Skip to content
Snippets Groups Projects
Commit 7f4630ce authored by Sören Henning's avatar Sören Henning
Browse files

Merge branch '237-add-crd-docu' into 'theodolite-kotlin' 

Add documentation for the crd benchmark and execution

See merge request !169
parents dfe7ad7d c99cb373
No related branches found
No related tags found
2 merge requests!169Add documentation for the crd benchmark and execution,!159Re-implementation of Theodolite with Kotlin/Quarkus
Pipeline #4017 passed
Stage: build
Stage: test
Stage: check
Stage: deploy
Hide whitespace changes
Inline Side-by-side
docs/crd-benchmark-doc.md 0 → 100644
+ 374
− 0
View file @ 7f4630ce
# API Reference
Packages:
- [theodolite.com/v1](#theodolitecomv1)
# theodolite.com/v1
Resource Types:
- [benchmark](#benchmark)
## benchmark
<sup><sup>[↩ Parent](#theodolitecomv1 )</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>apiVersion</b></td>
<td>string</td>
<td>theodolite.com/v1</td>
<td>true</td>
</tr>
<tr>
<td><b>kind</b></td>
<td>string</td>
<td>benchmark</td>
<td>true</td>
</tr>
<tr>
<td><b><a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#objectmeta-v1-meta">metadata</a></b></td>
<td>object</td>
<td>Refer to the Kubernetes API documentation for the fields of the `metadata` field.</td>
<td>true</td>
</tr><tr>
<td><b><a href="#benchmarkspec">spec</a></b></td>
<td>object</td>
<td>
<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### benchmark.spec
<sup><sup>[↩ Parent](#benchmark)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>name</b></td>
<td>string</td>
<td>
This field exists only for technical reasons and should not be set by the user. The value of the field will be overwritten.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>appResource</b></td>
<td>[]string</td>
<td>
A list of file names that reference Kubernetes resources that are deployed on the cluster for the system under test (SUT).<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#benchmarkspeckafkaconfig">kafkaConfig</a></b></td>
<td>object</td>
<td>
Contains the Kafka configuration.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>loadGenResource</b></td>
<td>[]string</td>
<td>
A list of file names that reference Kubernetes resources that are deployed on the cluster for the load generator.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#benchmarkspecloadtypesindex">loadTypes</a></b></td>
<td>[]object</td>
<td>
A list of load types that can be scaled for this benchmark. For each load type the concrete values are defined in the execution object.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#benchmarkspecresourcetypesindex">resourceTypes</a></b></td>
<td>[]object</td>
<td>
A list of resource types that can be scaled for this `benchmark` resource. For each resource type the concrete values are defined in the `execution` object.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### benchmark.spec.kafkaConfig
<sup><sup>[↩ Parent](#benchmarkspec)</sup></sup>
Contains the Kafka configuration.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>bootstrapServer</b></td>
<td>string</td>
<td>
The bootstrap servers connection string.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#benchmarkspeckafkaconfigtopicsindex">topics</a></b></td>
<td>[]object</td>
<td>
List of topics to be created for each experiment. Alternative theodolite offers the possibility to remove certain topics after each experiment.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### benchmark.spec.kafkaConfig.topics[index]
<sup><sup>[↩ Parent](#benchmarkspeckafkaconfig)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>removeOnly</b></td>
<td>boolean</td>
<td>
Determines if this topic should only be deleted after each experiement. For removeOnly topics the name can be a RegEx describing the topic.<br/>
<br/>
<i>Default</i>: false<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>name</b></td>
<td>string</td>
<td>
The name of the topic.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>true</td>
</tr><tr>
<td><b>numPartitions</b></td>
<td>integer</td>
<td>
The number of partitions of the topic.<br/>
<br/>
<i>Default</i>: 0<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>replicationFactor</b></td>
<td>integer</td>
<td>
The replication factor of the topic.<br/>
<br/>
<i>Default</i>: 0<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### benchmark.spec.loadTypes[index]
<sup><sup>[↩ Parent](#benchmarkspec)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b><a href="#benchmarkspecloadtypesindexpatchersindex">patchers</a></b></td>
<td>[]object</td>
<td>
List of patchers used to scale this resource type.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>typeName</b></td>
<td>string</td>
<td>
Name of the load type.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### benchmark.spec.loadTypes[index].patchers[index]
<sup><sup>[↩ Parent](#benchmarkspecloadtypesindex)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>properties</b></td>
<td>object</td>
<td>
(Optional) Patcher specific additional arguments.<br/>
<br/>
<i>Default</i>: map[]<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>resource</b></td>
<td>string</td>
<td>
Specifies the Kubernetes resource to be patched.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>true</td>
</tr><tr>
<td><b>type</b></td>
<td>string</td>
<td>
Type of the Patcher.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### benchmark.spec.resourceTypes[index]
<sup><sup>[↩ Parent](#benchmarkspec)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b><a href="#benchmarkspecresourcetypesindexpatchersindex">patchers</a></b></td>
<td>[]object</td>
<td>
List of patchers used to scale this resource type.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>typeName</b></td>
<td>string</td>
<td>
Name of the resource type.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### benchmark.spec.resourceTypes[index].patchers[index]
<sup><sup>[↩ Parent](#benchmarkspecresourcetypesindex)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>properties</b></td>
<td>object</td>
<td>
(Optional) Patcher specific additional arguments.<br/>
<br/>
<i>Default</i>: map[]<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>resource</b></td>
<td>string</td>
<td>
Specifies the Kubernetes resource to be patched.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>true</td>
</tr><tr>
<td><b>type</b></td>
<td>string</td>
<td>
Type of the patcher.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>true</td>
</tr></tbody>
</table>
\ No newline at end of file
docs/crd-execution-doc.md 0 → 100644
+ 438
− 0
View file @ 7f4630ce
# API Reference
Packages:
- [theodolite.com/v1](#theodolitecomv1)
# theodolite.com/v1
Resource Types:
- [execution](#execution)
## execution
<sup><sup>[↩ Parent](#theodolitecomv1 )</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>apiVersion</b></td>
<td>string</td>
<td>theodolite.com/v1</td>
<td>true</td>
</tr>
<tr>
<td><b>kind</b></td>
<td>string</td>
<td>execution</td>
<td>true</td>
</tr>
<tr>
<td><b><a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#objectmeta-v1-meta">metadata</a></b></td>
<td>object</td>
<td>Refer to the Kubernetes API documentation for the fields of the `metadata` field.</td>
<td>true</td>
</tr><tr>
<td><b><a href="#executionstatus">status</a></b></td>
<td>object</td>
<td>
<br/>
</td>
<td>false</td>
</tr><tr>
<td><b><a href="#executionspec">spec</a></b></td>
<td>object</td>
<td>
<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### execution.status
<sup><sup>[↩ Parent](#execution)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>executionDuration</b></td>
<td>string</td>
<td>
Duration of the execution in seconds<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>executionState</b></td>
<td>string</td>
<td>
<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
### execution.spec
<sup><sup>[↩ Parent](#execution)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>name</b></td>
<td>string</td>
<td>
This field exists only for technical reasons and should not be set by the user. The value of the field will be overwritten.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>false</td>
</tr><tr>
<td><b>benchmark</b></td>
<td>string</td>
<td>
The name of the benchmark this execution is referring to.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#executionspecconfigoverridesindex">configOverrides</a></b></td>
<td>[]object</td>
<td>
List of patchers that are used to override existing configurations.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#executionspecexecution">execution</a></b></td>
<td>object</td>
<td>
Defines the overall parameter for the execution.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#executionspecload">load</a></b></td>
<td>object</td>
<td>
Specifies the load values that are benchmarked.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#executionspecresources">resources</a></b></td>
<td>object</td>
<td>
Specifies the scaling resource that is benchmarked.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b><a href="#executionspecslosindex">slos</a></b></td>
<td>[]object</td>
<td>
List of resource values for the specified resource type.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### execution.spec.configOverrides[index]
<sup><sup>[↩ Parent](#executionspec)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b><a href="#executionspecconfigoverridesindexpatcher">patcher</a></b></td>
<td>object</td>
<td>
Patcher used to patch a resource<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>value</b></td>
<td>string</td>
<td>
<br/>
</td>
<td>false</td>
</tr></tbody>
</table>
### execution.spec.configOverrides[index].patcher
<sup><sup>[↩ Parent](#executionspecconfigoverridesindex)</sup></sup>
Patcher used to patch a resource
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>properties</b></td>
<td>object</td>
<td>
(Optional) Patcher specific additional arguments.<br/>
<br/>
<i>Default</i>: map[]<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>resource</b></td>
<td>string</td>
<td>
Specifies the Kubernetes resource to be patched.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>true</td>
</tr><tr>
<td><b>type</b></td>
<td>string</td>
<td>
Type of the Patcher.<br/>
<br/>
<i>Default</i>: <br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### execution.spec.execution
<sup><sup>[↩ Parent](#executionspec)</sup></sup>
Defines the overall parameter for the execution.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>loadGenerationDelay</b></td>
<td>integer</td>
<td>
Seconds to wait between the start of the SUT and the load generator.<br/>
</td>
<td>false</td>
</tr><tr>
<td><b>duration</b></td>
<td>integer</td>
<td>
Defines the duration of each experiment in seconds.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>repetitions</b></td>
<td>integer</td>
<td>
Numper of repititions for each experiments.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>restrictions</b></td>
<td>[]string</td>
<td>
List of restriction strategys used to delimit the search space.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>strategy</b></td>
<td>string</td>
<td>
Defines the used strategy for the execution, either 'LinearSearch' or 'BinarySearch'<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### execution.spec.load
<sup><sup>[↩ Parent](#executionspec)</sup></sup>
Specifies the load values that are benchmarked.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>loadType</b></td>
<td>string</td>
<td>
The type of the load. It must match one of the load types specified in the referenced benchmark.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>loadValues</b></td>
<td>[]integer</td>
<td>
List of load values for the specified load type.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### execution.spec.resources
<sup><sup>[↩ Parent](#executionspec)</sup></sup>
Specifies the scaling resource that is benchmarked.
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>resourceType</b></td>
<td>string</td>
<td>
The type of the resource. It must match one of the resource types specified in the referenced benchmark.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>resourceValues</b></td>
<td>[]integer</td>
<td>
<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
### execution.spec.slos[index]
<sup><sup>[↩ Parent](#executionspec)</sup></sup>
<table>
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
<th>Required</th>
</tr>
</thead>
<tbody><tr>
<td><b>externalSloUrl</b></td>
<td>string</td>
<td>
Connection string for a external slo analysis.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>offset</b></td>
<td>integer</td>
<td>
Hours by which the start and end timestamp will be shifted (for different timezones).<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>prometheusUrl</b></td>
<td>string</td>
<td>
Connection string for Promehteus.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>sloType</b></td>
<td>string</td>
<td>
The type of the SLO. It must match 'lag trend'.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>threshold</b></td>
<td>integer</td>
<td>
The threshold the SUT should meet for a sucessful experiment.<br/>
</td>
<td>true</td>
</tr><tr>
<td><b>warmup</b></td>
<td>integer</td>
<td>
Seconds of time that are ignored in the analysis.<br/>
</td>
<td>true</td>
</tr></tbody>
</table>
\ No newline at end of file
theodolite/crd/crd-benchmark.yaml
+ 29
− 1
View file @ 7f4630ce
......@@ -23,89 +23,117 @@ spec:
required: ["appResource", "loadGenResource", "resourceTypes", "loadTypes", "kafkaConfig"]
properties:
name:
description: This field exists only for technical reasons and should not be set by the user. The value of the field will be overwritten.
type: string
appResource:
description: A list of file names that reference Kubernetes resources that are deployed on the cluster for the system under test (SUT).
type: array
minItems: 1
items:
type: string
loadGenResource:
description: A list of file names that reference Kubernetes resources that are deployed on the cluster for the load generator.
type: array
minItems: 1
items:
type: string
resourceTypes:
description: A list of resource types that can be scaled for this `benchmark` resource. For each resource type the concrete values are defined in the `execution` object.
type: array
minItems: 1
items:
type: object
required: ["typeName", "patchers"]
properties:
typeName:
description: Name of the resource type.
type: string
patchers:
description: List of patchers used to scale this resource type.
examples: test test test
type: array
minItems: 1
items:
type: object
required: ["type", "resource"]
properties:
type:
description: Type of the patcher.
type: string
default: ""
resource:
description: Specifies the Kubernetes resource to be patched.
type: string
default: ""
properties:
description: (Optional) Patcher specific additional arguments.
type: object
additionalProperties: true
x-kubernetes-map-type: "granular"
default: {}
loadTypes:
description: A list of load types that can be scaled for this benchmark. For each load type the concrete values are defined in the execution object.
type: array
minItems: 1
items:
type: object
required: ["typeName","patchers"]
properties:
typeName:
description: Name of the load type.
type: string
patchers:
description: List of patchers used to scale this resource type.
type: array
minItems: 1
items:
type: object
required: ["type", "resource"]
properties:
type:
description: Type of the Patcher.
type: string
default: ""
resource:
description: Specifies the Kubernetes resource to be patched.
type: string
default: ""
properties:
description: (Optional) Patcher specific additional arguments.
type: object
additionalProperties: true
x-kubernetes-map-type: "granular"
default: {}
kafkaConfig:
description: Contains the Kafka configuration.
type: object
required: ["bootstrapServer", "topics"]
properties:
bootstrapServer:
description: The bootstrap servers connection string.
type: string
topics:
description: List of topics to be created for each experiment. Alternative theodolite offers the possibility to remove certain topics after each experiment.
type: array
minItems: 1
items:
type: object
required: []
required: ["name", "numPartitions", "replicationFactor"]
properties:
name:
description: The name of the topic.
type: string
default: ""
numPartitions:
description: The number of partitions of the topic.
type: integer
default: 0
replicationFactor:
description: The replication factor of the topic.
type: integer
default: 0
removeOnly:
description: Determines if this topic should only be deleted after each experiement. For removeOnly topics the name can be a RegEx describing the topic.
type: boolean
default: false
additionalPrinterColumns:
......
theodolite/crd/crd-execution.yaml
+ 28
− 1
View file @ 7f4630ce
......@@ -23,79 +23,106 @@ spec:
required: ["benchmark", "load", "resources", "slos", "execution", "configOverrides"]
properties:
name:
description: This field exists only for technical reasons and should not be set by the user. The value of the field will be overwritten.
type: string
default: ""
benchmark:
description: The name of the benchmark this execution is referring to.
type: string
load: # definition of the load dimension
description: Specifies the load values that are benchmarked.
type: object
required: ["loadType", "loadValues"]
properties:
loadType:
type: string
description: The type of the load. It must match one of the load types specified in the referenced benchmark.
type: string
loadValues:
description: List of load values for the specified load type.
type: array
items:
type: integer
resources: # definition of the resource dimension
description: Specifies the scaling resource that is benchmarked.
type: object
required: ["resourceType", "resourceValues"]
properties:
resourceType:
description: The type of the resource. It must match one of the resource types specified in the referenced benchmark.
type: string
resourceValues:
descriptoin: List of resource values for the specified resource type.
type: array
items:
type: integer
slos: # def of service level objectives
description: List of resource values for the specified resource type.
type: array
items:
type: object
required: ["sloType", "threshold", "prometheusUrl", "externalSloUrl", "offset", "warmup"]
properties:
sloType:
description: The type of the SLO. It must match 'lag trend'.
type: string
threshold:
description: The threshold the SUT should meet for a sucessful experiment.
type: integer
prometheusUrl:
description: Connection string for Promehteus.
type: string
externalSloUrl:
description: Connection string for a external slo analysis.
type: string
offset:
description: Hours by which the start and end timestamp will be shifted (for different timezones).
type: integer
warmup:
description: Seconds of time that are ignored in the analysis.
type: integer
execution: # def execution config
description: Defines the overall parameter for the execution.
type: object
required: ["strategy", "duration", "repetitions", "restrictions"]
properties:
strategy:
description: Defines the used strategy for the execution, either 'LinearSearch' or 'BinarySearch'
type: string
duration:
description: Defines the duration of each experiment in seconds.
type: integer
repetitions:
description: Numper of repititions for each experiments.
type: integer
loadGenerationDelay:
description: Seconds to wait between the start of the SUT and the load generator.
type: integer
restrictions:
description: List of restriction strategys used to delimit the search space.
type: array
items:
type: string
configOverrides:
description: List of patchers that are used to override existing configurations.
type: array
items:
type: object
properties:
patcher:
description: Patcher used to patch a resource
type: object
required: ["type", "resource"]
properties:
type:
description: Type of the Patcher.
type: string
default: ""
resource:
description: Specifies the Kubernetes resource to be patched.
type: string
default: ""
properties:
description: (Optional) Patcher specific additional arguments.
type: object
additionalProperties: true
x-kubernetes-map-type: "granular"
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment