Please note that to simply run a benchmark, it is not required to define one. Theodolite comes with a set of benchmarks, which are ready to be executed. See the Benchmarksand Executions docs page to learn more about our distinction between benchmarks and executions.
Please note that to simply run a benchmark, it is not required to define one. Theodolite comes with a [set of benchmarks](theodolite-benchmarks), which are ready to be executed. See the [fundamental concepts](benchmarks-and-executions) page to learn more about our distinction between benchmarks and executions.
A typical benchmark looks as follow.
A typical benchmark looks like this:
```yaml
apiVersion:theodolite.com/v1
...
...
@@ -77,6 +77,7 @@ Suppose the resources needed by your benchmark are defined as YAML files, locate
Benchmarks need to specify at least one supported load and resource type for which scalability can be benchmarked.
Load and resource types are described by a name (used for reference from an Execution) and a list of patchers.
If a benchmark is [executed by an Execution](running-benchmarks), these patchers are used to configure SUT and load generator according to the [load and resource values](creating-an-execution) set in the Execution.
@@ -51,11 +51,9 @@ Similar to [Kubernetes Jobs](https://kubernetes.io/docs/concepts/workloads/contr
## Reference to a Benchmark
An Execution always refers to a Benchmark. This benchmark has to be registered with Kubernetes once Theodolite starts an Execution. Otherwise, Theodolite aborts the Executions and marks it as *failed*.
An Execution always refers to a Benchmark. For the Execution to run, the Benchmark must be registered with Kubernetes and it must be in state *Ready*. If this is not the case, the Execution will remain in state *Pending*.
<!-- Benchmark state -->
As a Benchmark may define multiple supported load and resource types, an Execution has to pick exactly one of each by its name. Additionally, it defines the set of load values and resource values the benchmark should be executed with.
As a Benchmark may define multiple supported load and resource types, an Execution has to pick exactly one of each by its name. Additionally, it defines the set of load values and resource values the benchmark should be executed with. Both these values are represented as integers, which are interpreted in a [Benchmark-specific way](creating-a-benchmark) to configure the SUT and load generator.