diff --git a/docs/creating-a-benchmark.md b/docs/creating-a-benchmark.md
index f93c346c821e306b137eb71048fc9c425b87ae73..b61880e8619449d55ad04231da060358bb519699 100644
--- a/docs/creating-a-benchmark.md
+++ b/docs/creating-a-benchmark.md
@@ -14,18 +14,21 @@ A typical benchmark looks as follow.
apiVersion: theodolite.com/v1
kind: benchmark
metadata:
- name: uc1-kstreams
+ name: example-benchmark
spec:
- appResource:
- - "uc1-kstreams-deployment.yaml"
- loadGenResource:
- - "uc1-load-generator-deployment.yaml"
- - "uc1-load-generator-service.yaml"
- resourceTypes:
- - typeName: "Instances"
- patchers:
- - type: "ReplicaPatcher"
- resource: "uc1-kstreams-deployment.yaml"
+ sut:
+ resources:
+ - configMap:
+ name: "example-configmap"
+ files:
+ - "uc1-kstreams-deployment.yaml"
+ loadGenerator:
+ resources:
+ - configMap:
+ name: "example-configmap"
+ files:
+ - uc1-load-generator-service.yaml
+ - uc1-load-generator-deployment.yaml
loadTypes:
- typeName: "NumSensors"
patchers:
@@ -49,25 +52,33 @@ spec:
```
-### System under Test (SUT) and Load Generator Resources
+## System under Test (SUT) and Load Generator Resources
In Thedolite, the system under test (SUT) and the load generator are described by Kubernetes resources files.
Based on these files, both the SUT and the load generator are started and stopped for each SLO experiment.
+Optionally, you can also define infrastructure as Kubernetes resources files which lives over the entire duration of a benchmark run.
-All Kubernetes resource files listed under `appResource` and `loadGenResource` must be accessible by Theodolite.
-The recommended way to achieve this is by bundling them in one or multiple ConfigMaps, which can be done by:
+The recommended way to link Kubernetes resources files from a Benchmark is by bundling them in one or multiple ConfigMaps and refer to these ConfigMap from `sut.resources`, `loadGenerator.resources` or `infrastructure.resources`.
+To create a ConfigMap from all the Kubernetes resources in a directory run:
```sh
-kubectl create configmap <configmap-name> --from-file=<path-to-resources>
+kubectl create configmap <configmap-name> --from-file=<path-to-resource-dir>
```
-### Load and Resource Types
+<!--
+A Benchmark refers to other Kubernetes resources (e.g., Deployments, Services, ConfigMaps), which describe the system under test, the load generator and infrastructure components such as a middleware used in the benchmark. To manage those resources, Theodolite needs to have access to them. This is done by bundling resources in ConfigMaps.
+
+Suppose the resources needed by your benchmark are defined as YAML files, located in the `resources` directory. You can put them into the ConfigMap `benchmark-resources-custom` by running:
+-->
+
+
+## Load and Resource Types
Benchmarks need to specify at least one supported load and resource type for which scalability can be benchmarked.
Load and resource types are described by a name (used for reference from an Execution) and a list of patchers.
-### Kafka Configuration
+## Kafka Configuration
Theodolite allows to automatically create and remove Kafka topics for each SLO experiment.
Use the `removeOnly: True` property for topics which are created automatically by the SUT.
diff --git a/docs/installation.md b/docs/installation.md
index 203c40d5ab8c36eef30766cf5664db5f6dcebd1b..7c387e566de1eb284c87974e1d9f17d2b912ac95 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -85,10 +85,10 @@ helm uninstall theodolite
Helm does not remove any CRDs created by this chart. You can remove them manually with:
```sh
-# CRDs from Theodolite
+# CRDs for Theodolite
kubectl delete crd executions.theodolite.com
kubectl delete crd benchmarks.theodolite.com
-# CRDs from Prometheus operator (see https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#uninstall-chart)
+# CRDs for Prometheus operator (see https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#uninstall-chart)
kubectl delete crd alertmanagerconfigs.monitoring.coreos.com
kubectl delete crd alertmanagers.monitoring.coreos.com
kubectl delete crd podmonitors.monitoring.coreos.com
diff --git a/docs/quickstart.md b/docs/quickstart.md
index c8da052e9f1b34a319b4af76a2c84b85c4876518..2ad3824a6e6dc1df505f5ca1ffac8b45ec3fe591 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -41,12 +41,10 @@ All you need to get started is access to a Kubernetes cluster plus kubectl and H
The output is similar to this:
```
- NAME AGE
- uc1-kstreams 100s
+ NAME AGE STATUS
+ example-benchmark 81s Ready
```
- <!-- State with newest version -->
-
1. Run the Benchmark by deploying an Execution:
```sh
diff --git a/docs/running-benchmarks.md b/docs/running-benchmarks.md
index 9212003a992f68d11bb5e5917c2dd82334c9f4ed..e66fb8e346f95768f8e3f3717d8e263c20997964 100644
--- a/docs/running-benchmarks.md
+++ b/docs/running-benchmarks.md
@@ -16,27 +16,18 @@ Running scalability benchmarks with Theodolite involves two things:
A benchmark specification consists of two things:
* A Benchmark resource YAML file
-* A set of Kubernetes resource YAML files used by the benchmark
-<!-- - One or multiple ConfigMap YAML files containing all the Kubernetes resource used by the benchmark as YAML files -->
+* One or multiple ConfigMap YAML files containing all the Kubernetes resources used by the benchmark
These files are usually provided by benchmark designers.
For example, we ship Theodolite with a set of [benchmarks for event-driven microservices](theodolite-benchmarks).
Alternatively, you can also [create your own benchmarks](creating-a-benchmark).
-<!-- Theodolite >v0.5
-Once you have collected all Kubernetes resources for the benchmark (Benchmark resource and ConfigMaps) in a specific directory, you can deploy everything to Kubernetes by running:
-
-```sh
-kubectl apply -f .
-```
--->
-
-### Create the Benchmark
-
-Suppose your Benchmark resource is stored in `example-benchmark.yaml`, you can deploy it to Kubernetes by running:
+Suppose your Benchmark is defined in `example-benchmark.yaml` and all resources required by this benchmark are bundled in `example-configmap.yaml`.
+You can deploy both to Kubernetes by running:
```sh
kubectl apply -f example-benchmark.yaml
+kubectl apply -f example-configmap.yaml
```
To list all benchmarks currently deployed run:
@@ -45,17 +36,16 @@ To list all benchmarks currently deployed run:
kubectl get benchmarks
```
-<!-- TODO output-->
-
-### Create the Benchmark Resources ConfigMaps
-
-A Benchmark resource refers to other Kubernetes resources (e.g., Deployments, Services, ConfigMaps), which describe the system under test, the load generator and infrastructure components such as a middleware used in the benchmark. To manage those resources, Theodolite needs to have access to them. This is done by bundling resources in ConfigMaps.
-
-Suppose the resources needed by your benchmark are defined as YAML files, located in the `resources` directory. You can put them into the ConfigMap `benchmark-resources-custom` by running:
+The output is similar to this:
-```sh
-kubectl create configmap benchmark-resources-custom --from-file=./resources -o yaml --dry-run=client | kubectl apply -f -
```
+NAME AGE STATUS
+example-benchmark 81s Ready
+```
+
+The status of a benchmark tells you whether executions of it can be executed:
+* *Ready* means that Theodolite has access to all resources referred from the benchmark.
+* *Pending* implies that not all benchmark resources are available (yet). Please ensure that you have deployed them, for example, by running `kubectl get configmaps`.
## Creating an Execution
diff --git a/theodolite/examples/operator/example-benchmark.yaml b/theodolite/examples/operator/example-benchmark.yaml
index 9e6e0ef48c30803673bf6a7eb79337eb87724b2a..5f68af04750bcd779c9682ede69d6c68b9fb3e92 100644
--- a/theodolite/examples/operator/example-benchmark.yaml
+++ b/theodolite/examples/operator/example-benchmark.yaml
@@ -1,7 +1,7 @@
apiVersion: theodolite.com/v1
kind: benchmark
metadata:
- name: uc1-kstreams
+ name: example-benchmark
spec:
sut:
resources:
diff --git a/theodolite/examples/operator/example-execution.yaml b/theodolite/examples/operator/example-execution.yaml
index e2efb6e9a2bb6c08354b57a83506a601ac0ed96e..576a74b90dfc38483de79502ac14d42f6bedfb49 100644
--- a/theodolite/examples/operator/example-execution.yaml
+++ b/theodolite/examples/operator/example-execution.yaml
@@ -3,7 +3,7 @@ kind: execution
metadata:
name: theodolite-example-execution
spec:
- benchmark: "uc1-kstreams"
+ benchmark: "example-benchmark"
load:
loadType: "NumSensors"
loadValues: [25000, 50000, 75000, 100000, 125000, 150000]