From 68c1d2c13f8795f1d0911b93422d620c01c603da Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=B6ren=20Henning?= <soeren.henning@email.uni-kiel.de>
Date: Wed, 24 Nov 2021 18:36:10 +0100
Subject: [PATCH] Update docs for latest chnages in Theodolite
---
docs/creating-a-benchmark.md | 45 ++++++++++++-------
docs/installation.md | 4 +-
docs/quickstart.md | 6 +--
docs/running-benchmarks.md | 34 +++++---------
.../examples/operator/example-benchmark.yaml | 2 +-
.../examples/operator/example-execution.yaml | 2 +-
6 files changed, 46 insertions(+), 47 deletions(-)
diff --git a/docs/creating-a-benchmark.md b/docs/creating-a-benchmark.md
index f93c346c8..b61880e86 100644
--- a/docs/creating-a-benchmark.md
+++ b/docs/creating-a-benchmark.md
@@ -14,18 +14,21 @@ A typical benchmark looks as follow.
apiVersion: theodolite.com/v1
kind: benchmark
metadata:
- name: uc1-kstreams
+ name: example-benchmark
spec:
- appResource:
- - "uc1-kstreams-deployment.yaml"
- loadGenResource:
- - "uc1-load-generator-deployment.yaml"
- - "uc1-load-generator-service.yaml"
- resourceTypes:
- - typeName: "Instances"
- patchers:
- - type: "ReplicaPatcher"
- resource: "uc1-kstreams-deployment.yaml"
+ sut:
+ resources:
+ - configMap:
+ name: "example-configmap"
+ files:
+ - "uc1-kstreams-deployment.yaml"
+ loadGenerator:
+ resources:
+ - configMap:
+ name: "example-configmap"
+ files:
+ - uc1-load-generator-service.yaml
+ - uc1-load-generator-deployment.yaml
loadTypes:
- typeName: "NumSensors"
patchers:
@@ -49,25 +52,33 @@ spec:
```
-### System under Test (SUT) and Load Generator Resources
+## System under Test (SUT) and Load Generator Resources
In Thedolite, the system under test (SUT) and the load generator are described by Kubernetes resources files.
Based on these files, both the SUT and the load generator are started and stopped for each SLO experiment.
+Optionally, you can also define infrastructure as Kubernetes resources files which lives over the entire duration of a benchmark run.
-All Kubernetes resource files listed under `appResource` and `loadGenResource` must be accessible by Theodolite.
-The recommended way to achieve this is by bundling them in one or multiple ConfigMaps, which can be done by:
+The recommended way to link Kubernetes resources files from a Benchmark is by bundling them in one or multiple ConfigMaps and refer to these ConfigMap from `sut.resources`, `loadGenerator.resources` or `infrastructure.resources`.
+To create a ConfigMap from all the Kubernetes resources in a directory run:
```sh
-kubectl create configmap <configmap-name> --from-file=<path-to-resources>
+kubectl create configmap <configmap-name> --from-file=<path-to-resource-dir>
```
-### Load and Resource Types
+<!--
+A Benchmark refers to other Kubernetes resources (e.g., Deployments, Services, ConfigMaps), which describe the system under test, the load generator and infrastructure components such as a middleware used in the benchmark. To manage those resources, Theodolite needs to have access to them. This is done by bundling resources in ConfigMaps.
+
+Suppose the resources needed by your benchmark are defined as YAML files, located in the `resources` directory. You can put them into the ConfigMap `benchmark-resources-custom` by running:
+-->
+
+
+## Load and Resource Types
Benchmarks need to specify at least one supported load and resource type for which scalability can be benchmarked.
Load and resource types are described by a name (used for reference from an Execution) and a list of patchers.
-### Kafka Configuration
+## Kafka Configuration
Theodolite allows to automatically create and remove Kafka topics for each SLO experiment.
Use the `removeOnly: True` property for topics which are created automatically by the SUT.
diff --git a/docs/installation.md b/docs/installation.md
index 203c40d5a..7c387e566 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -85,10 +85,10 @@ helm uninstall theodolite
Helm does not remove any CRDs created by this chart. You can remove them manually with:
```sh
-# CRDs from Theodolite
+# CRDs for Theodolite
kubectl delete crd executions.theodolite.com
kubectl delete crd benchmarks.theodolite.com
-# CRDs from Prometheus operator (see https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#uninstall-chart)
+# CRDs for Prometheus operator (see https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#uninstall-chart)
kubectl delete crd alertmanagerconfigs.monitoring.coreos.com
kubectl delete crd alertmanagers.monitoring.coreos.com
kubectl delete crd podmonitors.monitoring.coreos.com
diff --git a/docs/quickstart.md b/docs/quickstart.md
index c8da052e9..2ad3824a6 100644
--- a/docs/quickstart.md
+++ b/docs/quickstart.md
@@ -41,12 +41,10 @@ All you need to get started is access to a Kubernetes cluster plus kubectl and H
The output is similar to this:
```
- NAME AGE
- uc1-kstreams 100s
+ NAME AGE STATUS
+ example-benchmark 81s Ready
```
- <!-- State with newest version -->
-
1. Run the Benchmark by deploying an Execution:
```sh
diff --git a/docs/running-benchmarks.md b/docs/running-benchmarks.md
index 9212003a9..e66fb8e34 100644
--- a/docs/running-benchmarks.md
+++ b/docs/running-benchmarks.md
@@ -16,27 +16,18 @@ Running scalability benchmarks with Theodolite involves two things:
A benchmark specification consists of two things:
* A Benchmark resource YAML file
-* A set of Kubernetes resource YAML files used by the benchmark
-<!-- - One or multiple ConfigMap YAML files containing all the Kubernetes resource used by the benchmark as YAML files -->
+* One or multiple ConfigMap YAML files containing all the Kubernetes resources used by the benchmark
These files are usually provided by benchmark designers.
For example, we ship Theodolite with a set of [benchmarks for event-driven microservices](theodolite-benchmarks).
Alternatively, you can also [create your own benchmarks](creating-a-benchmark).
-<!-- Theodolite >v0.5
-Once you have collected all Kubernetes resources for the benchmark (Benchmark resource and ConfigMaps) in a specific directory, you can deploy everything to Kubernetes by running:
-
-```sh
-kubectl apply -f .
-```
--->
-
-### Create the Benchmark
-
-Suppose your Benchmark resource is stored in `example-benchmark.yaml`, you can deploy it to Kubernetes by running:
+Suppose your Benchmark is defined in `example-benchmark.yaml` and all resources required by this benchmark are bundled in `example-configmap.yaml`.
+You can deploy both to Kubernetes by running:
```sh
kubectl apply -f example-benchmark.yaml
+kubectl apply -f example-configmap.yaml
```
To list all benchmarks currently deployed run:
@@ -45,17 +36,16 @@ To list all benchmarks currently deployed run:
kubectl get benchmarks
```
-<!-- TODO output-->
-
-### Create the Benchmark Resources ConfigMaps
-
-A Benchmark resource refers to other Kubernetes resources (e.g., Deployments, Services, ConfigMaps), which describe the system under test, the load generator and infrastructure components such as a middleware used in the benchmark. To manage those resources, Theodolite needs to have access to them. This is done by bundling resources in ConfigMaps.
-
-Suppose the resources needed by your benchmark are defined as YAML files, located in the `resources` directory. You can put them into the ConfigMap `benchmark-resources-custom` by running:
+The output is similar to this:
-```sh
-kubectl create configmap benchmark-resources-custom --from-file=./resources -o yaml --dry-run=client | kubectl apply -f -
```
+NAME AGE STATUS
+example-benchmark 81s Ready
+```
+
+The status of a benchmark tells you whether executions of it can be executed:
+* *Ready* means that Theodolite has access to all resources referred from the benchmark.
+* *Pending* implies that not all benchmark resources are available (yet). Please ensure that you have deployed them, for example, by running `kubectl get configmaps`.
## Creating an Execution
diff --git a/theodolite/examples/operator/example-benchmark.yaml b/theodolite/examples/operator/example-benchmark.yaml
index 9e6e0ef48..5f68af047 100644
--- a/theodolite/examples/operator/example-benchmark.yaml
+++ b/theodolite/examples/operator/example-benchmark.yaml
@@ -1,7 +1,7 @@
apiVersion: theodolite.com/v1
kind: benchmark
metadata:
- name: uc1-kstreams
+ name: example-benchmark
spec:
sut:
resources:
diff --git a/theodolite/examples/operator/example-execution.yaml b/theodolite/examples/operator/example-execution.yaml
index e2efb6e9a..576a74b90 100644
--- a/theodolite/examples/operator/example-execution.yaml
+++ b/theodolite/examples/operator/example-execution.yaml
@@ -3,7 +3,7 @@ kind: execution
metadata:
name: theodolite-example-execution
spec:
- benchmark: "uc1-kstreams"
+ benchmark: "example-benchmark"
load:
loadType: "NumSensors"
loadValues: [25000, 50000, 75000, 100000, 125000, 150000]
--
GitLab