From 085d255e11a940300866a9c366208dfcf429c6ef Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=B6ren=20Henning?= <soeren.henning@email.uni-kiel.de>
Date: Fri, 19 Nov 2021 18:26:31 +0100
Subject: [PATCH] Improve docs for creating new benchmarks

---
 docs/creating-a-benchmark.md  | 77 ++++++++++++++++++++++++++++++++++-
 docs/creating-an-execution.md |  1 +
 2 files changed, 77 insertions(+), 1 deletion(-)

diff --git a/docs/creating-a-benchmark.md b/docs/creating-a-benchmark.md
index ee55fb4ea..673d8e070 100644
--- a/docs/creating-a-benchmark.md
+++ b/docs/creating-a-benchmark.md
@@ -1,8 +1,83 @@
+---
+title: Creating a Benchmark
+has_children: false
+nav_order: 5
+---
 
-# Defining a Benchmark
+# Creating a Benchmark
 
 Please note that to simply run a benchmark, it is not required to define one. Theodolite comes with a set of benchmarks, which are ready to be executed. See the Benchmarks and Executions docs page to learn more about our distinction between benchmarks and executions.
 
+A typical benchmark looks as follow.
+
+```yaml
+apiVersion: theodolite.com/v1
+kind: benchmark
+metadata:
+  name: uc1-kstreams
+spec:
+  appResource:
+    - "custom/uc1-kstreams-deployment.yaml"
+  loadGenResource:
+    - "custom/uc1-load-generator-deployment.yaml"
+    - "custom/uc1-load-generator-service.yaml"
+  resourceTypes:
+    - typeName: "Instances"
+      patchers:
+        - type: "ReplicaPatcher"
+          resource: "custom/uc1-kstreams-deployment.yaml"
+  loadTypes:
+    - typeName: "NumSensors"
+      patchers:
+        - type: "EnvVarPatcher"
+          resource: "custom/uc1-load-generator-deployment.yaml"
+          properties:
+            variableName: "NUM_SENSORS"
+            container: "workload-generator"
+        - type: "NumSensorsLoadGeneratorReplicaPatcher"
+          resource: "custom/uc1-load-generator-deployment.yaml"
+          properties:
+            loadGenMaxRecords: "15000"
+  kafkaConfig:
+    bootstrapServer: "theodolite-cp-kafka:9092"
+    topics:
+      - name: "input"
+        numPartitions: 40
+        replicationFactor: 1
+      - name: "theodolite-.*"
+        removeOnly: True
+
+```
+
+### System under Test (SUT) and Load Generator Resources
+
+In Thedolite, the system under test (SUT) and the load generator are described by Kubernetes resources files.
+Based on these files, both the SUT and the load generator are started and stopped for each SLO experiment.
+
+All Kubernetes resource files listed under `appResource` and `loadGenResource` must be accessible by Theodolite.
+The recommended way to achieve this is by bundling them in one or multiple ConfigMaps, which can be done by:
+
+```sh
+kubectl create configmap <configmap-name> --from-file=<path-to-resources>
+```
+
+### Load and Resource Types
+
+Benchmarks need to specify at least one supported load and resource type for which scalability can be benchmarked.
+
+Load and resource types are described by a name (used for reference from an Execution) and a list of patchers.
+
+### Kafka Configuration
+
+Theodolite allows to automatically create and remove Kafka topics for each SLO experiment.
+Use the `removeOnly: True` property for topics which are created automatically by the SUT.
+For those topics, also wildcards are allowed in the topic name.
+
+
+<!-- Further information: API Reference -->
+<!-- Further information: How to deploy -->
+
+-----
 
 * Create a benchmark for operator mode
 * Create a benchmark for standalone mode
diff --git a/docs/creating-an-execution.md b/docs/creating-an-execution.md
index 9aeba6b28..2e96f6287 100644
--- a/docs/creating-an-execution.md
+++ b/docs/creating-an-execution.md
@@ -79,3 +79,4 @@ Instead, also Executions allow to do small reconfigurations, such as switching o
 This is done by defining `configOverrides` in the Execution. Each override consists of a patcher, defining which Kubernetes resource should be patched in which way, and a value the patcher is applied with.
 
 <!-- Further information: API Reference -->
+<!-- Further information: How to run -->
-- 
GitLab