docs/development/release-process.md

 ---
 title: Release Process
 has_children: false
-parent: Development
+parent: Contributing
 nav_order: 1
 ---
 

docs/example-teastore.md

+---
+title: "Example: TeaStore"
+has_children: false
+parent: "Creating Benchmarks"
+nav_order: 1
+---
+
+# Example: A Benchmark for the TeaStore
+
+The [TeaStore](https://github.com/DescartesResearch/TeaStore) is a microservice reference application.
+It resamples a web shop for tea, allowing customers, for example, to browse the shop catalog, receive product recommendations, or place orders.
+The TeaStore consists of six microservices and a MariaDB database.
+The entire application can easily deployed with Kubernetes using the provided resource files.
+
+In this example, we will create a Theodolite benchmark for the TeaStore.
+We use [Open Service Mesh (OSM)](https://openservicemesh.io/) to inject sidecar proxies into the TeaStore microservices, which allow us to gather latency and other metrics.
+
+## Prerequisites
+
+To get started, you need:
+
+* A running Kubernetes cluster (for testing purposes, you might want to use [Minikube](https://minikube.sigs.k8s.io/), [kind](https://kind.sigs.k8s.io/) or [k3d](https://k3d.io/))
+* [Helm installed](https://helm.sh/) on you local machine
+
+
+## Cluster Preparation
+
+Before running a benchmark, we need to install Theodolite and OSM on our cluster.
+
+### Install Theodolite
+
+In general, Theodolite can be installed using Helm as described in the [installation guide](installation).
+However, we need to make sure that no OSM sidecards are injected into the pod of Theodolite, Prometheus, etc.
+As we do not use Kafka in this example, we can omit the Strimzi installation.
+If no further configuration is required, run the following command to install Theodolite:
+
+```sh
+helm install theodolite theodolite/theodolite -f https://raw.githubusercontent.com/cau-se/theodolite/main/helm/preconfigs/osm-ready.yaml -f https://raw.githubusercontent.com/cau-se/theodolite/main/helm/preconfigs/kafka-less.yaml
+```
+
+### Install Open Service Mesh
+
+To install OSM, we use the Helm chart provided by the OSM project. Run the following commands to install and configure OSM:
+
+```sh
+export NAMESPACE=default # Kubernetes namespace to be monitored
+
+kubectl create ns osm-system
+helm install osm osm --repo https://openservicemesh.github.io/osm --namespace osm-system --version 0.9.2 # A newer version would probably work as well
+ 
+sleep 60s # Installation may take some time, so we wait a bit
+
+kubectl patch meshconfig osm-mesh-config -n osm-system -p '{"spec":{"traffic":{"enablePermissiveTrafficPolicyMode":true}}}'  --type=merge
+kubectl patch meshconfig osm-mesh-config -n osm-system -p '{"spec":{"traffic":{"enableEgress":true}}}' --type=merge
+
+kubectl label namespace $NAMESPACE openservicemesh.io/monitored-by=osm --overwrite
+kubectl annotate namespace $NAMESPACE openservicemesh.io/metrics=enabled --overwrite
+kubectl annotate namespace $NAMESPACE openservicemesh.io/sidecar-injection=enabled --overwrite
+```
+
+
+## Create a Benchmark for the TeaStore
+
+According to our Theodolite benchmarking method, we need to define a system under test (SUT) and a load generator for the benchmark.
+Quite obviously, the TeaStore enriched by OSM's Envoy sidecar proxies is the SUT.
+As load generator, we use JMeter along with the [browse profile](https://github.com/DescartesResearch/TeaStore/blob/master/examples/jmeter/teastore_browse_nogui.jmx) provided in the TeaStore repository.
+The desired benchmarking setup is shown in the following diagram:
+
+![TeaStore Benchmark](../assets/images/example-teastore-deployment.svg)
+
+Although the TeaStore comes with Kubernetes resources (Deployments and Services), we need some modifications such as one resource per file, resource limits, or readiness probes. (Note that these modifications are not specific for Theodolite, but are generally considered good practice.)
+We created a fork of the TeaStore repository all required modifications. Clone it by running:
+
+```sh
+git clone -b add-theodolite-example git@github.com:SoerenHenning/TeaStore.git
+```
+
+We now have to create ConfigMaps bundling these resources and a Benchmark resource describing the benchmark.
+
+### Create ConfigMaps containing all components
+
+To create a ConfigMap containing all TeaStore resources, simply run:
+
+```sh
+kubectl create configmap teastore-deployment --from-file=teastore/examples/kubernetes/teastore-clusterip-split/
+```
+
+Likewise, we have to create a ConfigMap for the JMeter profile and a ConfigMap containing a [JMeter deployment](https://github.com/SoerenHenning/TeaStore/blob/add-theodolite-example/examples/theodolite/jmeter.yaml):
+
+```sh
+kubectl create configmap teastore-jmeter-browse --from-file=teastore/examples/jmeter/teastore_browse_nogui.jmx
+kubectl create configmap teastore-jmeter-deployment --from-file=teastore/examples/theodolite/jmeter.yaml
+```
+
+### Create the Benchmark file
+
+Once all the required resources are bundled in ConfigMaps, we can define a Benchmark resource.
+The following resource defines a simple benchmark providing one load type, one resource type and one SLO.
+
+```yaml
+apiVersion: theodolite.rocks/v1beta1
+kind: benchmark
+metadata:
+  name: teastore
+spec:
+  waitForResourcesEnabled: true
+  sut:
+    resources:
+      - configMap:
+          name: teastore-deployment
+  loadGenerator:
+    resources:
+      - configMap:
+          name: teastore-jmeter-deployment
+  resourceTypes:
+    - typeName: "Instances"
+      patchers:
+        - type: "ReplicaPatcher"
+          resource: "teastore-auth-deployment.yaml"
+        - type: "ReplicaPatcher"
+          resource: "teastore-image-deployment.yaml"
+        - type: "ReplicaPatcher"
+          resource: "teastore-persistence-deployment.yaml"
+        - type: "ReplicaPatcher"
+          resource: "teastore-recommender-deployment.yaml"
+        - type: "ReplicaPatcher"
+          resource: "teastore-webui-deployment.yaml"
+  loadTypes:
+    - typeName: NumUsers
+      patchers:
+        - type: "EnvVarPatcher"
+          resource: "jmeter.yaml"
+          properties:
+            container: jmeter
+            variableName: NUM_USERS
+  slos:
+    - sloType: "generic"
+      name: "uiLatency"
+      prometheusUrl: "http://prometheus-operated:9090"
+      offset: 0
+      properties:
+        externalSloUrl: "http://localhost:8082"
+        promQLQuery: "histogram_quantile(0.95,sum(irate(osm_request_duration_ms_bucket{destination_name='teastore_webui'}[1m])) by (le, destination_name))"
+        warmup: 600 #in seconds
+        queryAggregation: max
+        repetitionAggregation: median
+        operator: lte
+        threshold: 200
+```
+
+#### SUT and Load Generator
+
+We simply use the ConfigMaps created previously as SUT and load generator.
+
+#### Resource Types
+
+We scale the number of replicas of the TeaStore's WebUI, Auth, Image, Persistence, and Recommender services equally to cope with increasing load.
+Hence, our resource type *Instances* defined *ReplicaPatcher*, which modify the number of replicas of all these services.
+
+See our [extended version of this benchmark](https://github.com/SoerenHenning/TeaStore/blob/add-theodolite-example/examples/theodolite/benchmark.yaml), also supporting two other resources types.
+
+#### Load Types
+
+We focus on increasing the load on the TeaStore by increasing the number of concurrent users. Each user is simulated by JMeter and performs a series of UI interactions in an endless loop. Our load type is called *NumUsers* and modifies the `NUM_USERS` environment variable of the JMeter Deployment with an *EnvVarPatcher*.
+
+#### SLOs
+
+The SLO is defined as that the 95th percentile of the response time of the TeaStore's WebUI service must not exceed 200ms.
+If multiple repetitions are performed, the median of the response times is used. Measurements from the first 600 seconds are discarded as warmup.
+
+
+## Run the Benchmark
+
+To run the benchmark, we first have to define an Execution resource, which we afterwards have to apply to our cluster along with the Benchmark resource.
+
+### Create an Execution for our Benchmark
+
+A simple Execution resource for our benchmark could look like this:
+
+```yaml
+apiVersion: theodolite.rocks/v1beta1
+kind: execution
+metadata:
+  name: teastore-example
+spec:
+  benchmark: teastore
+  load:
+    loadType: "NumUsers"
+    loadValues: [5, 10, 15, 20, 25, 30, 35, 40, 45, 50]
+  resources:
+    resourceType: "Instances"
+    resourceValues: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
+  slos:
+    - name: "uiLatency"
+  execution:
+    strategy:
+      name: "RestrictionSearch"
+      restrictions:
+        - "LowerBound"
+      searchStrategy: "LinearSearch"
+    duration: 1200  # in seconds
+    repetitions: 1
+  configOverrides: []
+```
+
+It is named `teastore-example` and defines that we want to execute the benchmark `teastore`.
+We evaluate load intensities from 5 to 50 users and provision 1 to 20 instances per scaled service.
+We apply the [*lower bound restriction search*](concepts/search-strategies#lower-bound-restriction-search), run each experiment for 1200 seconds and perform only one repetition.
+
+### Start the Benchmark
+
+To let Prometheus scrape OSM metrics, we need to create a PodMonitor.
+Download the [PodMonitor from GitHub](https://github.com/SoerenHenning/TeaStore/blob/add-theodolite-example/examples/theodolite/pod-monitors.yaml) (or use the repository already cloned in the previous step) and apply it: (Of course, this could also be made part of the benchmark.)
+
+```sh
+kubectl apply -f pod-monitors.yaml
+```
+
+Next, we need to deploy our benchmark:
+
+```sh
+kubectl apply -f benchmark.yaml
+```
+
+To now start benchmark execution, we deploy our Execution resource defined previously:
+
+```sh
+kubectl apply -f execution-users.yaml
+```
+As described at the [Running Benchmarks](running-benchmarks), we now have to wait, observe the benchmark execution, and finally access the results.
+
+## Further Reading
+
+We published a short paper about this example:
+* S. Henning, B. Wetzel, and W. Hasselbring. “[Cloud-Native Scalability Benchmarking with Theodolite Applied to the TeaStore Benchmark](https://oceanrep.geomar.de/id/eprint/57338/)”. In: *Symposium on Software Performance*. 2022.
+
+You might also want to have a look at the corresponding slides presented at the [Symposium on Software Performance 2022](https://www.performance-symposium.org/fileadmin/user_upload/palladio-conference/2022/presentations/Henning-Cloud-Native-Scalability-Benchmarking-with-Theodolite.pdf).

docs/index.yaml

 apiVersion: v1
 entries:
+  cp-helm-charts:
+  - apiVersion: v1
+    appVersion: "1.0"
+    created: "2023-04-14T16:35:13.695149306+02:00"
+    dependencies:
+    - condition: cp-kafka.enabled
+      name: cp-kafka
+      repository: file://./charts/cp-kafka
+      version: 0.1.0
+    - condition: cp-zookeeper.enabled
+      name: cp-zookeeper
+      repository: file://./charts/cp-zookeeper
+      version: 0.1.0
+    - condition: cp-schema-registry.enabled
+      name: cp-schema-registry
+      repository: file://./charts/cp-schema-registry
+      version: 0.1.0
+    - condition: cp-kafka-rest.enabled
+      name: cp-kafka-rest
+      repository: file://./charts/cp-kafka-rest
+      version: 0.1.0
+    - condition: cp-kafka-connect.enabled
+      name: cp-kafka-connect
+      repository: file://./charts/cp-kafka-connect
+      version: 0.1.0
+    - condition: cp-ksql-server.enabled
+      name: cp-ksql-server
+      repository: file://./charts/cp-ksql-server
+      version: 0.1.0
+    - condition: cp-control-center.enabled
+      name: cp-control-center
+      repository: file://./charts/cp-control-center
+      version: 0.1.0
+    description: A Helm chart for Confluent Platform Community Edition
+    digest: 45c1beba96b77f120f0d05e77be21b8d30431a9f2b63f05087defb54c5f3c60b
+    name: cp-helm-charts
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.6/charts/cp-helm-charts-0.6.0.tgz
+    version: 0.6.0
+  grafana:
+  - apiVersion: v2
+    appVersion: 8.2.5
+    created: "2023-04-14T16:35:13.696909309+02:00"
+    description: The leading tool for querying and visualizing time series and metrics.
+    digest: 56aec8d05f41792656f6a90a3e6ff1516b0e024a64fc2e39040128af9e3459c0
+    home: https://grafana.net
+    icon: https://raw.githubusercontent.com/grafana/grafana/master/public/img/logo_transparent_400x.png
+    kubeVersion: ^1.8.0-0
+    maintainers:
+    - email: zanhsieh@gmail.com
+      name: zanhsieh
+    - email: rluckie@cisco.com
+      name: rtluckie
+    - email: maor.friedman@redhat.com
+      name: maorfr
+    - email: miroslav.hadzhiev@gmail.com
+      name: Xtigyro
+    - email: mail@torstenwalter.de
+      name: torstenwalter
+    name: grafana
+    sources:
+    - https://github.com/grafana/grafana
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.6/charts/grafana-6.17.10.tgz
+    version: 6.17.10
+  kube-prometheus-stack:
+  - annotations:
+      artifacthub.io/links: |
+        - name: Chart Source
+          url: https://github.com/prometheus-community/helm-charts
+        - name: Upstream Project
+          url: https://github.com/prometheus-operator/kube-prometheus
+      artifacthub.io/operator: "true"
+    apiVersion: v2
+    appVersion: 0.60.1
+    created: "2023-04-14T16:35:13.720986306+02:00"
+    dependencies:
+    - condition: kubeStateMetrics.enabled
+      name: kube-state-metrics
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 4.22.*
+    - condition: nodeExporter.enabled
+      name: prometheus-node-exporter
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 4.4.*
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.43.*
+    description: kube-prometheus-stack collects Kubernetes manifests, Grafana dashboards,
+      and Prometheus rules combined with documentation and scripts to provide easy
+      to operate end-to-end Kubernetes cluster monitoring with Prometheus using the
+      Prometheus Operator.
+    digest: 8360468fa9ec4eb2152f5b629b488571dc92be404682deb493207c5b2b552f07
+    home: https://github.com/prometheus-operator/kube-prometheus
+    icon: https://raw.githubusercontent.com/prometheus/prometheus.github.io/master/assets/prometheus_logo-cb55bb5c346.png
+    keywords:
+    - operator
+    - prometheus
+    - kube-prometheus
+    kubeVersion: '>=1.16.0-0'
+    maintainers:
+    - email: andrew@quadcorps.co.uk
+      name: andrewgkew
+    - email: gianrubio@gmail.com
+      name: gianrubio
+    - email: github.gkarthiks@gmail.com
+      name: gkarthiks
+    - email: kube-prometheus-stack@sisti.pt
+      name: GMartinez-Sisti
+    - email: scott@r6by.com
+      name: scottrigby
+    - email: miroslav.hadzhiev@gmail.com
+      name: Xtigyro
+    name: kube-prometheus-stack
+    sources:
+    - https://github.com/prometheus-community/helm-charts
+    - https://github.com/prometheus-operator/kube-prometheus
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.6/charts/kube-prometheus-stack-41.7.4.tgz
+    version: 41.7.4
+  strimzi-kafka-operator:
+  - apiVersion: v2
+    appVersion: 0.29.0
+    created: "2023-04-14T16:35:13.725738879+02:00"
+    description: 'Strimzi: Apache Kafka running on Kubernetes'
+    digest: 87bb22f4b674a91cea51b61edf7c0b3b92c706ac7427534c8f40278c8c712a59
+    home: https://strimzi.io/
+    icon: https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/main/documentation/logo/strimzi_logo.png
+    keywords:
+    - kafka
+    - queue
+    - stream
+    - event
+    - messaging
+    - datastore
+    - topic
+    maintainers:
+    - name: Frawless
+    - name: ppatierno
+    - name: samuel-hawker
+    - name: scholzj
+    - name: tombentley
+    - name: sknot-rh
+    name: strimzi-kafka-operator
+    sources:
+    - https://github.com/strimzi/strimzi-kafka-operator
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.6/charts/strimzi-kafka-operator-helm-3-chart-0.29.0.tgz
+    version: 0.29.0
   theodolite:
+  - apiVersion: v2
+    appVersion: 0.9.0
+    created: "2023-07-19T09:58:16.207401357+02:00"
+    dependencies:
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.17.*
+    - condition: kube-prometheus-stack.enabled
+      name: kube-prometheus-stack
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 41.7.*
+    - condition: strimzi.enabled
+      name: strimzi-kafka-operator
+      repository: https://strimzi.io/charts/
+      version: 0.29.*
+    description: Theodolite is a framework for benchmarking the horizontal and vertical
+      scalability of cloud-native applications.
+    digest: d23f73c4b9c838d45be659cd27e6003b16ae22da52706d4d7111709389ffc9c2
+    home: https://www.theodolite.rocks
+    icon: https://www.theodolite.rocks/assets/logo/theodolite-stacked-transparent.svg
+    maintainers:
+    - email: soeren.henning@jku.at
+      name: Sören Henning
+      url: https://www.jku.at/lit-cyber-physical-systems-lab/ueber-uns/team/dr-ing-soeren-henning/
+    name: theodolite
+    sources:
+    - https://github.com/cau-se/theodolite
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.9.0/theodolite-0.9.0.tgz
+    version: 0.9.0
+  - apiVersion: v2
+    appVersion: 0.8.6
+    created: "2023-04-14T16:35:13.691461495+02:00"
+    dependencies:
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.17.*
+    - condition: kube-prometheus-stack.enabled
+      name: kube-prometheus-stack
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 41.7.*
+    - condition: cp-helm-charts.enabled
+      name: cp-helm-charts
+      repository: https://soerenhenning.github.io/cp-helm-charts
+      version: 0.6.0
+    - condition: strimzi.enabled
+      name: strimzi-kafka-operator
+      repository: https://strimzi.io/charts/
+      version: 0.29.*
+    description: Theodolite is a framework for benchmarking the horizontal and vertical
+      scalability of cloud-native applications.
+    digest: cb50c8f901462b8592232ca3af4a0316c82b89d6c66b83b018b9fdff0f9620e0
+    home: https://www.theodolite.rocks
+    icon: https://www.theodolite.rocks/assets/logo/theodolite-stacked-transparent.svg
+    maintainers:
+    - email: soeren.henning@email.uni-kiel.de
+      name: Sören Henning
+      url: https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc
+    name: theodolite
+    sources:
+    - https://github.com/cau-se/theodolite
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.6/theodolite-0.8.6.tgz
+    version: 0.8.6
+  - apiVersion: v2
+    appVersion: 0.8.5
+    created: "2023-02-09T12:34:28.130334604+01:00"
+    dependencies:
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.17.*
+    - condition: kube-prometheus-stack.enabled
+      name: kube-prometheus-stack
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 41.7.*
+    - condition: cp-helm-charts.enabled
+      name: cp-helm-charts
+      repository: https://soerenhenning.github.io/cp-helm-charts
+      version: 0.6.0
+    - condition: strimzi.enabled
+      name: strimzi-kafka-operator
+      repository: https://strimzi.io/charts/
+      version: 0.29.*
+    description: Theodolite is a framework for benchmarking the horizontal and vertical
+      scalability of cloud-native applications.
+    digest: 6b80c894c6db461a65262553d5dbe8880c4d22edebd150b9a97819a7e3355509
+    home: https://www.theodolite.rocks
+    icon: https://www.theodolite.rocks/assets/logo/theodolite-stacked-transparent.svg
+    maintainers:
+    - email: soeren.henning@email.uni-kiel.de
+      name: Sören Henning
+      url: https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc
+    name: theodolite
+    sources:
+    - https://github.com/cau-se/theodolite
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.5/theodolite-0.8.5.tgz
+    version: 0.8.5
+  - apiVersion: v2
+    appVersion: 0.8.4
+    created: "2023-02-01T14:02:42.124711907+01:00"
+    dependencies:
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.17.*
+    - condition: kube-prometheus-stack.enabled
+      name: kube-prometheus-stack
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 41.7.*
+    - condition: cp-helm-charts.enabled
+      name: cp-helm-charts
+      repository: https://soerenhenning.github.io/cp-helm-charts
+      version: 0.6.0
+    - condition: strimzi.enabled
+      name: strimzi-kafka-operator
+      repository: https://strimzi.io/charts/
+      version: 0.29.*
+    description: Theodolite is a framework for benchmarking the horizontal and vertical
+      scalability of cloud-native applications.
+    digest: 3f2815329938fcb018186d6db251e87ba05243adbbb29582370db6499ed69bcb
+    home: https://www.theodolite.rocks
+    icon: https://www.theodolite.rocks/assets/logo/theodolite-stacked-transparent.svg
+    maintainers:
+    - email: soeren.henning@email.uni-kiel.de
+      name: Sören Henning
+      url: https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc
+    name: theodolite
+    sources:
+    - https://github.com/cau-se/theodolite
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.4/theodolite-0.8.4.tgz
+    version: 0.8.4
+  - apiVersion: v2
+    appVersion: 0.8.3
+    created: "2023-01-31T18:28:08.273346921+01:00"
+    dependencies:
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.17.*
+    - condition: kube-prometheus-stack.enabled
+      name: kube-prometheus-stack
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 41.7.*
+    - condition: cp-helm-charts.enabled
+      name: cp-helm-charts
+      repository: https://soerenhenning.github.io/cp-helm-charts
+      version: 0.6.0
+    - condition: strimzi.enabled
+      name: strimzi-kafka-operator
+      repository: https://strimzi.io/charts/
+      version: 0.29.*
+    description: Theodolite is a framework for benchmarking the horizontal and vertical
+      scalability of cloud-native applications.
+    digest: f0b3ce50db9dec094993073cd8aebf548929e529d399710dda0235d1ea185546
+    home: https://www.theodolite.rocks
+    icon: https://www.theodolite.rocks/assets/logo/theodolite-stacked-transparent.svg
+    maintainers:
+    - email: soeren.henning@email.uni-kiel.de
+      name: Sören Henning
+      url: https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc
+    name: theodolite
+    sources:
+    - https://github.com/cau-se/theodolite
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.3/theodolite-0.8.3.tgz
+    version: 0.8.3
+  - apiVersion: v2
+    appVersion: 0.8.2
+    created: "2022-11-20T11:37:04.711009053+01:00"
+    dependencies:
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.17.*
+    - condition: kube-prometheus-stack.enabled
+      name: kube-prometheus-stack
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 41.7.*
+    - condition: cp-helm-charts.enabled
+      name: cp-helm-charts
+      repository: https://soerenhenning.github.io/cp-helm-charts
+      version: 0.6.0
+    - condition: strimzi.enabled
+      name: strimzi-kafka-operator
+      repository: https://strimzi.io/charts/
+      version: 0.29.*
+    description: Theodolite is a framework for benchmarking the horizontal and vertical
+      scalability of cloud-native applications.
+    digest: b6fc354d08b661dd75beb4e54efd0bb65b488247dcb528fd0c5e365f8f011808
+    home: https://www.theodolite.rocks
+    icon: https://www.theodolite.rocks/assets/logo/theodolite-stacked-transparent.svg
+    maintainers:
+    - email: soeren.henning@email.uni-kiel.de
+      name: Sören Henning
+      url: https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc
+    name: theodolite
+    sources:
+    - https://github.com/cau-se/theodolite
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.2/theodolite-0.8.2.tgz
+    version: 0.8.2
+  - apiVersion: v2
+    appVersion: 0.8.1
+    created: "2022-11-16T09:45:09.130711943+01:00"
+    dependencies:
+    - condition: grafana.enabled
+      name: grafana
+      repository: https://grafana.github.io/helm-charts
+      version: 6.17.5
+    - condition: kube-prometheus-stack.enabled
+      name: kube-prometheus-stack
+      repository: https://prometheus-community.github.io/helm-charts
+      version: 20.0.1
+    - condition: cp-helm-charts.enabled
+      name: cp-helm-charts
+      repository: https://soerenhenning.github.io/cp-helm-charts
+      version: 0.6.0
+    - condition: strimzi.enabled
+      name: strimzi-kafka-operator
+      repository: https://strimzi.io/charts/
+      version: 0.29.0
+    description: Theodolite is a framework for benchmarking the horizontal and vertical
+      scalability of cloud-native applications.
+    digest: 02a1c6a5a8d0295fb9bf2d704cb04e0a17624b83b2a03cd59c1d61b74d8fe4ab
+    home: https://www.theodolite.rocks
+    icon: https://www.theodolite.rocks/assets/logo/theodolite-stacked-transparent.svg
+    maintainers:
+    - email: soeren.henning@email.uni-kiel.de
+      name: Sören Henning
+      url: https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc
+    name: theodolite
+    sources:
+    - https://github.com/cau-se/theodolite
+    type: application
+    urls:
+    - https://github.com/cau-se/theodolite/releases/download/v0.8.1/theodolite-0.8.1.tgz
+    version: 0.8.1
   - apiVersion: v2
     appVersion: 0.8.0
     created: "2022-07-18T17:48:21.205921939+02:00"
@@ -387,4 +787,4 @@ entries:
     urls:
     - https://github.com/cau-se/theodolite/releases/download/v0.4.0/theodolite-0.4.0.tgz
     version: 0.4.0
-generated: "2022-07-18T17:48:21.163757427+02:00"
+generated: "2023-07-19T09:58:16.182895416+02:00"

docs/installation.md

@@ -15,9 +15,6 @@ helm repo update
 helm install theodolite theodolite/theodolite
 ```
 
-This installs Theodolite in operator mode. Operator mode is the easiest to use, but requires some permissions in the installation. If those cannot be granted, Theodolite can also be installed for standalone mode. 
-
-
 ## Installation Options
 
 As usual, the installation via Helm can be configured by passing a values YAML file:
@@ -38,6 +35,16 @@ To store the results of benchmark executions in a [PersistentVolume](https://kub
 You can also use an existing PersistentVolumeClaim by setting `operator.resultsVolume.persistent.existingClaim`.
 If persistence is not enabled, all results will be gone upon pod termination.
 
+### Exposing Grafana
+
+Per default, Theodolite exposes a Grafana instance as NodePort at port `31199`. This can configured by setting `grafana.service.nodePort`.
+
+### Additional Kubernetes cluster metrics
+
+As long as you have sufficient permissions on your cluster, you can integrate additional Kubernetes metrics into Prometheus. This involes enabling some exporters, additional Grafana dashboards and additional permissions. We provide a [values file for enabling extended metrics](https://github.com/cau-se/theodolite/blob/main/helm/preconfigs/extended-metrics.yaml).
+
+See the [kube-prometheus-stack](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) for more details on configuring the individual exporters.
+
 ### Random scheduler
 
 Installation of the random scheduler can be enabled via `randomScheduler.enabled`. Please note that the random scheduler is neither required in operator mode nor in standalone mode. However, it has to be installed if benchmark executions should use random scheduling.

docs/project-info.md

@@ -6,26 +6,41 @@ nav_order: 8
 
 # Project Info
 
-Theodolite is open-source research software, actively maintained at Kiel University's [Software Engineering Group](https://www.se.informatik.uni-kiel.de).
+Theodolite is open-source research software, actively maintained at Kiel University's [Software Engineering Group](https://www.se.informatik.uni-kiel.de) and Johannes Kepler University Linz' [LIT CPS Lab](https://www.jku.at/en/lit-cyber-physical-systems-lab/).
 
 ## Getting Help
 
-To get support with using Theodolite, feel free to directly contact [Sören Henning](https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc).
+To get support with using Theodolite, feel free to directly contact [Sören Henning](https://www.jku.at/en/lit-cyber-physical-systems-lab/about-us/our-team/dr-ing-soeren-henning/).
 You might also want to raise an issue on [GitHub](http://github.com/cau-se/theodolite).
 
 ## Project Management
 
-Theodolite's internal development including issue boards, merge requests and extensive CI pipelines is tracked in our [internal GitLab](https://git.se.informatik.uni-kiel.de/she/theodolite). We provide a public mirror on GitHub, [cau-se/theodolite](http://github.com/cau-se/theodolite), where we are also happy to welcome issues and pull requests.
+Theodolite's internal development including issue boards, merge requests and extensive CI pipelines is tracked in our [internal GitLab](https://git.se.informatik.uni-kiel.de/she/theodolite).
+While all internal development is publicly accessible, contributing requires an account to be set up.
+To ease contribution, we provide a public mirror on GitHub, [cau-se/theodolite](http://github.com/cau-se/theodolite), where we are also happy to welcome issues and pull requests.
+Also releases are published via GitHub. See the following table for an overview:
+
+| Project management | Public GitHub | Internal GitLab |
+|:---|:---|:---|
+| Source code | [GitHub](https://github.com/cau-se/theodolite) | [GitLab](https://git.se.informatik.uni-kiel.de/she/theodolite) |
+| Issue Tracking | [GitHub Issues](https://github.com/cau-se/theodolite/issues) | [GitLab Issues](https://git.se.informatik.uni-kiel.de/she/theodolite/-/issues) |
+| Pull/Merge requests | [GitHub Pull requests](https://github.com/cau-se/theodolite/pulls) | [GitLab Merge requests](https://git.se.informatik.uni-kiel.de/she/theodolite/-/merge_requests) | 
+| Roadmap | | [GitLab Milestones](https://git.se.informatik.uni-kiel.de/she/theodolite/-/milestones) | 
+| CI/CD pipelines | | [GitLab CI/CD](https://git.se.informatik.uni-kiel.de/she/theodolite/-/pipelines) | 
+| Releases   | [GitHub Releases](https://github.com/cau-se/theodolite/releases) | [GitLab Releases](https://git.se.informatik.uni-kiel.de/she/theodolite/-/releases) | 
+| Container images | [GitHub Packages](https://github.com/orgs/cau-se/packages?repo_name=theodolite) | |
 
 ## Contributors
 
-* [Sören Henning](https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc) (Maintainer)
+* [Sören Henning](https://www.jku.at/en/lit-cyber-physical-systems-lab/about-us/our-team/dr-ing-soeren-henning/) (Maintainer)
 * [Marcel Becker](https://www.linkedin.com/in/marcel-becker-11b39b246)
 * [Jan Bensien](https://oceanrep.geomar.de/id/eprint/52342/)
 * [Nico Biernat](https://github.com/NicoBiernat)
 * [Lorenz Boguhn](https://github.com/lorenzboguhn)
 * [Simon Ehrenstein](https://github.com/sehrenstein)
 * [Willi Hasselbring](https://www.se.informatik.uni-kiel.de/en/team/prof.-dr.-wilhelm-willi-hasselbring)
+* [Christopher Konkel](https://github.com/JustAnotherChristoph)
+* [Luca Mertens](https://www.linkedin.com/in/luca-mertens-35a932201)
 * [Tobias Pfandzelter](https://pfandzelter.com/)
 * [Julia Rossow](https://www.linkedin.com/in/julia-rossow/)
 * [Björn Vonheiden](https://github.com/bvonheid)

docs/publications.md

@@ -8,10 +8,13 @@ nav_order: 9
 
 Below you can find a list of publications that are directly related to Theodolite:
 
+* S. Henning. “[Scalability Benchmarking of Cloud-Native Applications Applied to Event-Driven Microservices](https://doi.org/10.21941/kcss/2023/2)”. In: *Kiel Computer Science Series 2023/2*. 2023. Dissertation, Faculty of Engineering, Kiel University. DOI: [10.21941/kcss/2023/2](https://doi.org/10.21941/kcss/2023/2).
+* S. Henning and W. Hasselbring. “[Benchmarking Scalability of Cloud-Native Applications](https://dl.gi.de/bitstream/handle/20.500.12116/40081/paper16.pdf)”. In: *Software Engineering*. 2023.
 * S. Henning and W. Hasselbring. “[A Configurable Method for Benchmarking Scalability of Cloud-Native Applications](https://doi.org/10.1007/s10664-022-10162-1)”. In: *Empirical Software Engineering* 27. 2022. DOI: [10.1007/s10664-022-10162-1](https://doi.org/10.1007/s10664-022-10162-1).
-* T. Pfandzelter, S. Henning, T. Schirmer, W. Hasselbring, and D. Bermbach. “[Streaming vs. Functions: A Cost Perspective on Cloud Event Processing](https://arxiv.org/pdf/2204.11509.pdf)”. In: *IEEE International Conference on Cloud Engineering*. 2022. In press.
-* S. Henning and W. Hasselbring. “Demo Paper: Benchmarking Scalability of Cloud-Native Applications with Theodolite”. In: *IEEE International Conference on Cloud Engineering*. 2022. In press.
+* T. Pfandzelter, S. Henning, T. Schirmer, W. Hasselbring, and D. Bermbach. “[Streaming vs. Functions: A Cost Perspective on Cloud Event Processing](https://arxiv.org/pdf/2204.11509.pdf)”. In: *IEEE International Conference on Cloud Engineering*. 2022. DOI: [10.1109/IC2E55432.2022.00015](https://doi.org/10.1109/IC2E55432.2022.00015).
+* S. Henning and W. Hasselbring. “[Demo Paper: Benchmarking Scalability of Cloud-Native Applications with Theodolite](https://oceanrep.geomar.de/id/eprint/57336/)”. In: *IEEE International Conference on Cloud Engineering*. 2022. DOI: [10.1109/IC2E55432.2022.00037](https://doi.org/10.1109/IC2E55432.2022.00037).
+* S. Henning, B. Wetzel, and W. Hasselbring. “[Cloud-Native Scalability Benchmarking with Theodolite Applied to the TeaStore Benchmark](https://oceanrep.geomar.de/id/eprint/57338/)”. In: *Softwaretechnik-Trends* 43 (1) (Symposium on Software Performance). 2022.
 * S. Henning, B. Wetzel, and W. Hasselbring. “[Reproducible Benchmarking of Cloud-Native Applications With the Kubernetes Operator Pattern](http://ceur-ws.org/Vol-3043/short5.pdf)”. In: *Symposium on Software Performance*. 2021.
 * S. Henning and W. Hasselbring. “[How to Measure Scalability of Distributed Stream Processing Engines?](https://research.spec.org/icpe_proceedings/2021/companion/p85.pdf)” In: *Companion of the ACM/SPEC International Conference on Performance Engineering*. 2021. DOI: [10.1145/3447545.3451190](https://doi.org/10.1145/3447545.3451190).
 * S. Henning and W. Hasselbring. “[Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures](https://arxiv.org/abs/2009.00304)”. In: *Big Data Research* 25. 2021. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209).
-* S. Henning and W. Hasselbring. “[Toward Efficient Scalability Benchmarking of Event-Driven Microservice Architectures at Large Scale](https://www.performance-symposium.org/fileadmin/user_upload/palladio-conference/2020/Papers/SSP2020_paper_14.pdf)”. In: *Softwaretechnik-Trends* 40 (3) (Symposium on Software Performance). 2020.
+* S. Henning and W. Hasselbring. “[Toward Efficient Scalability Benchmarking of Event-Driven Microservice Architectures at Large Scale](https://fb-swt.gi.de/fileadmin/FB/SWT/Softwaretechnik-Trends/Verzeichnis/Band_40_Heft_3/SSP2020_Henning.pdf)”. In: *Softwaretechnik-Trends* 40 (3) (Symposium on Software Performance). 2020.

docs/quickstart.md

@@ -18,6 +18,14 @@ All you need to get started is access to a Kubernetes cluster plus kubectl and H
    helm install theodolite theodolite/theodolite -f https://raw.githubusercontent.com/cau-se/theodolite/main/helm/preconfigs/minimal.yaml
    ```
 
+   After installation, it may take some time until all components are ready. You can check the status of the installation by running:
+
+   ```sh
+   kubectl get pods
+   ```
+
+   In particular, the Kafka Schema Registry may restart a couple of times.
+
 1. Get the Theodolite examples from the [Theodolite repository](https://github.com/cau-se/theodolite) and `cd` into its example directory:
 
    ```sh

docs/theodolite-benchmarks/benchmark-uc1.md

+---
+title: Benchmark UC1
+parent: Streaming Benchmarks
+has_children: false
+nav_order: 1
+---
+
+# Benchmark UC1: Database Storage
+
+A simple, but common use case in event-driven architectures is that events or messages should be stored permanently, for example, in a NoSQL database.
+
+## Dataflow Architecture
+
+![Theodolite Benchmark UC1: Database Storage](../../assets/images/arch-uc1.svg){: .d-block .mx-auto }
+
+The first step is to read data records from a messaging system. Then, these records are converted into another data format in order to match the often different formats required by the database. Finally, the converted records are written to an external database. 
+Per default, this benchmark does not use a real database, but instead writes all incoming records to system out. Otherwise, due to the simple, stateless stream processing topology, the benchmark would primarily test the database’s write capabilities. However, for implementations of some stream processing engines, also a real database can be configured.
+
+## Further Reading
+
+S. Henning and W. Hasselbring. “[Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures](https://arxiv.org/abs/2009.00304)”. In: *Big Data Research* 25. 2021. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209).

docs/theodolite-benchmarks/benchmark-uc2.md

+---
+title: Benchmark UC2
+parent: Streaming Benchmarks
+has_children: false
+nav_order: 1
+---
+
+# Benchmark UC2: Downsampling
+
+Another common use case for stream processing architectures is reducing the amount of events, messages, or measurements by aggregating multiple records within consecutive, non-overlapping time windows. Typical aggregations compute the average, minimum, or maximum of measurements within a time window or
+count the occurrence of same events. Such reduced amounts of data are required, for example, to save computing resources or to provide a better user experience (e.g., for data visualizations).
+When using aggregation windows of fixed size that succeed each other without gaps (called [tumbling windows](https://kafka.apache.org/30/documentation/streams/developer-guide/dsl-api.html#tumbling-time-windows) in many stream processing engines), the (potentially varying) message frequency is reduced to a constant value.
+This is also referred to as downsampling. Downsampling allows for applying many machine learning methods that require data of a fixed frequency.
+
+## Dataflow Architecture
+
+![Theodolite Benchmark UC2: Downsampling](../../assets/images/arch-uc2.svg){: .d-block .mx-auto }
+
+The dataflow architecture first reads measurement data from an input stream and then assigns each measurement to a time window of fixed, but statically configurable size. Afterwards, an aggregation operator computes the summary statistics sum, count, minimum, maximum, average and population variance for a time window. Finally, the aggregation result containing all summary statistics is written to an output stream.
+
+## Further Reading
+
+S. Henning and W. Hasselbring. “[Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures](https://arxiv.org/abs/2009.00304)”. In: *Big Data Research* 25. 2021. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209).

docs/theodolite-benchmarks/benchmark-uc3.md

+---
+title: Benchmark UC3
+parent: Streaming Benchmarks
+has_children: false
+nav_order: 1
+---
+
+# Benchmark UC3: Time Attribute-Based Aggregation
+
+A second type of temporal aggregation is aggregating messages that have the same time attribute. Such a time attribute is, for example, the hour of day, day of week, or day in the year. This type of aggregation can be used to compute, for example, an average course over the day, the week, or the year. It allows to demonstrate or discover seasonal patterns in the data.
+
+## Dataflow Architecture
+
+![Theodolite Benchmark UC3: Time Attribute-Based Aggregation](../../assets/images/arch-uc3.svg){: .d-block .mx-auto }
+
+The first step is to read measurement data from the input stream. Then, a new key is set for each message, which consists of the original key (i.e., the identifier of a sensor) and the selected time attribute (e.g., day of week) extracted from the record’s timestamp. In the next step, the message is duplicated for each sliding window it is contained in. Then, all measurements of the same sensor and the same time attribute are aggregated for each sliding time window by computing the summary statistics sum, count, minimum, maximum, average and population variance. The aggregation results per identifier, time attribute, and window are written to an output stream.
+
+## Further Reading
+
+S. Henning and W. Hasselbring. “[Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures](https://arxiv.org/abs/2009.00304)”. In: *Big Data Research* 25. 2021. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209).

docs/theodolite-benchmarks/benchmark-uc4.md

+---
+title: Benchmark UC4
+parent: Streaming Benchmarks
+has_children: false
+nav_order: 1
+---
+
+# Benchmark UC4: Hierarchical Aggregation
+
+For analyzing sensor data, often not only the individual measurements of sensors are of interest, but also aggregated data for
+groups of sensors. When monitoring energy consumption in industrial facilities, for example, comparing the total consumption
+of machine types often provides better insights than comparing the consumption of all individual machines. Additionally, it may
+be necessary to combine groups further into larger groups and adjust these group hierarchies at runtime.
+
+## Dataflow Architecture
+
+![Theodolite Benchmark UC4: Hierarchical Aggregation](../../assets/images/arch-uc4.svg){: .d-block .mx-auto }
+
+The dataflow architecture requires two input data streams: a stream of sensor measurements and a stream tracking changes to the hierarchies of sensor groups. In the consecutive steps, both streams are joined, measurements are duplicated for each relevant group, assigned to time windows, and the measurements for all sensors in a group per window are aggregated. Finally, the aggregation results are exposed via a new data stream. Additionally, the output stream is fed back as an input stream in order to compute aggregations for groups containing subgroups. To also support unknown record frequencies, this dataflow architecture can be configured to use sliding windows instead of tumbling windows (see [further reading](#further-reading)).
+
+## Further Reading
+
+* S. Henning and W. Hasselbring. “[Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures](https://arxiv.org/abs/2009.00304)”. In: *Big Data Research* 25. 2021. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209).
+* S. Henning and W. Hasselbring. “[Scalable and reliable multi-dimensional sensor data aggregation in data-streaming architectures](https://doi.org/10.1007/s41688-020-00041-3)”. In: *Data-Enabled Discovery and Applications* 4.1. 2020. DOI: [10.1007/s41688-020-00041-3](https://doi.org/10.1007/s41688-020-00041-3).
\ No newline at end of file

docs/theodolite-benchmarks/index.md

 ---
-title: Available Benchmarks
+title: Streaming Benchmarks
 has_children: true
 nav_order: 7
 ---
 
-# Theodolite Benchmarks
+# Theodolite's Stream Processing Benchmarks
 
-Theodolite comes with 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding [load generator](load-generator) is provided. Currently, Theodolite provides benchmark implementations for Apache Kafka Streams and Apache Flink.
+Theodolite comes with 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding [load generator](load-generator) is provided. Currently, Theodolite provides benchmark implementations for Apache Kafka Streams, Apache Flink, Hazelcast Jet and Apache Beam (with Samza and Flink).
 
+Theodolite's benchmarks (labeled UC1--UC4) represent some sort of event-driven microservice performing Industrial Internet of Things data analytics. Specifically, they are derived from a microservice-based research software for analyzing industrial power consumption data streams (the [Titan Control Center](https://github.com/cau-se/titan-ccp)).
 
-Theodolite's benchmarks are based on typical use cases for stream processing within microservices. Specifically, all benchmarks represent some sort of microservice doing Industrial Internet of Things data analytics. 
+| Stream processing engine  | [UC1](benchmark-uc1) | [UC2](benchmark-uc2) | [UC3](benchmark-uc3) | [UC4](benchmark-uc4) |
+|:--------------------------|:---:|:---:|:---:|:---:|
+| Apache Kafka Streams      | ✓   | ✓   | ✓   | ✓   |
+| Apache Flink              | ✓   | ✓   | ✓   | ✓   |
+| Hazelcast Jet             | ✓   | ✓   | ✓   | ✓   |
+| Apache Beam (Samza/Flink) | ✓   | ✓   | ✓   | ✓   |
 
-## UC1: Database Storage
+## Installation
 
-A simple, but common use case in event-driven architectures is that events or messages should be stored permanently, for example, in a NoSQL database.
+When [installing Theodolite](../installation) with Helm and the default configuration, also our stream processing benchmarks are automatically installed.
+This can be verified by running `kubectl get benchmarks`, which should yield something like:
 
+```
+NAME                AGE     STATUS
+uc1-beam-flink      2d20h   Ready
+uc1-beam-samza      2d20h   Ready
+uc1-flink           2d20h   Ready
+uc1-hazelcastjet    2d16h   Ready
+uc1-kstreams        2d20h   Ready
+uc2-beam-flink      2d20h   Ready
+uc2-beam-samza      2d20h   Ready
+uc2-flink           2d20h   Ready
+uc2-hazelcastjet    2d16h   Ready
+uc2-kstreams        2d20h   Ready
+uc3-beam-flink      2d20h   Ready
+uc3-beam-samza      2d20h   Ready
+uc3-flink           2d20h   Ready
+uc3-hazelcastjet    2d16h   Ready
+uc3-kstreams        2d20h   Ready
+uc4-beam-flink      2d20h   Ready
+uc4-beam-samza      2d20h   Ready
+uc4-flink           2d20h   Ready
+uc4-hazelcastjet    2d16h   Ready
+uc4-kstreams        2d20h   Ready
+```
 
-## UC2: Downsampling
+Alternatively, all benchmarks can also be found at [GitHub](https://github.com/cau-se/theodolite/tree/main/theodolite-benchmarks/definitions) and installed manually with `kubectl apply -f <benchmark-yaml-file>`. Additionally, you would need to package the benchmarks' Kubernetes resources into a ConfigMap by running:
 
-Another common use case for stream processing architectures is reducing the amount of events, messages, or measurements by aggregating multiple records within consecutive, non-overlapping time windows. Typical aggregations compute the average, minimum, or maximum of measurements within a time window or
-count the occurrence of same events. Such reduced amounts of data are required, for example, to save computing resources or to provide a better user experience (e.g., for data visualizations).
-When using aggregation windows of fixed size that succeed each other without gaps (called [tumbling windows](https://kafka.apache.org/30/documentation/streams/developer-guide/dsl-api.html#tumbling-time-windows) in many stream processing engines), the (potentially varying) message frequency is reduced to a constant value.
-This is also referred to as downsampling. Downsampling allows for applying many machine learning methods that require data of a fixed frequency.
+```sh
+kubectl create configmap <configmap-name-required-by-benchmark> --from-file <directory-with-benchmark-resources>
+```
 
+See the [install-configmaps.sh](https://github.com/cau-se/theodolite/blob/main/theodolite-benchmarks/definitions/install-configmaps.sh) script for examples.
 
-## UC3: Time Attribute-Based Aggregation
+## Running Benchmarks
 
-A second type of temporal aggregation is aggregating messages that have the same time attribute. Such a time attribute is, for example, the hour of day, day of week, or day in the year. This type of aggregation can be used to compute, for example, an average course over the day, the week, or the year. It allows to demonstrate or discover seasonal patterns in the data.
+To run a benchmark, you need to create and apply an `Execution` YAML file as described in the [running benchmarks documentation](../running-benchmarks).
+Some preliminary results of our benchmarks can be found in our publication:
 
-## UC4: Hierarchical Aggregation
+* S. Henning and W. Hasselbring. “[Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures](https://arxiv.org/abs/2009.00304)”. In: *Big Data Research* 25. 2021. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209).
 
-For analyzing sensor data, often not only the individual measurements of sensors are of interest, but also aggregated data for
-groups of sensors. When monitoring energy consumption in industrial facilities, for example, comparing the total consumption
-of machine types often provides better insights than comparing the consumption of all individual machines. Additionally, it may
-be necessary to combine groups further into larger groups and adjust these group hierarchies at runtime.
+## Control the Number of Load Generator Instances
+
+Depending on the load to be generated, the Theodolite benchmarks create multiple load generator instances.
+Per default, a single instance will generate up to 150&thinsp;000 messages per second.
+If higher loads are to be generated, accordingly more instances are deployed.
+However, the actual load that can be generated by a single load generator instance depends on the cluster configuration and might be lower.
+To change the maximum number of messages per instance, run the following commands.
+Set the `MAX_RECORDS_PER_INSTANCE` variable to the number of messages a single instance can generate in your cluster (use our Grafana dashboard to figure out that value).
+
+```sh
+export MAX_RECORDS_PER_INSTANCE=150000 # Change to your desired value
+kubectl patch benchmarks uc1-beam-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc1-beam-samza --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc1-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc1-hazelcastjet --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc1-kstreams --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc2-beam-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc2-beam-samza --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc2-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc2-hazelcastjet --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc2-kstreams --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc3-beam-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc3-beam-samza --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc3-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc3-hazelcastjet --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc3-kstreams --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc4-beam-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc4-beam-samza --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc4-flink --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc4-hazelcastjet --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+kubectl patch benchmarks uc4-kstreams --type json --patch "[{op: replace, path: /spec/loadTypes/0/patchers/1/properties/loadGenMaxRecords, value: $MAX_RECORDS_PER_INSTANCE}]"
+```
\ No newline at end of file

docs/theodolite-benchmarks/load-generator.md

 ---
 title: Load Generators
-parent: Available Benchmarks
+parent: Streaming Benchmarks
 has_children: false
-nav_order: 1
+nav_order: 5
 ---
 
 # Load Generator Framework

execution/README.md

-# Theodolite Execution Framework
-
-**Please note: Most of the content in this directory is deprecated. Please refer to the `helm` directory for installing the latest version of Theodolite.**
-
-This directory contains the Theodolite framework for executing scalability
-benchmarks in a Kubernetes cluster. As Theodolite aims for executing benchmarks
-in realistic execution environments, some third-party components are [required](#installation).
-After everything is installed and configured, you can move on the [execution of
-benchmarks](#execution).
-
-## Installation
-
-For executing benchmarks, access to a Kubernetes cluster is required. If you already run other applications inside your
-cluster, you might want to consider creating a dedicated namespace for your benchmarks.
-
-### Installing Dependencies
-
-The following third-party services need to be installed in your cluster. For most of them, the suggested way to install
-them is via [Helm](https://helm.sh).
-
-#### Prometheus
-
-We suggest to use the [Prometheus Operator](https://github.com/coreos/prometheus-operator)
-and create a dedicated Prometheus instance for these benchmarks.
-
-If Prometheus Operator is not already available on your cluster, a convenient way to install it is via the
-[Prometheus community Helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack).
-As you may not need an entire cluster monitoring stack, you can use our Helm configuration to only install the
-operator:
-
-```sh
-helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
-helm repo update
-helm install prometheus-operator prometheus-community/kube-prometheus-stack -f infrastructure/prometheus/helm-values.yaml
-```
-
-After installation, you need to create a Prometheus instance:
-
-```sh
-kubectl apply -f infrastructure/prometheus/prometheus.yaml
-```
-
-You might also need to apply the [ClusterRole](infrastructure/prometheus/cluster-role.yaml), the
-[CusterRoleBinding](infrastructure/prometheus/cluster-role-binding.yaml) and the
-[ServiceAccount](infrastructure/prometheus/service-account.yaml), depending on your cluster's security
-policies. If you are not in the *default* namespace, alter the namespace in
-[Prometheus' ClusterRoleBinding](infrastructure/prometheus/cluster-role-binding.yaml) accordingly.
-
-```sh
-kubectl apply -f infrastructure/prometheus/cluster-role.yaml
-kubectl apply -f infrastructure/prometheus/cluster-role-binding.yaml
-kubectl apply -f infrastructure/prometheus/service-account.yaml
-```
-
-For the individual benchmarking components to be monitored, [ServiceMonitors](https://github.com/coreos/prometheus-operator#customresourcedefinitions)
-are used. See the corresponding sections below for how to install them.
-
-#### Grafana
-
-As with Prometheus, we suggest to create a dedicated Grafana instance. Grafana
-with our default configuration can be installed with Helm:
-
-```sh
-helm repo add grafana https://grafana.github.io/helm-charts
-helm repo update
-helm install grafana grafana/grafana -f infrastructure/grafana/values.yaml
-```
-
-The official [Grafana Helm Chart repository](https://github.com/helm/charts/tree/master/stable/grafana)
-provides further documentation including a table of configuration options.
-
-We provide ConfigMaps for a [Grafana dashboard](infrastructure/grafana/dashboard-config-map.yaml) and a
-[Grafana data source](infrastructure/grafana/prometheus-datasource-config-map.yaml). Create them as follows:
-
-```sh
-kubectl apply -f infrastructure/grafana/dashboard-config-map.yaml
-kubectl apply -f infrastructure/grafana/prometheus-datasource-config-map.yaml
-```
-
-#### A Kafka cluster
-
-We suggest to set up a Kafka cluster via [Confluent's Helm Charts](https://github.com/confluentinc/cp-helm-charts).
-Currently, these charts do not expose all metrics, we would like to display. Thus, we provide a patched version of this chart.
-We also provide a [default configuration](infrastructure/kafka/values.yaml). If you do
-not want to deploy 10 Kafka and 3 Zookeeper instances, alter the configuration
-file accordingly. To install the patched Confluent's Kafka with our configuration:
-
-```sh
-helm install my-confluent https://github.com/SoerenHenning/cp-helm-charts/releases/download/v6.0.1-1-JMX-FIX-2/cp-helm-charts-0.6.0.tgz -f infrastructure/kafka/values.yaml
-```
-
-To let Prometheus scrape Kafka metrics, deploy a ServiceMonitor:
-
-```sh
-kubectl apply -f infrastructure/kafka/service-monitor.yaml
-```
-
-Other Kafka deployments, for example, using Strimzi, should work in a similar way.
-
-*Please note that currently, even if installed differently, the corresponding services must run at
-`my-confluent-cp-kafka:9092`, `my-confluent-cp-zookeeper:2181` and `my-confluent-cp-schema-registry:8081`.*
-
-#### A Kafka Client Pod
-
-A permanently running pod used for Kafka configuration is started via:
-
-```sh
-kubectl apply -f infrastructure/kafka/kafka-client.yaml
-```
-
-#### A Zookeeper Client Pod
-
-Also a permanently running pod for ZooKeeper access is started via:
-
-```sh
-kubectl apply -f infrastructure/zookeeper-client.yaml
-```
-
-#### The Kafka Lag Exporter
-
-[Lightbend's Kafka Lag Exporter](https://github.com/lightbend/kafka-lag-exporter)
-can be installed via Helm. We also provide a [default configuration](infrastructure/kafka-lag-exporter/values.yaml).
-To install it:
-
-```sh
-helm repo add kafka-lag-exporter https://lightbend.github.io/kafka-lag-exporter/repo/
-helm repo update
-helm install kafka-lag-exporter kafka-lag-exporter/kafka-lag-exporter -f infrastructure/kafka-lag-exporter/values.yaml
-```
-
-### Installing Theodolite
-
-While Theodolite itself has not be installed as it is loaded at runtime (see [execution](#Execution)), it requires some
-resources to be deployed in your cluster. These resources are grouped under RBAC and Volume in the following paragraphs.
-
-#### Theodolite RBAC
-
-**The following step is only required if RBAC is enabled in your cluster.** If you are not sure whether this is the
-case, you want to simply try it without the following step.
-
-If RBAC is enabled in your cluster, you have to allow Theodolite to start and stop pods etc. To do so, deploy the RBAC
-resources via:
-
-```sh
-kubectl apply -f infrastructure/kubernetes/rbac/role.yaml
-kubectl apply -f infrastructure/kubernetes/rbac/role-binding.yaml
-kubectl apply -f infrastructure/kubernetes/rbac/service-account.yaml
-```
-
-#### Theodolite Volume
-
-In order to persistently store benchmark results, Theodolite needs a volume mounted. We provide pre-configured
-declarations for different volume types.
-
-##### *hostPath* volume
-
-Using a [hostPath volume](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath) is the easiest option when
-running Theodolite locally, e.g., with minikube or kind.
-
-Just modify `infrastructure/kubernetes/volume-hostpath.yaml` by setting `path` to the directory on your host machine where
-all benchmark results should be stored and run:
-
-```sh
-kubectl apply -f infrastructure/kubernetes/volume-hostpath.yaml
-```
-
-##### *local* volume
-
-A [local volume](https://kubernetes.io/docs/concepts/storage/volumes/#local) is a simple option to use when having
-access (e.g. via SSH) to one of your cluster nodes.
-
-You first need to create a directory on a selected node where all benchmark results should be stored. Next, modify
-`infrastructure/kubernetes/volume-local.yaml` by setting `<node-name>` to your selected node. (This node will most
-likely also execute the [Theodolite job](#Execution).) Further, you have to set `path` to the directory on the node you just created. To deploy
-your volume run:
-
-```sh
-kubectl apply -f infrastructure/kubernetes/volume-local.yaml
-```
-
-##### *Oracle Cloud Infrastructure* volume
-
-When you are running in the Oracle Cloud, you can provision a persistent volume claim by attaching a volume from the
-Oracle Cloud Infrastructure Block Volume service. To create your volume, run: 
-
-```sh
-kubectl apply -f infrastructure/kubernetes/volume-oci.yaml
-```
-
-More information can be found in the official documentation:
-[Oracle Cloud Infrastructure: Creating a Persistent Volume Claim](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm)
-
-##### Other volumes
-
-To use volumes provided by other public cloud providers or network-based file systems, you can use the definitions in
-`infrastructure/kubernetes/` as a starting point. See the offical
-[volumes documentation](https://kubernetes.io/docs/concepts/storage/volumes/) for additional information.
-
-##### Accessing benchmark results via Kubernetes
-
-In cases where you do not have direct access to the underlying storage infrasturcture of your volume (e.g., if your
-admin configures a local or hostPath volume for you and you do not have SSH access to the node), you can deploy our
-Theodolite results access deployment:
-
-```sh
-kubectl apply -f infrastructure/kubernetes/volume-access.yaml
-```
-
-It allows you to browse the benchmark results or copy files your Kubernetes client via the following commands:
-
-```sh
-kubectl exec -it $(kubectl get pods -o=name -l app=theodolite-results-access) -- sh
-kubectl cp $(kubectl get pods --no-headers -o custom-columns=":metadata.name" -l app=theodolite-results-access):app/results <target-dir>
-```
-
-
-## Execution
-
-The preferred way to run scalability benchmarks with Theodolite is to deploy Theodolite
-[Kubernetes Jobs](https://kubernetes.io/docs/concepts/workloads/controllers/job/) in your cluster. For running
-Theodolite locally on your machine see the description below.
-
-`theodolite.yaml` provides a template for your own Theodolite job. To run your own job, create a copy, give it a name
-(`metadata.name`) and adjust configuration parameters as desired. For a description of available configuration options
-see the [Configuration](#configuration) section below. Note, that you might uncomment the `serviceAccountName` line if
-RBAC is enabled on your cluster (see installation of [Theodolite RBAC](#Theodolite-RBAC)).
-
-To start the execution of a benchmark create a ConfigMap which containts all required Kubernetes resource files for the SUT and the load generator, a ConfigMap for the execution and a ConfigMap for the benchmark.
-
-```sh
-kubectl create configmap app-resources-configmap --from-file=<folder-with-all-required-k8s-resources>
-kubectl create configmap execution-configmap --from-file=<execution.yaml>
-kubectl create configmap benchmark-configmap --from-file=<benchmark.yaml>
-```
-
-This will create three ConfigMaps. You can verify this via `kubectl get configmaps`.
-
-Start the Theodolite job (with `<your-theodolite-yaml>` being your job definition):
-
-```sh
-kubectl create -f <your-theodolite-yaml>
-```
-
-This will create a pod with a name such as `your-job-name-xxxxxx`. You can verifiy this via `kubectl get pods`. With
-`kubectl logs -f <your-job-name-xxxxxx>`, you can follow the benchmark execution logs.
-
-Once your job is completed (you can verify via `kubectl get jobs), its results are stored inside your configured
-Kubernetes volume.
-
-**Make sure to always run only one Theodolite job at a time.**
-
-### Configuration
-
-Be sure, that the names of the configmap corresponds correctly to the specifications of the mounted `configmaps`, `volumes`, `mountPath`. In particular: The name of the execution file and the benchmark file must match the value of the corresponding environment variable.
-
-### Domain Restriction
-
-For dimension value, we have a domain of the amounts of instances. As a consequence, for each dimension value the maximum number of lag experiments is equal to the size of the domain. How the domain is determined is defined by the following domain restriction strategies.
-
-* `no-domain-restriction`: For each dimension value, the domain of instances is equal to the set of all amounts of instances.
-* `restrict-domain`: For each dimension value, the domain is computed as follows:
-    * If the dimension value is the smallest dimension value the domain of the amounts of instances is equal to the set of all amounts of instances.
-    * If the dimension value is not the smallest dimension value and N is the amount of minimal amount of instances that was suitable for the last smaller dimension value the domain for this dimension value contains all amounts of instances greater than, or equal to N.
-
-### Benchmarking Search Strategies
-
-There are the following benchmarking strategies:
-
-* `check-all`: For each dimension value, execute one lag experiment for all amounts of instances within the current domain.
-* `linear-search`: A heuristic which works as follows: For each dimension value, execute one lag experiment for all number of instances within the current domain. The execution order is from the lowest number of instances to the highest amount of instances and the execution for each dimension value is stopped, when a suitable amount of instances is found or if all lag experiments for the dimension value were not successful.
-* `binary-search`: A heuristic which works as follows: For each dimension value, execute one lag experiment for all number of instances within the current domain. The execution order is in a binary-search-like manner. The execution is stopped, when a suitable amount of instances is found or if all lag experiments for the dimension value were not successful.
-
-## Observation
-
-The installed Grafana instance provides a dashboard to observe the benchmark execution. Unless configured otherwise,
-this dashboard can be accessed via `http://<cluster-ip>:31199` or via `http://localhost:31199` if proxied with
-`kubectl port-forward svc/grafana 8080:service`. Default credentials are user *admin* with password *admin*.
-
-
-## Local Execution (e.g. for Development)
-
-As an alternative to executing Theodolite as a Kubernetes Job, it is also possible to run it from your local system,
-for example, for development purposes. In addition to the generel installation instructions, the following adjustments
-are neccessary.
-
-### Installation
-
-For local execution a **Python 3.7** installation is required. We suggest to use a virtual environment placed in the `.venv`
-directory (in the Theodolite root directory). A set of requirements is needed. You can install them with the following
-command (make sure to be in your virtual environment if you use one):
-
-```sh
-pip install -r requirements.txt
-```
-
-Kubernetes volumes and service accounts, roles, and role bindings for Theodolite are not required in this case.
-
-### Local Execution
-
-The `theodolite.py` is the entrypoint for all benchmark executions. Is has to be called as follows:
-
-```python
-python theodolite.py --uc <uc> --loads <load> [<load> ...] --instances <instances> [<instances> ...]
-```
-
-This command is the minimal command for execution. Further configurations options are described [above](#configuration)
-or available via `python theodolite.py -h`.
\ No newline at end of file

execution/infrastructure/grafana/dashboard-config-map.yaml

-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: scalability
-  labels:
-    grafana_dashboard: "1"
-data:
-  k8s-dashboard.json: |-
-    {
-    "annotations": {
-      "list": [
-        {
-          "builtIn": 1,
-          "datasource": "-- Grafana --",
-          "enable": true,
-          "hide": true,
-          "iconColor": "rgba(0, 211, 255, 1)",
-          "name": "Annotations & Alerts",
-          "type": "dashboard"
-        }
-      ]
-    },
-    "editable": true,
-    "gnetId": null,
-    "graphTooltip": 0,
-    "id": 2,
-    "iteration": 1589140028684,
-    "links": [],
-    "panels": [
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 7,
-          "w": 12,
-          "x": 0,
-          "y": 0
-        },
-        "hiddenSeries": false,
-        "id": 2,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_input)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{Messages In Per Second}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Messages In Per Second",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 7,
-          "w": 12,
-          "x": 12,
-          "y": 0
-        },
-        "hiddenSeries": false,
-        "id": 3,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_output)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{Messages Out Per Second}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Messages Out Per Second",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 0,
-          "y": 7
-        },
-        "hiddenSeries": false,
-        "id": 9,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": true,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "sum by(group, topic) (kafka_consumergroup_group_lag >= 0)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{topic}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Record Lag",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 12,
-          "y": 7
-        },
-        "hiddenSeries": false,
-        "id": 5,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "count(count (kafka_consumer_consumer_fetch_manager_metrics_records_lag) by(pod))",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "instances",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Number of Instances",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "decimals": 0,
-            "format": "short",
-            "label": "",
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 0,
-          "y": 15
-        },
-        "hiddenSeries": false,
-        "id": 10,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "sum by(group,topic) (kafka_consumergroup_group_offset > 0)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{topic}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Records Consumed",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 12,
-          "y": 15
-        },
-        "hiddenSeries": false,
-        "id": 12,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "count by(group,topic) (kafka_consumergroup_group_offset > 0)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{topic}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Number of Partitions",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 0,
-          "y": 23
-        },
-        "hiddenSeries": false,
-        "id": 11,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "sum by(group,topic) (kafka_partition_latest_offset)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{topic}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Records Produced (Kafka Lag Exporter)",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 12,
-          "y": 23
-        },
-        "hiddenSeries": false,
-        "id": 8,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "count by(job, topic) (kafka_consumer_consumer_fetch_manager_metrics_records_lag)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{topic}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Number of Partitions (Kafka Streams Export)",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "decimals": null,
-            "format": "short",
-            "label": "",
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 0,
-          "y": 31
-        },
-        "hiddenSeries": false,
-        "id": 4,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": false,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "sum by(job, topic) (kafka_consumer_consumer_fetch_manager_metrics_records_lag)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "{{topic}}",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Record Lag (Kafka Streams Export)",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      },
-      {
-        "aliasColors": {},
-        "bars": false,
-        "dashLength": 10,
-        "dashes": false,
-        "datasource": null,
-        "fill": 1,
-        "fillGradient": 0,
-        "gridPos": {
-          "h": 8,
-          "w": 12,
-          "x": 12,
-          "y": 31
-        },
-        "hiddenSeries": false,
-        "id": 13,
-        "legend": {
-          "alignAsTable": false,
-          "avg": false,
-          "current": false,
-          "max": false,
-          "min": false,
-          "show": true,
-          "total": false,
-          "values": false
-        },
-        "lines": true,
-        "linewidth": 1,
-        "links": [],
-        "nullPointMode": "null",
-        "options": {
-          "dataLinks": []
-        },
-        "percentage": false,
-        "pointradius": 5,
-        "points": false,
-        "renderer": "flot",
-        "seriesOverrides": [],
-        "spaceLength": 10,
-        "stack": true,
-        "steppedLine": false,
-        "targets": [
-          {
-            "expr": "sum by(group) (kafka_consumergroup_group_lag > 0)",
-            "format": "time_series",
-            "intervalFactor": 1,
-            "legendFormat": "total lag",
-            "refId": "D"
-          }
-        ],
-        "thresholds": [],
-        "timeFrom": null,
-        "timeRegions": [],
-        "timeShift": null,
-        "title": "Total Record Lag (Kafka Lag Exporter)",
-        "tooltip": {
-          "shared": true,
-          "sort": 0,
-          "value_type": "individual"
-        },
-        "type": "graph",
-        "xaxis": {
-          "buckets": null,
-          "mode": "time",
-          "name": null,
-          "show": true,
-          "values": []
-        },
-        "yaxes": [
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": "0",
-            "show": true
-          },
-          {
-            "format": "short",
-            "label": null,
-            "logBase": 1,
-            "max": null,
-            "min": null,
-            "show": true
-          }
-        ],
-        "yaxis": {
-          "align": false,
-          "alignLevel": null
-        }
-      }
-    ],
-    "refresh": "10s",
-    "schemaVersion": 21,
-    "style": "dark",
-    "tags": [],
-    "templating": {
-      "list": [
-        {
-          "allValue": null,
-          "current": {
-            "tags": [],
-            "text": "titan-ccp-aggregation",
-            "value": "titan-ccp-aggregation"
-          },
-          "datasource": "Prometheus",
-          "definition": "label_values(kafka_consumer_consumer_fetch_manager_metrics_records_lag, job)",
-          "hide": 0,
-          "includeAll": false,
-          "label": "Job",
-          "multi": false,
-          "name": "Job",
-          "options": [
-            {
-              "selected": true,
-              "text": "titan-ccp-aggregation",
-              "value": "titan-ccp-aggregation"
-            }
-          ],
-          "query": "label_values(kafka_consumer_consumer_fetch_manager_metrics_records_lag, job)",
-          "refresh": 0,
-          "regex": "",
-          "skipUrlSync": false,
-          "sort": 0,
-          "tagValuesQuery": "",
-          "tags": [],
-          "tagsQuery": "",
-          "type": "query",
-          "useTags": false
-        }
-      ]
-    },
-    "time": {
-      "from": "now-1h",
-      "to": "now"
-    },
-    "timepicker": {
-      "refresh_intervals": [
-        "5s",
-        "10s",
-        "30s",
-        "1m",
-        "5m",
-        "15m",
-        "30m",
-        "1h",
-        "2h",
-        "1d"
-      ]
-    },
-    "timezone": "",
-    "title": "Scalability Benchmarking",
-    "uid": "dad0CNlZz",
-    "version": 25
-    }
\ No newline at end of file

execution/infrastructure/grafana/prometheus-datasource-config-map.yaml

-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: prometheus
-  labels:
-    grafana_datasource: "1"
-data:
-  datasource.yaml: |-
-    # config file version
-    apiVersion: 1
-    datasources:
-      # <string, required> name of the datasource. Required
-    - name: Prometheus
-      # <string, required> datasource type. Required
-      type: prometheus
-      # <string, required> access mode. proxy or direct (Server or Browser in the UI). Required
-      access: proxy
-      # <bool> mark as default datasource. Max one per org
-      isDefault: true
-      # <int> org id. will default to orgId 1 if not specified
-      orgId: 1
-      # <string> url
-      url: http://prometheus-operated:9090 #http://localhost:9090
-      # <map> fields that will be converted to json and stored in json_data
-      jsonData:
-        timeInterval: "15s"
-      version: 1
-      # <bool> allow users to edit datasources from the UI.
-      editable: true

execution/infrastructure/grafana/scalability-benchmarking-dashbaord.json

-{
-  "annotations": {
-    "list": [
-      {
-        "builtIn": 1,
-        "datasource": "-- Grafana --",
-        "enable": true,
-        "hide": true,
-        "iconColor": "rgba(0, 211, 255, 1)",
-        "name": "Annotations & Alerts",
-        "type": "dashboard"
-      }
-    ]
-  },
-  "editable": true,
-  "gnetId": null,
-  "graphTooltip": 0,
-  "id": 2,
-  "iteration": 1589140028684,
-  "links": [],
-  "panels": [
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 7,
-        "w": 12,
-        "x": 0,
-        "y": 0
-      },
-      "hiddenSeries": false,
-      "id": 2,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_input)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{Messages In Per Second}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Messages In Per Second",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 7,
-        "w": 12,
-        "x": 12,
-        "y": 0
-      },
-      "hiddenSeries": false,
-      "id": 3,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_output)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{Messages Out Per Second}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Messages Out Per Second",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 0,
-        "y": 7
-      },
-      "hiddenSeries": false,
-      "id": 9,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": true,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "sum by(group, topic) (kafka_consumergroup_group_lag > 0)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{topic}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Record Lag",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 12,
-        "y": 7
-      },
-      "hiddenSeries": false,
-      "id": 5,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "count(count (kafka_consumer_consumer_fetch_manager_metrics_records_lag) by(pod))",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "instances",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Number of Instances",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "decimals": 0,
-          "format": "short",
-          "label": "",
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 0,
-        "y": 15
-      },
-      "hiddenSeries": false,
-      "id": 10,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "sum by(group,topic) (kafka_consumergroup_group_offset > 0)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{topic}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Records Consumed",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 12,
-        "y": 15
-      },
-      "hiddenSeries": false,
-      "id": 12,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "count by(group,topic) (kafka_consumergroup_group_offset > 0)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{topic}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Number of Partitions",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 0,
-        "y": 23
-      },
-      "hiddenSeries": false,
-      "id": 11,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "sum by(group,topic) (kafka_partition_latest_offset)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{topic}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Records Produced (Kafka Lag Exporter)",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 12,
-        "y": 23
-      },
-      "hiddenSeries": false,
-      "id": 8,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "count by(job, topic) (kafka_consumer_consumer_fetch_manager_metrics_records_lag)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{topic}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Number of Partitions (Kafka Streams Export)",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "decimals": null,
-          "format": "short",
-          "label": "",
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 0,
-        "y": 31
-      },
-      "hiddenSeries": false,
-      "id": 4,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": false,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "sum by(job, topic) (kafka_consumer_consumer_fetch_manager_metrics_records_lag)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "{{topic}}",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Record Lag (Kafka Streams Export)",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    },
-    {
-      "aliasColors": {},
-      "bars": false,
-      "dashLength": 10,
-      "dashes": false,
-      "datasource": null,
-      "fill": 1,
-      "fillGradient": 0,
-      "gridPos": {
-        "h": 8,
-        "w": 12,
-        "x": 12,
-        "y": 31
-      },
-      "hiddenSeries": false,
-      "id": 13,
-      "legend": {
-        "alignAsTable": false,
-        "avg": false,
-        "current": false,
-        "max": false,
-        "min": false,
-        "show": true,
-        "total": false,
-        "values": false
-      },
-      "lines": true,
-      "linewidth": 1,
-      "links": [],
-      "nullPointMode": "null",
-      "options": {
-        "dataLinks": []
-      },
-      "percentage": false,
-      "pointradius": 5,
-      "points": false,
-      "renderer": "flot",
-      "seriesOverrides": [],
-      "spaceLength": 10,
-      "stack": true,
-      "steppedLine": false,
-      "targets": [
-        {
-          "expr": "sum by(group) (kafka_consumergroup_group_lag > 0)",
-          "format": "time_series",
-          "intervalFactor": 1,
-          "legendFormat": "total lag",
-          "refId": "D"
-        }
-      ],
-      "thresholds": [],
-      "timeFrom": null,
-      "timeRegions": [],
-      "timeShift": null,
-      "title": "Total Record Lag (Kafka Lag Exporter)",
-      "tooltip": {
-        "shared": true,
-        "sort": 0,
-        "value_type": "individual"
-      },
-      "type": "graph",
-      "xaxis": {
-        "buckets": null,
-        "mode": "time",
-        "name": null,
-        "show": true,
-        "values": []
-      },
-      "yaxes": [
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": "0",
-          "show": true
-        },
-        {
-          "format": "short",
-          "label": null,
-          "logBase": 1,
-          "max": null,
-          "min": null,
-          "show": true
-        }
-      ],
-      "yaxis": {
-        "align": false,
-        "alignLevel": null
-      }
-    }
-  ],
-  "refresh": "10s",
-  "schemaVersion": 21,
-  "style": "dark",
-  "tags": [],
-  "templating": {
-    "list": [
-      {
-        "allValue": null,
-        "current": {
-          "tags": [],
-          "text": "titan-ccp-aggregation",
-          "value": "titan-ccp-aggregation"
-        },
-        "datasource": "Prometheus",
-        "definition": "label_values(kafka_consumer_consumer_fetch_manager_metrics_records_lag, job)",
-        "hide": 0,
-        "includeAll": false,
-        "label": "Job",
-        "multi": false,
-        "name": "Job",
-        "options": [
-          {
-            "selected": true,
-            "text": "titan-ccp-aggregation",
-            "value": "titan-ccp-aggregation"
-          }
-        ],
-        "query": "label_values(kafka_consumer_consumer_fetch_manager_metrics_records_lag, job)",
-        "refresh": 0,
-        "regex": "",
-        "skipUrlSync": false,
-        "sort": 0,
-        "tagValuesQuery": "",
-        "tags": [],
-        "tagsQuery": "",
-        "type": "query",
-        "useTags": false
-      }
-    ]
-  },
-  "time": {
-    "from": "now-1h",
-    "to": "now"
-  },
-  "timepicker": {
-    "refresh_intervals": [
-      "5s",
-      "10s",
-      "30s",
-      "1m",
-      "5m",
-      "15m",
-      "30m",
-      "1h",
-      "2h",
-      "1d"
-    ]
-  },
-  "timezone": "",
-  "title": "Scalability Benchmarking",
-  "uid": "dad0CNlZz",
-  "version": 25
-}
\ No newline at end of file

execution/infrastructure/grafana/values.yaml

-image:
-  repository: grafana/grafana
-  tag: 6.7.3
-  pullPolicy: IfNotPresent
-
-# Administrator credentials when not using an existing secret (see below)
-adminUser: admin
-adminPassword: admin
-
-
-## Sidecars that collect the configmaps with specified label and stores the included files them into the respective folders
-## Requires at least Grafana 5 to work and can't be used together with parameters dashboardProviders, datasources and dashboards
-sidecar:
-  image:
-    repository: "kiwigrid/k8s-sidecar"
-    tag: "1.1.0"
-  imagePullPolicy: IfNotPresent
-  dashboards:
-    enabled: true
-    SCProvider: true
-    # label that the configmaps with dashboards are marked with
-    label: grafana_dashboard
-    # folder in the pod that should hold the collected dashboards (unless `defaultFolderName` is set)
-    folder: /tmp/dashboards
-    # The default folder name, it will create a subfolder under the `folder` and put dashboards in there instead
-    defaultFolderName: null
-    # If specified, the sidecar will search for dashboard config-maps inside this namespace.
-    # Otherwise the namespace in which the sidecar is running will be used.
-    # It's also possible to specify ALL to search in all namespaces
-    searchNamespace: null
-    # provider configuration that lets grafana manage the dashboards
-    provider:
-      # name of the provider, should be unique
-      name: sidecarProvider
-      # orgid as configured in grafana
-      orgid: 1
-      # folder in which the dashboards should be imported in grafana
-      folder: ''
-      # type of the provider
-      type: file
-      # disableDelete to activate a import-only behaviour
-      disableDelete: false
-      # allow updating provisioned dashboards from the UI
-      allowUiUpdates: true
-  datasources:
-    enabled: true
-    # label that the configmaps with datasources are marked with
-    label: grafana_datasource
-    # If specified, the sidecar will search for datasource config-maps inside this namespace.
-    # Otherwise the namespace in which the sidecar is running will be used.
-    # It's also possible to specify ALL to search in all namespaces
-    searchNamespace: null
-
-service:
-  nodePort: 31199
-  type: NodePort
\ No newline at end of file

execution/infrastructure/kafka-lag-exporter/install.sh

-helm install kafka-lag-exporter https://github.com/lightbend/kafka-lag-exporter/releases/download/v0.6.0/kafka-lag-exporter-0.6.0.tgz \
-  --set clusters\[0\].name=my-confluent-cp-kafka \
-  --set clusters\[0\].bootstrapBrokers=my-confluent-cp-kafka:9092 \
-  --set pollIntervalSeconds=15 #5
-
-# Helm could also create ServiceMonitor

execution/infrastructure/kafka-lag-exporter/values.yaml

-image:
-  pullPolicy: IfNotPresent
-
-clusters:
-  - name: "my-confluent-cp-kafka"
-    bootstrapBrokers: "my-confluent-cp-kafka:9092"
-
-## The interval between refreshing metrics
-pollIntervalSeconds: 15
-
-prometheus:
-  serviceMonitor:
-    enabled: true
-    interval: "5s"
-    additionalLabels:
-      appScope: titan-ccp
-    # service monitor label selectors: https://github.com/helm/charts/blob/f5a751f174263971fafd21eee4e35416d6612a3d/stable/prometheus-operator/templates/prometheus/prometheus.yaml#L74
-    # additionalLabels:
-    #   prometheus: k8s
\ No newline at end of file