Skip to content
Snippets Groups Projects
Commit 4d25f4f1 authored by Sören Henning's avatar Sören Henning
Browse files

Merge remote-tracking branch 'origin/master' into issue-30-add-doc-kube-res

parents 330d498a e18006d3
No related branches found
No related tags found
1 merge request!16Recreate additional Kubernetes resources on each benchmark subexperiment
Pipeline #509 passed with warnings
...@@ -2,4 +2,18 @@ ...@@ -2,4 +2,18 @@
> A theodolite is a precision optical instrument for measuring angles between designated visible points in the horizontal and vertical planes. -- <cite>[Wikipedia](https://en.wikipedia.org/wiki/Theodolite)</cite> > A theodolite is a precision optical instrument for measuring angles between designated visible points in the horizontal and vertical planes. -- <cite>[Wikipedia](https://en.wikipedia.org/wiki/Theodolite)</cite>
Theodolite is a framework for benchmarking the horizontal and vertical scalability of stream processing engines. Theodolite is a framework for benchmarking the horizontal and vertical scalability of stream processing engines. It consists of three modules:
## Theodolite Benchmarks
Theodolite contains 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding workload generator is provided. Currently, this repository provides benchmark implementations for Kafka Streams.
## Theodolite Execution Framework
Theodolite aims to benchmark scalability of stream processing engines for real use cases. Microservices that apply stream processing techniques are usually deployed in elastic cloud environments. Hence, Theodolite's cloud-native benchmarking framework deploys as components in a cloud environment, orchestrated by Kubernetes. More information on how to execute scalability benchmarks can be found in [Thedolite execution framework](execution).
## Theodolite Analysis Tools
Theodolite's benchmarking method create a *scalability graph* allowing to draw conclusions about the scalability of a stream processing engine or its deployment. A scalability graph shows how resource demand evolves with an increasing workload. Theodolite provides Jupyter notebooks for creating such scalability graphs based on benchmarking results from the execution framework. More information can be found in [Theodolite analysis tool](analysis).
...@@ -17,6 +17,6 @@ Python libraries, which can be installed via: ...@@ -17,6 +17,6 @@ Python libraries, which can be installed via:
pip install -r requirements.txt pip install -r requirements.txt
``` ```
We have tested these We have tested these
notebooks with [Visual Studio Code](https://code.visualstudio.com/docs/python/jupyter-support), notebooks with [Visual Studio Code](https://code.visualstudio.com/docs/python/jupyter-support),
however, every other server should be fine as well. however, every other server should be fine as well.
...@@ -95,7 +95,15 @@ kubectl apply -f infrastructure/kafka/service-monitor.yaml ...@@ -95,7 +95,15 @@ kubectl apply -f infrastructure/kafka/service-monitor.yaml
##### Other options for Kafka ##### Other options for Kafka
Other Kafka deployments, for example, using Strimzi, should work in similiar way. Other Kafka deployments, for example, using Strimzi, should work in a similar way.
#### A Kafka Client Pod
A permanently running pod used for Kafka configuration is started via:
```sh
kubectl apply -f infrastructure/kafka/kafka-client.yaml
```
#### The Kafka Lag Exporter #### The Kafka Lag Exporter
......
apiVersion: v1
kind: Pod
metadata:
name: kafka-client-2
spec:
containers:
- name: kafka-client
image: confluentinc/cp-enterprise-kafka:5.4.0
command:
- sh
- -c
- "exec tail -f /dev/null"
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment