diff --git a/CITATION.cff b/CITATION.cff
index 52e6e13286c0ba0aca34005a4d245d73b9869874..4ec1e6c14d466c84478208ecff81e0f24629af0f 100644
--- a/CITATION.cff
+++ b/CITATION.cff
@@ -8,7 +8,7 @@ authors:
given-names: Wilhelm
orcid: "https://orcid.org/0000-0001-6625-4335"
title: Theodolite
-version: "0.4.0"
+version: "0.5.0"
repository-code: "https://github.com/cau-se/theodolite"
license: "Apache-2.0"
doi: "10.1016/j.bdr.2021.100209"
diff --git a/README.md b/README.md
index f2673f4b9ed0c46987963f8b455e19def802db79..804a193df21f3883ecf9a727af5a743b77a9cceb 100644
--- a/README.md
+++ b/README.md
@@ -4,20 +4,17 @@
Theodolite is a framework for benchmarking the horizontal and vertical scalability of stream processing engines. It consists of three modules:
-## Theodolite Benchmarks
-
-Theodolite contains 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding workload generator is provided. Currently, this repository provides benchmark implementations for Apache Kafka Streams and Apache Flink. The benchmark sources can be found in [Thedolite benchmarks](benchmarks).
-
-
-## Theodolite Execution Framework
-
-Theodolite aims to benchmark scalability of stream processing engines for real use cases. Microservices that apply stream processing techniques are usually deployed in elastic cloud environments. Hence, Theodolite's cloud-native benchmarking framework deploys its components in a cloud environment, orchestrated by Kubernetes. More information on how to execute scalability benchmarks can be found in [Thedolite execution framework](execution).
+## Theodolite Benchmarking Tool
+Theodolite aims to benchmark scalability of stream processing engines for real use cases. Microservices that apply stream processing techniques are usually deployed in elastic cloud environments. Hence, Theodolite's cloud-native benchmarking framework deploys its components in a cloud environment, orchestrated by Kubernetes. It is recommended to install Theodolite with the package manager Helm. The Theodolite Helm chart along with instructions how to install it can be found in the [`helm`](helm) directory.
## Theodolite Analysis Tools
-Theodolite's benchmarking method creates a *scalability graph* allowing to draw conclusions about the scalability of a stream processing engine or its deployment. A scalability graph shows how resource demand evolves with an increasing workload. Theodolite provides Jupyter notebooks for creating such scalability graphs based on benchmarking results from the execution framework. More information can be found in [Theodolite analysis tool](analysis).
+Theodolite's benchmarking method maps load intensities to the resource amounts that are required for processing them. A plot showing how resource demand evolves with an increasing load allows to draw conclusions about the scalability of a stream processing engine or its deployment. Theodolite provides Jupyter notebooks for creating such plots based on benchmarking results from the execution framework. More information can be found in [Theodolite analysis tool](analysis).
+
+## Theodolite Benchmarks
+Theodolite comes with 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding load generator is provided. Currently, this repository provides benchmark implementations for Apache Kafka Streams and Apache Flink. The benchmark sources can be found in [Thedolite benchmarks](theodolite-benchmarks).
## How to Cite
diff --git a/analysis/demand-metric-plot.ipynb b/analysis/demand-metric-plot.ipynb
index 90ef227dbf6a4566760329b615d5f59b4cc2bc25..71e08f0590f819a63b1bdd6bf13b57ac665f65bc 100644
--- a/analysis/demand-metric-plot.ipynb
+++ b/analysis/demand-metric-plot.ipynb
@@ -1,22 +1,22 @@
{
"cells": [
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"# Theodolite Analysis - Plotting the Demand Metric\n",
"\n",
"This notebook creates a plot, showing scalability as a function that maps load intensities to the resources required for processing them. It is able to combine multiple such plots in one figure, for example, to compare multiple systems or configurations.\n",
"\n",
"The notebook takes a CSV file for each plot mapping load intensities to minimum required resources, computed by the `demand-metric-plot.ipynb` notebook."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"First, we need to import some libraries, which are required for creating the plots."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -33,11 +33,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"We need to specify the directory, where the demand CSV files can be found, and a dictionary that maps a system description (e.g. its name) to the corresponding CSV file (prefix). To use Unicode narrow non-breaking spaces in the description format it as `u\"1000\\u202FmCPU\"`."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -53,11 +53,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"Now, we combie all systems described in `experiments`."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -71,11 +71,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"We might want to display the mappings before we plot it."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -87,11 +87,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"The following code creates a MatPlotLib figure showing the scalability plots for all specified systems. You might want to adjust its styling etc. according to your preferences. Make sure to also set a filename."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -149,27 +149,33 @@
}
],
"metadata": {
+ "file_extension": ".py",
+ "interpreter": {
+ "hash": "e9e076445e1891a25f59b525adcc71b09846b3f9cf034ce4147fc161b19af121"
+ },
+ "kernelspec": {
+ "display_name": "Python 3.8.10 64-bit ('.venv': venv)",
+ "name": "python3"
+ },
"language_info": {
- "name": "python",
"codemirror_mode": {
"name": "ipython",
"version": 3
},
- "version": "3.8.5-final"
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.10"
},
- "orig_nbformat": 2,
- "file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
+ "orig_nbformat": 2,
"pygments_lexer": "ipython3",
- "version": 3,
- "kernelspec": {
- "name": "python37064bitvenvvenv6c432ee1239d4f3cb23f871068b0267d",
- "display_name": "Python 3.7.0 64-bit ('.venv': venv)",
- "language": "python"
- }
+ "version": 3
},
"nbformat": 4,
"nbformat_minor": 2
-}
\ No newline at end of file
+}
diff --git a/analysis/demand-metric.ipynb b/analysis/demand-metric.ipynb
index bcea129b7cb07465fa99f32b6f8b2b6115e8a0aa..fbf3ee02960a1e06457eef5dda96cb6d0a1a75ac 100644
--- a/analysis/demand-metric.ipynb
+++ b/analysis/demand-metric.ipynb
@@ -1,6 +1,8 @@
{
"cells": [
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"# Theodolite Analysis - Demand Metric\n",
"\n",
@@ -9,11 +11,11 @@
"Theodolite's *demand* metric is a function, mapping load intensities to the minimum required resources (e.g., instances) that are required to process this load. With this notebook, the *demand* metric function is approximated by a map of tested load intensities to their minimum required resources.\n",
"\n",
"The final output when running this notebook will be a CSV file, providig this mapping. It can be used to create nice plots of a system's scalability using the `demand-metric-plot.ipynb` notebook."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"In the following cell, we need to specifiy:\n",
"\n",
@@ -22,9 +24,7 @@
"* `max_lag_trend_slope`: The maximum tolerable increase in queued messages per second.\n",
"* `measurement_dir`: The directory where the measurement data files are to be found.\n",
"* `results_dir`: The directory where the computed demand CSV files are to be stored."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -40,11 +40,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"With the following call, we compute our demand mapping."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -58,11 +58,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"We might already want to plot a simple visualization here:"
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -74,11 +74,11 @@
]
},
{
+ "cell_type": "markdown",
+ "metadata": {},
"source": [
"Finally we store the results in a CSV file."
- ],
- "cell_type": "markdown",
- "metadata": {}
+ ]
},
{
"cell_type": "code",
@@ -93,27 +93,33 @@
}
],
"metadata": {
+ "file_extension": ".py",
+ "interpreter": {
+ "hash": "e9e076445e1891a25f59b525adcc71b09846b3f9cf034ce4147fc161b19af121"
+ },
+ "kernelspec": {
+ "display_name": "Python 3.8.10 64-bit ('.venv': venv)",
+ "name": "python3"
+ },
"language_info": {
- "name": "python",
"codemirror_mode": {
"name": "ipython",
"version": 3
},
- "version": "3.8.5-final"
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.8.10"
},
- "orig_nbformat": 2,
- "file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
+ "orig_nbformat": 2,
"pygments_lexer": "ipython3",
- "version": 3,
- "kernelspec": {
- "name": "python37064bitvenvvenv6c432ee1239d4f3cb23f871068b0267d",
- "display_name": "Python 3.7.0 64-bit ('.venv': venv)",
- "language": "python"
- }
+ "version": 3
},
"nbformat": 4,
"nbformat_minor": 2
-}
\ No newline at end of file
+}
diff --git a/analysis/src/demand.py b/analysis/src/demand.py
index dfb20c05af8e9a134eedd2cdb584c961a82369f5..2178ab7c5dc5f7e4c04ebb58d4c14c9bf8b1aeff 100644
--- a/analysis/src/demand.py
+++ b/analysis/src/demand.py
@@ -1,59 +1,51 @@
import os
from datetime import datetime, timedelta, timezone
import pandas as pd
+from pandas.core.frame import DataFrame
from sklearn.linear_model import LinearRegression
def demand(exp_id, directory, threshold, warmup_sec):
raw_runs = []
- # Compute SL, i.e., lag trend, for each tested configuration
- filenames = [filename for filename in os.listdir(directory) if filename.startswith(f"exp{exp_id}") and filename.endswith("totallag.csv")]
+ # Compute SLI, i.e., lag trend, for each tested configuration
+ filenames = [filename for filename in os.listdir(directory) if filename.startswith(f"exp{exp_id}") and "lag-trend" in filename and filename.endswith(".csv")]
for filename in filenames:
- #print(filename)
run_params = filename[:-4].split("_")
- dim_value = run_params[2]
- instances = run_params[3]
+ dim_value = run_params[1]
+ instances = run_params[2]
df = pd.read_csv(os.path.join(directory, filename))
- #input = df.loc[df['topic'] == "input"]
input = df
- #print(input)
+
input['sec_start'] = input.loc[0:, 'timestamp'] - input.iloc[0]['timestamp']
- #print(input)
- #print(input.iloc[0, 'timestamp'])
+
regress = input.loc[input['sec_start'] >= warmup_sec] # Warm-Up
- #regress = input
- #input.plot(kind='line',x='timestamp',y='value',color='red')
- #plt.show()
+ X = regress.iloc[:, 1].values.reshape(-1, 1) # values converts it into a numpy array
+ Y = regress.iloc[:, 2].values.reshape(-1, 1) # -1 means that calculate the dimension of rows, but have 1 column
- X = regress.iloc[:, 2].values.reshape(-1, 1) # values converts it into a numpy array
- Y = regress.iloc[:, 3].values.reshape(-1, 1) # -1 means that calculate the dimension of rows, but have 1 column
linear_regressor = LinearRegression() # create object for the class
linear_regressor.fit(X, Y) # perform linear regression
Y_pred = linear_regressor.predict(X) # make predictions
trend_slope = linear_regressor.coef_[0][0]
- #print(linear_regressor.coef_)
row = {'load': int(dim_value), 'resources': int(instances), 'trend_slope': trend_slope}
- #print(row)
raw_runs.append(row)
runs = pd.DataFrame(raw_runs)
- # Set suitable = True if SLOs are met, i.e., lag trend is below threshold
- runs["suitable"] = runs.apply(lambda row: row['trend_slope'] < threshold, axis=1)
-
- # Sort results table (unsure if required)
- runs.columns = runs.columns.str.strip()
- runs.sort_values(by=["load", "resources"])
+ # Group by the load and resources to handle repetitions, and take from the reptitions the median
+ # for even reptitions, the mean of the two middle values is used
+ medians = runs.groupby(by=['load', 'resources'], as_index=False).median()
- # Filter only suitable configurations
- filtered = runs[runs.apply(lambda x: x['suitable'], axis=1)]
-
- # Compute demand per load intensity
- grouped = filtered.groupby(['load'])['resources'].min()
- demand_per_load = grouped.to_frame().reset_index()
+ # Set suitable = True if SLOs are met, i.e., lag trend slope is below threshold
+ medians["suitable"] = medians.apply(lambda row: row['trend_slope'] < threshold, axis=1)
+ suitable = medians[medians.apply(lambda x: x['suitable'], axis=1)]
+
+ # Compute minimal demand per load intensity
+ demand_per_load = suitable.groupby(by=['load'], as_index=False)['resources'].min()
+
return demand_per_load
+
diff --git a/codemeta.json b/codemeta.json
index 5696996592f63bf8ece23239d8204e0f25b9cce1..fd2fbe137fc4f8e854452998c62ee988829693fb 100644
--- a/codemeta.json
+++ b/codemeta.json
@@ -8,7 +8,7 @@
"dateModified": "2021-03-18",
"downloadUrl": "https://github.com/cau-se/theodolite/releases",
"name": "Theodolite",
- "version": "0.4.0",
+ "version": "0.5.0",
"description": "Theodolite is a framework for benchmarking the horizontal and vertical scalability of stream processing engines.",
"developmentStatus": "active",
"referencePublication": "https://doi.org/10.1016/j.bdr.2021.100209",
diff --git a/docs/README.md b/docs/README.md
index 4fd13bdfc157efe8b3491695bb83972f96a82c5d..eb0848d52ec4235c6325ba0a373ea2628e52a102 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -10,16 +10,20 @@ permalink: /
Theodolite is a framework for benchmarking the horizontal and vertical scalability of stream processing engines. It consists of three modules:
-## Theodolite Benchmarks
+## Theodolite Benchmarking Tool
-Theodolite contains 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding workload generator is provided. Currently, this repository provides benchmark implementations for Kafka Streams.
+Theodolite aims to benchmark scalability of stream processing engines for real use cases. Microservices that apply stream processing techniques are usually deployed in elastic cloud environments. Hence, Theodolite's cloud-native benchmarking framework deploys its components in a cloud environment, orchestrated by Kubernetes. It is recommended to install Theodolite with the package manager Helm. The Theodolite Helm chart along with instructions how to install it can be found in the [`helm`](helm) directory.
+## Theodolite Analysis Tools
-## Theodolite Execution Framework
+Theodolite's benchmarking method maps load intensities to the resource amounts that are required for processing them. A plot showing how resource demand evolves with an increasing load allows to draw conclusions about the scalability of a stream processing engine or its deployment. Theodolite provides Jupyter notebooks for creating such plots based on benchmarking results from the execution framework. More information can be found in [Theodolite analysis tool](analysis).
-Theodolite aims to benchmark scalability of stream processing engines for real use cases. Microservices that apply stream processing techniques are usually deployed in elastic cloud environments. Hence, Theodolite's cloud-native benchmarking framework deploys as components in a cloud environment, orchestrated by Kubernetes. More information on how to execute scalability benchmarks can be found in [Thedolite execution framework](execution).
+## Theodolite Benchmarks
+Theodolite comes with 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding load generator is provided. Currently, this repository provides benchmark implementations for Apache Kafka Streams and Apache Flink. The benchmark sources can be found in [Thedolite benchmarks](theodolite-benchmarks).
-## Theodolite Analysis Tools
+## How to Cite
+
+If you use Theodolite, please cite
-Theodolite's benchmarking method create a *scalability graph* allowing to draw conclusions about the scalability of a stream processing engine or its deployment. A scalability graph shows how resource demand evolves with an increasing workload. Theodolite provides Jupyter notebooks for creating such scalability graphs based on benchmarking results from the execution framework. More information can be found in [Theodolite analysis tool](analysis).
+> Sören Henning and Wilhelm Hasselbring. (2021). Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures. Big Data Research, Volume 25. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209). arXiv:[2009.00304](https://arxiv.org/abs/2009.00304).
diff --git a/docs/index.yaml b/docs/index.yaml
index 087124d158794e1b48dfc880e26da2c91d78808f..635cf8e5fe879cd89927b1b7d87ae621187b3e94 100644
--- a/docs/index.yaml
+++ b/docs/index.yaml
@@ -1,6 +1,41 @@
apiVersion: v1
entries:
theodolite:
+ - apiVersion: v2
+ appVersion: 0.5.0
+ created: "2021-11-04T17:45:14.153231798+01:00"
+ dependencies:
+ - condition: grafana.enabled
+ name: grafana
+ repository: https://grafana.github.io/helm-charts
+ version: 6.0.0
+ - condition: kube-prometheus-stack.enabled
+ name: kube-prometheus-stack
+ repository: https://prometheus-community.github.io/helm-charts
+ version: 12.0.0
+ - condition: cp-helm-charts.enabled
+ name: cp-helm-charts
+ repository: https://soerenhenning.github.io/cp-helm-charts
+ version: 0.6.0
+ - condition: kafka-lag-exporter.enabled
+ name: kafka-lag-exporter
+ repository: https://lightbend.github.io/kafka-lag-exporter/repo/
+ version: 0.6.6
+ description: Theodolite is a framework for benchmarking the scalability stream
+ processing engines.
+ digest: 8a4f218e44341eb8fb09ddc58c6aaa0a14aded685f3423088c21fe0ffc112281
+ home: https://cau-se.github.io/theodolite
+ maintainers:
+ - email: soeren.henning@email.uni-kiel.de
+ name: Sören Henning
+ url: https://www.se.informatik.uni-kiel.de/en/team/soeren-henning-m-sc
+ name: theodolite
+ sources:
+ - https://github.com/cau-se/theodolite
+ type: application
+ urls:
+ - https://github.com/cau-se/theodolite/releases/download/v0.5.0/theodolite-0.5.0.tgz
+ version: 0.5.0
- apiVersion: v2
appVersion: 0.4.0
created: "2021-03-18T15:50:50.930902088+01:00"
@@ -36,4 +71,4 @@ entries:
urls:
- https://github.com/cau-se/theodolite/releases/download/v0.4.0/theodolite-0.4.0.tgz
version: 0.4.0
-generated: "2021-03-18T15:50:50.897801281+01:00"
+generated: "2021-11-04T17:45:14.120866992+01:00"
diff --git a/docs/release-process.md b/docs/release-process.md
index 981306b0762e43eacb29a434cc1e505593548fce..103d8d1ac65472459bcaad648f921240eaf508c8 100644
--- a/docs/release-process.md
+++ b/docs/release-process.md
@@ -18,8 +18,11 @@ again be merged into master.
3. Update all references to artifacts which are versioned. This includes:
- 1. Update all references to Theodolite Docker images to tag `v0.3.1`. These are the Kubernetes resource definitions in
-`execution`, the references to *latest* in `run_uc.py`, the Docker Compose files in `theodolite-benchmarks/docker-test` and the example `theodolite.yaml` job.
+ 1. Update all references to Theodolite Docker images to tag `v0.3.1`. These are:
+ 1. the default `helm/values.yaml` file,
+ 2. the example `execution/theodolite.yaml` job,
+ 3. the Kubernetes benchmark resources in `theodolite-benchmarks/definitions/**/resources` and
+ 2. the Docker Compose files in `theodolite-benchmarks/docker-test`.
2. Update both, the `version` and the `appVersion` fields, in the Helm `Charts.yaml` file to `0.3.1`.
diff --git a/execution/.gitignore b/execution/.gitignore
deleted file mode 100644
index bac9a5d1eeb12d9e40d38376904e8fb69c0e5231..0000000000000000000000000000000000000000
--- a/execution/.gitignore
+++ /dev/null
@@ -1,2 +0,0 @@
-exp_counter.txt
-results
diff --git a/helm/Chart.yaml b/helm/Chart.yaml
index b09b4022d6727029311815b3b2b1bfcf4b4d2bd1..0e56a156832ed6f9159f436ec63f825d132e8dd3 100644
--- a/helm/Chart.yaml
+++ b/helm/Chart.yaml
@@ -13,11 +13,11 @@ type: application
dependencies:
- name: grafana
- version: 6.0.0
+ version: 6.17.5
repository: https://grafana.github.io/helm-charts
condition: grafana.enabled
- name: kube-prometheus-stack
- version: 12.0.0
+ version: 20.0.1
repository: https://prometheus-community.github.io/helm-charts
condition: kube-prometheus-stack.enabled
- name: cp-helm-charts
@@ -25,10 +25,10 @@ dependencies:
repository: https://soerenhenning.github.io/cp-helm-charts
condition: cp-helm-charts.enabled
- name: kafka-lag-exporter
- version: 0.6.6
+ version: 0.6.7
repository: https://lightbend.github.io/kafka-lag-exporter/repo/
condition: kafka-lag-exporter.enabled
-version: 0.5.0-SNAPSHOT
+version: 0.6.0-SNAPSHOT
-appVersion: 0.5.0-SNAPSHOT
+appVersion: 0.6.0-SNAPSHOT
diff --git a/helm/README.md b/helm/README.md
index c545804aaec8eb8ed91054f1f7ee97dd293816a4..fe5c15e6fd52122d52c016bff62b8a33e2a22018 100644
--- a/helm/README.md
+++ b/helm/README.md
@@ -2,55 +2,49 @@
## Installation
-Install the chart via:
+The Theodolite Helm chart with all its dependencies can be installed via:
```sh
helm dependencies update .
helm install theodolite .
```
-This chart installs requirements to execute benchmarks with Theodolite.
+Additionally, a PersistentVolumeClaim with the name `theodolite-pv-claim` needs to be created or the results volume can be disabled as done with the minimal configuration, described below.
-Dependencies and subcharts:
+## Customize Installation
-- Prometheus Operator
-- Prometheus
-- Grafana (incl. dashboard and data source configuration)
-- Kafka
-- Zookeeper
-- A Kafka client pod
+As usual, the installation with Helm can be configured by passing a values YAML file:
-## Test
-
-Test the installation:
-
-```sh
-helm test theodolite
+```
+helm install theodolite . -f <your-config.yaml>
```
-Our test files are located [here](templates/../../theodolite-chart/templates/tests). Many subcharts have their own tests, these are also executed and are placed in the respective /templates folders.
-
-Please note: If a test fails, Helm will stop testing.
+We provide a minimal configuration, especially suited for development environments, with the `preconfigs/minimal.yaml`
+file.
-It is possible that the tests are not running successfully at the moment. This is because the Helm tests of the subchart cp-confluent receive a timeout exception. There is an [issue](https://github.com/confluentinc/cp-helm-charts/issues/318) for this problem on GitHub.
+Per default, Helm installs the Theodolite CRDs used for the operator. If Theodolite will not be used as operator or if
+the CRDs are already installed, you can skip their installation by adding the flag `--skip-crds`.
-## Configuration
+## Test Installation
-In development environments Kubernetes resources are often low. To reduce resource consumption, we provide an `one-broker-value.yaml` file. This file can be used with:
+Test the installation with:
```sh
-helm install theodolite . -f preconfigs/one-broker-values.yaml
+helm test theodolite
```
+Our test files are located [here](templates/tests). Many subcharts have their own tests, which are also executed.
+Please note: If a test fails, Helm will stop testing.
+
## Uninstall this Chart
-To uninstall/delete the `theodolite` deployment:
+The Theodolite Helm can easily be removed with:
```sh
-helm delete theodolite
+helm uninstall theodolite
```
-This command does not remove the CRDs which are created by this chart. Remove them manually with:
+Helm does not remove any CRDs created by this chart. You can remove them manually with:
```sh
# CRDs from Theodolite
@@ -69,9 +63,20 @@ kubectl delete crd thanosrulers.monitoring.coreos.com
## Development
-**Hints**:
+### Dependencies
+
+The following 3rd party charts are used by Theodolite:
+
+- Kube Prometheus Stack (to install the Prometheus Operator, which is used to create a Prometheus instances)
+- Grafana (including a dashboard and a data source configuration)
+- Confluent Platform (for Kafka and Zookeeper)
+- Kafka Lag Exporter (used to collect monitoring data of the Kafka lag)
+
+### Hints
+
+#### Grafana
-- Grafana configuration: Grafana ConfigMaps contains expressions like {{ topic }}. Helm uses the same syntax for template function. More information [here](https://github.com/helm/helm/issues/2798)
+Grafana ConfigMaps contain expressions like `{{ topic }}`. Helm uses the same syntax for template function. More information [here](https://github.com/helm/helm/issues/2798)
- Escape braces: {{ "{{" topic }}
- Let Helm render the template as raw string: {{ `{{ <config>}}` }}
\ No newline at end of file
diff --git a/helm/crds/benchmark.yaml b/helm/crds/benchmark.yaml
new file mode 120000
index 0000000000000000000000000000000000000000..fb100de7a1407462bfb6488a54b7f70014a58474
--- /dev/null
+++ b/helm/crds/benchmark.yaml
@@ -0,0 +1 @@
+./../../theodolite/crd/crd-benchmark.yaml
\ No newline at end of file
diff --git a/helm/crds/execution.yaml b/helm/crds/execution.yaml
new file mode 120000
index 0000000000000000000000000000000000000000..62d268c23c391cd7bbfbaffeaee8af1697dc446a
--- /dev/null
+++ b/helm/crds/execution.yaml
@@ -0,0 +1 @@
+./../../theodolite/crd/crd-execution.yaml
\ No newline at end of file
diff --git a/helm/preconfigs/minimal.yaml b/helm/preconfigs/minimal.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..b0828c2f424e8456933dc626a66a199cd60aa5da
--- /dev/null
+++ b/helm/preconfigs/minimal.yaml
@@ -0,0 +1,12 @@
+cp-helm-charts:
+ cp-zookeeper:
+ servers: 1
+
+ cp-kafka:
+ brokers: 1
+ configurationOverrides:
+ offsets.topic.replication.factor: "1"
+
+operator:
+ resultsVolume:
+ enabled: false
diff --git a/helm/preconfigs/one-broker-values.yaml b/helm/preconfigs/one-broker-values.yaml
deleted file mode 100644
index c53c1f1eb8bc7a17f192d70a6f10f8cacc09c98f..0000000000000000000000000000000000000000
--- a/helm/preconfigs/one-broker-values.yaml
+++ /dev/null
@@ -1,15 +0,0 @@
-cp-helm-charts:
- ## ------------------------------------------------------
- ## Zookeeper
- ## ------------------------------------------------------
- cp-zookeeper:
- servers: 1 # default: 3
-
- ## ------------------------------------------------------
- ## Kafka
- ## ------------------------------------------------------
- cp-kafka:
- brokers: 1 # default: 10
-
- configurationOverrides:
- offsets.topic.replication.factor: "1"
\ No newline at end of file
diff --git a/helm/templates/grafana/dashboard-config-map.yaml b/helm/templates/grafana/dashboard-config-map.yaml
index 41365e5efefaddc92a9f2f25f867a9d895e4ca3d..0df01b20efa0fb1100fe4b7289b00b3058eb032f 100644
--- a/helm/templates/grafana/dashboard-config-map.yaml
+++ b/helm/templates/grafana/dashboard-config-map.yaml
@@ -24,7 +24,7 @@ data:
"editable": true,
"gnetId": null,
"graphTooltip": 0,
- "id": 2,
+ "id": 1,
"iteration": 1589140028684,
"links": [],
"panels": [
@@ -1004,4 +1004,4 @@ data:
"uid": "dad0CNlZz",
"version": 25
}`}}
-{{- end }}
\ No newline at end of file
+{{- end }}
diff --git a/helm/templates/grafana/osp-dashboard-config-map.yaml b/helm/templates/grafana/osp-dashboard-config-map.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..22685adf8e93d95614bdbb71ef6bc993a009a150
--- /dev/null
+++ b/helm/templates/grafana/osp-dashboard-config-map.yaml
@@ -0,0 +1,1029 @@
+{{- if .Values.grafana.enabled -}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ template "theodolite.fullname" . }}-grafana-scalability-osp
+ labels:
+ grafana_dashboard: "2"
+data:
+ osp-dashboard.json: |-
+ {{`{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": 2,
+ "iteration": 1631777972723,
+ "links": [],
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 7,
+ "w": 12,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 2,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_ndwflow)",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "ndwflow",
+ "refId": "A"
+ },
+ {
+ "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_ndwspeed)",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "ndwspeed",
+ "refId": "B"
+ },
+ {
+ "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_ndwflow+cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_ndwspeed)",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "ndwflow+ndwspeed",
+ "refId": "C"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Messages In Per Second",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 7,
+ "w": 12,
+ "x": 12,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 3,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "sum(cp_kafka_server_brokertopicmetrics_messagesinpersec_topic_metrics)",
+ "format": "time_series",
+ "interval": "",
+ "intervalFactor": 1,
+ "legendFormat": "Metrics",
+ "refId": "A"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Messages Out Per Second",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 0,
+ "y": 7
+ },
+ "hiddenSeries": false,
+ "id": 9,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": true,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "sum by(group, topic) (kafka_consumergroup_group_lag >= 0)",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "{{topic}}",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Record Lag",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 12,
+ "y": 7
+ },
+ "hiddenSeries": false,
+ "id": 5,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "count(count (kafka_consumer_consumer_fetch_manager_metrics_records_lag) by(pod))",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "instances",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Number of Instances",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "decimals": 0,
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 0,
+ "y": 15
+ },
+ "hiddenSeries": false,
+ "id": 10,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "sum by(group,topic) (kafka_consumergroup_group_offset >= 0)",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "{{topic}}",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Records Consumed",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 12,
+ "y": 15
+ },
+ "hiddenSeries": false,
+ "id": 12,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "count by(group,topic) (kafka_consumergroup_group_offset >= 0)",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "{{topic}}",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Number of Partitions",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 0,
+ "y": 23
+ },
+ "hiddenSeries": false,
+ "id": 11,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "sum by(group,topic) (kafka_partition_latest_offset)",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "{{topic}}",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Records Produced (Kafka Lag Exporter)",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 12,
+ "y": 23
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "count by(job, topic) (kafka_consumer_consumer_fetch_manager_metrics_records_lag)",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "{{topic}}",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Number of Partitions (Kafka Streams Export)",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "decimals": null,
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 0,
+ "y": 31
+ },
+ "hiddenSeries": false,
+ "id": 4,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "sum by(job, topic) (kafka_consumer_consumer_fetch_manager_metrics_records_lag)",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "{{topic}}",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Record Lag (Kafka Streams Export)",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": null,
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 8,
+ "w": 12,
+ "x": 12,
+ "y": 31
+ },
+ "hiddenSeries": false,
+ "id": 13,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": true,
+ "steppedLine": false,
+ "targets": [
+ {
+ "expr": "sum by(group) (kafka_consumergroup_group_lag >= 0)",
+ "format": "time_series",
+ "intervalFactor": 1,
+ "legendFormat": "total lag",
+ "refId": "D"
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Total Record Lag (Kafka Lag Exporter)",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ }
+ ],
+ "refresh": "10s",
+ "schemaVersion": 22,
+ "style": "dark",
+ "tags": [],
+ "templating": {
+ "list": [
+ {
+ "allValue": null,
+ "current": {
+ "tags": [],
+ "text": "titan-ccp-aggregation",
+ "value": "titan-ccp-aggregation"
+ },
+ "datasource": "Prometheus",
+ "definition": "label_values(kafka_consumer_consumer_fetch_manager_metrics_records_lag, job)",
+ "hide": 0,
+ "includeAll": false,
+ "index": -1,
+ "label": "Job",
+ "multi": false,
+ "name": "Job",
+ "options": [
+ {
+ "selected": true,
+ "text": "titan-ccp-aggregation",
+ "value": "titan-ccp-aggregation"
+ }
+ ],
+ "query": "label_values(kafka_consumer_consumer_fetch_manager_metrics_records_lag, job)",
+ "refresh": 0,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ }
+ ]
+ },
+ "time": {
+ "from": "now-1h",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ]
+ },
+ "timezone": "",
+ "title": "OSPBench",
+ "uid": "01O646v7z",
+ "variables": {
+ "list": []
+ },
+ "version": 1
+ }`}}
+{{- end }}
diff --git a/helm/templates/theodolite/crd-benchmark.yaml b/helm/templates/theodolite/crd-benchmark.yaml
deleted file mode 100644
index c2eb8d0d355578d5c247c4e4ce622c8764d71cb4..0000000000000000000000000000000000000000
--- a/helm/templates/theodolite/crd-benchmark.yaml
+++ /dev/null
@@ -1,186 +0,0 @@
-{{- if .Values.operator.benchmarkCRD.create -}}
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
- name: benchmarks.theodolite.com
-spec:
- group: theodolite.com
- names:
- kind: benchmark
- plural: benchmarks
- shortNames:
- - bench
- versions:
- - name: v1
- served: true
- storage: true
- schema:
- openAPIV3Schema:
- type: object
- required: ["spec"]
- properties:
- spec:
- type: object
- required: ["appResourceSets", "loadGenResourceSets", "resourceTypes", "loadTypes", "kafkaConfig"]
- properties:
- name:
- description: This field exists only for technical reasons and should not be set by the user. The value of the field will be overwritten.
- type: string
- resourceTypes:
- description: A list of resource types that can be scaled for this `benchmark` resource. For each resource type the concrete values are defined in the `execution` object.
- type: array
- minItems: 1
- items:
- type: object
- required: ["typeName", "patchers"]
- properties:
- typeName:
- description: Name of the resource type.
- type: string
- patchers:
- description: List of patchers used to scale this resource type.
- type: array
- minItems: 1
- items:
- type: object
- required: ["type", "resource"]
- properties:
- type:
- description: Type of the patcher.
- type: string
- default: ""
- resource:
- description: Specifies the Kubernetes resource to be patched.
- type: string
- default: ""
- properties:
- description: (Optional) Patcher specific additional arguments.
- type: object
- additionalProperties: true
- x-kubernetes-map-type: "granular"
- default: {}
- loadTypes:
- description: A list of load types that can be scaled for this benchmark. For each load type the concrete values are defined in the execution object.
- type: array
- minItems: 1
- items:
- type: object
- required: ["typeName","patchers"]
- properties:
- typeName:
- description: Name of the load type.
- type: string
- patchers:
- description: List of patchers used to scale this resource type.
- type: array
- minItems: 1
- items:
- type: object
- required: ["type", "resource"]
- properties:
- type:
- description: Type of the Patcher.
- type: string
- default: ""
- resource:
- description: Specifies the Kubernetes resource to be patched.
- type: string
- default: ""
- properties:
- description: (Optional) Patcher specific additional arguments.
- type: object
- additionalProperties: true
- x-kubernetes-map-type: "granular"
- default: {}
- kafkaConfig:
- description: Contains the Kafka configuration.
- type: object
- required: ["bootstrapServer", "topics"]
- properties:
- bootstrapServer:
- description: The bootstrap servers connection string.
- type: string
- topics:
- description: List of topics to be created for each experiment. Alternative theodolite offers the possibility to remove certain topics after each experiment.
- type: array
- minItems: 1
- items:
- type: object
- required: ["name"]
- properties:
- name:
- description: The name of the topic.
- type: string
- default: ""
- numPartitions:
- description: The number of partitions of the topic.
- type: integer
- default: 0
- replicationFactor:
- description: The replication factor of the topic.
- type: integer
- default: 0
- removeOnly:
- description: Determines if this topic should only be deleted after each experiement. For removeOnly topics the name can be a RegEx describing the topic.
- type: boolean
- default: false
- appResourceSets:
- type: array
- items:
- type: object
- oneOf:
- - required: [configMap]
- - required: [fileSystem]
- properties:
- configMap:
- type: object
- properties:
- name:
- type: string
- files:
- type: array
- items:
- type: string
- fileSystem:
- type: object
- properties:
- path:
- type: string
- files:
- type: array
- items:
- type: string
- loadGenResourceSets:
- type: array
- items:
- type: object
- oneOf:
- - required: [configMap]
- - required: [fileSystem]
- properties:
- configMap:
- type: object
- properties:
- name:
- type: string
- files:
- type: array
- items:
- type: string
- fileSystem:
- type: object
- properties:
- path:
- type: string
- files:
- type: array
- items:
- type: string
- additionalPrinterColumns:
- - name: Age
- type: date
- jsonPath: .metadata.creationTimestamp
- subresources:
- status: {}
- scope: Namespaced
-{{- end }}
diff --git a/helm/templates/theodolite/crd-execution.yaml b/helm/templates/theodolite/crd-execution.yaml
deleted file mode 100644
index 163835e9b37aca774983d4f019cc61d4bde06510..0000000000000000000000000000000000000000
--- a/helm/templates/theodolite/crd-execution.yaml
+++ /dev/null
@@ -1,133 +0,0 @@
-{{- if .Values.operator.executionCRD.create -}}
-apiVersion: apiextensions.k8s.io/v1
-kind: CustomResourceDefinition
-metadata:
- name: executions.theodolite.com
-spec:
- group: theodolite.com
- names:
- kind: execution
- plural: executions
- shortNames:
- - exec
- versions:
- - name: v1
- served: true
- storage: true
- schema:
- openAPIV3Schema:
- type: object
- required: ["spec"]
- properties:
- spec:
- type: object
- required: ["benchmark", "load", "resources", "slos", "execution", "configOverrides"]
- properties:
- name:
- type: string
- default: ""
- benchmark:
- type: string
- load: # definition of the load dimension
- type: object
- required: ["loadType", "loadValues"]
- properties:
- loadType:
- type: string
- loadValues:
- type: array
- items:
- type: integer
- resources: # definition of the resource dimension
- type: object
- required: ["resourceType", "resourceValues"]
- properties:
- resourceType:
- type: string
- resourceValues:
- type: array
- items:
- type: integer
- slos: # def of service level objectives
- type: array
- items:
- type: object
- required: ["sloType", "prometheusUrl", "offset"]
- properties:
- sloType:
- description: The type of the SLO. It must match 'lag trend'.
- type: string
- prometheusUrl:
- description: Connection string for Promehteus.
- type: string
- offset:
- description: Hours by which the start and end timestamp will be shifted (for different timezones).
- type: integer
- properties:
- description: (Optional) SLO specific additional arguments.
- type: object
- additionalProperties: true
- x-kubernetes-map-type: "granular"
- default: {}
- execution: # def execution config
- type: object
- required: ["strategy", "duration", "repetitions", "restrictions"]
- properties:
- strategy:
- type: string
- duration:
- type: integer
- repetitions:
- type: integer
- loadGenerationDelay:
- type: integer
- restrictions:
- type: array
- items:
- type: string
- configOverrides:
- type: array
- items:
- type: object
- properties:
- patcher:
- type: object
- properties:
- type:
- type: string
- default: ""
- resource:
- type: string
- default: ""
- properties:
- type: object
- additionalProperties: true
- x-kubernetes-map-type: "granular"
- default: {}
- value:
- type: string
- status:
- type: object
- properties:
- executionState:
- description: ""
- type: string
- executionDuration:
- description: "Duration of the execution in seconds"
- type: string
- additionalPrinterColumns:
- - name: STATUS
- type: string
- description: State of the execution
- jsonPath: .status.executionState
- - name: Duration
- type: string
- description: Duration of the execution
- jsonPath: .status.executionDuration
- - name: Age
- type: date
- jsonPath: .metadata.creationTimestamp
- subresources:
- status: {}
- scope: Namespaced
-{{- end }}
diff --git a/helm/templates/theodolite/random-scheduler/deployment.yaml b/helm/templates/theodolite/random-scheduler/deployment.yaml
index a1ea535d52d3dce971806dd638a90e9acb81c5d0..55b6e4ad5f8fafccc9623e69ef1df1fccf81ed39 100644
--- a/helm/templates/theodolite/random-scheduler/deployment.yaml
+++ b/helm/templates/theodolite/random-scheduler/deployment.yaml
@@ -22,8 +22,8 @@ spec:
serviceAccount: {{ include "theodolite.fullname" . }}-random-scheduler
containers:
- name: random-scheduler
- image: ghcr.io/cau-se/theodolite-random-scheduler:latest
- #imagePullPolicy: Always
+ image: "{{ .Values.randomScheduler.image }}:{{ .Values.randomScheduler.imageTag }}"
+ imagePullPolicy: "{{ .Values.randomScheduler.imagePullPolicy }}"
env:
- name: TARGET_NAMESPACE
value: {{ .Release.Namespace }}
diff --git a/helm/templates/theodolite/role-binding.yaml b/helm/templates/theodolite/role-binding.yaml
index 93d8c34e7bc544c3b0c231e986bc58c792cce38e..3b327bb246f9716be0939416db55fc1b2cc5dd70 100644
--- a/helm/templates/theodolite/role-binding.yaml
+++ b/helm/templates/theodolite/role-binding.yaml
@@ -1,5 +1,5 @@
{{- if .Values.rbac.create -}}
-apiVersion: rbac.authorization.k8s.io/v1beta1
+apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ include "theodolite.fullname" . }}
diff --git a/helm/update-index.sh b/helm/update-index.sh
index 286724dd87718387df58ed993af417bf0fd4d8ec..66c55bb8b79e18e3d06d156cb1859f2a53078999 100755
--- a/helm/update-index.sh
+++ b/helm/update-index.sh
@@ -3,7 +3,7 @@
RELEASE_NAME=$1 # Supposed to be equal to tag, e.g., v0.3.0
RELEASE_PATH="https://github.com/cau-se/theodolite/releases/download"
-REPO_INDEX="../../docs/index.yaml"
+REPO_INDEX="../docs/index.yaml"
helm repo index . --url $RELEASE_PATH/$RELEASE_NAME --merge $REPO_INDEX && \
mv index.yaml $REPO_INDEX
\ No newline at end of file
diff --git a/helm/values.yaml b/helm/values.yaml
index 917ec5cdbda7541b56b366aff2412ccf7d856f3b..c51c82d0063cbe0ccdc4e058658c88a6711eaf1b 100644
--- a/helm/values.yaml
+++ b/helm/values.yaml
@@ -250,11 +250,6 @@ operator:
imageTag: latest
imagePullPolicy: Always
- executionCRD:
- create: true
- benchmarkCRD:
- create: true
-
sloChecker:
lagTrend:
enabled: true
@@ -281,6 +276,9 @@ rbac:
randomScheduler:
enabled: true
+ image: ghcr.io/cau-se/theodolite-random-scheduler
+ imageTag: latest
+ imagePullPolicy: Always
rbac:
create: true
serviceAccount:
diff --git a/theodolite-benchmarks/definitions/uc1-flink/resources/jobmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc1-flink/resources/jobmanager-deployment.yaml
index 512d4fe3c786e1b2c44e6ec57fccadf41a2e2eeb..1f328b1cd553c8036e570d28b97795fb2b00ec81 100644
--- a/theodolite-benchmarks/definitions/uc1-flink/resources/jobmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc1-flink/resources/jobmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: jobmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: jobmanager
image: ghcr.io/cau-se/theodolite-uc1-flink:latest
@@ -90,4 +91,4 @@ spec:
emptyDir: {}
# - name: job-artifacts-volume
# hostPath:
-# path: /host/path/to/job/artifacts
\ No newline at end of file
+# path: /host/path/to/job/artifacts
diff --git a/theodolite-benchmarks/definitions/uc1-flink/resources/taskmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc1-flink/resources/taskmanager-deployment.yaml
index 7d46554692696b194736df6023eed5686040497d..c2266a4aeb21302262279f147e6512d5264e1dc1 100644
--- a/theodolite-benchmarks/definitions/uc1-flink/resources/taskmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc1-flink/resources/taskmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: taskmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: taskmanager
image: ghcr.io/cau-se/theodolite-uc1-flink:latest
diff --git a/theodolite-benchmarks/definitions/uc2-flink/resources/jobmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc2-flink/resources/jobmanager-deployment.yaml
index cece4286d49a3f6ff139ca7f1e01c647acd5d9f3..87ea174f71c592bbffab4e5fc9ce6e3963596b9c 100644
--- a/theodolite-benchmarks/definitions/uc2-flink/resources/jobmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc2-flink/resources/jobmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: jobmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: jobmanager
image: ghcr.io/cau-se/theodolite-uc2-flink:latest
@@ -90,4 +91,4 @@ spec:
emptyDir: {}
# - name: job-artifacts-volume
# hostPath:
-# path: /host/path/to/job/artifacts
\ No newline at end of file
+# path: /host/path/to/job/artifacts
diff --git a/theodolite-benchmarks/definitions/uc2-flink/resources/taskmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc2-flink/resources/taskmanager-deployment.yaml
index c0a10f65aae92e4ac1fd8fb92bae97794c142232..c37df972a334a4a0e27f0420030f99f1dff15b53 100644
--- a/theodolite-benchmarks/definitions/uc2-flink/resources/taskmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc2-flink/resources/taskmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: taskmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: taskmanager
image: ghcr.io/cau-se/theodolite-uc2-flink:latest
diff --git a/theodolite-benchmarks/definitions/uc3-flink/resources/jobmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc3-flink/resources/jobmanager-deployment.yaml
index 33bf1f1121a9764785db7a504799314a7ed40cf3..d01123b13fe2d63637ee4000051091a99bad0546 100644
--- a/theodolite-benchmarks/definitions/uc3-flink/resources/jobmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc3-flink/resources/jobmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: jobmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: jobmanager
image: ghcr.io/cau-se/theodolite-uc3-flink:latest
@@ -90,4 +91,4 @@ spec:
emptyDir: {}
# - name: job-artifacts-volume
# hostPath:
-# path: /host/path/to/job/artifacts
\ No newline at end of file
+# path: /host/path/to/job/artifacts
diff --git a/theodolite-benchmarks/definitions/uc3-flink/resources/taskmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc3-flink/resources/taskmanager-deployment.yaml
index 8f70b7308429f79cfd8f8bda7a7a96e2bc8d8689..495f97817e43d692c30fe898c4ef3118cae682d7 100644
--- a/theodolite-benchmarks/definitions/uc3-flink/resources/taskmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc3-flink/resources/taskmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: taskmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: taskmanager
image: ghcr.io/cau-se/theodolite-uc3-flink:latest
diff --git a/theodolite-benchmarks/definitions/uc4-flink/resources/jobmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc4-flink/resources/jobmanager-deployment.yaml
index b6533a2c4355e227a16aeface2080253bce19958..032499ea498f8155fd80e42ec4cbdd850498b217 100644
--- a/theodolite-benchmarks/definitions/uc4-flink/resources/jobmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc4-flink/resources/jobmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: jobmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: jobmanager
image: ghcr.io/cau-se/theodolite-uc4-flink:latest
@@ -90,4 +91,4 @@ spec:
emptyDir: {}
# - name: job-artifacts-volume
# hostPath:
-# path: /host/path/to/job/artifacts
\ No newline at end of file
+# path: /host/path/to/job/artifacts
diff --git a/theodolite-benchmarks/definitions/uc4-flink/resources/taskmanager-deployment.yaml b/theodolite-benchmarks/definitions/uc4-flink/resources/taskmanager-deployment.yaml
index 7363b013b21ad29b481e449113ccf31538505634..7af13f20b6b2edf3c8878adf4f381dc1c1add115 100644
--- a/theodolite-benchmarks/definitions/uc4-flink/resources/taskmanager-deployment.yaml
+++ b/theodolite-benchmarks/definitions/uc4-flink/resources/taskmanager-deployment.yaml
@@ -14,6 +14,7 @@ spec:
app: flink
component: taskmanager
spec:
+ terminationGracePeriodSeconds: 0
containers:
- name: taskmanager
image: ghcr.io/cau-se/theodolite-uc4-flink:latest
diff --git a/theodolite-benchmarks/kstreams-commons/build.gradle b/theodolite-benchmarks/kstreams-commons/build.gradle
index e177aa5c4770c1a77fc21084a7766741fdb9bdec..c5a880acd4377056cc0b0f06b33a2d74c9f87c4e 100644
--- a/theodolite-benchmarks/kstreams-commons/build.gradle
+++ b/theodolite-benchmarks/kstreams-commons/build.gradle
@@ -7,6 +7,9 @@ repositories {
maven {
url "https://oss.sonatype.org/content/repositories/snapshots/"
}
+ maven {
+ url 'https://packages.confluent.io/maven/'
+ }
}
dependencies {
diff --git a/theodolite/src/main/kotlin/theodolite/evaluation/SloCheckerFactory.kt b/theodolite/src/main/kotlin/theodolite/evaluation/SloCheckerFactory.kt
index 76b158a580102e209b13e247864dd7481b557638..93e8e6180f5a99486e500af022869d896067d128 100644
--- a/theodolite/src/main/kotlin/theodolite/evaluation/SloCheckerFactory.kt
+++ b/theodolite/src/main/kotlin/theodolite/evaluation/SloCheckerFactory.kt
@@ -22,13 +22,13 @@ class SloCheckerFactory {
* - `warmup`: time from the beginning to skip in the analysis.
*
*
- * ### `lag trend percent`
+ * ### `lag trend ratio`
* Creates an [ExternalSloChecker] with defined parameters.
- * The required threshold is computed using a percentage and the load of the experiment.
+ * The required threshold is computed using a ratio and the load of the experiment.
*
* The properties map needs the following fields:
* - `externalSlopeURL`: Url to the concrete SLO checker service.
- * - `percent`: of the executed load that is accepted for the slope.
+ * - `ratio`: of the executed load that is accepted for the slope.
* - `warmup`: time from the beginning to skip in the analysis.
*
* @param sloType Type of the [SloChecker].
@@ -50,18 +50,15 @@ class SloCheckerFactory {
threshold = properties["threshold"]?.toInt() ?: throw IllegalArgumentException("threshold expected"),
warmup = properties["warmup"]?.toInt() ?: throw IllegalArgumentException("warmup expected")
)
- "lag trend percent" -> {
- if (!properties["loadType"].equals("NumSensors")) {
- throw IllegalArgumentException("Percent Threshold is only allowed with load type NumSensors")
- }
- var thresholdPercent =
- properties["percent"]?.toDouble()
- ?: throw IllegalArgumentException("percent for threshold expected")
- if (thresholdPercent < 0.0 || thresholdPercent > 1.0) {
- throw IllegalArgumentException("Threshold percent need to be an Double in the range between 0.0 and 1.0 (inclusive)")
+ "lag trend ratio" -> {
+ var thresholdRatio =
+ properties["ratio"]?.toDouble()
+ ?: throw IllegalArgumentException("ratio for threshold expected")
+ if (thresholdRatio < 0.0) {
+ throw IllegalArgumentException("Threshold ratio needs to be an Double greater or equal 0.0")
}
// cast to int, as rounding is not really necessary
- var threshold = (load.get() * thresholdPercent).toInt()
+ var threshold = (load.get() * thresholdRatio).toInt()
ExternalSloChecker(
externalSlopeURL = properties["externalSloUrl"]
diff --git a/theodolite/src/main/kotlin/theodolite/execution/TheodoliteExecutor.kt b/theodolite/src/main/kotlin/theodolite/execution/TheodoliteExecutor.kt
index bc9371763c30e4cef913a368b64e9989e7f2286b..addf30acde31ee8e3e53c20a5e2b57a03587d08e 100644
--- a/theodolite/src/main/kotlin/theodolite/execution/TheodoliteExecutor.kt
+++ b/theodolite/src/main/kotlin/theodolite/execution/TheodoliteExecutor.kt
@@ -55,9 +55,6 @@ class TheodoliteExecutor(
this.kubernetesBenchmark.loadTypes
)
- // Add load type to check if the percentage lag trend is applicable
- config.slos.forEach { it.properties["loadType"] = config.load.loadType }
-
executor =
BenchmarkExecutorImpl(
benchmark = kubernetesBenchmark,
@@ -118,10 +115,10 @@ class TheodoliteExecutor(
val ioHandler = IOHandler()
val resultsFolder = ioHandler.getResultFolderURL()
this.config.executionId = getAndIncrementExecutionID(resultsFolder + "expID.txt")
- ioHandler.writeToJSONFile(this.config, "$resultsFolder${this.config.executionId}-execution-configuration")
+ ioHandler.writeToJSONFile(this.config, "${resultsFolder}exp${this.config.executionId}-execution-configuration")
ioHandler.writeToJSONFile(
kubernetesBenchmark,
- "$resultsFolder${this.config.executionId}-benchmark-configuration"
+ "${resultsFolder}exp${this.config.executionId}-benchmark-configuration"
)
val config = buildConfig()
@@ -133,7 +130,7 @@ class TheodoliteExecutor(
}
ioHandler.writeToJSONFile(
config.compositeStrategy.benchmarkExecutor.results,
- "$resultsFolder${this.config.executionId}-result"
+ "${resultsFolder}exp${this.config.executionId}-result"
)
}
diff --git a/theodolite/src/main/kotlin/theodolite/patcher/PatcherFactory.kt b/theodolite/src/main/kotlin/theodolite/patcher/PatcherFactory.kt
index d89e1f0089ed43f76d69ce98cdb91c348d4aa451..ebad5de74a6b819dbf7887dfad91faac37ed5074 100644
--- a/theodolite/src/main/kotlin/theodolite/patcher/PatcherFactory.kt
+++ b/theodolite/src/main/kotlin/theodolite/patcher/PatcherFactory.kt
@@ -32,7 +32,7 @@ class PatcherFactory {
k8sResources.filter { it.first == patcherDefinition.resource }
.map { resource -> resource.second }
.firstOrNull()
- ?: throw DeploymentFailedException("Could not find resource ${patcherDefinition.resource}")
+ ?: throw InvalidPatcherConfigurationException("Could not find resource ${patcherDefinition.resource}")
return try {
when (patcherDefinition.type) {
@@ -86,10 +86,10 @@ class PatcherFactory {
)
else -> throw InvalidPatcherConfigurationException("Patcher type ${patcherDefinition.type} not found.")
}
- } catch (e: Exception) {
+ } catch (e: NullPointerException) {
throw InvalidPatcherConfigurationException(
"Could not create patcher with type ${patcherDefinition.type}" +
- " Probably a required patcher argument was not specified."
+ " Probably a required patcher argument was not specified.", e
)
}
}
diff --git a/theodolite/src/main/kotlin/theodolite/util/InvalidPatcherConfigurationException.kt b/theodolite/src/main/kotlin/theodolite/util/InvalidPatcherConfigurationException.kt
index e8ecd11d524f5c365149ac0b37c7b985812f8c4b..81ea227d0d9871c2420a414d81749a34b97676b8 100644
--- a/theodolite/src/main/kotlin/theodolite/util/InvalidPatcherConfigurationException.kt
+++ b/theodolite/src/main/kotlin/theodolite/util/InvalidPatcherConfigurationException.kt
@@ -1,4 +1,4 @@
package theodolite.util
-class InvalidPatcherConfigurationException(message: String) : Exception(message)
+class InvalidPatcherConfigurationException(message: String, e: Exception? = null) : Exception(message, e)