Skip to content
Snippets Groups Projects
Commit 5e0a2841 authored by Sören Henning's avatar Sören Henning
Browse files

Add docs for generic SLO checker

parent 2f0d0e19
No related branches found
No related tags found
Loading
Pipeline #6024 passed
...@@ -58,7 +58,29 @@ As a Benchmark may define multiple supported load and resource types, an Executi ...@@ -58,7 +58,29 @@ As a Benchmark may define multiple supported load and resource types, an Executi
## Definition of SLOs ## Definition of SLOs
SLOs provide a way to quantify whether a certain load intensity can be handled by a certain amount of provisioned resources. SLOs provide a way to quantify whether a certain load intensity can be handled by a certain amount of provisioned resources.
An Execution must at least specify one SLO to be checked. In Theodolite, SLO are evaluated by requesting monitoring data from Prometheus and analyzing it in a benchmark-specific way.
An Execution must at least define one SLO to be checked.
A good choice to get started is defining an SLO of type `generic`:
```yaml
- sloType: "generic"
prometheusUrl: "http://prometheus-operated:9090"
offset: 0
properties:
externalSloUrl: "http://localhost:8082"
promQLQuery: "sum by(job) (kafka_streams_stream_task_metrics_dropped_records_total>=0)"
warmup: 60 # in seconds
queryAggregation: max
repetitionAggregation: median
operator: lte
threshold: 1000
```
All you have to do is to define a [PromQL query](https://prometheus.io/docs/prometheus/latest/querying/basics/) describing which metrics should be requested (`promQLQuery`) and how the resulting time series should be evaluated. With `queryAggregation` you specify how the resulting time series is aggregated to a single value and `repetitionAggregation` describes how the results of multiple repetitions are aggregated. Possible values are
`mean`, `median`, `mode`, `sum`, `count`, `max`, `min`, `std`, `var`, `skew`, `kurt` as well as percentiles such as `p99` or `p99.9`. The result of aggregation all repetitions is checked against `threshold`. This check is performed using an `operator`, which describes that the result must be "less than" (`lt`), "less than equal" (`lte`), "greater than" (`gt`) or "greater than equal" (`gte`) to the threshold.
In case you need to evaluate monitoring data in a more flexible fashion, you can also change the value of `externalSloUrl` to your custom SLO checker. Have a look at the source code of the [generic SLO checker](https://github.com/cau-se/theodolite/tree/master/slo-checker/generic) to get started.
## Experimental Setup ## Experimental Setup
...@@ -72,7 +94,7 @@ The experimental setup can be configured by: ...@@ -72,7 +94,7 @@ The experimental setup can be configured by:
## Configuration Overrides ## Configuration Overrides
In cases where only small modifications of a system under test should be benchmarked, it is not necessarily required to [create a new benchmark](creating-a-benchmark). In cases where only small modifications of a system under test should be benchmarked, it is not necessary to [create a new benchmark](creating-a-benchmark).
Instead, also Executions allow to do small reconfigurations, such as switching on or off a specific Pod scheduler. Instead, also Executions allow to do small reconfigurations, such as switching on or off a specific Pod scheduler.
This is done by defining `configOverrides` in the Execution. Each override consists of a patcher, defining which Kubernetes resource should be patched in which way, and a value the patcher is applied with. This is done by defining `configOverrides` in the Execution. Each override consists of a patcher, defining which Kubernetes resource should be patched in which way, and a value the patcher is applied with.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment