Skip to content
Snippets Groups Projects

Adjust Benchmark Definitions for Operator

Merged Sören Henning requested to merge adjust-benchmark-def-for-operator into theodolite-kotlin
1 file
+ 43
0
Compare changes
  • Side-by-side
  • Inline
+ 43
0
 
---
 
title: Benchmarks and Executions
 
has_children: false
 
nav_order: 2
 
---
 
 
# Benchmarks and Executions
 
 
In Theodolite, we distinguish between the static description of a scalability benchmark and its execution.
 
 
## Benchmarks
 
 
Benchmarks define what should be executed in scalability experiments. They
 
consists of system under test (SUT) and an associated load generator, where
 
both SUT and load generator are represented as sets of Kubernetes resources
 
such Pods, Services, or ConfigMaps.
 
Additionally, benchmarks define one or more load and resource types, scalability
 
can be evaluated for.
 
 
[TODO]: # (Link to metrics)
 
 
Benchmarks are portable, that is they can be provided by standardization
 
organizations, researchers, etc. and installed by other researchers,
 
engineers, etc.
 
They do not have a life-cycle. Instead, to run a benchmark, an execution of a
 
benchmark is created and passed to Theodolite.
 
 
## Execution
 
 
An execution represents a one-time execution of a benchmark with a
 
specific configuration. Hence, an execution refers to a benchmark and one of
 
the load and resource types defined by that benchmark. Additionally, executions
 
allow to alter the SUT or the load generator to evaluate regarding specific
 
configurations.
 
 
Executions also describe details regarding the scalability measurement method,
 
such as for long experiment should be performed or how often they should be
 
repeated.
 
 
In contrast to benchmarks, execution have a life-cycle. They can be planned,
 
executed, or aborted. Each execution of a benchmark is represented by an
 
individual entity. This supports repeatability as executions can be archived
 
and shared.
Loading