Addd more content on metrics and strategies

docs/_sass/custom/custom.scss

+.main-content a {
+    white-space: normal;
+}
+
 .theodolite-logo {
     height: 18em;
 }
+
+.MJXc-display {
+    overflow-x: auto;
+}
+
+table.fixed-colums {
+    th, td {
+       width: 1%;
+    }
+}

docs/concepts/index.md

@@ -11,9 +11,9 @@ nav_order: 2
 Theodolite is a framework for benchmarking the horizontal and vertical scalability of cloud-native applications.
 
 Theodolite adopts established definitions of scalability in cloud computing for its benchmarking method. It quantifies
-scalability by running isolated experiments for different load intensities and provisioned resource amounts, which assess whether specified SLOs are fulfilled. Two metrics are available: The demand metric describes how the amount of minimal required resources evolve with increasing load intensities, while the capacity metric describes how the maximal processable load evolves with increasing resources. Hence, both metrics are functions. <!--Example?-->
+scalability by running isolated experiments for different load intensities and provisioned resource amounts, which assess whether specified SLOs are fulfilled. [Two metrics are available](metrics): The demand metric describes how the amount of minimal required resources evolve with increasing load intensities, while the capacity metric describes how the maximal processable load evolves with increasing resources. Hence, both metrics are functions. <!--Example?-->
 
-Load, resources and SLOs are deliberately kept abstract in Theodolite as Theodolite supports different types of these. For example, horizontal scalability can be benchmarked by varying the amount of Kubernetes Pods, while vertical scalability can be benchmarked by varying CPU and memory constraints of Pods.
+The terms load, resources and SLOs are consciously kept abstract as Theodolite leaves it to the benchmark designer to define what type of load, resources, and SLOs should be evaluated. For example, horizontal scalability can be benchmarked by varying the amount of Kubernetes Pods, while vertical scalability can be benchmarked by varying CPU and memory constraints of Pods.
 
-To balance statistical grounding and time-efficient benchmark execution, Theodolite comes with different heuristic for
-evaluating the search space of load and resource combinations. Other configuration options include the number of repetitions, the experiment and warm-up duration, as well as the amount of different load and resource values to be evaluated.
+To balance statistical grounding and time-efficient benchmark execution, Theodolite comes with different [heuristic for
+evaluating the search space](search-strategies) of load and resource combinations. Other configuration options include the number of repetitions, the experiment and warm-up duration, as well as the amount of different load and resource values to be evaluated.
\ No newline at end of file

docs/concepts/metrics.md

@@ -8,35 +8,61 @@ usemathjax: true
 
 # Theodolite's Scalability Metrics
 
+Theodolite's scalability metrics are based on the following definition:
+
+> Scalability is the ability of [a] system to sustain increasing workloads by making use of additional resources. -- <cite>[Herbst et al. (2013)](https://www.usenix.org/conference/icac13/technical-sessions/presentation/herbst)</cite>
+
+Based on this definition, scalability can be characterized by the following three attributes:
+
+* **Load intensity** is the input variable to which a system is subjected. Scalability is evaluated within a range of load intensities.
+* **Service levels objectives (SLOs)** are measurable quality criteria that have to be fulfilled for every load intensity.
+* **Provisioned resources** can be increased to meet the SLOs if load intensities increase.
+
+
+## Scalability Metrics
+
+Theodolite uses these attributes to define two scalability metrics:
+
+| Resource Demand Metric | Load Capacity Metric |
+|:----|:----|
+| The resource demand metric quantifies scalability by describing how the amount of minimal required resources (i.e., all SLOs are fulfilled) evolves with increasing load intensities. | The load capacity metric quantifies scalability by describing how the maximal processable load (i.e., all SLOs are fulfilled) evolves with increasing resources. |
+| ![](../../assets/images/demand.svg){: .d-block .mx-auto } *Example: Scalability of two stream processing engines measured with the demand metric.*{: .d-block .text-center } | ![](../../assets/images/capacity.svg){: .d-block .mx-auto }  *Example: Scalability of two stream processing engines measured with the capacity metric.*{: .d-block .text-center } |
+{: .fixed-colums }
+
+<!--
 ## Resource Demand Metric
 
+![](../../assets/images/capacity.svg){: .d-block .mx-auto }
+
+
+## Load Capacity Metric
+
+![](../../assets/images/demand.svg){: .d-block .mx-auto }
+-->
+
+
+## Formal Definition
+
+For a more formal definition of both metrics, we define the load type as the set of possible load intensities for that
+type, denoted as $$L$$.
+Similarly, we define the resource type as the set of possible resources, denoted as $$R$$.
+We also require that there exists an ordering on both sets $$L$$ and $$R$$.
+We define the set of all SLOs as $$S$$ and denote an SLO $$s \in S$$ as Boolean-valued function
+$$\text{slo}_s: L \times R \to \{\text{false},\text{true}\}$$ with $$\text{slo}_s(l,r) = \text{true}$$ if a system deployed with $$r$$ resource amounts does not violate SLO $$s$$ when processing load intensity $$l$$.
+
+We can denote our **resource demand** metric as $$\text{demand: } L \to R$$, defined as:
+
 $$
 \forall l \in L: \text{demand}(l) = \min\{r \in R \mid \forall s \in S: \text{slo}_s(l,r) = \text{true}\}
 $$
 
-## Load Capacity Metric
+And similarly denote our **resource capacity** metric as $$\text{capacity: } R \to L$$, defined as:
 
 $$
 \forall r \in R: \text{capacity}(r) = \max\{l \in L \mid \forall s \in S: \text{slo}_s(l,r) = \text{true}\}
 $$
 
+## Further Reading
 
-
-## Fun with Math
-
-$$
-\begin{aligned}
-  & \phi(x,y) = \phi \left(\sum_{i=1}^n x_ie_i, \sum_{j=1}^n y_je_j \right)
-  = \sum_{i=1}^n \sum_{j=1}^n x_i y_j \phi(e_i, e_j) = \\
-  & (x_1, \ldots, x_n) \left( \begin{array}{ccc}
-      \phi(e_1, e_1) & \cdots & \phi(e_1, e_n) \\
-      \vdots & \ddots & \vdots \\
-      \phi(e_n, e_1) & \cdots & \phi(e_n, e_n)
-    \end{array} \right)
-  \left( \begin{array}{c}
-      y_1 \\
-      \vdots \\
-      y_n
-    \end{array} \right)
-\end{aligned}
-$$
+S. Henning and W. Hasselbring. “[How to Measure Scalability of Distributed Stream Processing Engines?](https://research.spec.org/icpe_proceedings/2021/companion/p85.pdf)” In: International Conference on Performance Engineering. 2021. DOI: [10.1145/3447545.3451190](https://doi.org/10.1145/3447545.3451190).
+<!-- TODO replace with EMSE paper one day-->

docs/concepts/search-strategies.md

@@ -6,3 +6,6 @@ nav_order: 2
 ---
 
 # Theodolite's Search Strategies
+
+Theodolite measures [its metrics](metrics) by performing isolated experiments for different load intensities and provisioned resource amounts.
+However, to determine a system's resource demand or load capacity or a good approximation of those, it is often not necessary to evaluate each possible combination. Instead, Theodolite provides search strategies, which decide at benchmark runtime which combinations of load and resources to evaluate.
\ No newline at end of file