"# Theodolite Analysis - Plotting the Demand Metric\n",
"\n",
"This notebook creates a plot, showing scalability as a function that maps load intensities to the resources required for processing them. It is able to combine multiple such plots in one figure, for example, to compare multiple systems or configurations.\n",
"\n",
"The notebook takes a CSV file for each plot mapping load intensities to minimum required resources, computed by the `demand-metric-plot.ipynb` notebook."
],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, we need to import some libraries, which are required for creating the plots."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import pandas as pd\n",
...
...
@@ -28,20 +30,20 @@
"import matplotlib.pyplot as plt\n",
"from matplotlib.ticker import FuncFormatter\n",
"from matplotlib.ticker import MaxNLocator"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We need to specify the directory, where the demand CSV files can be found, and a dictionary that maps a system description (e.g. its name) to the corresponding CSV file (prefix). To use Unicode narrow non-breaking spaces in the description format it as `u\"1000\\u202FmCPU\"`."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"results_dir = '<path-to>/results'\n",
"plot_name = '<plot-name>'\n",
...
...
@@ -49,102 +51,54 @@
"experiments = {\n",
" 'System XYZ': 'exp200',\n",
"}\n"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we combie all systems described in `experiments`."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dataframes = [pd.read_csv(os.path.join(results_dir, f'{v}_demand.csv')).set_index('load').rename(columns={\"resources\": k}) for k, v in experiments.items()]\n",
"We might want to display the mappings before we plot it."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df"
],
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>System XYZ</th>\n",
" </tr>\n",
" <tr>\n",
" <th>load</th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>50000</th>\n",
" <td>1</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" System XYZ\n",
"load \n",
"50000 1"
]
},
"metadata": {},
"execution_count": 13
}
],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The following code creates a MatPlotLib figure showing the scalability plots for all specified systems. You might want to adjust its styling etc. according to your preferences. Make sure to also set a filename."
# Theodolite Analysis - Plotting the Demand Metric
This notebook creates a plot, showing scalability as a function that maps load intensities to the resources required for processing them. It is able to combine multiple such plots in one figure, for example, to compare multiple systems or configurations.
The notebook takes a CSV file for each plot mapping load intensities to minimum required resources, computed by the `demand-metric-plot.ipynb` notebook.
%% Cell type:markdown id: tags:
First, we need to import some libraries, which are required for creating the plots.
%% Cell type:code id: tags:
```
import os
import pandas as pd
from functools import reduce
import matplotlib.pyplot as plt
from matplotlib.ticker import FuncFormatter
from matplotlib.ticker import MaxNLocator
```
%% Cell type:markdown id: tags:
We need to specify the directory, where the demand CSV files can be found, and a dictionary that maps a system description (e.g. its name) to the corresponding CSV file (prefix). To use Unicode narrow non-breaking spaces in the description format it as `u"1000\u202FmCPU"`.
%% Cell type:code id: tags:
```
results_dir = '<path-to>/results'
plot_name = '<plot-name>'
experiments = {
'System XYZ': 'exp200',
}
```
%% Cell type:markdown id: tags:
Now, we combie all systems described in `experiments`.
%% Cell type:code id: tags:
```
dataframes = [pd.read_csv(os.path.join(results_dir, f'{v}_demand.csv')).set_index('load').rename(columns={"resources": k}) for k, v in experiments.items()]
We might want to display the mappings before we plot it.
%% Cell type:code id: tags:
```
df
```
%% Output
System XYZ
load
50000 1
%% Cell type:markdown id: tags:
The following code creates a MatPlotLib figure showing the scalability plots for all specified systems. You might want to adjust its styling etc. according to your preferences. Make sure to also set a filename.
"Theodolite's *demand* metric is a function, mapping load intensities to the minimum required resources (e.g., instances) that are required to process this load. With this notebook, the *demand* metric function is approximated by a map of tested load intensities to their minimum required resources.\n",
"\n",
"The final output when running this notebook will be a CSV file, providig this mapping. It can be used to create nice plots of a system's scalability using the `demand-metric-plot.ipynb` notebook."
],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the following cell, we need to specifiy:\n",
"\n",
...
...
@@ -23,126 +24,102 @@
"* `max_lag_trend_slope`: The maximum tolerable increase in queued messages per second.\n",
"* `measurement_dir`: The directory where the measurement data files are to be found.\n",
"* `results_dir`: The directory where the computed demand CSV files are to be stored."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"exp_id = 200\n",
"warmup_sec = 60\n",
"max_lag_trend_slope = 2000\n",
"measurement_dir = '<path-to>/measurements'\n",
"results_dir = '<path-to>/results'\n"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With the following call, we compute our demand mapping."
This notebook applies Theodolite's *demand* metric to describe scalability of a SUT based on Theodolite measurement data.
Theodolite's *demand* metric is a function, mapping load intensities to the minimum required resources (e.g., instances) that are required to process this load. With this notebook, the *demand* metric function is approximated by a map of tested load intensities to their minimum required resources.
The final output when running this notebook will be a CSV file, providig this mapping. It can be used to create nice plots of a system's scalability using the `demand-metric-plot.ipynb` notebook.
%% Cell type:markdown id: tags:
In the following cell, we need to specifiy:
*`exp_id`: The experiment id that is to be analyzed.
*`warmup_sec`: The number of seconds which are to be ignored in the beginning of each experiment.
*`max_lag_trend_slope`: The maximum tolerable increase in queued messages per second.
*`measurement_dir`: The directory where the measurement data files are to be found.
*`results_dir`: The directory where the computed demand CSV files are to be stored.
%% Cell type:code id: tags:
```
exp_id = 200
warmup_sec = 60
max_lag_trend_slope = 2000
measurement_dir = '<path-to>/measurements'
results_dir = '<path-to>/results'
```
%% Cell type:markdown id: tags:
With the following call, we compute our demand mapping.
