Add capacity metric plot notebook and minor fixes

analysis/README.md

@@ -5,6 +5,7 @@ benchmark execution results and plotting. The following notebooks are provided:
 
 * [demand-metric.ipynb](demand-metric.ipynb): Create CSV files describing scalability according to the Theodolite `demand` metric.
 * [demand-metric-plot.ipynb](demand-metric-plot.ipynb): Create plots based on such CSV files of the `demand` metric.
+* [capacity-metric-plot.ipynb](capacity-metric-plot.ipynb): Create plots based on such CSV files of the `capacity` metric.
 
 For legacy reasons, we also provide the following notebooks, which, however, are not documented:
 

analysis/capacity-metric-plot.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Theodolite Analysis - Plotting the Capacity Metric\n",
+    "\n",
+    "This notebook creates a plot, showing scalability as a function that maps provisioned resources to the load intensities they process. It is able to combine multiple such plots in one figure, for example, to compare multiple systems or configurations.\n",
+    "\n",
+    "The notebook takes a CSV file for each plot mapping provisioned resources to the maximum processable load intensities, provided by Theodolite directly or computed by the `capacity-metric.ipynb` notebook."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First, we need to import some libraries, which are required for creating the plots."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "import pandas as pd\n",
+    "from functools import reduce\n",
+    "import matplotlib.pyplot as plt\n",
+    "from matplotlib.ticker import FuncFormatter\n",
+    "from matplotlib.ticker import MaxNLocator"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We need to specify the directory, where the capacity CSV files can be found, and a dictionary that maps a system description (e.g. its name) to the corresponding CSV file (prefix). To use Unicode narrow non-breaking spaces in the description format it as `u\"1000\\u202FmCPU\"`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "results_dir = '<path-to>/results'\n",
+    "\n",
+    "experiments = {\n",
+    "    'System XYZ': 'exp200',\n",
+    "}\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now, we combie all systems described in `experiments`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataframes = [pd.read_csv(os.path.join(results_dir, f'{v}_capacity.csv')).set_index('resource').rename(columns={\"loads\": k}) for k, v in experiments.items()]\n",
+    "\n",
+    "df = reduce(lambda df1,df2: df1.join(df2,how='outer'), dataframes)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We might want to display the mappings before we plot it."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "df"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The following code creates a MatPlotLib figure showing the scalability plots for all specified systems. You might want to adjust its styling etc. according to your preferences. Make sure to also set a filename."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "plt.style.use('ggplot')\n",
+    "plt.rcParams['pdf.fonttype'] = 42 # TrueType fonts\n",
+    "plt.rcParams['ps.fonttype'] = 42 # TrueType fonts\n",
+    "plt.rcParams['axes.facecolor']='w'\n",
+    "plt.rcParams['axes.edgecolor']='555555'\n",
+    "#plt.rcParams['ytick.color']='black'\n",
+    "plt.rcParams['grid.color']='dddddd'\n",
+    "plt.rcParams['axes.spines.top']='false'\n",
+    "plt.rcParams['axes.spines.right']='false'\n",
+    "plt.rcParams['legend.frameon']='true'\n",
+    "plt.rcParams['legend.framealpha']='1'\n",
+    "plt.rcParams['legend.edgecolor']='1'\n",
+    "plt.rcParams['legend.borderpad']='1'\n",
+    "\n",
+    "@FuncFormatter\n",
+    "def load_formatter(x, pos):\n",
+    "    return f'{(x/1000):.0f}k'\n",
+    "\n",
+    "markers = ['s', 'D', 'o', 'v', '^', '<', '>', 'p', 'X']\n",
+    "\n",
+    "def splitSerToArr(ser):\n",
+    "    return [ser.index, ser.as_matrix()]\n",
+    "\n",
+    "plt.figure()\n",
+    "#plt.figure(figsize=(4.8, 3.6)) # For other plot sizes\n",
+    "#ax = df.plot(kind='line', marker='o')\n",
+    "for i, column in enumerate(df):\n",
+    "    plt.plot(df[column].dropna(), marker=markers[i], label=column)\n",
+    "plt.legend()\n",
+    "ax = plt.gca()\n",
+    "#ax = df.plot(kind='line',x='dim_value', legend=False, use_index=True)\n",
+    "ax.set_ylabel('messages/second')\n",
+    "ax.set_xlabel('number of instances')\n",
+    "ax.set_ylim(ymin=0)\n",
+    "#ax.set_xlim(xmin=0)\n",
+    "ax.xaxis.set_major_locator(MaxNLocator(integer=True))\n",
+    "ax.yaxis.set_major_formatter(FuncFormatter(load_formatter))\n",
+    "\n",
+    "plt.savefig('temp.pdf', bbox_inches='tight')"
+   ]
+  }
+ ],
+ "metadata": {
+  "file_extension": ".py",
+  "interpreter": {
+   "hash": "e9e076445e1891a25f59b525adcc71b09846b3f9cf034ce4147fc161b19af121"
+  },
+  "kernelspec": {
+   "display_name": "Python 3.8.10 64-bit ('.venv': venv)",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.17"
+  },
+  "mimetype": "text/x-python",
+  "name": "python",
+  "npconvert_exporter": "python",
+  "orig_nbformat": 2,
+  "pygments_lexer": "ipython3",
+  "version": 3
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

analysis/demand-metric-plot.ipynb

@@ -8,7 +8,7 @@
     "\n",
     "This notebook creates a plot, showing scalability as a function that maps load intensities to the resources required for processing them. It is able to combine multiple such plots in one figure, for example, to compare multiple systems or configurations.\n",
     "\n",
-    "The notebook takes a CSV file for each plot mapping load intensities to minimum required resources, computed by the `demand-metric-plot.ipynb` notebook."
+    "The notebook takes a CSV file for each plot mapping load intensities to minimum required resources, provided by Theodolite directly or computed by the `demand-metric.ipynb` notebook."
    ]
   },
   {

analysis/demand-metric.ipynb

@@ -10,7 +10,7 @@
     "\n",
     "Theodolite's *demand* metric is a function, mapping load intensities to the minimum required resources (e.g., instances) that are required to process this load. With this notebook, the *demand* metric function is approximated by a map of tested load intensities to their minimum required resources.\n",
     "\n",
-    "The final output when running this notebook will be a CSV file, providig this mapping. It can be used to create nice plots of a system's scalability using the `demand-metric-plot.ipynb` notebook."
+    "The final output when running this notebook will be a CSV file, providing this mapping. It can be used to create nice plots of a system's scalability using the `demand-metric-plot.ipynb` notebook."
    ]
   },
   {

analysis/requirements.txt

@@ -2,3 +2,4 @@ jupyter==1.0.0
 matplotlib==3.2.0
 pandas==1.0.1
 scikit-learn==0.22.2.post1
+numpy==1.23.1