Introduce USL extensions

analysis/usl-offline-analysis-tool.Rmd

+---
+title: "USL Offline Analysis Tool"
+output: html_notebook
+---
+
+This is an [R Markdown](http://rmarkdown.rstudio.com) Notebook for performing USL analysis on the benchmark results of Theodolite. The notebook is configured to write the USL analysis results to the file system.
+
+At the top of the document there are the following parameters that can be set:
+
+- resources and load dimension names: According to the version of Theodolite, the format of the CSV files may be different. Therefore, it is possible to set the resources and load dimension names to the values of the input CSV files
+- directory: Specifies the directory where the benchmark results files are located
+- expID: Specifies the ID of the benchmark in Theodolite
+- resources and load normalization: There are cases where the computation of the USL model lacks numerical stability because of floating point precision. Consequently, it can make sense to normalize the load values in order to be able to generate a USL model. This only influences the gamma parameter of the USL proportionally by the normalization factor. In other cases, one may be interested to also normalize the resources dimension values.
+
+```{r}
+library(usl)
+library(jsonlite)
+library(dplyr)
+library(purrr)
+
+resources_dimension_name <- "resource" # may be set to the according CSV file column
+load_dimension_name <- "loads"  # may be set to the according CSV file column
+directory <- "./" # directory which contains the exp{N}_capacity.csv and exp{N}-results files
+expID <- "28" # the experiment ID {N}
+resources_normalization <- 1 # all the resources values are divided by this value
+load_normalization <- 1000 # all the load values are divided by this value
+filename <- paste("./exp",  expID,  "_capacity.csv", sep = "")
+
+# import data and compute usl model
+# capacity units are changed to a thousand * (load value)
+dataset <- read.csv(filename, colClasses = c("numeric", "numeric"))
+dataset[resources_dimension_name] = map(dataset[resources_dimension_name], function(x) x/resources_normalization)
+dataset[load_dimension_name] = map(dataset[load_dimension_name], function(x) ((x/load_normalization)))
+usl.model <- usl(as.formula(paste(load_dimension_name, resources_dimension_name, sep = " ~ ")), dataset)
+
+# print summary of model + visualizations
+sink("summary.txt")
+print(summary(usl.model))
+sink()
+
+# plot efficiency bars
+plot.new()
+pdf("efficiency-bars.pdf")
+par(ps=21)
+barplot(efficiency((usl.model)))
+dev.off()
+
+# save confidence interval for level of 0.95
+sink("confidence-intervals.txt")
+print(confint(usl.model, level = 0.95))
+sink()
+
+# store efficiency as plain text
+sink("efficiency.txt")
+print(efficiency(usl.model))
+sink()
+
+# plot efficiency as line
+plot.new()
+pdf("efficiency-plot.pdf")
+efficiencyValues <- efficiency(usl.model)
+par(ps=21)
+names(efficiencyValues)
+plot(efficiencyValues, x = names(efficiencyValues), type = "b", pch = 19, xlab = resources_dimension_name, ylab = "efficiency rate", xaxt = "n", ylim = c(0, 2))
+axis(side=1, at=dataset[[resources_dimension_name]])
+dev.off()
+
+# plot usl function
+plot.new()
+pdf("usl-plotted.pdf") # pdf export start
+par(ps=19)
+plot(dataset, pch = 16, ylim = c(0, max(dataset[load_dimension_name])), xlim = c(0, max(dataset[resources_dimension_name])), xaxt="n", xlab=resources_dimension_name, ylab=load_dimension_name)
+axis(side=1, at=dataset[[resources_dimension_name]])
+plot(function(x) x*coef(usl.model)[["gamma"]], 0, max(dataset[resources_dimension_name]), add = TRUE)
+plot(usl.model, col="red", add=TRUE, bounds = FALSE)
+dev.off()  # pdf export end
+
+# compute and print usl search strategy efficiency
+plot.new()
+results <- unlist(read_json(path = paste("exp", expID, "-result", sep = ""))["results"][[1]])
+results_df <- data.frame(matrix(results, ncol = 3, byrow = TRUE))
+colnames(results_df) <- c("load", "resources", "success")
+slo_experiments <- results_df %>% group_by("resources") %>% summarise(n_distinct("load"))
+pdf("number-experiments-bars.pdf")
+par(ps=21)
+barplot(slo_experiments$`n_distinct(load)`, names.arg = slo_experiments$resources, xlab = resources_dimension_name, ylab = "executed SLO experiments")
+dev.off()
+```
+
+Add a new chunk by clicking the *Insert Chunk* button on the toolbar or by pressing *Ctrl+Alt+I*.
+
+When you save the notebook, an HTML file containing the code and output will be saved alongside it (click the *Preview* button or press *Ctrl+Shift+K* to preview the HTML file).
+
+The preview shows you a rendered HTML copy of the contents of the editor. Consequently, unlike *Knit*, *Preview* does not run any R code chunks. Instead, the output of the chunk when it was last run in the editor is displayed.