diff --git a/analysis/README.md b/analysis/README.md index 3c96cf0b6e67a60ebbb4c610ca69fcbcb27876a0..8d37f01c011e74bf258e2d411bc72f32f0ddcfdc 100644 --- a/analysis/README.md +++ b/analysis/README.md @@ -9,7 +9,7 @@ benchmark execution results and plotting. The following notebooks are provided: For legacy reasons, we also provide the following notebooks, which, however, are not documented: * [scalability-graph.ipynb](scalability-graph.ipynb): Creates a scalability graph for a certain benchmark execution. -* [scalability-graph-final.ipynb](scalability-graph-final.ipynb): Combines the scalability graphs of multiple benchmarks executions (e.g. for comparing different configuration). +* [scalability-graph-plotter.ipynb](scalability-graph-plotter.ipynb): Combines the scalability graphs of multiple benchmarks executions (e.g. for comparing different configuration). * [lag-trend-graph.ipynb](lag-trend-graph.ipynb): Visualizes the consumer lag evaluation over time along with the computed trend. ## Usage diff --git a/analysis/demand-metric-plot.ipynb b/analysis/demand-metric-plot.ipynb index 95f371510bbcc8af785739c50bce42e969ea2b80..985d1fc91caec847f1795234903d1cbb34e3ddba 100644 --- a/analysis/demand-metric-plot.ipynb +++ b/analysis/demand-metric-plot.ipynb @@ -34,7 +34,7 @@ }, { "source": [ - "We need to specify the directory, where the demand CSV files can be found, and a dictionary that maps a system description (e.g. its name) to the corresponding CSV file (prefix). " + "We need to specify the directory, where the demand CSV files can be found, and a dictionary that maps a system description (e.g. its name) to the corresponding CSV file (prefix). To use Unicode narrow non-breaking spaces in the description format it as `u\"1000\\u202FmCPU\"`." ], "cell_type": "markdown", "metadata": {} diff --git a/analysis/demand-metric.ipynb b/analysis/demand-metric.ipynb index 525bde211afcabeecf52f1e88f3c91c02a77a152..bcea129b7cb07465fa99f32b6f8b2b6115e8a0aa 100644 --- a/analysis/demand-metric.ipynb +++ b/analysis/demand-metric.ipynb @@ -4,7 +4,7 @@ "source": [ "# Theodolite Analysis - Demand Metric\n", "\n", - "This notebook allows applies Theodolite's *demand* metric to describe scalability of a SUT based on Theodolite measurement data.\n", + "This notebook applies Theodolite's *demand* metric to describe scalability of a SUT based on Theodolite measurement data.\n", "\n", "Theodolite's *demand* metric is a function, mapping load intensities to the minimum required resources (e.g., instances) that are required to process this load. With this notebook, the *demand* metric function is approximated by a map of tested load intensities to their minimum required resources.\n", "\n", diff --git a/analysis/scalability-graph-finish.ipynb b/analysis/scalability-graph-plotter.ipynb similarity index 100% rename from analysis/scalability-graph-finish.ipynb rename to analysis/scalability-graph-plotter.ipynb diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000000000000000000000000000000000000..4fd13bdfc157efe8b3491695bb83972f96a82c5d --- /dev/null +++ b/docs/README.md @@ -0,0 +1,25 @@ +--- +title: Theodolite +nav_order: 1 +permalink: / +--- + +# Theodolite + +> A theodolite is a precision optical instrument for measuring angles between designated visible points in the horizontal and vertical planes. -- <cite>[Wikipedia](https://en.wikipedia.org/wiki/Theodolite)</cite> + +Theodolite is a framework for benchmarking the horizontal and vertical scalability of stream processing engines. It consists of three modules: + +## Theodolite Benchmarks + +Theodolite contains 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding workload generator is provided. Currently, this repository provides benchmark implementations for Kafka Streams. + + +## Theodolite Execution Framework + +Theodolite aims to benchmark scalability of stream processing engines for real use cases. Microservices that apply stream processing techniques are usually deployed in elastic cloud environments. Hence, Theodolite's cloud-native benchmarking framework deploys as components in a cloud environment, orchestrated by Kubernetes. More information on how to execute scalability benchmarks can be found in [Thedolite execution framework](execution). + + +## Theodolite Analysis Tools + +Theodolite's benchmarking method create a *scalability graph* allowing to draw conclusions about the scalability of a stream processing engine or its deployment. A scalability graph shows how resource demand evolves with an increasing workload. Theodolite provides Jupyter notebooks for creating such scalability graphs based on benchmarking results from the execution framework. More information can be found in [Theodolite analysis tool](analysis). diff --git a/docs/_config.yml b/docs/_config.yml new file mode 100644 index 0000000000000000000000000000000000000000..b0f0a13c22083b21a7c90ceaed44b846ffe55550 --- /dev/null +++ b/docs/_config.yml @@ -0,0 +1,6 @@ +title: "Theodolite" +remote_theme: pmarsceill/just-the-docs +#color_scheme: "dark" +aux_links: + "Theodolite on GitHub": + - "//github.com/cau-se/theodolite" \ No newline at end of file diff --git a/docs/release-process.md b/docs/release-process.md index f267d611fb08931dd766c2f9952655f9dae62e32..c53ea4423eb1dbf521d13286448f33a9613b71ef 100644 --- a/docs/release-process.md +++ b/docs/release-process.md @@ -1,3 +1,9 @@ +--- +title: Release Process +has_children: false +nav_order: 2 +--- + # Release Process We assume that we are creating the release `v0.1.1`. Please make sure to adjust diff --git a/execution/infrastructure/kafka/values.yaml b/execution/infrastructure/kafka/values.yaml index e65a5fc567d39c7389479d406fa9e6d7156b0f0a..15fd8a822a18521f247584d1becbd09c19c137d2 100644 --- a/execution/infrastructure/kafka/values.yaml +++ b/execution/infrastructure/kafka/values.yaml @@ -48,14 +48,15 @@ cp-kafka: # cpu: 100m # memory: 128Mi configurationOverrides: - #"offsets.topic.replication.factor": "3" + # offsets.topic.replication.factor: "3" "message.max.bytes": "134217728" # 128 MB "replica.fetch.max.bytes": "134217728" # 128 MB # "default.replication.factor": 3 # "min.insync.replicas": 2 "auto.create.topics.enable": false "log.retention.ms": "10000" # 10s - #"log.retention.ms": "86400000" # 24h + # "log.retention.ms": "86400000" # 24h + # "group.initial.rebalance.delay.ms": "30000" # 30s "metrics.sample.window.ms": "5000" #5s ## ------------------------------------------------------ diff --git a/execution/infrastructure/kafka/values_kafka_nodeport.yaml b/execution/infrastructure/kafka/values_kafka_nodeport.yaml new file mode 100644 index 0000000000000000000000000000000000000000..cf1deb3a0eda97039ad4609a1f07fa54d4d5d1ea --- /dev/null +++ b/execution/infrastructure/kafka/values_kafka_nodeport.yaml @@ -0,0 +1,97 @@ +## ------------------------------------------------------ +## Zookeeper +## ------------------------------------------------------ +cp-zookeeper: + enabled: true + servers: 1 + image: confluentinc/cp-zookeeper + imageTag: 5.4.0 + ## Optionally specify an array of imagePullSecrets. Secrets must be manually created in the namespace. + ## https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod + imagePullSecrets: + # - name: "regcred" + heapOptions: "-Xms512M -Xmx512M" + persistence: + enabled: false + resources: {} + ## If you do want to specify resources, uncomment the following lines, adjust them as necessary, + ## and remove the curly braces after 'resources:' + # limits: + # cpu: 100m + # memory: 128Mi + # requests: + # cpu: 100m + # memory: 128Mi + +## ------------------------------------------------------ +## Kafka +## ------------------------------------------------------ +cp-kafka: + enabled: true + brokers: 1 + image: confluentinc/cp-enterprise-kafka + imageTag: 5.4.0 + ## Optionally specify an array of imagePullSecrets. Secrets must be manually created in the namespace. + ## https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod + imagePullSecrets: + # - name: "regcred" + heapOptions: "-Xms512M -Xmx512M" + persistence: + enabled: false + resources: {} + ## If you do want to specify resources, uncomment the following lines, adjust them as necessary, + ## and remove the curly braces after 'resources:' + # limits: + # cpu: 100m + # memory: 128Mi + # requests: + # cpu: 100m + # memory: 128Mi + configurationOverrides: + offsets.topic.replication.factor: "1" + "message.max.bytes": "134217728" # 128 MB + "replica.fetch.max.bytes": "134217728" # 128 MB + # "default.replication.factor": 3 + # "min.insync.replicas": 2 + "auto.create.topics.enable": false + "log.retention.ms": "10000" # 10s + #"log.retention.ms": "86400000" # 24h + "metrics.sample.window.ms": "5000" #5s + + # access kafka from outside + nodeport: + enabled: true + +## ------------------------------------------------------ +## Schema Registry +## ------------------------------------------------------ +cp-schema-registry: + enabled: true + image: confluentinc/cp-schema-registry + imageTag: 5.4.0 + ## Optionally specify an array of imagePullSecrets. Secrets must be manually created in the namespace. + ## https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod + imagePullSecrets: + # - name: "regcred" + heapOptions: "-Xms512M -Xmx512M" + resources: {} + ## If you do want to specify resources, uncomment the following lines, adjust them as necessary, + ## and remove the curly braces after 'resources:' + # limits: + # cpu: 100m + # memory: 128Mi + # requests: + # cpu: 100m + # memory: 128Mi + +cp-kafka-rest: + enabled: false + +cp-kafka-connect: + enabled: false + +cp-ksql-server: + enabled: false + +cp-control-center: + enabled: false diff --git a/theodolite-quarkus/build.gradle b/theodolite-quarkus/build.gradle index 71ddbf657f49e196593a31231b4f2ca0b9ee497c..a63d3f9af4c45e45631a277bea38f8e9b5f17b7f 100644 --- a/theodolite-quarkus/build.gradle +++ b/theodolite-quarkus/build.gradle @@ -20,9 +20,11 @@ dependencies { testImplementation 'io.quarkus:quarkus-junit5' testImplementation 'io.rest-assured:rest-assured' - + implementation 'org.slf4j:slf4j-simple:1.7.29' + implementation 'io.github.microutils:kotlin-logging:1.12.0' implementation 'io.fabric8:kubernetes-client:5.0.0-alpha-2' - //implementation 'com.fkorotkov:kubernetes-dsl:2.8.1' + compile group: 'org.apache.kafka', name: 'kafka-clients', version: '2.7.0' + compile group: 'org.apache.zookeeper', name: 'zookeeper', version: '3.6.2' } group 'theodolite' @@ -47,9 +49,8 @@ compileKotlin { compileTestKotlin { kotlinOptions.jvmTarget = JavaVersion.VERSION_11 } - detekt { failFast = true // fail build on any finding buildUponDefaultConfig = true ignoreFailures = true -} +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/GreetingResource.kt b/theodolite-quarkus/src/main/kotlin/theodolite/GreetingResource.kt deleted file mode 100644 index 2cf79f2d805344ba2b19f8e0aea58ad682a5e36f..0000000000000000000000000000000000000000 --- a/theodolite-quarkus/src/main/kotlin/theodolite/GreetingResource.kt +++ /dev/null @@ -1,14 +0,0 @@ -package theodolite - -import javax.ws.rs.GET -import javax.ws.rs.Path -import javax.ws.rs.Produces -import javax.ws.rs.core.MediaType - -@Path("/hello-resteasy") -class GreetingResource { - - @GET - @Produces(MediaType.TEXT_PLAIN) - fun hello() = "Hello RESTEasy" -} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/Main.kt b/theodolite-quarkus/src/main/kotlin/theodolite/Main.kt index cf6a7cb5d1fa028e0841d58839b674800f251edc..3aae49e5a805c62284c54d0a905eabf89c306588 100644 --- a/theodolite-quarkus/src/main/kotlin/theodolite/Main.kt +++ b/theodolite-quarkus/src/main/kotlin/theodolite/Main.kt @@ -1,14 +1,17 @@ package theodolite + import io.quarkus.runtime.annotations.QuarkusMain +import theodolite.execution.TheodoliteExecutor +import mu.KotlinLogging +private val logger = KotlinLogging.logger {} @QuarkusMain object Main { @JvmStatic fun main(args: Array<String>) { - println("Running main method") - - - //Quarkus.run() + val theodolite = TheodoliteExecutor() + theodolite.run() + logger.info("Application started") } } diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/evaluation/SLOChecker.kt b/theodolite-quarkus/src/main/kotlin/theodolite/evaluation/SLOChecker.kt new file mode 100644 index 0000000000000000000000000000000000000000..10e113fdf1110e0c8bb30cad8a4b5f04f9e82c0d --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/evaluation/SLOChecker.kt @@ -0,0 +1,4 @@ +package theodolite.evaluation + +interface SLOChecker { +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/execution/BenchmarkExecutor.kt b/theodolite-quarkus/src/main/kotlin/theodolite/execution/BenchmarkExecutor.kt new file mode 100644 index 0000000000000000000000000000000000000000..82a40908990879624edd54dbb9c999df14ad4f2f --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/execution/BenchmarkExecutor.kt @@ -0,0 +1,40 @@ +package theodolite.execution + +import mu.KotlinLogging +import theodolite.util.AbstractBenchmark +import theodolite.util.LoadDimension +import theodolite.util.Resource +import theodolite.util.Results +import java.time.Duration + +private val logger = KotlinLogging.logger {} + +/** + * The Benchmark Executor runs a single experiment. + * + * @property benchmark + * @property results + * @property executionDuration + * @constructor Create empty Benchmark executor + */ +abstract class BenchmarkExecutor(val benchmark: AbstractBenchmark, val results: Results, val executionDuration: Duration) { + /** + * Run a experiment for the given parametrization, evaluate the experiment and save the result. + * + * @param load load to be tested. + * @param res resources to be tested. + * @return True, if the number of resources are suitable for the given load, false otherwise. + */ + abstract fun runExperiment(load: LoadDimension, res: Resource): Boolean + + /** + * Wait while the benchmark is running and log the number of minutes executed every 1 minute. + * + */ + fun waitAndLog() { + for (i in 1.rangeTo(executionDuration.toMinutes())) { + Thread.sleep(Duration.ofMinutes(1).toMillis()) + logger.info { "Executed: $i minutes" } + } + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/execution/BenchmarkExecutorImpl.kt b/theodolite-quarkus/src/main/kotlin/theodolite/execution/BenchmarkExecutorImpl.kt new file mode 100644 index 0000000000000000000000000000000000000000..a975085bf38f7d00146a4f4e415aa4444d85e991 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/execution/BenchmarkExecutorImpl.kt @@ -0,0 +1,19 @@ +package theodolite.execution + +import theodolite.util.AbstractBenchmark +import theodolite.util.LoadDimension +import theodolite.util.Resource +import theodolite.util.Results +import java.time.Duration + +class BenchmarkExecutorImpl(benchmark: AbstractBenchmark, results: Results, executionDuration: Duration) : BenchmarkExecutor(benchmark, results, executionDuration) { + override fun runExperiment(load: LoadDimension, res: Resource): Boolean { + benchmark.start(load, res) + this.waitAndLog() + benchmark.clearClusterEnvironment() + // todo evaluate + val result = false // if success else false + this.results.setResult(Pair(load, res), result) + return result; + } +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/execution/TestBenchmarkExecutorImpl.kt b/theodolite-quarkus/src/main/kotlin/theodolite/execution/TestBenchmarkExecutorImpl.kt new file mode 100644 index 0000000000000000000000000000000000000000..9823c4dd5c9800955e07eded87a76871a30c9e86 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/execution/TestBenchmarkExecutorImpl.kt @@ -0,0 +1,18 @@ +package theodolite.execution + +import theodolite.util.AbstractBenchmark +import theodolite.util.LoadDimension +import theodolite.util.Resource +import theodolite.util.Results +import java.time.Duration + +class TestBenchmarkExecutorImpl(private val mockResults: Array<Array<Boolean>>, benchmark: AbstractBenchmark, results: Results): + BenchmarkExecutor(benchmark, results, executionDuration = Duration.ofSeconds(1)) { + + override fun runExperiment(load: LoadDimension, res: Resource): Boolean { + val result = this.mockResults[load.get()][res.get()] + + this.results.setResult(Pair(load, res), result) + return result; + } +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/execution/TheodoliteExecutor.kt b/theodolite-quarkus/src/main/kotlin/theodolite/execution/TheodoliteExecutor.kt new file mode 100644 index 0000000000000000000000000000000000000000..ed5a651593324291bad5b367e59f3f224405e81b --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/execution/TheodoliteExecutor.kt @@ -0,0 +1,68 @@ +package theodolite.execution + +import mu.KotlinLogging +import theodolite.k8s.UC1Benchmark +import theodolite.strategies.restriction.LowerBoundRestriction +import theodolite.strategies.searchstrategy.CompositeStrategy +import theodolite.strategies.searchstrategy.LinearSearch +import theodolite.util.* +import java.nio.file.Paths +import java.time.Duration + +private val logger = KotlinLogging.logger {} + +class TheodoliteExecutor() { + val projectDirAbsolutePath = Paths.get("").toAbsolutePath().toString() + val resourcesPath = Paths.get(projectDirAbsolutePath, "./../../../resources/main/yaml/") + private fun loadConfig(): Config { + logger.info { resourcesPath } + val benchmark: UC1Benchmark = UC1Benchmark( + AbstractBenchmark.Config( + clusterZookeeperConnectionString = "my-confluent-cp-zookeeper:2181", + clusterKafkaConnectionString = "my-confluent-cp-kafka:9092", + externalZookeeperConnectionString = "localhost:2181", + externalKafkaConnectionString = "localhost:9092", + schemaRegistryConnectionString = "http://my-confluent-cp-schema-registry:8081", + kafkaPartition = 40, + kafkaReplication = 1, + kafkaTopics = listOf("input", "output"), + // TODO("handle path in a more nice way (not absolut)") + ucDeploymentPath = "$resourcesPath/aggregation-deployment.yaml", + ucServicePath = "$resourcesPath/aggregation-service.yaml", + wgDeploymentPath = "$resourcesPath/workloadGenerator.yaml", + configMapPath = "$resourcesPath/jmx-configmap.yaml", + ucImageURL = "ghcr.io/cau-se/theodolite-uc1-kstreams-app:latest", + wgImageURL = "ghcr.io/cau-se/theodolite-uc1-workload-generator:theodolite-kotlin-latest" + ) + ) + val results: Results = Results() + + val executionDuration = Duration.ofSeconds(60 * 5) + + val executor: BenchmarkExecutor = BenchmarkExecutorImpl(benchmark, results, executionDuration) + + val restrictionStrategy = LowerBoundRestriction(results) + val searchStrategy = LinearSearch(executor) + + return Config( + loads = listOf(5000, 10000).map { number -> LoadDimension(number) }, + resources = (1..6).map { number -> Resource(number) }, + compositeStrategy = CompositeStrategy( + executor, + searchStrategy, + restrictionStrategies = setOf(restrictionStrategy) + ), + executionDuration = executionDuration + ) + } + + fun run() { + // read or get benchmark config + val config = this.loadConfig() + + // execute benchmarks for each load + for (load in config.loads) { + config.compositeStrategy.findSuitableResource(load, config.resources) + } + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/k8s/ConfigMapManager.kt b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/ConfigMapManager.kt new file mode 100644 index 0000000000000000000000000000000000000000..bf18ff7df07b4eb1e13d4a8c273fecd9283be267 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/ConfigMapManager.kt @@ -0,0 +1,15 @@ +package theodolite.k8s + +import io.fabric8.kubernetes.api.model.ConfigMap +import io.fabric8.kubernetes.client.NamespacedKubernetesClient + +class ConfigMapManager(private val client: NamespacedKubernetesClient) { + + fun deploy(configMap: ConfigMap) { + this.client.configMaps().createOrReplace(configMap) + } + + fun delete(configMap: ConfigMap) { + this.client.configMaps().delete(configMap) + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/k8s/DeploymentManager.kt b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/DeploymentManager.kt new file mode 100644 index 0000000000000000000000000000000000000000..af59c523633d448da96d533c819823b5be5215c7 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/DeploymentManager.kt @@ -0,0 +1,80 @@ +package theodolite.k8s + +import io.fabric8.kubernetes.api.model.Container +import io.fabric8.kubernetes.api.model.EnvVar +import io.fabric8.kubernetes.api.model.EnvVarSource +import io.fabric8.kubernetes.api.model.Quantity +import io.fabric8.kubernetes.api.model.apps.Deployment +import io.fabric8.kubernetes.client.NamespacedKubernetesClient +import mu.KotlinLogging + +private val logger = KotlinLogging.logger {} + +class DeploymentManager(private val client: NamespacedKubernetesClient) { + + /** + * Sets the ContainerEvironmentVariables, creates new if variable don t exist. + * @param container - The Container + * @param map - Map of k=Name,v =Value of EnviromentVariables + */ + private fun setContainerEnv(container: Container, map: Map<String, String>) { + map.forEach { k, v -> + // filter for mathing name and set value + val x = container.env.filter { envVar -> envVar.name == k } + + if (x.isEmpty()) { + val newVar = EnvVar(k, v, EnvVarSource()) + container.env.add(newVar) + } else { + x.forEach { + it.value = v + } + } + } + } + + /** + * Set the environment Variable for a container + */ + fun setWorkloadEnv(workloadDeployment: Deployment, containerName: String, map: Map<String, String>) { + workloadDeployment.spec.template.spec.containers.filter { it.name == containerName } + .forEach { it: Container -> + setContainerEnv(it, map) + } + } + + /** + * Change the RessourceLimit of a container (Usally SUT) + */ + fun changeRessourceLimits(deployment: Deployment, ressource: String, containerName: String, limit: String) { + deployment.spec.template.spec.containers.filter { it.name == containerName }.forEach { + it.resources.limits.replace(ressource, Quantity(limit)) + } + } + + /** + * Change the image name of a container (SUT and the Worklaodgenerators) + */ + fun setImageName(deployment: Deployment, containerName: String, image: String) { + deployment.spec.template.spec.containers.filter { it.name == containerName }.forEach { + it.image = image + } + } + + /** + * Change the image name of a container (SUT and the Worklaodgenerators) + */ + fun setReplica(deployment: Deployment, replicas: Int) { + deployment.spec.setReplicas(replicas) + } + + // TODO potential add exception handling + fun deploy(deployment: Deployment) { + this.client.apps().deployments().createOrReplace(deployment) + } + + // TODO potential add exception handling + fun delete(deployment: Deployment) { + this.client.apps().deployments().delete(deployment) + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/k8s/ServiceManager.kt b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/ServiceManager.kt new file mode 100644 index 0000000000000000000000000000000000000000..a976849fac0e0df9d224e96f3d4d87bda1d97695 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/ServiceManager.kt @@ -0,0 +1,22 @@ +package theodolite.k8s + +import io.fabric8.kubernetes.api.model.Service +import io.fabric8.kubernetes.client.NamespacedKubernetesClient + +class ServiceManager(private val client: NamespacedKubernetesClient) { + + fun changeServiceName(service: Service, newName: String) { + + service.metadata.apply { + name = newName + } + } + + fun deploy(service: Service) { + client.services().createOrReplace(service) + } + + fun delete(service: Service) { + client.services().delete(service) + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/k8s/TopicManager.kt b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/TopicManager.kt new file mode 100644 index 0000000000000000000000000000000000000000..fe1f4f89c1b853d967f3cc8b94fcf4900d1e5f91 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/TopicManager.kt @@ -0,0 +1,73 @@ +package theodolite.k8s + +import mu.KotlinLogging +import org.apache.kafka.clients.admin.AdminClient +import org.apache.kafka.clients.admin.AdminClientConfig +import org.apache.kafka.clients.admin.ListTopicsResult +import org.apache.kafka.clients.admin.NewTopic + +private val logger = KotlinLogging.logger {} + +/** + * Manages the topics related tasks + * @param bootstrapServers Ip of the kafka server + */ +class TopicManager(bootstrapServers: String) { + private val props = hashMapOf<String, Any>(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG to bootstrapServers) + lateinit var kafkaAdmin: AdminClient + + init { + try { + kafkaAdmin = AdminClient.create(props) + } catch (e: Exception) { + logger.error { e.toString() } + } + } + + /** + * Creates topics. + * @param topics Map that holds a numPartition for each topic it should create + * @param replicationFactor + */ + fun createTopics(topics: Map<String, Int>, replicationFactor: Short) { + + val newTopics = mutableSetOf<NewTopic>() + for (i in topics) { + val tops = NewTopic(i.key, i.value, replicationFactor) + newTopics.add(tops) + } + kafkaAdmin.createTopics(newTopics) + logger.info { "Topics created" } + } + + fun createTopics(topics: List<String>, numPartitions: Int, replicationFactor: Short) { + + val newTopics = mutableSetOf<NewTopic>() + for (i in topics) { + val tops = NewTopic(i, numPartitions, replicationFactor) + newTopics.add(tops) + } + kafkaAdmin.createTopics(newTopics) + logger.info { "Creation of $topics started" } + } + + /** + * Deletes topics. + * @param topics + */ + fun deleteTopics(topics: List<String>) { + + val result = kafkaAdmin.deleteTopics(topics) + + try { + result.all().get() + } catch (ex: Exception) { + logger.error { ex.toString() } + } + logger.info { "Topics deleted" } + } + + fun getTopics(): ListTopicsResult? { + return kafkaAdmin.listTopics() + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/k8s/UC1Benchmark.kt b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/UC1Benchmark.kt new file mode 100644 index 0000000000000000000000000000000000000000..23a712bfa72c08cf19c0e4845a33714755a6e2d0 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/UC1Benchmark.kt @@ -0,0 +1,98 @@ +package theodolite.k8s + +import io.fabric8.kubernetes.api.model.ConfigMap +import io.fabric8.kubernetes.api.model.Service +import io.fabric8.kubernetes.api.model.apps.Deployment +import io.fabric8.kubernetes.client.DefaultKubernetesClient +import io.fabric8.kubernetes.client.NamespacedKubernetesClient +import mu.KotlinLogging +import theodolite.util.AbstractBenchmark +import theodolite.util.LoadDimension +import theodolite.util.Resource + +private val logger = KotlinLogging.logger {} + +class UC1Benchmark(config: Config) : AbstractBenchmark(config) { + private val workloadGeneratorStateCleaner: WorkloadGeneratorStateCleaner + private val topicManager: TopicManager + + // TODO("service monitor") + private val kubernetesClient: NamespacedKubernetesClient + private val yamlLoader: YamlLoader + private val deploymentManager: DeploymentManager + private val serviceManager: ServiceManager + private val configMapManager: ConfigMapManager + private var ucDeployment: Deployment + private var ucService: Service + private var wgDeployment: Deployment + private var configMap: ConfigMap + + init { + this.workloadGeneratorStateCleaner = + WorkloadGeneratorStateCleaner(this.config.externalZookeeperConnectionString, path = "/workload-generation") + this.topicManager = TopicManager(this.config.externalKafkaConnectionString) + this.kubernetesClient = DefaultKubernetesClient().inNamespace("default") + this.yamlLoader = YamlLoader(this.kubernetesClient) + this.deploymentManager = DeploymentManager(this.kubernetesClient) + this.serviceManager = ServiceManager(this.kubernetesClient) + this.configMapManager = ConfigMapManager(this.kubernetesClient) + ucDeployment = this.yamlLoader.loadDeployment(this.config.ucDeploymentPath) + ucService = this.yamlLoader.loadService(this.config.ucServicePath) + wgDeployment = this.yamlLoader.loadDeployment(this.config.wgDeploymentPath) + configMap = this.yamlLoader.loadConfigmap(this.config.configMapPath) + } + + override fun clearClusterEnvironment() { + this.workloadGeneratorStateCleaner.deleteAll() + this.topicManager.deleteTopics(this.config.kafkaTopics) + this.deploymentManager.delete(this.ucDeployment) + this.serviceManager.delete(this.ucService) + this.deploymentManager.delete(this.wgDeployment) + } + + override fun initializeClusterEnvironment() { + this.topicManager.createTopics( + this.config.kafkaTopics, + this.config.kafkaPartition, + this.config.kafkaReplication + ) + } + + override fun startSUT(resources: Resource) { + this.deploymentManager.setImageName(ucDeployment, "uc-application", this.config.ucImageURL) + + // set environment variables + val environmentVariables: MutableMap<String, String> = mutableMapOf() + environmentVariables.put("KAFKA_BOOTSTRAP_SERVERS", this.config.clusterKafkaConnectionString) + environmentVariables.put("SCHEMA_REGISTRY_URL", this.config.schemaRegistryConnectionString) + + + // setup deployment + this.deploymentManager.setReplica(ucDeployment, resources.get()) + this.deploymentManager.setWorkloadEnv(ucDeployment, "uc-application", environmentVariables) + + + // create kubernetes resources + this.deploymentManager.deploy(ucDeployment) + this.serviceManager.deploy(ucService) + this.configMapManager.deploy(configMap) + } + + override fun startWorkloadGenerator(load: LoadDimension) { + this.deploymentManager.setImageName(wgDeployment, "workload-generator", this.config.wgImageURL) + + + // TODO ("calculate number of required instances") + val requiredInstances = 1 + val environmentVariables: MutableMap<String, String> = mutableMapOf() + environmentVariables.put("KAFKA_BOOTSTRAP_SERVERS", this.config.clusterKafkaConnectionString) + environmentVariables.put("ZK_HOST", this.config.clusterZookeeperConnectionString.split(":")[0]) + environmentVariables.put("ZK_PORT", this.config.clusterZookeeperConnectionString.split(":")[1]) + environmentVariables["NUM_SENSORS"] = load.get().toString() + environmentVariables["INSTANCES"] = requiredInstances.toString() + + this.deploymentManager.setWorkloadEnv(this.wgDeployment, "workload-generator", environmentVariables) + this.deploymentManager.deploy(this.wgDeployment) + } + +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/k8s/WorkloadGeneratorStateCleaner.kt b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/WorkloadGeneratorStateCleaner.kt new file mode 100644 index 0000000000000000000000000000000000000000..7b1f8c29d60032e7e0980af2192d2928cf8c0cf6 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/WorkloadGeneratorStateCleaner.kt @@ -0,0 +1,82 @@ +package theodolite.k8s + +import mu.KotlinLogging +import org.apache.zookeeper.KeeperException +import org.apache.zookeeper.WatchedEvent +import org.apache.zookeeper.Watcher +import org.apache.zookeeper.ZooKeeper +import java.time.Duration + +private val logger = KotlinLogging.logger {} + +/** + * Resets the workloadgenerator states in zookeper (and potentially watches for Zookeper events) + * + * @param ip of zookeeper + * @param path path of the zookeeper node + */ +class WorkloadGeneratorStateCleaner(ip: String, val path: String) { + private val timeout: Duration = Duration.ofMillis(500) + private val retryAfter: Duration = Duration.ofSeconds(5) + lateinit var zookeeperClient: ZooKeeper + + init { + try { + val watcher: Watcher = ZookeeperWatcher() // defined below + zookeeperClient = ZooKeeper(ip, timeout.toMillis().toInt(), watcher) + } catch (e: Exception) { + logger.error { e.toString() } + } + } + + fun deleteAll() { + deleteRecusiveAll(this.path) + logger.info { "ZooKeeper reset was successful" } + } + + /** + * Deletes a Zookeeper node and its children with the corresponding path. + */ + private fun deleteRecusiveAll(nodePath: String) { + + while (true) { + var children: List<String> + try { + children = zookeeperClient.getChildren(nodePath, true) + } catch (e: KeeperException.NoNodeException) { + break; + } + // recursivly delete all children nodes + for (s: String in children) { + try { + deleteRecusiveAll("$nodePath/$s") + } catch (ex: Exception) { + logger.info { "$ex" } + } + } + + // delete main node + try { + zookeeperClient.delete(nodePath, -1) + break; + } catch (ex: Exception) { + // no instance of node found + if (ex is KeeperException.NoNodeException) { + break; + } else { + logger.error { ex.toString() } + } + } + Thread.sleep(retryAfter.toMillis()) + logger.info { "ZooKeeper reset was not successful. Retrying in 5s" } + } + } + + /** + * Currently empty, could be used to watch(and react) on certain zookeeper events + */ + private class ZookeeperWatcher : Watcher { + + override fun process(event: WatchedEvent) {} + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/k8s/YamlLoader.kt b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/YamlLoader.kt new file mode 100644 index 0000000000000000000000000000000000000000..5e37e97dfaefeb2761d13173efbfbd2b6adc41e7 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/k8s/YamlLoader.kt @@ -0,0 +1,71 @@ +package theodolite.k8s + +import io.fabric8.kubernetes.api.model.ConfigMap +import io.fabric8.kubernetes.api.model.Service +import io.fabric8.kubernetes.api.model.apiextensions.v1beta1.CustomResourceDefinition +import io.fabric8.kubernetes.api.model.apps.Deployment +import io.fabric8.kubernetes.client.NamespacedKubernetesClient +import mu.KotlinLogging + +private val logger = KotlinLogging.logger {} + +class YamlLoader(private val client: NamespacedKubernetesClient) { + + /** + * Parses a Service from a servive yaml + * @param path of the yaml file + * @return service from fabric8 + */ + fun loadService(path: String): Service { + return loadGenericRessource(path) { x: String -> client.services().load(x).get() } + } + + /** + * Parses a Service from a servive yaml + * @param path of the yaml file + * @return service from fabric8 + */ + fun loadServiceMonitor(path: String): CustomResourceDefinition { + return loadGenericRessource(path) { x: String -> client.customResourceDefinitions().load(x).get() } + } + + /** + * Parses a Deployment from a Deployment yaml + * @param path of the yaml file + * @return Deployment from fabric8 + */ + fun loadDeployment(path: String): Deployment { + return loadGenericRessource(path) { x: String -> client.apps().deployments().load(x).get() } + } + + /** + * Parses a ConfigMap from a ConfigMap yaml + * @param path of the yaml file + * @return ConfigMap from fabric8 + */ + fun loadConfigmap(path: String): ConfigMap { + return loadGenericRessource(path) { x: String -> client.configMaps().load(x).get() } + } + + /** + * Generic helper function to load a resource. + * @param path of the resource + * @param f fuction that shall be applied to the resource. + */ + private fun <T> loadGenericRessource(path: String, f: (String) -> T): T { + var resource: T? = null + + try { + resource = f(path) + } catch (e: Exception) { + logger.info("You potentially misspelled the path: $path") + logger.info("$e") + } + + if (resource == null) { + throw IllegalArgumentException("The Resource at path: $path could not be loaded") + } + + return resource + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/strategies/restriction/LowerBoundRestriction.kt b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/restriction/LowerBoundRestriction.kt new file mode 100644 index 0000000000000000000000000000000000000000..093ef3b100ae53babf0b873d6133a9571196bcdd --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/restriction/LowerBoundRestriction.kt @@ -0,0 +1,22 @@ +package theodolite.strategies.restriction + +import theodolite.util.Results +import theodolite.util.LoadDimension +import theodolite.util.Resource + +/** + * The Lower Bound Restriction sets the lower bound of the resources to be examined to the value + * needed to successfully execute the next smaller load. + * + * @param results Result object used as a basis to restrict the resources. + */ +class LowerBoundRestriction(results: Results) : RestrictionStrategy(results) { + override fun next(load: LoadDimension, resources: List<Resource>): List<Resource> { + val maxLoad: LoadDimension? = this.results.getMaxBenchmarkedLoad(load) + var lowerBound: Resource? = this.results.getMinRequiredInstances(maxLoad) + if(lowerBound == null) { + lowerBound = resources.get(0) + } + return resources.filter{x -> x.get() >= lowerBound.get()} + } +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/strategies/restriction/RestrictionStrategy.kt b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/restriction/RestrictionStrategy.kt new file mode 100644 index 0000000000000000000000000000000000000000..2479e9429ba2f82522a28f24ce1aba76816063ec --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/restriction/RestrictionStrategy.kt @@ -0,0 +1,20 @@ +package theodolite.strategies.restriction + +import theodolite.util.Results +import theodolite.util.LoadDimension +import theodolite.util.Resource + + +/** + * A "Restriction Strategy" restricts a list of resources based on the current results of all previously performed benchmarks. + */ +abstract class RestrictionStrategy(val results: Results) { + /** + * Next Restrict the given resource list for the given load based on the result object. + * + * @param load Load dimension for which a subset of resources are required. + * @param resources List of resources to be restricted. + * @return Returns a list containing only elements that have not been filtered out by the restriction (possibly empty). + */ + public abstract fun next(load: LoadDimension, resources: List<Resource>): List<Resource> +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/BinarySearch.kt b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/BinarySearch.kt new file mode 100644 index 0000000000000000000000000000000000000000..474389c9283b08c672186fe11504351db88995c6 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/BinarySearch.kt @@ -0,0 +1,47 @@ +package theodolite.strategies.searchstrategy + +import theodolite.execution.BenchmarkExecutor +import theodolite.util.LoadDimension +import theodolite.util.Resource +import java.lang.IllegalArgumentException + +/** + * Search for the smallest suitable resource with binary search. + * + * @param benchmarkExecutor Benchmark executor which runs the individual benchmarks. + */ +class BinarySearch(benchmarkExecutor: BenchmarkExecutor) : SearchStrategy(benchmarkExecutor) { + override fun findSuitableResource(load: LoadDimension, resources: List<Resource>): Resource? { + val result = binarySearch(load, resources, 0, resources.size - 1) + if( result == -1 ) { + return null; + } + return resources[result] + } + + private fun binarySearch (load: LoadDimension, resources: List<Resource>, lower: Int, upper: Int): Int { + if (lower > upper) { + throw IllegalArgumentException() + } + // special case: length == 1 or 2 + if (lower == upper) { + if (this.benchmarkExecutor.runExperiment(load, resources[lower])) return lower + else { + if (lower + 1 == resources.size) return - 1 + return lower + 1 + } + } else { + // apply binary search for a list with length > 2 and adjust upper and lower depending on the result for `resources[mid]` + val mid = (upper + lower) / 2 + if (this.benchmarkExecutor.runExperiment(load, resources[mid])) { + if (mid == lower) { + return lower + } + return binarySearch(load, resources, lower, mid - 1 ) + } else { + return binarySearch(load, resources, mid + 1 , upper) + } + } + } + +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/CompositeStrategy.kt b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/CompositeStrategy.kt new file mode 100644 index 0000000000000000000000000000000000000000..b44d5ea5e842f205c0a5ead7f5eba3887f5da591 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/CompositeStrategy.kt @@ -0,0 +1,18 @@ +package theodolite.strategies.searchstrategy + +import theodolite.execution.BenchmarkExecutor +import theodolite.strategies.restriction.RestrictionStrategy +import theodolite.util.LoadDimension +import theodolite.util.Resource +import theodolite.util.Results + +class CompositeStrategy(benchmarkExecutor: BenchmarkExecutor, private val searchStrategy: SearchStrategy, val restrictionStrategies: Set<RestrictionStrategy>) : SearchStrategy(benchmarkExecutor) { + + override fun findSuitableResource(load: LoadDimension, resources: List<Resource>): Resource? { + var restrictedResources = resources.toList() + for (strategy in this.restrictionStrategies) { + restrictedResources = restrictedResources.intersect(strategy.next(load, resources)).toList() + } + return this.searchStrategy.findSuitableResource(load, restrictedResources) + } +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/LinearSearch.kt b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/LinearSearch.kt new file mode 100644 index 0000000000000000000000000000000000000000..67a254989a9e56775787bf395934a3d5b299d694 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/LinearSearch.kt @@ -0,0 +1,16 @@ +package theodolite.strategies.searchstrategy + +import theodolite.execution.BenchmarkExecutor +import theodolite.util.LoadDimension +import theodolite.util.Resource +import theodolite.util.Results + +class LinearSearch(benchmarkExecutor: BenchmarkExecutor) : SearchStrategy(benchmarkExecutor) { + + override fun findSuitableResource(load: LoadDimension, resources: List<Resource>): Resource? { + for (res in resources) { + if (this.benchmarkExecutor.runExperiment(load, res)) return res + } + return null; + } +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/SearchStrategy.kt b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/SearchStrategy.kt new file mode 100644 index 0000000000000000000000000000000000000000..118d2f1625f704f4967a66e6c27a0a3fdaeb2895 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/strategies/searchstrategy/SearchStrategy.kt @@ -0,0 +1,17 @@ +package theodolite.strategies.searchstrategy + +import theodolite.execution.BenchmarkExecutor +import theodolite.util.LoadDimension +import theodolite.util.Resource +import theodolite.util.Results + +abstract class SearchStrategy(val benchmarkExecutor: BenchmarkExecutor) { + /** + * Find smallest suitable resource from the specified resource list for the given load. + * + * @param load Load to be tested. + * @param resources List of all possible resources. + * @return suitable resource for the specified load, or null if no suitable resource exists. + */ + abstract fun findSuitableResource(load: LoadDimension, resources: List<Resource>): Resource?; +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/util/AbstractBenchmark.kt b/theodolite-quarkus/src/main/kotlin/theodolite/util/AbstractBenchmark.kt new file mode 100644 index 0000000000000000000000000000000000000000..2411e1b8bb62bf40ed724737e76280603ab4f97b --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/util/AbstractBenchmark.kt @@ -0,0 +1,27 @@ +package theodolite.util + +abstract class AbstractBenchmark(val config: Config): Benchmark { + override fun start(load: LoadDimension, resources: Resource) { + this.clearClusterEnvironment() + this.initializeClusterEnvironment() + this.startSUT(resources) + this.startWorkloadGenerator(load) + } + + data class Config( + val clusterZookeeperConnectionString: String, + val clusterKafkaConnectionString: String, + val externalZookeeperConnectionString: String, + val externalKafkaConnectionString: String, + val schemaRegistryConnectionString: String, + val kafkaTopics: List<String>, + val kafkaReplication: Short, + val kafkaPartition: Int, + val ucDeploymentPath: String, + val ucServicePath: String, + val configMapPath: String, + val wgDeploymentPath: String, + val ucImageURL: String, + val wgImageURL: String + ) {} +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/util/Benchmark.kt b/theodolite-quarkus/src/main/kotlin/theodolite/util/Benchmark.kt new file mode 100644 index 0000000000000000000000000000000000000000..44f98da2ec34df02d2b010dc1844f1ffa57c0f50 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/util/Benchmark.kt @@ -0,0 +1,12 @@ +package theodolite.util + +interface Benchmark { + fun start(load: LoadDimension, resources: Resource) { + } + + fun initializeClusterEnvironment(); + fun clearClusterEnvironment(); + + fun startSUT(resources: Resource); + fun startWorkloadGenerator(load: LoadDimension) +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/util/Config.kt b/theodolite-quarkus/src/main/kotlin/theodolite/util/Config.kt new file mode 100644 index 0000000000000000000000000000000000000000..6fb680c0ae0dfeb80418f98093494386eda97fb4 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/util/Config.kt @@ -0,0 +1,15 @@ +package theodolite.util + +import theodolite.strategies.restriction.RestrictionStrategy +import theodolite.strategies.searchstrategy.CompositeStrategy +import theodolite.strategies.searchstrategy.SearchStrategy +import java.time.Duration + +data class Config( + val loads: List<LoadDimension>, + val resources: List<Resource>, + val compositeStrategy: CompositeStrategy, + val executionDuration: Duration +) { + +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/util/LoadDimension.kt b/theodolite-quarkus/src/main/kotlin/theodolite/util/LoadDimension.kt new file mode 100644 index 0000000000000000000000000000000000000000..17f1726902cfb61ee9162420ff781ae99abb55ea --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/util/LoadDimension.kt @@ -0,0 +1,7 @@ +package theodolite.util + +data class LoadDimension(private val number: Int) { + public fun get(): Int { + return this.number; + } +} diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/util/Resource.kt b/theodolite-quarkus/src/main/kotlin/theodolite/util/Resource.kt new file mode 100644 index 0000000000000000000000000000000000000000..bd3e1fb691084c07617b8ecc47d674443e38590b --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/util/Resource.kt @@ -0,0 +1,7 @@ +package theodolite.util + +data class Resource(private val number: Int) { + public fun get(): Int { + return this.number; + } +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/util/Results.kt b/theodolite-quarkus/src/main/kotlin/theodolite/util/Results.kt new file mode 100644 index 0000000000000000000000000000000000000000..fdd38896d664e6f793a1fffe7cc39a0c25d09067 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/util/Results.kt @@ -0,0 +1,50 @@ +package theodolite.util + +import theodolite.util.LoadDimension +import theodolite.util.Resource +import kotlin.math.exp + +class Results { + // load, instances + private val results: MutableMap<Pair<LoadDimension, Resource>, Boolean> = mutableMapOf() // multi map guava + + public fun setResult(experiment: Pair<LoadDimension, Resource>, successful: Boolean) { + this.results[experiment] = successful + } + + public fun getResult (experiment: Pair<LoadDimension, Resource>): Boolean? { + return this.results[experiment] + } + + public fun getMinRequiredInstances(load: LoadDimension?): Resource? { + if (this.results.isEmpty()) return Resource(Int.MIN_VALUE) + + var requiredInstances: Resource? = Resource(Int.MAX_VALUE) + for(experiment in results) { + if(experiment.key.first == load && experiment.value){ + if(requiredInstances == null) { + requiredInstances = experiment.key.second + }else if (experiment.key.second.get() < requiredInstances.get()) { + requiredInstances = experiment.key.second + } + } + } + return requiredInstances + } + + public fun getMaxBenchmarkedLoad(load: LoadDimension): LoadDimension? { + var maxBenchmarkedLoad: LoadDimension? = null; + for(experiment in results) { + if (experiment.value) { + if(experiment.key.first.get() <= load.get()) { + if (maxBenchmarkedLoad == null) { + maxBenchmarkedLoad = experiment.key.first + } else if (maxBenchmarkedLoad.get() < experiment.key.first.get()) { + maxBenchmarkedLoad = experiment.key.first + } + } + } + } + return maxBenchmarkedLoad + } +} \ No newline at end of file diff --git a/theodolite-quarkus/src/main/kotlin/theodolite/util/TestBenchmark.kt b/theodolite-quarkus/src/main/kotlin/theodolite/util/TestBenchmark.kt new file mode 100644 index 0000000000000000000000000000000000000000..0358ff1a61a66fe0dd0380a3b56dc2e8fddd3fa0 --- /dev/null +++ b/theodolite-quarkus/src/main/kotlin/theodolite/util/TestBenchmark.kt @@ -0,0 +1,37 @@ +package theodolite.util + +class TestBenchmark : AbstractBenchmark( + AbstractBenchmark.Config( + clusterZookeeperConnectionString = "", + clusterKafkaConnectionString = "", + externalZookeeperConnectionString = "", + externalKafkaConnectionString = "", + schemaRegistryConnectionString = "", + kafkaTopics = emptyList(), + kafkaReplication = 0, + kafkaPartition = 0, + ucServicePath = "", + ucDeploymentPath = "", + wgDeploymentPath = "", + configMapPath = "", + ucImageURL = "", + wgImageURL = "" + ) +) { + + override fun initializeClusterEnvironment() { + TODO("Not yet implemented") + } + + override fun clearClusterEnvironment() { + TODO("Not yet implemented") + } + + override fun startSUT(resources: Resource) { + TODO("Not yet implemented") + } + + override fun startWorkloadGenerator(load: LoadDimension) { + TODO("Not yet implemented") + } +} diff --git a/theodolite-quarkus/src/main/resources/META-INF/resources/index.html b/theodolite-quarkus/src/main/resources/META-INF/resources/index.html deleted file mode 100644 index 1a04c85cdee291a34d26b80d75ae97b037090d87..0000000000000000000000000000000000000000 --- a/theodolite-quarkus/src/main/resources/META-INF/resources/index.html +++ /dev/null @@ -1,242 +0,0 @@ -<!DOCTYPE html> -<html lang="en"> -<head> - <meta charset="UTF-8"> - <title>theodolite-quarkus - 1.0.0-SNAPSHOT</title> - <style> - h1, h2, h3, h4, h5, h6 { - margin-bottom: 0.5rem; - font-weight: 400; - line-height: 1.5; - } - - h1 { - font-size: 2.5rem; - } - - h2 { - font-size: 2rem - } - - h3 { - font-size: 1.75rem - } - - h4 { - font-size: 1.5rem - } - - h5 { - font-size: 1.25rem - } - - h6 { - font-size: 1rem - } - - .lead { - font-weight: 300; - font-size: 2rem; - } - - .banner { - font-size: 2.7rem; - margin: 0; - padding: 2rem 1rem; - background-color: #0d1c2c; - color: white; - } - - body { - margin: 0; - font-family: -apple-system, system-ui, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; - } - - code { - font-family: SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; - font-size: 87.5%; - color: #e83e8c; - word-break: break-word; - } - - .left-column { - padding: .75rem; - max-width: 75%; - min-width: 55%; - } - - .right-column { - padding: .75rem; - max-width: 25%; - } - - .container { - display: flex; - width: 100%; - } - - li { - margin: 0.75rem; - } - - .right-section { - margin-left: 1rem; - padding-left: 0.5rem; - } - - .right-section h3 { - padding-top: 0; - font-weight: 200; - } - - .right-section ul { - border-left: 0.3rem solid #71aeef; - list-style-type: none; - padding-left: 0; - } - - .examples { - display: flex; - flex-wrap: wrap; - margin: 20px 0 20px -40px; - } - - .example { - display: flex; - margin-left: 20px; - margin-bottom: 20px; - flex-direction: column; - width: 350px; - background-color: #205894; - color: white; - } - - .example code { - color: lightgrey; - } - - .example-header { - padding: 20px; - display: flex; - position: relative; - } - - .example-header h4 { - margin: 0; - font-size: 1.4rem; - flex-grow: 1; - line-height: 1.5; - } - - .example-description { - padding: 0 20px; - flex-grow: 1; - } - - .example-paths { - display: flex; - flex-direction: column; - } - - .example-paths a { - display: block; - background-color: transparent; - font-family: SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; - color: white; - padding: 10px; - text-decoration: none; - } - - .example-paths a:before { - content: '⇨'; - font-weight: bold; - font-size: 1.5rem; - margin: 20px; - } - - .example-paths a:hover { - background-color: #0d1c2c; - } - - .guide-link { - background-color: #71aeef; - position: absolute; - color: white; - text-decoration: none; - top: 0; - right: 0; - padding: 7px; - font-weight: bold; - } - - .guide-link:hover { - background-color: #0d1c2c; - } - </style> -</head> -<body> - -<div class="banner lead"> - Your new Cloud-Native application is ready! -</div> - -<div class="container"> - <div class="left-column"> - <p class="lead"> Congratulations, you have created a new Quarkus cloud application.</p> - - <h2>Why do you see this?</h2> - - <p>This page is served by Quarkus. The source is in - <code>src/main/resources/META-INF/resources/index.html</code>.</p> - - <h2>What can I do from here?</h2> - - <p>If not already done, run the application in <em>dev mode</em> using: <code>./gradlew quarkusDev</code>. - </p> - <ul> - <li>Play with your example code in <code>src/main/kotlin</code>: - <div class="examples"> -<div class="example"> - <div class="example-header"> - <h4>RESTEasy JAX-RS</h4> - <a href="https://quarkus.io/guides/rest-json" target="_blank" class="guide-link">Guide</a> - </div> - <div class="example-description"> - <p>A Hello World RESTEasy resource</p> - - </div> - <div class="example-paths"> - <a href="/hello-resteasy" class="path-link" target="_blank">GET /hello-resteasy</a> - </div> -</div> - - </div> - </li> - <li>Your static assets are located in <code>src/main/resources/META-INF/resources</code>.</li> - <li>Configure your application in <code>src/main/resources/application.properties</code>.</li> - </ul> - <h2>Do you like Quarkus?</h2> - <p>Go give it a star on <a href="https://github.com/quarkusio/quarkus">GitHub</a>.</p> - </div> - <div class="right-column"> - <div class="right-section"> - <h3>Application</h3> - <ul> - <li>GroupId: theodolite</li> - <li>ArtifactId: theodolite-quarkus</li> - <li>Version: 1.0.0-SNAPSHOT</li> - <li>Quarkus Version: 1.10.3.Final</li> - </ul> - </div> - <div class="right-section"> - <h3>Next steps</h3> - <ul> - <li><a href="https://quarkus.io/guides/gradle-tooling" target="_blank">Setup your IDE</a></li> - <li><a href="https://quarkus.io/guides/getting-started.html" target="_blank">Getting started</a></li> - <li><a href="https://quarkus.io" target="_blank">Quarkus Web Site</a></li> - </ul> - </div> - </div> -</div> -</body> -</html> \ No newline at end of file diff --git a/theodolite-quarkus/src/main/resources/yaml/aggregation-deployment.yaml b/theodolite-quarkus/src/main/resources/yaml/aggregation-deployment.yaml new file mode 100644 index 0000000000000000000000000000000000000000..07732ca1dd1e6b2b06f098dfb10a53d38e8d5cae --- /dev/null +++ b/theodolite-quarkus/src/main/resources/yaml/aggregation-deployment.yaml @@ -0,0 +1,55 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: titan-ccp-aggregation +spec: + selector: + matchLabels: + app: titan-ccp-aggregation + replicas: 1 + template: + metadata: + labels: + app: titan-ccp-aggregation + spec: + terminationGracePeriodSeconds: 0 + containers: + - name: uc-application + image: uc-app:latest + ports: + - containerPort: 5555 + name: jmx + env: + - name: KAFKA_BOOTSTRAP_SERVERS + value: "my-confluent-cp-kafka:9092" + - name: SCHEMA_REGISTRY_URL + value: "http://my-confluent-cp-schema-registry:8081" + - name: JAVA_OPTS + value: "-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=5555" + - name: COMMIT_INTERVAL_MS # Set as default for the applications + value: "100" + resources: + limits: + memory: 4Gi + cpu: 1000m + - name: prometheus-jmx-exporter + image: "solsson/kafka-prometheus-jmx-exporter@sha256:6f82e2b0464f50da8104acd7363fb9b995001ddff77d248379f8788e78946143" + command: + - java + - -XX:+UnlockExperimentalVMOptions + - -XX:+UseCGroupMemoryLimitForHeap + - -XX:MaxRAMFraction=1 + - -XshowSettings:vm + - -jar + - jmx_prometheus_httpserver.jar + - "5556" + - /etc/jmx-aggregation/jmx-kafka-prometheus.yml + ports: + - containerPort: 5556 + volumeMounts: + - name: jmx-config + mountPath: /etc/jmx-aggregation + volumes: + - name: jmx-config + configMap: + name: aggregation-jmx-configmap diff --git a/theodolite-quarkus/src/main/resources/yaml/aggregation-service.yaml b/theodolite-quarkus/src/main/resources/yaml/aggregation-service.yaml new file mode 100644 index 0000000000000000000000000000000000000000..916dd6677d60370b1d62e5d7e708c3ee966bda23 --- /dev/null +++ b/theodolite-quarkus/src/main/resources/yaml/aggregation-service.yaml @@ -0,0 +1,17 @@ +apiVersion: v1 +kind: Service +metadata: + name: titan-ccp-aggregation + labels: + app: titan-ccp-aggregation +spec: + #type: NodePort + selector: + app: titan-ccp-aggregation + ports: + - name: http + port: 80 + targetPort: 80 + protocol: TCP + - name: metrics + port: 9980 diff --git a/theodolite-quarkus/src/main/resources/yaml/jmx-configmap.yaml b/theodolite-quarkus/src/main/resources/yaml/jmx-configmap.yaml new file mode 100644 index 0000000000000000000000000000000000000000..78496a86b1242a89b9e844ead3e700fd0b9a9667 --- /dev/null +++ b/theodolite-quarkus/src/main/resources/yaml/jmx-configmap.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: aggregation-jmx-configmap +data: + jmx-kafka-prometheus.yml: |+ + jmxUrl: service:jmx:rmi:///jndi/rmi://localhost:5555/jmxrmi + lowercaseOutputName: true + lowercaseOutputLabelNames: true + ssl: false diff --git a/theodolite-quarkus/src/main/resources/yaml/service-monitor.yaml b/theodolite-quarkus/src/main/resources/yaml/service-monitor.yaml new file mode 100644 index 0000000000000000000000000000000000000000..4e7e758cacb5086305efa26292ddef2afc958096 --- /dev/null +++ b/theodolite-quarkus/src/main/resources/yaml/service-monitor.yaml @@ -0,0 +1,14 @@ +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + labels: + app: titan-ccp-aggregation + appScope: titan-ccp + name: titan-ccp-aggregation +spec: + selector: + matchLabels: + app: titan-ccp-aggregation + endpoints: + - port: metrics + interval: 10s diff --git a/theodolite-quarkus/src/main/resources/yaml/workloadGenerator.yaml b/theodolite-quarkus/src/main/resources/yaml/workloadGenerator.yaml new file mode 100644 index 0000000000000000000000000000000000000000..242ce5f2dc0eb9523a530710c5774f440627a8f6 --- /dev/null +++ b/theodolite-quarkus/src/main/resources/yaml/workloadGenerator.yaml @@ -0,0 +1,34 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: titan-ccp-load-generator +spec: + selector: + matchLabels: + app: titan-ccp-load-generator + replicas: 1 + template: + metadata: + labels: + app: titan-ccp-load-generator + spec: + terminationGracePeriodSeconds: 0 + containers: + - name: workload-generator + image: workload-generator:latest + env: + # Order need to be preserved for run_uc.py + - name: NUM_SENSORS + value: "25000" + - name: INSTANCES + value: "1" + - name: NUM_NESTED_GROUPS + value: "5" + - name: ZK_HOST + value: "my-confluent-cp-zookeeper" + - name: ZK_PORT + value: "2181" + - name: KAFKA_BOOTSTRAP_SERVERS + value: "my-confluent-cp-kafka:9092" + - name: SCHEMA_REGISTRY_URL + value: "http://my-confluent-cp-schema-registry:8081" \ No newline at end of file diff --git a/theodolite-quarkus/src/native-test/kotlin/theodolite/NativeGreetingResourceIT.kt b/theodolite-quarkus/src/native-test/kotlin/theodolite/NativeGreetingResourceIT.kt deleted file mode 100644 index 26e8900ffe1a17b71fc23e1327fa649421548e36..0000000000000000000000000000000000000000 --- a/theodolite-quarkus/src/native-test/kotlin/theodolite/NativeGreetingResourceIT.kt +++ /dev/null @@ -1,6 +0,0 @@ -package theodolite - -import io.quarkus.test.junit.NativeImageTest - -@NativeImageTest -class NativeGreetingResourceIT : GreetingResourceTest() \ No newline at end of file diff --git a/theodolite-quarkus/src/test/kotlin/theodolite/CompositeStrategyTest.kt b/theodolite-quarkus/src/test/kotlin/theodolite/CompositeStrategyTest.kt new file mode 100644 index 0000000000000000000000000000000000000000..fb98f11f212e40616e0907c2c61c71e7214c3d65 --- /dev/null +++ b/theodolite-quarkus/src/test/kotlin/theodolite/CompositeStrategyTest.kt @@ -0,0 +1,108 @@ +package theodolite + +import io.quarkus.test.junit.QuarkusTest +import org.junit.jupiter.api.Assertions.* +import org.junit.jupiter.api.Test +import theodolite.strategies.searchstrategy.LinearSearch +import theodolite.strategies.searchstrategy.BinarySearch +import theodolite.strategies.restriction.LowerBoundRestriction +import theodolite.strategies.searchstrategy.CompositeStrategy +import theodolite.execution.TestBenchmarkExecutorImpl +import theodolite.util.* + +@QuarkusTest +class CompositeStrategyTest { + + @Test + fun testEnd2EndLinearSearch() { + val mockResults = arrayOf( + arrayOf( true , true , true , true , true , true , true), + arrayOf( false, false, true , true , true , true , true), + arrayOf( false, false, true , true , true , true , true), + arrayOf( false, false, false, true , true , true , true), + arrayOf( false, false, false, false, true , true , true), + arrayOf( false, false, false, false, false, false, true), + arrayOf( false, false, false, false, false, false, false) + ) + val mockLoads: List<LoadDimension> = (0..6).map{number -> LoadDimension(number)} + val mockResources: List<Resource> = (0..6).map{number -> Resource(number)} + val results: Results = Results(); + val benchmark = TestBenchmark() + val benchmarkExecutor: TestBenchmarkExecutorImpl = TestBenchmarkExecutorImpl(mockResults, benchmark, results) + val linearSearch: LinearSearch = LinearSearch(benchmarkExecutor); + val lowerBoundRestriction: LowerBoundRestriction = LowerBoundRestriction(results); + val strategy: CompositeStrategy = CompositeStrategy(benchmarkExecutor, linearSearch, setOf(lowerBoundRestriction)) + + val actual: ArrayList<Resource?> = ArrayList<Resource?>() + val expected: ArrayList<Resource?> = ArrayList(listOf(0,2,2,3,4,6).map{ x -> Resource(x)}) + expected.add(null) + + for(load in mockLoads) { + actual.add(strategy.findSuitableResource(load, mockResources)) + } + + assertEquals(actual, expected) + } + + @Test + fun testEnd2EndBinarySearch() { + val mockResults = arrayOf( + arrayOf( true , true , true , true , true , true , true), + arrayOf( false, false, true , true , true , true , true), + arrayOf( false, false, true , true , true , true , true), + arrayOf( false, false, false, true , true , true , true), + arrayOf( false, false, false, false, true , true , true), + arrayOf( false, false, false, false, false, false, true), + arrayOf( false, false, false, false, false, false, false) + ) + val mockLoads: List<LoadDimension> = (0..6).map{number -> LoadDimension(number)} + val mockResources: List<Resource> = (0..6).map{number -> Resource(number)} + val results: Results = Results(); + val benchmark = TestBenchmark() + val benchmarkExecutorImpl: TestBenchmarkExecutorImpl = TestBenchmarkExecutorImpl(mockResults, benchmark, results) + val binarySearch: BinarySearch = BinarySearch(benchmarkExecutorImpl); + val lowerBoundRestriction: LowerBoundRestriction = LowerBoundRestriction(results); + val strategy: CompositeStrategy = CompositeStrategy(benchmarkExecutorImpl, binarySearch, setOf(lowerBoundRestriction)) + + val actual: ArrayList<Resource?> = ArrayList<Resource?>() + val expected: ArrayList<Resource?> = ArrayList(listOf(0,2,2,3,4,6).map{ x -> Resource(x)}) + expected.add(null) + + for(load in mockLoads) { + actual.add(strategy.findSuitableResource(load, mockResources)) + } + + assertEquals(actual, expected) + } + + @Test + fun testEnd2EndBinarySearch2() { + val mockResults = arrayOf( + arrayOf( true , true , true , true , true , true , true, true), + arrayOf( false, false, true , true , true , true , true, true), + arrayOf( false, false, true , true , true , true , true, true), + arrayOf( false, false, false, true , true , true , true, true), + arrayOf( false, false, false, false, true , true , true, true), + arrayOf( false, false, false, false, false, false, true, true), + arrayOf( false, false, false, false, false, false, false, true) + ) + val mockLoads: List<LoadDimension> = (0..6).map{number -> LoadDimension(number)} + val mockResources: List<Resource> = (0..7).map{number -> Resource(number)} + val results: Results = Results(); + val benchmark = TestBenchmark() + val benchmarkExecutor: TestBenchmarkExecutorImpl = TestBenchmarkExecutorImpl(mockResults, benchmark, results) + val binarySearch: BinarySearch = BinarySearch(benchmarkExecutor); + val lowerBoundRestriction: LowerBoundRestriction = LowerBoundRestriction(results); + val strategy: CompositeStrategy = CompositeStrategy(benchmarkExecutor, binarySearch, setOf(lowerBoundRestriction)) + + val actual: ArrayList<Resource?> = ArrayList<Resource?>() + val expected: ArrayList<Resource?> = ArrayList(listOf(0,2,2,3,4,6,7).map{ x -> Resource(x)}) + + for(load in mockLoads) { + actual.add(strategy.findSuitableResource(load, mockResources)) + } + + assertEquals(actual, expected) + } + +} \ No newline at end of file diff --git a/theodolite-quarkus/src/test/kotlin/theodolite/GreetingResourceTest.kt b/theodolite-quarkus/src/test/kotlin/theodolite/GreetingResourceTest.kt deleted file mode 100644 index 048093a7ec71389f7bfc59e22f7f4911ef485b94..0000000000000000000000000000000000000000 --- a/theodolite-quarkus/src/test/kotlin/theodolite/GreetingResourceTest.kt +++ /dev/null @@ -1,20 +0,0 @@ -package theodolite - -import io.quarkus.test.junit.QuarkusTest -import io.restassured.RestAssured.given -import org.hamcrest.CoreMatchers.`is` -import org.junit.jupiter.api.Test - -@QuarkusTest -class GreetingResourceTest { - - @Test - fun testHelloEndpoint() { - given() - .`when`().get("/hello-resteasy") - .then() - .statusCode(200) - .body(`is`("Hello RESTEasy")) - } - -} \ No newline at end of file