docs/README.md

@@ -36,8 +36,8 @@ We use the [crdoc](https://github.com/fybrik/crdoc) tool to generate the API ref
 crdoc --resources ../theodolite/crd/ --template api-reference/crds.tmpl  --output api-reference/crds.md
 ```
 
-With the following command, crdoc is installed and executed in Docker:
+With the following command, crdoc is executed in Docker:
 
 ```sh
-docker run --rm -v "`pwd`/../theodolite/crd/":/crd -v "`pwd`/api-reference":/api-reference golang sh -c "go install fybrik.io/crdoc@latest && crdoc --resources /crd/ --template /api-reference/crds.tmpl --output /api-reference/crds.md"
+docker run --rm -v "`pwd`/../theodolite/crd/":/crd -u $UID -v "`pwd`/api-reference":/api-reference ghcr.io/fybrik/crdoc:0.6.0 --resources /crd/ --template /api-reference/crds.tmpl --output /api-reference/crds.md
 ```

docs/index.md

@@ -5,7 +5,7 @@ nav_order: 1
 permalink: /
 ---
 
-![Theodolite](assets/theodolite-stacked-transparent.svg){: .d-block .mx-auto .mb-8 .theodolite-logo }
+![Theodolite](assets/logo/theodolite-stacked-transparent.svg){: .d-block .mx-auto .mb-8 .theodolite-logo }
 
 
 Theodolite is a framework for benchmarking the horizontal and vertical scalability of cloud-native applications.
@@ -16,78 +16,3 @@ Theodolite is a framework for benchmarking the horizontal and vertical scalabili
 {: .text-center }
 
 ---
-
-
-
-## Kubernetes Operator for Cloud-Native Benchmarking Experience
-
-With Theodolite, you run and manage scalability benchmarks via the Kubernetes API.
-<!--After installing Theodolite, -->
-Benchmarks and their executions are purely declaratively described in Kubernetes YAML files, which can be submitted to Kubernetes via `kubectl`.
-This way you get the full experience of Kubernetes tooling. <!-- including auto-completion, schema validation and fehler behandlung-->
-Theodolite itself runs as an operator in your cluster and watches for newly scheduled benchmark executions.
-After installation, Theodolite runs continuously inside your Kubernetes cluster and watches for newly scheduled benchmark executions.
-
-<!-- Integration with established cloud-native tooling such as Prometheus>
-
-<!-- Designers of scalability benchmarks are freed from ... -->
-<!-- Benchmarks -->
-
-<!--
-* ... for cloud-native experience
-* Use kubectl to run benchmarks
-* runs in background on your Kubernetes cluster
-* Benchmarks and their executions are defined as
-  * CRDs/YAMl files, managed by the Kubernetes API
--->
-
-## Grounded on Established Scalability Definitions and Benchmarking Best Practices
-
-<!-- Theodolite measures scalability by running individual experiments for different load intensities and resource amounts -->
-Theodolite measures scalability by running a system under test (SUT) for different load intensities and resource amounts.
-For each combination of generated load and provisioned resources, Theodolite checks whether certain service-level objects (SLOs) are fulfilled.
-This way, Theodolite empirically derives how resource demand evolves with increasing load.
-
-Each load-resource-combination is tested individually for a configurable amount of time and with multiple repetitions to ensure statistically grounded results.
-To balance statistical grounding and time-efficient execution, Theodolite provides heuristics to shrink the search space.
- 
-For best applicability, Theodolite does not make any restrictions on the type of SUT, load, resources or SLOs. Everything that can be deployed in Kubernetes can also be benchmarked with Theodolite.
-
-<!--* Cloud-native benchmarking by using CRDs
-* Operator based (easy to use)
-* Arbitrary load and resource types
-* statistical grounding + time-efficient execution-->
-
-## Ready-to-use Benchmarks for Event-Driven Microservices
-
-Theodolite comes with 4 benchmarks for big data stream processing frameworks. 
- inspired by typical use for big data stream processing within microservices.
-Implementations: Kstreams, flink + 
-Load Generator, Metrics (SLO checkers), implementations
-
-
-<!--
-## Theodolite Benchmarking Tool
-
-Theodolite aims to benchmark scalability of stream processing engines for real use cases. Microservices that apply stream processing techniques are usually deployed in elastic cloud environments. Hence, Theodolite's cloud-native benchmarking framework deploys its components in a cloud environment, orchestrated by Kubernetes. It is recommended to install Theodolite with the package manager Helm. The Theodolite Helm chart along with instructions how to install it can be found in the [`helm`](helm) directory.
-
-## Theodolite Analysis Tools
-
-Theodolite's benchmarking method maps load intensities to the resource amounts that are required for processing them. A plot showing how resource demand evolves with an increasing load allows to draw conclusions about the scalability of a stream processing engine or its deployment. Theodolite provides Jupyter notebooks for creating such plots based on benchmarking results from the execution framework. More information can be found in [Theodolite analysis tool](analysis).
-
-## Theodolite Benchmarks
-
-Theodolite comes with 4 application benchmarks, which are based on typical use cases for stream processing within microservices. For each benchmark, a corresponding load generator is provided. Currently, this repository provides benchmark implementations for Apache Kafka Streams and Apache Flink. The benchmark sources can be found in [Thedolite benchmarks](theodolite-benchmarks).
--->
-
-## Free and Open Source Research Software
-
-Theodolite is entwickelt as free and open source research software at Kiel University's [Software Engineering Group](https://www.se.informatik.uni-kiel.de).
-
-Feel free to use, extend or modify for your research or ... engineering.
-
-You find a list of publications associated with Theodolite.
-
-If you use Theodolite in your research, please cite as:
-
-> Sören Henning and Wilhelm Hasselbring. (2021). Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures. Big Data Research, Volume 25. DOI: [10.1016/j.bdr.2021.100209](https://doi.org/10.1016/j.bdr.2021.100209). arXiv:[2009.00304](https://arxiv.org/abs/2009.00304).

theodolite-benchmarks/README.md

+# Theodolite Benchmarks
+
+Theodolite comes with a set of 4 benchmarks for event-driven microservices, which are implemented with Kafka Streams
+and Apache Flink. The benchmarks are based on typical use cases for stream processing and named: UC1, UC2, UC3 and UC4.
+
+## Building and packaging the benchmarks
+
+All benchmarks can be built with:
+
+```sh
+./gradlew build
+```
+
+This produces `.tar` files in `<benchmark-impl>/build/distribution`, where `<benchmark-impl>` is for example
+`uc1-kstreams`.
+
+## Building Docker images of the benchmarks
+
+Each benchmark implementation directory contains a Dockerfile. To build an image (in this case of `uc1-kstreams`), run:
+
+```sh
+docker build -t theodolite-uc1-kstreams ./uc1-kstreams
+```

theodolite-benchmarks/buildSrc/src/main/groovy/theodolite.flink.gradle

@@ -6,9 +6,6 @@ plugins {
 
 applicationDefaultJvmArgs = ["-Dlog4j.configuration=log4j.properties"]
 
-
-run.classpath = sourceSets.main.runtimeClasspath
-
 jar {
     manifest {
         attributes 'Built-By': System.getProperty('user.name'),
@@ -17,7 +14,6 @@ jar {
 }
 
 shadowJar {
-    configurations = [project.configurations.compile]
     zip64 true
 }
 
@@ -43,24 +39,22 @@ dependencies {
     implementation('org.industrial-devops:titan-ccp-common:0.1.0-flink-ready-SNAPSHOT') { changing = true }
     implementation('org.industrial-devops:titan-ccp-common-kafka:0.1.0-SNAPSHOT') { changing = true }
 
-    // These dependencies are used internally, and not exposed to consumers on their own compile classpath.
     implementation 'org.apache.kafka:kafka-clients:2.2.0'
-    implementation 'com.google.guava:guava:24.1-jre'
+    implementation 'com.google.guava:guava:30.1-jre'
     implementation 'com.google.code.gson:gson:2.8.2'
     implementation 'org.slf4j:slf4j-simple:1.6.1'
     implementation project(':flink-commons')
 
-    //compile group: 'org.apache.kafka', name: 'kafka-clients', version: "2.2.0"
     implementation "org.apache.flink:flink-java:${flinkVersion}"
     implementation "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"
-    implementation "org.apache.flink:flink-table-api-java-bridge_${scalaBinaryVersion}:${flinkVersion}"
-    implementation "org.apache.flink:flink-table-planner-blink_${scalaBinaryVersion}:${flinkVersion}"
     implementation "org.apache.flink:flink-connector-kafka_${scalaBinaryVersion}:${flinkVersion}"
     implementation "org.apache.flink:flink-avro:${flinkVersion}"
     implementation "org.apache.flink:flink-avro-confluent-registry:${flinkVersion}"
-    implementation "org.apache.flink:flink-runtime-web_${scalaBinaryVersion}:${flinkVersion}" // For debugging
     implementation "org.apache.flink:flink-statebackend-rocksdb_${scalaBinaryVersion}:${flinkVersion}"
     implementation "org.apache.flink:flink-metrics-prometheus_${scalaBinaryVersion}:${flinkVersion}"
+    implementation "org.apache.flink:flink-runtime-web_${scalaBinaryVersion}:${flinkVersion}" // For debugging
+    
+    
 
     // Use JUnit test framework
     testImplementation 'junit:junit:4.12'

theodolite-benchmarks/docker-test/uc1-flink-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1
@@ -44,8 +45,8 @@ services:
       NUM_SENSORS: 10
   benchmark-jobmanager:
     image: ghcr.io/cau-se/theodolite-uc1-flink:latest
-    ports:
-      - "8080:8081"
+    #ports:
+    #  - "8080:8081"
     command: standalone-job --job-classname theodolite.uc1.application.HistoryServiceFlinkJob
     environment:
       - KAFKA_BOOTSTRAP_SERVERS=kafka:9092

theodolite-benchmarks/docker-test/uc1-kstreams-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1

theodolite-benchmarks/docker-test/uc2-flink-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1
@@ -44,8 +45,8 @@ services:
       NUM_SENSORS: 10  
   benchmark-jobmanager:
     image: ghcr.io/cau-se/theodolite-uc2-flink:latest
-    ports:
-      - "8080:8081"
+    #ports:
+    #  - "8080:8081"
     command: standalone-job --job-classname theodolite.uc2.application.HistoryServiceFlinkJob
     environment:
       - KAFKA_BOOTSTRAP_SERVERS=kafka:9092

theodolite-benchmarks/docker-test/uc2-kstreams-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1

theodolite-benchmarks/docker-test/uc3-flink-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1
@@ -44,8 +45,8 @@ services:
       NUM_SENSORS: 10
   benchmark-jobmanager:
     image: ghcr.io/cau-se/theodolite-uc3-flink:latest
-    ports:
-      - "8080:8081"
+    #ports:
+    #  - "8080:8081"
     command: standalone-job --job-classname theodolite.uc3.application.HistoryServiceFlinkJob
     environment:
       - KAFKA_BOOTSTRAP_SERVERS=kafka:9092

theodolite-benchmarks/docker-test/uc3-kstreams-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1

theodolite-benchmarks/docker-test/uc4-flink-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1
@@ -45,8 +46,8 @@ services:
       NUM_NESTED_GROUPS: 4
   benchmark-jobmanager:
     image: ghcr.io/cau-se/theodolite-uc4-flink:latest
-    ports:
-      - "8080:8081"
+    #ports:
+    #  - "8080:8081"
     command: standalone-job --job-classname theodolite.uc4.application.AggregationServiceFlinkJob
     environment:
       - KAFKA_BOOTSTRAP_SERVERS=kafka:9092

theodolite-benchmarks/docker-test/uc4-kstreams-docker-compose/docker-compose.yml

@@ -18,6 +18,7 @@ services:
       KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
       KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 30000
+      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
       KAFKA_CREATE_TOPICS: "input:3:1,output:3:1,configuration:3:1,aggregation-feedback:3:1"
   schema-registry:
     image: confluentinc/cp-schema-registry:5.3.1

theodolite-benchmarks/flink-commons/build.gradle

@@ -20,17 +20,13 @@ repositories {
 dependencies {
     // Special version required because of https://issues.apache.org/jira/browse/FLINK-13703
     implementation('org.industrial-devops:titan-ccp-common:0.1.0-flink-ready-SNAPSHOT') { changing = true }
-    implementation('org.industrial-devops:titan-ccp-common-kafka:0.1.0-SNAPSHOT') { changing = true }
     implementation 'com.google.guava:guava:30.1-jre'
 
-    implementation "org.apache.flink:flink-java:${flinkVersion}"    
+    implementation "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"
     implementation "org.apache.flink:flink-connector-kafka_${scalaBinaryVersion}:${flinkVersion}"
-    implementation "org.apache.flink:flink-avro:${flinkVersion}"
     implementation "org.apache.flink:flink-avro-confluent-registry:${flinkVersion}"
-    implementation "org.apache.flink:flink-runtime-web_${scalaBinaryVersion}:${flinkVersion}" // For debugging
     implementation "org.apache.flink:flink-statebackend-rocksdb_${scalaBinaryVersion}:${flinkVersion}"
-    implementation "org.apache.flink:flink-metrics-prometheus_${scalaBinaryVersion}:${flinkVersion}"
-    
+
     // Use JUnit test framework
     testImplementation 'junit:junit:4.12'
   }

theodolite-benchmarks/uc2-flink/build.gradle

@@ -2,16 +2,4 @@ plugins {
   id 'theodolite.flink'
 }
 
-allprojects {
-	repositories {
-    	maven {
-    		url 'https://packages.confluent.io/maven/'
-    	}
-	}
-}
-
-dependencies {
-    compile('org.industrial-devops:titan-ccp-common-kafka:0.1.0-SNAPSHOT')
-}
-
 mainClassName = "theodolite.uc2.application.HistoryServiceFlinkJob"

theodolite-benchmarks/uc3-flink/build.gradle

@@ -2,16 +2,4 @@ plugins {
   id 'theodolite.flink'
 }
 
-allprojects {
-	repositories {
-    	maven {
-    		url 'https://packages.confluent.io/maven/'
-    	}
-	}
-}
-
-dependencies {
-    compile('org.industrial-devops:titan-ccp-common-kafka:0.1.0-SNAPSHOT')
-}
-
 mainClassName = "theodolite.uc3.application.HistoryServiceFlinkJob"

theodolite/README.md

@@ -29,7 +29,7 @@ The application can be packaged using:
 ./gradlew build
 ```
 
-It produces the `theodolite-1.0.0-SNAPSHOT-runner.jar` file in the `/build` directory. Be aware that it’s not
+It produces the `theodolite-0.6.0-SNAPSHOT-runner.jar` file in the `/build` directory. Be aware that it’s not
 an _über-jar_ as the dependencies are copied into the `build/lib` directory.
 
 If you want to build an _über-jar_, execute the following command:
@@ -38,7 +38,7 @@ If you want to build an _über-jar_, execute the following command:
 ./gradlew build -Dquarkus.package.type=uber-jar
 ```
 
-The application is now runnable using `java -jar build/theodolite-1.0.0-SNAPSHOT-runner.jar`.
+The application is now runnable using `java -jar build/theodolite-0.6.0-SNAPSHOT-runner.jar`.
 
 ## Creating a native executable
 
@@ -57,7 +57,7 @@ Or, if you don't have GraalVM installed, you can run the native executable build
 ```
 
 You can then execute your native executable with:
-```./build/theodolite-1.0.0-SNAPSHOT-runner```
+```./build/theodolite-0.6.0-SNAPSHOT-runner```
 
 If you want to learn more about building native executables, please consult <https://quarkus.io/guides/gradle-tooling>.