Skip to content

Migrate Beam benchmark implementation

Lorenz Boguhn requested to merge stu203404/theodolite:feature/147-add-beam-implementations into master

Co-authored-by: Jan Bensien stu128012@mail.uni-kiel.de

This merge request will fix #147 (closed). It is based on the existing implementation in the https://git.se.informatik.uni-kiel.de/stu128012/theodolite/-/tree/beam_current_theodolite fork but has to tackle the following requirements:

  • Migrate the individual benchmarks with their new enumeration
    • Migrate UC1
    • Migrate UC2 (f.k.a. UC3)
    • Migrate UC3 (f.k.a. UC4)
    • Migrate UC4 (f.k.a. UC2)
  • Migrate to Titan Control Center Commons 0.1.0 (Avro records and Schema Registry are already used)
  • Fix code quality issues if any
  • Support Docker builds (already available in the base branch)
  • Build and push Docker images in CI pipeline
  • Update Docker Compose files in docker-test
  • Share code among Beam runners (either create subprojects or dedicated Gradle tasks or dedicated Dockerfiles or environment variables or ...)
  • Clean up Gradle build files
    • Build Gradle plugins:
      • Build beam commons plugin
      • Build beam flink plugin
      • Build beam samza plugin
    • Check if same versions of dependecies are used (slf4, etc)
  • Provide corresponding Kubernetes benchmark definitions -> Moved to #281 (closed)
  • Solve #278 (closed)
  • Fix #293 (closed)
  • Solve #291 (not part of this MR)
  • Make samza configurable through environment variables
  • Resolve ./gradlew build --warning-mode all
  • @she Image names in Docker Compose files must be replaced when merging and pushing images

Depends on !188 (merged)

Edited by Sören Henning

Merge request reports