Skip to content

Open Points for Beam benchmark implementations

Remaining things from of #147 (closed) and !187 (merged)

-- TODO: --

  • Clear usage of samza configuration. Is this needed with multiple instances for the shared zookeeper state?
  • Clear usage of samza pipeline option `--maxSourceParallelism=$MAX_SOURCE_PARALLELISM**
  • Clear usage of flink pipeline option --parallelism=$PARALLELISM
  • Clear usage of flink pipeline option --flinkMaster=flink-jobmanager:8081
  • Clear usage of --fasterCopy
  • Clear usage of --enableMetrics=false
  • Test implementations in more detail: eg. multiple instances, ...
  • Check when a TimestampPolicie is needed.
  • Clear usage of Uc2ApplicationBeamNoFeedback.java (old use case 2 / new use case 4).
  • Configure Logger to not always log the kafka Subscription State.
  • Maybe shorten Uc4Pipeline.
  • ConfigurationKeys not optimal
  • ENV MAX_SOURCE_PARALLELISM=1024 as default
Edited by Sören Henning