Open Points for Beam benchmark implementations

changed the description

Regarding the TimestampPolicy: I think it is needed for the timestamp-sensitive benchmarks (UC2-4). I think, we can also use it for all benchmarks.

The TimestampPolicy was not used in uc2(old uc4). Im in the progress of adding it.

Just for the sake of completeness: UC2 is old UC3. Old UC4 is now UC3. Sorry for making this so complicated.

changed the description

mentioned in merge request !187 (merged)

Clear usage of samza pipeline option --maxSourceParallelism=$MAX_SOURCE_PARALLELISM

-> See Jan's thesis

Clear usage of flink pipeline option --parallelism=$PARALLELISM

See Flink implementations

Clear usage of flink pipeline option --flinkMaster=flink-jobmanager:8081

If needed (probably yes), should be configurable via environment variable

Test implementations in more detail: eg. multiple instances, ...

should be part of !187 (merged)

Check when a TimestampPolicy is needed.

Try using TimestampPolicy for all benchmarks

changed milestone to %0.7.0

Information about the pipeline options: Flink Runner: Flink Config

--disableMetrics to disable the internal metrics
--fasterCopy to disable unneeded serialization and deserialization steps.
--parallelism Degree of parallelism for the workers. Default:1 (set via flink configuration in docker-compose)
--flinkMaster=flink-jobmanager:8081 to connect to the jobmanager (not needed in docker-compose)
-stateBackend=rocksdb --stateBackendStoragePath=file:///data/flink/checkpoints for persistenz.

Samza:

--samzaExecutionEnvironment=STANDALONE to use kafka and zookeper instead of Hadoop for coordination
--maxSourceParallelism=$MAX_SOURCE_PARALLELISM The maximum parallelism allowed for any data source.
--enableMetrics=false to disable internal metrics.

Regarding --faster-copy, I think it's okay to enable it. You find an evaluation of this configuration in Jan's thesis. Also regarding the metrics you find some information in Jan's thesis. However, it's important whether these are beam metrics or Flink/Samza metrics or both.
If we have to set parallelism is still open, right?
Why is the Flink Master address not needed in docker-compose? How do the taskmanagers coordinate?
Regarding the state backend in Flink: We should use the same aas for the native Flink benchmarks. However, we should support both backends and document this somewhere.
For the --maxSourceParallelism argument, we should simply set a high value because of:

To set each Kafka partition in a split, we can set a large “maxSourceParallelism” value which is the upper bound of the number of splits. -- https://samza.apache.org/startup/code-examples/latest/beam.html

In the docs, they set --maxSourceParallelism=1024.
--samzaExecutionEnvironment=STANDALONE is what we want.

I added --fastercopy and disabled the metrics. They are disabled for both.
In Flink: If we want to set parallelism depends on the use case (testing, Benchmarking). This can be done by parallelism.default: 1 in docker-compose
The Flink Master address is set in the docker-compose file with jobmanager.rpc.address: benchmark-jobmanager. I have added it for exemplary reasons just to uc4-beam-flink until now. I think we should discuss the docker-compose file.
--maxSourceParallelism will be set to --maxSourceParallelism=$maxSourceParallelism
Set in the dockerfiles.

Can we make maxSourceParallelism configurable with 1024 as default? This should be possible with

ENV MAX_SOURCE_PARALLELISM=1024

in the Dockerfile.

Testet and it works. The values of the docker build serve as defaults. They will be over-riden if they are set elsewhere.

marked the checklist item Clear usage of samza configuration. Is this needed with multiple instances for the shared zookeeper state? as completed

marked the checklist item Clear usage of flink pipeline option --parallelism=$PARALLELISM as completed

marked the checklist item Clear usage of flink pipeline option --flinkMaster=flink-jobmanager:8081 as completed

marked the checklist item Clear usage of samza pipeline option --maxSourceParallelism=$MAX_SOURCE_PARALLELISM as completed

marked the checklist item Check when a TimestampPolicie is needed. as completed

marked the checklist item ConfigurationKeys not optimal as completed

changed the description

The usage of Uc2ApplicationBeamNoFeedback.java is not documented or part of Jan's thesis. It is an old artefact of the development process. Therefore we can delete it.

marked the checklist item Clear usage of Uc2ApplicationBeamNoFeedback.java (old use case 2 / new use case 4). as completed

Open Points for Beam benchmark implementations

Designs

Child items ...

Activity