Benchmarking streaming frameworks

[Giselle van Dongen]

Dear users of Streaming Technologies,

As a PhD student in big data analytics, I am currently in the process of
compiling a list of benchmarks (to test multiple streaming frameworks) in
order to create an expanded benchmarking suite. The benchmark suite is being
developed as a part of my current work at Ghent University.

The included frameworks at this time are, in no particular order, Spark,
Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
previous work or relevant benchmarks would be appreciated.

Best regards,
Giselle van Dongen

[Christophe Salperwyck]

Good idea! You could test Akka streams too. 

Lots of documents exist:

https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at

https://github.com/yahoo/streaming-benchmarks

Cheers,

Christophe

2017-03-23 11:09 GMT+01:00 Giselle van Dongen <[hidden email]>:

Dear users of Streaming Technologies,

As a PhD student in big data analytics, I am currently in the process of
compiling a list of benchmarks (to test multiple streaming frameworks) in
order to create an expanded benchmarking suite. The benchmark suite is being
developed as a part of my current work at Ghent University.

The included frameworks at this time are, in no particular order, Spark,
Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
previous work or relevant benchmarks would be appreciated.

Best regards,
Giselle van Dongen

[Felix Neutatz]

Hi,

our team already created a benchmark framework for batch processing (including MR,Yarn, Spark, Flink), maybe you like to extend it for streaming: https://github.com/peelframework/peel

Best regards,

Felix

On Mar 23, 2017 11:51, "Christophe Salperwyck" <[hidden email]> wrote:

Good idea! You could test Akka streams too. 

Lots of documents exist:

https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at

https://github.com/yahoo/streaming-benchmarks

Cheers,

Christophe

2017-03-23 11:09 GMT+01:00 Giselle van Dongen <[hidden email]>:

Dear users of Streaming Technologies,

As a PhD student in big data analytics, I am currently in the process of
compiling a list of benchmarks (to test multiple streaming frameworks) in
order to create an expanded benchmarking suite. The benchmark suite is being
developed as a part of my current work at Ghent University.

The included frameworks at this time are, in no particular order, Spark,
Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
previous work or relevant benchmarks would be appreciated.

Best regards,
Giselle van Dongen

[Michael Noll]

A recent one is "Analytics on Fast Data: Main-Memory Database Systems versus Modern Streaming Systems" (http://db.in.tum.de/~kipf/papers/fastdata.pdf)

For the record, the paper above doesn't yet cover/realize that, nowadays, the Kafka project includes native stream processing capabilities aka the Kafka Streams API.

-Michael

On Thu, Mar 23, 2017 at 2:00 PM, Felix Neutatz <[hidden email]> wrote:

Hi,

our team already created a benchmark framework for batch processing (including MR,Yarn, Spark, Flink), maybe you like to extend it for streaming: https://github.com/peelframework/peel

Best regards,

Felix

On Mar 23, 2017 11:51, "Christophe Salperwyck" <[hidden email]> wrote:

Good idea! You could test Akka streams too. 

Lots of documents exist:

https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at

https://github.com/yahoo/streaming-benchmarks

Cheers,

Christophe

2017-03-23 11:09 GMT+01:00 Giselle van Dongen <[hidden email]>:

Dear users of Streaming Technologies,

As a PhD student in big data analytics, I am currently in the process of
compiling a list of benchmarks (to test multiple streaming frameworks) in
order to create an expanded benchmarking suite. The benchmark suite is being
developed as a part of my current work at Ghent University.

The included frameworks at this time are, in no particular order, Spark,
Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
previous work or relevant benchmarks would be appreciated.

Best regards,
Giselle van Dongen

[Dominik Safaric]

Dear Giselle,

Various stream processing engines benchmarks already exist. Here are only a few of them I believe are worthwhile mentioning:

• http://ieeexplore.ieee.org/document/7530084/
• https://www.usenix.org/node/188989
• https://pdfs.semanticscholar.org/c82f/170fbc837291d94dc0a18f0223d182144339.pdf
• https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F14/projects/reports/project11_report_ver3.pdf
• https://hal.inria.fr/hal-01347638/document

Regards,

Dominik

On 23 Mar 2017, at 11:09, Giselle van Dongen <[hidden email]> wrote:

Dear users of Streaming Technologies,

As a PhD student in big data analytics, I am currently in the process of
compiling a list of benchmarks (to test multiple streaming frameworks) in
order to create an expanded benchmarking suite. The benchmark suite is being
developed as a part of my current work at Ghent University.

The included frameworks at this time are, in no particular order, Spark,
Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
previous work or relevant benchmarks would be appreciated.

Best regards,
Giselle van Dongen