Just do a survey, how many people give up the storm and turn to Flink ?

[zhangminglei]

Hi, flink user

    

     I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much!

Thanks

zhangminglei

 

[Niels Basjes-2]

Hi,

The company I work for switched about 2 years ago because of these reasons AT THAT moment!

1) Storm doesn't run on Yarn

2) Storm doesn't support statefull processing components.

3) Storm has a bad Java api.

4) Storm is not fast enough.

Some of these things have changed over the last 2 years. But comparing the two at this moment would still let me choose Flink.

Niels

On 19 Aug 2017 10:28, "mingleizhang" <[hidden email]> wrote:

Hi, flink user

    

     I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much!

Thanks

zhangminglei

 

[zhangminglei]

Thanks Niels for your answer. Yes, storm does not support stateful processing components. So, I have to use something like Redis to store it's stateful.

At 2017-08-19 16:57:13, "Niels Basjes" <[hidden email]> wrote:

Hi,

The company I work for switched about 2 years ago because of these reasons AT THAT moment!

1) Storm doesn't run on Yarn

2) Storm doesn't support statefull processing components.

3) Storm has a bad Java api.

4) Storm is not fast enough.

Some of these things have changed over the last 2 years. But comparing the two at this moment would still let me choose Flink.

Niels

On 19 Aug 2017 10:28, "mingleizhang" <[hidden email]> wrote:

Hi, flink user

    

     I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much!

Thanks

zhangminglei

 

 

[Niels Basjes-2]

If you combine Storm with Redis for managing state you still do not have "exactly once" when a failure of a processing node occurs.

With Flink you do have that.

Niels

On 19 Aug 2017 12:00, "mingleizhang" <[hidden email]> wrote:

Thanks Niels for your answer. Yes, storm does not support stateful processing components. So, I have to use something like Redis to store it's stateful.

At 2017-08-19 16:57:13, "Niels Basjes" <[hidden email]> wrote:

Hi,

The company I work for switched about 2 years ago because of these reasons AT THAT moment!

1) Storm doesn't run on Yarn

2) Storm doesn't support statefull processing components.

3) Storm has a bad Java api.

4) Storm is not fast enough.

Some of these things have changed over the last 2 years. But comparing the two at this moment would still let me choose Flink.

Niels

On 19 Aug 2017 10:28, "mingleizhang" <[hidden email]> wrote:

Hi, flink user

    

     I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much!

Thanks

zhangminglei

 

 

[kaniska Mandal]

Storm supports 'exactly once' - http://storm.apache.org/releases/current/Trident-tutorial.html

Storm supports 'stateful processing' - http://storm.apache.org/releases/2.0.0-SNAPSHOT/State-checkpointing.html

Storm is blazing fast - when its properly configured to use distributed (via some client like redisson) and local cache. 

On Sun, Aug 20, 2017 at 11:27 AM, Niels Basjes <[hidden email]> wrote:

If you combine Storm with Redis for managing state you still do not have "exactly once" when a failure of a processing node occurs.

With Flink you do have that.

Niels

On 19 Aug 2017 12:00, "mingleizhang" <[hidden email]> wrote:

Thanks Niels for your answer. Yes, storm does not support stateful processing components. So, I have to use something like Redis to store it's stateful.

At 2017-08-19 16:57:13, "Niels Basjes" <[hidden email]> wrote:

Hi,

The company I work for switched about 2 years ago because of these reasons AT THAT moment!

1) Storm doesn't run on Yarn

2) Storm doesn't support statefull processing components.

3) Storm has a bad Java api.

4) Storm is not fast enough.

Some of these things have changed over the last 2 years. But comparing the two at this moment would still let me choose Flink.

Niels

On 19 Aug 2017 10:28, "mingleizhang" <[hidden email]> wrote:

Hi, flink user

    

     I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much!

Thanks

zhangminglei

 

 

[Renjie Liu]

We are running both systems. Old existing systems are still running on storm and new jobs are all running on flink.

On Mon, Aug 21, 2017 at 3:06 AM kaniska Mandal <[hidden email]> wrote:

Storm supports 'exactly once' - http://storm.apache.org/releases/current/Trident-tutorial.html

Storm supports 'stateful processing' - http://storm.apache.org/releases/2.0.0-SNAPSHOT/State-checkpointing.html

Storm is blazing fast - when its properly configured to use distributed (via some client like redisson) and local cache. 

On Sun, Aug 20, 2017 at 11:27 AM, Niels Basjes <[hidden email]> wrote:

If you combine Storm with Redis for managing state you still do not have "exactly once" when a failure of a processing node occurs.

With Flink you do have that.

Niels

On 19 Aug 2017 12:00, "mingleizhang" <[hidden email]> wrote:

Thanks Niels for your answer. Yes, storm does not support stateful processing components. So, I have to use something like Redis to store it's stateful.

At 2017-08-19 16:57:13, "Niels Basjes" <[hidden email]> wrote:

Hi,

The company I work for switched about 2 years ago because of these reasons AT THAT moment!

1) Storm doesn't run on Yarn

2) Storm doesn't support statefull processing components.

3) Storm has a bad Java api.

4) Storm is not fast enough.

Some of these things have changed over the last 2 years. But comparing the two at this moment would still let me choose Flink.

Niels

On 19 Aug 2017 10:28, "mingleizhang" <[hidden email]> wrote:

Hi, flink user

    

     I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much!

Thanks

zhangminglei

 

 

--

[zhangminglei]

Thanks, Renjie. Same here.

minglei

At 2017-08-21 11:19:48, "Renjie Liu" <[hidden email]> wrote:

We are running both systems. Old existing systems are still running on storm and new jobs are all running on flink.

On Mon, Aug 21, 2017 at 3:06 AM kaniska Mandal <[hidden email]> wrote:

Storm supports 'exactly once' - http://storm.apache.org/releases/current/Trident-tutorial.html

Storm supports 'stateful processing' - http://storm.apache.org/releases/2.0.0-SNAPSHOT/State-checkpointing.html

Storm is blazing fast - when its properly configured to use distributed (via some client like redisson) and local cache. 

On Sun, Aug 20, 2017 at 11:27 AM, Niels Basjes <[hidden email]> wrote:

If you combine Storm with Redis for managing state you still do not have "exactly once" when a failure of a processing node occurs.

With Flink you do have that.

Niels

On 19 Aug 2017 12:00, "mingleizhang" <[hidden email]> wrote:

Thanks Niels for your answer. Yes, storm does not support stateful processing components. So, I have to use something like Redis to store it's stateful.

At 2017-08-19 16:57:13, "Niels Basjes" <[hidden email]> wrote:

Hi,

The company I work for switched about 2 years ago because of these reasons AT THAT moment!

1) Storm doesn't run on Yarn

2) Storm doesn't support statefull processing components.

3) Storm has a bad Java api.

4) Storm is not fast enough.

Some of these things have changed over the last 2 years. But comparing the two at this moment would still let me choose Flink.

Niels

On 19 Aug 2017 10:28, "mingleizhang" <[hidden email]> wrote:

Hi, flink user

    

     I just want to do a survey as the subject said. How many of you that used to use storm as your real-time computing framework, but now, turn to Flink instead. And why ? Could you tell me ? Thank you very much!

Thanks

zhangminglei

 

 

--