Is it reasonable to use Flink in a local machine

[Soheil Pourbafrani]

Hi,

My purpose is to generate data in a process and process those data in another process, something like stream processing, but all done in just one node, not cluster!

The rate of generating data is 12000 sample per second. I want the processing phase to done in parallel, So as Flink use node CPU cores to distribute the task on them and do the process, I'm wondering is it reasonable to use Flink in such a scenario for processing phase?

The node specification is 32GB RAM and 12 CPU cores.

[zhangminglei]

Hi, Soheil

You can set up several taskmanager processes in one node. I think it is reasonable to use Flink like this if you do not have enough machines.

Generate data we can think it is source operator and you can set the parallelism for this operator.
Process to those data we can think, like a flatmap function apply on a datastream. You can also set the parallelism for this operator.

Cheers
Minglei

> 在 2018年6月24日，下午5:36，Soheil Pourbafrani <[hidden email]> 写道：
>
> Hi,
>
> My purpose is to generate data in a process and process those data in another process, something like stream processing, but all done in just one node, not cluster!
>
> The rate of generating data is 12000 sample per second. I want the processing phase to done in parallel, So as Flink use node CPU cores to distribute the task on them and do the process, I'm wondering is it reasonable to use Flink in such a scenario for processing phase?
>
> The node specification is 32GB RAM and 12 CPU cores.