How to achieve co-location constraints in Flink 1.9.1

[Si-li Liu]

Hi

Flink only have slotSharingGroup API on DataStream class, I can't find any public API to achieve co-location constraints. Could anyone provide me an example?

Another question is that if I use slotSharing group, Flink will schedule two sub tasks to same slot is possible. I think such schedule will always success because two tasks run in one slot just decrease the resource usage. Could anyone provide me an example when slotSharing group schedule fail ?

Thanks

--

[Arvid Heise-3]

Hi Si-li,

slot sharing is indeed the way that Flink performs co-location. It's actually enabled by default. It should work as expected if upstream and downstream operators have the same parallelism.

In certain cases, two operators can be even chained into one task where no serialization/network traffic happens.

Btw, I also responded to your initial mail to point out an alternative solution.

On Thu, Nov 19, 2020 at 12:52 PM Si-li Liu <[hidden email]> wrote:

Hi

Flink only have slotSharingGroup API on DataStream class, I can't find any public API to achieve co-location constraints. Could anyone provide me an example?

Another question is that if I use slotSharing group, Flink will schedule two sub tasks to same slot is possible. I think such schedule will always success because two tasks run in one slot just decrease the resource usage. Could anyone provide me an example when slotSharing group schedule fail ?

Thanks

--

Best regards

Sili Liu

--