KeyedStream and keyedProcessFunction

Hi All,

I have a keyed data stream and calling a keyedProcessFunction after keyBy operation on datastream. Till now my understanding was, "For all different n- elements in keyed stream if their keys are same, same instance of keyedProcessFunction is called and for another elements with different keyes, another instances are invoked. But, I am observing some different , different behaviours. Can anyone please correct me on this?

Thanks,

Jaswin

[Tzu-Li (Gordon) Tai]

Hi,

Records with the same key will be processed by the same partition.
Note there isn't an instance of a keyed process function for each key.
There is a single instance per partition, and all keys that are distributed
to the same partition will get processed by the same keyed process function
instance.

Gordon



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[1048262223]

Hi

+1. Because there is no need to generate an instance for each key, flink just maintain the key collection in one instance. Imagine what would happen if the number of keys were unlimited.

Best,

Yichao Yang

------------------ Original ------------------

From: "Tzu-Li (Gordon) Tai"<[hidden email]>;

Date: Tue, Jun 9, 2020 06:33 PM

To: "user"<[hidden email]>;

Subject: Re: KeyedStream and keyedProcessFunction

Hi,

Records with the same key will be processed by the same partition.
Note there isn't an instance of a keyed process function for each key.
There is a single instance per partition, and all keys that are distributed
to the same partition will get processed by the same keyed process function
instance.

Gordon



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/