The Role of TimerService in ProcessFunction

[chiggi_dev]

Hi,

Currently, both ProcessFunction and KeyedProcessFunction (and their CoProcess counterparts) expose the Context and TimerService in the processElement() method. However, if we use the TimerService in non keyed context, it gives a runtime error. 

I am a bit confused about these APIs. Is there any specific reason for exposing TimerService in non-keyed context especially if it cant be used without keyed stream?

Any leads are much appreciated.

Thanks,

Chirag

[Dawid Wysakowicz-2]

Hi Chirag,

I agree it might be a little bit confusing.

Let me try to explain the reasoning. To do that I'll first try to rephrase the reasoning from FLINK-8560 for introducing the KeyedProcessFunction. It was introduced so that users have a typed access to the current key via Context and OnTimerContext. This is actually the only difference between the two function.

Somewhat as a consequence of the above, the KeyedProcessFunction can be used solely on a keyed stream, however ProcessFunction can be used in both. That was actually the only way to use a ProcessFunction on a keyed stream prior to introducing the KeyedProcessFunction. If you don't need access to the current key you should be fine with using the ProcessFunction on a keyed stream and there you can use the TimerService. It is advised to use a KeyedProcessFunction on a keyed stream, however for backwards compatibility the old behaviour has been kept.

Hope that it clarifies the things a bit.

Best,

Dawid

On 17/03/2021 07:47, Chirag Dewan wrote:

Hi,

Currently, both ProcessFunction and KeyedProcessFunction (and their CoProcess counterparts) expose the Context and TimerService in the processElement() method. However, if we use the TimerService in non keyed context, it gives a runtime error. 

I am a bit confused about these APIs. Is there any specific reason for exposing TimerService in non-keyed context especially if it cant be used without keyed stream?

Any leads are much appreciated.

Thanks,

Chirag

[chiggi_dev]

Thanks for the clarification Dawid. Resolves my confusion.

Sent from Yahoo Mail on Android

On Fri, 19 Mar 2021 at 2:41 pm, Dawid Wysakowicz

<[hidden email]> wrote:

Hi Chirag,

I agree it might be a little bit confusing.

Let me try to explain the reasoning. To do that I'll first try to rephrase the reasoning from FLINK-8560 for introducing the KeyedProcessFunction. It was introduced so that users have a typed access to the current key via Context and OnTimerContext. This is actually the only difference between the two function.

Somewhat as a consequence of the above, the KeyedProcessFunction can be used solely on a keyed stream, however ProcessFunction can be used in both. That was actually the only way to use a ProcessFunction on a keyed stream prior to introducing the KeyedProcessFunction. If you don't need access to the current key you should be fine with using the ProcessFunction on a keyed stream and there you can use the TimerService. It is advised to use a KeyedProcessFunction on a keyed stream, however for backwards compatibility the old behaviour has been kept.

Hope that it clarifies the things a bit.

Best,

Dawid

On 17/03/2021 07:47, Chirag Dewan wrote:

Hi,

Currently, both ProcessFunction and KeyedProcessFunction (and their CoProcess counterparts) expose the Context and TimerService in the processElement() method. However, if we use the TimerService in non keyed context, it gives a runtime error. 

I am a bit confused about these APIs. Is there any specific reason for exposing TimerService in non-keyed context especially if it cant be used without keyed stream?

Any leads are much appreciated.

Thanks,

Chirag