Re: Why the current time of my window never reaches the end time when I use KeyedProcessFunction?

Posted by Piotr Nowojski-3 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Why-the-current-time-of-my-window-never-reaches-the-end-time-when-I-use-KeyedProcessFunction-tp28294p28309.html

Hi,

Isn’t your problem that the source is constantly emitting the data and bumping your timers? Keep in mind that the code that you are basing on has the following characteristic:

> In the following example a KeyedProcessFunction maintains counts per key, and emits a key/count pair whenever a minute passes without an update for that key

Piotrek

On 17 Jun 2019, at 15:51, Felipe Gutierrez <[hidden email]> wrote:

Hi,

I used this example of KeyedProcessFunction from the FLink website [1] and I have implemented my own KeyedProcessFunction to process some approximation counting [2]. This worked very well. Then I switched the data source to consume strings from Twitter [3]. The data source is consuming the strings because I can see it when I debug. However, the time comparison is always different on the onTimer() method, and I never get the results of the window processing. I don't know the exact reason that this is happening. I guess it is because my state is too heavy. But, still shouldn't the time be correct at some point to finish the evaluation of my window?

[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/operators/process_function.html#example
[2] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/WordHLLKeyedProcessWindowSocket.java
[3] https://github.com/felipegutierrez/explore-flink/blob/master/src/main/java/org/sense/flink/examples/stream/WordHLLKeyedProcessWindowTwitter.java

Kind Regards,

Felipe