How to apply watermark on datastream and then do join operation on it

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

How to apply watermark on datastream and then do join operation on it

Rakesh Kumar
Hi all,

I am new to flink, i written a small flink streaming program which take two inputs from kafka and put it to two different data streams. i wanted to apply watermarks on it and then i wanted to join both datastream.

For reference i have attached an image.

flinkJoinDataStream.png (936K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to apply watermark on datastream and then do join operation on it

Fabian Hueske-2
Hi,

Welcome to the mailing list.

What exactly is your problem? Do you receive an error message? Is the program not compiling? Do you receive no output?
Regardless of that, I would recommend to provide the timestamp extractors to the Kafka source functions.
Also, I would have a close look at the window join semantics and validate that it provides the semantics that you need.
The interval join is an alternative that should be applicable to more use cases.

Best, Fabian


Am Fr., 30. Nov. 2018 um 14:11 Uhr schrieb Rakesh Kumar <[hidden email]>:
Hi all,

I am new to flink, i written a small flink streaming program which take two inputs from kafka and put it to two different data streams. i wanted to apply watermarks on it and then i wanted to join both datastream.

For reference i have attached an image.