Filtering and mapping data after window opertator

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Filtering and mapping data after window opertator

Soheil Pourbafrani
Hi, I'm getting data stream from a source and after gathering data in a time window I want to do some operation like filtering and mapping on windowed data, but the output of time window operation just allow reduce, aggregate or ... function and after that, I want to apply functions like filter or map. How can I apply filter function to the windowed data without using reduce function before that?

temp.keyBy( 0).
timeWindow(Time.milliseconds(INTERVAL_TIME)).reduce(new ReduceFunction<Tuple2<Long, byte[]>>() {

Reply | Threaded
Open this post in threaded view
|

Re: Filtering and mapping data after window opertator

Hequn Cheng
Hi Soheil,

We can't apply FilterFunction or MapFunction on WindowedStream. It is recommended to do these operations on DataStream, for example, temp.filter().map().keyBy(0).timeWindow(). 

Best, Hequn

On Sat, Jul 14, 2018 at 9:14 PM, Soheil Pourbafrani <[hidden email]> wrote:
Hi, I'm getting data stream from a source and after gathering data in a time window I want to do some operation like filtering and mapping on windowed data, but the output of time window operation just allow reduce, aggregate or ... function and after that, I want to apply functions like filter or map. How can I apply filter function to the windowed data without using reduce function before that?

temp.keyBy( 0).
timeWindow(Time.milliseconds(INTERVAL_TIME)).reduce(new ReduceFunction<Tuple2<Long, byte[]>>() {