Setting SlotSharingGroups for Joins and Sorting Keyed Streams

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Setting SlotSharingGroups for Joins and Sorting Keyed Streams

Dimitris Palyvos
Hello,

I have two unrelated questions regarding the Flink API. Both of these are connected to some proof-of-concepts and will not be deployed in an actual production environment, so even less-than-ideal suggestions are welcome!

First, maybe this is a silly question, but is it possible to set the slotSharingGroup for a join? I cannot seem to be able to set this using the standard join API: stream.join(..).where(...).equalTo(...).window(...).apply(...)

Second, is there a good way to sort a keyed stream before pushing it through an aggregate? I am thinking of something like what is described here (I know this does not actually exist): https://cwiki.apache.org/confluence/display/FLINK/Time+and+Order+in+Streams

I was thinking of something like stream.keyBy(...).flatMap(new MyBufferingAndSortingOp()).window(..) but the flatMap returns a SingleOutputStreamOperator so this does not work. Is there any other way to sort the tuples in a window except doing it inside the AggregateFunction/ProcessWindowFunction?

Thank you in advance!

Kind regards,
Dimitris Palyvos