Intermediate solution set of delta iteration

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Intermediate solution set of delta iteration

Mengqi Yang
Hi all,

Are there any approaches here I could get intermediate solution sets from every delta iteration? I tried union but the compiler gave me the error:
Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: Error: The only operations allowed on the solution set are Join and CoGroup.


Best,
Mengqi
Reply | Threaded
Open this post in threaded view
|

Re: Intermediate solution set of delta iteration

Vasiliki Kalavri
Hi Mengqi,

if what you are trying to do is output the solution set of every iteration, before the iteration has finished, then that is not possible.
i.e. you can not output the solution set to a sink or another operator during the iteration.

However, you can add elements to the solution set and grow the dataset as the iteration proceeds and retrieve it after it is finished.
Currently, you cannot union with the solution set, but a CoGroup could give you the desired result.

Alternatively, you might want to look at bulk iterations (Iterate abstraction) [1]. With these you can produce a completely new dataset after every iteration.

I hope that helps!
-Vasia.


On 20 March 2016 at 18:12, Mengqi Yang <[hidden email]> wrote:
Hi all,

Are there any approaches here I could get intermediate solution sets from
every delta iteration? I tried union but the compiler gave me the error:
Exception in thread "main"
org.apache.flink.api.common.InvalidProgramException: Error: The only
operations allowed on the solution set are Join and CoGroup.


Best,
Mengqi



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Intermediate-solution-set-of-delta-iteration-tp5656.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.