dataset dataframe join

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

dataset dataframe join

vishnuviswanath
Hi All,

Is there any workaround/hack to join a dataset with datastream since https://issues.apache.org/jira/browse/FLINK-2320 is still in progress.

Regards,
Vishnu
Reply | Threaded
Open this post in threaded view
|

Re: dataset dataframe join

Till Rohrmann
Hi Vishnu,

currently the only way to do this, is to persist the DataSet (e.g. writing to a file) and then reading from the persisted form (e.g. file) in the open method of a rich function in the DataStream program. That way you can keep the data in your operator and then join with incoming stream records.

Cheers,
Till

On Wed, Jun 15, 2016 at 11:43 PM, Vishnu Viswanath <[hidden email]> wrote:
Hi All,

Is there any workaround/hack to join a dataset with datastream since https://issues.apache.org/jira/browse/FLINK-2320 is still in progress.

Regards,
Vishnu

Reply | Threaded
Open this post in threaded view
|

Re: dataset dataframe join

vishnuviswanath
Thank you Till,

On Thu, Jun 16, 2016 at 10:08 AM, Till Rohrmann <[hidden email]> wrote:
Hi Vishnu,

currently the only way to do this, is to persist the DataSet (e.g. writing to a file) and then reading from the persisted form (e.g. file) in the open method of a rich function in the DataStream program. That way you can keep the data in your operator and then join with incoming stream records.

Cheers,
Till

On Wed, Jun 15, 2016 at 11:43 PM, Vishnu Viswanath <[hidden email]> wrote:
Hi All,

Is there any workaround/hack to join a dataset with datastream since https://issues.apache.org/jira/browse/FLINK-2320 is still in progress.

Regards,
Vishnu




--
Thanks and Regards,
Vishnu Viswanath,