Distributed Cache support for StreamExecutionEnvironment

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Distributed Cache support for StreamExecutionEnvironment

Swapnil Chougule
Hi Team,

Is there support for Distributed Cache in StreamExecutionEnvironment? I didn't find any such things in StreamExecutionEnvironment. I am using flink 1.1.1
I found distributed cache for ExecutionEnvironment but not for StreamExecutionEnvironment

If Yes, Can anybody tell me how to use same ? Else any work around is there for same ?

Regards,
Swapnil
Reply | Threaded
Open this post in threaded view
|

Re: Distributed Cache support for StreamExecutionEnvironment

rmetzger0
Hi Swapnil,

there's no support for something like DistributedCache in the DataStream API.
However, as a workaround, you can rely on the RichFunction's open() method's to load such data directly from a distributed file system.

Regards,
Robert

On Fri, Sep 9, 2016 at 8:13 AM, Swapnil Chougule <[hidden email]> wrote:
Hi Team,

Is there support for Distributed Cache in StreamExecutionEnvironment? I didn't find any such things in StreamExecutionEnvironment. I am using flink 1.1.1
I found distributed cache for ExecutionEnvironment but not for StreamExecutionEnvironment

If Yes, Can anybody tell me how to use same ? Else any work around is there for same ?

Regards,
Swapnil

Reply | Threaded
Open this post in threaded view
|

Re: Distributed Cache support for StreamExecutionEnvironment

Swapnil Chougule
Thanks Robert. It worked for me. I have used RichFunction's open() method.

Regards,
Swapnil

On Fri, Sep 9, 2016 at 3:40 PM, Robert Metzger <[hidden email]> wrote:
Hi Swapnil,

there's no support for something like DistributedCache in the DataStream API.
However, as a workaround, you can rely on the RichFunction's open() method's to load such data directly from a distributed file system.

Regards,
Robert

On Fri, Sep 9, 2016 at 8:13 AM, Swapnil Chougule <[hidden email]> wrote:
Hi Team,

Is there support for Distributed Cache in StreamExecutionEnvironment? I didn't find any such things in StreamExecutionEnvironment. I am using flink 1.1.1
I found distributed cache for ExecutionEnvironment but not for StreamExecutionEnvironment

If Yes, Can anybody tell me how to use same ? Else any work around is there for same ?

Regards,
Swapnil