Re: Using Stateful Functions within a Flink pipeline

Posted by Oytun Tez on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Using-Stateful-Functions-within-a-Flink-pipeline-tp34582p34586.html

@Igal, this sounds more comprehensive (better) than just opening DataStreams: "basically exposing the core Flink job that is the heart of stateful functions. "

Great!

On Wed, Apr 22, 2020 at 10:56 AM Igal Shilman <[hidden email]> wrote:

Hi Annemarie,

There are plans to make stateful functions more easily embeddable within a Flink Job, 

perhaps skipping ingress/egress routing abstracting all together and basically exposing the 

core Flink job that is the heart of stateful functions. 

Although these plans are not concrete yet I believe this would be brought to discussion with the community in the upcoming weeks.

Currently, you can split your pipeline to a preprocessing Flink job, and a stateful functions job.

Good luck,

Igal.

On Wed, Apr 22, 2020 at 4:26 PM Annemarie Burger <[hidden email]> wrote:

I was wondering if it is possible to use a Stateful Function within a Flink
pipeline. I know they work with different API's, so I was wondering if it is
possible to have a DataStream as ingress for a Stateful Function.

Some context: I'm working on a streaming graph analytics system, and want to
save the state of the graph within a window. Stateful functions could then
allow me to process these distributed graph states by making use of the SF
messaging.

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/