Re: [PROPOSAL] Contribute Stateful Functions to Apache Flink

Posted by Jark Wu-3 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/PROPOSAL-Contribute-Stateful-Functions-to-Apache-Flink-tp30468p30475.html

Hi Stephan,

big +1 for adding statefun to Apache Flink. 

I think we can add it in the main repository as a library just like State Process API. 

Having it in the main repository can closely co-develop with Apache Flink which be beneficial for both side.

Regards,

Jark

On Fri, 11 Oct 2019 at 19:12, Till Rohrmann <[hidden email]> wrote:

Hi Stephan,

+1 for adding stateful functions to Flink. I believe the new set of applications this feature will unlock will be super interesting for new and existing Flink users alike.

One reason for not including it in the main repository would to not being bound to Flink's release cadence. This would allow to release faster and more often. However, I believe that having it eventually in Flink's main repository would be beneficial in the long run.

Cheers,

Till

On Fri, Oct 11, 2019 at 12:56 PM Trevor Grant <[hidden email]> wrote:

+1 non-binding on contribution. 

Separate repo, or feature branch to start maybe? I just feel like in the beginning this thing is going to have lots of breaking changes that maybe aren't going to fit well with tests / other "v1+" release code. Just my .02. 

On Fri, Oct 11, 2019 at 4:38 AM Stephan Ewen <[hidden email]> wrote:

Dear Flink Community!

Some of you probably heard it already: On Tuesday, at Flink Forward Berlin, we announced **Stateful Functions**.

Stateful Functions is a library on Flink to implement general purpose applications. It is built around stateful functions (who would have thunk)

that can communicate arbitrarily through messages, have consistent state, and a small resource footprint. They are a bit like keyed ProcessFunctions

that can send each other messages.

As simple as this sounds, this means you can now communicate in non-DAG patterns, so it allows users to build programs they cannot build with Flink.

It also has other neat properties, like multiplexing of functions, modular composition, tooling both container-based deployments and as-a-Flink-job deployments.

You can find out more about it here

  - Website: https://statefun.io/

  - Code: https://github.com/ververica/stateful-functions

  - Talk with motivation: https://speakerdeck.com/stephanewen/stateful-functions-building-general-purpose-applications-and-services-on-apache-flink?slide=12

Now for the main issue: **We would like to contribute this project to Apache Flink**

I believe that this is a great fit for both sides.

For the Flink community, it would be a way to extend the capabilities and use cases of Flink into a completely different type of applications and thus grow the community into this new field.

Many discussions recently about evolving the Flink runtime (both on the mailing list and at conferences) show the interest in Flink users in the space that Stateful Functions covers.

It seems natural that Stateful Functions should closely co-develop with Apache Flink, ideally as part of the project.

There are many details to be discusses, for example whether this should be added to the Flink core repository, or whether we and to create a separate repository

for this. But I think we should start discussing this after we have consensus on whether the community wants this contribution.

Really looking forward to hear what you think!

Best Regards,

Stephan