[ANNOUNCE] Weekly Community Update 2019/45

[Konstantin Knauf-2]

Dear community,

happy to share this week's community digest with updates on stateful functions & Flink 1.8.3, a discussion on a connector for Hortonworks/Cloudera's schema registry, a couple of meetups and a bit more.

Flink Development

==============

* [stateful functions] After the successful vote to accept Stateful Functions into Flink Igal has started a thread to discuss a few details of the contribution like repository name, mailing lists, component name, etc. [1]

* [releases] Jingcheng has started a conversation about the release of Flink 1.8.3. Probably still waiting for a few fixes to come in, but looks like there could be a first RC soon. [2]

* [connectors] Őrhidi Mátyás and Gyula propose to contribute a connector for Hortonworks/Cloudera Schema Registry, which can be used during de-/serialization in Flink's Kafka Connector. [3]

* [python] Apache Flink will start Python processes to execute Python UDFs in the Table API, planned for 1.10. Dian Fu has published a proposal how to integrate the resource requirements of these Python processes into the unified memory configuration framework, which is currently introduced in FLIP-49. [4]

[1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Stateful-Functions-Contribution-Details-tp34737.html

[2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-3-tp34811.html

[3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Avro-Cloudera-Registry-FLINK-14577-tp34647.html

[4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tp34631.html

Notable Bugs

==========

* [FLINK-14382] [1.9.1] [1.8.2] [yarn] In Flink 1.9+ filesystem dependencies can be loaded via a plugin mechanism, each with its own classloader. This does currently not work on YARN, where the plugin directory is directly added to the classpath instead. [5]

[5] https://issues.apache.org/jira/browse/FLINK-14382

Events, Blog Posts, Misc

===================

* Jark Wu is now a member of the Apache Flink PMC. Congratulations! [6]

* This blog post by Sreekanth Krishnavajjala & Vinod Kataria (AWS) includes a hands-on introduction to Apache Flink on AWS EMR. [7]

* Upcoming Meetups

    * At the next Athens Big Data Group on the 14th of November Chaoran Yu of Lightbend will talk about Flink and Spark on Kubernetes. [8]

    * Bowen Li will speak about "The Rise of Apache and Stream Processing" at the next Big Data Bellevue in Seattle on the 20th of November. [9]

    * The next edition of the Bay Area Apache Flink meetup will happen on the 20th of November with talks by Gyula Fora (Cloudera) and Lakshmi Rao (Lyft).[10]

    * We will have our next Apache Flink Meetup in Munich on November 27th with talks by Heiko Udluft & Giuseppe Sirigu, Airbus, and Konstantin Knauf (on Stateful Functions). [11]

[6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Jark-Wu-is-now-part-of-the-Flink-PMC-tp34768.html

[7] https://idk.dev/extract-oracle-oltp-data-in-real-time-with-goldengate-and-query-from-amazon-athena/

[8] https://www.meetup.com/Athens-Big-Data/events/265957761/

[9] https://www.meetup.com/Big-Data-Bellevue-BDB/events/fxbnllyzpbbc/

[10] https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/266226960/

[11] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/266072196/

Cheers,

Konstantin (@snntrable)

--

[phoenixjiangnan]

FYI, 

I'll be at the ScaleByTheBay conference in Oakland on 11/14-15 to give a Flink talk (https://sched.co/RoSY). Shoot me an email if anyone is also there and would like to meet in person and talk about Flink, data, AI/ML, etc.

On Sun, Nov 10, 2019 at 8:17 AM Konstantin Knauf <[hidden email]> wrote:

Dear community,

happy to share this week's community digest with updates on stateful functions & Flink 1.8.3, a discussion on a connector for Hortonworks/Cloudera's schema registry, a couple of meetups and a bit more.

Flink Development

==============

* [stateful functions] After the successful vote to accept Stateful Functions into Flink Igal has started a thread to discuss a few details of the contribution like repository name, mailing lists, component name, etc. [1]

* [releases] Jingcheng has started a conversation about the release of Flink 1.8.3. Probably still waiting for a few fixes to come in, but looks like there could be a first RC soon. [2]

* [connectors] Őrhidi Mátyás and Gyula propose to contribute a connector for Hortonworks/Cloudera Schema Registry, which can be used during de-/serialization in Flink's Kafka Connector. [3]

* [python] Apache Flink will start Python processes to execute Python UDFs in the Table API, planned for 1.10. Dian Fu has published a proposal how to integrate the resource requirements of these Python processes into the unified memory configuration framework, which is currently introduced in FLIP-49. [4]

[1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Stateful-Functions-Contribution-Details-tp34737.html

[2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Releasing-Flink-1-8-3-tp34811.html

[3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-Avro-Cloudera-Registry-FLINK-14577-tp34647.html

[4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tp34631.html

Notable Bugs

==========

* [FLINK-14382] [1.9.1] [1.8.2] [yarn] In Flink 1.9+ filesystem dependencies can be loaded via a plugin mechanism, each with its own classloader. This does currently not work on YARN, where the plugin directory is directly added to the classpath instead. [5]

[5] https://issues.apache.org/jira/browse/FLINK-14382

Events, Blog Posts, Misc

===================

* Jark Wu is now a member of the Apache Flink PMC. Congratulations! [6]

* This blog post by Sreekanth Krishnavajjala & Vinod Kataria (AWS) includes a hands-on introduction to Apache Flink on AWS EMR. [7]

* Upcoming Meetups

    * At the next Athens Big Data Group on the 14th of November Chaoran Yu of Lightbend will talk about Flink and Spark on Kubernetes. [8]

    * Bowen Li will speak about "The Rise of Apache and Stream Processing" at the next Big Data Bellevue in Seattle on the 20th of November. [9]

    * The next edition of the Bay Area Apache Flink meetup will happen on the 20th of November with talks by Gyula Fora (Cloudera) and Lakshmi Rao (Lyft).[10]

    * We will have our next Apache Flink Meetup in Munich on November 27th with talks by Heiko Udluft & Giuseppe Sirigu, Airbus, and Konstantin Knauf (on Stateful Functions). [11]

[6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Jark-Wu-is-now-part-of-the-Flink-PMC-tp34768.html

[7] https://idk.dev/extract-oracle-oltp-data-in-real-time-with-goldengate-and-query-from-amazon-athena/

[8] https://www.meetup.com/Athens-Big-Data/events/265957761/

[9] https://www.meetup.com/Big-Data-Bellevue-BDB/events/fxbnllyzpbbc/

[10] https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/266226960/

[11] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/266072196/

Cheers,

Konstantin (@snntrable)

--

Konstantin Knauf | Solutions Architect

[phone]

Follow us @VervericaData Ververica

--

Join Flink Forward - The Apache Flink Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--

Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Tony) Cheng