[ANNOUNCE] Weekly Community Update 2019/44

[Konstantin Knauf-2]

Dear community,

happy to share this week's community digest with updates on the ongoing release cycle, end-to-end performance testing, changes to the Table API and a discussion on the Flink Per-Job Mode (aka Application Clusters).

Flink Development

==============

* [releases] We have about one month until the planned feature freeze for Flink 1.10 and Gary has shared another status update. Check it out for a good overview of the ongoing development threads. [1]

* [development process] Yu proposes to integrate end-to-end performance tests into our build process. For this he has published FLIP-83 [2] which describes two benchmark jobs and a list of configuration scenarios to consider. In a first step the tests would focus on throughput in a small standalone cluster. [3]

* [development process] Yu reminds all FLIP authors to keep the FLIP status in Confluence up-to-date to facilitate release planning [4]

* [sql] Terry Wang proposes a few changes to the API of the TableEnvironment (TableEnvironment#execute/sqlQuery/sqlUpdate etc.) to better support asynchronous submission and multi-line sql statements and to make the API more consistent overall. [5] 

* [clients] Tison has started a discussion on Flink's per-job mode and highlights two shortcomings of the current implementation. First, the per-job mode only allows a single JobGraph to be executed. Second, on YARN, the JobGraph is compiled on the client side, not in the Flink Master as in the Standalone Per-Job mode. There has already been quite some feedback on the proposal, which mostly emphasizes the need for multiple JobGraphs in per-job mode and advantages of the current client-side compilation of the JobGraph. [6]

* [connectors] flink-orc has been moved from "flink-connectors" to "flink-formats". [7,8]

[1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Progress-of-Apache-Flink-1-10-2-tp34585.html

[2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-83%3A+Flink+End-to-end+Performance+Testing+Framework

[3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-83-Flink-End-to-end-Performance-Testing-Framework-tp34517.html

[4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Reminder-please-update-FLIP-document-to-latest-status-tp34555.html

[5] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-84-Improve-Refactor-API-of-Table-Module-tp34537.html

[6] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Semantic-and-implementation-of-per-job-mode-tp34502p34520.html

[7] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Move-flink-orc-to-flink-formats-from-flink-connectors-tp34438.html

[8] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Move-flink-orc-to-flink-formats-from-flink-connectors-tp34496.html

Notable Bugs

==========

* [FLINK-14546] [1.9.1] [1.8.2] The RabbitMQSource might leave open consumers around after a job is cancelled or stopped. PR available. [9]

[9] https://issues.apache.org/jira/browse/FLINK-14546

Events, Blog Posts, Misc

===================

* Becket Qin is now a member of the Apache Flink PMC. Congratulations! [10]

* Euroa Nova has published a Flink Forward Europe recap on their blog including key takeaways and summaries of selected talks. [11]

* Preetdeep Kumar has published the first part of a series of articles on Streaming ETL with Apache Flink on DZone. [12]

* Upcoming Meetups

    * There will be Flink/Spark talk at the next Chicago Big Data [13] on the 7th of November. No idea what it will be about (can't join the group) :)

    * At the next Athens Big Data Group on the 14th of November Chaoran Yu of Lightbend will talk about Flink and Spark on Kubernetes. [14]

    * We will have our next Apache Flink Meetup in Munich on November 27th with talks by Heiko Udluft & Giuseppe Sirigu, Airbus, and Konstantin Knauf (on Stateful Functions). [15]

[10] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Becket-Qin-joins-the-Flink-PMC-tp34400p34452.html

[11] https://research.euranova.eu/flink-forward-the-key-takeaways/

[12] https://dzone.com/articles/introduction-to-streaming-etl-with-apache-flink

[13] https://www.meetup.com/Chicago-area-Hadoop-User-Group-CHUG/events/265675851/

[14] https://www.meetup.com/Athens-Big-Data/events/265957761/

[15] https://www.meetup.com/Apache-Flink-Meetup-Munich/events/266072196/

Cheers,

Konstantin (@snntrable)

--