[ANNOUNCE] Weekly Community Update 2019/38

[Konstantin Knauf-2]

Dear community,

happy to share this week's community update with a FLIP for the Pulsar Connector contribution, three FLIPs for the SQL Ecosystem (plugin system, computed columns, extended support for views), and a bit more. Enjoy!

Flink Development

==============

* [connectors]  After some discussion on the mailing list over the last weeks, Sijie has opened a FLIP to add an exactly-once Pulsar Connector (DataStream API, Table API, Catalog API) to Flink. [1]

* [sql] The discussion on supporting Hive built-in function in Flink SQL lead to FLIP-69 to extend the core table system with modular plugins. [2] While focusing on function modules as a first step, the FLIP proposes a more general plugin system also covering user defined types, operators, rules, etc. As part of this FLIP the existing functions in Flink SQL would also be migrated into a "CorePlugin". [3]

* [sql] Danny proposes to add support for computed columns in Flink SQL (as FLIP-10). [4]

* [sql] Zhenghua has started a discussion on extending support for VIEWs in Flink SQL (as FLIP-71).  He proposes to add support to store views in a catalog and to add support for "SHOW VIEWS" and "DESCRIBE VIEW". [5]

[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-72%3A+Introduce+Pulsar+Connector

[2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-68%3A+Extend+Core+Table+System+with+Modular+Plugins

[3] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-68-Extend-Core-Table-System-with-Modular-Plugins-tp33161.html

[4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-70-Support-Computed-Column-for-Flink-SQL-tp33126.html

[5] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-71-E2E-View-support-in-Flink-SQL-tp33131.html

Notable Bugs

==========

* [FLINK-14010] [ 1.9.0] [1.8.2] [1.7.2] [yarn] When the Flink Yarn Application Manager receives a shut down request by the YARN Resource Manager, the Flink cluster can get into an inconsistent state, where leaderhship for JobManager, ResourceManager and Dispatcher components is split between two master processes. Tison is working on a fix. [6]

* [FLINK-14107] [ 1.9.0] [ 1.8.2] [kinesis] When using event time alignment with the Kinsesis Consumer the consumer might deadlock in one corner case. Fixed for 1.9.1 and 1.8.3. [7]

[6] https://issues.apache.org/jira/browse/FLINK-14010

[7] https://issues.apache.org/jira/browse/FLINK-14107

Events, Blog Posts, Misc

===================

* Upcoming Meetups

    * Enrico Canzonieri of Yelp and David Massart of Tentative will share their Apache Flink user stories of Yelp and BNP Paribas at the next Bay Area Apache Flink Meetup 24th of September.  [8]

* Ana Esguerra has published a blog post on how to run Flink on YARN with Kerberos for Kafka & YARN. [9]

[8] https://www.meetup.com/Bay-Area-Apache-Flink-Meetup/events/262680261/

[9] https://medium.com/@minyodev/apache-flink-on-yarn-with-kerberos-authentication-adeb62ef47d2

Cheers,

Konstantin

--