Support for Flink in EMR 6.0

[orips]

Hi,

EMR 6.0.0 has been released [1], and this release ignores Apache Flink (as well as other applications).

Are there any plans to add support for Apache Flink for EMR 6.0.0 in the future?

Thanks.

[1] https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-emr-announces-emr-release-6-with-new-major-versions-hadoop-hive-hbase-amazon-linux-2-docker/

[rmetzger0]

Hey Ori,

thanks a lot for reaching out to the user@ mailing list. This is more a question for the EMR Team at Amazon than the Flink community. But it seems that the lack of Hadoop 3 support in Flink [1] seems to be the reason why EMR 6.0 doesn't have Flink support.

Flink 1.11 will most likely have support for Hadoop 3, and I hope that Amazon will add Flink back into EMR.

However, if you have a EMR 6.0.0 with Hadoop 3 deployed, you should be able deploy vanilla Flink 1.10.0 there as well!

[1] https://issues.apache.org/jira/browse/FLINK-11086?focusedCommentId=17088936&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17088936

On Mon, May 4, 2020 at 2:55 PM Ori Popowski <[hidden email]> wrote:

Hi,

EMR 6.0.0 has been released [1], and this release ignores Apache Flink (as well as other applications).

Are there any plans to add support for Apache Flink for EMR 6.0.0 in the future?

Thanks.

[1] https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-emr-announces-emr-release-6-with-new-major-versions-hadoop-hive-hbase-amazon-linux-2-docker/

[orips]

Thanks.

I also contacted AWS support and they should keep me updated about it.

Hoping to see this soon :)

On Mon, May 4, 2020 at 4:41 PM Robert Metzger <[hidden email]> wrote:

Hey Ori,

thanks a lot for reaching out to the user@ mailing list. This is more a question for the EMR Team at Amazon than the Flink community. But it seems that the lack of Hadoop 3 support in Flink [1] seems to be the reason why EMR 6.0 doesn't have Flink support.

Flink 1.11 will most likely have support for Hadoop 3, and I hope that Amazon will add Flink back into EMR.

However, if you have a EMR 6.0.0 with Hadoop 3 deployed, you should be able deploy vanilla Flink 1.10.0 there as well!

[1] https://issues.apache.org/jira/browse/FLINK-11086?focusedCommentId=17088936&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17088936

On Mon, May 4, 2020 at 2:55 PM Ori Popowski <[hidden email]> wrote:

Hi,

EMR 6.0.0 has been released [1], and this release ignores Apache Flink (as well as other applications).

Are there any plans to add support for Apache Flink for EMR 6.0.0 in the future?

Thanks.

[1] https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-emr-announces-emr-release-6-with-new-major-versions-hadoop-hive-hbase-amazon-linux-2-docker/

[KristoffSC]

Actually it seems there is already ongoing discussion about installing Flink
1.10 on EMR

http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/upgrade-flink-from-1-9-1-to-1-10-0-on-EMR-td34114.html



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/