Flink support for Kafka versions

[Prasanna kumar]

Hi Flinksters,

We are researching about if we could use the latest version of kafka (2.6.1 or 2.7.0) 

Since we are using Flink as a processor , we came across this https://issues.apache.org/jira/browse/FLINK-19168.

It says that it does not support version 2.5.0 and beyond.

That was created 8 months back , just checking if there is any effort on that front. 

Thanks,

Prasanna

[austin.ce]

Hi Prasanna,

It looks like the Kafka 2.5.0 connector upgrade is tied to dropping support for Scala 2.11. The best place to track that would be the ticket for Scala 2.13 support, FLINK-13414 [1], and its subtask FLINK-20845 [2].

I have listed FLINK-20845 as a blocker for FLINK-19168 for better visibility.

Best,

Austin

[1]: https://issues.apache.org/jira/browse/FLINK-13414

[2]: https://issues.apache.org/jira/browse/FLINK-20845

On Tue, Apr 20, 2021 at 9:08 AM Prasanna kumar <[hidden email]> wrote:

Hi Flinksters,

We are researching about if we could use the latest version of kafka (2.6.1 or 2.7.0) 

Since we are using Flink as a processor , we came across this https://issues.apache.org/jira/browse/FLINK-19168.

It says that it does not support version 2.5.0 and beyond.

That was created 8 months back , just checking if there is any effort on that front. 

Thanks,

Prasanna

[Arvid Heise-4]

I'm wondering if we could shade scala 1.13 dependencies inside the Kafka connector? Then we would be independent of the rather big FLINK-20845.

On Tue, Apr 20, 2021 at 5:54 PM Austin Cawley-Edwards <[hidden email]> wrote:

Hi Prasanna,

It looks like the Kafka 2.5.0 connector upgrade is tied to dropping support for Scala 2.11. The best place to track that would be the ticket for Scala 2.13 support, FLINK-13414 [1], and its subtask FLINK-20845 [2].

I have listed FLINK-20845 as a blocker for FLINK-19168 for better visibility.

Best,

Austin

[1]: https://issues.apache.org/jira/browse/FLINK-13414

[2]: https://issues.apache.org/jira/browse/FLINK-20845

On Tue, Apr 20, 2021 at 9:08 AM Prasanna kumar <[hidden email]> wrote:

Hi Flinksters,

We are researching about if we could use the latest version of kafka (2.6.1 or 2.7.0) 

Since we are using Flink as a processor , we came across this https://issues.apache.org/jira/browse/FLINK-19168.

It says that it does not support version 2.5.0 and beyond.

That was created 8 months back , just checking if there is any effort on that front. 

Thanks,

Prasanna