[DISCUSS] Dropping Scala 2.10

[Aljoscha Krettek]

Hi,

Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.

Are there any concerns about this?

Best,
Aljoscha

[Till Rohrmann]

Given that the last maintenance release Scala 2.10.6 is from about 2 years ago, I would also be in favour of dropping Scala 2.10 support from Flink. This will make maintenance easier for us and allow us to drop artifacts like Flakka.

Cheers,

Till

On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]> wrote:

Hi,

Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.

Are there any concerns about this?

Best,
Aljoscha

[Márton Balassi]

Hi Aljoscha,

I am in favor of the change. No concerns on my side, just one remark that I have talked to Sean last week (ccd) and he mentioned that he has faced some technical issues while driving the transition from 2.10 to 2.12 for Spark. It had to do with changes in the scope of implicits. You might end up hitting the same.

Best,

Marton

On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]> wrote:

Hi,

Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.

Are there any concerns about this?

Best,
Aljoscha

[Sean Owen]

For the curious, here's the overall task in Spark:

https://issues.apache.org/jira/browse/SPARK-14220

and  most of the code-related changes:

https://github.com/apache/spark/pull/18645 

and where it's stuck at the moment:

https://mail-archives.apache.org/mod_mbox/spark-dev/201709.mbox/%3CCAMAsSdKe7Os80mX7jYaD2vNWLGWioBgCb4GG55eaN_iotFxZvw%40mail.gmail.com%3E

On Tue, Sep 19, 2017 at 11:07 AM Márton Balassi <[hidden email]> wrote:

Hi Aljoscha,

I am in favor of the change. No concerns on my side, just one remark that I have talked to Sean last week (ccd) and he mentioned that he has faced some technical issues while driving the transition from 2.10 to 2.12 for Spark. It had to do with changes in the scope of implicits. You might end up hitting the same.

Best,

Marton

On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]> wrote:

Hi,

Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.

Are there any concerns about this?

Best,
Aljoscha

[bowen.li]

+1 for dropping support for Scala 2.10

On Tue, Sep 19, 2017 at 3:29 AM, Sean Owen <[hidden email]> wrote:

For the curious, here's the overall task in Spark:

https://issues.apache.org/jira/browse/SPARK-14220

and  most of the code-related changes:

https://github.com/apache/spark/pull/18645 

and where it's stuck at the moment:

https://mail-archives.apache.org/mod_mbox/spark-dev/201709.mbox/%3CCAMAsSdKe7Os80mX7jYaD2vNWLGWioBgCb4GG55eaN_iotFxZvw%40mail.gmail.com%3E

On Tue, Sep 19, 2017 at 11:07 AM Márton Balassi <[hidden email]> wrote:

Hi Aljoscha,

I am in favor of the change. No concerns on my side, just one remark that I have talked to Sean last week (ccd) and he mentioned that he has faced some technical issues while driving the transition from 2.10 to 2.12 for Spark. It had to do with changes in the scope of implicits. You might end up hitting the same.

Best,

Marton

On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]> wrote:

Hi,

Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.

Are there any concerns about this?

Best,
Aljoscha

[Hai Zhou]

+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

[Ted Yu]

+1

-------- Original message --------

From: Hai Zhou <[hidden email]>

Date: 9/20/17 12:44 AM (GMT-08:00)

To: Aljoscha Krettek <[hidden email]>, [hidden email], user <[hidden email]>

Subject: Re: [DISCUSS] Dropping Scala 2.10

+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

[Federico D'Ambrosio-2]

Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10 as part of their hadoop distribution.
Could the use of a different scala version cause issues with the Kafka connector? I'm asking because we are using HDP 2.6 and we once already had some issue with conflicting scala versions concerning Kafka (though, we were using Storm, I still haven't tested the Flink connector in this context).

Regards,

Federico

2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email]>:

+1

-------- Original message --------

From: Hai Zhou <[hidden email]>

Date: 9/20/17 12:44 AM (GMT-08:00)

To: Aljoscha Krettek <[hidden email]>, [hidden email], user <[hidden email]>

Subject: Re: [DISCUSS] Dropping Scala 2.10

+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

--

[Aljoscha Krettek]

Hi Federico,

As far as I know, the Kafka client code has been rewritten in Java for version 0.9, meaning there is no more Scala dependency in there. Only the server (broker) code still contains Scala but it doesn't matter what Scala version a client uses, if any.

Best,

Aljoscha 

On 20. Sep 2017, at 14:32, Federico D'Ambrosio <[hidden email]> wrote:

Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10 as part of their hadoop distribution.
Could the use of a different scala version cause issues with the Kafka connector? I'm asking because we are using HDP 2.6 and we once already had some issue with conflicting scala versions concerning Kafka (though, we were using Storm, I still haven't tested the Flink connector in this context).

Regards,

Federico

2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email]>:

+1

-------- Original message --------

From: Hai Zhou <[hidden email]>

Date: 9/20/17 12:44 AM (GMT-08:00)

To: Aljoscha Krettek <[hidden email]>, [hidden email], user <[hidden email]>

Subject: Re: [DISCUSS] Dropping Scala 2.10

+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

--

[Stephan Ewen]

+1

@ Frederico: I think Aljoscha is right - Flink only executes Kafka client code, which is Scala independent from 0.9 on. Do you use Kafka 0.8 still?

On Wed, Sep 20, 2017 at 10:00 PM, Aljoscha Krettek <[hidden email]> wrote:

Hi Federico,

As far as I know, the Kafka client code has been rewritten in Java for version 0.9, meaning there is no more Scala dependency in there. Only the server (broker) code still contains Scala but it doesn't matter what Scala version a client uses, if any.

Best,

Aljoscha 

On 20. Sep 2017, at 14:32, Federico D'Ambrosio <[hidden email]> wrote:

Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10 as part of their hadoop distribution.
Could the use of a different scala version cause issues with the Kafka connector? I'm asking because we are using HDP 2.6 and we once already had some issue with conflicting scala versions concerning Kafka (though, we were using Storm, I still haven't tested the Flink connector in this context).

Regards,

Federico

2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email]>:

+1

-------- Original message --------

From: Hai Zhou <[hidden email]>

Date: 9/20/17 12:44 AM (GMT-08:00)

To: Aljoscha Krettek <[hidden email]>, [hidden email], user <[hidden email]>

Subject: Re: [DISCUSS] Dropping Scala 2.10

+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

--

[Federico D'Ambrosio-2]

Ok, thank you all for the clarification.

@Stephan: I'm using Kafka 0.10, I guess the problem I had then was actually unrelated to specific Kafka version

Il 21 set 2017 16:30, "Stephan Ewen" <[hidden email]> ha scritto:

+1

@ Frederico: I think Aljoscha is right - Flink only executes Kafka client code, which is Scala independent from 0.9 on. Do you use Kafka 0.8 still?

On Wed, Sep 20, 2017 at 10:00 PM, Aljoscha Krettek <[hidden email]> wrote:

Hi Federico,

As far as I know, the Kafka client code has been rewritten in Java for version 0.9, meaning there is no more Scala dependency in there. Only the server (broker) code still contains Scala but it doesn't matter what Scala version a client uses, if any.

Best,

Aljoscha 

On 20. Sep 2017, at 14:32, Federico D'Ambrosio <[hidden email]> wrote:

Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10 as part of their hadoop distribution.
Could the use of a different scala version cause issues with the Kafka connector? I'm asking because we are using HDP 2.6 and we once already had some issue with conflicting scala versions concerning Kafka (though, we were using Storm, I still haven't tested the Flink connector in this context).

Regards,

Federico

2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email]>:

+1

-------- Original message --------

From: Hai Zhou <[hidden email]>

Date: 9/20/17 12:44 AM (GMT-08:00)

To: Aljoscha Krettek <[hidden email]>, [hidden email], user <[hidden email]>

Subject: Re: [DISCUSS] Dropping Scala 2.10

+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

--