Flink 0.9 built with Scala 2.11
Locked
12
12
Flink 0.9 built with Scala 2.11
 
|
Hi community!
We started a new project called Piglet (https://github.com/ksattler/piglet). For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. Do you have an official repository for Flink 0.9 (built with Scala 2.11)? Best Regards, Philipp |
Re: Flink 0.9 built with Scala 2.11
|
|
Nice! On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote:
Hi community! |
Re: Flink 0.9 built with Scala 2.11
|
|
Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in master branch.
`mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command builds Flink with Scala 2.11. Regards, Chiwan Park > On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier <[hidden email]> wrote: > > Nice! > > On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote: > Hi community! > > We started a new project called Piglet (https://github.com/ksattler/piglet). > For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. > > Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. > > Do you have an official repository for Flink 0.9 (built with Scala 2.11)? > > Best Regards, > Philipp |
Re: Flink 0.9 built with Scala 2.11
|
|
Thank you Chiwan!
I did not know the master has a 2.11 profile. But there is no pre-built Flink with 2.11, which I could refer to in sbt or maven, is it? Best Regards, Philipp On 10.06.2015 15:03, Chiwan Park wrote: > Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in master branch. > `mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command builds Flink with Scala 2.11. > > Regards, > Chiwan Park > >> On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier <[hidden email]> wrote: >> >> Nice! >> >> On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote: >> Hi community! >> >> We started a new project called Piglet (https://github.com/ksattler/piglet). >> For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. >> >> Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. >> >> Do you have an official repository for Flink 0.9 (built with Scala 2.11)? >> >> Best Regards, >> Philipp > > > > |
Re: Flink 0.9 built with Scala 2.11
|
|
No there are no Scala 2.11 Flink binaries which you can download. You have to build it yourself. Cheers, Till On Wed, Jun 10, 2015 at 3:19 PM Philipp Goetze <[hidden email]> wrote: Thank you Chiwan! |
Re: Flink 0.9 built with Scala 2.11
|
|
In reply to this post by Philipp Goetze
No. Currently, there are no Flink binaries with scala 2.11 which are downloadable.
Regards, Chiwan Park > On Jun 10, 2015, at 10:18 PM, Philipp Goetze <[hidden email]> wrote: > > Thank you Chiwan! > > I did not know the master has a 2.11 profile. > > But there is no pre-built Flink with 2.11, which I could refer to in sbt or maven, is it? > > Best Regards, > Philipp > > On 10.06.2015 15:03, Chiwan Park wrote: >> Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in master branch. >> `mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command builds Flink with Scala 2.11. >> >> Regards, >> Chiwan Park >> >>> On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier <[hidden email]> wrote: >>> >>> Nice! >>> >>> On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote: >>> Hi community! >>> >>> We started a new project called Piglet (https://github.com/ksattler/piglet). >>> For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. >>> >>> Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. >>> >>> Do you have an official repository for Flink 0.9 (built with Scala 2.11)? >>> >>> Best Regards, >>> Philipp >> >> >> >> > |
Re: Flink 0.9 built with Scala 2.11
|
|
But I think uploading Flink API with scala 2.11 to maven repository is nice idea.
Could you create a JIRA issue? Regards, Chiwan Park > On Jun 10, 2015, at 10:23 PM, Chiwan Park <[hidden email]> wrote: > > No. Currently, there are no Flink binaries with scala 2.11 which are downloadable. > > Regards, > Chiwan Park > >> On Jun 10, 2015, at 10:18 PM, Philipp Goetze <[hidden email]> wrote: >> >> Thank you Chiwan! >> >> I did not know the master has a 2.11 profile. >> >> But there is no pre-built Flink with 2.11, which I could refer to in sbt or maven, is it? >> >> Best Regards, >> Philipp >> >> On 10.06.2015 15:03, Chiwan Park wrote: >>> Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in master branch. >>> `mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command builds Flink with Scala 2.11. >>> >>> Regards, >>> Chiwan Park >>> >>>> On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier <[hidden email]> wrote: >>>> >>>> Nice! >>>> >>>> On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote: >>>> Hi community! >>>> >>>> We started a new project called Piglet (https://github.com/ksattler/piglet). >>>> For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. >>>> >>>> Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. >>>> >>>> Do you have an official repository for Flink 0.9 (built with Scala 2.11)? >>>> >>>> Best Regards, >>>> Philipp >>> >>> >>> >>> >> > > > > |
Re: Flink 0.9 built with Scala 2.11
|
|
Done. Can be found here: https://issues.apache.org/jira/browse/FLINK-2200
Best Regards, Philipp On 10.06.2015 15:29, Chiwan Park wrote: > But I think uploading Flink API with scala 2.11 to maven repository is nice idea. > Could you create a JIRA issue? > > Regards, > Chiwan Park > >> On Jun 10, 2015, at 10:23 PM, Chiwan Park <[hidden email]> wrote: >> >> No. Currently, there are no Flink binaries with scala 2.11 which are downloadable. >> >> Regards, >> Chiwan Park >> >>> On Jun 10, 2015, at 10:18 PM, Philipp Goetze <[hidden email]> wrote: >>> >>> Thank you Chiwan! >>> >>> I did not know the master has a 2.11 profile. >>> >>> But there is no pre-built Flink with 2.11, which I could refer to in sbt or maven, is it? >>> >>> Best Regards, >>> Philipp >>> >>> On 10.06.2015 15:03, Chiwan Park wrote: >>>> Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in master branch. >>>> `mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command builds Flink with Scala 2.11. >>>> >>>> Regards, >>>> Chiwan Park >>>> >>>>> On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier <[hidden email]> wrote: >>>>> >>>>> Nice! >>>>> >>>>> On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote: >>>>> Hi community! >>>>> >>>>> We started a new project called Piglet (https://github.com/ksattler/piglet). >>>>> For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. >>>>> >>>>> Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. >>>>> >>>>> Do you have an official repository for Flink 0.9 (built with Scala 2.11)? >>>>> >>>>> Best Regards, >>>>> Philipp >>>> >>>> >>>> >> >> >> > > > |
Re: Flink 0.9 built with Scala 2.11
|
|
Please do ping this list if you encounter any problems with Flink during your project (you have done so already :-), but also if you find that the Flink API needs additions to map Pig well to Flink On Wed, Jun 10, 2015 at 3:47 PM, Philipp Goetze <[hidden email]> wrote: Done. Can be found here: https://issues.apache.org/jira/browse/FLINK-2200 |
|
|
I agree that we should ship a 2.11 build of Flink if downstream projects need that. The only thing that we should keep in mind when doing this is that the number of jars we're pushing to maven will explode (but that is fine) We have currently 46 maven modules and we would create 4 versions of each of the modules (hd1,hd2 x scala210,scala211) so we end up with 184 jars per release ;) The other big question that I have regarding this is how we want to name the modules. We could add the scala version to all the modules, like "flink-java_2.10", which would mean that users have to change a bit more when upgrading to the release supporting different scala versions. If we all agree on that, we can move on changing our maven setup. On Wed, Jun 10, 2015 at 8:34 AM, Kostas Tzoumas <[hidden email]> wrote:
|
Re: Flink 0.9 built with Scala 2.11
|
|
Hi. I think that we don’t need deploy all modules with scala variation. The pure java-based modules such as flink-java, flink-core, flink-optimizers, …, etc. don’t need to be deployed with scala version variation. We need only scala related modules such as flink-ml, flink-runtime, flink-scala, …, etc. with version variation.
So we can reduce a number of deployed modules. Regards, Chiwan Park > On Jun 13, 2015, at 9:17 AM, Robert Metzger <[hidden email]> wrote: > > I agree that we should ship a 2.11 build of Flink if downstream projects need that. > > The only thing that we should keep in mind when doing this is that the number of jars we're pushing to maven will explode (but that is fine) > We have currently 46 maven modules and we would create 4 versions of each of the modules (hd1,hd2 x scala210,scala211) so we end up with 184 jars per release ;) > > The other big question that I have regarding this is how we want to name the modules. > We could add the scala version to all the modules, like "flink-java_2.10", which would mean that users have to change a bit more when upgrading to the release supporting different scala versions. > > If we all agree on that, we can move on changing our maven setup. > > > On Wed, Jun 10, 2015 at 8:34 AM, Kostas Tzoumas <[hidden email]> wrote: > Please do ping this list if you encounter any problems with Flink during your project (you have done so already :-), but also if you find that the Flink API needs additions to map Pig well to Flink > > On Wed, Jun 10, 2015 at 3:47 PM, Philipp Goetze <[hidden email]> wrote: > Done. Can be found here: https://issues.apache.org/jira/browse/FLINK-2200 > > Best Regards, > Philipp > > > > On 10.06.2015 15:29, Chiwan Park wrote: > But I think uploading Flink API with scala 2.11 to maven repository is nice idea. > Could you create a JIRA issue? > > Regards, > Chiwan Park > > On Jun 10, 2015, at 10:23 PM, Chiwan Park <[hidden email]> wrote: > > No. Currently, there are no Flink binaries with scala 2.11 which are downloadable. > > Regards, > Chiwan Park > > On Jun 10, 2015, at 10:18 PM, Philipp Goetze <[hidden email]> wrote: > > Thank you Chiwan! > > I did not know the master has a 2.11 profile. > > But there is no pre-built Flink with 2.11, which I could refer to in sbt or maven, is it? > > Best Regards, > Philipp > > On 10.06.2015 15:03, Chiwan Park wrote: > Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in master branch. > `mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command builds Flink with Scala 2.11. > > Regards, > Chiwan Park > > On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier <[hidden email]> wrote: > > Nice! > > On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote: > Hi community! > > We started a new project called Piglet (https://github.com/ksattler/piglet). > For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. > > Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. > > Do you have an official repository for Flink 0.9 (built with Scala 2.11)? > > Best Regards, > Philipp > > > > > > > > > > > > |
Re: Flink 0.9 built with Scala 2.11
|
|
Good idea, Chiwan! On Sat, Jun 13, 2015 at 6:25 PM, Chiwan Park <[hidden email]> wrote: Hi. I think that we don’t need deploy all modules with scala variation. The pure java-based modules such as flink-java, flink-core, flink-optimizers, …, etc. don’t need to be deployed with scala version variation. We need only scala related modules such as flink-ml, flink-runtime, flink-scala, …, etc. with version variation. |
|
|
There was already a discussion regarding the two options here [1], back then we had a majority for giving all modules a scala suffix. I'm against giving all modules a suffix because we force our users to migrate the name and its confusing for Java users (I was confused myself when I was trying out Spark two years ago (back then I didn't know anything about Scala ;) )) On Sun, Jun 14, 2015 at 3:47 AM, Stephan Ewen <[hidden email]> wrote:
|
Re: Flink 0.9 built with Scala 2.11
|
|
+1 for giving only those modules a version suffix which depend on Scala.
On Sun, Jun 14, 2015 at 8:03 PM Robert Metzger <[hidden email]> wrote:
|
|
|
Okay, it seems like we have consensus on this. Who is interested in working on this? https://issues.apache.org/jira/browse/FLINK-2200 On Mon, Jun 15, 2015 at 1:26 AM, Till Rohrmann <[hidden email]> wrote:
|
Re: Flink 0.9 built with Scala 2.11
|
|
I’m interested in working on this. :) I’ll assign to me.
Regards, Chiwan Park > On Jun 21, 2015, at 8:22 AM, Robert Metzger <[hidden email]> wrote: > > Okay, it seems like we have consensus on this. Who is interested in working on this? https://issues.apache.org/jira/browse/FLINK-2200 > > On Mon, Jun 15, 2015 at 1:26 AM, Till Rohrmann <[hidden email]> wrote: > +1 for giving only those modules a version suffix which depend on Scala. > > > On Sun, Jun 14, 2015 at 8:03 PM Robert Metzger <[hidden email]> wrote: > There was already a discussion regarding the two options here [1], back then we had a majority for giving all modules a scala suffix. > > I'm against giving all modules a suffix because we force our users to migrate the name and its confusing for Java users (I was confused myself when I was trying out Spark two years ago (back then I didn't know anything about Scala ;) )) > > [1] https://github.com/apache/flink/pull/477#issuecomment-82266786 > > On Sun, Jun 14, 2015 at 3:47 AM, Stephan Ewen <[hidden email]> wrote: > Good idea, Chiwan! > > On Sat, Jun 13, 2015 at 6:25 PM, Chiwan Park <[hidden email]> wrote: > Hi. I think that we don’t need deploy all modules with scala variation. The pure java-based modules such as flink-java, flink-core, flink-optimizers, …, etc. don’t need to be deployed with scala version variation. We need only scala related modules such as flink-ml, flink-runtime, flink-scala, …, etc. with version variation. > > So we can reduce a number of deployed modules. > > Regards, > Chiwan Park > > > On Jun 13, 2015, at 9:17 AM, Robert Metzger <[hidden email]> wrote: > > > > I agree that we should ship a 2.11 build of Flink if downstream projects need that. > > > > The only thing that we should keep in mind when doing this is that the number of jars we're pushing to maven will explode (but that is fine) > > We have currently 46 maven modules and we would create 4 versions of each of the modules (hd1,hd2 x scala210,scala211) so we end up with 184 jars per release ;) > > > > The other big question that I have regarding this is how we want to name the modules. > > We could add the scala version to all the modules, like "flink-java_2.10", which would mean that users have to change a bit more when upgrading to the release supporting different scala versions. > > > > If we all agree on that, we can move on changing our maven setup. > > > > > > On Wed, Jun 10, 2015 at 8:34 AM, Kostas Tzoumas <[hidden email]> wrote: > > Please do ping this list if you encounter any problems with Flink during your project (you have done so already :-), but also if you find that the Flink API needs additions to map Pig well to Flink > > > > On Wed, Jun 10, 2015 at 3:47 PM, Philipp Goetze <[hidden email]> wrote: > > Done. Can be found here: https://issues.apache.org/jira/browse/FLINK-2200 > > > > Best Regards, > > Philipp > > > > > > > > On 10.06.2015 15:29, Chiwan Park wrote: > > But I think uploading Flink API with scala 2.11 to maven repository is nice idea. > > Could you create a JIRA issue? > > > > Regards, > > Chiwan Park > > > > On Jun 10, 2015, at 10:23 PM, Chiwan Park <[hidden email]> wrote: > > > > No. Currently, there are no Flink binaries with scala 2.11 which are downloadable. > > > > Regards, > > Chiwan Park > > > > On Jun 10, 2015, at 10:18 PM, Philipp Goetze <[hidden email]> wrote: > > > > Thank you Chiwan! > > > > I did not know the master has a 2.11 profile. > > > > But there is no pre-built Flink with 2.11, which I could refer to in sbt or maven, is it? > > > > Best Regards, > > Philipp > > > > On 10.06.2015 15:03, Chiwan Park wrote: > > Hi. You can build Flink with Scala 2.11 with scala-2.11 profile in master branch. > > `mvn clean install -DskipTests -P \!scala-2.10,scala-2.11` command builds Flink with Scala 2.11. > > > > Regards, > > Chiwan Park > > > > On Jun 10, 2015, at 9:56 PM, Flavio Pompermaier <[hidden email]> wrote: > > > > Nice! > > > > On 10 Jun 2015 14:49, "Philipp Goetze" <[hidden email]> wrote: > > Hi community! > > > > We started a new project called Piglet (https://github.com/ksattler/piglet). > > For that we use i.a. Flink as a backend. The project is based on Scala 2.11. Thus we need a 2.11 build of Flink. > > > > Until now we used the 2.11 branch of the stratosphere project and built Flink ourselves. Unfortunately this branch is not up-to-date. > > > > Do you have an official repository for Flink 0.9 (built with Scala 2.11)? > > > > Best Regards, > > Philipp > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
|
|
(adding dev@ to the conversation) Since flink-runtime also needs scala all modules are affected by this. I would vote for naming the Scala 2.10 version of flink modules without a suffix: flink-java flink-core flink-runtime And for the Scala 2.11 builds: flink-java_2.11 flink-core_2.11 flink-runtime_2.11 On Sun, Jun 21, 2015 at 9:31 AM, Chiwan Park <[hidden email]> wrote: I’m interested in working on this. :) I’ll assign to me. |
Re: Flink 0.9 built with Scala 2.11
|
|
+1, like that approach On Wed, Jul 1, 2015 at 10:28 AM, Robert Metzger <[hidden email]> wrote:
|
Re: Flink 0.9 built with Scala 2.11
|
|
On 01 Jul 2015, at 10:34, Stephan Ewen <[hidden email]> wrote: > +1, like that approach +1 I like that this is not breaking for non-Scala users :-) |
Re: Flink 0.9 built with Scala 2.11
|
|
Okay, I will apply this suggestion.
Regards, Chiwan Park > On Jul 1, 2015, at 5:41 PM, Ufuk Celebi <[hidden email]> wrote: > > > On 01 Jul 2015, at 10:34, Stephan Ewen <[hidden email]> wrote: > >> +1, like that approach > > +1 > > I like that this is not breaking for non-Scala users :-) |
«
Return to (DEPRECATED) Apache Flink User Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |