Resolving dependencies when using sbt
Locked
Resolving dependencies when using sbt
 
|
Hello, Earlier today, I was able to get a Flink cluster running, and successfully ran the wordcount jar file in the examples folder. I then tried to compile the WordCount example using sbt found here: https://ci.apache.org/projects/flink/flink-docs-master/quickstart/scala_api_quickstart.html#alternative-build-tools-sbt and could not resolve the sbt dependencies listed on that page. More specifically, I did the following steps: -- Created a directory with the WordCountJob.scala file copied and pasted from the above link -- Created a build.sbt file (scala -version told me that I have 2.9.2 installed) name := "sbt-test" version := "1.0" scalaVersion := "2.9.2" libraryDependencies ++= Seq( "org.apache.flink" % "flink-scala" % "0.10-SNAPSHOT", "org.apache.flink" % "flink-clients" % "0.10-SNAPSHOT") fork in run := true I then ran sbt > compile and got the error: :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: org.apache.flink#flink-scala;0.10-SNAPSHOT: not found [warn] :: org.apache.flink#flink-clients;0.10-SNAPSHOT: not found [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] I then installed Fink on my local Mac laptop, and the same problem: I was able to run the wordcount jar file, but had the same issue when trying to build using sbt Thank you, Dan |
Re: Resolving dependencies when using sbt
|
|
nevermind on this issue, based on this (a whole different issue with Kafka): https://issues.apache.org/jira/browse/FLINK-2408 I saw the following (instead of 0.10-SNAPSHOT in build.sbt), and the error message went away: val flinkVersion = "0.9.0" libraryDependencies ++= Seq("org.apache.flink" % "flink-scala" % flinkVersion, "org.apache.flink" % "flink-clients" % flinkVersion) I'm doing more debugging now with the actual wordcount.scala file -- so I might ask another question about that here -- but thought I'd mention that that error message is no longer popping up. Dan On Sun, Sep 13, 2015 at 3:53 PM, Daniel Blazevski <[hidden email]> wrote:
|
Re: Resolving dependencies when using sbt
|
|
If you want to use Flink 0.10-SNAPSHOT you can add the Apache Snapshot repository, in sbt:
resolvers += "apache-snapshot" at "https://repository.apache.org/content/repositories/snapshots/“ It would probably be better to use scala 2.10 rather than 2.9.
Giancarlo
On 13 Sep 2015, at 21:36, Daniel Blazevski <[hidden email]> wrote: |
Re: Resolving dependencies when using sbt
|
|
Hi, Giancarlo is correct about the Scala version. The provided Flink libraries should only work with 2.10. I'm actually wondering why it seems to be working for you with 2.9.x. Cheers, Aljoscha On Mon, 14 Sep 2015 at 11:07 Giancarlo Pagano <[hidden email]> wrote:
|
Re: Resolving dependencies when using sbt
|
|
Thanks for the feedback. I have another question about building using sbt: how can one go from import statements to figuring out library dependencies? Would be nice to write small programs w/o an IDE and be able to go from the import statements to appending the library dependency list in a .sbt file to build. For example, I would like to test algorithm in the machine learning library ( namely: https://github.com/chiwanpark/flink/blob/FLINK-1745/flink-staging/flink-ml/src/main/scala/org/apache/flink/ml/nn/KNN.scala ), and when I run `sbt package`, I get errors since I need to add more dependencies in my build.sbt file. I can try to get an IDE and have this automated, but I would like to be able to add the right dependencies quickly on the command line. Thanks, Dan On Mon, Sep 14, 2015 at 5:51 AM, Aljoscha Krettek <[hidden email]> wrote:
|
Re: Resolving dependencies when using sbt
|
|
BTW, I previously had an already built version of Flink, am now building from scratch so that I can use an IDE -- Got Scala 2.10.5, and building Flink from source will naturally make the
whole process of getting the dependencies right more straight-forward. Decided will need to do this eventually anyway since I'm planning to contribute to Flink. Cheers, Dan On Mon, Sep 14, 2015 at 3:01 PM, Daniel Blazevski <[hidden email]> wrote:
|
Re: Resolving dependencies when using sbt
|
|
Answering to our other mail: I think getting from import statements to dependencies is not straightworward easy. Usually if you add flink-core and flink-java or flink-scala, plus maybe flink-streaming-core you have what you need to program (transitive libraries will be resolved by Maven / sbt) and if you need to execute the program, add "flink-clients"... Greetings, Stephan On Mon, Sep 14, 2015 at 11:11 PM, Daniel Blazevski <[hidden email]> wrote:
|
«
Return to (DEPRECATED) Apache Flink User Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |