No yarn option in self-built flink version
Locked
No yarn option in self-built flink version
 
|
Hello,
I am trying to build my own flink distribution with proper Cloudera dependencies. Reading https://ci.apache.org/projects/flink/flink-docs-stable/flinkDev/building.html I've done : git clone https://github.com/apache/flink cd flink git checkout tags/release-1.8.0 $MAVEN_HOME/bin/mvn clean install -DskipTests -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.16.1 -Dfast cd flink-dist $MAVEN_HOME/bin/mvn install -DskipTests -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.16.1 Everything was successful. However when running using flink-dist/target/flink-1.8.0-bin/flink-1.8.0 Running /bin/flink -h prints no yarn/Hadoop options. And running bin/flink run -m yarn-cluster -yn 4 -yjm 1024m -ytm 4096m ./examples/batch/WordCount.jar Prints : Could not build the program from JAR file. Am I missing something? Best regards, Arnaud ________________________________ L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender. |
Re: No yarn option in self-built flink version
|
|
Hey Arnaud, I think you need to active the Hadoop profile via -Pinclude-hadoop (the default was changed to not include Hadoop as far as I know). For more details, check out: If this does not work for you, I would wait for Chesnay's input (cc'd). – Ufuk On Fri, Jun 7, 2019 at 11:04 AM LINZ, Arnaud <[hidden email]> wrote: Hello, |
RE: No yarn option in self-built flink version
|
|
Hello, Thanks a lot, it works. However, may I suggest that you update the documentation page :
has absolutely no interest if you don’t include hadoop, that’s why I thought that
-Pvendor-repos was including the
-Pinclude-hadoop stated in the above paragraph… Arnaud De : Ufuk Celebi <[hidden email]>
Hey Arnaud, I think you need to active the Hadoop profile via -Pinclude-hadoop (the default was changed to not include Hadoop as far as I know). For more details, check out: If this does not work for you, I would wait for Chesnay's input (cc'd). – Ufuk On Fri, Jun 7, 2019 at 11:04 AM LINZ, Arnaud <[hidden email]> wrote:
|
Re: No yarn option in self-built flink version
|
|
I created https://issues.apache.org/jira/browse/FLINK-12813 for this. @Arnaud: Would you be interested in opening a PR with a fix? – Ufuk On Tue, Jun 11, 2019 at 11:10 AM LINZ, Arnaud <[hidden email]> wrote:
|
Re: No yarn option in self-built flink version
|
|
@Arnaud: Turns out those examples are on purpose. As Chesnay pointed out in the ticket, there are also cases where you don't necessarily want to bundle the Hadoop dependency, but still want to set a version like that. On Wed, Jun 12, 2019 at 9:32 AM Ufuk Celebi <[hidden email]> wrote:
|
| Free forum by Nabble | Edit this page |