Flink 1.10 classpath

[Richard Moorhead]

We're submitting jobs to a flink session running on YARN. Our yarn-site.xml does not have our mapreduce libs in yarn.application.classpath. Our application code has some provided dependencies from hadoop-mapreduce-client-core. Without editing our yarn-site, or adding those libraries to the flink lib dir, how can we extend our session's classpath?

[Yang Wang]

It is an interesting requirement. Usually, we suggest our users to add these dependencies in

lib directory or ship via --yarnship cli option.

If you really do not want to do this, I think you could ship an updated yarn-site.xml, set "yarn.application.classpath"

including the mapreduce jars. Then all the TaskManagers will have them in the classpath.

Best,

Yang

Richard Moorhead <[hidden email]> 于2020年9月26日周六 上午4:56写道：

We're submitting jobs to a flink session running on YARN. Our yarn-site.xml does not have our mapreduce libs in yarn.application.classpath. Our application code has some provided dependencies from hadoop-mapreduce-client-core. Without editing our yarn-site, or adding those libraries to the flink lib dir, how can we extend our session's classpath?