Re: How to retrieve values from yarn.taskmanager.env in a Job?

Posted by Chesnay Schepler on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/How-to-retrieve-values-from-yarn-taskmanager-env-in-a-Job-tp10566p10579.html

Hello,

can you clarify one small thing for me: Do you want to access this parameter when you define the plan
(aka when you call methods on the StreamExecutionEnvironment or DataStream instances)
or from within your functions/operators?

Regards,
Chesnay Schepler


On 12.12.2016 14:21, Till Rohrmann wrote:

Hi Shannon,

have you tried accessing the environment variables via System.getenv()? This should give you a map of string-string key value pairs where the key is the environment variable name.

If your values are not set in the returned map, then this indicates a bug in Flink and it would be great if you could open a JIRA issue.

Cheers,
Till

​

On Fri, Dec 9, 2016 at 7:33 PM, Shannon Carey <[hidden email]> wrote:

This thread http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/passing-environment-variables-to-flink-program-td3337.html describes the impetus for the addition of yarn.taskmanager.env.

I have configured a value within yarn.taskmanager.env, and I see it appearing in the Flink web UI in the list underneath Job Manager -> Configuration. However, I can't figure out how to retrieve the value from within a Flink job. It doesn't appear in the environment, the system properties, or my ParameterTool instance, and I can't figure out how I would get to it via the StreamExecutionEnvironment. Can anyone point me in the right direction?

All I want to do is inform my Flink jobs which environment they're running on, so that programmers don't have to specify the environment as a job parameter every time they run it. I also see that there is a "env.java.opts" configuration… does that work in YARN apps (would my jobs be able to see it?)

Thanks!

Shannon