start-cluster.sh issue

[Lior Amar]

Hi,

I am new here.

My name is Lior and I am working at Parallel Machines.

Was assigned recently to work on Flink run/use/improve :-)

I am using the FLINK_CONF_DIR environment variable to pass the config location

to the start-cluster.sh (e.g. env FLINK_CONF_DIR=/tmp/parallelmachine/lior/flink/conf)

The directory contain the generated config for the setup and is avail on multiple machines.

The issue is that when the start-cluster.sh is running the tasknmanager.sh (via ssh)

the FLINK_CONF_DIR is not passed via the ssh.

Note: that the taskmanager.sh is making use of this environment variable (line 96).

Is this a bug?

or expected?

Should I open a Jira?

Regards

--lior

[Greg Hogan]

Hi Lior,

Try adding this to your flink-conf.yaml:
  env.ssh.opts: FLINK_CONF_DIR=/tmp/parallelmachine/lior/flink/conf

I think this is expected and not a bug (setting FLINK_CONF_DIR from the environment is supported for YARN). Please do file a JIRA for this feature as I think it would be a nice improvement.

I created FLINK-5680 to add env.ssh.opts to the configuration documentation [1].

[1] https://ci.apache.org/projects/flink/flink-docs-master/setup/config.html

Greg

On Thu, Jan 26, 2017 at 12:10 PM, Lior Amar <[hidden email]> wrote:

Hi,

I am new here.

My name is Lior and I am working at Parallel Machines.

Was assigned recently to work on Flink run/use/improve :-)

I am using the FLINK_CONF_DIR environment variable to pass the config location

to the start-cluster.sh (e.g. env FLINK_CONF_DIR=/tmp/parallelmachine/lior/flink/conf)

The directory contain the generated config for the setup and is avail on multiple machines.

The issue is that when the start-cluster.sh is running the tasknmanager.sh (via ssh)

the FLINK_CONF_DIR is not passed via the ssh.

Note: that the taskmanager.sh is making use of this environment variable (line 96).

Is this a bug?

or expected?

Should I open a Jira?

Regards

--lior