Best way to read property file in flink

[Janardhan Reddy]

We are running multiple flink jobs inside a yarn session.  For each flink job we have a separate property file. We are copying the property files to each node in the cluster before submitting the job.

Is there a better way to read the properties file?  Can we read it from hdfs or s3. Do we need to write custom code or flink already exposes an api to read properties from a file in hdfs or s3.

[Aljoscha Krettek]

Hi,

could you load the properties file when starting the application and add it to the user functions so that it would be serialized along with them? This way, you wouldn't have to ship the file to each node.

Cheers,

Aljoscha

On Wed, 29 Jun 2016 at 12:09 Janardhan Reddy <[hidden email]> wrote:

We are running multiple flink jobs inside a yarn session.  For each flink job we have a separate property file. We are copying the property files to each node in the cluster before submitting the job.

Is there a better way to read the properties file?  Can we read it from hdfs or s3. Do we need to write custom code or flink already exposes an api to read properties from a file in hdfs or s3.