Config files content read

[Flavio Pompermaier]

Hi to all,

in many of my jobs I have to read a config file that can be either on local fs either on hdfs.

I'm looking for an intuitive API to read the content of such config files (JSON) before converting them to Java objects through jackson. Is there any Flink API to easily achieve this?

I really like something like 

• String content = FileSystem.get(myFileUri).readAsString() or
• String content = new Path(myFilePath).readAsString();

but at the moment the only solution I found is something like:

StringBuffer content = new StringBuffer();

Path path = new Path(myFilePath);

FSDataInputStream stream =  FileSystem.get(path.toUri()).open(path);

BufferedReader reader = new BufferedReader(new InputStreamReader(stream));

String line;

try {

while ((line = reader.readLine()) != null) {

content.append(line);

}

} finally {

reader.close();

}

String contentStr = content.toString();

Am I the only one that need such a feature?

Best,

Flavio

[Fabian Hueske-2]

Hi Flavio,

I don't think this is a feature that needs to go into Flink core.

To me it looks like this be implemented as a utility method by anybody who needs it without major effort.

Best, Fabian

2015-10-02 15:27 GMT+02:00 Flavio Pompermaier <[hidden email]>:

Hi to all,

in many of my jobs I have to read a config file that can be either on local fs either on hdfs.

I'm looking for an intuitive API to read the content of such config files (JSON) before converting them to Java objects through jackson. Is there any Flink API to easily achieve this?

I really like something like 

• String content = FileSystem.get(myFileUri).readAsString() or
• String content = new Path(myFilePath).readAsString();

but at the moment the only solution I found is something like:

StringBuffer content = new StringBuffer();

Path path = new Path(myFilePath);

FSDataInputStream stream =  FileSystem.get(path.toUri()).open(path);

BufferedReader reader = new BufferedReader(new InputStreamReader(stream));

String line;

try {

while ((line = reader.readLine()) != null) {

content.append(line);

}

} finally {

reader.close();

}

String contentStr = content.toString();

Am I the only one that need such a feature?

Best,

Flavio