Re: Get file metadata

Posted by Ronny Bräunlich on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Get-file-metadata-tp1891p1894.html

Hi Robert,

thank you for your quick answer.

Just one additional question:

When I use the ExecutionEnvironment like this: DataSource<String> files = env.readTextFile("file:///Users/me/path/to/file/dir“);

Shouldn’t it read all the files in dir? I have three .json files there but when I print the result, nothing is shown.

Cheers,

Ronny

Am 01.07.2015 um 11:35 schrieb Robert Metzger <[hidden email]>:

Hi Ronny,

check out this answer on SO: http://stackoverflow.com/questions/30599616/create-objects-from-input-files-in-apache-flink

It is a similar use case ... I guess you can get the metadata from the input split as well.

On Wed, Jul 1, 2015 at 11:30 AM, Ronny Bräunlich <[hidden email]> wrote:

Hello,

I want to read a file containing textfiles with Flink.
As I already found out I can simply point the environment to the directory and it will read all the files.
What I couldn’t find out is if it’s possible to keep the file metadata somehow.
Concrete, I need the timestamp, the filename and the file content. Is there a way to do this with the ExecutionEnvironment?

Cheers,
Ronny