Get get file name when reading from files? Or assign timestamps from file creation time?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Get get file name when reading from files? Or assign timestamps from file creation time?

joshlemer
Hey there, is it possible to somehow read the filename of elements that are read from `env.readFile`? In our case, the date of creation is encoded in the file name. 

Otherwise, maybe it is possible to assign timestamps somehow by the file's creation time directly?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Get get file name when reading from files? Or assign timestamps from file creation time?

Fabian Hueske-2
Hi Josh,

FileInputFormat stores the currently read FileInputSplit in a protected variable FileInputFormat.currentSplit.
You can override your FileInputFormat and access the path of the read file from the FileInputSplit in the method that emits the records from the format.

Best,
Fabian


2018-04-06 17:46 GMT+02:00 Josh Lemer <[hidden email]>:
Hey there, is it possible to somehow read the filename of elements that are read from `env.readFile`? In our case, the date of creation is encoded in the file name. 

Otherwise, maybe it is possible to assign timestamps somehow by the file's creation time directly?

Thanks!