Re: Recursive Traversal of the Input Path Directory, Not working

Posted by Stefan Richter on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Recursive-Traversal-of-the-Input-Path-Directory-Not-working-tp13926p13930.html

Hi,

I am not sure I am getting the problem right: the code works if you use a file name, but it does not work for directories? What exactly is not working? Do you get any exceptions?

Best,

Stefan

Am 22.06.2017 um 17:01 schrieb Adarsh Jain <[hidden email]>:

Hi,

I am trying to use "Recursive Traversal of the Input Path Directory" in Flink 1.3 using scala. Snippet of my code below. If I give exact file name it is working fine. Ref https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/batch/index.html

import org.apache.flink.api.java.utils.ParameterTool

import org.apache.flink.api.java.{DataSet, ExecutionEnvironment}

import org.apache.flink.configuration.Configuration

val config = new Configuration

    config.setBoolean("recursive.file.enumeration",true)

val featuresSource: String = "file:///Users/adarsh/Documents/testData/featurecsv/31c710ac40/2017/06/22"

val testInput = env.readTextFile(featuresSource).withParameters(config)

testInput.print()

Please guide how to fix this.

Regards,

Adarsh