Re: Discarding header from CSV file

Posted by nsengupta on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Discarding-header-from-CSV-file-tp6474p6477.html

Chiwan and other Flinksters,

I am stuck with the following. Somehow, I am an unable to spot the error, if any! Please help.

I have this case class:
case class BuildingInformation(buildingID: Int, buildingManager: Int, buildingAge: Int, productID: String, country: String)

I intend to read from a CSV file which has a one-line header:
BuildingID,BuildingMgr,BuildingAge,HVACproduct,Country

I attempt to read the file in this manner:
private def readBuildingInfo(env: ExecutionEnvironment, inputPath: String) = {
env.readCsvFile [BuildingInformation] (
      inputPath,
      ignoreFirstLine = true,
      pojoFields = Array("buildingID","buildingManager","buildingAge","productID","country")
    )
}

Then, I use this function in the driver's main():
val envDefault = ExecutionEnvironment.getExecutionEnvironment
val buildings = readBuildingInfo(envDefault,"./SensorFiles/building.csv").collect().toList

The 'buildings' list is always empty!

I fail to figure out, why! I have checked that the path of the CSV file is correct and accessible. Also, I can read the same stuff by following the usual method of reading as a text-line, parsing the commas and creating the POJOs (case-classes).


-- Nirmalya