Re: Error while reading binary file

Posted by Saliya Ekanayake on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Error-while-reading-binary-file-tp4759p4779.html

Till,

I am still having trouble getting this to work. Here's my code (https://github.com/esaliya/flinkit)

String binaryFile = "src/main/resources/sample.bin";
SerializedInputFormat<ShortValue> sif = new SerializedInputFormat<>();
sif.setFilePath(binaryFile);
DataSet<ShortValue> ds = env.createInput(sif);
System.out.println(ds.count());

I still get the same error as shown below

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: The type returned by the input format could not be automatically determined. Please specify the TypeInformation of the produced type explicitly by using the 'createInput(InputFormat, TypeInformation)' method instead.

at org.apache.flink.api.java.ExecutionEnvironment.createInput(ExecutionEnvironment.java:511)

at org.saliya.flinkit.WordCount.main(WordCount.java:24)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

On Mon, Feb 8, 2016 at 5:42 AM, Till Rohrmann <[hidden email]> wrote:

Hi Saliya,

in order to set the file path for the SerializedInputFormat you first have to create it and then explicitly call setFilePath.

final SerializedInputFormat<Record> inputFormat = new SerializedInputFormat<Record>();
inputFormat.setFilePath(PATH_TO_FILE);

env.createInput(inputFormat, myTypeInfo);

Cheers,
Till

​

On Mon, Feb 8, 2016 at 7:00 AM, Saliya Ekanayake <[hidden email]> wrote:

Hi,

I was trying to read a simple binary file using SerializedInputFormat as suggested in a different thread, but encounters the following error. I tried to do what the exception suggests, but eventhough createInput() returns a DataSet object I couldn't find how to specify which file to read.

Any help is appreciated. The file I am trying to read is a simple binary file with containing java short values. Is there any example on reading binary files available?

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: The type returned by the input format could not be automatically determined. Please specify the TypeInformation of the produced type explicitly by using the 'createInput(InputFormat, TypeInformation)' method instead.

Thank you,

Saliya

--

Saliya Ekanayake

Ph.D. Candidate | Research Assistant

School of Informatics and Computing | Digital Science Center

Indiana University, Bloomington
Cell <a href="tel:[phone]" value="[phone]" target="_blank">812-391-4914
http://saliya.org

--