(DEPRECATED) Apache Flink User Mailing List archive.

Re: Error while reading binary file

Posted by Fabian Hueske-2 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Error-while-reading-binary-file-tp4759p4783.html

Hi,

please try to replace
DataSet<ShortValue> ds = env.createInput(sif);

by
DataSet<ShortValue> ds = env.createInput(sif, ValueTypeInfo.SHORT_VALUE_TYPE_INFO);

Best, Fabian

2016-02-08 19:33 GMT+01:00 Saliya Ekanayake <[hidden email]>:

Till,

I am still having trouble getting this to work. Here's my code (https://github.com/esaliya/flinkit)
String binaryFile = "src/main/resources/sample.bin";
SerializedInputFormat<ShortValue> sif = new SerializedInputFormat<>();
sif.setFilePath(binaryFile);
DataSet<ShortValue> ds = env.createInput(sif);
System.out.println(ds.count());
I still get the same error as shown below

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: The type returned by the input format could not be automatically determined. Please specify the TypeInformation of the produced type explicitly by using the 'createInput(InputFormat, TypeInformation)' method instead.
at org.apache.flink.api.java.ExecutionEnvironment.createInput(ExecutionEnvironment.java:511)
at org.saliya.flinkit.WordCount.main(WordCount.java:24)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
On Mon, Feb 8, 2016 at 5:42 AM, Till Rohrmann <[hidden email]> wrote:
Hi Saliya,

in order to set the file path for the SerializedInputFormat you first have to create it and then explicitly call setFilePath.
final SerializedInputFormat<Record> inputFormat = new SerializedInputFormat<Record>();
inputFormat.setFilePath(PATH_TO_FILE);

env.createInput(inputFormat, myTypeInfo);
Cheers,
Till
On Mon, Feb 8, 2016 at 7:00 AM, Saliya Ekanayake <[hidden email]> wrote:
Hi,

I was trying to read a simple binary file using SerializedInputFormat as suggested in a different thread, but encounters the following error. I tried to do what the exception suggests, but eventhough createInput() returns a DataSet object I couldn't find how to specify which file to read.

Any help is appreciated. The file I am trying to read is a simple binary file with containing java short values. Is there any example on reading binary files available?

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: The type returned by the input format could not be automatically determined. Please specify the TypeInformation of the produced type explicitly by using the 'createInput(InputFormat, TypeInformation)' method instead.

Thank you,
Saliya

--
Saliya Ekanayake
Ph.D. Candidate | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington
Cell <a href="tel:812-391-4914" value="+18123914914" target="_blank">812-391-4914
http://saliya.org
--
Saliya Ekanayake
Ph.D. Candidate | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington
Cell <a href="tel:812-391-4914" value="+18123914914" target="_blank">812-391-4914
http://saliya.org