HashMap/HashSet Serialization Issue

[Charith Wickramarachchi]

Hi All, 

I am using flink-gelly and using a custom POJO type as the VertexValue/MessageType (I am using the vertex-centric model). The POJO contains HashMap/HashSet as members. While executing the job I am getting following Log message. 

17:50:53,582 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - No fields detected for class java.util.HashSet. Cannot be used as a PojoType. Will be handled as GenericType

17:50:53,583 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - class java.util.HashMap is not a valid POJO type

Is there a way to resolve this issue? 

Thanks,
Charith

--

[Chen Qin]

My understanding is HashMap doesn't work with Flink Native serialization framework, though I might be wrong.

This might worth reading

​https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization

-Chen​

On Fri, Jan 6, 2017 at 6:06 PM, Charith Wickramarachchi <[hidden email]> wrote:

Hi All, 

I am using flink-gelly and using a custom POJO type as the VertexValue/MessageType (I am using the vertex-centric model). The POJO contains HashMap/HashSet as members. While executing the job I am getting following Log message. 

17:50:53,582 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - No fields detected for class java.util.HashSet. Cannot be used as a PojoType. Will be handled as GenericType

17:50:53,583 INFO  org.apache.flink.api.java.typeutils.

​​

TypeExtractor             - class java.util.HashMap is not a valid POJO type

Is there a way to resolve this issue? 

Thanks,
Charith

--

[Charith Wickramarachchi]

Thanks very much. 

Regards,

Charith

On Fri, Jan 6, 2017 at 7:44 PM, Chen Qin <[hidden email]> wrote:

My understanding is HashMap doesn't work with Flink Native serialization framework, though I might be wrong.

This might worth reading

​https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization

-Chen​

On Fri, Jan 6, 2017 at 6:06 PM, Charith Wickramarachchi <[hidden email]> wrote:

Hi All, 

I am using flink-gelly and using a custom POJO type as the VertexValue/MessageType (I am using the vertex-centric model). The POJO contains HashMap/HashSet as members. While executing the job I am getting following Log message. 

17:50:53,582 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - No fields detected for class java.util.HashSet. Cannot be used as a PojoType. Will be handled as GenericType

17:50:53,583 INFO  org.apache.flink.api.java.typeutils.

​​

TypeExtractor             - class java.util.HashMap is not a valid POJO type

Is there a way to resolve this issue? 

Thanks,
Charith

--

--

[Jark Wu]

Hi Charith,

The LOG message has nothing to worry about. The HashMap and HashSet members in your POJO is not a PojoType, 

so they will be handled as GenericType, i.e. will be serialized/deserialized by Kryo. But your custom POJO type will still 

recognized as PojoType, i.e. will be serialized/deserialized by Flink serialization framework.

In a word, your custom POJO can be serialized/deserialized correctly. There is no issue in it.  :)

- Jark Wu 

在 2017年1月7日，下午12:02，Charith Wickramarachchi <[hidden email]> 写道：

Thanks very much. 

Regards,

Charith

On Fri, Jan 6, 2017 at 7:44 PM, Chen Qin <[hidden email]> wrote:

My understanding is HashMap doesn't work with Flink Native serialization framework, though I might be wrong.

This might worth reading

​https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization

-Chen​

On Fri, Jan 6, 2017 at 6:06 PM, Charith Wickramarachchi <[hidden email]> wrote:

Hi All, 

I am using flink-gelly and using a custom POJO type as the VertexValue/MessageType (I am using the vertex-centric model). The POJO contains HashMap/HashSet as members. While executing the job I am getting following Log message. 

17:50:53,582 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - No fields detected for class java.util.HashSet. Cannot be used as a PojoType. Will be handled as GenericType

17:50:53,583 INFO  org.apache.flink.api.java.typeutils.

​​

TypeExtractor             - class java.util.HashMap is not a valid POJO type

Is there a way to resolve this issue? 

Thanks,
Charith

--
Charith Dhanushka Wickramaarachchi

Tel  [phone]

Blog  http://charith.wickramaarachchi.org/

Twitter  @charithwiki

This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have
received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions

--

[Charith Wickramarachchi]

Thanks very much. 

Regards,

Charith

On Sat, Jan 7, 2017 at 3:42 AM, Jark Wu <[hidden email]> wrote:

Hi Charith,

The LOG message has nothing to worry about. The HashMap and HashSet members in your POJO is not a PojoType, 

so they will be handled as GenericType, i.e. will be serialized/deserialized by Kryo. But your custom POJO type will still 

recognized as PojoType, i.e. will be serialized/deserialized by Flink serialization framework.

In a word, your custom POJO can be serialized/deserialized correctly. There is no issue in it.  :)

- Jark Wu 

在 2017年1月7日，下午12:02，Charith Wickramarachchi <[hidden email]> 写道：

Thanks very much. 

Regards,

Charith

On Fri, Jan 6, 2017 at 7:44 PM, Chen Qin <[hidden email]> wrote:

My understanding is HashMap doesn't work with Flink Native serialization framework, though I might be wrong.

This might worth reading

​https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization

-Chen​

On Fri, Jan 6, 2017 at 6:06 PM, Charith Wickramarachchi <[hidden email]> wrote:

Hi All, 

I am using flink-gelly and using a custom POJO type as the VertexValue/MessageType (I am using the vertex-centric model). The POJO contains HashMap/HashSet as members. While executing the job I am getting following Log message. 

17:50:53,582 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - No fields detected for class java.util.HashSet. Cannot be used as a PojoType. Will be handled as GenericType

17:50:53,583 INFO  org.apache.flink.api.java.typeutils.

​​

TypeExtractor             - class java.util.HashMap is not a valid POJO type

Is there a way to resolve this issue? 

Thanks,
Charith

--
Charith Dhanushka Wickramaarachchi

Tel  [phone]

Blog  http://charith.wickramaarachchi.org/

Twitter  @charithwiki

This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have
received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions

--

--