Class not found when deserializing

[Paolo Cristofanelli]

Hi, 

I am trying to write and read in a Kafka topic a user-defined class (that implements serializable, and all the fields are serializable). Everything works fine when I am executing the program in the IDE or with the mvn exec command.

When I try to execute the program in standalone mode I get the ClassNotFoundException.

More specifically I get the exception only during the deserialization parts :

@Override

public TheGlobalModel deserialize(byte[] message) throws IOException {

outlierDetection.network.TheGlobalModel model;

   model = (outlierDetection.network.TheGlobalModel) SerializationUtils.deserialize(message);

   

return model;

}

It seems that the problem lies in the deserialize method. If I remove it and simply return "new TheGlobalModel()" the exception is not thrown. I don´t understand why in this case the program seems to be aware of the existence of the class, I guess the problem is in the deserialize function.

I only know this method for sending a serializable class through Kafka, I would be glad to hear other ways. 

Thanks in advance for your time.

Cheers 

Paolo

[Fabian Hueske-2]

Hi Paolo,

do you get the ClassNotFoundException for TheGlobalModel or for another class?

Did you maybe forget to include SerializationUtils in the classpath?

Best, Fabian

2017-07-26 16:14 GMT+02:00 Paolo Cristofanelli <[hidden email]>:

Hi, 

I am trying to write and read in a Kafka topic a user-defined class (that implements serializable, and all the fields are serializable). Everything works fine when I am executing the program in the IDE or with the mvn exec command.

When I try to execute the program in standalone mode I get the ClassNotFoundException.

More specifically I get the exception only during the deserialization parts :

@Override

public TheGlobalModel deserialize(byte[] message) throws IOException {

outlierDetection.network.TheGlobalModel model;

   model = (outlierDetection.network.TheGlobalModel) SerializationUtils.deserialize(message);

   

return model;

}

It seems that the problem lies in the deserialize method. If I remove it and simply return "new TheGlobalModel()" the exception is not thrown. I don´t understand why in this case the program seems to be aware of the existence of the class, I guess the problem is in the deserialize function.

I only know this method for sending a serializable class through Kafka, I would be glad to hear other ways. 

Thanks in advance for your time.

Cheers 

Paolo

[Aljoscha Krettek]

Hi,

I think the problem here is that the SerializationUtils don't use the correct ClassLoader when deserialising. When running a Flink job on a cluster the user code (from the user-code jar) is only available in the user-code ClassLoader. If you can pass a ClassLoader to your deserialize method you can try passing the result of Thread.currentThread().getContextClassLoader().

Best,

Aljoscha

On 27. Jul 2017, at 09:52, Fabian Hueske <[hidden email]> wrote:

Hi Paolo,

do you get the ClassNotFoundException for TheGlobalModel or for another class?
Did you maybe forget to include SerializationUtils in the classpath?

Best, Fabian

2017-07-26 16:14 GMT+02:00 Paolo Cristofanelli <[hidden email]>:
Hi, 

I am trying to write and read in a Kafka topic a user-defined class (that implements serializable, and all the fields are serializable). Everything works fine when I am executing the program in the IDE or with the mvn exec command.
When I try to execute the program in standalone mode I get the ClassNotFoundException.

More specifically I get the exception only during the deserialization parts :

@Override
public TheGlobalModel deserialize(byte[] message) throws IOException {

outlierDetection.network.TheGlobalModel model;

    model = (outlierDetection.network.TheGlobalModel) SerializationUtils.deserialize(message);
    
return model;

}

It seems that the problem lies in the deserialize method. If I remove it and simply return "new TheGlobalModel()" the exception is not thrown. I don´t understand why in this case the program seems to be aware of the existence of the class, I guess the problem is in the deserialize function.

I only know this method for sending a serializable class through Kafka, I would be glad to hear other ways. 

Thanks in advance for your time.
Cheers 
Paolo