Details of the downsides of “falling back” to Kyro rather than using Flink’s built in serde

[kb]

Hi all,

I was looking at
https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html,
and https://www.youtube.com/watch?v=euFMWFDThiE to improve our state
management and back pressure.

Both of these resources mention ensuring objects used in the flow are valid
POJOs to avoid “falling back” to Kyro.

Are the specific downsides listed anywhere?

Thanks
kb




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Konstantin Knauf-2]

Hi Kevin,

there are no performance downsides to using Flink POJOs. You are just limited in the types you can use (e.g. you can not use Collections).

In Flink 1.7 you might want to use Avro (SpecificRecord) for your state objects to benefit from Flink's built-in state schema evolution capabilities. As of Flink 1.8, state schema evolution also works out-of-the-box with Flink POJOs. So, this potential downside does not exist going forward. 

[1] contains a very basic benchmark comparing different serializers in Flink. The results are not representative and the absolute values don't have any meaning, but it should give you an idea of the potential differences.

Hope this helps,

Konstantin

[1] https://github.com/ververica/flink-training-troubleshooting/blob/master/slides/2019-04-01%20_%20OPERATIONS%20_%206%20SERIALIZATION.pdf

On Tue, Apr 9, 2019 at 6:11 PM kb <[hidden email]> wrote:

Hi all,

I was looking at
https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html,
and https://www.youtube.com/watch?v=euFMWFDThiE to improve our state
management and back pressure.

Both of these resources mention ensuring objects used in the flow are valid
POJOs to avoid “falling back” to Kyro.

Are the specific downsides listed anywhere?

Thanks
kb




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

--

[kb]

Hi Konstantin,

Thank you! This is exactly what I was looking for.

Thanks
Kevin



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/