Questions about Flink RichSinkFunction constructor VS open()

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions about Flink RichSinkFunction constructor VS open()

Jiawei Wu
I have posted this question in StackOverflow: https://stackoverflow.com/questions/61334549/flink-richsinkfunction-constructor-vs-open

The question is: 
> Let's say I need to implemnt a custom sink using RichSinkFunction, and I need some variables like DBConnection in the sink. Where should I initialize the DBConnection? I see most of the articles init the DBConnection in the open() method, why not in the constructor?

A folow up questions is what kind of variables should be inited in constructor and what should be init in open()?

Reply | Threaded
Open this post in threaded view
|

Re: Questions about Flink RichSinkFunction constructor VS open()

Jark Wu-3
Hi Jiawei,

Yes, you should initialize connection in open() method, because constructor is only called in client side (where may can't connect to your database).
Besides, after construction, the RichSinkFunction instance will be serialized into binary and ship to server (TaskManagers) for deserialization and execution.
However, connection instances are mostly not serializable (e.g. jdbc connection). So you can't and shouldn't initialize it in constructor. open() is the suggested way to do that.

Best,
Jark

On Tue, 21 Apr 2020 at 09:26, Jiawei Wu <[hidden email]> wrote:
I have posted this question in StackOverflow: https://stackoverflow.com/questions/61334549/flink-richsinkfunction-constructor-vs-open

The question is: 
> Let's say I need to implemnt a custom sink using RichSinkFunction, and I need some variables like DBConnection in the sink. Where should I initialize the DBConnection? I see most of the articles init the DBConnection in the open() method, why not in the constructor?

A folow up questions is what kind of variables should be inited in constructor and what should be init in open()?