Implementation of ElasticsearchSinkFunction, how to handle class level variables

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Implementation of ElasticsearchSinkFunction, how to handle class level variables

Jayant Ameta
Hi,
I'm trying to integrate ElasticsearchSink in my pipeline. The example shows using Anonymous class which implements ElasticsearchSinkFunction. This is passed as a constructor argument to another anonymous class which extends ElasticsearchSink.

Can I create a separate class instead of using anonymous classes?
If I do so, I'll have to have index and type as class level fields. How is state maintenance done for class level fields? Do I need to mark the fields as transient, and implement CheckpointedFunction?

Thanks,
Jayant
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of ElasticsearchSinkFunction, how to handle class level variables

Tzu-Li (Gordon) Tai
Hi Jayant,

Yes, you don’t have to use an anonymous class for the sink function. An actual separate class works just as fine.
The class fields don’t need to be marked as transient or checkpointed, since they should just be constants that come with instantiation of the sink function, or could even be static fields, for example.

Cheers,
Gordon

On 8 June 2018 at 12:55:57 PM, Jayant Ameta ([hidden email]) wrote:

Hi,
I'm trying to integrate ElasticsearchSink in my pipeline. The example shows using Anonymous class which implements ElasticsearchSinkFunction. This is passed as a constructor argument to another anonymous class which extends ElasticsearchSink.

Can I create a separate class instead of using anonymous classes?
If I do so, I'll have to have index and type as class level fields. How is state maintenance done for class level fields? Do I need to mark the fields as transient, and implement CheckpointedFunction?

Thanks,
Jayant