How can I set configuration of process function from job's main?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

How can I set configuration of process function from job's main?

Main Frame
Hi guys! Iam newbie in flink and I have probably silly question about streaming api.

So for the instance:

I trying to apply SomeProcessFunction to stream1


DataStream<ObjectNode> stream2 = stream1.process(new MyProcessFunction()).name("Ingest data»);


I have created package-private class with MyProcessFunction which extends ProcessFunction.

class MyProcessFunction extends ProcessFunction<ObjectNode, ObjectNode> {

    @Override
    public void open(Configuration parameters) throws Exception {
         ...
    }
    …
}

How can I set parameters variable before execution of MyProcessFunction?

———————————————

Timofeev Dmitry
VoIP Engineer
Open source telephony solutions
Skype: itsroot
Linkedin: https://www.linkedin.com/in/itsroot

Reply | Threaded
Open this post in threaded view
|

Re: How can I set configuration of process function from job's main?

Timo Walther
Hi,

the configuration parameter is just legacy API. You can simply pass any
serializable object to the constructor of your process function.

Regards,
Timo


Am 29.03.18 um 20:38 schrieb Main Frame:

> Hi guys! Iam newbie in flink and I have probably silly question about streaming api.
>
> So for the instance:
>
> I trying to apply SomeProcessFunction to stream1
>
> …
> DataStream<ObjectNode> stream2 = stream1.process(new MyProcessFunction()).name("Ingest data»);
> …
>
> I have created package-private class with MyProcessFunction which extends ProcessFunction.
>
> class MyProcessFunction extends ProcessFunction<ObjectNode, ObjectNode> {
>
>      @Override
>      public void open(Configuration parameters) throws Exception {
>           ...
>      }
>      …
> }
>
> How can I set parameters variable before execution of MyProcessFunction?
>
> ———————————————
>
> Timofeev Dmitry
> VoIP Engineer
> Open source telephony solutions
> Skype: itsroot
> Linkedin: https://www.linkedin.com/in/itsroot