Receiving context information through JobListener interface

[Barak Ben Nathan]

 

Hi all,

 

I am building an application that launches Flink Jobs and monitors them.

 

I want to use the JobListener interface to output job evemts to a Kafka Topic.

 

The problem:

In the application we have RuleId, i.e.  business logic identifier for the job,  and there’s JobId which is  the internal identifier generated by Flink.

I need the events emitted to Kafka to be partitioned by *RuleId*.

 

Is there a way to pass this kind of information to Flink and get it through the JobListener interface?

 

Thanks,

Barak

[Yangze Guo]

It seems that the JobListener interface could not expose such
information. Maybe you can set the RuleId as the jobName(or the suffix
of the jobName) of the application, then you can get the mappings of
jobId to jobName(RuleId) throw /jobs/overview.

[1] https://ci.apache.org/projects/flink/flink-docs-master/docs/ops/rest_api/#jobs-overview

Best,
Yangze Guo

On Sun, Apr 25, 2021 at 4:17 PM Barak Ben Nathan
<[hidden email]> wrote:

>
>
>
> Hi all,
>
>
>
> I am building an application that launches Flink Jobs and monitors them.
>
>
>
> I want to use the JobListener interface to output job evemts to a Kafka Topic.
>
>
>
> The problem:
>
> In the application we have RuleId, i.e.  business logic identifier for the job,  and there’s JobId which is  the internal identifier generated by Flink.
>
> I need the events emitted to Kafka to be partitioned by *RuleId*.
>
>
>
> Is there a way to pass this kind of information to Flink and get it through the JobListener interface?
>
>
>
> Thanks,
>
> Barak