Deploying Operator on concrete Task Manager

[KristoffSC]

Hi,
I have a question regarding job/operator deployment on Task Managers.

If I understand correctly, my job will be spitted into individual tasks,
which will be "deployed and executed" on particular task slot/s of Task
Manager (depending on parallelism level of course).

Lets imagine I have a Job that has:

1. Kafka Source with map to RawEvent
2. Enrichment that has AsyncCall + map to EnrichedEvent
3. Key stream with another Map/Process functions maybe with some Windowing.
4. Sink

I have a grid that has 1 Job Manager and 3 Task Managers nodes.
For a security reason one Task manager should be in a special network zone
(red zone).

The point 2 of my Job (Enrichment that has AsyncCall + map to EnrichedEvent)
should be executed on that particular node that is located in a Secured
Zone.

All other Operations should not be putted on this node.
Is there a way to configure this?


As an alternative I can see that we could have a separate job that will just
have a source, enrich, sink and this job will be "deployed" on this Secured
Task Manager. Or maybe we should have separate Flink cluster for this?




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Zhu Zhu]

Hi KristoffSC,

Flink does not support specifying the TM for tasks. 

So I think you need to launch a separate job to do the "AsyncCall + map" in the secured zone.

Thanks,

Zhu Zhu

KristoffSC <[hidden email]> 于2019年12月18日周三 下午8:04写道：

Hi,
I have a question regarding job/operator deployment on Task Managers.

If I understand correctly, my job will be spitted into individual tasks,
which will be "deployed and executed" on particular task slot/s of Task
Manager (depending on parallelism level of course).

Lets imagine I have a Job that has:

1. Kafka Source with map to RawEvent
2. Enrichment that has AsyncCall + map to EnrichedEvent
3. Key stream with another Map/Process functions maybe with some Windowing.
4. Sink

I have a grid that has 1 Job Manager and 3 Task Managers nodes.
For a security reason one Task manager should be in a special network zone
(red zone).

The point 2 of my Job (Enrichment that has AsyncCall + map to EnrichedEvent)
should be executed on that particular node that is located in a Secured
Zone.

All other Operations should not be putted on this node.
Is there a way to configure this?


As an alternative I can see that we could have a separate job that will just
have a source, enrich, sink and this job will be "deployed" on this Secured
Task Manager. Or maybe we should have separate Flink cluster for this?




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[KristoffSC]

Hi,
thanks for the replay.

Just to clarify, I will have to have *a new Flink Cluster* (Job Manager and
Task Manager) that will run in the secure zone which will ran the
AsyncEnrich Job right?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/