Passing checkpoint lock object to StreamSourceContexts.getSourceContext after StreamTask.getCheckpointLock deprecation

[Yuval Itzchakov]

Hi,

I have an implementation of a custom source, which uses StreamSourceContexts.getSourceContext (https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamSourceContexts.java#L46) which has a checkpointLock argument that's used for watermark emission and possibly additional work.

Flink 1.10 marks StreamTask.getCheckpointLock as deprecated and points to using YieldingOperatorFactory instead.

Question is, with the above method now being deprecated, which object should we pass to getSourceContext for the required checkpointLock?

--

[Yuval Itzchakov]

One thing that had just occurred to me is that the context is later used to emit watermarks and send elements downstream using ctx.collect. Perhaps all these operations should now be switched to the new mailbox executor instead?

On Thu, Apr 9, 2020 at 8:52 PM Yuval Itzchakov <[hidden email]> wrote:

Hi,

I have an implementation of a custom source, which uses StreamSourceContexts.getSourceContext (https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamSourceContexts.java#L46) which has a checkpointLock argument that's used for watermark emission and possibly additional work.

Flink 1.10 marks StreamTask.getCheckpointLock as deprecated and points to using YieldingOperatorFactory instead.

Question is, with the above method now being deprecated, which object should we pass to getSourceContext for the required checkpointLock?

--

Best Regards,
Yuval Itzchakov.

--

[Arvid Heise-3]

Hi Yuval,

sorry for the late response on the mailing list.

I'm assuming your stackoverflow post is referring to the same question and would just cross-reference it here. [1]

[1] https://stackoverflow.com/questions/61141240/streamtask-getcheckpointlock-deprecation-and-custom-flink-sources

On Thu, Apr 9, 2020 at 7:56 PM Yuval Itzchakov <[hidden email]> wrote:

One thing that had just occurred to me is that the context is later used to emit watermarks and send elements downstream using ctx.collect. Perhaps all these operations should now be switched to the new mailbox executor instead?

On Thu, Apr 9, 2020 at 8:52 PM Yuval Itzchakov <[hidden email]> wrote:

Hi,

I have an implementation of a custom source, which uses StreamSourceContexts.getSourceContext (https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamSourceContexts.java#L46) which has a checkpointLock argument that's used for watermark emission and possibly additional work.

Flink 1.10 marks StreamTask.getCheckpointLock as deprecated and points to using YieldingOperatorFactory instead.

Question is, with the above method now being deprecated, which object should we pass to getSourceContext for the required checkpointLock?

--

Best Regards,
Yuval Itzchakov.

--

Best Regards,
Yuval Itzchakov.

--