Testing Flink Streaming applications - controlling the clock

[Rohit Agarwal]

Hi,

I am writing tests for my flink streaming application. I mostly use event-time. But there are some aspects which are still controlled by wall-clock time. For example, I am using AssignerWithPeriodicWatermarks and so watermarks are triggered based on wall-clock time. Similarly, checkpoints are also triggered based on wall-clock time. Is there a way I can manually control the clock which flink uses from my tests.

--

Rohit Agarwal

[Till Rohrmann]

Hi Rohit,

it depends a little bit on your tests. If you test individual operators you can use the AbstractStreamOperatorTestHarness class which allows to set the processing time via AbstractStreamOperatorTestHarness#setProcessingTime. You can also set the ProcessingTimeService used by a StreamTask via the StreamTask#setProcessingTimeService method.

If you have an IT case where you want to test the whole topology, there is, unfortunately, not much tooling around to set the test up so that you can easily define your own ProcessingTimeService. But if you like, then you could contribute it to Flink. It will be highly appreciated.

Cheers,
Till

​

On Fri, Dec 9, 2016 at 11:06 PM, Rohit Agarwal <[hidden email]> wrote:

Hi,

I am writing tests for my flink streaming application. I mostly use event-time. But there are some aspects which are still controlled by wall-clock time. For example, I am using AssignerWithPeriodicWatermarks and so watermarks are triggered based on wall-clock time. Similarly, checkpoints are also triggered based on wall-clock time. Is there a way I can manually control the clock which flink uses from my tests.

--

Rohit Agarwal