Re: [Flink Unit Tests] Unit test for Flink streaming codes

Posted by Arvid Heise-3 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-Unit-Tests-Unit-test-for-Flink-streaming-codes-tp37119p37134.html

Hi Vijay,

Any unit test of Flink operators is actually an IT case as it involves a large portion of the stack. A real unit test, would be over a factored out logic class.

Similar to Niels, I'd recommend to use simple sources (env.fromElements) and sinks to inject the data and retrieve the data and put the logic under test in the middle. That may be a part of your pipeline or even the whole pipeline.

If you want to have some scala inspiration, have a look at:

https://github.com/apache/flink/blob/5f0183fe79d10ac36101f60f2589062a39630f96/flink-table/flink-table-planner-blink/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/SortITCase.scala#L56-L82 . It's on table API but should be quite easy to translate to datastream API if needed.

On Sat, Aug 1, 2020 at 4:03 PM Niels Basjes <[hidden email]> wrote:

No, I only have Java.

On Fri, 31 Jul 2020, 21:57 Vijayendra Yadav, <[hidden email]> wrote:

Thank You Niels. Would you have something for the scala object class.  Say for example if I want to implement a unit test ( not integration test) for below code or similar  :

https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples/socket/SocketWindowWordCount.scala

Regards,

Vijay

On Fri, Jul 31, 2020 at 12:22 PM Niels Basjes <[hidden email]> wrote:

Does this test in one of my own projects do what you are looking for?

https://github.com/nielsbasjes/yauaa/blob/1e1ceb85c507134614186e3e60952112a2daabff/udfs/flink/src/test/java/nl/basjes/parse/useragent/flink/TestUserAgentAnalysisMapperClass.java#L107

On Fri, 31 Jul 2020, 20:20 Vijayendra Yadav, <[hidden email]> wrote:

Hi Team,

Looking for some help and reference code / material to implement unit tests of possible scenarios in Flink streaming Code that should assert specific cases.

Regards,

Vijay

--