Document an example pattern that makes sources and sinks pluggable in the production code for testing

[Hung]

Hi guys,

I found the testing part mentioned

make sources and sinks pluggable in your production code and inject special
test sources and test sinks in your tests.
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/testing.html#testing-flink-jobs

I think it would be useful to have a documented example as the section
*testing stateful operato*r does, which demonstrates by WindowOperatorTest
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/testing.html#unit-testing-stateful-or-timely-udfs--custom-operators

or, is there perhaps already a test that plugs sources and sinks?





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[vino yang]

Hi Hung,

Your suggestion is reasonable. Giving an example of a pluggable source and sink can make it more user-friendly, you can open a JIRA issue to see if there is anyone who wants to improve this.

IMO, it's not very difficult to implement it. Because the source and sink in Flink has two unified abstract interfaces: `SourceFunction` and `SinkFunction`. You can pass them into your method which builds your DAG(except source and sink). Then, you can provide different implementation of source and sink for production and testing purposes.

Best,

Vino

Hung <[hidden email]> 于2019年11月11日周一 下午8:44写道：

Hi guys,

I found the testing part mentioned

make sources and sinks pluggable in your production code and inject special
test sources and test sinks in your tests.
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/testing.html#testing-flink-jobs

I think it would be useful to have a documented example as the section
*testing stateful operato*r does, which demonstrates by WindowOperatorTest
https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/testing.html#unit-testing-stateful-or-timely-udfs--custom-operators

or, is there perhaps already a test that plugs sources and sinks?





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Piotr Nowojski-3]

Hi,

Doesn’t the included example `ExampleIntegrationTest` demonstrate the idea of

>  inject special test sources and test sinks in your tests.

?

Piotrek

> On 11 Nov 2019, at 13:44, Hung <[hidden email]> wrote:
>
> Hi guys,
>
> I found the testing part mentioned
>
> make sources and sinks pluggable in your production code and inject special
> test sources and test sinks in your tests.
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/testing.html#testing-flink-jobs
>
> I think it would be useful to have a documented example as the section
> *testing stateful operato*r does, which demonstrates by WindowOperatorTest
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/testing.html#unit-testing-stateful-or-timely-udfs--custom-operators
>
> or, is there perhaps already a test that plugs sources and sinks?
>
>
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/