[flink :: connected-streams :: integration-tests]

[Rinat]

Hi mates, I got some troubles with the implementation of integration tests for the job, based on connected streams.

It has the following logic:

1. I got two streams, first one is a stream of rules, and another one is a stream of events
2. to apply events on each rule, I’ve implemented a KeyedBroadcastProcessFunction , that broadcasts the set of rules, received from the stream
3. in the processBroadcastElement 
   I'm updating the broadcast state
4. in the processElement method I’m evaluating all rules, from the broadcast state, using input event

 

I would like to implement an integration test, that will send a rule into pipeline and then, when it’ll be added to the broadcast state, send an event and check, that the output item is a result of rule evaluation.

For the test needs, I’ve replaced source functions with FromElementsFunction, that gives me a bounded streams with pre-defined items that will be passed over pipeline. 

But I couldn’t understand, how I can pass rules before sending events, maybe you know some practises or workarounds, how to achieve such behaviour, or maybe I’m doing something wrong ?

Another approach, that is also suitable for me, is to initialize a broadcast state manually on job startup, but I still can’t find the way hotw to do that. 

Thx for your advices.

Sincerely yours,

Rinat Sharipov

Software Engineer at 1DMP CORE Team

email: [hidden email]

mobile: [phone]

CleverDATA

make your data clever