Hi Soheil,
I think I knew what you meant by passing a function's name B to another method A: assuming in function "A" you are trying to dynamically load another function "B" based on either (1) some characteristic of the message in your data stream, or (2) some configuration during start up.
And I further assume here "method A" is part of your job graph, otherwise it would be as easy to just get rid of it.
You can definitely include "function B" in your job JAR and ship it over to the execution cluster; and then simple use Classloader to load by name and instantiate. However I am not sure this is the best programming paradigm.
To add to Hequn's comment [1], some alternatives you can try to eliminate the need to load function "B" during execution:
- Consolidate all of the "function B variances" into one function, and use a filterBy or a keyBy [2] for choosing different code path upon message arrival;
- You can use broadcast state [3] to assist you when choosing different code path if it depends on some external configurations;
Thanks,
Rong
Hi Soheil,
What do you mean by "give it a written function's name B" and "function A will apply function B"? Do you mean function A override B?
Perhaps the DataStream api guide[1] may gives you some guidance.
Best, Hequn
Locked
