How do I call an algorithm written in C++ in Flink?

[苏喜 张]

The company has provided an algorithm written in C++, which has been packaged into a.so file. I have built a SpringBoot project, which uses JNI to operate the algorithm written in C++. Could you please tell me how to call it in Flink? Do i need to define operators, chains of operators?

 

[Yun Gao]

Hi Suxi,

Do you mean you want to call the algorithm in C++ ? If so, I think you could

do it the same with as you wrap it in SpringBoot project via JNI. I think you 

do not need to add a new operator, and you could use existing Flink API, and

you could load you library in open() and call the algorithm in the following 

processing method.

Best,

Yun

------------------Original Mail ------------------

Sender:苏喜 张 <[hidden email]>

Send Date:Mon Mar 8 14:12:02 2021

Recipients:[hidden email] <[hidden email]>

Subject:How do I call an algorithm written in C++ in Flink?

The company has provided an algorithm written in C++, which has been packaged into a.so file. I have built a SpringBoot project, which uses JNI to operate the algorithm written in C++. Could you please tell me how to call it in Flink? Do i need to define operators, chains of operators?

 

[Arvid Heise-4]

Hi Suxi,

to expand a bit on the answer of Yun: it depends on which kind of algorithm do you have. If you want to apply your C++ function to each record, then you can go Yun's way and use a RichMapFunction to load your library and invoke it for each record in map. If you need more records, then you need to group then accordingly.

If you give us more details, we can help you figure it out. I'd be especially interested to understand why you want to use Flink in the first place.

Best,

Arvid

On Mon, Mar 8, 2021 at 10:48 AM Yun Gao <[hidden email]> wrote:

Hi Suxi,

Do you mean you want to call the algorithm in C++ ? If so, I think you could

do it the same with as you wrap it in SpringBoot project via JNI. I think you 

do not need to add a new operator, and you could use existing Flink API, and

you could load you library in open() and call the algorithm in the following 

processing method.

Best,

Yun

------------------Original Mail ------------------

Sender:苏喜 张 <[hidden email]>

Send Date:Mon Mar 8 14:12:02 2021

Recipients:[hidden email] <[hidden email]>

Subject:How do I call an algorithm written in C++ in Flink?

The company has provided an algorithm written in C++, which has been packaged into a.so file. I have built a SpringBoot project, which uses JNI to operate the algorithm written in C++. Could you please tell me how to call it in Flink? Do i need to define operators, chains of operators?