Does flink have a plan to support flink sql udf of any language?

[Joshua Fan]

Hi,

Does the flink community have a plan to support flink sql udf in any language? For example, a udf in c or php. Because in my company, many developers do not know java or scala, they use c in their usual work.

Now we have a workaround to support this situation by creating a process running the c logic, and the process communicates with flink sql by a wrapper udf function.

So, here I have two questions:

1.  Does the flink community have a plan to support flink sql udf in any language?

2. If the answer to question 1 is no, is there any possibility to push our solution to flink? Is this udf process staff  in line with the ideas of the community aboud flink sql?

Thanks

Yours Sincerely

Josh

[Till Rohrmann]

Hi Josh,

Currently Flink supports Java, Scala and Python for defining SQL UDFs. There are no concrete plans to extend the set of supported languages at the moment.

In general, these kinds of contributions are always welcome. What we have to make sure is to see how it fits into the overall story. In order to assess this, I would recommend writing to [hidden email] to propose your idea and the solution proposal.

Cheers,

Till

On Thu, Dec 17, 2020 at 11:53 AM Joshua Fan <[hidden email]> wrote:

Hi,

Does the flink community have a plan to support flink sql udf in any language? For example, a udf in c or php. Because in my company, many developers do not know java or scala, they use c in their usual work.

Now we have a workaround to support this situation by creating a process running the c logic, and the process communicates with flink sql by a wrapper udf function.

So, here I have two questions:

1.  Does the flink community have a plan to support flink sql udf in any language?

2. If the answer to question 1 is no, is there any possibility to push our solution to flink? Is this udf process staff  in line with the ideas of the community aboud flink sql?

Thanks

Yours Sincerely

Josh

[Arvid Heise-3]

Just a quick idea for using C. I'd create a generic Java UDF and use JNA [1] in it to call a function written in C.

UDF could be something like invokeC("libraryName", "functionName", args) and would get JNA's Function [2] and invoke it accordingly.

[1] https://github.com/java-native-access/jna

[2] https://github.com/java-native-access/jna/blob/578220eff46dc0b808cab62f87408ceb33c34d37/src/com/sun/jna/Function.java#L101

On Thu, Dec 17, 2020 at 1:45 PM Till Rohrmann <[hidden email]> wrote:

Hi Josh,

Currently Flink supports Java, Scala and Python for defining SQL UDFs. There are no concrete plans to extend the set of supported languages at the moment.

In general, these kinds of contributions are always welcome. What we have to make sure is to see how it fits into the overall story. In order to assess this, I would recommend writing to [hidden email] to propose your idea and the solution proposal.

Cheers,

Till

On Thu, Dec 17, 2020 at 11:53 AM Joshua Fan <[hidden email]> wrote:

Hi,

Does the flink community have a plan to support flink sql udf in any language? For example, a udf in c or php. Because in my company, many developers do not know java or scala, they use c in their usual work.

Now we have a workaround to support this situation by creating a process running the c logic, and the process communicates with flink sql by a wrapper udf function.

So, here I have two questions:

1.  Does the flink community have a plan to support flink sql udf in any language?

2. If the answer to question 1 is no, is there any possibility to push our solution to flink? Is this udf process staff  in line with the ideas of the community aboud flink sql?

Thanks

Yours Sincerely

Josh

--