Flink build-in functions

[sunfulin]

As far as I can see, the latest flink version does not have a fullfilled support for blink build-in functions. Many date functions and string functions can not be used in Flink. I want to know that when shall we use flink just as to use blink in the same way.  

 

[Wyatt Chun]

They are two different systems for differentiated usage. For your question, why don’t give a direct try on Blink?

Regards 

On Tue, Feb 4, 2020 at 10:02 AM sunfulin <[hidden email]> wrote:

As far as I can see, the latest flink version does not have a fullfilled support for blink build-in functions. Many date functions and string functions can not be used in Flink. I want to know that when shall we use flink just as to use blink in the same way.  

 

[Jingsong Li]

Hi Sunfulin,

Did you use blink-planner? What functions are missing?

Best,

Jingsong Lee

On Tue, Feb 4, 2020 at 12:23 PM Wyatt Chun <[hidden email]> wrote:

They are two different systems for differentiated usage. For your question, why don’t give a direct try on Blink?

Regards 

On Tue, Feb 4, 2020 at 10:02 AM sunfulin <[hidden email]> wrote:

As far as I can see, the latest flink version does not have a fullfilled support for blink build-in functions. Many date functions and string functions can not be used in Flink. I want to know that when shall we use flink just as to use blink in the same way.  

 

--

[sunfulin]

Hi, Jingsong

Yep, I'm using blink planner as the following approach.

EnvironmentSettings bsSettings = EnvironmentSettings.newInstance().useBlinkPlanner().inStreamingMode().build();

initPack.tableEnv = org.apache.flink.table.api.java.StreamTableEnvironment.create(initPack.env, bsSettings);

While running the job, I can see one log from console. 

[main] INFO  org.apache.flink.table.module.ModuleManager  - Cannot find FunctionDefinition date_fromat from any loaded modules

Why Flink cannot load the function definition? From what I can see, Flink 1.10 with blink-planner shall support this kind of function. 

At 2020-02-04 12:35:12, "Jingsong Li" <[hidden email]> wrote:

Hi Sunfulin,

Did you use blink-planner? What functions are missing?

Best,

Jingsong Lee

On Tue, Feb 4, 2020 at 12:23 PM Wyatt Chun <[hidden email]> wrote:

They are two different systems for differentiated usage. For your question, why don’t give a direct try on Blink?

Regards 

On Tue, Feb 4, 2020 at 10:02 AM sunfulin <[hidden email]> wrote:

As far as I can see, the latest flink version does not have a fullfilled support for blink build-in functions. Many date functions and string functions can not be used in Flink. I want to know that when shall we use flink just as to use blink in the same way.  

 

--

 

[Jingsong Li]

Hi sunfulin,

When merging blink, we combed the semantics of all functions at present, and removed a few functions whose semantics are not clearly defined at present. "date_format" should be one of the victim.
You can implement your UDF.

And you can create a JIRA to support "date_format" too.

Best,

Jingsong Lee

On Thu, Feb 6, 2020 at 10:32 AM sunfulin <[hidden email]> wrote:

Hi, Jingsong

Yep, I'm using blink planner as the following approach.

EnvironmentSettings bsSettings = EnvironmentSettings.newInstance().useBlinkPlanner().inStreamingMode().build();

initPack.tableEnv = org.apache.flink.table.api.java.StreamTableEnvironment.create(initPack.env, bsSettings);

While running the job, I can see one log from console. 

[main] INFO  org.apache.flink.table.module.ModuleManager  - Cannot find FunctionDefinition date_fromat from any loaded modules

Why Flink cannot load the function definition? From what I can see, Flink 1.10 with blink-planner shall support this kind of function. 

At 2020-02-04 12:35:12, "Jingsong Li" <[hidden email]> wrote:

Hi Sunfulin,

Did you use blink-planner? What functions are missing?

Best,

Jingsong Lee

On Tue, Feb 4, 2020 at 12:23 PM Wyatt Chun <[hidden email]> wrote:

They are two different systems for differentiated usage. For your question, why don’t give a direct try on Blink?

Regards 

On Tue, Feb 4, 2020 at 10:02 AM sunfulin <[hidden email]> wrote:

As far as I can see, the latest flink version does not have a fullfilled support for blink build-in functions. Many date functions and string functions can not be used in Flink. I want to know that when shall we use flink just as to use blink in the same way.  

 

--

Best, Jingsong Lee

 

--