Flink SQL as DSL for flink CEP

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink SQL as DSL for flink CEP

Dipanjan Mazumder
Hi,

    Can we say that Flink SQL is kind of a DSL overlay on flink CEP , i mean i need a DSL for flink CEP , so that i can decouple the CEP rules from code and pass them dynamically to be applied on different data streams. Flink CEP doen't have any DSL implementation , so is it that Flink SQL can be used for the same purpose , where flink SQL has integration with Flink CEP and underlying flink SQL uses flink CEP for data processing. If that is the case can we use flink SQL as a streaming CEP DSL.


Regards
Dipanjan
Reply | Threaded
Open this post in threaded view
|

Re: Flink SQL as DSL for flink CEP

rmetzger0
Hi Dipanjan,

Using Flink SQL's MATCH_RECOGNIZE operator is certainly a good idea if you are looking for a non-programmatic way to do CEP with Flink.

On Wed, Jun 16, 2021 at 6:44 AM Dipanjan Mazumder <[hidden email]> wrote:
Hi,

    Can we say that Flink SQL is kind of a DSL overlay on flink CEP , i mean i need a DSL for flink CEP , so that i can decouple the CEP rules from code and pass them dynamically to be applied on different data streams. Flink CEP doen't have any DSL implementation , so is it that Flink SQL can be used for the same purpose , where flink SQL has integration with Flink CEP and underlying flink SQL uses flink CEP for data processing. If that is the case can we use flink SQL as a streaming CEP DSL.


Regards
Dipanjan
Reply | Threaded
Open this post in threaded view
|

Re: Flink SQL as DSL for flink CEP

JING ZHANG
In reply to this post by Dipanjan Mazumder
Hi Dipanjan,
I'm not an expert on Flink CEP, however I would like to share my immature ideas, hope it helps.
Flink provides CEP library[1] which is a programmatic library which allows user define patterns based on pattern API.
Besides, Flink allows user define patterns by MATCH_RECOGNIZE clause in SQL [2].
I think it's the right decision to use Flink SQL as a streaming CEP DSL. Please note most of the important features about CEP are already supported by Flink SQL, while a few features are not supported yet, you could view the document [2] to check.



Best,
JING ZHANG

Dipanjan Mazumder <[hidden email]> 于2021年6月16日周三 下午12:44写道:
Hi,

    Can we say that Flink SQL is kind of a DSL overlay on flink CEP , i mean i need a DSL for flink CEP , so that i can decouple the CEP rules from code and pass them dynamically to be applied on different data streams. Flink CEP doen't have any DSL implementation , so is it that Flink SQL can be used for the same purpose , where flink SQL has integration with Flink CEP and underlying flink SQL uses flink CEP for data processing. If that is the case can we use flink SQL as a streaming CEP DSL.


Regards
Dipanjan