Flink SQL as DSL for flink CEP

[Dipanjan Mazumder]

Hi,

    Can we say that Flink SQL is kind of a DSL overlay on flink CEP , i mean i need a DSL for flink CEP , so that i can decouple the CEP rules from code and pass them dynamically to be applied on different data streams. Flink CEP doen't have any DSL implementation , so is it that Flink SQL can be used for the same purpose , where flink SQL has integration with Flink CEP and underlying flink SQL uses flink CEP for data processing. If that is the case can we use flink SQL as a streaming CEP DSL.

Regards

Dipanjan

[rmetzger0]

Hi Dipanjan,

Using Flink SQL's MATCH_RECOGNIZE operator is certainly a good idea if you are looking for a non-programmatic way to do CEP with Flink.

On Wed, Jun 16, 2021 at 6:44 AM Dipanjan Mazumder <[hidden email]> wrote:

Hi,

    Can we say that Flink SQL is kind of a DSL overlay on flink CEP , i mean i need a DSL for flink CEP , so that i can decouple the CEP rules from code and pass them dynamically to be applied on different data streams. Flink CEP doen't have any DSL implementation , so is it that Flink SQL can be used for the same purpose , where flink SQL has integration with Flink CEP and underlying flink SQL uses flink CEP for data processing. If that is the case can we use flink SQL as a streaming CEP DSL.

Regards

Dipanjan

[JING ZHANG]

Hi Dipanjan,

I'm not an expert on Flink CEP, however I would like to share my immature ideas, hope it helps.

Flink provides CEP library[1] which is a programmatic library which allows user define patterns based on pattern API.

Besides, Flink allows user define patterns by MATCH_RECOGNIZE clause in SQL [2].

I think it's the right decision to use Flink SQL as a streaming CEP DSL. Please note most of the important features about CEP are already supported by Flink SQL, while a few features are not supported yet, you could view the document [2] to check.

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/libs/cep/

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/table/sql/queries/match_recognize/

Best,

JING ZHANG

Dipanjan Mazumder <[hidden email]> 于2021年6月16日周三 下午12:44写道：

Hi,

    Can we say that Flink SQL is kind of a DSL overlay on flink CEP , i mean i need a DSL for flink CEP , so that i can decouple the CEP rules from code and pass them dynamically to be applied on different data streams. Flink CEP doen't have any DSL implementation , so is it that Flink SQL can be used for the same purpose , where flink SQL has integration with Flink CEP and underlying flink SQL uses flink CEP for data processing. If that is the case can we use flink SQL as a streaming CEP DSL.

Regards

Dipanjan