DSL for Flink CEP

[Dipanjan Mazumder]

Hi ,

   I am currently using Siddhi CEP with flink , but the flink-siddhi library has limited support for flnk versions and i will either need to fix the library or get tied to a fix version of Flink to use th library.

 I am looking at Flink CEP as an option , and also came across a Flink CEP DSL library (https://github.com/phil3k3/flink-cep-dsl) , but i am not sure about the acceptance for the same by the Flink community and DEV. Also is Flink CEP supporting the Siddhi CEP constructs and is rich on the same aspect.

Please let me know the same , so that i can take a cautious decision on the same.

Regards

Dipanjan

[Fabian Paul]

Hi Dipanjan,

Unfortunately, I have no experience with Siddhi but I am not aware of any official joined efforts between Flink and Siddhi.

I can imagine that not all Siddhi CEP expressions are compatible with Flink’s CEP. At the moment there is also no active

development for Flink’s CEP. 

I think to get a better understanding what the caveats are of the third party solution you have to directly reach out to the 

maintainers.

Best,

Fabian

On 2. Jun 2021, at 08:37, Dipanjan Mazumder <[hidden email]> wrote:

Hi ,

   I am currently using Siddhi CEP with flink , but the flink-siddhi library has limited support for flnk versions and i will either need to fix the library or get tied to a fix version of Flink to use th library.

 I am looking at Flink CEP as an option , and also came across a Flink CEP DSL library (https://github.com/phil3k3/flink-cep-dsl) , but i am not sure about the acceptance for the same by the Flink community and DEV. Also is Flink CEP supporting the Siddhi CEP constructs and is rich on the same aspect.

Please let me know the same , so that i can take a cautious decision on the same.

Regards

Dipanjan

[Dipanjan Mazumder]

Hi Fabian,

     Understood but is there any plan to grow the flink  CEP and build a friendly DSL around flink CEP by any chance.

Regards

Dipanjan

On Wednesday, June 2, 2021, 03:22:46 PM GMT+5:30, Fabian Paul <[hidden email]> wrote:

Hi Dipanjan,

Unfortunately, I have no experience with Siddhi but I am not aware of any official joined efforts between Flink and Siddhi.

I can imagine that not all Siddhi CEP expressions are compatible with Flink’s CEP. At the moment there is also no active

development for Flink’s CEP. 

I think to get a better understanding what the caveats are of the third party solution you have to directly reach out to the 

maintainers.

Best,

Fabian

On 2. Jun 2021, at 08:37, Dipanjan Mazumder <[hidden email]> wrote:

Hi ,

   I am currently using Siddhi CEP with flink , but the flink-siddhi library has limited support for flnk versions and i will either need to fix the library or get tied to a fix version of Flink to use th library.

 I am looking at Flink CEP as an option , and also came across a Flink CEP DSL library (https://github.com/phil3k3/flink-cep-dsl) , but i am not sure about the acceptance for the same by the Flink community and DEV. Also is Flink CEP supporting the Siddhi CEP constructs and is rich on the same aspect.

Please let me know the same , so that i can take a cautious decision on the same.

Regards

Dipanjan

[Fabian Paul]

Hi Dipanjan,

I am afraid there are no foreseeable efforts planned but if you find a nice addition, you can 

start a discussion in the community about this feature.

Best,

Fabian

On 2. Jun 2021, at 12:10, Dipanjan Mazumder <[hidden email]> wrote:

Hi Fabian,

     Understood but is there any plan to grow the flink  CEP and build a friendly DSL around flink CEP by any chance.

Regards

Dipanjan

On Wednesday, June 2, 2021, 03:22:46 PM GMT+5:30, Fabian Paul <[hidden email]> wrote:

Hi Dipanjan,

Unfortunately, I have no experience with Siddhi but I am not aware of any official joined efforts between Flink and Siddhi.

I can imagine that not all Siddhi CEP expressions are compatible with Flink’s CEP. At the moment there is also no active

development for Flink’s CEP. 

I think to get a better understanding what the caveats are of the third party solution you have to directly reach out to the 

maintainers.

Best,

Fabian

On 2. Jun 2021, at 08:37, Dipanjan Mazumder <[hidden email]> wrote:

Hi ,

   I am currently using Siddhi CEP with flink , but the flink-siddhi library has limited support for flnk versions and i will either need to fix the library or get tied to a fix version of Flink to use th library.

 I am looking at Flink CEP as an option , and also came across a Flink CEP DSL library (https://github.com/phil3k3/flink-cep-dsl) , but i am not sure about the acceptance for the same by the Flink community and DEV. Also is Flink CEP supporting the Siddhi CEP constructs and is rich on the same aspect.

Please let me know the same , so that i can take a cautious decision on the same.

Regards

Dipanjan

[Dawid Wysakowicz-2]

Hi,

Just to add on top to what Fabian said.

The only community supported CEP library is the one that comes with Flink[1]. It is also used internally to support the MATCH_RECOGNIZE clause in Flink SQL[2]. Therefore there is a both programmatic library native DSL for defining patterns. Moreover you can use SQL. As Fabian mentioned, you can say the library is in a maintenance mode, primarily because there is no one interested in actively working on it.

If you use other CEP libraries that are integrated with Flink, it's probably a better idea to reach out to the maintainers of said libraries.

Lastly, I am not aware of any comparisons of CEP libraries/extensions that work with Flink. I am afraid you have to do the feature comparison yourself. I think the documentation for community supported library is a good start for it.

Best,

Dawid

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/libs/cep/

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/table/sql/queries/match_recognize/

On 02/06/2021 13:23, Fabian Paul wrote:

Hi Dipanjan,

I am afraid there are no foreseeable efforts planned but if you find a nice addition, you can 

start a discussion in the community about this feature.

Best,

Fabian

On 2. Jun 2021, at 12:10, Dipanjan Mazumder <[hidden email]> wrote:

Hi Fabian,

     Understood but is there any plan to grow the flink  CEP and build a friendly DSL around flink CEP by any chance.

Regards

Dipanjan

On Wednesday, June 2, 2021, 03:22:46 PM GMT+5:30, Fabian Paul <[hidden email]> wrote:

Hi Dipanjan,

Unfortunately, I have no experience with Siddhi but I am not aware of any official joined efforts between Flink and Siddhi.

I can imagine that not all Siddhi CEP expressions are compatible with Flink’s CEP. At the moment there is also no active

development for Flink’s CEP. 

I think to get a better understanding what the caveats are of the third party solution you have to directly reach out to the 

maintainers.

Best,

Fabian

On 2. Jun 2021, at 08:37, Dipanjan Mazumder <[hidden email]> wrote:

Hi ,

   I am currently using Siddhi CEP with flink , but the flink-siddhi library has limited support for flnk versions and i will either need to fix the library or get tied to a fix version of Flink to use th library.

 I am looking at Flink CEP as an option , and also came across a Flink CEP DSL library (https://github.com/phil3k3/flink-cep-dsl) , but i am not sure about the acceptance for the same by the Flink community and DEV. Also is Flink CEP supporting the Siddhi CEP constructs and is rich on the same aspect.

Please let me know the same , so that i can take a cautious decision on the same.

Regards

Dipanjan

[Dipanjan Mazumder]

Thanks a lot ...

On Thursday, June 3, 2021, 12:49:58 PM GMT+5:30, Dawid Wysakowicz <[hidden email]> wrote:

Hi,

Just to add on top to what Fabian said.

The only community supported CEP library is the one that comes with Flink[1]. It is also used internally to support the MATCH_RECOGNIZE clause in Flink SQL[2]. Therefore there is a both programmatic library native DSL for defining patterns. Moreover you can use SQL. As Fabian mentioned, you can say the library is in a maintenance mode, primarily because there is no one interested in actively working on it.

If you use other CEP libraries that are integrated with Flink, it's probably a better idea to reach out to the maintainers of said libraries.

Lastly, I am not aware of any comparisons of CEP libraries/extensions that work with Flink. I am afraid you have to do the feature comparison yourself. I think the documentation for community supported library is a good start for it.

Best,

Dawid

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/libs/cep/

[2] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/table/sql/queries/match_recognize/

On 02/06/2021 13:23, Fabian Paul wrote:

Hi Dipanjan,

I am afraid there are no foreseeable efforts planned but if you find a nice addition, you can 

start a discussion in the community about this feature.

Best,

Fabian

On 2. Jun 2021, at 12:10, Dipanjan Mazumder <[hidden email]> wrote:

Hi Fabian,

     Understood but is there any plan to grow the flink  CEP and build a friendly DSL around flink CEP by any chance.

Regards

Dipanjan

On Wednesday, June 2, 2021, 03:22:46 PM GMT+5:30, Fabian Paul <[hidden email]> wrote:

Hi Dipanjan,

Unfortunately, I have no experience with Siddhi but I am not aware of any official joined efforts between Flink and Siddhi.

I can imagine that not all Siddhi CEP expressions are compatible with Flink’s CEP. At the moment there is also no active

development for Flink’s CEP. 

I think to get a better understanding what the caveats are of the third party solution you have to directly reach out to the 

maintainers.

Best,

Fabian

On 2. Jun 2021, at 08:37, Dipanjan Mazumder <[hidden email]> wrote:

Hi ,

   I am currently using Siddhi CEP with flink , but the flink-siddhi library has limited support for flnk versions and i will either need to fix the library or get tied to a fix version of Flink to use th library.

 I am looking at Flink CEP as an option , and also came across a Flink CEP DSL library (https://github.com/phil3k3/flink-cep-dsl) , but i am not sure about the acceptance for the same by the Flink community and DEV. Also is Flink CEP supporting the Siddhi CEP constructs and is rich on the same aspect.

Please let me know the same , so that i can take a cautious decision on the same.

Regards

Dipanjan