Using async io in cep

Hi，

        Is there a way to use asynchronous io to query the database in the process of cep?

---

[hidden email]

[Dawid Wysakowicz-2]

Hi,

You cannot use the Async IO as described here[1] in the CEP library, if that's what you are asking for.

It is also not that straightforward to say what would an async processing in that case mean. Primary use case for Async IO is to execute parallel computations of independent data. In case of CEP it does not stand for processing of records within a single key, as those assume strict ordering and in general case depend on results of processing previous records. One could think of asynchronous processing of records from different keys in a single parallel instance, but that would require a careful key processing. If I am not mistaken Async IO also does not support a stateful processing on a keyed stream.

Best,

Dawid

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/asyncio.html

On 06/01/2020 09:59, 郑 洁锋 wrote:

Hi，

        Is there a way to use asynchronous io to query the database in the process of cep?

---

[hidden email]

Hi，

        Our business is to dynamically query (such as left join/right join) the mysql / oracle database during the cep process, or is there any other way to achieve this function?

---

[hidden email]

 

From: [hidden email]

Date: 2020-01-06 18:09

To: [hidden email]; [hidden email]

Subject: Re: Using async io in cep

Hi,

You cannot use the Async IO as described here[1] in the CEP library, if that's what you are asking for.

It is also not that straightforward to say what would an async processing in that case mean. Primary use case for Async IO is to execute parallel computations of independent data. In case of CEP it does not stand for processing of records within a single key, as those assume strict ordering and in general case depend on results of processing previous records. One could think of asynchronous processing of records from different keys in a single parallel instance, but that would require a careful key processing. If I am not mistaken Async IO also does not support a stateful processing on a keyed stream.

Best,

Dawid

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/asyncio.html

On 06/01/2020 09:59, 郑 洁锋 wrote:

Hi，

        Is there a way to use asynchronous io to query the database in the process of cep?

---

[hidden email]

[Dawid Wysakowicz-2]

If I am not mistaken, my previous answer is still valid. There is no way to have true asynchronicity within CEP conditions.

Why do you want to use async io there? Did you hit performance issues? If so, you could try increasing the parallelism.

Best,

Dawid

On 07/01/2020 02:47, 郑 洁锋 wrote:

Hi，

        Our business is to dynamically query (such as left join/right join) the mysql / oracle database during the cep process, or is there any other way to achieve this function?

---

[hidden email]

 

From: [hidden email]

Date: 2020-01-06 18:09

To: [hidden email]; [hidden email]

Subject: Re: Using async io in cep

Hi,

You cannot use the Async IO as described here[1] in the CEP library, if that's what you are asking for.

It is also not that straightforward to say what would an async processing in that case mean. Primary use case for Async IO is to execute parallel computations of independent data. In case of CEP it does not stand for processing of records within a single key, as those assume strict ordering and in general case depend on results of processing previous records. One could think of asynchronous processing of records from different keys in a single parallel instance, but that would require a careful key processing. If I am not mistaken Async IO also does not support a stateful processing on a keyed stream.

Best,

Dawid

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/asyncio.html

On 06/01/2020 09:59, 郑 洁锋 wrote:

Hi，

        Is there a way to use asynchronous io to query the database in the process of cep?

---

[hidden email]