Long running time based Patterns

[Moiz Jinia]

Does Flink (with a persistent State backend such as RocksDB) work well with long running Patterns of this type? (running into days)

Pattern.<Integer>begin("start").followedBy("end").within(Time.days(3))

Is there some gotchas here or things to watch out for?

Thanks,

Moiz

[Kostas Kloudas]

Hi Moiz,

You are on Flink 1.2 or 1.3? 

In Flink 1.2 (latest stable) there are no known issues, so this will work correctly. 

Keep in mind that without any conditions (where-clauses), you will only get all possible 

2-tuples of incoming elements, which could also be done with a simple process function I would say.

In Flink 1.3 (unreleased) there is this issue: https://issues.apache.org/jira/browse/FLINK-6445

Thanks,

Kostas

On May 4, 2017, at 1:45 PM, Moiz S Jinia <[hidden email]> wrote:

Does Flink (with a persistent State backend such as RocksDB) work well with long running Patterns of this type? (running into days)

Pattern.<Integer>begin("start").followedBy("end").within(Time.days(3))

Is there some gotchas here or things to watch out for?

Thanks,

Moiz

[Moiz Jinia]

It'll definitely have a where clause. Just forgot to include it in the example. Just meant to focus on the within clause.

Am on 1.3 - expect it'll be fixed by the time stable is out?

Thanks!

Moiz

—

sent from phone

On 04-May-2017, at 8:12 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

You are on Flink 1.2 or 1.3? 

In Flink 1.2 (latest stable) there are no known issues, so this will work correctly. 

Keep in mind that without any conditions (where-clauses), you will only get all possible 

2-tuples of incoming elements, which could also be done with a simple process function I would say.

In Flink 1.3 (unreleased) there is this issue: https://issues.apache.org/jira/browse/FLINK-6445

Thanks,

Kostas

On May 4, 2017, at 1:45 PM, Moiz S Jinia <[hidden email]> wrote:

Does Flink (with a persistent State backend such as RocksDB) work well with long running Patterns of this type? (running into days)

Pattern.<Integer>begin("start").followedBy("end").within(Time.days(3))

Is there some gotchas here or things to watch out for?

Thanks,

Moiz

[Kostas Kloudas]

Hi Moiz,

Then it should work.

And the previous issue is already fixed on the master.

Kostas

On May 4, 2017, at 6:02 PM, Moiz Jinia <[hidden email]> wrote:

It'll definitely have a where clause. Just forgot to include it in the example. Just meant to focus on the within clause.

Am on 1.3 - expect it'll be fixed by the time stable is out?

Thanks!

Moiz

—

sent from phone

On 04-May-2017, at 8:12 PM, Kostas Kloudas <[hidden email]> wrote:

Hi Moiz,

You are on Flink 1.2 or 1.3? 

In Flink 1.2 (latest stable) there are no known issues, so this will work correctly. 

Keep in mind that without any conditions (where-clauses), you will only get all possible 

2-tuples of incoming elements, which could also be done with a simple process function I would say.

In Flink 1.3 (unreleased) there is this issue: https://issues.apache.org/jira/browse/FLINK-6445

Thanks,

Kostas

On May 4, 2017, at 1:45 PM, Moiz S Jinia <[hidden email]> wrote:

Does Flink (with a persistent State backend such as RocksDB) work well with long running Patterns of this type? (running into days)

Pattern.<Integer>begin("start").followedBy("end").within(Time.days(3))

Is there some gotchas here or things to watch out for?

Thanks,

Moiz