Status of FLINK-12692 (Support disk spilling in HeapKeyedStateBackend)

[Ken Krugler]

Hi Yu Li,

It looks like this stalled out a bit, from May of last year, and won’t make it into 1.10.

I’m wondering if there’s a version in Blink (as a completely separate state backend?) that could be tried out?

Thanks,

— Ken

--------------------------

Ken Krugler

http://www.scaleunlimited.com

custom big data solutions & training

Hadoop, Cascading, Cassandra & Solr

[Yu Li]

Hi Ken,

Thanks for watching this feature.

Unfortunately yes this didn't make into 1.10, and we will try our best to complete the upstreaming work in 1.11.0. Meantime, we are actively preparing a try-out version and will publish it onto flink-packages [1] once ready. Will also get back here once the package is ready.

On Thu, 30 Jan 2020 at 08:47, Ken Krugler <[hidden email]> wrote:

Hi Yu Li,

It looks like this stalled out a bit, from May of last year, and won’t make it into 1.10.

I’m wondering if there’s a version in Blink (as a completely separate state backend?) that could be tried out?

Thanks,

— Ken

--------------------------

Ken Krugler

http://www.scaleunlimited.com

custom big data solutions & training

Hadoop, Cascading, Cassandra & Solr

[Yu Li]

btw, the upstreaming work is lagged but not stalled, and the biggest PR (with more than 6k lines of codes) [1] [2] was already merged on Nov. 2019.

We are sorry about the lagging, but we're still actively working on this feature, JFYI (smile).

[1] https://issues.apache.org/jira/browse/FLINK-12697

[2] https://github.com/apache/flink/pull/9501

On Fri, 31 Jan 2020 at 15:27, Yu Li <[hidden email]> wrote:

Hi Ken,

Thanks for watching this feature.

Unfortunately yes this didn't make into 1.10, and we will try our best to complete the upstreaming work in 1.11.0. Meantime, we are actively preparing a try-out version and will publish it onto flink-packages [1] once ready. Will also get back here once the package is ready.

Best Regards,

Yu

[1] https://flink-packages.org/

On Thu, 30 Jan 2020 at 08:47, Ken Krugler <[hidden email]> wrote:

Hi Yu Li,

It looks like this stalled out a bit, from May of last year, and won’t make it into 1.10.

I’m wondering if there’s a version in Blink (as a completely separate state backend?) that could be tried out?

Thanks,

— Ken

--------------------------

Ken Krugler

http://www.scaleunlimited.com

custom big data solutions & training

Hadoop, Cascading, Cassandra & Solr