Accumulating a batch

[austin.ce]

Hi there,

Is it possible to use an AggregationFunction to accumulate n values in a buffer until a threshold is met, then stream down all records in the batch?

Thank you!

Austin Cawley-Edwards

[Hequn Cheng]

Hi Austin,

You can use GroupBy Window[1], such as TUMBLE Window. The size of the window either as time or row-count interval. You can also define your own User-Defined Aggregate Functions[2] to be used in window.

Best, Hequn

[1] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#tumble-tumbling-windows

[2] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/udfs.html#aggregation-functions

On Fri, Oct 26, 2018 at 5:08 AM Austin Cawley-Edwards <[hidden email]> wrote:

Hi there,

Is it possible to use an AggregationFunction to accumulate n values in a buffer until a threshold is met, then stream down all records in the batch?

Thank you!

Austin Cawley-Edwards

[austin.ce]

Hi Hequn,

Thank you for the info! Much appreciated! Do you know if a GroupBy Window could be used to buffer until an arbitrary record came through?

Best,

Austin

On Thu, Oct 25, 2018, 9:17 PM Hequn Cheng <[hidden email]> wrote:

Hi Austin,

You can use GroupBy Window[1], such as TUMBLE Window. The size of the window either as time or row-count interval. You can also define your own User-Defined Aggregate Functions[2] to be used in window.

Best, Hequn

[1] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#tumble-tumbling-windows

[2] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/udfs.html#aggregation-functions

On Fri, Oct 26, 2018 at 5:08 AM Austin Cawley-Edwards <[hidden email]> wrote:

Hi there,

Is it possible to use an AggregationFunction to accumulate n values in a buffer until a threshold is met, then stream down all records in the batch?

Thank you!

Austin Cawley-Edwards