How autoscaling works on Kinesis Data Analytics for Java ?

[Maxim Parkachov]

Hi,

I had impression, that in order to change parallelism, one need to stop Flink streaming job and re-start with new settings. 

According to https://docs.aws.amazon.com/kinesisanalytics/latest/java/how-scaling.html auto-scaling works out of the box. Could someone with experience of running Flink on AWS Data Analytics for Java give a short explanation ?

Thanks in advance.

Maxim.

[Maxim Parkachov]

Hi,

Answering to myself in case someone else is interested as well. As per https://aws.amazon.com/blogs/big-data/build-and-run-streaming-applications-with-apache-flink-and-amazon-kinesis-data-analytics-for-java-applications/ it does autoscaling itself, but in order to change parallelism it takes snapshot and restarts streaming job, so no magic here, but nicely automated. 

Regards,

Maxim.

On Tue, Jan 29, 2019 at 5:23 AM Maxim Parkachov <[hidden email]> wrote:

Hi,

I had impression, that in order to change parallelism, one need to stop Flink streaming job and re-start with new settings. 

According to https://docs.aws.amazon.com/kinesisanalytics/latest/java/how-scaling.html auto-scaling works out of the box. Could someone with experience of running Flink on AWS Data Analytics for Java give a short explanation ?

Thanks in advance.

Maxim.