hi,
I change the following config from
flink.shard.getrecords.maxrecordcount: 1000
flink.shard.getrecords.intervalmillis: 200
to
flink.shard.getrecords.maxrecordcount: 10000
flink.shard.getrecords.intervalmillis: 1000
and found a few task managers around (10/1000) are becoming very slow.
We also found that slow task manager's input stream shard (kineis) keeps aging. And other task manager's kinesis shard age stays low.
Anyone knows why increase maxrecordcount and increase intervalmillis can cause a few task-managers throughput drops a lot?
Best,
Terry