Re: understadning kafka connector - rebalance

Posted by Taher Koitawala on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/understadning-kafka-connector-rebalance-tp24724p24725.html

Hi Avi,

          No, rebalance is not changing the number of kafka partitions. Lets say you have 6 kafka partitions and your flink parallelism is 8, in this case using rebalance will send records to all downstream operators in a round robin fashion. 

On Mon, Nov 26, 2018 at 11:33 AM Avi Levi <[hidden email]> wrote:

Hi

Looking at this example, doing the "rebalance" (e.g messageStream.rebalance().map(...) ) operation on heavy load stream wouldn't slow the stream ? is the rebalancing action occurs only when there is a partition change ? 

it says that "the rebelance call is causing a repartitioning of the data so that all machines" is it actually changing the num of partitions of the topic to match the num of flink operators ?

Avi