Mirror Maker 2.0 cluster and starting from latest offset and other queries

[Vishal Santoshi]

2 queries

1.   I am trying to configure MM2 to start replicating from the head ( latest of the topic ) .  Should  auto.offset.reset = latest in mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.

2. I  do not have "Authorizer is configured on the broker "  and see this exception java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is configured on the broker From https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync

sync.topic.acls = false should do it but does not..

Any ideas? 

[Vishal Santoshi]

1.  still no clue, apart from the fact that ConsumerConfig gets it from somewhere ( need to override it and have tried both auto.offset.reset =latest and  consumer.auto.offset.reset = latest

[2019-10-16 13:50:34,260] INFO ConsumerConfig values: 

allow.auto.create.topics = true

auto.commit.interval.ms = 5000

auto.offset.reset = earliest

2. sync.topic.acls.enabled = false 

On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <[hidden email]> wrote:

2 queries

1.   I am trying to configure MM2 to start replicating from the head ( latest of the topic ) .  Should  auto.offset.reset = latest in mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.

2. I  do not have "Authorizer is configured on the broker "  and see this exception java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is configured on the broker From https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync

sync.topic.acls = false should do it but does not..

Any ideas? 

[Piotr Nowojski-3]

Hi,

It sounds like setting up Mirror Maker has very little to do with Flink. Shouldn’t you try asking this question on the Kafka mailing list?

Piotrek

On 16 Oct 2019, at 16:06, Vishal Santoshi <[hidden email]> wrote:

1.  still no clue, apart from the fact that ConsumerConfig gets it from somewhere ( need to override it and have tried both auto.offset.reset =latest and  consumer.auto.offset.reset = latest

[2019-10-16 13:50:34,260] INFO ConsumerConfig values: 

allow.auto.create.topics = true

auto.commit.interval.ms = 5000

auto.offset.reset = earliest

2. sync.topic.acls.enabled = false 

On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <[hidden email]> wrote:

2 queries

1.   I am trying to configure MM2 to start replicating from the head ( latest of the topic ) .  Should  auto.offset.reset = latest in mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.

2. I  do not have "Authorizer is configured on the broker "  and see this exception java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is configured on the broker From https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync

sync.topic.acls = false should do it but does not..

Any ideas? 

[Vishal Santoshi]

oh shit.. sorry wrong wrong forum :) 

On Thu, Oct 17, 2019 at 1:41 PM Piotr Nowojski <[hidden email]> wrote:

Hi,

It sounds like setting up Mirror Maker has very little to do with Flink. Shouldn’t you try asking this question on the Kafka mailing list?

Piotrek

On 16 Oct 2019, at 16:06, Vishal Santoshi <[hidden email]> wrote:

1.  still no clue, apart from the fact that ConsumerConfig gets it from somewhere ( need to override it and have tried both auto.offset.reset =latest and  consumer.auto.offset.reset = latest

[2019-10-16 13:50:34,260] INFO ConsumerConfig values: 

allow.auto.create.topics = true

auto.commit.interval.ms = 5000

auto.offset.reset = earliest

2. sync.topic.acls.enabled = false 

On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <[hidden email]> wrote:

2 queries

1.   I am trying to configure MM2 to start replicating from the head ( latest of the topic ) .  Should  auto.offset.reset = latest in mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.

2. I  do not have "Authorizer is configured on the broker "  and see this exception java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is configured on the broker From https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync

sync.topic.acls = false should do it but does not..

Any ideas? 

[Piotr Nowojski-3]

No problem, cheers :)

Piotrek

On 17 Oct 2019, at 19:56, Vishal Santoshi <[hidden email]> wrote:

oh shit.. sorry wrong wrong forum :) 

On Thu, Oct 17, 2019 at 1:41 PM Piotr Nowojski <[hidden email]> wrote:

Hi,

It sounds like setting up Mirror Maker has very little to do with Flink. Shouldn’t you try asking this question on the Kafka mailing list?

Piotrek

On 16 Oct 2019, at 16:06, Vishal Santoshi <[hidden email]> wrote:

1.  still no clue, apart from the fact that ConsumerConfig gets it from somewhere ( need to override it and have tried both auto.offset.reset =latest and  consumer.auto.offset.reset = latest

[2019-10-16 13:50:34,260] INFO ConsumerConfig values: 

allow.auto.create.topics = true

auto.commit.interval.ms = 5000

auto.offset.reset = earliest

2. sync.topic.acls.enabled = false 

On Tue, Oct 15, 2019 at 4:00 PM Vishal Santoshi <[hidden email]> wrote:

2 queries

1.   I am trying to configure MM2 to start replicating from the head ( latest of the topic ) .  Should  auto.offset.reset = latest in mm2.properties be enough ? Unfortunately MM2 will start from the EARLIEST.

2. I  do not have "Authorizer is configured on the broker "  and see this exception java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SecurityDisabledException: No Authorizer is configured on the broker From https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-Config,ACLSync

sync.topic.acls = false should do it but does not..

Any ideas?