Reading messages from start - new job submission

[avilevi]

I'm updating a job without savepoint.

The consumer properties is set to   prop.setProperty("auto.offset.reset", "earliest")

The start strategy is not explicitly set (using the default setStartFromGroupOffsets).

In this case I expect that the consumer will read the messages from the beginning since there is no offset restored, but is actually reading from latest. What am I missing? 

Best

Avi

[sohimankotia]

Hi,

Which version of flink you r using ?

Reset offset to earliest :

https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-start-position-configuration


Thanks




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[avilevi]

Hi ,
I'm using 1.7.0

Cheers



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[avilevi]

Thanks for the answer,
But my question is why do  I need to set
/myConsumer.setStartFromEarliest();/  if I set this property
/setProperty("auto.offset.reset", "earliest") /in consumer properties ?
I want the consumer to start reading from earliest only If offsets could not
be found as stated in the documentation.
isn't that the expected behaviour ?

Best wishes
Avi



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Dian Fu]

Hi Avi,

As described in the documentation: "If offsets could not be found for a partition, the auto.offset.reset setting in the properties will be used.". For starting from GroupOffset, the property "auto.offset.reset" will ONLY be respected when the group offset cannot be found for a partition. Otherwise, it will use the offset found.

Regards,
Dian

> 在 2019年2月20日，上午1:49，avilevi <[hidden email]> 写道：
>
> Thanks for the answer,
> But my question is why do  I need to set
> /myConsumer.setStartFromEarliest();/  if I set this property
> /setProperty("auto.offset.reset", "earliest") /in consumer properties ?
> I want the consumer to start reading from earliest only If offsets could not
> be found as stated in the documentation.
> isn't that the expected behaviour ?
>
> Best wishes
> Avi
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/