About KafkaConsumerBase

[aitozi]

Hello:

    i am new to Flink, ijust read the source code . i am doubt that , why in FlinkKafkaConsumerBase.java (version1.2),  like method : notifyCheckpointComplete  may change the pendingOffsetsToCommit in parallel , why dont need to be surrouned with "synchronized"  

thanks

[Piotr Nowojski]

Hi,

pendingOffsetsToCommit is a private field which is not accessed from outside of the FlinkKafkaConsumerBase class. It is only used in state manipulation methods, which are not executed in parallel.

Thanks, Piotrek

> On Aug 1, 2017, at 1:20 PM, aitozi <[hidden email]> wrote:
>
> Hello:
>
>    i am new to Flink, ijust read the source code . i am doubt that , why in
> FlinkKafkaConsumerBase.java (version1.2),  like method :
> notifyCheckpointComplete  may change the pendingOffsetsToCommit in parallel
> , why dont need to be surrouned with "synchronized"  
>
> thanks
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/About-KafkaConsumerBase-tp14601.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[aitozi]

Hi,Piotr Nowojski

      i think you are right, but i think it is executed in parallel, but in each parallel , it maintain itself a individual   instance of FlinkKafkaConsumerBase, and it contains a individual pendingOffsetsToCommit , am right ?

thanks, aitozi

[Tzu-Li (Gordon) Tai]

Hi,

it maintain itself a individual instance of 
FlinkKafkaConsumerBase, and it contains a individual pendingOffsetsToCommit 
, am right ? 

That is correct! The FlinkKafkaConsumerBase is code executed for each parallel subtask instance, and therefore have their own pendingOffsetsToCommit which would not be manipulated / accessed concurrently.

The only places where that map is accessed is in the snapshotState and notifyCheckpointComplete method, which I think is guaranteed to not be concurrently called.

Cheers,

Gordon

On 2 August 2017 at 1:02:57 PM, aitozi ([hidden email]) wrote:

Hi,Piotr Nowojski

i think you are right, but i think it is executed in parallel, but in
each parallel , it maintain itself a individual instance of
FlinkKafkaConsumerBase, and it contains a individual pendingOffsetsToCommit
, am right ?

thanks, aitozi



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/About-KafkaConsumerBase-tp14601p14619.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[aitozi]

Hi，

     thanks，you explained clearly!