SinkFunction.Context

[Durga Durga]

HI Folks,

Was following the documentation for 

https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html

        

long	currentProcessingTime() Returns the current processing time.
long	currentWatermark() Returns the current event-time watermark.
Long	timestamp() Returns the timestamp of the current input record or null if the element does not have an assigned timestamp.

- CurrentProcessing time - is this is the Event Time ? i.e the time when the Event Occured ? (or) when the flink window got the message ?.

- timeStamp - is this the time the record is persisted in to the sync ?. (or) the aggregated data timestamp ?. Say if I have 100 records in my  time window - which time comes in to affect.

- CurrentWaterMark - which time is this - the time the even occured - what will this value be - when there are 1000 records in my time window

PS - We want to use some ID ( timestamp ) and associate with  all the records that are persisted (aggregated) in a given time window - i.e if there are 1000 records aggregated - and they resulted in 10 aggregated records - we want to give these 10 aggregated records the same ID and we want to use one of the above time stamp

Thanks much.

[Rong Rong]

Hi Durga,

1. currentProcessingTime: refers to this operator(SinkFunction)'s system time at the moment of invoke

1a. the time you are referring to as "flink window got the message" is the currentProcessingTime() invoked at the window operator (which provided by the WindowContext similar to this one [1])

2 currentWatermark: refers to the current watermark [2] received by this operator(SinkFunction)

3. timestamp: is actually the input record's event-time (this "input" is referring to the input to the SinkFunction, not to the entire Flink topology)

Hope these help.

--

Rong

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#processwindowfunction

[2] https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_time.html#event-time-and-watermarks

On Thu, Feb 21, 2019 at 4:59 PM Durga Durga <[hidden email]> wrote:

HI Folks,

Was following the documentation for 

https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html

        

long	currentProcessingTime() Returns the current processing time.
long	currentWatermark() Returns the current event-time watermark.
Long	timestamp() Returns the timestamp of the current input record or null if the element does not have an assigned timestamp.

- CurrentProcessing time - is this is the Event Time ? i.e the time when the Event Occured ? (or) when the flink window got the message ?.

- timeStamp - is this the time the record is persisted in to the sync ?. (or) the aggregated data timestamp ?. Say if I have 100 records in my  time window - which time comes in to affect.

- CurrentWaterMark - which time is this - the time the even occured - what will this value be - when there are 1000 records in my time window

PS - We want to use some ID ( timestamp ) and associate with  all the records that are persisted (aggregated) in a given time window - i.e if there are 1000 records aggregated - and they resulted in 10 aggregated records - we want to give these 10 aggregated records the same ID and we want to use one of the above time stamp

Thanks much.

[Durga Durga]

Excellent ! Thanks Rong..

On Thu, Feb 21, 2019 at 7:26 PM Rong Rong <[hidden email]> wrote:

Hi Durga,

1. currentProcessingTime: refers to this operator(SinkFunction)'s system time at the moment of invoke

1a. the time you are referring to as "flink window got the message" is the currentProcessingTime() invoked at the window operator (which provided by the WindowContext similar to this one [1])

2 currentWatermark: refers to the current watermark [2] received by this operator(SinkFunction)

3. timestamp: is actually the input record's event-time (this "input" is referring to the input to the SinkFunction, not to the entire Flink topology)

Hope these help.

--

Rong

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#processwindowfunction

[2] https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_time.html#event-time-and-watermarks

On Thu, Feb 21, 2019 at 4:59 PM Durga Durga <[hidden email]> wrote:

HI Folks,

Was following the documentation for 

https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.Context.html

        

long	currentProcessingTime() Returns the current processing time.
long	currentWatermark() Returns the current event-time watermark.
Long	timestamp() Returns the timestamp of the current input record or null if the element does not have an assigned timestamp.

- CurrentProcessing time - is this is the Event Time ? i.e the time when the Event Occured ? (or) when the flink window got the message ?.

- timeStamp - is this the time the record is persisted in to the sync ?. (or) the aggregated data timestamp ?. Say if I have 100 records in my  time window - which time comes in to affect.

- CurrentWaterMark - which time is this - the time the even occured - what will this value be - when there are 1000 records in my time window

PS - We want to use some ID ( timestamp ) and associate with  all the records that are persisted (aggregated) in a given time window - i.e if there are 1000 records aggregated - and they resulted in 10 aggregated records - we want to give these 10 aggregated records the same ID and we want to use one of the above time stamp

Thanks much.

--