WatermarkStrategy.for_bounded_out_of_orderness in table API

[joris.vanagtmaal]

Can i set the watermark strategy to bounded out of orderness when using the
table api and sql DDL to assign watermarks?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[joris.vanagtmaal]

Or is this only possible with the data stream api? I tried converting a table
to a datastream of rows, but being a noob, finding examples of how to do
this has been difficult and not sure how to provide the required
RowTypeInfo.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[joris.vanagtmaal]

I worked out the rowtype input for the conversion to datastream;

type_info = Types.ROW_NAMED(['sender', 'stw', 'time'],[Types.STRING(),
Types.DOUBLE(), Types.LONG()])
datastream=table_env.to_append_stream(my_table, type_info)

But if i try to assign rowtime and watermarks to the datastream and convert
it back to a table, the rowtime/watermarks either get dropped or maybe they
are not created properly.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Roman Khachatryan]

Hi,

You can use watermark strategy with bounded out of orderness in DDL, please refer to [1].

[1]

https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/sql/create.html#watermark

On Tue, Feb 23, 2021 at 12:48 PM joris.vanagtmaal <[hidden email]> wrote:

I worked out the rowtype input for the conversion to datastream;

type_info = Types.ROW_NAMED(['sender', 'stw', 'time'],[Types.STRING(),
Types.DOUBLE(), Types.LONG()])
datastream=table_env.to_append_stream(my_table, type_info)

But if i try to assign rowtime and watermarks to the datastream and convert
it back to a table, the rowtime/watermarks either get dropped or maybe they
are not created properly.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[joris.vanagtmaal]

Thanks Roman,

somehow i must have missed this in the documentation.

What is the difference (if any) between:

Ascending timestamps:
WATERMARK FOR rowtime_column AS rowtime_column - INTERVAL '0.001' SECOND.

Bounded out of orderness timestamps:
WATERMARK FOR rowtime_column AS rowtime_column - INTERVAL 'string' timeUnit.





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Roman Khachatryan]

The watermark resolution in Flink is one millisecond [1], so the 1st form essentially doesn't allow out-of-orderness (though the elements with the same timestamp are not considered late in this case).

[1]

https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_timestamps_watermarks.html

On Wed, Feb 24, 2021 at 7:54 AM joris.vanagtmaal <[hidden email]> wrote:

Thanks Roman,

somehow i must have missed this in the documentation.

What is the difference (if any) between:

Ascending timestamps:
WATERMARK FOR rowtime_column AS rowtime_column - INTERVAL '0.001' SECOND.

Bounded out of orderness timestamps:
WATERMARK FOR rowtime_column AS rowtime_column - INTERVAL 'string' timeUnit.





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[joris.vanagtmaal]

Ah thanks, so event though the method of describing it is exactly the same,
because you're using the max resolution it isn't useful for
out-of-orderness. Ok, clear



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/