Kafka table descriptor missing startFromTimestamp()

[Steve Whelan]

Examining the org.apache.flink.table.descriptors.Kafka class in Flink v1.8, it has the following startUpModes for consumers:
    .startFromEarliest()
    .startFromLatest()
    .startFromSpecificOffsets(...)

However, it does not have a method to support starting from a Timestamp. The FlinkKafkaConsumer supports this feature though. Was it a conscience decision to leave that start up mode out of the table descriptor? If so, what was the reasoning? 

[Dawid Wysakowicz-2]

Hi,

The only reason why it was not exposed at the beginning is that not all
versions of the consumers support starting from a specific timestamp. I
think we could expose such setting now. Would you like to create an
issue for it?

Best,

Dawid

On 19/12/2019 06:56, Steve Whelan wrote:

> Examining the org.apache.flink.table.descriptors.Kafka class in Flink
> v1.8, it has the following startUpModes for consumers:
>     .startFromEarliest()
>     .startFromLatest()
>     .startFromSpecificOffsets(...)
>
> However, it does not have a method to support starting from a
> Timestamp. The FlinkKafkaConsumer supports this feature though. Was it
> a conscience decision to leave that start up mode out of the table
> descriptor? If so, what was the reasoning? 

[Timo Walther]

This issue is work in progress:

https://issues.apache.org/jira/browse/FLINK-15220


On 19.12.19 09:07, Dawid Wysakowicz wrote:

> Hi,
>
> The only reason why it was not exposed at the beginning is that not all
> versions of the consumers support starting from a specific timestamp. I
> think we could expose such setting now. Would you like to create an
> issue for it?
>
> Best,
>
> Dawid
>
> On 19/12/2019 06:56, Steve Whelan wrote:
>> Examining the org.apache.flink.table.descriptors.Kafka class in Flink
>> v1.8, it has the following startUpModes for consumers:
>>      .startFromEarliest()
>>      .startFromLatest()
>>      .startFromSpecificOffsets(...)
>>
>> However, it does not have a method to support starting from a
>> Timestamp. The FlinkKafkaConsumer supports this feature though. Was it
>> a conscience decision to leave that start up mode out of the table
>> descriptor? If so, what was the reasoning?
>