Connecting the new DataSource API (FLIP-27) with DynamicTableSource

[Yuval Itzchakov]

Hi,

I've implemented a new custom source using the new DataSource API (https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/stream/sources.html) and I want to connect it to the new DynamicTableSource API. 

However, in ScanTableSource, the getScanRuntimeProvider method returns a ScanRuntimeProvider interface, which only has an implementation for the old SourceFunction[T], and not for the new Source[T] class:

Does the Table API not yet support the new Source API? or am I missing something? 

--

[Jingsong Li]

Hi Yuval,

Yes, The new table source does not support the new Source API in Flink 1.11. The integration is introduced in Flink master (1.12):

https://cwiki.apache.org/confluence/display/FLINK/FLIP-146%3A+Improve+new+TableSource+and+TableSink+interfaces

Best,

Jingsong

On Sun, Nov 1, 2020 at 10:54 PM Yuval Itzchakov <[hidden email]> wrote:

Hi,

I've implemented a new custom source using the new DataSource API (https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/stream/sources.html) and I want to connect it to the new DynamicTableSource API. 

However, in ScanTableSource, the getScanRuntimeProvider method returns a ScanRuntimeProvider interface, which only has an implementation for the old SourceFunction[T], and not for the new Source[T] class:

Does the Table API not yet support the new Source API? or am I missing something? 

--

Best Regards,
Yuval Itzchakov.

--