LookupableTableSource from kafka consumer

[Clay Teeter]

Hey, does anyone have any examples that i can use to create a LookupableTableSource from a kafka topic?

Thanks!

Clay

[Jark Wu-3]

Hey, 

You can take JDBCTableSource [1] as an example about how to implement a LookupableTableSource.

However, I'm not sure how to support lookup for kafka. Because AFAIK, kafka doesn't have the ability to lookup by key?

Best,

Jark

[1]: https://github.com/apache/flink/blob/master/flink-connectors/flink-jdbc/src/main/java/org/apache/flink/api/java/io/jdbc/JDBCTableSource.java

On Tue, 21 Apr 2020 at 19:44, Clay Teeter <[hidden email]> wrote:

Hey, does anyone have any examples that i can use to create a LookupableTableSource from a kafka topic?

Thanks!

Clay

[Danny Chan]

We usually implementation a LookupableTableSource based on k-v store data sources, such as the ES, Hbase and Redis.

For Kafka, what we usually do is a regular stream join [1]

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/streaming/joins.html#regular-joins

Best,

Danny Chan

在 2020年4月21日 +0800 PM8:02，Jark Wu <[hidden email]>，写道：

Hey, 

You can take JDBCTableSource [1] as an example about how to implement a LookupableTableSource.

However, I'm not sure how to support lookup for kafka. Because AFAIK, kafka doesn't have the ability to lookup by key?

Best,

Jark

[1]: https://github.com/apache/flink/blob/master/flink-connectors/flink-jdbc/src/main/java/org/apache/flink/api/java/io/jdbc/JDBCTableSource.java

On Tue, 21 Apr 2020 at 19:44, Clay Teeter <[hidden email]> wrote:

Hey, does anyone have any examples that i can use to create a LookupableTableSource from a kafka topic?

Thanks!

Clay