Kafka SQL Connector Behavior (v1.11)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Kafka SQL Connector Behavior (v1.11)

Aeden Jameson
Based on these docs,
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/kafka.html,
the default partitioning behavior is not quite clear to me.
If no value for sink-partitioner is given, is the default behavior
just that of the native Kafka library? (with key use murmur2 , without
key round robin)

Thank you,
Aeden
Reply | Threaded
Open this post in threaded view
|

Re: Kafka SQL Connector Behavior (v1.11)

Arvid Heise-3
Hi Aeden,

I just checked the code and your assumption is correct. Without an explicit partitioner, Flink just writes ProducerRecord without partition (null), so that whatever Kafka usually does applies.

On Tue, Jan 5, 2021 at 1:53 AM Aeden Jameson <[hidden email]> wrote:
Based on these docs,
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/connectors/kafka.html,
the default partitioning behavior is not quite clear to me.
If no value for sink-partitioner is given, is the default behavior
just that of the native Kafka library? (with key use murmur2 , without
key round robin)

Thank you,
Aeden


--

Arvid Heise | Senior Java Developer


Follow us @VervericaData

--

Join Flink Forward - The Apache Flink Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--

Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng