Parameter to config read frequency in Kafka SQL connector

[Kai Fu]

Hi team,

We are in a situatoin that we want to reduce the read frequency of Kafka SQL connector. I did some investigation on the properties of Kafka client, while it seems it does not have such options. Athough I found the batch size config('properties.max.partition.fetch.bytes') among the config options.

https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html

Any suggestions on this?

[Dawid Wysakowicz-2]

Hi,

Unfortunately I have no experience with this. You can pass any Kafka
client parameters through the properties.* option[1] and see if the
setting works for you.

Best,

Dawid

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connectors/kafka.html#properties

On 18/03/2021 12:50, eef hhj wrote:

> Hi team,
>
> We are in a situatoin that we want to reduce the read frequency of
> Kafka SQL connector. I did some investigation on the properties of
> Kafka client, while it seems it does not have such options. Athough I
> found the batch size config('properties.max.partition.fetch.bytes')
> among the config options.
>
> https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html
> <https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html>
>
> Any suggestions on this?
>

[Kai Fu]

Hi Dawid,

Thank you for the reply, I do not find such property in Kafka connector's config, while it's easy to customize with the passed in properties parameter.

-- Best wishes

Kai 

On Fri, Mar 19, 2021 at 5:21 PM Dawid Wysakowicz <[hidden email]> wrote:

Hi,

Unfortunately I have no experience with this. You can pass any Kafka
client parameters through the properties.* option[1] and see if the
setting works for you.

Best,

Dawid

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/table/connectors/kafka.html#properties

On 18/03/2021 12:50, eef hhj wrote:
> Hi team,
>
> We are in a situatoin that we want to reduce the read frequency of
> Kafka SQL connector. I did some investigation on the properties of
> Kafka client, while it seems it does not have such options. Athough I
> found the batch size config('properties.max.partition.fetch.bytes')
> among the config options.
>
> https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html
> <https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html>
>
> Any suggestions on this?
>

--