Does apache flink support stream input from Postgresql ?

[Yu Watanabe]

Hello .

I would like to ask question about possibility of stream read table rows  from PostgresQL using pyflink.

In use cases in online document, there is a example which is reading data from transactional database.

https://flink.apache.org/img/usecases-datapipelines.png

Is it possible to stream read from PostgresQL  using pyflink ?

Thanks,

Yu Watanabe

--

[Zhenghua Gao]

The jdbc connector can read data from PostgreSQL for Table/SQL users.

For pyflink, cc @Hequn

On Wed, Nov 20, 2019 at 7:56 PM Yu Watanabe <[hidden email]> wrote:

Hello .

I would like to ask question about possibility of stream read table rows  from PostgresQL using pyflink.

In use cases in online document, there is a example which is reading data from transactional database.

https://flink.apache.org/img/usecases-datapipelines.png

Is it possible to stream read from PostgresQL  using pyflink ?

Thanks,

Yu Watanabe

--

Yu Watanabe

Weekend Freelancer who loves to challenge building data platform

[hidden email]

    

[Theo]

Hi,

You could check out Debezium [1] and get your database CDC into a kafka stream which can then be read nicely by flink.

Best regards

Theo

[1] https://debezium.io/documentation/reference/1.0/tutorial.html

---

Von: "Yu Watanabe" <[hidden email]>
An: "user" <[hidden email]>
Gesendet: Mittwoch, 20. November 2019 12:56:18
Betreff: Does apache flink support stream input from Postgresql ?

Hello .

I would like to ask question about possibility of stream read table rows  from PostgresQL using pyflink.

In use cases in online document, there is a example which is reading data from transactional database.

https://flink.apache.org/img/usecases-datapipelines.png

Is it possible to stream read from PostgresQL  using pyflink ?

Thanks,

Yu Watanabe

--

Yu Watanabe

Weekend Freelancer who loves to challenge building data platform

[hidden email]

    

--

SCOOP Software GmbH - Gut Maarhausen - Eiler Straße 3 P - D-51107 Köln
Theo Diefenthal

T [phone] - F [phone] - M [phone]
[hidden email] - www.scoop-software.de
Sitz der Gesellschaft: Köln, Handelsregister: Köln,
Handelsregisternummer: HRB 36625
Geschäftsführung: Dr. Oleg Balovnev, Frank Heinen,
Martin Müller-Rohde, Dr. Wolfgang Reddig, Roland Scheel