Flink Kafka connector in Python

Posted by Manas Kale on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-Kafka-connector-in-Python-tp36270.html

Hi,

I want to consume and write to Kafak from Flink's python API. 

The only way I found to do this was through this question on SO where the user essentially copies FlinkKafka connector JARs into the Flink runtime's lib/ directory. 

• Is this the recommended method to do this? If not, what is?
  
• Is there any official documentation for using Kafka with pyFlink? Is this officially supported? 
• How does the method described in the link work? Does the Flink runtime load and expose all JARs in /lib to the python script? Can I write custom operators in Java and use those through python?

Thanks,

Manas