Use s3 on Flink on kubernetes

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Use s3 on Flink on kubernetes

William Saar-2

How can I easiest use s3 from a Flink job deployed in a session cluster on kubernetes?

I've tried including the flink-s3-fs-hadoop dependency in the sbt file for my job, can I programmatically set the properties to point to it? Is there a ready-made docker image for a flink with s3 dependencies configured to deploy in the jobmanager/taskmanager deployments?