S3 Checkpoint Storage

[John Sherwood]

Hello all,

I'm attempting to set up a taskmanager cluster using S3 as the highly-available store. It looks like the main thing is just setting the `state.backend.fs.checkpointdir` to the appropriate s3:// URI, but as someone rather new to accessing S3 from Java, how should I provide Flink with the credentials necessary to access the bucket I've set up?

Thanks for your time & any help!

[Fabian Hueske-2]

Hi John,

S3 keys are configured via Hadoop's configuration files.

Check out the documentation for AWS setups [1].

Cheers, Fabian

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.0/setup/aws.html

2016-05-02 20:22 GMT+02:00 John Sherwood <[hidden email]>:

Hello all,

I'm attempting to set up a taskmanager cluster using S3 as the highly-available store. It looks like the main thing is just setting the `state.backend.fs.checkpointdir` to the appropriate s3:// URI, but as someone rather new to accessing S3 from Java, how should I provide Flink with the credentials necessary to access the bucket I've set up?

Thanks for your time & any help!