Flink with Openstack Swift example

[Tejash Tarun]

Hi Users,

Can somebody help me by pointing to any document listing out the steps if I want to configure checkpoint and savepoint in OpenStack Swift ?

Thanks in advance,

Tejash

[Andrey Zagrebin]

Hi Tejash,

Flink has an implementation for swift file system (see SwiftFileSystemFactory).

It should be enough just prepend your checkpoint/savepoint paths with “swift://“ (e.g. instead of  "hdfs://“).

See configuration 'how to’ in Flink docs [1], [2].

Best,

Andrey

[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/state/checkpoints.html

[2] https://ci.apache.org/projects/flink/flink-docs-master/ops/state/savepoints.html

On 12 Dec 2018, at 14:54, Tejash Tarun <[hidden email]> wrote:

Hi Users,

Can somebody help me by pointing to any document listing out the steps if I want to configure checkpoint and savepoint in OpenStack Swift ?

Thanks in advance,

Tejash

[Tejash Tarun]

Hi Andrey,

Thanks for the help. Let me try it out.

Regards,

Tejash

On Wed, Dec 12, 2018 at 10:32 PM Andrey Zagrebin <[hidden email]> wrote:

Hi Tejash,

Flink has an implementation for swift file system (see SwiftFileSystemFactory).

It should be enough just prepend your checkpoint/savepoint paths with “swift://“ (e.g. instead of  "hdfs://“).

See configuration 'how to’ in Flink docs [1], [2].

Best,

Andrey

[1] https://ci.apache.org/projects/flink/flink-docs-master/ops/state/checkpoints.html

[2] https://ci.apache.org/projects/flink/flink-docs-master/ops/state/savepoints.html

On 12 Dec 2018, at 14:54, Tejash Tarun <[hidden email]> wrote:

Hi Users,

Can somebody help me by pointing to any document listing out the steps if I want to configure checkpoint and savepoint in OpenStack Swift ?

Thanks in advance,

Tejash