Starting beam pipeline from savepoint

[borisbolvig]

Is it possible to start Beam pipelines from savepoints when running on Flink
(v.1.4)?

I am running flink in a remote environment, and executing the pipelines as
.jars specifying the flink jobmanager via cmd line arguments. This is
opposed to passing the .jar to `flink run`.

In this way the jar files are not directly available to the jobmanager, and
indeed the API /jars returns empty.

Now to restart from a savepoint (either using the REST API or CLI) it is
expected that I pass the pipeline jar. However this is not available to the
jobmanager.

Note that automatic restarting from checkpoints and creating savepoints is
working.

Thank you for your help.





--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Piotr Nowojski]

Hi,

I am not sure, but it probably brings down to the question, whether you can restore from a save point when using RemoteEnvironment - and answer for this question is unfortunately no.

Maybe Aljoscha will know something more.

Piotrek

> On 23 May 2018, at 19:25, borisbolvig <[hidden email]> wrote:
>
> Is it possible to start Beam pipelines from savepoints when running on Flink
> (v.1.4)?
>
> I am running flink in a remote environment, and executing the pipelines as
> .jars specifying the flink jobmanager via cmd line arguments. This is
> opposed to passing the .jar to `flink run`.
>
> In this way the jar files are not directly available to the jobmanager, and
> indeed the API /jars returns empty.
>
> Now to restart from a savepoint (either using the REST API or CLI) it is
> expected that I pass the pipeline jar. However this is not available to the
> jobmanager.
>
> Note that automatic restarting from checkpoints and creating savepoints is
> working.
>
> Thank you for your help.
>
>
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/