Uploaded jar disappears when web monitor restarts

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Uploaded jar disappears when web monitor restarts

Emanuele Cesena
Hi,

When the web monitor restarts the uploaded jars disappear — in fact, every time it restarts the upload directory is different.

This seems intentional:
https://github.com/apache/flink/blob/master/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java#L162

Could anyone confirm?

Wouldn’t it be useful to have a config param to be able to set it permanently (and thus avoiding jars to be deleted)? Or, what is the intended way to “add jars to my cluster”?

Thank you,
E.

Reply | Threaded
Open this post in threaded view
|

Re: Uploaded jar disappears when web monitor restarts

Till Rohrmann
Hi Emanuele,

you're right, the upload directory for the jars is deleted whenever the WebRuntimeMonitor is stopped and a new directory is created when the WebRuntimeMonitor is started.

 It would be doable to introduce a configuration parameter for the uploaded jar directory so that you don't have to upload the jars every time again. But then the user would have to responsible for cleaning it up. If you want, then you can open a JIRA issue for this feature.

I would recommend using the CLI for starting Flink jobs. The reason is that the WebRuntimeMonitor does not support jobs which contain eager execution trigger calls such as collect, count and print, for example.

Cheers,
Till

On Wed, Jun 8, 2016 at 5:06 AM, Emanuele Cesena <[hidden email]> wrote:
Hi,

When the web monitor restarts the uploaded jars disappear — in fact, every time it restarts the upload directory is different.

This seems intentional:
https://github.com/apache/flink/blob/master/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java#L162

Could anyone confirm?

Wouldn’t it be useful to have a config param to be able to set it permanently (and thus avoiding jars to be deleted)? Or, what is the intended way to “add jars to my cluster”?

Thank you,
E.


Reply | Threaded
Open this post in threaded view
|

Re: Uploaded jar disappears when web monitor restarts

Emanuele Cesena
Thanks for the clarifications (and for the note on eager exec, I’ll read more about it).

I’ll open a JIRA ticket then.

Best,


> On Jun 8, 2016, at 1:57 AM, Till Rohrmann <[hidden email]> wrote:
>
> Hi Emanuele,
>
> you're right, the upload directory for the jars is deleted whenever the WebRuntimeMonitor is stopped and a new directory is created when the WebRuntimeMonitor is started.
>
>  It would be doable to introduce a configuration parameter for the uploaded jar directory so that you don't have to upload the jars every time again. But then the user would have to responsible for cleaning it up. If you want, then you can open a JIRA issue for this feature.
>
> I would recommend using the CLI for starting Flink jobs. The reason is that the WebRuntimeMonitor does not support jobs which contain eager execution trigger calls such as collect, count and print, for example.
>
> Cheers,
> Till
>
> On Wed, Jun 8, 2016 at 5:06 AM, Emanuele Cesena <[hidden email]> wrote:
> Hi,
>
> When the web monitor restarts the uploaded jars disappear — in fact, every time it restarts the upload directory is different.
>
> This seems intentional:
> https://github.com/apache/flink/blob/master/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/WebRuntimeMonitor.java#L162
>
> Could anyone confirm?
>
> Wouldn’t it be useful to have a config param to be able to set it permanently (and thus avoiding jars to be deleted)? Or, what is the intended way to “add jars to my cluster”?
>
> Thank you,
> E.
>
>