Is it a good practice to create lots of jobs in a flink cluster ?

[Christophe Julien]

Hi,

I wonder if create and delete lots of jobs in a cluster can be a problem ?

From my point of view the impact will be limited, because the task Manager won't be impacted during the process, it is only a task slot available or not.

But is it really "free" or a good practice to do that ? Especially in production, is it a "stable" practice ?

Thanks,

Christophe

[Maximilian Michels]

It is safe to run multiple jobs concurrently in a Flink cluster. If we
care about resource isolation, you are probably better off with Flink
on Yarn.

That said, we're moving away from this model in FLIP-6 towards a
cluster-per-job model:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077

-Max

On Thu, Oct 6, 2016 at 10:42 AM, Christophe Julien
<[hidden email]> wrote:

> Hi,
>
> I wonder if create and delete lots of jobs in a cluster can be a problem ?
>
> From my point of view the impact will be limited, because the task Manager
> won't be impacted during the process, it is only a task slot available or
> not.
> But is it really "free" or a good practice to do that ? Especially in
> production, is it a "stable" practice ?
>
>
> Thanks,
> Christophe