How to run Flink 1.6 job cluster in "standalone" mode?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to run Flink 1.6 job cluster in "standalone" mode?

Hao Sun
"Standalone" here I mean job-mananger + taskmanager on the same JVM. I have an issue to debug on our K8S environment, I can not reproduce it in local docker env or Intellij. If JM and TM are running in different VMs, it makes things harder to debug.

Or is there a way to debug a job running on JM + TM on different VMs?
Is reverting to session cluster the only way to get JM + TM on the same VM?
Reply | Threaded
Open this post in threaded view
|

Re: How to run Flink 1.6 job cluster in "standalone" mode?

Timo Walther
Hi,

a session cluster does not imply that JM + TM are always executed in the
same JVM. Debugging a job running on different JVMs might be a bit more
difficult to debug but it should still be straightforward.

Maybe you can tell us what wrong behavior you observe?

Btw. Flink's metrics can also already be quite helpful.

Regards,
Timo

Am 07.11.18 um 14:15 schrieb Hao Sun:
> "Standalone" here I mean job-mananger + taskmanager on the same JVM. I
> have an issue to debug on our K8S environment, I can not reproduce it
> in local docker env or Intellij. If JM and TM are running in different
> VMs, it makes things harder to debug.
>
> Or is there a way to debug a job running on JM + TM on different VMs?
> Is reverting to session cluster the only way to get JM + TM on the
> same VM?


Reply | Threaded
Open this post in threaded view
|

Re: How to run Flink 1.6 job cluster in "standalone" mode?

Hao Sun
Hi Tim, I am trying to debug this issue
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/java-io-IOException-NSS-is-already-initialized-td24247.html

And in general, how to debug code in a distributed JM/TM architecture is very interesting topic to me.
Any hints will be appreciated. Thanks

On Mon, Nov 12, 2018 at 7:12 AM Timo Walther <[hidden email]> wrote:
Hi,

a session cluster does not imply that JM + TM are always executed in the
same JVM. Debugging a job running on different JVMs might be a bit more
difficult to debug but it should still be straightforward.

Maybe you can tell us what wrong behavior you observe?

Btw. Flink's metrics can also already be quite helpful.

Regards,
Timo

Am 07.11.18 um 14:15 schrieb Hao Sun:
> "Standalone" here I mean job-mananger + taskmanager on the same JVM. I
> have an issue to debug on our K8S environment, I can not reproduce it
> in local docker env or Intellij. If JM and TM are running in different
> VMs, it makes things harder to debug.
>
> Or is there a way to debug a job running on JM + TM on different VMs?
> Is reverting to session cluster the only way to get JM + TM on the
> same VM?