Need assistance : creating remote environment

[Mohan mohan]

Hi,

I have started flink in cluster mode.   ..flink1.4.2/bin/$ ./start-cluster.sh (no config changes ie., default settings)

And trying to connect to it,

ExecutionEnvironment.createRemoteEnvironment("localhost", 6123, "xxx.jar");

I am not seeing any response, did not find anything in jobmanager log. Please guide how to trace the issue. Or Do we need any preconfiguration?

In local environment everything works fine. 

Using : flink-1.4.2-bin-scala_2.11.tgz

Thanks in advance.

[Chesnay Schepler]

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:

> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.

[Mohan mohan]

Attached log file. (Log level : Trace)

Is this the issue ? Trying with very minimal graph (execution plan is printed in log file)

WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [/127.0.0.1:44322] failed with org.apache.flink.shaded.akka.org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 10485760: 1195725860 - discarded

On Wed, Jul 11, 2018 at 5:44 PM Chesnay Schepler <[hidden email]> wrote:

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:
> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.

[Chesnay Schepler]

Based on the logs your client is using the RestClusterClient, which means that the client is either
a) running 1.4 with the flip6 profile enabled
b) running 1.5.

Please ensure that both the flink versions match for client and server, and that both run/do not run in flip6 mode.

On 11.07.2018 14:26, Mohan mohan wrote:

Attached log file. (Log level : Trace)

Is this the issue ? Trying with very minimal graph (execution plan is printed in log file)

WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [/127.0.0.1:44322] failed with org.apache.flink.shaded.akka.org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 10485760: 1195725860 - discarded

On Wed, Jul 11, 2018 at 5:44 PM Chesnay Schepler <[hidden email]> wrote:

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:
> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.

[Mohan mohan]

Yep, that is due to version mismatch. Working fine now. 

Thank you Chesnay Schepler.

On Wed, Jul 11, 2018 at 6:05 PM Chesnay Schepler <[hidden email]> wrote:

Based on the logs your client is using the RestClusterClient, which means that the client is either
a) running 1.4 with the flip6 profile enabled
b) running 1.5.

Please ensure that both the flink versions match for client and server, and that both run/do not run in flip6 mode.

On 11.07.2018 14:26, Mohan mohan wrote:

Attached log file. (Log level : Trace)

Is this the issue ? Trying with very minimal graph (execution plan is printed in log file)

WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [/127.0.0.1:44322] failed with org.apache.flink.shaded.akka.org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 10485760: 1195725860 - discarded

On Wed, Jul 11, 2018 at 5:44 PM Chesnay Schepler <[hidden email]> wrote:

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:
> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.

[Mohan mohan]

Hi,

 what is flip6? "Flink Improvement Proposal" ?

How to enable/disable flip mode in below cases,

  1. While starting cluster using ./start-cluster.sh

  2. While connecting using ExecutionEnvironment.createRemoteEnvironment("localhost", 6123, "xyz.jar") 

Thank you.

On Thu, Jul 12, 2018 at 9:32 PM Mohan mohan <[hidden email]> wrote:

Yep, that is due to version mismatch. Working fine now. 

Thank you Chesnay Schepler.

On Wed, Jul 11, 2018 at 6:05 PM Chesnay Schepler <[hidden email]> wrote:

Based on the logs your client is using the RestClusterClient, which means that the client is either
a) running 1.4 with the flip6 profile enabled
b) running 1.5.

Please ensure that both the flink versions match for client and server, and that both run/do not run in flip6 mode.

On 11.07.2018 14:26, Mohan mohan wrote:

Attached log file. (Log level : Trace)

Is this the issue ? Trying with very minimal graph (execution plan is printed in log file)

WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [/127.0.0.1:44322] failed with org.apache.flink.shaded.akka.org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 10485760: 1195725860 - discarded

On Wed, Jul 11, 2018 at 5:44 PM Chesnay Schepler <[hidden email]> wrote:

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:
> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.

[Chesnay Schepler]

FLIP-6  is/was an initiative to rework Flink distributed model.

Contrary to my previous reply this wasn't introduced in 1.4, please disregard explanation a).

Thus, the only remaining explanation is that you're running 1.5 in the client/IDE.
I heavily recommend to make sure that your client/IDE uses 1.4

For completeness sake, to disable FLIP-6
* within the IDE, enable the "legacyCode" maven profile
* for the client, configure the mode parameter.

On 13.07.2018 12:20, Mohan mohan wrote:

Hi,

 what is flip6? "Flink Improvement Proposal" ?

How to enable/disable flip mode in below cases,

  1. While starting cluster using ./start-cluster.sh

  2. While connecting using ExecutionEnvironment.createRemoteEnvironment("localhost", 6123, "xyz.jar") 

Thank you.

On Thu, Jul 12, 2018 at 9:32 PM Mohan mohan <[hidden email]> wrote:

Yep, that is due to version mismatch. Working fine now. 

Thank you Chesnay Schepler.

On Wed, Jul 11, 2018 at 6:05 PM Chesnay Schepler <[hidden email]> wrote:

Based on the logs your client is using the RestClusterClient, which means that the client is either
a) running 1.4 with the flip6 profile enabled
b) running 1.5.

Please ensure that both the flink versions match for client and server, and that both run/do not run in flip6 mode.

On 11.07.2018 14:26, Mohan mohan wrote:

Attached log file. (Log level : Trace)

Is this the issue ? Trying with very minimal graph (execution plan is printed in log file)

WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [/127.0.0.1:44322] failed with org.apache.flink.shaded.akka.org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 10485760: 1195725860 - discarded

On Wed, Jul 11, 2018 at 5:44 PM Chesnay Schepler <[hidden email]> wrote:

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:
> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.

[Mohan mohan]

Hi Chesnay,

Now server and client/IDE both are using 1.4.0. 

When I submit job from IDE getting below exception,

com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka.remote.artery'

I have used shadow jar. Thought problem is with merging jars. But I could not find above property 'akka.remote.artery' in any of akka jars.

Please guide.

On Fri, Jul 13, 2018 at 4:03 PM Chesnay Schepler <[hidden email]> wrote:

FLIP-6  is/was an initiative to rework Flink distributed model.

Contrary to my previous reply this wasn't introduced in 1.4, please disregard explanation a).

Thus, the only remaining explanation is that you're running 1.5 in the client/IDE.
I heavily recommend to make sure that your client/IDE uses 1.4

For completeness sake, to disable FLIP-6
* within the IDE, enable the "legacyCode" maven profile
* for the client, configure the mode parameter.

On 13.07.2018 12:20, Mohan mohan wrote:

Hi,

 what is flip6? "Flink Improvement Proposal" ?

How to enable/disable flip mode in below cases,

  1. While starting cluster using ./start-cluster.sh

  2. While connecting using ExecutionEnvironment.createRemoteEnvironment("localhost", 6123, "xyz.jar") 

Thank you.

On Thu, Jul 12, 2018 at 9:32 PM Mohan mohan <[hidden email]> wrote:

Yep, that is due to version mismatch. Working fine now. 

Thank you Chesnay Schepler.

On Wed, Jul 11, 2018 at 6:05 PM Chesnay Schepler <[hidden email]> wrote:

Based on the logs your client is using the RestClusterClient, which means that the client is either
a) running 1.4 with the flip6 profile enabled
b) running 1.5.

Please ensure that both the flink versions match for client and server, and that both run/do not run in flip6 mode.

On 11.07.2018 14:26, Mohan mohan wrote:

Attached log file. (Log level : Trace)

Is this the issue ? Trying with very minimal graph (execution plan is printed in log file)

WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [/127.0.0.1:44322] failed with org.apache.flink.shaded.akka.org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 10485760: 1195725860 - discarded

On Wed, Jul 11, 2018 at 5:44 PM Chesnay Schepler <[hidden email]> wrote:

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:
> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.

[Mohan mohan]

Updated akka's 'reference.conf' file in my app shadow jar. Working fine now.

Thank you.

On Fri, Jul 13, 2018 at 10:46 PM Mohan mohan <[hidden email]> wrote:

Hi Chesnay,

Now server and client/IDE both are using 1.4.0. 

When I submit job from IDE getting below exception,

com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka.remote.artery'

I have used shadow jar. Thought problem is with merging jars. But I could not find above property 'akka.remote.artery' in any of akka jars.

Please guide.

On Fri, Jul 13, 2018 at 4:03 PM Chesnay Schepler <[hidden email]> wrote:

FLIP-6  is/was an initiative to rework Flink distributed model.

Contrary to my previous reply this wasn't introduced in 1.4, please disregard explanation a).

Thus, the only remaining explanation is that you're running 1.5 in the client/IDE.
I heavily recommend to make sure that your client/IDE uses 1.4

For completeness sake, to disable FLIP-6
* within the IDE, enable the "legacyCode" maven profile
* for the client, configure the mode parameter.

On 13.07.2018 12:20, Mohan mohan wrote:

Hi,

 what is flip6? "Flink Improvement Proposal" ?

How to enable/disable flip mode in below cases,

  1. While starting cluster using ./start-cluster.sh

  2. While connecting using ExecutionEnvironment.createRemoteEnvironment("localhost", 6123, "xyz.jar") 

Thank you.

On Thu, Jul 12, 2018 at 9:32 PM Mohan mohan <[hidden email]> wrote:

Yep, that is due to version mismatch. Working fine now. 

Thank you Chesnay Schepler.

On Wed, Jul 11, 2018 at 6:05 PM Chesnay Schepler <[hidden email]> wrote:

Based on the logs your client is using the RestClusterClient, which means that the client is either
a) running 1.4 with the flip6 profile enabled
b) running 1.5.

Please ensure that both the flink versions match for client and server, and that both run/do not run in flip6 mode.

On 11.07.2018 14:26, Mohan mohan wrote:

Attached log file. (Log level : Trace)

Is this the issue ? Trying with very minimal graph (execution plan is printed in log file)

WARN  akka.remote.transport.netty.NettyTransport                    - Remote connection to [/127.0.0.1:44322] failed with org.apache.flink.shaded.akka.org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame length exceeds 10485760: 1195725860 - discarded

On Wed, Jul 11, 2018 at 5:44 PM Chesnay Schepler <[hidden email]> wrote:

Did/could you enable logging in the submitting code?

On 11.07.2018 13:57, Mohan mohan wrote:
> Hi,
>
> I have started flink in cluster mode.   ..flink1.4.2/bin/$
> ./start-cluster.sh (no config changes ie., default settings)
> And trying to connect to it,
> ExecutionEnvironment.createRemoteEnvironment("localhost", 6123,
> "xxx.jar");
>
> I am not seeing any response, did not find anything in jobmanager log.
> Please guide how to trace the issue. Or Do we need any preconfiguration?
> In local environment everything works fine.
>
> Using : flink-1.4.2-bin-scala_2.11.tgz
>
> Thanks in advance.