Flink 1.4.2 in Zeppelin Notebook

[Rico Bergmann]

Hi!

Has someone successfully integrated Flink 1.4.2 into Zeppelin notebook
(using Flink in cluster mode, not local mode)?

Best,

Rico.

[kedar mhaswade]

Yes. You need to add the two properties for the job manager (I agree, it is confusing because the properties named "host" and "port" already available, but the names of the useful properties are different):

Could you please try this and let us know if it works for you?

Regards,

Kedar

On Fri, Apr 6, 2018 at 5:51 AM, Dipl.-Inf. Rico Bergmann <[hidden email]> wrote:

Hi!

Has someone successfully integrated Flink 1.4.2 into Zeppelin notebook (using Flink in cluster mode, not local mode)?

Best,

Rico.

[Rico Bergmann]

Hi. 

Thanks for your reply. But this also didn’t work for me. 

In the JM log I get an akka Error („dropping message for non-local recipient“). 

My setup: I have Flink running on Kubernetes cluster, version 1.4.2. zeppelin is version 0.8 using the flink interpreter compiled against flink 1.1.3. 

When submitting a job with the CLI tool everything is working fine. The CLI tool is version 1.4.2 ...

Any other suggestions?

Thanks a lot. 

Best,

Rico. 

Am 06.04.2018 um 18:44 schrieb kedar mhaswade <[hidden email]>:

Yes. You need to add the two properties for the job manager (I agree, it is confusing because the properties named "host" and "port" already available, but the names of the useful properties are different):

Could you please try this and let us know if it works for you?

Regards,

Kedar

On Fri, Apr 6, 2018 at 5:51 AM, Dipl.-Inf. Rico Bergmann <[hidden email]> wrote:

Hi!

Has someone successfully integrated Flink 1.4.2 into Zeppelin notebook (using Flink in cluster mode, not local mode)?

Best,

Rico.

[kedar mhaswade]

Hmm. What error do you see on the Zeppelin console when you click the run (flink code) button after making these changes for flink interpreter config (I assume you restart the interpreter)?

Regards,

Kedar

On Mon, Apr 9, 2018 at 12:50 AM, Rico Bergmann <[hidden email]> wrote:

Hi. 

Thanks for your reply. But this also didn’t work for me. 

In the JM log I get an akka Error („dropping message for non-local recipient“). 

My setup: I have Flink running on Kubernetes cluster, version 1.4.2. zeppelin is version 0.8 using the flink interpreter compiled against flink 1.1.3. 

When submitting a job with the CLI tool everything is working fine. The CLI tool is version 1.4.2 ...

Any other suggestions?

Thanks a lot. 

Best,

Rico. 

Am 06.04.2018 um 18:44 schrieb kedar mhaswade <[hidden email]>:

Yes. You need to add the two properties for the job manager (I agree, it is confusing because the properties named "host" and "port" already available, but the names of the useful properties are different):

Could you please try this and let us know if it works for you?

Regards,

Kedar

On Fri, Apr 6, 2018 at 5:51 AM, Dipl.-Inf. Rico Bergmann <[hidden email]> wrote:

Hi!

Has someone successfully integrated Flink 1.4.2 into Zeppelin notebook (using Flink in cluster mode, not local mode)?

Best,

Rico.

[Rico Bergmann]

The error message is:

org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Communication with JobManager failed: Lost connection to the JobManager.

Am 09.04.2018 um 14:12 schrieb kedar mhaswade:

Hmm. What error do you see on the Zeppelin console when you click the run (flink code) button after making these changes for flink interpreter config (I assume you restart the interpreter)?

Regards,

Kedar

On Mon, Apr 9, 2018 at 12:50 AM, Rico Bergmann <[hidden email]> wrote:

Hi. 

Thanks for your reply. But this also didn’t work for me. 

In the JM log I get an akka Error („dropping message for non-local recipient“). 

My setup: I have Flink running on Kubernetes cluster, version 1.4.2. zeppelin is version 0.8 using the flink interpreter compiled against flink 1.1.3. 

When submitting a job with the CLI tool everything is working fine. The CLI tool is version 1.4.2 ...

Any other suggestions?

Thanks a lot. 

Best,

Rico. 

Am 06.04.2018 um 18:44 schrieb kedar mhaswade <[hidden email]>:

Yes. You need to add the two properties for the job manager (I agree, it is confusing because the properties named "host" and "port" already available, but the names of the useful properties are different):

Could you please try this and let us know if it works for you?

Regards,

Kedar

On Fri, Apr 6, 2018 at 5:51 AM, Dipl.-Inf. Rico Bergmann <[hidden email]> wrote:

Hi!

Has someone successfully integrated Flink 1.4.2 into Zeppelin notebook (using Flink in cluster mode, not local mode)?

Best,

Rico.

[Rico Bergmann]

FYI: I finally managed to get the new Flink version running in Zeppelin. Besides adding the parameters mentioned below you have to build Zeppelin with profile scala-2.11 and the new Flink version 1.4.2.

Best,

Rico.

Am 09.04.2018 um 14:43 schrieb Rico Bergmann:

The error message is:

org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Communication with JobManager failed: Lost connection to the JobManager.

Am 09.04.2018 um 14:12 schrieb kedar mhaswade:

Hmm. What error do you see on the Zeppelin console when you click the run (flink code) button after making these changes for flink interpreter config (I assume you restart the interpreter)?

Regards,

Kedar

On Mon, Apr 9, 2018 at 12:50 AM, Rico Bergmann <[hidden email]> wrote:

Hi. 

Thanks for your reply. But this also didn’t work for me. 

In the JM log I get an akka Error („dropping message for non-local recipient“). 

My setup: I have Flink running on Kubernetes cluster, version 1.4.2. zeppelin is version 0.8 using the flink interpreter compiled against flink 1.1.3. 

When submitting a job with the CLI tool everything is working fine. The CLI tool is version 1.4.2 ...

Any other suggestions?

Thanks a lot. 

Best,

Rico. 

Am 06.04.2018 um 18:44 schrieb kedar mhaswade <[hidden email]>:

Yes. You need to add the two properties for the job manager (I agree, it is confusing because the properties named "host" and "port" already available, but the names of the useful properties are different):

Could you please try this and let us know if it works for you?

Regards,

Kedar

On Fri, Apr 6, 2018 at 5:51 AM, Dipl.-Inf. Rico Bergmann <[hidden email]> wrote:

Hi!

Has someone successfully integrated Flink 1.4.2 into Zeppelin notebook (using Flink in cluster mode, not local mode)?

Best,

Rico.