Re: jar dependency in the cluster

Posted by Matthias J. Sax-2 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/jar-dependency-in-the-cluster-tp6137p6143.html

Did you double check that your jar does contain the Kafka connector
classes? I would assume that the jar is not assembled correctly.

See her for some help on how to package jars correctly:
https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/cluster_execution.html#linking-with-modules-not-contained-in-the-binary-distribution


One way would be this (even if I think, this would not be required to
solve your problem):
https://stackoverflow.com/questions/31784051/how-to-reference-the-external-jar-in-flink

-Matthias


On 04/16/2016 08:31 PM, Radu Tudoran wrote:

> Hi,
>
>  
>
> Could anyone help me with the following problem:
>
>  
>
> I have a flink cluster of a couple of nodes (i am using the old version
> 0.10).
>
> I am packaging a jar that needs to use kafka connector. When I create
> the jar in eclipse I am adding the flink connector dependency and set to
> be packed with the jar. Nevertheless, when I submitted it to be executed
> on the cluster I get an error that the jar connector is not visible for
> the class loader. Is there a way in which I can set flink to use a
> certain library path where to look for dependencies or maybe when I
> deploy either the flink cluster or submit the job to add extra
> dependencies.
>
>  
>
> Many thanks
>
>  
>
>  
>
> Dr. Radu Tudoran
>
> Research Engineer - Big Data Expert
>
> IT R&D Division
>
>  
>
> cid:[e-mail]060EE0
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>
> European Research Center
>
> Riesstrasse 25, 80992 München
>
>  
>
> E-mail: [e-mail]_
>
> Mobile: [phone]
>
> Telephone: [phone]
>
>  
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> <http://www.huawei.com/>
> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>
> This e-mail and its attachments contain confidential information from
> HUAWEI, which is intended only for the person or entity whose address is
> listed above. Any use of the information contained herein in any way
> (including, but not limited to, total or partial disclosure,
> reproduction, or dissemination) by persons other than the intended
> recipient(s) is prohibited. If you receive this e-mail in error, please
> notify the sender by phone or email immediately and delete it!
>
>  
>