ClasNotFound when submitting job from command line

[Flavio Pompermaier]

Hi to all,

I just tied to dubmit my application to the Flink cluster (1.0.1) but I get ClassNotFound exceptions for classes inside my shaded jar (like oracle.jdbc.OracleDriver or org.apache.commons.pool2.PooledObjectFactory). Those classes are in the shaded jar but aren't found.

If I put the jars in the flink's lib dir (for every node of the cluster) things work.

How can I solve that?

Best,

Flavio

[Radu Tudoran]

Hi,

 

In my case the root cause for this was mainly that I was using eclipse to package the jar. Try using mvn instead. Additioanlly you can copy the dependency jars in the lib of the task managers and restart them

 

Dr. Radu Tudoran

Research Engineer - Big Data Expert

IT R&D Division

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: [phone]

Telephone: [phone]

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!

 

From: Flavio Pompermaier [mailto:[hidden email]]
Sent: Tuesday, April 19, 2016 5:40 PM
To: user
Subject: ClasNotFound when submitting job from command line

 

Hi to all,

I just tied to dubmit my application to the Flink cluster (1.0.1) but I get ClassNotFound exceptions for classes inside my shaded jar (like oracle.jdbc.OracleDriver or org.apache.commons.pool2.PooledObjectFactory). Those classes are in the shaded jar but aren't found.

If I put the jars in the flink's lib dir (for every node of the cluster) things work.

How can I solve that?

 

Best,

Flavio

[Balaji Rajagopalan]

In your pom.xml add the maven.plugins like this, and you will have to add all the dependent artifacts, this works for me, if you fire mvn clean compile package, the created jar is a fat jar. 

<plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-dependency-plugin</artifactId>

                <version>2.9</version>

                <executions>

                    <execution>

                        <id>unpack</id>

                        <!-- executed just before the package phase -->

                        <phase>prepare-package</phase>

                        <goals>

                            <goal>unpack</goal>

                        </goals>

                        <configuration>

                            <artifactItems>

                                <!-- For Flink connector classes -->

                                <artifactItem>

                                    <groupId>org.apache.flink</groupId>

                                    <artifactId>flink-connector-kafka-0.8_${scala.version}</artifactId>

                                    <version>${flink.version}</version>

                                    <type>jar</type>

                                    <overWrite>false</overWrite>

                                    <outputDirectory>${project.build.directory}/classes</outputDirectory>

                                    <includes>org/apache/flink/**</includes>

                                </artifactItem>

       ...

      ...

                        </artifactItems>

                        </configuration>

                    </execution>

                </executions>

            </plugin>

On Tue, Apr 19, 2016 at 9:10 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I just tied to dubmit my application to the Flink cluster (1.0.1) but I get ClassNotFound exceptions for classes inside my shaded jar (like oracle.jdbc.OracleDriver or org.apache.commons.pool2.PooledObjectFactory). Those classes are in the shaded jar but aren't found.

If I put the jars in the flink's lib dir (for every node of the cluster) things work.

How can I solve that?

Best,

Flavio

[Flavio Pompermaier]

I use maven to generate the shaded jar (and the classes are inside it) but when the job starts it can load those classes using Class.forName() (required to instantiate the JDBC connections).

I think it's probably a problem related to class loading of Flink

On Tue, Apr 19, 2016 at 6:02 PM, Balaji Rajagopalan <[hidden email]> wrote:

In your pom.xml add the maven.plugins like this, and you will have to add all the dependent artifacts, this works for me, if you fire mvn clean compile package, the created jar is a fat jar. 

<plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-dependency-plugin</artifactId>

                <version>2.9</version>

                <executions>

                    <execution>

                        <id>unpack</id>

                        <!-- executed just before the package phase -->

                        <phase>prepare-package</phase>

                        <goals>

                            <goal>unpack</goal>

                        </goals>

                        <configuration>

                            <artifactItems>

                                <!-- For Flink connector classes -->

                                <artifactItem>

                                    <groupId>org.apache.flink</groupId>

                                    <artifactId>flink-connector-kafka-0.8_${scala.version}</artifactId>

                                    <version>${flink.version}</version>

                                    <type>jar</type>

                                    <overWrite>false</overWrite>

                                    <outputDirectory>${project.build.directory}/classes</outputDirectory>

                                    <includes>org/apache/flink/**</includes>

                                </artifactItem>

       ...

      ...

                        </artifactItems>

                        </configuration>

                    </execution>

                </executions>

            </plugin>

On Tue, Apr 19, 2016 at 9:10 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I just tied to dubmit my application to the Flink cluster (1.0.1) but I get ClassNotFound exceptions for classes inside my shaded jar (like oracle.jdbc.OracleDriver or org.apache.commons.pool2.PooledObjectFactory). Those classes are in the shaded jar but aren't found.

If I put the jars in the flink's lib dir (for every node of the cluster) things work.

How can I solve that?

Best,

Flavio

[rmetzger0]

Hi Flavio,

in which class are you calling Class.forName()? Is the class where the Class.forName() call is loaded from the user jar or is it a class from the Flink distribution?

I'm asking because Class.forName() is using the classloader of the class where the call is located. So if the class has been loaded from the system class loader, it can not access classes from the user jar.

On Tue, Apr 19, 2016 at 6:13 PM, Flavio Pompermaier <[hidden email]> wrote:

I use maven to generate the shaded jar (and the classes are inside it) but when the job starts it can load those classes using Class.forName() (required to instantiate the JDBC connections).

I think it's probably a problem related to class loading of Flink

On Tue, Apr 19, 2016 at 6:02 PM, Balaji Rajagopalan <[hidden email]> wrote:

In your pom.xml add the maven.plugins like this, and you will have to add all the dependent artifacts, this works for me, if you fire mvn clean compile package, the created jar is a fat jar. 

<plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-dependency-plugin</artifactId>

                <version>2.9</version>

                <executions>

                    <execution>

                        <id>unpack</id>

                        <!-- executed just before the package phase -->

                        <phase>prepare-package</phase>

                        <goals>

                            <goal>unpack</goal>

                        </goals>

                        <configuration>

                            <artifactItems>

                                <!-- For Flink connector classes -->

                                <artifactItem>

                                    <groupId>org.apache.flink</groupId>

                                    <artifactId>flink-connector-kafka-0.8_${scala.version}</artifactId>

                                    <version>${flink.version}</version>

                                    <type>jar</type>

                                    <overWrite>false</overWrite>

                                    <outputDirectory>${project.build.directory}/classes</outputDirectory>

                                    <includes>org/apache/flink/**</includes>

                                </artifactItem>

       ...

      ...

                        </artifactItems>

                        </configuration>

                    </execution>

                </executions>

            </plugin>

On Tue, Apr 19, 2016 at 9:10 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I just tied to dubmit my application to the Flink cluster (1.0.1) but I get ClassNotFound exceptions for classes inside my shaded jar (like oracle.jdbc.OracleDriver or org.apache.commons.pool2.PooledObjectFactory). Those classes are in the shaded jar but aren't found.

If I put the jars in the flink's lib dir (for every node of the cluster) things work.

How can I solve that?

Best,

Flavio

[Flavio Pompermaier]

At the moment my 2 PRs about jdbc batch connector and input format fix are pending so I have my customized implementation of the inputformat in my jar andas jdbc backend I need ijdbc6.jar as dependency (that I shade on the jar).
Thus the class.forName() is called on the inputformat open().

On 20 Apr 2016 09:32, "Robert Metzger" <[hidden email]> wrote:

Hi Flavio,

in which class are you calling Class.forName()? Is the class where the Class.forName() call is loaded from the user jar or is it a class from the Flink distribution?

I'm asking because Class.forName() is using the classloader of the class where the call is located. So if the class has been loaded from the system class loader, it can not access classes from the user jar.

On Tue, Apr 19, 2016 at 6:13 PM, Flavio Pompermaier <[hidden email]> wrote:

I use maven to generate the shaded jar (and the classes are inside it) but when the job starts it can load those classes using Class.forName() (required to instantiate the JDBC connections).

I think it's probably a problem related to class loading of Flink

On Tue, Apr 19, 2016 at 6:02 PM, Balaji Rajagopalan <[hidden email]> wrote:

In your pom.xml add the maven.plugins like this, and you will have to add all the dependent artifacts, this works for me, if you fire mvn clean compile package, the created jar is a fat jar. 

<plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-dependency-plugin</artifactId>

                <version>2.9</version>

                <executions>

                    <execution>

                        <id>unpack</id>

                        <!-- executed just before the package phase -->

                        <phase>prepare-package</phase>

                        <goals>

                            <goal>unpack</goal>

                        </goals>

                        <configuration>

                            <artifactItems>

                                <!-- For Flink connector classes -->

                                <artifactItem>

                                    <groupId>org.apache.flink</groupId>

                                    <artifactId>flink-connector-kafka-0.8_${scala.version}</artifactId>

                                    <version>${flink.version}</version>

                                    <type>jar</type>

                                    <overWrite>false</overWrite>

                                    <outputDirectory>${project.build.directory}/classes</outputDirectory>

                                    <includes>org/apache/flink/**</includes>

                                </artifactItem>

       ...

      ...

                        </artifactItems>

                        </configuration>

                    </execution>

                </executions>

            </plugin>

On Tue, Apr 19, 2016 at 9:10 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi to all,

I just tied to dubmit my application to the Flink cluster (1.0.1) but I get ClassNotFound exceptions for classes inside my shaded jar (like oracle.jdbc.OracleDriver or org.apache.commons.pool2.PooledObjectFactory). Those classes are in the shaded jar but aren't found.

If I put the jars in the flink's lib dir (for every node of the cluster) things work.

How can I solve that?

Best,

Flavio