Re: flink run jar throw java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
Posted by
Jörn Franke on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/flink-run-jar-throw-java-lang-ClassNotFoundException-com-mysql-jdbc-Driver-tp30757p30758.html
You can create a fat jar (also called Uber jar) that includes all dependencies in your application jar.
I would avoid to put things in the Flink lib directory as it can make maintenance difficult. Eg deployment is challenging, upgrade of flink, providing it on new nodes etc.
Hello everyone, I am a newbie.
I am learning the flink-sql-submit project. From @Jark Wu :
https://github.com/wuchong/flink-sql-submitMy local environment is:
1. flink1.9.0 standalone
2. kafka_2.11-2.2.0 single
I configured Flink Connectors and Formats jars to $FLINK_HOME/lib .
Reference:
https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/connect.html#connectorsThen I run flink-sql-submit , sh run.sh q1
Throw java.lang.ClassNotFoundException: com.mysql.jdbc.Driver.
My question is: I configured mysql-connector-java in the pom.xml file, mvn build jar include com.mysql.jdbc.Driver.
Why is this error still reported? I put the jar package in $FLINK_HOME/lib and the problem can be solved.
Do you need to put these jars in $FLINK_HOME/lib when the project relies on too many jar packages?
If I don't put mysql-connector-java.jar in $FLINK_HOME/lib, how can I solve this problem?
Can @Jark Wu give me some advice? Or can someone give me some advice? Thank you.
1. pom.xml
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.38</version>
</dependency>
2. mvn clean; mvn package
$ ll -rth target [±master ●]
total 32312
drwxr-xr-x 3 alex staff 96B Oct 30 11:32 generated-sources
drwxr-xr-x 5 alex staff 160B Oct 30 11:32 classes
drwxr-xr-x 3 alex staff 96B Oct 30 11:32 maven-archiver
-rw-r--r-- 1 alex staff 7.2M Oct 30 11:32 flink-sql-submit-1.0-SNAPSHOT.jar
-rw-r--r-- 1 alex staff 8.2M Oct 30 11:32 flink-sql-submit.jar
3. flink-sql-submit.jar include java.sql.Driver
" zip.vim version v28
" Browsing zipfile /Users/alex/IdeaProjects/alex/flink_learn/flink-sql-submit/target/flink-sql-submit.jar
" Select a file with cursor and press ENTER
META-INF/MANIFEST.MF
META-INF/
q1.sql
user_behavior.log
com/
com/github/
com/github/wuchong/
com/github/wuchong/sqlsubmit/
com/github/wuchong/sqlsubmit/SqlSubmit$1.class
com/github/wuchong/sqlsubmit/SqlSubmit.class
com/github/wuchong/sqlsubmit/SourceGenerator.class
com/github/wuchong/sqlsubmit/cli/
com/github/wuchong/sqlsubmit/cli/SqlCommandParser$SqlCommandCall.class
com/github/wuchong/sqlsubmit/cli/SqlCommandParser.class
com/github/wuchong/sqlsubmit/cli/SqlCommandParser$SqlCommand.class
com/github/wuchong/sqlsubmit/cli/CliOptions.class
com/github/wuchong/sqlsubmit/cli/CliOptionsParser.class
META-INF/maven/
META-INF/maven/com.github.wuchong/
META-INF/maven/com.github.wuchong/flink-sql-submit/
META-INF/maven/com.github.wuchong/flink-sql-submit/pom.xml
META-INF/maven/com.github.wuchong/flink-sql-submit/pom.properties
META-INF/services/
META-INF/services/java.sql.Driver
com/mysql/
com/mysql/fabric/
com/mysql/fabric/FabricCommunicationException.class
com/mysql/fabric/FabricConnection.class
com/mysql/fabric/FabricStateResponse.class
com/mysql/fabric/HashShardMapping$ReverseShardIndexSorter.class
com/mysql/fabric/HashShardMapping.class
com/mysql/fabric/RangeShardMapping$RangeShardIndexSorter.class
com/mysql/fabric/RangeShardMapping.class
com/mysql/fabric/Response.class
com/mysql/fabric/Server.class
com/mysql/fabric/ServerGroup.class
com/mysql/fabric/ServerMode.class
com/mysql/fabric/ServerRole.class
etc ...
$FLINK_DIR/bin/flink run -d -p 3 target/flink-sql-submit.jar -w "${PROJECT_DIR}"/src/main/resources/ -f "$1".sql
Eerror:
2019-10-30 10:27:35
java.lang.IllegalArgumentException: JDBC driver class not found.
At org.apache.flink.api.java.io.jdbc.JDBCUpsertOutputFormat.open(JDBCUpsertOutputFormat.java:112)
At org.apache.flink.api.java.io.jdbc.JDBCUpsertSinkFunction.open(JDBCUpsertSinkFunction.java:42)
At org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
At org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
At org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48)
At org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:532)
At org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:396)
At org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
At org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
At java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
At java.net.URLClassLoader.findClass(URLClassLoader.java:381)
At java.lang.ClassLoader.loadClass(ClassLoader.java:424)
At sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
At java.lang.ClassLoader.loadClass(ClassLoader.java:357)
At java.lang.Class.forName0(Native Method)
At java.lang.Class.forName(Class.java:264)
At org.apache.flink.api.java.io.jdbc.AbstractJDBCOutputFormat.establishConnection(AbstractJDBCOutputFormat.java:66)
At org.apache.flink.api.java.io.jdbc.JDBCUpsertOutputFormat.open(JDBCUpsertOutputFormat.java:99)
... 9 more
--
Best