Hello everyone, I am a newbie.
I am learning the flink-sql-submit project. From @Jark Wu :https://github.com/wuchong/flink-sql-submit My local environment is: 1. flink1.9.0 standalone 2. kafka_2.11-2.2.0 single I configured Flink Connectors and Formats jars to $FLINK_HOME/lib . Reference: https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/table/connect.html#connectors Then I run flink-sql-submit , sh run.sh q1 Throw java.lang.ClassNotFoundException: com.mysql.jdbc.Driver. My question is: I configured mysql-connector-java in the pom.xml file, mvn build jar include com.mysql.jdbc.Driver. Why is this error still reported? I put the jar package in $FLINK_HOME/lib and the problem can be solved. Do you need to put these jars in $FLINK_HOME/lib when the project relies on too many jar packages? If I don't put mysql-connector-java.jar in $FLINK_HOME/lib, how can I solve this problem? Can @Jark Wu give me some advice? Or can someone give me some advice? Thank you. 1. pom.xml <dependency> 2. mvn clean; mvn package $ ll -rth target [±master ●] 3. flink-sql-submit.jar include java.sql.Driver " zip.vim version v28 $FLINK_DIR/bin/flink run -d -p 3 target/flink-sql-submit.jar -w "${PROJECT_DIR}"/src/main/resources/ -f "$1".sql Eerror: 2019-10-30 10:27:35 java.lang.IllegalArgumentException: JDBC driver class not found. At org.apache.flink.api.java.io.jdbc.JDBCUpsertOutputFormat.open(JDBCUpsertOutputFormat.java:112) At org.apache.flink.api.java.io.jdbc.JDBCUpsertSinkFunction.open(JDBCUpsertSinkFunction.java:42) At org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) At org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102) At org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:48) At org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:532) At org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:396) At org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) At org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) At java.lang.Thread.run(Thread.java:748) Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver At java.net.URLClassLoader.findClass(URLClassLoader.java:381) At java.lang.ClassLoader.loadClass(ClassLoader.java:424) At sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) At java.lang.ClassLoader.loadClass(ClassLoader.java:357) At java.lang.Class.forName0(Native Method) At java.lang.Class.forName(Class.java:264) At org.apache.flink.api.java.io.jdbc.AbstractJDBCOutputFormat.establishConnection(AbstractJDBCOutputFormat.java:66) At org.apache.flink.api.java.io.jdbc.JDBCUpsertOutputFormat.open(JDBCUpsertOutputFormat.java:99) ... 9 more -- Best |
You can create a fat jar (also called Uber jar) that includes all dependencies in your application jar. I would avoid to put things in the Flink lib directory as it can make maintenance difficult. Eg deployment is challenging, upgrade of flink, providing it on new nodes etc. Am 30.10.2019 um 04:46 schrieb Alex Wang <[hidden email]>:
|
Hi Franke, From the information provided by Alex: >> mvn build jar include com.mysql.jdbc.Driver. it seems he has packaged a fat jar? Best, Vino Jörn Franke <[hidden email]> 于2019年10月30日周三 下午2:47写道:
|
Hello Yang, Frank Thank you for your reply. Frank I have created a fat jar called flink-sql-submit.jar , the file size is 8.2M .
I am in favor of your point of view, I want to maintain the project's dependencies, rather than put it in the Flink lib directory. I will continue to solve this problem. I will give you feedback if I have an answer. vino yang <[hidden email]> 于2019年10月30日周三 下午4:05写道:
Best |
Free forum by Nabble | Edit this page |