Table API Compilation Error in Flink

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Table API Compilation Error in Flink

Nagananda M

Hi All,

Am trying to compile a sample program in apache flink using TableEnvironment and facing some issue in compilation for the addition of the below line.

"StreamTableEnvironment tableEnv = StreamTableEnvironment.getTableEnvironment(env);"

The error is as below

"cannot access org.apache.flink.api.scala.ExecutionEnvironment"

Am compiling the code using maven. Can anyone help on this.

Cheers,

Nagananda M


Virus-free. www.avast.com
Reply | Threaded
Open this post in threaded view
|

Re: Table API Compilation Error in Flink

Xingcan Cui
Hi Nagananda,

adding `flink-streaming-scala_${scala version}` to your maven dependency would solve this.

Best,
Xingcan



On 5 Mar 2018, at 2:21 PM, Nagananda M <[hidden email]> wrote:

Hi All,
Am trying to compile a sample program in apache flink using TableEnvironment and facing some issue in compilation for the addition of the below line.
"StreamTableEnvironment tableEnv = StreamTableEnvironment.getTableEnvironment(env);"
The error is as below
"cannot access org.apache.flink.api.scala.ExecutionEnvironment"
Am compiling the code using maven. Can anyone help on this.
Cheers,
Nagananda M

Virus-free. www.avast.com

Reply | Threaded
Open this post in threaded view
|

RE: Table API Compilation Error in Flink

Nagananda M

Hi Xingcan,

 

I have included the below portion in my pom.xml. Still the same error is coming.

 

</dependency>

<dependency>

   <groupId>org.apache.flink</groupId>

   <artifactId>flink-streaming-scala_2.11</artifactId>

   <version>1.4.0</version>

</dependency>

 

Regards,

Nagananda M

 

From: Xingcan Cui [mailto:[hidden email]]
Sent: Monday, March 05, 2018 1:01 PM
To: Nagananda M
Cc: [hidden email]
Subject: Re: Table API Compilation Error in Flink

 

Hi Nagananda,

 

adding `flink-streaming-scala_${scala version}` to your maven dependency would solve this.

 

Best,

Xingcan

 

 



On 5 Mar 2018, at 2:21 PM, Nagananda M <[hidden email]> wrote:

 

Hi All,

Am trying to compile a sample program in apache flink using TableEnvironment and facing some issue in compilation for the addition of the below line.

"StreamTableEnvironment tableEnv = StreamTableEnvironment.getTableEnvironment(env);"

The error is as below

"cannot access org.apache.flink.api.scala.ExecutionEnvironment"

Am compiling the code using maven. Can anyone help on this.

Cheers,

Nagananda M

 

Virus-free. www.avast.com

 

Reply | Threaded
Open this post in threaded view
|

Re: Table API Compilation Error in Flink

Timo Walther
Hi Nagananda,

could you show us your entire pom.xml?

From what I see it seems that you are using the wrong StreamTableEnvironment. First you need to decide if you want to program in Scala or Java. Depending on that you can add the dependencies as descriped in [1].

There are two environments specialized for Java and Scala development (org.apache.flink.table.api.scala.StreamTableEnvironment OR org.apache.flink.table.api.java.StreamTableEnvironment).

Regards,
Timo

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/table/#setup

Am 3/5/18 um 8:39 AM schrieb Nagananda M:

Hi Xingcan,

 

I have included the below portion in my pom.xml. Still the same error is coming.

 

</dependency>

<dependency>

   <groupId>org.apache.flink</groupId>

   <artifactId>flink-streaming-scala_2.11</artifactId>

   <version>1.4.0</version>

</dependency>

 

Regards,

Nagananda M

 

From: Xingcan Cui [[hidden email]]
Sent: Monday, March 05, 2018 1:01 PM
To: Nagananda M
Cc: [hidden email]
Subject: Re: Table API Compilation Error in Flink

 

Hi Nagananda,

 

adding `flink-streaming-scala_${scala version}` to your maven dependency would solve this.

 

Best,

Xingcan

 

 



On 5 Mar 2018, at 2:21 PM, Nagananda M <[hidden email]> wrote:

 

Hi All,

Am trying to compile a sample program in apache flink using TableEnvironment and facing some issue in compilation for the addition of the below line.

"StreamTableEnvironment tableEnv = StreamTableEnvironment.getTableEnvironment(env);"

The error is as below

"cannot access org.apache.flink.api.scala.ExecutionEnvironment"

Am compiling the code using maven. Can anyone help on this.

Cheers,

Nagananda M

 

Virus-free. www.avast.com

 


Reply | Threaded
Open this post in threaded view
|

RE: Table API Compilation Error in Flink

Nagananda M

Hi Timo Walther,

 

Please find the entire pom.xml below.

 

 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

        <modelVersion>4.0.0</modelVersion>

 

        <groupId>flink-test</groupId>

        <artifactId>flink-test</artifactId>

        <version>0.1</version>

        <packaging>jar</packaging>

 

        <name>Flink Quickstart Job</name>

        <url>http://www.myorganization.org</url>

 

        <properties>

                <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

                <flink.version>1.4.0</flink.version>

                <slf4j.version>1.7.7</slf4j.version>

                <log4j.version>1.2.17</log4j.version>

                <scala.binary.version>2.11</scala.binary.version>

        </properties>

 

        <repositories>

                <repository>

                        <id>apache.snapshots</id>

                        <name>Apache Development Snapshot Repository</name>

                        <url>https://repository.apache.org/content/repositories/snapshots/</url>

                        <releases>

                                <enabled>false</enabled>

                        </releases>

                        <snapshots>

                                <enabled>true</enabled>

                        </snapshots>

                </repository>

        </repositories>

 

        <!--

 

                Execute "mvn clean package -Pbuild-jar"

                to build a jar file out of this project!

 

                How to use the Flink Quickstart pom:

 

                a) Adding new dependencies:

                        You can add dependencies to the list below.

 

                b) Build a jar for running on the cluster:

 

                        "mvn clean package -Pbuild-jar"

                        This will create a fat-jar which contains all dependencies necessary for running the created jar in a cluster.

        -->

 

        <dependencies>

                <!-- Apache Flink dependencies -->

                <dependency>

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-core</artifactId>

                        <version>${flink.version}</version>

                </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-java -->

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-java</artifactId>

         <version>1.4.1</version>

      </dependency>

                <dependency>

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-java</artifactId>

                        <version>${flink.version}</version>

                </dependency>

                <dependency>

                        <!-- This dependency is required to actually execute jobs. It is currently pulled in by

                                flink-streaming-java, but we explicitly depend on it to safeguard against future changes. -->

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-clients_${scala.binary.version}</artifactId>

                        <version>${flink.version}</version>

                </dependency>

                <dependency>

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>

                        <version>${flink.version}</version>

                </dependency>

                <!-- explicitly add a standard logging framework, as Flink does not have

                        a hard dependency on one specific framework by default -->

                <dependency>

                        <groupId>org.slf4j</groupId>

                        <artifactId>slf4j-log4j12</artifactId>

                        <version>${slf4j.version}</version>

                </dependency>

                <dependency>

                        <groupId>log4j</groupId>

                        <artifactId>log4j</artifactId>

                        <version>${log4j.version}</version>

                </dependency>

                <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka -->

                <dependency>

                        <groupId>org.apache.kafka</groupId>

                        <artifactId>kafka_2.12</artifactId>

                        <version>1.0.0</version>

                </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients -->

                <dependency>

                    <groupId>org.apache.kafka</groupId>

                    <artifactId>kafka-clients</artifactId>

                    <version>1.0.0</version>

                </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-streams -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>kafka-streams</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-api -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-api</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-json -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-json</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-runtime -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-runtime</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-transforms -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-transforms</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-log4j-appender -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>kafka-log4j-appender</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-tools -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>kafka-tools</artifactId>

         <version>1.0.0</version>

      </dependency>

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-cep_2.11</artifactId>

         <version>1.4.1</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-table -->

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-table_2.11</artifactId>

         <version>1.4.0</version>

      <scope>provided</scope>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka-0.10 -->

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-connector-kafka-0.10_2.11</artifactId>

         <version>1.4.0</version>

      </dependency>

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-connector-kafka-0.11_${scala.binary.version}</artifactId>

         <version>${flink.version}</version>

      </dependency>

        </dependencies>

 

        <profiles>

                <profile>

                        <!-- Profile for packaging correct JAR files -->

                        <id>build-jar</id>

                        <activation>

                                <activeByDefault>false</activeByDefault>

                        </activation>

 

                        <dependencies>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-core</artifactId>

                                        <version>${flink.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-java</artifactId>

                                        <version>${flink.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-clients_${scala.binary.version}</artifactId>

                                        <version>${flink.version}</version>

                                       <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>

                                        <version>${flink.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.slf4j</groupId>

                                        <artifactId>slf4j-log4j12</artifactId>

                                        <version>${slf4j.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>log4j</groupId>

                                        <artifactId>log4j</artifactId>

                                        <version>${log4j.version}</version>

                                        <scope>provided</scope>

                                </dependency>

            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-scala -->

            <!--

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-scala_${scala.binary.version}</artifactId>

               <version>${flink.version}</version>

            </dependency>-->

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-table_2.11</artifactId>

               <version>1.4.0</version>

            </dependency>

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-scala_2.11</artifactId>

               <version>1.4.0</version>

            </dependency>

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-streaming-scala_2.11</artifactId>

               <version>1.4.0</version>

            </dependency>

            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-core

           <dependency>

                <groupId>org.apache.flink</groupId>

                <artifactId>flink-core</artifactId>

                <version>${flink.version}</version>

            </dependency> -->

            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-scala -->

            <!--

            <dependency>

            <groupId>org.apache.flink</groupId>

            <artifactId>flink-streaming-scala_${scala.binary.version}</artifactId>

                <version>${flink.version}</version>

            </dependency>

            -->

                        </dependencies>

 

                        <build>

                                <plugins>

                                        <!-- We use the maven-shade plugin to create a fat jar that contains all dependencies

                        except flink and its transitive dependencies. The resulting fat-jar can be executed

                        on a cluster. Change the value of Program-Class if your program entry point changes. -->

                                        <plugin>

                                                <groupId>org.apache.maven.plugins</groupId>

                                                <artifactId>maven-shade-plugin</artifactId>

                                                <version>2.4.1</version>

                                                <executions>

                                                        <!-- Run shade goal on package phase -->

                                                        <execution>

                                                                <phase>package</phase>

                                                                <goals>

                                                                        <goal>shade</goal>

                                                                </goals>

                                                                <configuration>

                                                                        <artifactSet>

                                                                                <excludes>

                                                                                        <exclude>org.apache.flink:force-shading</exclude>

                                                                                        <exclude>com.google.code.findbgs:jsr305</exclude>

                                                                                        <exclude>org.slf4j:slf4j-api</exclude>

                                                                                </excludes>

                                                                        </artifactSet>

                                                                        <filters>

                                                                                <filter>

                                                                                        <!-- Do not copy the signatures in the META-INF folder.

                                            Otherwise, this might cause SecurityExceptions when using the JAR. -->

                                                                                        <artifact>*:*</artifact>

                                                                                        <excludes>

                                                                                                <exclude>META-INF/*.SF</exclude>

                                                                                                <exclude>META-INF/*.DSA</exclude>

                                                                                                <exclude>META-INF/*.RSA</exclude>

                                                                                        </excludes>

                                                                                </filter>

                                                                        </filters>

                                                                        <!-- If you want to use ./bin/flink run <quickstart jar> uncomment the following lines.

                                                This will add a Main-Class entry to the manifest file -->

                                                                        <!--

                                        <transformers>

                                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">

                                                <mainClass>flinktest.StreamingJob</mainClass>

                                                </transformer>

                                        </transformers>

                                        -->

                                                                </configuration>

                                                        </execution>

                                                </executions>

                                        </plugin>

                                </plugins>

                        </build>

                </profile>

        </profiles>

 

        <build>

                <plugins>

                        <plugin>

                                <groupId>org.apache.maven.plugins</groupId>

                                <artifactId>maven-compiler-plugin</artifactId>

                                <version>3.1</version>

                                <configuration>

                                        <source>1.8</source>

                                        <target>1.8</target>

                                </configuration>

                        </plugin>

       <!--

         <plugin>

            <artifactId>maven-assembly-plugin</artifactId>

            <configuration>

               <source>1.8</source>

               <target>1.8</target>

               <archive>

                  <manifest>

                     <mainClass>flinktest.KafkaStreamingJob</mainClass>

                  </manifest>

               </archive>

                 <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef>

                  </descriptorRefs>

            </configuration>

            <executions>

               <execution>

                  <id>make-assembly</id>

                  <phase>package</phase>

                  <goals>

                     <goal>single</goal>

                  </goals>

                  <configuration>

                  <outputDirectory>${basedir}/build</outputDirectory>

                  </configuration>

               </execution>

            </executions>

         </plugin>

      -->

                </plugins>

                <!-- If you want to use Java 8 Lambda Expressions uncomment the following lines -->

 

         <!--

                <pluginManagement>

                        <plugins>

                                <plugin>

                                        <artifactId>maven-compiler-plugin</artifactId>

                                        <configuration>

                                                <source>1.8</source>

                                                <target>1.8</target>

                                                <compilerId>jdt</compilerId>

                                        </configuration>

                                        <dependencies>

                                                <dependency>

                                                        <groupId>org.eclipse.tycho</groupId>

                                                        <artifactId>tycho-compiler-jdt</artifactId>

                                                        <version>0.21.0</version>

                                                </dependency>

                                        </dependencies>

                                </plugin>

                                <plugin>

                                        <groupId>org.eclipse.m2e</groupId>

                                        <artifactId>lifecycle-mapping</artifactId>

                                        <version>1.0.0</version>

                                        <configuration>

                                                <lifecycleMappingMetadata>

                                                        <pluginExecutions>

                                                                <pluginExecution>

                                                                        <pluginExecutionFilter>

                                                                                <groupId>org.apache.maven.plugins</groupId>

                                                                                <artifactId>maven-assembly-plugin</artifactId>

                                                                                <versionRange>[2.4,)</versionRange>

                                                                                <goals>

                                                                                        <goal>single</goal>

                                                                                </goals>

                                                                        </pluginExecutionFilter>

                                                                        <action>

                                                                                <ignore/>

                                                                        </action>

                                                                </pluginExecution>

                                                                <pluginExecution>

                                                                        <pluginExecutionFilter>

                                                                                <groupId>org.apache.maven.plugins</groupId>

                                                                                <artifactId>maven-compiler-plugin</artifactId>

                                                                                <versionRange>[3.1,)</versionRange>

                                                                                <goals>

                                                                                        <goal>testCompile</goal>

                                                                                        <goal>compile</goal>

                                                                                </goals>

                                                                        </pluginExecutionFilter>

                                                                        <action>

                                                                                <ignore/>

                                                                        </action>

                                                                </pluginExecution>

                                                        </pluginExecutions>

                                                </lifecycleMappingMetadata>

                                        </configuration>

                                </plugin>

                        </plugins>

                </pluginManagement> -->

        </build>

</project>

 

 

Regards,

Nagananda M

 

From: Timo Walther [mailto:[hidden email]]
Sent: Monday, March 05, 2018 6:56 PM
To: [hidden email]
Subject: Re: Table API Compilation Error in Flink

 

Hi Nagananda,

could you show us your entire pom.xml?

From what I see it seems that you are using the wrong StreamTableEnvironment. First you need to decide if you want to program in Scala or Java. Depending on that you can add the dependencies as descriped in [1].

There are two environments specialized for Java and Scala development (org.apache.flink.table.api.scala.StreamTableEnvironment OR org.apache.flink.table.api.java.StreamTableEnvironment).

Regards,
Timo

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/table/#setup

Am 3/5/18 um 8:39 AM schrieb Nagananda M:

Hi Xingcan,

 

I have included the below portion in my pom.xml. Still the same error is coming.

 

</dependency>

<dependency>

   <groupId>org.apache.flink</groupId>

   <artifactId>flink-streaming-scala_2.11</artifactId>

   <version>1.4.0</version>

</dependency>

 

Regards,

Nagananda M

 

From: Xingcan Cui [[hidden email]]
Sent: Monday, March 05, 2018 1:01 PM
To: Nagananda M
Cc: [hidden email]
Subject: Re: Table API Compilation Error in Flink

 

Hi Nagananda,

 

adding `flink-streaming-scala_${scala version}` to your maven dependency would solve this.

 

Best,

Xingcan

 

 




On 5 Mar 2018, at 2:21 PM, Nagananda M <[hidden email]> wrote:

 

Hi All,

Am trying to compile a sample program in apache flink using TableEnvironment and facing some issue in compilation for the addition of the below line.

"StreamTableEnvironment tableEnv = StreamTableEnvironment.getTableEnvironment(env);"

The error is as below

"cannot access org.apache.flink.api.scala.ExecutionEnvironment"

Am compiling the code using maven. Can anyone help on this.

Cheers,

Nagananda M

 

Virus-free. www.avast.com

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Table API Compilation Error in Flink

Timo Walther
Hi Nagananda,

did my advice in the last mail help you? In general I would recommend to clean up your pom.xml. There are so many unnessary dependencies. Actually all you needs to do is.

- Create a project from here [1].
- If you want to use Kafka add only [2]
- And for table these two dependencies from [3].

Regards,
Timo

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/quickstart/java_api_quickstart.html
[2] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/kafka.html
[3] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/table/

Am 3/7/18 um 5:34 AM schrieb Nagananda M:

Hi Timo Walther,

 

Please find the entire pom.xml below.

 

 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

        <modelVersion>4.0.0</modelVersion>

 

        <groupId>flink-test</groupId>

        <artifactId>flink-test</artifactId>

        <version>0.1</version>

        <packaging>jar</packaging>

 

        <name>Flink Quickstart Job</name>

        <url>http://www.myorganization.org</url>

 

        <properties>

                <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

                <flink.version>1.4.0</flink.version>

                <slf4j.version>1.7.7</slf4j.version>

                <log4j.version>1.2.17</log4j.version>

                <scala.binary.version>2.11</scala.binary.version>

        </properties>

 

        <repositories>

                <repository>

                        <id>apache.snapshots</id>

                        <name>Apache Development Snapshot Repository</name>

                        <url>https://repository.apache.org/content/repositories/snapshots/</url>

                        <releases>

                                <enabled>false</enabled>

                        </releases>

                        <snapshots>

                                <enabled>true</enabled>

                        </snapshots>

                </repository>

        </repositories>

 

        <!--

 

                Execute "mvn clean package -Pbuild-jar"

                to build a jar file out of this project!

 

                How to use the Flink Quickstart pom:

 

                a) Adding new dependencies:

                        You can add dependencies to the list below.

 

                b) Build a jar for running on the cluster:

 

                        "mvn clean package -Pbuild-jar"

                        This will create a fat-jar which contains all dependencies necessary for running the created jar in a cluster.

        -->

 

        <dependencies>

                <!-- Apache Flink dependencies -->

                <dependency>

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-core</artifactId>

                        <version>${flink.version}</version>

                </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-java -->

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-java</artifactId>

         <version>1.4.1</version>

      </dependency>

                <dependency>

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-java</artifactId>

                        <version>${flink.version}</version>

                </dependency>

                <dependency>

                        <!-- This dependency is required to actually execute jobs. It is currently pulled in by

                                flink-streaming-java, but we explicitly depend on it to safeguard against future changes. -->

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-clients_${scala.binary.version}</artifactId>

                        <version>${flink.version}</version>

                </dependency>

                <dependency>

                        <groupId>org.apache.flink</groupId>

                        <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>

                        <version>${flink.version}</version>

                </dependency>

                <!-- explicitly add a standard logging framework, as Flink does not have

                        a hard dependency on one specific framework by default -->

                <dependency>

                        <groupId>org.slf4j</groupId>

                        <artifactId>slf4j-log4j12</artifactId>

                        <version>${slf4j.version}</version>

                </dependency>

                <dependency>

                        <groupId>log4j</groupId>

                        <artifactId>log4j</artifactId>

                        <version>${log4j.version}</version>

                </dependency>

                <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka -->

                <dependency>

                        <groupId>org.apache.kafka</groupId>

                        <artifactId>kafka_2.12</artifactId>

                        <version>1.0.0</version>

                </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients -->

                <dependency>

                    <groupId>org.apache.kafka</groupId>

                    <artifactId>kafka-clients</artifactId>

                    <version>1.0.0</version>

                </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-streams -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>kafka-streams</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-api -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-api</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-json -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-json</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-runtime -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-runtime</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/connect-transforms -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>connect-transforms</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-log4j-appender -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>kafka-log4j-appender</artifactId>

         <version>1.0.0</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-tools -->

      <dependency>

         <groupId>org.apache.kafka</groupId>

         <artifactId>kafka-tools</artifactId>

         <version>1.0.0</version>

      </dependency>

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-cep_2.11</artifactId>

         <version>1.4.1</version>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-table -->

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-table_2.11</artifactId>

         <version>1.4.0</version>

      <scope>provided</scope>

      </dependency>

      <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka-0.10 -->

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-connector-kafka-0.10_2.11</artifactId>

         <version>1.4.0</version>

      </dependency>

      <dependency>

         <groupId>org.apache.flink</groupId>

         <artifactId>flink-connector-kafka-0.11_${scala.binary.version}</artifactId>

         <version>${flink.version}</version>

      </dependency>

        </dependencies>

 

        <profiles>

                <profile>

                        <!-- Profile for packaging correct JAR files -->

                        <id>build-jar</id>

                        <activation>

                                <activeByDefault>false</activeByDefault>

                        </activation>

 

                        <dependencies>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-core</artifactId>

                                        <version>${flink.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-java</artifactId>

                                        <version>${flink.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-clients_${scala.binary.version}</artifactId>

                                        <version>${flink.version}</version>

                                       <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.apache.flink</groupId>

                                        <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>

                                        <version>${flink.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>org.slf4j</groupId>

                                        <artifactId>slf4j-log4j12</artifactId>

                                        <version>${slf4j.version}</version>

                                        <scope>provided</scope>

                                </dependency>

                                <dependency>

                                        <groupId>log4j</groupId>

                                        <artifactId>log4j</artifactId>

                                        <version>${log4j.version}</version>

                                        <scope>provided</scope>

                                </dependency>

            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-scala -->

            <!--

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-scala_${scala.binary.version}</artifactId>

               <version>${flink.version}</version>

            </dependency>-->

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-table_2.11</artifactId>

               <version>1.4.0</version>

            </dependency>

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-scala_2.11</artifactId>

               <version>1.4.0</version>

            </dependency>

            <dependency>

               <groupId>org.apache.flink</groupId>

               <artifactId>flink-streaming-scala_2.11</artifactId>

               <version>1.4.0</version>

            </dependency>

            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-core

           <dependency>

                <groupId>org.apache.flink</groupId>

                <artifactId>flink-core</artifactId>

                <version>${flink.version}</version>

            </dependency> -->

            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-scala -->

            <!--

            <dependency>

            <groupId>org.apache.flink</groupId>

            <artifactId>flink-streaming-scala_${scala.binary.version}</artifactId>

                <version>${flink.version}</version>

            </dependency>

            -->

                        </dependencies>

 

                        <build>

                                <plugins>

                                        <!-- We use the maven-shade plugin to create a fat jar that contains all dependencies

                        except flink and its transitive dependencies. The resulting fat-jar can be executed

                        on a cluster. Change the value of Program-Class if your program entry point changes. -->

                                        <plugin>

                                                <groupId>org.apache.maven.plugins</groupId>

                                                <artifactId>maven-shade-plugin</artifactId>

                                                <version>2.4.1</version>

                                                <executions>

                                                        <!-- Run shade goal on package phase -->

                                                        <execution>

                                                                <phase>package</phase>

                                                                <goals>

                                                                        <goal>shade</goal>

                                                                </goals>

                                                                <configuration>

                                                                        <artifactSet>

                                                                                <excludes>

                                                                                        <exclude>org.apache.flink:force-shading</exclude>

                                                                                        <exclude>com.google.code.findbgs:jsr305</exclude>

                                                                                        <exclude>org.slf4j:slf4j-api</exclude>

                                                                                </excludes>

                                                                        </artifactSet>

                                                                        <filters>

                                                                                <filter>

                                                                                        <!-- Do not copy the signatures in the META-INF folder.

                                            Otherwise, this might cause SecurityExceptions when using the JAR. -->

                                                                                        <artifact>*:*</artifact>

                                                                                        <excludes>

                                                                                                <exclude>META-INF/*.SF</exclude>

                                                                                                <exclude>META-INF/*.DSA</exclude>

                                                                                                <exclude>META-INF/*.RSA</exclude>

                                                                                        </excludes>

                                                                                </filter>

                                                                        </filters>

                                                                        <!-- If you want to use ./bin/flink run <quickstart jar> uncomment the following lines.

                                                This will add a Main-Class entry to the manifest file -->

                                                                        <!--

                                        <transformers>

                                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">

                                                <mainClass>flinktest.StreamingJob</mainClass>

                                                </transformer>

                                        </transformers>

                                        -->

                                                                </configuration>

                                                        </execution>

                                                </executions>

                                        </plugin>

                                </plugins>

                        </build>

                </profile>

        </profiles>

 

        <build>

                <plugins>

                        <plugin>

                                <groupId>org.apache.maven.plugins</groupId>

                                <artifactId>maven-compiler-plugin</artifactId>

                                <version>3.1</version>

                                <configuration>

                                        <source>1.8</source>

                                        <target>1.8</target>

                                </configuration>

                        </plugin>

       <!--

         <plugin>

            <artifactId>maven-assembly-plugin</artifactId>

            <configuration>

               <source>1.8</source>

               <target>1.8</target>

               <archive>

                  <manifest>

                     <mainClass>flinktest.KafkaStreamingJob</mainClass>

                  </manifest>

               </archive>

                 <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef>

                  </descriptorRefs>

            </configuration>

            <executions>

               <execution>

                  <id>make-assembly</id>

                  <phase>package</phase>

                  <goals>

                     <goal>single</goal>

                  </goals>

                  <configuration>

                  <outputDirectory>${basedir}/build</outputDirectory>

                  </configuration>

               </execution>

            </executions>

         </plugin>

      -->

                </plugins>

                <!-- If you want to use Java 8 Lambda Expressions uncomment the following lines -->

 

         <!--

                <pluginManagement>

                        <plugins>

                                <plugin>

                                        <artifactId>maven-compiler-plugin</artifactId>

                                        <configuration>

                                                <source>1.8</source>

                                                <target>1.8</target>

                                                <compilerId>jdt</compilerId>

                                        </configuration>

                                        <dependencies>

                                                <dependency>

                                                        <groupId>org.eclipse.tycho</groupId>

                                                        <artifactId>tycho-compiler-jdt</artifactId>

                                                        <version>0.21.0</version>

                                                </dependency>

                                        </dependencies>

                                </plugin>

                                <plugin>

                                        <groupId>org.eclipse.m2e</groupId>

                                        <artifactId>lifecycle-mapping</artifactId>

                                        <version>1.0.0</version>

                                        <configuration>

                                                <lifecycleMappingMetadata>

                                                        <pluginExecutions>

                                                                <pluginExecution>

                                                                        <pluginExecutionFilter>

                                                                                <groupId>org.apache.maven.plugins</groupId>

                                                                                <artifactId>maven-assembly-plugin</artifactId>

                                                                                <versionRange>[2.4,)</versionRange>

                                                                                <goals>

                                                                                        <goal>single</goal>

                                                                                </goals>

                                                                        </pluginExecutionFilter>

                                                                        <action>

                                                                                <ignore/>

                                                                        </action>

                                                                </pluginExecution>

                                                                <pluginExecution>

                                                                        <pluginExecutionFilter>

                                                                                <groupId>org.apache.maven.plugins</groupId>

                                                                                <artifactId>maven-compiler-plugin</artifactId>

                                                                                <versionRange>[3.1,)</versionRange>

                                                                                <goals>

                                                                                        <goal>testCompile</goal>

                                                                                        <goal>compile</goal>

                                                                                </goals>

                                                                        </pluginExecutionFilter>

                                                                        <action>

                                                                                <ignore/>

                                                                        </action>

                                                                </pluginExecution>

                                                        </pluginExecutions>

                                                </lifecycleMappingMetadata>

                                        </configuration>

                                </plugin>

                        </plugins>

                </pluginManagement> -->

        </build>

</project>

 

 

Regards,

Nagananda M

 

From: Timo Walther [[hidden email]]
Sent: Monday, March 05, 2018 6:56 PM
To: [hidden email]
Subject: Re: Table API Compilation Error in Flink

 

Hi Nagananda,

could you show us your entire pom.xml?

From what I see it seems that you are using the wrong StreamTableEnvironment. First you need to decide if you want to program in Scala or Java. Depending on that you can add the dependencies as descriped in [1].

There are two environments specialized for Java and Scala development (org.apache.flink.table.api.scala.StreamTableEnvironment OR org.apache.flink.table.api.java.StreamTableEnvironment).

Regards,
Timo

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/table/#setup

Am 3/5/18 um 8:39 AM schrieb Nagananda M:

Hi Xingcan,

 

I have included the below portion in my pom.xml. Still the same error is coming.

 

</dependency>

<dependency>

   <groupId>org.apache.flink</groupId>

   <artifactId>flink-streaming-scala_2.11</artifactId>

   <version>1.4.0</version>

</dependency>

 

Regards,

Nagananda M

 

From: Xingcan Cui [[hidden email]]
Sent: Monday, March 05, 2018 1:01 PM
To: Nagananda M
Cc: [hidden email]
Subject: Re: Table API Compilation Error in Flink

 

Hi Nagananda,

 

adding `flink-streaming-scala_${scala version}` to your maven dependency would solve this.

 

Best,

Xingcan

 

 




On 5 Mar 2018, at 2:21 PM, Nagananda M <[hidden email]> wrote:

 

Hi All,

Am trying to compile a sample program in apache flink using TableEnvironment and facing some issue in compilation for the addition of the below line.

"StreamTableEnvironment tableEnv = StreamTableEnvironment.getTableEnvironment(env);"

The error is as below

"cannot access org.apache.flink.api.scala.ExecutionEnvironment"

Am compiling the code using maven. Can anyone help on this.

Cheers,

Nagananda M

 

Virus-free. www.avast.com