flink-1.2 and unit testing / flinkspector

[Tarandeep Singh]

Hi,

Is someone using flinkspector unit testing framework with flink-1.2?

I added the following dependencies in my pom.xml file:

        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-datastream_2.10</artifactId>
            <version>0.5</version>
        </dependency>
        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-core_2.10</artifactId>
            <version>0.5</version>
        </dependency>

However, I keep getting NullPointer exception. It seems it is using ForkableFlinkMiniCluster which is probably removed in 1.2.

I tried to build flinkspector locally by replacing ForkableFlinkMiniCluster by  org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster

However, I am getting this error-
ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

I am not able to resolve this error.
Has anyone successfully used flinkspector with flink-1.2 (or build it locally)?

Flinkspector seems like a easy to use unit test framework. Is this the recommended framework? If other people are using some other tool, can someone please point me to an example.

Thank you,

Tarandeep

[Ted Yu]

Can you post the patch for flink-specter where the mini cluster is replaced ?

I assume you upgraded the version of Flink in the pom. 

Cheers

On Mar 17, 2017, at 4:26 PM, Tarandeep Singh <[hidden email]> wrote:

Hi,

Is someone using flinkspector unit testing framework with flink-1.2?

I added the following dependencies in my pom.xml file:

        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-datastream_2.10</artifactId>
            <version>0.5</version>
        </dependency>
        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-core_2.10</artifactId>
            <version>0.5</version>
        </dependency>

However, I keep getting NullPointer exception. It seems it is using ForkableFlinkMiniCluster which is probably removed in 1.2.

I tried to build flinkspector locally by replacing ForkableFlinkMiniCluster by  org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster

However, I am getting this error-
ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

I am not able to resolve this error.
Has anyone successfully used flinkspector with flink-1.2 (or build it locally)?

Flinkspector seems like a easy to use unit test framework. Is this the recommended framework? If other people are using some other tool, can someone please point me to an example.

Thank you,

Tarandeep

[Tarandeep Singh]

Hi Ted,

See the attached patch.

I am able to run test examples (e.g. org.flinkspector.datastream.examples.TestMapper) via IntelliJ. But when I try to build via maven-

mvn clean install

I get that dependency issue (ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

Thanks,

Tarandeep

On Fri, Mar 17, 2017 at 4:55 PM, Ted Yu <[hidden email]> wrote:

Can you post the patch for flink-specter where the mini cluster is replaced ?

I assume you upgraded the version of Flink in the pom. 

Cheers

On Mar 17, 2017, at 4:26 PM, Tarandeep Singh <[hidden email]> wrote:

Hi,

Is someone using flinkspector unit testing framework with flink-1.2?

I added the following dependencies in my pom.xml file:

        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-datastream_2.10</artifactId>
            <version>0.5</version>
        </dependency>
        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-core_2.10</artifactId>
            <version>0.5</version>
        </dependency>

However, I keep getting NullPointer exception. It seems it is using ForkableFlinkMiniCluster which is probably removed in 1.2.

I tried to build flinkspector locally by replacing ForkableFlinkMiniCluster by  org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster

However, I am getting this error-
ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

I am not able to resolve this error.
Has anyone successfully used flinkspector with flink-1.2 (or build it locally)?

Flinkspector seems like a easy to use unit test framework. Is this the recommended framework? If other people are using some other tool, can someone please point me to an example.

Thank you,

Tarandeep

[Ted Yu]

Here is where the dependency comes in:

[INFO] org.apache.flink:flink-test-utils_2.10:jar:1.3-SNAPSHOT

[INFO] +- org.apache.hadoop:hadoop-minikdc:jar:2.7.2:compile

[INFO] |  +- org.apache.directory.server:apacheds-core-api:jar:2.0.0-M15:compile

[INFO] |  |  \- org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2:compile

Using the following change, I am able to build flink-spector based on your changes:

http://pastebin.com/Dxerr5KM

Cheers

On Fri, Mar 17, 2017 at 5:19 PM, Tarandeep Singh <[hidden email]> wrote:

Hi Ted,

See the attached patch.

I am able to run test examples (e.g. org.flinkspector.datastream.examples.TestMapper) via IntelliJ. But when I try to build via maven-

mvn clean install

I get that dependency issue (ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

Thanks,

Tarandeep

On Fri, Mar 17, 2017 at 4:55 PM, Ted Yu <[hidden email]> wrote:

Can you post the patch for flink-specter where the mini cluster is replaced ?

I assume you upgraded the version of Flink in the pom. 

Cheers

On Mar 17, 2017, at 4:26 PM, Tarandeep Singh <[hidden email]> wrote:

Hi,

Is someone using flinkspector unit testing framework with flink-1.2?

I added the following dependencies in my pom.xml file:

        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-datastream_2.10</artifactId>
            <version>0.5</version>
        </dependency>
        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-core_2.10</artifactId>
            <version>0.5</version>
        </dependency>

However, I keep getting NullPointer exception. It seems it is using ForkableFlinkMiniCluster which is probably removed in 1.2.

I tried to build flinkspector locally by replacing ForkableFlinkMiniCluster by  org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster

However, I am getting this error-
ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

I am not able to resolve this error.
Has anyone successfully used flinkspector with flink-1.2 (or build it locally)?

Flinkspector seems like a easy to use unit test framework. Is this the recommended framework? If other people are using some other tool, can someone please point me to an example.

Thank you,

Tarandeep

[Tarandeep Singh]

Thank you Ted. It worked!

Best,

Tarandeep

On Fri, Mar 17, 2017 at 5:31 PM, Ted Yu <[hidden email]> wrote:

Here is where the dependency comes in:

[INFO] org.apache.flink:flink-test-utils_2.10:jar:1.3-SNAPSHOT

[INFO] +- org.apache.hadoop:hadoop-minikdc:jar:2.7.2:compile

[INFO] |  +- org.apache.directory.server:apacheds-core-api:jar:2.0.0-M15:compile

[INFO] |  |  \- org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2:compile

Using the following change, I am able to build flink-spector based on your changes:

http://pastebin.com/Dxerr5KM

Cheers

On Fri, Mar 17, 2017 at 5:19 PM, Tarandeep Singh <[hidden email]> wrote:

Hi Ted,

See the attached patch.

I am able to run test examples (e.g. org.flinkspector.datastream.examples.TestMapper) via IntelliJ. But when I try to build via maven-

mvn clean install

I get that dependency issue (ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

Thanks,

Tarandeep

On Fri, Mar 17, 2017 at 4:55 PM, Ted Yu <[hidden email]> wrote:

Can you post the patch for flink-specter where the mini cluster is replaced ?

I assume you upgraded the version of Flink in the pom. 

Cheers

On Mar 17, 2017, at 4:26 PM, Tarandeep Singh <[hidden email]> wrote:

Hi,

Is someone using flinkspector unit testing framework with flink-1.2?

I added the following dependencies in my pom.xml file:

        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-datastream_2.10</artifactId>
            <version>0.5</version>
        </dependency>
        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-core_2.10</artifactId>
            <version>0.5</version>
        </dependency>

However, I keep getting NullPointer exception. It seems it is using ForkableFlinkMiniCluster which is probably removed in 1.2.

I tried to build flinkspector locally by replacing ForkableFlinkMiniCluster by  org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster

However, I am getting this error-
ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

I am not able to resolve this error.
Has anyone successfully used flinkspector with flink-1.2 (or build it locally)?

Flinkspector seems like a easy to use unit test framework. Is this the recommended framework? If other people are using some other tool, can someone please point me to an example.

Thank you,

Tarandeep

[Ted Yu]

You're welcome.

Maybe send out pull request to flink-spector so that other people don't have to go thru this route.

On Fri, Mar 17, 2017 at 7:16 PM, Tarandeep Singh <[hidden email]> wrote:

Thank you Ted. It worked!

Best,

Tarandeep

On Fri, Mar 17, 2017 at 5:31 PM, Ted Yu <[hidden email]> wrote:

Here is where the dependency comes in:

[INFO] org.apache.flink:flink-test-utils_2.10:jar:1.3-SNAPSHOT

[INFO] +- org.apache.hadoop:hadoop-minikdc:jar:2.7.2:compile

[INFO] |  +- org.apache.directory.server:apacheds-core-api:jar:2.0.0-M15:compile

[INFO] |  |  \- org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2:compile

Using the following change, I am able to build flink-spector based on your changes:

http://pastebin.com/Dxerr5KM

Cheers

On Fri, Mar 17, 2017 at 5:19 PM, Tarandeep Singh <[hidden email]> wrote:

Hi Ted,

See the attached patch.

I am able to run test examples (e.g. org.flinkspector.datastream.examples.TestMapper) via IntelliJ. But when I try to build via maven-

mvn clean install

I get that dependency issue (ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

Thanks,

Tarandeep

On Fri, Mar 17, 2017 at 4:55 PM, Ted Yu <[hidden email]> wrote:

Can you post the patch for flink-specter where the mini cluster is replaced ?

I assume you upgraded the version of Flink in the pom. 

Cheers

On Mar 17, 2017, at 4:26 PM, Tarandeep Singh <[hidden email]> wrote:

Hi,

Is someone using flinkspector unit testing framework with flink-1.2?

I added the following dependencies in my pom.xml file:

        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-datastream_2.10</artifactId>
            <version>0.5</version>
        </dependency>
        <dependency>
            <groupId>org.flinkspector</groupId>
            <artifactId>flinkspector-core_2.10</artifactId>
            <version>0.5</version>
        </dependency>

However, I keep getting NullPointer exception. It seems it is using ForkableFlinkMiniCluster which is probably removed in 1.2.

I tried to build flinkspector locally by replacing ForkableFlinkMiniCluster by  org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster

However, I am getting this error-
ERROR] Failed to execute goal on project flinkspector-core_2.11: Could not resolve dependencies for project org.flinkspector:flinkspector-core_2.11:jar:0.6-SNAPSHOT: Could not find artifact org.apache.directory.jdbm:apacheds-jdbm1:bundle:2.0.0-M2 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

I am not able to resolve this error.
Has anyone successfully used flinkspector with flink-1.2 (or build it locally)?

Flinkspector seems like a easy to use unit test framework. Is this the recommended framework? If other people are using some other tool, can someone please point me to an example.

Thank you,

Tarandeep

[Nancy Estrada]

Hi Tarandeep and Ted,

I am in this route now. I am trying to use Flinkspector with Flink 1.2 using your instructions but failing miserably. After applying the changes, when I try to run "mvn clean install", some Tests fail and therefore I am not able to build successfully.

I am wondering if there is a pull request, where I can access to a version of Flinkspector working with Flink 1.2.

Thank you,
Nancy

[Ted Yu]

Nancy:

You can start another thread for the failed unit tests.

You can pass "-DskipTests" to get over the install command.

Cheers

On Thu, Mar 23, 2017 at 11:06 AM, Nancy Estrada <[hidden email]> wrote:

Hi Tarandeep and Ted,

I am in this route now. I am trying to use Flinkspector with Flink 1.2 using
your instructions but failing miserably. After applying the changes, when I
try to run "mvn clean install", some Tests fail and therefore I am not able
to build successfully.

I am wondering if there is a pull request, where I can access to a version
of Flinkspector working with Flink 1.2.

Thank you,
Nancy



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/flink-1-2-and-unit-testing-flinkspector-tp12281p12377.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[Tarandeep Singh]

Hi Nancy,

I also get 1 test failed when I build/run tests on flink-spector:

- should stop if all triggers fire
Run completed in 3 seconds, 944 milliseconds.
Total number of tests run: 19
Suites: completed 5, aborted 0
Tests: succeeded 18, failed 1, canceled 0, ignored 0, pending 0
*** 1 TEST FAILED ***

I didn't spend time on it and proceeded to build the project by ignoring the failed test.

Thanks,

Tarandeep

On Thu, Mar 23, 2017 at 11:16 AM, Ted Yu <[hidden email]> wrote:

Nancy:

You can start another thread for the failed unit tests.

You can pass "-DskipTests" to get over the install command.

Cheers

On Thu, Mar 23, 2017 at 11:06 AM, Nancy Estrada <[hidden email]> wrote:

Hi Tarandeep and Ted,

I am in this route now. I am trying to use Flinkspector with Flink 1.2 using
your instructions but failing miserably. After applying the changes, when I
try to run "mvn clean install", some Tests fail and therefore I am not able
to build successfully.

I am wondering if there is a pull request, where I can access to a version
of Flinkspector working with Flink 1.2.

Thank you,
Nancy



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/flink-1-2-and-unit-testing-flinkspector-tp12281p12377.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.