Execution environments for testing: local vs collection vs mini cluster

Posted by Juan Rodríguez Hortalá on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Execution-environments-for-testing-local-vs-collection-vs-mini-cluster-tp28926.html

Hi,

In https://ci.apache.org/projects/flink/flink-docs-stable/dev/local_execution.html and https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/runtime/minicluster/MiniCluster.html I see there are 3 ways to create an execution environment for testing:

• StreamExecutionEnvironment.createLocalEnvironment and ExecutionEnvironment.createLocalEnvironment create an execution environment running on a single JVM using different threads.
  
• CollectionEnvironment runs on a single JVM on a single thread.
  
• I haven't found not much documentation on the Mini Cluster, but it sounds similar to the Hadoop MiniCluster. If that is then case, then it would run on many local JVMs, each of them running multiple threads.
  

Am I correct about the Mini Cluster? Is there any additional documentation about it? I discovered it looking at the source code of AbstractTestBase, that is mentioned on https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/testing.html#integration-testing. Also, it looks like launching the mini cluster registers it somewhere, so subsequent calls to `StreamExecutionEnvironment.getExecutionEnvironment` return an environment that uses the mini cluster. Is that performed by `executionEnvironment.setAsContext()` in https://github.com/apache/flink/blob/master/flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterWithClientResource.java#L56 ? Is that execution environment registration process documented anywhere?

Which test execution environment is recommended for each test use case? For example I don't see why would I use CollectionEnvironment when I have the local environment available and running on several threads, what is a good use case for CollectionEnvironment?

Are all these 3 environments supported equality, or maybe some of them is expected to be deprecated?

Are there any additional execution environments that could be useful for testing on a single host?

Thanks,

Juan