Cluster execution -jar files-

[Juan Fumero]

Hi,
  I would like to use the createRemoteEnvironment to run the application
in a cluster and I have some questions. Following the documentation in
[1] It is not clear to me how to use it.

What should be the content of the jar file? All the external libraries
that I use? or need to include the program map/reduce to be distributed
as well? In the last case, why should I redefine all the operations
again in the main source? Shouldn't be included in the jar files?

Many thanks
Juan

[Juan Fumero]

Missing reference:

[1]
https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/cluster_execution.html

On Don, 2015-07-16 at 16:04 +0200, Juan Fumero wrote:

> Hi,
>   I would like to use the createRemoteEnvironment to run the application
> in a cluster and I have some questions. Following the documentation in
> [1] It is not clear to me how to use it.
>
> What should be the content of the jar file? All the external libraries
> that I use? or need to include the program map/reduce to be distributed
> as well? In the last case, why should I redefine all the operations
> again in the main source? Shouldn't be included in the jar files?
>
> Many thanks
> Juan
>

[Matthias J. Sax]

As the JavaDoc explains:

> * @param jarFiles The JAR files with code that needs to be shipped to the cluster. If the program uses
> *                 user-defined functions, user-defined input formats, or any libraries, those must be
> *                 provided in the JAR files.

-> external libraries, yes
-> your program code, no
  -> except your UDFs, those yes

-Matthias


On 07/16/2015 04:06 PM, Juan Fumero wrote:

> Missing reference:
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/cluster_execution.html
>
> On Don, 2015-07-16 at 16:04 +0200, Juan Fumero wrote:
>> Hi,
>>   I would like to use the createRemoteEnvironment to run the application
>> in a cluster and I have some questions. Following the documentation in
>> [1] It is not clear to me how to use it.
>>
>> What should be the content of the jar file? All the external libraries
>> that I use? or need to include the program map/reduce to be distributed
>> as well? In the last case, why should I redefine all the operations
>> again in the main source? Shouldn't be included in the jar files?
>>
>> Many thanks
>> Juan
>>
>
>