/home/user/udfs.jar same question

[Deng Jie]

Dear Flink org,i have same question,like:

public static void main(String[] args) throws Exception {
    ExecutionEnvironment env = ExecutionEnvironment
        .createRemoteEnvironment("flink-master", 6123, "/home/user/udfs.jar");

    DataSet<String> data = env.readTextFile("hdfs://path/to/file");

    data
        .filter(new FilterFunction<String>() {
            public boolean filter(String value) {
                return value.startsWith("http://");
            }
        })
        .writeAsText("hdfs://path/to/result");

    env.execute();
}

How to write the file(udfs.jar),can you give me an example?In addition,can this parameter as an option?

-- Best wishes for you

[Chiwan Park-2]

Hi Deng,

The jarFiles parameter of `createRemoteEnvironment` means that the path of your custom library jar. If you don’t need custom library, you can omit the parameter.

Regards,
Chiwan Park

> On Sep 25, 2015, at 10:48 AM, Deng Jie <[hidden email]> wrote:
>
> Dear Flink org,i have same question,like:
> public static void main(String[] args) throws Exception {
>
>     ExecutionEnvironment env
> =
>  ExecutionEnvironment
>        
> .createRemoteEnvironment("flink-master", 6123, "/home/user/udfs.jar");
>
>
>     DataSet
> <String> data = env.readTextFile("hdfs://path/to/file");
>
>
>     data
>        
> .filter(new FilterFunction<String>() {
>
>            
> public boolean filter(String value) {
>
>                
> return value.startsWith("http://");
>
>            
> }
>
>        
> })
>
>        
> .writeAsText("hdfs://path/to/result");
>
>
>     env
> .execute();
> }
>
> How to write the file(udfs.jar),can you give me an example?In addition,can this parameter as an option?
>
> -- Best wishes for you

[Deng Jie]

Dear Chiwan,

However, 'createRemoteEnvironment' method must specify this parameter.

2015-09-25 13:42 GMT+08:00 Chiwan Park <[hidden email]>:

Hi Deng,

The jarFiles parameter of `createRemoteEnvironment` means that the path of your custom library jar. If you don’t need custom library, you can omit the parameter.

Regards,
Chiwan Park

> On Sep 25, 2015, at 10:48 AM, Deng Jie <[hidden email]> wrote:
>
> Dear Flink org,i have same question,like:
> public static void main(String[] args) throws Exception {
>
>     ExecutionEnvironment env
> =
>  ExecutionEnvironment
>
> .createRemoteEnvironment("flink-master", 6123, "/home/user/udfs.jar");
>
>
>     DataSet
> <String> data = env.readTextFile("hdfs://path/to/file");
>
>
>     data
>
> .filter(new FilterFunction<String>() {
>
>
> public boolean filter(String value) {
>
>
> return value.startsWith("http://");
>
>
> }
>
>
> })
>
>
> .writeAsText("hdfs://path/to/result");
>
>
>     env
> .execute();
> }
>
> How to write the file(udfs.jar),can you give me an example?In addition,can this parameter as an option?
>
> -- Best wishes for you

[Chiwan Park-2]

Hi Deng,

It sounds weird. In code [1], `jarFiles` parameter is defined as a varargs parameter. From this, we can omit the parameter.

Which version of Flink are you using?

Regards,
Chiwan Park

[1] https://github.com/apache/flink/blob/master/flink-java/src/main/java/org/apache/flink/api/java/ExecutionEnvironment.java#L1148

> On Sep 25, 2015, at 5:15 PM, Deng Jie <[hidden email]> wrote:
>
> Dear Chiwan,
> However, 'createRemoteEnvironment' method must specify this parameter.
>
> -- Best wishes for you
> 邓杰
> 15994846535
>
> 2015-09-25 13:42 GMT+08:00 Chiwan Park <[hidden email]>:
> Hi Deng,
>
> The jarFiles parameter of `createRemoteEnvironment` means that the path of your custom library jar. If you don’t need custom library, you can omit the parameter.
>
> Regards,
> Chiwan Park
>
> > On Sep 25, 2015, at 10:48 AM, Deng Jie <[hidden email]> wrote:
> >
> > Dear Flink org,i have same question,like:
> > public static void main(String[] args) throws Exception {
> >
> >     ExecutionEnvironment env
> > =
> >  ExecutionEnvironment
> >
> > .createRemoteEnvironment("flink-master", 6123, "/home/user/udfs.jar");
> >
> >
> >     DataSet
> > <String> data = env.readTextFile("hdfs://path/to/file");
> >
> >
> >     data
> >
> > .filter(new FilterFunction<String>() {
> >
> >
> > public boolean filter(String value) {
> >
> >
> > return value.startsWith("http://");
> >
> >
> > }
> >
> >
> > })
> >
> >
> > .writeAsText("hdfs://path/to/result");
> >
> >
> >     env
> > .execute();
> > }
> >
> > How to write the file(udfs.jar),can you give me an example?In addition,can this parameter as an option?
> >
> > -- Best wishes for you
>
>
>
>

[Fabian Hueske-2]

Hi Deng Jie,

your Flink program needs to be packaged into a JAR file.
The Flink quickstart Maven archetypes for Java [1] and Scala [2] help to setup a Maven project that packages your program correctly into a JAR file.

Best, Fabian

[1] https://ci.apache.org/projects/flink/flink-docs-release-0.9/quickstart/java_api_quickstart.html
[2] https://ci.apache.org/projects/flink/flink-docs-release-0.9/quickstart/scala_api_quickstart.html

2015-09-25 10:15 GMT+02:00 Deng Jie <[hidden email]>:

Dear Chiwan,

However, 'createRemoteEnvironment' method must specify this parameter.

-- Best wishes for you

邓杰

15994846535

2015-09-25 13:42 GMT+08:00 Chiwan Park <[hidden email]>:

Hi Deng,

The jarFiles parameter of `createRemoteEnvironment` means that the path of your custom library jar. If you don’t need custom library, you can omit the parameter.

Regards,
Chiwan Park

> On Sep 25, 2015, at 10:48 AM, Deng Jie <[hidden email]> wrote:
>
> Dear Flink org,i have same question,like:
> public static void main(String[] args) throws Exception {
>
>     ExecutionEnvironment env
> =
>  ExecutionEnvironment
>
> .createRemoteEnvironment("flink-master", 6123, "/home/user/udfs.jar");
>
>
>     DataSet
> <String> data = env.readTextFile("hdfs://path/to/file");
>
>
>     data
>
> .filter(new FilterFunction<String>() {
>
>
> public boolean filter(String value) {
>
>
> return value.startsWith("http://");
>
>
> }
>
>
> })
>
>
> .writeAsText("hdfs://path/to/result");
>
>
>     env
> .execute();
> }
>
> How to write the file(udfs.jar),can you give me an example?In addition,can this parameter as an option?
>
> -- Best wishes for you

[Chiwan Park-2]

Oh, sorry for wrong information.
I have misunderstood about `jarFiles` parameter.

Regards,
Chiwan Park

> On Sep 25, 2015, at 5:27 PM, Fabian Hueske <[hidden email]> wrote:
>
> Hi Deng Jie,
>
> your Flink program needs to be packaged into a JAR file.
> The Flink quickstart Maven archetypes for Java [1] and Scala [2] help to setup a Maven project that packages your program correctly into a JAR file.
>
> Best, Fabian
>
> [1] https://ci.apache.org/projects/flink/flink-docs-release-0.9/quickstart/java_api_quickstart.html
> [2] https://ci.apache.org/projects/flink/flink-docs-release-0.9/quickstart/scala_api_quickstart.html
>
> 2015-09-25 10:15 GMT+02:00 Deng Jie <[hidden email]>:
> Dear Chiwan,
> However, 'createRemoteEnvironment' method must specify this parameter.
>
> -- Best wishes for you
> 邓杰
> 15994846535
>
> 2015-09-25 13:42 GMT+08:00 Chiwan Park <[hidden email]>:
> Hi Deng,
>
> The jarFiles parameter of `createRemoteEnvironment` means that the path of your custom library jar. If you don’t need custom library, you can omit the parameter.
>
> Regards,
> Chiwan Park
>
> > On Sep 25, 2015, at 10:48 AM, Deng Jie <[hidden email]> wrote:
> >
> > Dear Flink org,i have same question,like:
> > public static void main(String[] args) throws Exception {
> >
> >     ExecutionEnvironment env
> > =
> >  ExecutionEnvironment
> >
> > .createRemoteEnvironment("flink-master", 6123, "/home/user/udfs.jar");
> >
> >
> >     DataSet
> > <String> data = env.readTextFile("hdfs://path/to/file");
> >
> >
> >     data
> >
> > .filter(new FilterFunction<String>() {
> >
> >
> > public boolean filter(String value) {
> >
> >
> > return value.startsWith("http://");
> >
> >
> > }
> >
> >
> > })
> >
> >
> > .writeAsText("hdfs://path/to/result");
> >
> >
> >     env
> > .execute();
> > }
> >
> > How to write the file(udfs.jar),can you give me an example?In addition,can this parameter as an option?
> >
> > -- Best wishes for you
>
>
>
>
>