Unsatisfied Link Error

[Debaditya Roy]

I was trying to run a basic program in java by submitting to the job manager in Flink. I have a native library from open CV. When I try to submit the job I get "java.lang.UnsatisfiedLinkError: no opencv_java310 in java.library.path", however when I run it on Intellij by setting up the flink execution environment I get correct results. I have followed some solutions from the apache flink support website:https://mail-archives.apache.org/mod_mbox/flink-user/201604.mbox/%3CCAO0MGUj_h==sw76-TWF6x8fnT_Vdc84mwu=YLejjn=bG-up+MQ@...%3E and have modified my conf.yaml file accordingly (by pointing env.java.opts: -Djava.library.path="/path of Open CV library", but no luck, Maybe my question is very basic , but still I am stuck, any help would be highly appreciated.

 Thanks :-)

[arpit8622]

I am not sure but the reason could be that you are not submitting "fat-jar" containing all the libs when submitting it to job manager. 

When you run it through intellij it takes care of including the libraries.

To create fat jar use maven-shade-plugin.

Let me know if it works.

Thanks,

Arpit

On Mon, May 30, 2016 at 2:41 PM, Debaditya Roy <[hidden email]> wrote:

I was trying to run a basic program in java by submitting to the job manager in Flink. I have a native library from open CV. When I try to submit the job I get "java.lang.UnsatisfiedLinkError: no opencv_java310 in java.library.path", however when I run it on Intellij by setting up the flink execution environment I get correct results. I have followed some solutions from the apache flink support website:https://mail-archives.apache.org/mod_mbox/flink-user/201604.mbox/%3CCAO0MGUj_h==sw76-TWF6x8fnT_Vdc84mwu=YLejjn=bG-up+MQ@...%3E and have modified my conf.yaml file accordingly (by pointing env.java.opts: -Djava.library.path="/path of Open CV library", but no luck, Maybe my question is very basic , but still I am stuck, any help would be highly appreciated.

 Thanks :-)

[Debaditya Roy]

Actually I have worked on IntelliJ only. As you said I also suspect the fat jar (created from intelliJ) is unable to have the link to the native library when I generate it and also the maven plugin is present in the pom.xml . I also used another fat jar creator application to create the fat jar and link it manually with my native library. In the second case I do not find the error of linking however I get a new error:
org.apache.flink.client.program.ProgramInvocationException: The program plan could not be fetched - the program aborted pre-maturely. However the program is executing all the steps it is required to do because I can see all the files written properly as expected.

On Mon, May 30, 2016 at 9:10 PM, arpit srivastava <[hidden email]> wrote:

I am not sure but the reason could be that you are not submitting "fat-jar" containing all the libs when submitting it to job manager. 

When you run it through intellij it takes care of including the libraries.

To create fat jar use maven-shade-plugin.

Let me know if it works.

Thanks,

Arpit

On Mon, May 30, 2016 at 2:41 PM, Debaditya Roy <[hidden email]> wrote:

I was trying to run a basic program in java by submitting to the job manager in Flink. I have a native library from open CV. When I try to submit the job I get "java.lang.UnsatisfiedLinkError: no opencv_java310 in java.library.path", however when I run it on Intellij by setting up the flink execution environment I get correct results. I have followed some solutions from the apache flink support website:https://mail-archives.apache.org/mod_mbox/flink-user/201604.mbox/%3CCAO0MGUj_h==sw76-TWF6x8fnT_Vdc84mwu=YLejjn=bG-up+MQ@...%3E and have modified my conf.yaml file accordingly (by pointing env.java.opts: -Djava.library.path="/path of Open CV library", but no luck, Maybe my question is very basic , but still I am stuck, any help would be highly appreciated.

 Thanks :-)

[Stephan Ewen]

Hi!

Concerning the "the program aborted pre-maturely" exception - I assume you were using the web dashboard to submit the program.

There is a trick that we use to fetch the plan without executing the program, but it can be voided by catching certain exceptions.

I would try to simply execute the program without showing the plan first, and then looking at the plan in the dashboard, under "plan".

Greetings,

Stephan

On Tue, May 31, 2016 at 10:55 AM, Debaditya Roy <[hidden email]> wrote:

Actually I have worked on IntelliJ only. As you said I also suspect the fat jar (created from intelliJ) is unable to have the link to the native library when I generate it and also the maven plugin is present in the pom.xml . I also used another fat jar creator application to create the fat jar and link it manually with my native library. In the second case I do not find the error of linking however I get a new error:
org.apache.flink.client.program.ProgramInvocationException: The program plan could not be fetched - the program aborted pre-maturely. However the program is executing all the steps it is required to do because I can see all the files written properly as expected.

On Mon, May 30, 2016 at 9:10 PM, arpit srivastava <[hidden email]> wrote:

I am not sure but the reason could be that you are not submitting "fat-jar" containing all the libs when submitting it to job manager. 

When you run it through intellij it takes care of including the libraries.

To create fat jar use maven-shade-plugin.

Let me know if it works.

Thanks,

Arpit

On Mon, May 30, 2016 at 2:41 PM, Debaditya Roy <[hidden email]> wrote:

I was trying to run a basic program in java by submitting to the job manager in Flink. I have a native library from open CV. When I try to submit the job I get "java.lang.UnsatisfiedLinkError: no opencv_java310 in java.library.path", however when I run it on Intellij by setting up the flink execution environment I get correct results. I have followed some solutions from the apache flink support website:https://mail-archives.apache.org/mod_mbox/flink-user/201604.mbox/%3CCAO0MGUj_h==sw76-TWF6x8fnT_Vdc84mwu=YLejjn=bG-up+MQ@...%3E and have modified my conf.yaml file accordingly (by pointing env.java.opts: -Djava.library.path="/path of Open CV library", but no luck, Maybe my question is very basic , but still I am stuck, any help would be highly appreciated.

 Thanks :-)

[Debaditya Roy]

Hello,

Yes I ran it from the cli successfully. :-)

Regards

On Tue, May 31, 2016 at 11:03 AM, Stephan Ewen <[hidden email]> wrote:

Hi!

Concerning the "the program aborted pre-maturely" exception - I assume you were using the web dashboard to submit the program.

There is a trick that we use to fetch the plan without executing the program, but it can be voided by catching certain exceptions.

I would try to simply execute the program without showing the plan first, and then looking at the plan in the dashboard, under "plan".

Greetings,

Stephan

On Tue, May 31, 2016 at 10:55 AM, Debaditya Roy <[hidden email]> wrote:

Actually I have worked on IntelliJ only. As you said I also suspect the fat jar (created from intelliJ) is unable to have the link to the native library when I generate it and also the maven plugin is present in the pom.xml . I also used another fat jar creator application to create the fat jar and link it manually with my native library. In the second case I do not find the error of linking however I get a new error:
org.apache.flink.client.program.ProgramInvocationException: The program plan could not be fetched - the program aborted pre-maturely. However the program is executing all the steps it is required to do because I can see all the files written properly as expected.

On Mon, May 30, 2016 at 9:10 PM, arpit srivastava <[hidden email]> wrote:

I am not sure but the reason could be that you are not submitting "fat-jar" containing all the libs when submitting it to job manager. 

When you run it through intellij it takes care of including the libraries.

To create fat jar use maven-shade-plugin.

Let me know if it works.

Thanks,

Arpit

On Mon, May 30, 2016 at 2:41 PM, Debaditya Roy <[hidden email]> wrote:

I was trying to run a basic program in java by submitting to the job manager in Flink. I have a native library from open CV. When I try to submit the job I get "java.lang.UnsatisfiedLinkError: no opencv_java310 in java.library.path", however when I run it on Intellij by setting up the flink execution environment I get correct results. I have followed some solutions from the apache flink support website:https://mail-archives.apache.org/mod_mbox/flink-user/201604.mbox/%3CCAO0MGUj_h==sw76-TWF6x8fnT_Vdc84mwu=YLejjn=bG-up+MQ@...%3E and have modified my conf.yaml file accordingly (by pointing env.java.opts: -Djava.library.path="/path of Open CV library", but no luck, Maybe my question is very basic , but still I am stuck, any help would be highly appreciated.

 Thanks :-)