I want run flink program in ubuntu x64 Mult Node Cluster what is configuration?

[hagersaleh]

I run flink program in ubuntu x64  Single Node Cluster
I want run flink program in ubuntu x64  Mult Node Cluster
what is configuration for run program in Mult Node?

[rmetzger0]

Hi,

this guide in our documentation should get you started: http://ci.apache.org/projects/flink/flink-docs-master/setup/cluster_setup.html

You basically have to copy flink to all machines and put the hostnames into the slaves file.

On Tue, Jun 2, 2015 at 4:00 PM, hagersaleh <[hidden email]> wrote:

I run flink program in ubuntu x64  Single Node Cluster
I want run flink program in ubuntu x64  Mult Node Cluster
what is configuration for run program in Mult Node?



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/I-want-run-flink-program-in-ubuntu-x64-Mult-Node-Cluster-what-is-configuration-tp1444.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[hagersaleh]

1- I copy flink in the same path all machines
2- in master machine write 10.0.0.1 in conf/flink-conf.yaml
3- in slave machine write 10.0.0.1 in conf/flink-conf.yaml and 10.0.0.2 in conf/slaves

then in master machine open command line write this
bin/start-cluster.sh

display
Starting job manager
Starting task manager on host ubuntu

when run program not work ‎ bin/flink run /home/hadoop/Desktop/simple_code_flink/dist/simple_code_flink.jar

[Matthias J. Sax]

Hi,

in order to use bin/start-cluster.sh you need to configure conf/slaves
on your master machine. (conf/slave is ignored on slave machines).

Hope this helps. If not, please post the error message you get. It's
hard to figure out what wrong without the stacktrace.

-Matthias


On 06/08/2015 09:43 AM, hagersaleh wrote:

> 1- I copy flink in the same path all machines
> 2- in master machine write 10.0.0.1 in conf/flink-conf.yaml
> 3- in slave machine write 10.0.0.1 in conf/flink-conf.yaml and 10.0.0.2 in
> conf/slaves
>
> then in master machine open command line write this
> bin/start-cluster.sh
>
> display
> Starting job manager
> Starting task manager on host ubuntu
>
> when run program not work ‎ bin/flink run
> /home/hadoop/Desktop/simple_code_flink/dist/simple_code_flink.jar
>
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/I-want-run-flink-program-in-ubuntu-x64-Mult-Node-Cluster-what-is-configuration-tp1444p1532.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
>

[hagersaleh]

when run progam

display
error:null

[Matthias J. Sax]

Can you please share the whole console output. It is unclear what the
problem might be from this short message.


On 06/08/2015 10:24 AM, hagersaleh wrote:

> when run progam
>
> display
> error:null
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/I-want-run-flink-program-in-ubuntu-x64-Mult-Node-Cluster-what-is-configuration-tp1444p1534.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
>

[hagersaleh]

I change IP to good IP
but when run program
Error: The program execution failed: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Not enough free slots available to run the job. You can decrease the operator parallelism or increase the number of slots per TaskManager in the configuration. Task to schedule: < Attempt #0 (CHAIN DataSource (at main(TPCHQuery3.java:53) (org.apache.flink.api.java.io.CsvInputFormat)) -> Filter (Filter at main(TPCHQuery3.java:57)) (1/1)) @ (unassigned) - [SCHEDULED] > with groupID < ee0fdf61005cacb7748513264adf86f1 > in sharing group < SlotSharingGroup [5cb8ec8990d20836f80f8db43c61e0b0, 58e498ffd5b50c985489b6c2ca9391b0, ee0fdf61005cacb7748513264adf86f1] >. Resources available to scheduler: Number of instances=0, total number of slots=0, available slots=0

[Matthias J. Sax]

Are you sure, that the TaskManager registered to the JobManager
correctly? You can check on the master machine in your browser:

> localhost:8081

Additionally, you might need to increase the number of slots for the
TaskManager. Increase 'taskmanager.numberOfTaskSlots' within
conf/flink-conf.yaml.


-Matthias


On 06/08/2015 10:52 AM, hagersaleh wrote:

> I change IP to good IP
> but when run program
> Error: The program execution failed:
> org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
> Not enough free slots available to run the job. You can decrease the
> operator parallelism or increase the number of slots per TaskManager in the
> configuration. Task to schedule: < Attempt #0 (CHAIN DataSource (at
> main(TPCHQuery3.java:53) (org.apache.flink.api.java.io.CsvInputFormat)) ->
> Filter (Filter at main(TPCHQuery3.java:57)) (1/1)) @ (unassigned) -
> [SCHEDULED] > with groupID < ee0fdf61005cacb7748513264adf86f1 > in sharing
> group < SlotSharingGroup [5cb8ec8990d20836f80f8db43c61e0b0,
> 58e498ffd5b50c985489b6c2ca9391b0, ee0fdf61005cacb7748513264adf86f1] >.
> Resources available to scheduler: Number of instances=0, total number of
> slots=0, available slots=0
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/I-want-run-flink-program-in-ubuntu-x64-Mult-Node-Cluster-what-is-configuration-tp1444p1537.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
>

[hagersaleh]

very thanks

[Mar_zieh]

Hello

I run "start-cluster.bat" on windows very easy and it works fine. Even
though, when I run "start-cluster.sh" on terminal of ubuntu, I get these
errors:

Starting cluster.
./start-cluster.sh: line 48:
/home/pooya/IdeaProjects/flink-1.6.0/bin/jobmanager.sh: Permission denied
/home/pooya/IdeaProjects/flink-1.6.0/bin/config.sh: line 656:
/home/pooya/IdeaProjects/flink-1.6.0/bin/taskmanager.sh: Permission denied

Could you please help me? What should I do and how to config it?

Thanks in advance.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[miki haiat]

The easiest way to tun it without adding user and root permission  is to run it with sudo command

sudo /start-cluster.sh

If you want to run  high  availability cluster you need to follow those instruction [1]

1.https://ci.apache.org/projects/flink/flink-docs-release-1.6/ops/jobmanager_high_availability.html#standalone-cluster-high-availability

On Sun, Sep 30, 2018 at 12:31 PM Mar_zieh <[hidden email]> wrote:

Hello

I run "start-cluster.bat" on windows very easy and it works fine. Even
though, when I run "start-cluster.sh" on terminal of ubuntu, I get these
errors:

Starting cluster.
./start-cluster.sh: line 48:
/home/pooya/IdeaProjects/flink-1.6.0/bin/jobmanager.sh: Permission denied
/home/pooya/IdeaProjects/flink-1.6.0/bin/config.sh: line 656:
/home/pooya/IdeaProjects/flink-1.6.0/bin/taskmanager.sh: Permission denied

Could you please help me? What should I do and how to config it?

Thanks in advance.



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Mar_zieh]

I run "sudo /start-cluster.sh" but I got this error:

sudo: /start-cluster.sh : command not found.

Would you please tell me for just one node, is it necessary to configure it?

Thank you



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Fabian Hueske-2]

Hi,

these issues are not related to Flink but rather generic Linux / bash issues.

Ensure that the start scripts are executable (can be changed with chmod) your user has the right permissions to executed the start scripts.

Also, you have to use the right path to the scripts. If you are in the base folder of the Flink setup, the scripts are located in the bin folder, i.e., you can run them with ./bin/start-cluster.sh

Best, Fabian

Am So., 30. Sep. 2018 um 13:02 Uhr schrieb Mar_zieh <[hidden email]>:

I run "sudo /start-cluster.sh" but I got this error:

sudo: /start-cluster.sh : command not found.

Would you please tell me for just one node, is it necessary to configure it?

Thank you



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/