how many 'run -c' commands to start?

classic Classic list List threaded Threaded
7 messages Options
Rob
Reply | Threaded
Open this post in threaded view
|

how many 'run -c' commands to start?

Rob
Hello

I successfully ran a job with 'flink run -c', but this is for the local

setup.

How should i proceed with a cluster? Will flink automagically instantiate

the job on all servers - i hope i don't have to start 'flink run -c' on all

machines.

New to flink and bigdata, so sorry for the probably silly question



Thanks!

Rob

Reply | Threaded
Open this post in threaded view
|

Re: how many 'run -c' commands to start?

Chesnay Schepler
Hi!

Given a Flink cluster, you would only call `flink run ...` to submit a
job once; for simplicity i would submit it on the node where you started
the cluster. Flink will automatically distribute job across the cluster,
in smaller independent parts known as Tasks.

Regards,
Chesnay

On 28.09.2017 08:31, r. r. wrote:

> Hello
>
> I successfully ran a job with 'flink run -c', but this is for the local
>
> setup.
>
> How should i proceed with a cluster? Will flink automagically instantiate
>
> the job on all servers - i hope i don't have to start 'flink run -c' on all
>
> machines.
>
> New to flink and bigdata, so sorry for the probably silly question
>
>
>
> Thanks!
>
> Rob
>
>

Rob
Reply | Threaded
Open this post in threaded view
|

Re: how many 'run -c' commands to start?

Rob
Thank you, Chesnay
to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
Do I need to ensure this by configuration?

btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?

Thanks!
Rob






 >-------- Оригинално писмо --------

 >От: Chesnay Schepler [hidden email]

 >Относно: Re: how many 'run -c' commands to start?

 >До: [hidden email]

 >Изпратено на: 28.09.2017 15:05



 
> Hi!
 
>
 
> Given a Flink cluster, you would only call `flink run ...` to submit a
 
> job once; for simplicity i would submit it on the node where you started
 
> the cluster. Flink will automatically distribute job across the cluster,
 
> in smaller independent parts known as Tasks.
 
>
 
> Regards,
 
> Chesnay
 
>
 
> On 28.09.2017 08:31, r. r. wrote:
 
> > Hello
 
> >
 
> > I successfully ran a job with 'flink run -c', but this is for the local
 
> >
 
> > setup.
 
> >
 
> > How should i proceed with a cluster? Will flink automagically instantiate
 
> >
 
> > the job on all servers - i hope i don't have to start 'flink run -c' on all
 
> >
 
> > machines.
 
> >
 
> > New to flink and bigdata, so sorry for the probably silly question
 
> >
 
> >
 
> >
 
> > Thanks!
 
> >
 
> > Rob
 
> >
 
> >
Reply | Threaded
Open this post in threaded view
|

Re: how many 'run -c' commands to start?

Chesnay Schepler
The only nodes that matter are those on which the Flink processes,  i.e Job- and TaskManagers, are being run.

To prevent a JobManager node failure from causing the job to fail you have to look into an HA setup.
(The jobmanager is responsible for distributing/coordinating work)

In case of a TaskManager node failure the job will fail and restart, provided that enough TaskManagers are
still alive to satisfy the resource requirements of the job.


Can you elaborate a bit more what happened when you used the  --detached param?

On 28.09.2017 16:33, r. r. wrote:
Thank you, Chesnay
to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
Do I need to ensure this by configuration?

btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?

Thanks!
Rob






 >-------- Оригинално писмо --------

 >От: Chesnay Schepler [hidden email]

 >Относно: Re: how many 'run -c' commands to start?

 >До: [hidden email]

 >Изпратено на: 28.09.2017 15:05



 
Hi!
 

      
 
Given a Flink cluster, you would only call `flink run ...` to submit a 
 
job once; for simplicity i would submit it on the node where you started 
 
the cluster. Flink will automatically distribute job across the cluster, 
 
in smaller independent parts known as Tasks.
 

      
 
Regards,
 
Chesnay
 

      
 
On 28.09.2017 08:31, r. r. wrote:
 
Hello
 

        
 
I successfully ran a job with 'flink run -c', but this is for the local
 

        
 
setup.
 

        
 
How should i proceed with a cluster? Will flink automagically instantiate
 

        
 
the job on all servers - i hope i don't have to start 'flink run -c' on all
 

        
 
machines.
 

        
 
New to flink and bigdata, so sorry for the probably silly question
 

        
 

        
 

        
 
Thanks!
 

        
 
Rob
 

        
 

        

    


Rob
Reply | Threaded
Open this post in threaded view
|

Re: how many 'run -c' commands to start?

Rob
Sure, here is the cmdline output:

flink-1.3.2/bin/flink run -c com.corp.flink.KafkaJob quickstart.jar --topic InputQ --bootstrap.servers localhost:9092 --zookeeper.connect localhost:2181 --group.id Consumers -p 5 --detached
Cluster configuration: Standalone cluster with JobManager at localhost/127.0.0.1:6123
Using address localhost:6123 to connect to JobManager.
JobManager web interface address http://localhost:8081
Starting execution of program
Submitting job with JobID: 5cc74547361cec2ff9874764dac9ee91. Waiting for job completion.
Connected to JobManager at Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-100905913] with leader session id 00000000-0000-0000-0000-000000000000.
09/28/2017 16:30:40    Job execution switched to status RUNNING.
09/28/2017 16:30:40    Source: Custom Source -> Map(1/5) switched to SCHEDULED
09/28/2017 16:30:40    Source: Custom Source -> Map(2/5) switched to SCHEDULED
09/28/2017 16:30:40    Source: Custom Source -> Map(3/5) switched to SCHEDULED
09/28/2017 16:30:40    Source: Custom Source -> Map(4/5) switched to SCHEDULED
09/28/2017 16:30:40    Source: Custom Source -> Map(5/5) switched to SCHEDULED
09/28/2017 16:30:40    Sink: Unnamed(1/5) switched to SCHEDULED
09/28/2017 16:30:40    Sink: Unnamed(2/5) switched to SCHEDULED
09/28/2017 16:30:40    Sink: Unnamed(3/5) switched to SCHEDULED
09/28/2017 16:30:40    Sink: Unnamed(4/5) switched to SCHEDULED
09/28/2017 16:30:40    Sink: Unnamed(5/5) switched to SCHEDULED
...

I thought --detach will put the process in the background, and give me back the cmdline, but maybe I got the meaning behind this option wrong?

Thank you!







 >-------- Оригинално писмо --------

 >От: Chesnay Schepler [hidden email]

 >Относно: Re: how many 'run -c' commands to start?

 >До: [hidden email]

 >Изпратено на: 29.09.2017 18:01



 
>
 
>    
 
>  
 
>  
 
>  
 
>    The only nodes that matter are those on which the Flink processes,  i.e
 
>    Job- and TaskManagers, are being run.
 
>    
 
>     To prevent a JobManager node failure from causing the job to fail you have to look into an
 
>    HA setup.
 
>     (The jobmanager is responsible for distributing/coordinating work)
 
>    
 
>     In case of a TaskManager node failure the job will fail and
 
>    restart, provided that enough TaskManagers are
 
>     still alive to satisfy the resource requirements of the job.
 
>    
 
>    
 
>     Can you elaborate a bit more what happened when you used the  --detached param?
 
>    
 
>     On 28.09.2017 16:33, r. r. wrote:
 
>    
 
>    
 
>    
 
>    Thank you, Chesnay
 
> to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
 
> Do I need to ensure this by configuration?
 
>
 
> btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
 
>
 
> Thanks!
 
> Rob
 
>
 
>  >-------- Оригинално писмо --------
 
>
 
>  >От: Chesnay Schepler [hidden email]
 
>
 
>  >Относно: Re: how many 'run -c' commands to start?
 
>
 
>  >До: [hidden email]
 
>
 
>  >Изпратено на: 28.09.2017 15:05
 
>
 
>  
 
>  
 
>    
 
>     Hi!
 
>  
 
>    
 
>    
 
>  
 
>    
 
>      
 
>    
 
>    
 
>  
 
>    
 
>     Given a Flink cluster, you would only call `flink run ...` to submit a
 
>  
 
>    
 
>    
 
>  
 
>    
 
>     job once; for simplicity i would submit it on the node where you started
 
>  
 
>    
 
>    
 
>  
 
>    
 
>     the cluster. Flink will automatically distribute job across the cluster,
 
>  
 
>    
 
>    
 
>  
 
>    
 
>     in smaller independent parts known as Tasks.
 
>  
 
>    
 
>    
 
>  
 
>    
 
>      
 
>    
 
>    
 
>  
 
>    
 
>     Regards,
 
>  
 
>    
 
>    
 
>  
 
>    
 
>     Chesnay
 
>  
 
>    
 
>    
 
>  
 
>    
 
>      
 
>    
 
>    
 
>  
 
>    
 
>     On 28.09.2017 08:31, r. r. wrote:
 
>  
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      Hello
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      I successfully ran a job with 'flink run -c', but this is for the local
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      setup.
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      How should i proceed with a cluster? Will flink automagically instantiate
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      the job on all servers - i hope i don't have to start 'flink run -c' on all
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      machines.
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      New to flink and bigdata, so sorry for the probably silly question
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      Thanks!
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      Rob
 
>  
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>  
 
>    
 
>      
 
>      
 
>      
 
>    
 
>    
 
>    
 
>      
 
>  
Reply | Threaded
Open this post in threaded view
|

Re: how many 'run -c' commands to start?

Chesnay Schepler
The order in which you pass program arguments is quite important. Can
you try again putting --detached directly after run?
I.e. flink run --detached -c ...

The reason being that arguments after the jar are treated as arguments
for the user-program, and not as arguments for the job execution.

On 29.09.2017 17:23, r. r. wrote:

> Sure, here is the cmdline output:
>
> flink-1.3.2/bin/flink run -c com.corp.flink.KafkaJob quickstart.jar --topic InputQ --bootstrap.servers localhost:9092 --zookeeper.connect localhost:2181 --group.id Consumers -p 5 --detached
> Cluster configuration: Standalone cluster with JobManager at localhost/127.0.0.1:6123
> Using address localhost:6123 to connect to JobManager.
> JobManager web interface address http://localhost:8081
> Starting execution of program
> Submitting job with JobID: 5cc74547361cec2ff9874764dac9ee91. Waiting for job completion.
> Connected to JobManager at Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-100905913] with leader session id 00000000-0000-0000-0000-000000000000.
> 09/28/2017 16:30:40    Job execution switched to status RUNNING.
> 09/28/2017 16:30:40    Source: Custom Source -> Map(1/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Source: Custom Source -> Map(2/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Source: Custom Source -> Map(3/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Source: Custom Source -> Map(4/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Source: Custom Source -> Map(5/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Sink: Unnamed(1/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Sink: Unnamed(2/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Sink: Unnamed(3/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Sink: Unnamed(4/5) switched to SCHEDULED
> 09/28/2017 16:30:40    Sink: Unnamed(5/5) switched to SCHEDULED
> ...
>
> I thought --detach will put the process in the background, and give me back the cmdline, but maybe I got the meaning behind this option wrong?
>
> Thank you!
>
>
>
>
>
>
>
>   >-------- Оригинално писмо --------
>
>   >От: Chesnay Schepler [hidden email]
>
>   >Относно: Re: how many 'run -c' commands to start?
>
>   >До: [hidden email]
>
>   >Изпратено на: 29.09.2017 18:01
>
>
>
>  
>  
>>    
>  
>>  
>  
>>    
>  
>>    
>  
>>     The only nodes that matter are those on which the Flink processes,  i.e
>  
>>     Job- and TaskManagers, are being run.
>  
>>      
>  
>>      To prevent a JobManager node failure from causing the job to fail you have to look into an
>  
>>     HA setup.
>  
>>      (The jobmanager is responsible for distributing/coordinating work)
>  
>>      
>  
>>      In case of a TaskManager node failure the job will fail and
>  
>>     restart, provided that enough TaskManagers are
>  
>>      still alive to satisfy the resource requirements of the job.
>  
>>      
>  
>>      
>  
>>      Can you elaborate a bit more what happened when you used the  --detached param?
>  
>>      
>  
>>      On 28.09.2017 16:33, r. r. wrote:
>  
>>      
>  
>>    
>  
>>    
>  
>>     Thank you, Chesnay
>  
>> to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
>  
>> Do I need to ensure this by configuration?
>  
>  
>> btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
>  
>  
>> Thanks!
>  
>> Rob
>  
>  
>>   >-------- Оригинално писмо --------
>  
>  
>>   >От: Chesnay Schepler [hidden email]
>  
>  
>>   >Относно: Re: how many 'run -c' commands to start?
>  
>  
>>   >До: [hidden email]
>  
>  
>>   >Изпратено на: 28.09.2017 15:05
>  
>  
>>  
>  
>>  
>  
>>      
>  
>>      Hi!
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      Given a Flink cluster, you would only call `flink run ...` to submit a
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      job once; for simplicity i would submit it on the node where you started
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      the cluster. Flink will automatically distribute job across the cluster,
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      in smaller independent parts known as Tasks.
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      Regards,
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      Chesnay
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      On 28.09.2017 08:31, r. r. wrote:
>  
>>  
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       Hello
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       I successfully ran a job with 'flink run -c', but this is for the local
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       setup.
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       How should i proceed with a cluster? Will flink automagically instantiate
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       the job on all servers - i hope i don't have to start 'flink run -c' on all
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       machines.
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       New to flink and bigdata, so sorry for the probably silly question
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       Thanks!
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>       Rob
>  
>>  
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>  
>  
>>      
>  
>>      
>  
>>        
>  
>>      
>  
>>      
>  
>>      
>  
>>    
>  
>>        
>  
>>  


Rob
Reply | Threaded
Open this post in threaded view
|

Re: how many 'run -c' commands to start?

Rob
Thanks, Chesnay, that was indeed the problem.
It also explains why -p5 was not working for me from the cmdline

Best regards
Robert






 >-------- Оригинално писмо --------

 >От: Chesnay Schepler [hidden email]

 >Относно: Re: how many 'run -c' commands to start?

 >До: "r. r." <[hidden email]>

 >Изпратено на: 01.10.2017 00:58



 
> The order in which you pass program arguments is quite important. Can
 
> you try again putting --detached directly after run?
 
> I.e. flink run --detached -c ...
 
>
 
> The reason being that arguments after the jar are treated as arguments
 
> for the user-program, and not as arguments for the job execution.
 
>
 
> On 29.09.2017 17:23, r. r. wrote:
 
> > Sure, here is the cmdline output:
 
> >
 
> > flink-1.3.2/bin/flink run -c com.corp.flink.KafkaJob quickstart.jar --topic InputQ --bootstrap.servers localhost:9092 --zookeeper.connect localhost:2181 --group.id Consumers -p 5 --detached
 
> > Cluster configuration: Standalone cluster with JobManager at localhost/127.0.0.1:6123
 
> > Using address localhost:6123 to connect to JobManager.
 
> > JobManager web interface address http://localhost:8081
 
> > Starting execution of program
 
> > Submitting job with JobID: 5cc74547361cec2ff9874764dac9ee91. Waiting for job completion.
 
> > Connected to JobManager at Actor[akka.tcp://flink@localhost:6123/user/jobmanager#-100905913] with leader session id 00000000-0000-0000-0000-000000000000.
 
> > 09/28/2017 16:30:40    Job execution switched to status RUNNING.
 
> > 09/28/2017 16:30:40    Source: Custom Source -> Map(1/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Source: Custom Source -> Map(2/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Source: Custom Source -> Map(3/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Source: Custom Source -> Map(4/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Source: Custom Source -> Map(5/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Sink: Unnamed(1/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Sink: Unnamed(2/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Sink: Unnamed(3/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Sink: Unnamed(4/5) switched to SCHEDULED
 
> > 09/28/2017 16:30:40    Sink: Unnamed(5/5) switched to SCHEDULED
 
> > ...
 
> >
 
> > I thought --detach will put the process in the background, and give me back the cmdline, but maybe I got the meaning behind this option wrong?
 
> >
 
> > Thank you!
 
> >
 
> >
 
> >
 
> >
 
> >
 
> >
 
> >
 
> >   >-------- Оригинално писмо --------
 
> >
 
> >   >От: Chesnay Schepler [hidden email]
 
> >
 
> >   >Относно: Re: how many 'run -c' commands to start?
 
> >
 
> >   >До: [hidden email]
 
> >
 
> >   >Изпратено на: 29.09.2017 18:01
 
> >
 
> >
 
> >
 
> >  
 
> >  
 
> >>    
 
> >  
 
> >>  
 
> >  
 
> >>    
 
> >  
 
> >>    
 
> >  
 
> >>     The only nodes that matter are those on which the Flink processes,  i.e
 
> >  
 
> >>     Job- and TaskManagers, are being run.
 
> >  
 
> >>      
 
> >  
 
> >>      To prevent a JobManager node failure from causing the job to fail you have to look into an
 
> >  
 
> >>     HA setup.
 
> >  
 
> >>      (The jobmanager is responsible for distributing/coordinating work)
 
> >  
 
> >>      
 
> >  
 
> >>      In case of a TaskManager node failure the job will fail and
 
> >  
 
> >>     restart, provided that enough TaskManagers are
 
> >  
 
> >>      still alive to satisfy the resource requirements of the job.
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      Can you elaborate a bit more what happened when you used the  --detached param?
 
> >  
 
> >>      
 
> >  
 
> >>      On 28.09.2017 16:33, r. r. wrote:
 
> >  
 
> >>      
 
> >  
 
> >>    
 
> >  
 
> >>    
 
> >  
 
> >>     Thank you, Chesnay
 
> >  
 
> >> to make sure - should the node where the job has been submitted goes down, the processing will continue, I hope?
 
> >  
 
> >> Do I need to ensure this by configuration?
 
> >  
 
> >  
 
> >> btw I added --detached param to the run cmd, but it didn't go into background process as I would've expected. Am I guessing wrong?
 
> >  
 
> >  
 
> >> Thanks!
 
> >  
 
> >> Rob
 
> >  
 
> >  
 
> >>   >-------- Оригинално писмо --------
 
> >  
 
> >  
 
> >>   >От: Chesnay Schepler [hidden email]
 
> >  
 
> >  
 
> >>   >Относно: Re: how many 'run -c' commands to start?
 
> >  
 
> >  
 
> >>   >До: [hidden email]
 
> >  
 
> >  
 
> >>   >Изпратено на: 28.09.2017 15:05
 
> >  
 
> >  
 
> >>  
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      Hi!
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      Given a Flink cluster, you would only call `flink run ...` to submit a
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      job once; for simplicity i would submit it on the node where you started
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      the cluster. Flink will automatically distribute job across the cluster,
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      in smaller independent parts known as Tasks.
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      Regards,
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      Chesnay
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      On 28.09.2017 08:31, r. r. wrote:
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       Hello
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       I successfully ran a job with 'flink run -c', but this is for the local
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       setup.
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       How should i proceed with a cluster? Will flink automagically instantiate
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       the job on all servers - i hope i don't have to start 'flink run -c' on all
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       machines.
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       New to flink and bigdata, so sorry for the probably silly question
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       Thanks!
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>       Rob
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>  
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>        
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>      
 
> >  
 
> >>    
 
> >  
 
> >>        
 
> >  
 
> >>