Job Manager HA manual setup
Locked
 
|
Hi All, We have already try to setup the Job Manager HA based on the documentation and using script and provided zookeeper. It works. However currently everything is done using start-cluster script that I believe will require passwordlress ssh between node. We are restricted with our environment so this one is not possible. Is it possible to setup the Job Manager HA manually ? By starting each job manager with in each node and task manager. We have our zookeeper and hdfs cluster already. We have tried this one the job manager can failover, but the task manager can be relocated to the new task manager. Is there some settings for this one ? Or is the task manager also can be relocate to the new job manager ? Any more details on the mechanism used on Job Manager HA and interaction with Zookeeper ? Is task manager also registered on Zookeeper ? How they find the right job manager master ? Thanks for your help. Cheers --
|
|
|
typos We have tried this one the job manager can failover, but the task manager CAN'T be relocated to the new task manager. Is there some settings for this one ? Or is the task manager also can be relocate to the new job manager ? Cheers On Sat, Feb 27, 2016 at 7:27 AM, Welly Tambunan <[hidden email]> wrote:
-- |
|
|
Hey Welly!
Yes, it is possible to do manually via the jobmanager.sh and taskmanager.sh script like this: jobmanager.sh start cluster $HOST $WEB-UI-PORT taskmanager.sh start The start-cluster.sh script is just a wrapper around these scripts. From experience, it's often forgotten to sync the configuration files. Make sure to have the same configuration file on each host (both job and task managers), because that is relevant to parse the ZooKeeper quorum etc. The task managers retrieve the currently leading job manager via ZooKeeper. If job manager failover works as expected, but the task managers don't connect to the new job manager, I would suspect that the task manager configuration is out of sync. Could you check this please? Moreover, it will be helpful to have a look at the jobmanager and task manager logs to further investigate this. Can you share these? (Privately works as well of course.) – Ufuk On Sat, Feb 27, 2016 at 1:28 AM, Welly Tambunan <[hidden email]> wrote: > typos > > > We have tried this one the job manager can failover, but the task manager > CAN'T be relocated to the new task manager. Is there some settings for this > one ? Or is the task manager also can be relocate to the new job manager ? > > Cheers > > On Sat, Feb 27, 2016 at 7:27 AM, Welly Tambunan <[hidden email]> wrote: >> >> Hi All, >> >> We have already try to setup the Job Manager HA based on the documentation >> and using script and provided zookeeper. It works. >> >> However currently everything is done using start-cluster script that I >> believe will require passwordlress ssh between node. We are restricted with >> our environment so this one is not possible. >> >> Is it possible to setup the Job Manager HA manually ? By starting each job >> manager with in each node and task manager. We have our zookeeper and hdfs >> cluster already. >> >> We have tried this one the job manager can failover, but the task manager >> can be relocated to the new task manager. Is there some settings for this >> one ? Or is the task manager also can be relocate to the new job manager ? >> >> Any more details on the mechanism used on Job Manager HA and interaction >> with Zookeeper ? >> >> Is task manager also registered on Zookeeper ? How they find the right job >> manager master ? >> >> >> Thanks for your help. >> >> Cheers >> -- >> Welly Tambunan >> Triplelands >> >> http://weltam.wordpress.com >> http://www.triplelands.com > > > > > -- > Welly Tambunan > Triplelands > > http://weltam.wordpress.com > http://www.triplelands.com |
|
|
Hi Ufuk, Thanks, it's working fine now with your suggestion. Cheers On Sun, Feb 28, 2016 at 10:10 PM, Ufuk Celebi <[hidden email]> wrote: Hey Welly! -- |
«
Return to (DEPRECATED) Apache Flink User Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |