(DEPRECATED) Apache Flink User Mailing List archive.

unable to add more servers in zookeeper quorum peers in flink 1.2

Classic

List

Threaded

9 messages Options

kanagaraj.vengidasamy

unable to add more servers in zookeeper quorum peers in flink 1.2

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers

server.0= server1:2888:3888

server.1= server2:2888:3888

server.2= server3:2888:3888

server.3= server4:2888:3888

server.4= server5:2888:3888

server.5= server6:2888:3888

server.6= server7:2888:3888

server.7= server8:2888:3888

server.8= server9:2888:3888

server.9= server10:2888:3888

#server.10= server11:2888:3888

#server.11= server12:2888:3888

Thanks

Kanagaraj

Greg Hogan

Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers
server.0= server1:2888:3888
server.1= server2:2888:3888
server.2= server3:2888:3888
server.3= server4:2888:3888
server.4= server5:2888:3888
server.5= server6:2888:3888
server.6= server7:2888:3888
server.7= server8:2888:3888
server.8= server9:2888:3888
server.9= server10:2888:3888
#server.10= server11:2888:3888
#server.11= server12:2888:3888

Thanks
Kanagaraj

Stephan Ewen

Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Side note: It's quite uncommon to have more than 9 ZK nodes (at least if the ZK is dedicated to Flink alone).

The ZooKeeper docs suggest to add "observers" to scale out ZooKeeper further, not quorum peers, because many quorum peers make ZooKeeper transactions slower (because the quorum that has to reach consensus is larger).

On Wed, Mar 22, 2017 at 3:08 PM, Greg Hogan <[hidden email]> wrote:

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.
^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers
server.0= server1:2888:3888
server.1= server2:2888:3888
server.2= server3:2888:3888
server.3= server4:2888:3888
server.4= server5:2888:3888
server.5= server6:2888:3888
server.6= server7:2888:3888
server.7= server8:2888:3888
server.8= server9:2888:3888
server.9= server10:2888:3888
#server.10= server11:2888:3888
#server.11= server12:2888:3888

Thanks
Kanagaraj

kanagaraj.vengidasamy

RE: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

In reply to this post by Greg Hogan

Greg,

Sorry about that.. when I copy the config and replaced the real server names . I don’t have space in my configurations. The issue is when I have server.10=, server.11=

And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and server.1

Thanks

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

From: Greg Hogan [mailto:[hidden email]]
Sent: Wednesday, March 22, 2017 10:08 AM
To: [hidden email]
Subject: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers

server.0=server1:2888:3888

server.1=server2:2888:3888

server.2=server3:2888:3888

server.3=server4:2888:3888

server.4=server5:2888:3888

server.5=server6:2888:3888

server.6=server7:2888:3888

server.7=server8:2888:3888

server.8=server9:2888:3888

server.9=server10:2888:3888

#server.10=server11:2888:3888

#server.11=server12:2888:3888

Thanks

Kanagaraj

Chesnay Schepler

Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

I guess that's because the grouping is wrong.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

should probably be

^server\.([0-9]+)[[:space:]]*\=([^: \#]+)

Could you modify the .sh script as such and try again?

Regards,
Chesnay

On 22.03.2017 16:10, [hidden email] wrote:

Greg,

Sorry about that.. when I copy the config and replaced the real server names . I don’t have space in my configurations. The issue is when I have server.10=, server.11=

And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and server.1

Thanks

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

From: Greg Hogan [[hidden email]]
Sent: Wednesday, March 22, 2017 10:08 AM
To: [hidden email]
Subject: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers

server.0=server1:2888:3888

server.1=server2:2888:3888

server.2=server3:2888:3888

server.3=server4:2888:3888

server.4=server5:2888:3888

server.5=server6:2888:3888

server.6=server7:2888:3888

server.7=server8:2888:3888

server.8=server9:2888:3888

server.9=server10:2888:3888

#server.10=server11:2888:3888

#server.11=server12:2888:3888

Thanks

Kanagaraj

kanagaraj.vengidasamy

RE: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

That worked.. Thanks Chesnay,

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

From: Chesnay Schepler [mailto:[hidden email]]
Sent: Wednesday, March 22, 2017 11:23 AM
To: [hidden email]
Subject: Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

I guess that's because the grouping is wrong.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

should probably be

^server\.([0-9]+)[[:space:]]*\=([^: \#]+)

Could you modify the .sh script as such and try again?

Regards,
Chesnay

On 22.03.2017 16:10, [hidden email] wrote:

Greg,

Sorry about that.. when I copy the config and replaced the real server names . I don’t have space in my configurations. The issue is when I have server.10=, server.11=

And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and server.1

Thanks

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

From: Greg Hogan [[hidden email]]
Sent: Wednesday, March 22, 2017 10:08 AM
To: [hidden email]
Subject: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers

server.0=server1:2888:3888

server.1=server2:2888:3888

server.2=server3:2888:3888

server.3=server4:2888:3888

server.4=server5:2888:3888

server.5=server6:2888:3888

server.6=server7:2888:3888

server.7=server8:2888:3888

server.8=server9:2888:3888

server.9=server10:2888:3888

#server.10=server11:2888:3888

#server.11=server12:2888:3888

Thanks

Kanagaraj

rmetzger0

Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Thank you for verifying. Fixed in master: http://git-wip-us.apache.org/repos/asf/flink/commit/3e860b40

On Wed, Mar 22, 2017 at 9:25 PM, <[hidden email]> wrote:

That worked.. Thanks Chesnay,

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

From: Chesnay Schepler [mailto:[hidden email]]
Sent: Wednesday, March 22, 2017 11:23 AM
To: [hidden email]
Subject: Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2
I guess that's because the grouping is wrong.
^server\.([0-9])+[[:space:]]*\=([^: \#]+)
should probably be
^server\.([0-9]+)[[:space:]]*\=([^: \#]+)
Could you modify the .sh script as such and try again?

Regards,
Chesnay

On 22.03.2017 16:10, [hidden email] wrote:
Greg,

Sorry about that.. when I copy the config and replaced the real server names . I don’t have space in my configurations. The issue is when I have server.10=, server.11=

And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and server.1

Thanks

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

From: Greg Hogan [[hidden email]]
Sent: Wednesday, March 22, 2017 10:08 AM
To: [hidden email]
Subject: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers

server.0=server1:2888:3888

server.1=server2:2888:3888

server.2=server3:2888:3888

server.3=server4:2888:3888

server.4=server5:2888:3888

server.5=server6:2888:3888

server.6=server7:2888:3888

server.7=server8:2888:3888

server.8=server9:2888:3888

server.9=server10:2888:3888

#server.10=server11:2888:3888

#server.11=server12:2888:3888

Thanks

Kanagaraj

Greg Hogan

Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

A PR and review may have noted that the same regex is in stop-zookeeper-quorum.sh and recommended ignoring whitespace both before and after the equals sign.

On Mar 23, 2017, at 10:12 AM, Robert Metzger <[hidden email]> wrote:
Thank you for verifying. Fixed in master: http://git-wip-us.apache.org/repos/asf/flink/commit/3e860b40
On Wed, Mar 22, 2017 at 9:25 PM, <[hidden email]> wrote:
That worked.. Thanks Chesnay,

<image001.png>

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

<image002.png> <image003.png> <image004.png>

From: Chesnay Schepler [mailto:[hidden email]]
Sent: Wednesday, March 22, 2017 11:23 AM
To: [hidden email]
Subject: Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2
I guess that's because the grouping is wrong.
^server\.([0-9])+[[:space:]]*\=([^: \#]+)
should probably be
^server\.([0-9]+)[[:space:]]*\=([^: \#]+)
Could you modify the .sh script as such and try again?

Regards,
Chesnay

On 22.03.2017 16:10, [hidden email] wrote:
Greg,

Sorry about that.. when I copy the config and replaced the real server names . I don’t have space in my configurations. The issue is when I have server.10=, server.11=
And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and server.1

Thanks

<image001.png>

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

<image002.png> <image003.png> <image004.png>

From: Greg Hogan [[hidden email]]
Sent: Wednesday, March 22, 2017 10:08 AM
To: [hidden email]
Subject: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers

server.0=server1:2888:3888

server.1=server2:2888:3888

server.2=server3:2888:3888

server.3=server4:2888:3888

server.4=server5:2888:3888

server.5=server6:2888:3888

server.6=server7:2888:3888

server.7=server8:2888:3888

server.8=server9:2888:3888

server.9=server10:2888:3888

#server.10=server11:2888:3888

#server.11=server12:2888:3888

Thanks

Kanagaraj

rmetzger0

Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

You are right and I'm sorry. I've opened a PR: https://github.com/apache/flink/pull/3605

On Thu, Mar 23, 2017 at 3:47 PM, Greg Hogan <[hidden email]> wrote:

A PR and review may have noted that the same regex is in stop-zookeeper-quorum.sh and recommended ignoring whitespace both before and after the equals sign.
On Mar 23, 2017, at 10:12 AM, Robert Metzger <[hidden email]> wrote:
Thank you for verifying. Fixed in master: http://git-wip-us.apache.org/repos/asf/flink/commit/3e860b40
On Wed, Mar 22, 2017 at 9:25 PM, <[hidden email]> wrote:
That worked.. Thanks Chesnay,

<image001.png>

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

<image002.png> <image003.png> <image004.png>

From: Chesnay Schepler [mailto:[hidden email]]
Sent: Wednesday, March 22, 2017 11:23 AM
To: [hidden email]
Subject: Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2
I guess that's because the grouping is wrong.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)
should probably be

^server\.([0-9]+)[[:space:]]*\=([^: \#]+)
Could you modify the .sh script as such and try again?

Regards,
Chesnay

On 22.03.2017 16:10, [hidden email] wrote:

Greg,

Sorry about that.. when I copy the config and replaced the real server names . I don’t have space in my configurations. The issue is when I have server.10=, server.11=
And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and server.1

Thanks

<image001.png>

Kanagaraj Vengidasamy
RTCI

7701 E Telecom PKWY
Temple Terrace, FL 33637

O 813.978.4372 | M 813.455.9757

<image002.png> <image003.png> <image004.png>

From: Greg Hogan [[hidden email]]
Sent: Wednesday, March 22, 2017 10:08 AM
To: [hidden email]
Subject: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj,

None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign.

^server\.([0-9])+[[:space:]]*\=([^: \#]+)

Greg

On Mar 22, 2017, at 8:49 AM, [hidden email] wrote:

Hi All,

We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers?

# ZooKeeper quorum peers

server.0=server1:2888:3888

server.1=server2:2888:3888

server.2=server3:2888:3888

server.3=server4:2888:3888

server.4=server5:2888:3888

server.5=server6:2888:3888

server.6=server7:2888:3888

server.7=server8:2888:3888

server.8=server9:2888:3888

server.9=server10:2888:3888

#server.10=server11:2888:3888

#server.11=server12:2888:3888

Thanks

Kanagaraj