Flink - excessive amount of yarn container requests for versions > 1.4.0

[sohimankotia]

Hi ,

While running Flink streaming job it is requesting more than specified
resources from yarn. I am giving 17 TM but it is requesting more than > 35
containers from yarn .

This is happening for all versions greater than 1.4.0.


Attaching JM logs.

logs.zip
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t894/logs.zip>  



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Shuyi Chen]

We've seen similar issue in our production, you can refer to this JIRA (https://issues.apache.org/jira/browse/FLINK-10848) for more detail.

Shuyi

On Sun, Dec 9, 2018 at 11:27 PM sohimankotia <[hidden email]> wrote:

Hi ,

While running Flink streaming job it is requesting more than specified
resources from yarn. I am giving 17 TM but it is requesting more than > 35
containers from yarn .

This is happening for all versions greater than 1.4.0.


Attaching JM logs.

logs.zip
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t894/logs.zip> 



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

--

[sohimankotia]

Thanks for your information. 

So there is no solution for this as of now ?

On Mon 10 Dec, 2018, 1:16 PM Shuyi Chen <[hidden email] wrote:

We've seen similar issue in our production, you can refer to this JIRA (https://issues.apache.org/jira/browse/FLINK-10848) for more detail.

Shuyi

On Sun, Dec 9, 2018 at 11:27 PM sohimankotia <[hidden email]> wrote:

Hi ,

While running Flink streaming job it is requesting more than specified
resources from yarn. I am giving 17 TM but it is requesting more than > 35
containers from yarn .

This is happening for all versions greater than 1.4.0.


Attaching JM logs.

logs.zip
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t894/logs.zip> 



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

--