Maximizing resource allocation in EMR (EC2)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Maximizing resource allocation in EMR (EC2)

Ishwara Varnasi
Hello, my question is regarding running Flink (streaming) on EMR. How do I maximize resource allocation? For example, how do I allocate optimal number of yarn job managers, slots, heap sizes etc given EC2 instance types I'll be using in EMR? I see that (by trial/error) I'm able to allocate just two third of available memory to yarn. Thanks for any suggestions. 
thanks
Ishwara Varnasi