Login  Register

Configuration Best Practices for Batch Processing

classic Classic list List threaded Threaded
1 message Options Options
Embed post
Permalink
Reply | Threaded
Open this post in threaded view
| More
Print post
Permalink

Configuration Best Practices for Batch Processing

Micah Whitacre
I was curious suggested best practices as it relates to running batch processes on Flink.  Does anyone have any good guides on good default settings and configuration?

One question I'm really curious about is what suggestions there might be for the relationship of memory of TaskManagers? Number of slots? It is it better to have larger TaskManagers with more slots?  Smaller TaskManagers with fewer slots?  Are there any settings related to stuff like netty/akka that are known to be good to tune to higher values for batch that wouldn't be appropriate for streaming?

Thanks,
Micah