Re: CPU Cores of JobManager

Posted by Timo Walther on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/CPU-Cores-of-JobManager-tp17022p17030.html

I had some profiling tool like jvisualvm in mind. Are you executing
streaming or batch jobs? If streaming, is checkpointing enabled and
which type of statebackend?

@Chesnay do you have experience with slow behavior of the Web UI?

Regards,
Timo


Am 12/5/17 um 10:37 AM schrieb Yuta Morisawa:

> Hi Timo
>
> Thank you for your early reply.
>
> These are commands which I run my apps.
> ./bin/yarn-session.sh -n 20 -jm 6000 -tm 24000 -s 10
> ./bin/flink run -p 100 <jar1> <jar1 parameters>
> ./bin/flink run -p 100 <jar2> <jar2 parameters>
>
> So, JobManager Heap Memory = 6000 MB and it manages 2 jobs.
>
> > Maybe you can use a profiler and find out which component consumes so
> > much CPU resources?
> You mean Java Flight Recorder or JITWatch?
> Or, Flink has original profiler?
> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/application_profiling.html 
>
>
> Regards,
> Yuta
>
> On 2017/12/05 18:02, Timo Walther wrote:
>> Hi Yuta,
>>
>> as far as I know you cannot assign more cores to a JobManager.
>>
>> Can you tell us a bit more about your environment? How many jobs does
>> the JobManager has to manage? How much heap memory is assigned to the
>> JobManager?
>>
>> Maybe you can use a profiler and find out which component consumes so
>> much CPU resources?
>>
>> Regards,
>> Timo
>>
>>
>> Am 12/5/17 um 5:13 AM schrieb Yuta Morisawa:
>>> Hi
>>>
>>> Now I am looking for the way to increase the number of allocated CPU
>>> cores because my JobManagaer WEBUI is very heavy and sometimes freeze.
>>>
>>> I think this is caused by the resource shortage of JobManager.
>>> How can I increase the number of CPU for JobManager in YARN mode?
>>>
>>>
>>>
>>> Thanks
>>> Yuta
>>
>>
>>