Re: Flink stress testing and metrics

Posted by Timo Walther on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Flink-stress-testing-and-metrics-tp16851p16862.html

Hi Sadok,

it would be helpful if you could tell us a bit more about your job. E.g. a skewed key distribution where keys are only sent to one third of your operators can not use your CPUs full capabilities.

The latency tracking interval is in milliseconds. Can you try if 1000 would fix your problem? I could not find an open issue describing your problem. Maybe more information about your environment can help. How are you executing your Flink application? Are you using a parallelism of 8?

Regards,
Timo


Am 11/22/17 um 9:49 AM schrieb Ladhari Sadok:

Hi All,

I want to do a stress testing of my Flink app implementation: event generation with ParallelSourceFunction then measuring the latency ,throughput, CPU & memry leak ...

But when testing, I noticed that :

• the maximum of CPU usage is 30-33%
• latency is always NaNd NaNh in the dashboard ( even I have set this configuration executionConfig.setLatencyTrackingInterval(1); )

Can some one help me find the best solution to smoke testing Flink ?

Note: I'm using Flink 1.3 and the Flink Web UI to visualize the metrics.
Also my PC have a 12Go RAM and 8 Core CPU.

Regards,

Sadok