The memory usage of the job is very different between Flink1.9 and Flink1.12

[Haihang Jing]

Ask a question, the same business logic, the same resource configuration, the
memory usage of the job is very different between Flink1.9 and Flink1.12.
Using jemalloc analysis, it is found that the
UncompressBlockContentsForCompressionType method of rocksdb takes up more
memory and runs the same time , This method occupies 200MB of memory in
Flink1.9 and about 4G in Flink1.12. Have you ever encountered this
phenomenon?
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t3050/1.png>
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t3050/12.png>



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Piotr Nowojski-4]

Hi,

always when upgrading I would suggest to check release notes first [1]

Best,

Piotrek

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/release-notes/flink-1.11.html#memory-management

pt., 18 cze 2021 o 12:24 Haihang Jing <[hidden email]> napisał(a):

Ask a question, the same business logic, the same resource configuration, the
memory usage of the job is very different between Flink1.9 and Flink1.12.
Using jemalloc analysis, it is found that the
UncompressBlockContentsForCompressionType method of rocksdb takes up more
memory and runs the same time , This method occupies 200MB of memory in
Flink1.9 and about 4G in Flink1.12. Have you ever encountered this
phenomenon?
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t3050/1.png>
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t3050/12.png>



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/