AllWindowed vs Windowed with 1 key

Posted by Adrienne Kole on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/AllWindowed-vs-Windowed-with-1-key-tp13039.html

Hi,

I am doing simple aggregation with a keyed and global windows in flink.

When I compare the keyed window aggregation with 1 key and global window (which has parallelism 1) I would expect that both of them would have similar performance.

However, keyed stream with 1 key performs with 2x more throughput than global window.

My configuration is 8 node cluster, 16 core in each node, parallelism = 128.

AFAIK, Flink doesn't manage skew by default and uses hash function to assign keys to partitions. So if I have 1 key only, it should go to only one partition always, which is semantically similar to global windows in flink.

What can be the reason behind this difference in performance?

Thanks,

Adrienne