Re: fliter and flatMap operation VS only a flatMap operation

Posted by Tzu-Li (Gordon) Tai on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/fliter-and-flatMap-operation-VS-only-a-flatMap-operation-tp32474p32483.html

Hi,

If your filter and flatMap operators are chained, then the performance difference should not be noticeable.
If a shuffle (i.e. a keyBy operation) occurs after the filter and before the flatMap, then applying the filter first will be more efficient.

Cheers,
Gordon

On Thu, Jan 30, 2020 at 4:03 AM Soheil Pourbafrani <[hidden email]> wrote:

Hi,

In case we need to filter operation followed by a transformation, which one is more efficient in Flink, applying the filter operation first and then a flatMap operation separately OR using only a flatMap operation that internally includes the filter logic, too?

best

Soheil