Re: Counting the number of elements in a dataset

Posted by Márton Balassi on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Counting-the-number-of-elements-in-a-dataset-tp437p438.html

Hey,

There was a thread recently on the dev list that might be interesting to you [1].

I do not know the exact state of the code though.

[1] http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Hi-Aggregation-support-td2311.html#a2547

Cheers,

Marton

On Sat, Nov 22, 2014 at 8:09 PM, Sebastian Schelter <[hidden email]> wrote:

Hi,

Is there a simple way to count the number of elements of a dataset? At the moment, I have to use the following code, which is pretty verbose and unefficient.

    val numVertices =
      (srcVertices union targetVertices).distinct.reduceGroup { iter =>
        var count = 1L
        while (iter.hasNext) {
          count += 1
          iter.next
        }
        count
      }

Best,
Sebastian