Count on grouped keys

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Count on grouped keys

Punit Naik
I have a dataset which has maps. I have performed a groupBy on a key and I want to count all the elements in a particular group. How do I do this?

--
Thank You

Regards

Punit Naik
Reply | Threaded
Open this post in threaded view
|

Re: Count on grouped keys

Punit Naik
Anyways, I managed to do it. you should attach the following code block to your groupBy
.reduceGroup {
      (in, out: org.apache.flink.util.Collector[(Map[String,String], Int)]) => // Map[String,String] is the data type I want to output along with the count as Int in a Tuple
        var v:Int = 0;
        var k:Map[String,String]=Map()
        for (t <- in) {
          v+=1;
          k=t
        }
        out.collect((k,v))

On Fri, Apr 29, 2016 at 3:59 PM, Punit Naik <[hidden email]> wrote:
I have a dataset which has maps. I have performed a groupBy on a key and I want to count all the elements in a particular group. How do I do this?

--
Thank You

Regards

Punit Naik



--
Thank You

Regards

Punit Naik
Reply | Threaded
Open this post in threaded view
|

Re: Count on grouped keys

stefanobaghino
Thanks for sharing the solution, Punit.

On Fri, Apr 29, 2016 at 1:40 PM, Punit Naik <[hidden email]> wrote:
Anyways, I managed to do it. you should attach the following code block to your groupBy
.reduceGroup {
      (in, out: org.apache.flink.util.Collector[(Map[String,String], Int)]) => // Map[String,String] is the data type I want to output along with the count as Int in a Tuple
        var v:Int = 0;
        var k:Map[String,String]=Map()
        for (t <- in) {
          v+=1;
          k=t
        }
        out.collect((k,v))

On Fri, Apr 29, 2016 at 3:59 PM, Punit Naik <[hidden email]> wrote:
I have a dataset which has maps. I have performed a groupBy on a key and I want to count all the elements in a particular group. How do I do this?

--
Thank You

Regards

Punit Naik



--
Thank You

Regards

Punit Naik



--
BR,
Stefano Baghino

Software Engineer @ Radicalbit
Reply | Threaded
Open this post in threaded view
|

Re: Count on grouped keys

Punit Naik

Yeah no problem. Its not an optimised solution but I think it gives enough understanding of how reduceGroup works.

On 29-Apr-2016 5:17 PM, "Stefano Baghino" <[hidden email]> wrote:
Thanks for sharing the solution, Punit.

On Fri, Apr 29, 2016 at 1:40 PM, Punit Naik <[hidden email]> wrote:
Anyways, I managed to do it. you should attach the following code block to your groupBy
.reduceGroup {
      (in, out: org.apache.flink.util.Collector[(Map[String,String], Int)]) => // Map[String,String] is the data type I want to output along with the count as Int in a Tuple
        var v:Int = 0;
        var k:Map[String,String]=Map()
        for (t <- in) {
          v+=1;
          k=t
        }
        out.collect((k,v))

On Fri, Apr 29, 2016 at 3:59 PM, Punit Naik <[hidden email]> wrote:
I have a dataset which has maps. I have performed a groupBy on a key and I want to count all the elements in a particular group. How do I do this?

--
Thank You

Regards

Punit Naik



--
Thank You

Regards

Punit Naik



--
BR,
Stefano Baghino

Software Engineer @ Radicalbit