Flattening of events

[Satish Saley]

Hi team,

I am trying to figure out a way to flatten events in my Flink app.

Event that i am consuming from Kafka is

UpperLevelData {

int upperId;

List<ModuleData> listOfModules

}

ModuleData {

 int moduleId;

 string info;

}

After consuming this event, i want to flatten it out in following format -

FlinkReshapedData{

int upperId;

int moduleId;

string info;

}

How would I do it? Shall I use process function for it https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/ ? 

[Chesnay Schepler]

Have a look at flatMaps:
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/overview/#datastream-rarr-datastream-1

On 6/3/2021 8:28 PM, Satish Saley wrote:

> Hi team,
> I am trying to figure out a way to flatten events in my Flink app.
>
> Event that i am consuming from Kafka is
>
> UpperLevelData {
> int upperId;
> List<ModuleData> listOfModules
> }
>
> ModuleData {
>  int moduleId;
>  string info;
> }
>
> After consuming this event, i want to flatten it out in following format -
>
> FlinkReshapedData{
> int upperId;
> int moduleId;
> string info;
> }
>
> How would I do it? Shall I use process function for it
> https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/ 
> <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/datastream/operators/process_function/>
> ?