is dataset immutable in Flink?

Posted by Xing Feng on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/is-dataset-immutable-in-Flink-tp14051.html

Hi, all,

I have a case where each element in the Dataset is an Arraylist. And I may add some elements in the Arraylist. Therefore, I use map operator to add elements. However, due to dataset is immutable. Every time I use a mapper to add an element, it actually creates another arraylist and add element in it.

Below is the code as an example.

DataSet<ArrayList<Integer>> dataset0 = env.fromElements(new ArrayList<>(), new ArrayList<>());
DataSet<ArrayList<Integer>> dataset1 = dataset0.map(
new MapFunction<ArrayList<Integer>, ArrayList<Integer>>() {
  @Override
  public ArrayList<Integer> map(ArrayList<Integer> value) throws Exception {
    value.add(10);
    return value;
  }
});
DataSet<ArrayList<Integer>> unioned = dataset0.union(dataset1);
unioned.print();

The output is “[] [] [10] [10]”

But I expect “[10] [10] [10] [10]”

Any advice is appreciated.

--