Split Stream on a Split Stream

[Taher Koitawala]

Hi All,

          We are currently working with Flink 1.7.2 version and we are get the below given exception when doing a split on a split.

SplitStream<String>splitStream=stream1.split(new SomeSplitLogic());

DataStream select1=splitStream.select("1");

DataStream select2=splitStream.select("2");

select2.split(new AnotherSplitLogic()).select("3");

  

Basically the exception is recommending to use SideOutput, however the only way I see to get a side output is by using a process function. Can someone suggest a better way of doing this?

Exception :

Caused by: java.lang.IllegalStateException:  Consecutive multiple splits are not supported. Splits are deprecated. Please use side-outputs

[Konstantin Knauf-2]

Hi Taher ,

a ProcessFunction is actually the way to do this. When chained to the previous operator the overhead of such a ProcessFunction in negligible.

Any particular reason you don't want to go for a ProcessFunctio?

Cheers,

Konstantin

On Wed, Feb 27, 2019 at 8:36 AM Taher Koitawala <[hidden email]> wrote:

Hi All,

          We are currently working with Flink 1.7.2 version and we are get the below given exception when doing a split on a split.

SplitStream<String>splitStream=stream1.split(new SomeSplitLogic());

DataStream select1=splitStream.select("1");

DataStream select2=splitStream.select("2");

select2.split(new AnotherSplitLogic()).select("3");

  

Basically the exception is recommending to use SideOutput, however the only way I see to get a side output is by using a process function. Can someone suggest a better way of doing this?

Exception :

Caused by: java.lang.IllegalStateException:  Consecutive multiple splits are not supported. Splits are deprecated. Please use side-outputs

--

[Taher Koitawala]

No particular reason for not using the process function, just wanted to clarify if that was the correct way to do it. Thanks Knauf.

On Wed, Feb 27, 2019 at 8:23 PM Konstantin Knauf <[hidden email]> wrote:

Hi Taher ,

a ProcessFunction is actually the way to do this. When chained to the previous operator the overhead of such a ProcessFunction in negligible.

Any particular reason you don't want to go for a ProcessFunctio?

Cheers,

Konstantin

On Wed, Feb 27, 2019 at 8:36 AM Taher Koitawala <[hidden email]> wrote:

Hi All,

          We are currently working with Flink 1.7.2 version and we are get the below given exception when doing a split on a split.

SplitStream<String>splitStream=stream1.split(new SomeSplitLogic());

DataStream select1=splitStream.select("1");

DataStream select2=splitStream.select("2");

select2.split(new AnotherSplitLogic()).select("3");

  

Basically the exception is recommending to use SideOutput, however the only way I see to get a side output is by using a process function. Can someone suggest a better way of doing this?

Exception :

Caused by: java.lang.IllegalStateException:  Consecutive multiple splits are not supported. Splits are deprecated. Please use side-outputs

--