Out of the blue: "Cannot use split/select with side outputs"

[Julio Biason]

Hey guys,

I got a weird problem with my pipeline.

The pipeline process lines from our logs and generate different metrics based on it (I mean, quite the standard procedure). It uses side outputs for dead letter queues, in case it finds something wrong with the logs and a metric can't be generated. At the end, because some metrics differ in content, we send them to different sinks (based on their class, which is quite simple); for this, we split the pipeline (based, again, on the metrics class) and each split goes to a different metric.

Everything worked fine so far, but I spent some time writing tests for some weird results in generating the metrics but right now, when I tried to run the whole process, I get a "Cannot use split/select with side outputs".

This is really weird to me, cause a couple of weeks, the combination of splits and side outputs worked like a charm.

We didn't update Flink (still running 1.4.0) so I'm really confused on what's going on here.

Any ideas?

--

[Julio Biason]

Update:

Even weirder, I stopped Flink (jobmanager and taskmanager) to increase the number of slots and, upon restart, it crashed again and then processed everything just fine.

On Mon, Mar 19, 2018 at 3:01 PM, Julio Biason <[hidden email]> wrote:

Hey guys,

I got a weird problem with my pipeline.

The pipeline process lines from our logs and generate different metrics based on it (I mean, quite the standard procedure). It uses side outputs for dead letter queues, in case it finds something wrong with the logs and a metric can't be generated. At the end, because some metrics differ in content, we send them to different sinks (based on their class, which is quite simple); for this, we split the pipeline (based, again, on the metrics class) and each split goes to a different metric.

Everything worked fine so far, but I spent some time writing tests for some weird results in generating the metrics but right now, when I tried to run the whole process, I get a "Cannot use split/select with side outputs".

This is really weird to me, cause a couple of weeks, the combination of splits and side outputs worked like a charm.

We didn't update Flink (still running 1.4.0) so I'm really confused on what's going on here.

Any ideas?

--

--