what happened to flink on tez?

[Yingda Chen]

All,

Back in version 0.9, there has been discussion about building Flink on top of Tez: https://ci.apache.org/projects/flink/flink-docs-release-0.9/setup/flink_on_tez.html 

But looks like things have gone the other way and there has been no follow up about leveraging Tez for yarn environment. Wondering what is the decision behind not to moving forward with Tez? 

Is there anything particular that the community find in Tez that does not work well with Flink? Maybe the support for streaming scenario on Tez is inadequate? Or there is concern about Tez support for non-YARN platform?

Anyone care to comment?

Thanks,

-Yingda

[Fabian Hueske-2]

Hi Yingda,

Flink on Tez was dropped in preparation of the 1.0 release as discussed on the mailing list [1].

In addition to the reasons in the thread, Tez did not support pipelined shuffles which are a prerequisite for low-latency stream processing, the most common use case for Flink.

Cheers,
Fabian

[1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Dripping-the-Flink-on-Tez-code-for-Flink-1-0-tp9744.html

2017-07-07 23:27 GMT+02:00 Yingda Chen <[hidden email]>:

All,

Back in version 0.9, there has been discussion about building Flink on top of Tez: https://ci.apache.org/projects/flink/flink-docs-release-0.9/setup/flink_on_tez.html 

But looks like things have gone the other way and there has been no follow up about leveraging Tez for yarn environment. Wondering what is the decision behind not to moving forward with Tez? 

Is there anything particular that the community find in Tez that does not work well with Flink? Maybe the support for streaming scenario on Tez is inadequate? Or there is concern about Tez support for non-YARN platform?

Anyone care to comment?

Thanks,

-Yingda