Re: flink-dist shading

Posted by rmetzger0 on Nov 19, 2016; 11:57am
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Re-flink-dist-shading-tp10221p10225.html

Hi Craig,

I also received only this email (and I'm a moderator of the dev@ list, so the message never made it into Apache's infra)

When this issue was first reported [1][2] I asked on the Maven mailing list what's going on [3]. I think this JIRA contains the most information on the issue: https://issues.apache.org/jira/browse/MNG-5899

It doesn't seem that Maven is going to fix the issue anytime soon.

One idea I had regarding this issue was to print a warning in the maven output if we detect Maven 3.3 at build time.

[1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/flink-dist-packaging-including-unshaded-classes-td9504.html

[2] https://issues.apache.org/jira/browse/FLINK-3158 

[3] http://comments.gmane.org/gmane.comp.jakarta.turbine.maven.user/137621 

On Fri, Nov 18, 2016 at 7:23 PM, Tzu-Li (Gordon) Tai <[hidden email]> wrote:

Hi Craig,

I think the email wasn't sent to the ‘dev’ list, somehow.

Have you tried this:

mvn clean install -DskipTests
# In Maven 3.3 the shading of flink-dist doesn't work properly in one run, so we need to run mvn for flink-dist again.
cd flink-dist
mvn clean install -DskipTests

I agree that it’ll affect downstream users who need to build Flink themselves, and would be best if it can be resolved.

The above is still more or less a “workaround”, but since I don’t really know the reason for why the newer Maven versions

won’t properly shade, we’ll probably need to wait for others more knowledgable on the build infrastructure to chime in and

see if there’s a good long-term solution.

Best Regards,

Gordon

On November 19, 2016 at 8:48:32 AM, Foster, Craig ([hidden email]) wrote:

I’m not even sure this was delivered to the ‘dev’ list but I’ll go ahead and forward the same email to the ‘user’ list since I haven’t seen a response.

-------

 

I’m following up on the issue in FLINK-5013 about flink-dist specifically requiring Maven 3.0.5 through to <3.3. This affects people who build Flink with BigTop (not only EMR), so I’m wondering about the context and how we can properly shade the Apache HTTP libraries so that flink-dist can be built with a current version of Maven. Any insight into this would be helpful.

 

Thanks!

Craig