Re: flink-dist shading

Posted by Foster, Craig on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Re-flink-dist-shading-tp10221p10280.html

Thanks Max. Yep, I just confirmed it works.

On 11/22/16, 2:09 AM, "Maximilian Michels" <[hidden email]> wrote:

    Hi Craig,
   
    I've left a comment on the original Maven JIRA issue to revive the
    discussion. For BigTop, you can handle this in the build script by
    building flink-dist again after a successful build. That will always
    work independently of the Maven 3.x version.
   
    -Max
   
   
    On Mon, Nov 21, 2016 at 6:27 PM, Foster, Craig <[hidden email]> wrote:
    > Thanks for explaining, Robert and Gordon. For however it helps, I’ll comment
    > on the original Maven issue which seems to be affecting many people. I am
    > trying to create RPMs using BigTop and am still not quite sure how to handle
    > this case. I guess what I could do is build the parent first in our script
    > and then build flink-dist again as described in the instructions. I’ll try
    > that out and see if it resolves the issue.
    >
    >
    >
    > From: Robert Metzger <[hidden email]>
    > Reply-To: "[hidden email]" <[hidden email]>
    > Date: Saturday, November 19, 2016 at 4:08 AM
    > To: "[hidden email]" <[hidden email]>
    > Subject: Re: flink-dist shading
    >
    >
    >
    > Hi Craig,
    >
    > I also received only this email (and I'm a moderator of the dev@ list, so
    > the message never made it into Apache's infra)
    >
    >
    >
    > When this issue was first reported [1][2] I asked on the Maven mailing list
    > what's going on [3]. I think this JIRA contains the most information on the
    > issue: https://issues.apache.org/jira/browse/MNG-5899
    >
    > It doesn't seem that Maven is going to fix the issue anytime soon.
    >
    >
    >
    > One idea I had regarding this issue was to print a warning in the maven
    > output if we detect Maven 3.3 at build time.
    >
    >
    >
    >
    >
    > [1]
    > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/flink-dist-packaging-including-unshaded-classes-td9504.html
    >
    > [2] https://issues.apache.org/jira/browse/FLINK-3158
    >
    > [3] http://comments.gmane.org/gmane.comp.jakarta.turbine.maven.user/137621
    >
    >
    >
    >
    >
    >
    >
    > On Fri, Nov 18, 2016 at 7:23 PM, Tzu-Li (Gordon) Tai <[hidden email]>
    > wrote:
    >
    > Hi Craig,
    >
    >
    >
    > I think the email wasn't sent to the ‘dev’ list, somehow.
    >
    >
    >
    > Have you tried this:
    >
    >
    >
    > mvn clean install -DskipTests
    >
    > # In Maven 3.3 the shading of flink-dist doesn't work properly in one run,
    > so we need to run mvn for flink-dist again.
    >
    > cd flink-dist
    >
    > mvn clean install -DskipTests
    >
    > I agree that it’ll affect downstream users who need to build Flink
    > themselves, and would be best if it can be resolved.
    >
    > The above is still more or less a “workaround”, but since I don’t really
    > know the reason for why the newer Maven versions
    >
    > won’t properly shade, we’ll probably need to wait for others more
    > knowledgable on the build infrastructure to chime in and
    >
    > see if there’s a good long-term solution.
    >
    >
    >
    > Best Regards,
    >
    > Gordon
    >
    > On November 19, 2016 at 8:48:32 AM, Foster, Craig ([hidden email])
    > wrote:
    >
    > I’m not even sure this was delivered to the ‘dev’ list but I’ll go ahead and
    > forward the same email to the ‘user’ list since I haven’t seen a response.
    >
    > -------
    >
    >
    >
    > I’m following up on the issue in FLINK-5013 about flink-dist specifically
    > requiring Maven 3.0.5 through to <3.3. This affects people who build Flink
    > with BigTop (not only EMR), so I’m wondering about the context and how we
    > can properly shade the Apache HTTP libraries so that flink-dist can be built
    > with a current version of Maven. Any insight into this would be helpful.
    >
    >
    >
    > Thanks!
    >
    > Craig
    >
    >
    >
    >