Problem: Please check that all IDs specified via `uid(String)` are unique.

[Rami Al-Isawi]

Hi,

I am trying to set uids. I keep getting this (Flink.1.2):

Exception in thread "main" java.lang.IllegalArgumentException: Hash collision on user-specified ID. Most likely cause is a non-unique ID. Please check that all IDs specified via `uid(String)` are unique.

Here is the code snippet. 

public DataStream<UserAction> processUserActions(DataStream<UserRaw> userRawDataStream, Time duration, Sonar.ReportSpan span) {
    return userRawDataStream.map(userRaw->new UserActionValue(userRaw)).startNewChain().uid("111111")
             .keyBy("userRaw.eventRaw.env_key","userRaw.eventRaw.tag","userRaw.eventRaw."+span.name(),"userRaw.type")
            .window(GlobalWindows.create())
            .trigger(new TimedTrigger(duration))
            .sum("count")//.startNewChain().uid("22222222")
            .map(userActionsCount->new UserAction(userActionsCount,span));//.startNewChain().uid("33333333");

}

I made sure that they are unique and even with one uid, the error is there. I guess there is an easy fix, but I cannot see it.

Regards,

-Rami 

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

[Chesnay Schepler]

Hello,

was a uid set on "userRawDataStream", or any of it's parent transformations?

On 03.05.2017 12:59, Rami Al-Isawi wrote:

Hi,

I am trying to set uids. I keep getting this (Flink.1.2):

Exception in thread "main" java.lang.IllegalArgumentException: Hash collision on user-specified ID. Most likely cause is a non-unique ID. Please check that all IDs specified via `uid(String)` are unique.

Here is the code snippet. 

public DataStream<UserAction> processUserActions(DataStream<UserRaw> userRawDataStream, Time duration, Sonar.ReportSpan span) {
    return userRawDataStream.map(userRaw->new UserActionValue(userRaw)).startNewChain().uid("111111")
             .keyBy("userRaw.eventRaw.env_key","userRaw.eventRaw.tag","userRaw.eventRaw."+span.name(),"userRaw.type")
            .window(GlobalWindows.create())
            .trigger(new TimedTrigger(duration))
            .sum("count")//.startNewChain().uid("22222222")
            .map(userActionsCount->new UserAction(userActionsCount,span));//.startNewChain().uid("33333333");

}

I made sure that they are unique and even with one uid, the error is there. I guess there is an easy fix, but I cannot see it.

Regards,

-Rami 

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

[Rami Al-Isawi]

Hi,

Nope, that is the only uid in the whole project. The “userRawDataStream" comes from a splitter:

public static class Splitter implements OutputSelector<EventRaw>{
    @Override
    public Iterable<String> select(EventRaw value) {
        List<String> output = new ArrayList<>();
        output.add(value.type);
        return output;
    }
}

Any ideas how to get better debug messages? 

Regards,

-Rami

On 3 May 2017, at 14:12, Chesnay Schepler <[hidden email]> wrote:

Hello,

was a uid set on "userRawDataStream", or any of it's parent transformations?

On 03.05.2017 12:59, Rami Al-Isawi wrote:

Hi,

I am trying to set uids. I keep getting this (Flink.1.2):

Exception in thread "main" java.lang.IllegalArgumentException: Hash collision on user-specified ID. Most likely cause is a non-unique ID. Please check that all IDs specified via `uid(String)` are unique.

Here is the code snippet. 

public DataStream<UserAction> processUserActions(DataStream<UserRaw> userRawDataStream, Time duration, Sonar.ReportSpan span) {
    return userRawDataStream.map(userRaw->new UserActionValue(userRaw)).startNewChain().uid("111111")
             .keyBy("userRaw.eventRaw.env_key","userRaw.eventRaw.tag","userRaw.eventRaw."+span.name(),"userRaw.type")
            .window(GlobalWindows.create())
            .trigger(new TimedTrigger(duration))
            .sum("count")//.startNewChain().uid("22222222")
            .map(userActionsCount->new UserAction(userActionsCount,span));//.startNewChain().uid("33333333");

}

I made sure that they are unique and even with one uid, the error is there. I guess there is an easy fix, but I cannot see it.

Regards,

-Rami 

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

[Chesnay Schepler]

Is the method called multiple times with different input sets?

Does the issue occur regardless of which uid you set?

On 03.05.2017 20:21, Rami Al-Isawi wrote:

Hi,

Nope, that is the only uid in the whole project. The “userRawDataStream" comes from a splitter:

public static class Splitter implements OutputSelector<EventRaw>{
    @Override
    public Iterable<String> select(EventRaw value) {
        List<String> output = new ArrayList<>();
        output.add(value.type);
        return output;
    }
}

Any ideas how to get better debug messages? 

Regards,

-Rami

On 3 May 2017, at 14:12, Chesnay Schepler <[hidden email]> wrote:

Hello,

was a uid set on "userRawDataStream", or any of it's parent transformations?

On 03.05.2017 12:59, Rami Al-Isawi wrote:

Hi,

I am trying to set uids. I keep getting this (Flink.1.2):

Exception in thread "main" java.lang.IllegalArgumentException: Hash collision on user-specified ID. Most likely cause is a non-unique ID. Please check that all IDs specified via `uid(String)` are unique.

Here is the code snippet. 

public DataStream<UserAction> processUserActions(DataStream<UserRaw> userRawDataStream, Time duration, Sonar.ReportSpan span) {
    return userRawDataStream.map(userRaw->new UserActionValue(userRaw)).startNewChain().uid("111111")
             .keyBy("userRaw.eventRaw.env_key","userRaw.eventRaw.tag","userRaw.eventRaw."+span.name(),"userRaw.type")
            .window(GlobalWindows.create())
            .trigger(new TimedTrigger(duration))
            .sum("count")//.startNewChain().uid("22222222")
            .map(userActionsCount->new UserAction(userActionsCount,span));//.startNewChain().uid("33333333");

}

I made sure that they are unique and even with one uid, the error is there. I guess there is an easy fix, but I cannot see it.

Regards,

-Rami 

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.

Disclaimer: This message and any attachments thereto are intended solely for the addressed recipient(s) and may contain confidential information. If you are not the intended recipient, please notify the sender by reply e-mail and delete the e-mail (including any attachments thereto) without producing, distributing or retaining any copies thereof. Any review, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. Thank you.