key

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

key

Radu Tudoran

Hi,

 

I want to apply a “keyBy operator on a stream”.

The string is of type MyEvent. This is a simple type that contains 2 longs and and int or string

 

However, when applying this I get

 

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: This type (GenericType<Event>) cannot be used as key.

 

Can you give me a hint about a solution to this?

 

Thanks

 

Dr. Radu Tudoran

Research Engineer

IT R&D Division

 

cid:image007.jpg@01CD52EB.AD060EE0

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: +49 15209084330

Telephone: +49 891588344173

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!

 

Reply | Threaded
Open this post in threaded view
|

Re: key

Till Rohrmann

Hi Radu,

if you want to use custom types as keys, then these custom types have to implement the Key interface.

Cheers,
Till


On Mon, Nov 30, 2015 at 5:28 PM, Radu Tudoran <[hidden email]> wrote:

Hi,

 

I want to apply a “keyBy operator on a stream”.

The string is of type MyEvent. This is a simple type that contains 2 longs and and int or string

 

However, when applying this I get

 

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: This type (GenericType<Event>) cannot be used as key.

 

Can you give me a hint about a solution to this?

 

Thanks

 

Dr. Radu Tudoran

Research Engineer

IT R&D Division

 

cid:image007.jpg@01CD52EB.AD060EE0

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: <a href="tel:%2B49%2015209084330" value="+4915209084330" target="_blank">+49 15209084330

Telephone: <a href="tel:%2B49%20891588344173" value="+49891588344173" target="_blank">+49 891588344173

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!

 


Reply | Threaded
Open this post in threaded view
|

Re: key

Márton Balassi
Hey Radu,

To add to Till's comment: do you need the whole Event type to be the key are would you like to group the records based on the value of one of your attributes (the 2 longs, int or string as mentioned)? If the latter is true Flink comes with utilities to use standard types as keys. In the former case Till's comment holds.

Best,

Marton

On Mon, Nov 30, 2015 at 5:38 PM, Till Rohrmann <[hidden email]> wrote:

Hi Radu,

if you want to use custom types as keys, then these custom types have to implement the Key interface.

Cheers,
Till


On Mon, Nov 30, 2015 at 5:28 PM, Radu Tudoran <[hidden email]> wrote:

Hi,

 

I want to apply a “keyBy operator on a stream”.

The string is of type MyEvent. This is a simple type that contains 2 longs and and int or string

 

However, when applying this I get

 

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: This type (GenericType<Event>) cannot be used as key.

 

Can you give me a hint about a solution to this?

 

Thanks

 

Dr. Radu Tudoran

Research Engineer

IT R&D Division

 

cid:image007.jpg@01CD52EB.AD060EE0

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: <a href="tel:%2B49%2015209084330" value="+4915209084330" target="_blank">+49 15209084330

Telephone: <a href="tel:%2B49%20891588344173" value="+49891588344173" target="_blank">+49 891588344173

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!

 



Reply | Threaded
Open this post in threaded view
|

RE: key

Radu Tudoran

Thank you both for the information.

Meanwhile I realized I had a mistmatch between the setters/getters names and fields, after correcting this, it worked. It seems that it works also without implementing the Key interface.

My new question is, should it still implement the key interface? Is it expected to go wrong somewhere else without having it?

 

@Marton – the answer was that I need 2 of these fields to be used to do the keyBy (long, string)

 

 

Dr. Radu Tudoran

Research Engineer

IT R&D Division

 

cid:image007.jpg@01CD52EB.AD060EE0

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: +49 15209084330

Telephone: +49 891588344173

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!

 

From: Márton Balassi [mailto:[hidden email]]
Sent: Monday, November 30, 2015 5:42 PM
To: [hidden email]
Subject: Re: key

 

Hey Radu,

To add to Till's comment: do you need the whole Event type to be the key are would you like to group the records based on the value of one of your attributes (the 2 longs, int or string as mentioned)? If the latter is true Flink comes with utilities to use standard types as keys. In the former case Till's comment holds.

Best,

Marton

 

On Mon, Nov 30, 2015 at 5:38 PM, Till Rohrmann <[hidden email]> wrote:

Hi Radu,

if you want to use custom types as keys, then these custom types have to implement the Key interface.

Cheers,
Till

 

On Mon, Nov 30, 2015 at 5:28 PM, Radu Tudoran <[hidden email]> wrote:

Hi,

 

I want to apply a “keyBy operator on a stream”.

The string is of type MyEvent. This is a simple type that contains 2 longs and and int or string

 

However, when applying this I get

 

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: This type (GenericType<Event>) cannot be used as key.

 

Can you give me a hint about a solution to this?

 

Thanks

 

Dr. Radu Tudoran

Research Engineer

IT R&D Division

 

cid:image007.jpg@01CD52EB.AD060EE0

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: <a href="tel:%2B49%2015209084330" target="_blank">+49 15209084330

Telephone: <a href="tel:%2B49%20891588344173" target="_blank">+49 891588344173

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: key

Fabian Hueske-2
Hi Radu,

with the corrected setters/getters, Flink accepts your data type as a POJO data type which automatically is a key (in contrast to the GenericType it was before).
There is no need to implement the Key interface.

Best, Fabian

2015-11-30 17:46 GMT+01:00 Radu Tudoran <[hidden email]>:

Thank you both for the information.

Meanwhile I realized I had a mistmatch between the setters/getters names and fields, after correcting this, it worked. It seems that it works also without implementing the Key interface.

My new question is, should it still implement the key interface? Is it expected to go wrong somewhere else without having it?

 

@Marton – the answer was that I need 2 of these fields to be used to do the keyBy (long, string)

 

 

Dr. Radu Tudoran

Research Engineer

IT R&D Division

 

cid:image007.jpg@01CD52EB.AD060EE0

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: <a href="tel:%2B49%2015209084330" value="+4915209084330" target="_blank">+49 15209084330

Telephone: <a href="tel:%2B49%20891588344173" value="+49891588344173" target="_blank">+49 891588344173

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!

 

From: Márton Balassi [mailto:[hidden email]]
Sent: Monday, November 30, 2015 5:42 PM
To: [hidden email]
Subject: Re: key

 

Hey Radu,

To add to Till's comment: do you need the whole Event type to be the key are would you like to group the records based on the value of one of your attributes (the 2 longs, int or string as mentioned)? If the latter is true Flink comes with utilities to use standard types as keys. In the former case Till's comment holds.

Best,

Marton

 

On Mon, Nov 30, 2015 at 5:38 PM, Till Rohrmann <[hidden email]> wrote:

Hi Radu,

if you want to use custom types as keys, then these custom types have to implement the Key interface.

Cheers,
Till

 

On Mon, Nov 30, 2015 at 5:28 PM, Radu Tudoran <[hidden email]> wrote:

Hi,

 

I want to apply a “keyBy operator on a stream”.

The string is of type MyEvent. This is a simple type that contains 2 longs and and int or string

 

However, when applying this I get

 

Exception in thread "main" org.apache.flink.api.common.InvalidProgramException: This type (GenericType<Event>) cannot be used as key.

 

Can you give me a hint about a solution to this?

 

Thanks

 

Dr. Radu Tudoran

Research Engineer

IT R&D Division

 

cid:image007.jpg@01CD52EB.AD060EE0

HUAWEI TECHNOLOGIES Duesseldorf GmbH

European Research Center

Riesstrasse 25, 80992 München

 

E-mail: [hidden email]

Mobile: <a href="tel:%2B49%2015209084330" target="_blank">+49 15209084330

Telephone: <a href="tel:%2B49%20891588344173" target="_blank">+49 891588344173

 

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN

This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!