(DEPRECATED) Apache Flink User Mailing List archive.

the attribute order in sql 'select * from...'

classic Classic list List threaded Threaded
♦
♦
 6 messages Options
Hongyuhong
Reply | Threaded
Open this post in threaded view
♦
♦
|

the attribute order in sql 'select * from...'

Hongyuhong

Hi,

I’m now using streaming sql, And I have the sql like

select *  FROM OrderA where user > 2

the OrderA has 3 attr (user, product, amount)

and I expect the result is as the order like input, but it has been sorted by attr name,

and I found the order has already been sorted when call addSource,

What is the purpose of doing so?cause it’s a little not meet our requirements.

 

Thanks very much.

 

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

                   super(typeClass);

 

                   checkArgument(Modifier.isPublic(typeClass.getModifiers()),

                                     "POJO %s is not public", typeClass);

 

                   this.fields = fields.toArray(new PojoField[fields.size()]);

 

                   Arrays.sort(this.fields, new Comparator<PojoField>() {

                            @Override

                            public int compare(PojoField o1, PojoField o2) {

                                     return o1.getField().getName().compareTo(o2.getField().getName());

                            }

                   });

 

 

Best,

Yuhong
Fabian Hueske-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: the attribute order in sql 'select * from...'

Fabian Hueske-2
Hi Yuhong,

I assume that OrderA is a table of POJO objects and you are expecting the order of the attribute to be as the order in which the fields of the POJO are defined in the source code.

Flink accepts fields which are either public members or accessible via a getter and setter.
This makes it difficult to automatically define an order, esp. if some fields use getter and setter or public fields. Would the order depend on the field (which might not exist in case of getter/setter) or setter or getter methods (which might also not exist).
I'm also not sure if it is possible to extract the line number of a method or field via reflection.

Best, Fabian



2017-01-13 9:54 GMT+01:00 Hongyuhong <[hidden email]>:

Hi,

I’m now using streaming sql, And I have the sql like

select *  FROM OrderA where user > 2

the OrderA has 3 attr (user, product, amount)

and I expect the result is as the order like input, but it has been sorted by attr name,

and I found the order has already been sorted when call addSource,

What is the purpose of doing so?cause it’s a little not meet our requirements.

 

Thanks very much.

 

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

                   super(typeClass);

 

                   checkArgument(Modifier.isPublic(typeClass.getModifiers()),

                                     "POJO %s is not public", typeClass);

 

                   this.fields = fields.toArray(new PojoField[fields.size()]);

 

                   Arrays.sort(this.fields, new Comparator<PojoField>() {

                            @Override

                            public int compare(PojoField o1, PojoField o2) {

                                     return o1.getField().getName().compareTo(o2.getField().getName());

                            }

                   });

 

 

Best,

Yuhong
Timo Walther
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: the attribute order in sql 'select * from...'

Timo Walther
Hi Yuhong,

as a solution you can specify the order of your Pojo fields when converting from DataStream to Table.

Table table = tableEnv
   .fromDataSet(env.fromCollection(data),
      "department AS a, " +
      "age AS b, " +
      "salary AS c, " +
      "name AS d")
   .select("a, b, c, d");

Timo


Am 13/01/17 um 10:22 schrieb Fabian Hueske:
Hi Yuhong,

I assume that OrderA is a table of POJO objects and you are expecting the order of the attribute to be as the order in which the fields of the POJO are defined in the source code.

Flink accepts fields which are either public members or accessible via a getter and setter.
This makes it difficult to automatically define an order, esp. if some fields use getter and setter or public fields. Would the order depend on the field (which might not exist in case of getter/setter) or setter or getter methods (which might also not exist).
I'm also not sure if it is possible to extract the line number of a method or field via reflection.

Best, Fabian



2017-01-13 9:54 GMT+01:00 Hongyuhong <[hidden email]>:

Hi,

I’m now using streaming sql, And I have the sql like

select *  FROM OrderA where user > 2

the OrderA has 3 attr (user, product, amount)

and I expect the result is as the order like input, but it has been sorted by attr name,

and I found the order has already been sorted when call addSource,

What is the purpose of doing so?cause it’s a little not meet our requirements.

 

Thanks very much.

 

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

                   super(typeClass);

 

                   checkArgument(Modifier.isPublic(typeClass.getModifiers()),

                                     "POJO %s is not public", typeClass);

 

                   this.fields = fields.toArray(new PojoField[fields.size()]);

 

                   Arrays.sort(this.fields, new Comparator<PojoField>() {

                            @Override

                            public int compare(PojoField o1, PojoField o2) {

                                     return o1.getField().getName().compareTo(o2.getField().getName());

                            }

                   });

 

 

Best,

Yuhong
Hongyuhong
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: the attribute order in sql 'select * from...'

Hongyuhong
In reply to this post by Fabian Hueske-2

Hi Fabian,

 

Yes, OrderA is a table of POJO.

But what I consume is that in construct func PojoTypeInfo

The input param (fields)’s order is right, it ‘s change after the sort operation, and I’m wonder if the sort operation can be removed?

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

super(typeClass);

 checkArgument(Modifier.isPublic(typeClass.getModifiers()),"POJO %s is not public", typeClass);

 this.fields = fields.toArray(new PojoField[fields.size()]);

 Arrays.sort(this.fields, new Comparator<PojoField>() {

 @Override

 public int compare(PojoField o1, PojoField o2) {

  return o1.getField().getName().compareTo(o2.getField().getName());

 }

});

 

 

发件人: Fabian Hueske [mailto:[hidden email]]
发送时间: 2017年1月13日 17:23
收件人: [hidden email]
主题: Re: the attribute order in sql 'select * from...'

 

Hi Yuhong,

I assume that OrderA is a table of POJO objects and you are expecting the order of the attribute to be as the order in which the fields of the POJO are defined in the source code.

Flink accepts fields which are either public members or accessible via a getter and setter.

This makes it difficult to automatically define an order, esp. if some fields use getter and setter or public fields. Would the order depend on the field (which might not exist in case of getter/setter) or setter or getter methods (which might also not exist).

I'm also not sure if it is possible to extract the line number of a method or field via reflection.

Best, Fabian

 

 

2017-01-13 9:54 GMT+01:00 Hongyuhong <[hidden email]>:

Hi,

I’m now using streaming sql, And I have the sql like

select *  FROM OrderA where user > 2

the OrderA has 3 attr (user, product, amount)

and I expect the result is as the order like input, but it has been sorted by attr name,

and I found the order has already been sorted when call addSource,

What is the purpose of doing so?cause it’s a little not meet our requirements.

 

Thanks very much.

 

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

                   super(typeClass);

 

                   checkArgument(Modifier.isPublic(typeClass.getModifiers()),

                                     "POJO %s is not public", typeClass);

 

                   this.fields = fields.toArray(new PojoField[fields.size()]);

 

                   Arrays.sort(this.fields, new Comparator<PojoField>() {

                            @Override

                            public int compare(PojoField o1, PojoField o2) {

                                     return o1.getField().getName().compareTo(o2.getField().getName());

                            }

                   });

 

 

Best,

Yuhong

 
Fabian Hueske-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: the attribute order in sql 'select * from...'

Fabian Hueske-2
I think the sorting is done for consistency reasons, i.e., that all PojoTypeInfos for the same class behave the same.
Since this code is used in many parts of Flink and many jobs (DataSet, DataStream, etc.) I would be very careful to change the default behavior here.

Maybe we can add a constructor that does not sort the fields.

2017-01-13 10:46 GMT+01:00 Hongyuhong <[hidden email]>:

Hi Fabian,

 

Yes, OrderA is a table of POJO.

But what I consume is that in construct func PojoTypeInfo

The input param (fields)’s order is right, it ‘s change after the sort operation, and I’m wonder if the sort operation can be removed?

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

super(typeClass);

 checkArgument(Modifier.isPublic(typeClass.getModifiers()),"POJO %s is not public", typeClass);

 this.fields = fields.toArray(new PojoField[fields.size()]);

 Arrays.sort(this.fields, new Comparator<PojoField>() {

 @Override

 public int compare(PojoField o1, PojoField o2) {

  return o1.getField().getName().compareTo(o2.getField().getName());

 }

});

 

 

发件人: Fabian Hueske [mailto:[hidden email]]
发送时间: 2017年1月13日 17:23
收件人: [hidden email]
主题: Re: the attribute order in sql 'select * from...'

 

Hi Yuhong,

I assume that OrderA is a table of POJO objects and you are expecting the order of the attribute to be as the order in which the fields of the POJO are defined in the source code.

Flink accepts fields which are either public members or accessible via a getter and setter.

This makes it difficult to automatically define an order, esp. if some fields use getter and setter or public fields. Would the order depend on the field (which might not exist in case of getter/setter) or setter or getter methods (which might also not exist).

I'm also not sure if it is possible to extract the line number of a method or field via reflection.

Best, Fabian

 

 

2017-01-13 9:54 GMT+01:00 Hongyuhong <[hidden email]>:

Hi,

I’m now using streaming sql, And I have the sql like

select *  FROM OrderA where user > 2

the OrderA has 3 attr (user, product, amount)

and I expect the result is as the order like input, but it has been sorted by attr name,

and I found the order has already been sorted when call addSource,

What is the purpose of doing so?cause it’s a little not meet our requirements.

 

Thanks very much.

 

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

                   super(typeClass);

 

                   checkArgument(Modifier.isPublic(typeClass.getModifiers()),

                                     "POJO %s is not public", typeClass);

 

                   this.fields = fields.toArray(new PojoField[fields.size()]);

 

                   Arrays.sort(this.fields, new Comparator<PojoField>() {

                            @Override

                            public int compare(PojoField o1, PojoField o2) {

                                     return o1.getField().getName().compareTo(o2.getField().getName());

                            }

                   });

 

 

Best,

Yuhong

 
Timo Walther
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: the attribute order in sql 'select * from...'

Timo Walther
Especially, as it might also change the serialized binary format.


Am 13/01/17 um 11:24 schrieb Fabian Hueske:
I think the sorting is done for consistency reasons, i.e., that all PojoTypeInfos for the same class behave the same.
Since this code is used in many parts of Flink and many jobs (DataSet, DataStream, etc.) I would be very careful to change the default behavior here.

Maybe we can add a constructor that does not sort the fields.

2017-01-13 10:46 GMT+01:00 Hongyuhong <[hidden email]>:

Hi Fabian,

 

Yes, OrderA is a table of POJO.

But what I consume is that in construct func PojoTypeInfo

The input param (fields)’s order is right, it ‘s change after the sort operation, and I’m wonder if the sort operation can be removed?

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

super(typeClass);

 checkArgument(Modifier.isPublic(typeClass.getModifiers()),"POJO %s is not public", typeClass);

 this.fields = fields.toArray(new PojoField[fields.size()]);

 Arrays.sort(this.fields, new Comparator<PojoField>() {

 @Override

 public int compare(PojoField o1, PojoField o2) {

  return o1.getField().getName().compareTo(o2.getField().getName());

 }

});

 

 

发件人: Fabian Hueske [mailto:[hidden email]]
发送时间: 2017年1月13日 17:23
收件人: [hidden email]
主题: Re: the attribute order in sql 'select * from...'

 

Hi Yuhong,

I assume that OrderA is a table of POJO objects and you are expecting the order of the attribute to be as the order in which the fields of the POJO are defined in the source code.

Flink accepts fields which are either public members or accessible via a getter and setter.

This makes it difficult to automatically define an order, esp. if some fields use getter and setter or public fields. Would the order depend on the field (which might not exist in case of getter/setter) or setter or getter methods (which might also not exist).

I'm also not sure if it is possible to extract the line number of a method or field via reflection.

Best, Fabian

 

 

2017-01-13 9:54 GMT+01:00 Hongyuhong <[hidden email]>:

Hi,

I’m now using streaming sql, And I have the sql like

select *  FROM OrderA where user > 2

the OrderA has 3 attr (user, product, amount)

and I expect the result is as the order like input, but it has been sorted by attr name,

and I found the order has already been sorted when call addSource,

What is the purpose of doing so?cause it’s a little not meet our requirements.

 

Thanks very much.

 

 

public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {

                   super(typeClass);

 

                   checkArgument(Modifier.isPublic(typeClass.getModifiers()),

                                     "POJO %s is not public", typeClass);

 

                   this.fields = fields.toArray(new PojoField[fields.size()]);

 

                   Arrays.sort(this.fields, new Comparator<PojoField>() {

                            @Override

                            public int compare(PojoField o1, PojoField o2) {

                                     return o1.getField().getName().compareTo(o2.getField().getName());

                            }

                   });

 

 

Best,

Yuhong

 
Free forum by Nabble Edit this page