Calcite SQL Map to Pojo Map

[shkob1]

Im trying to convert a SQL query that has a select map[..] into a pojo with
Map (using tableEnv.toRestractedStream )
It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
GenericType<java.util.Map> while the field type itself is MapTypeInfo with
Map<String,String>


Exception in thread "main" org.apache.flink.table.api.TableException: Result
field does not match requested type. Requested: GenericType<java.util.Map>;
Actual: Map<String, String>

Any suggestion?
Shahar



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Rong Rong]

If your conversion is done using a UDF you need to override the getResultType method [1] to explicitly specify the key and value type information. As generic erasure will not preseve the <String,String> part of your code.

Thanks,

Rong

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/udfs.html#scalar-functions

On Wed, Mar 27, 2019 at 10:14 AM shkob1 <[hidden email]> wrote:

Im trying to convert a SQL query that has a select map[..] into a pojo with
Map (using tableEnv.toRestractedStream )
It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
GenericType<java.util.Map> while the field type itself is MapTypeInfo with
Map<String,String>


Exception in thread "main" org.apache.flink.table.api.TableException: Result
field does not match requested type. Requested: GenericType<java.util.Map>;
Actual: Map<String, String>

Any suggestion?
Shahar



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[shkob1]

Hey Rong,

I don't think this is about a UDF, i reproduce the same exception with a simple map['a','b'] where the Pojo has a Map<String,String> property

btw for the UDF i'm already doing it (clazz is based on the specific map im creating):

@Override
public TypeInformation<?> getResultType(Class<?>[] signature) {
    return Types.MAP(Types.STRING, TypeInformation.of(clazz));
}

The table schema looks good but looking at the PojoTypeInfo fields the Map field is a GenericType<Map> - this causes the exception to be thrown on TableEnvironment.generateRowConverterFunction

On Thu, Mar 28, 2019 at 8:56 AM Rong Rong <[hidden email]> wrote:

If your conversion is done using a UDF you need to override the getResultType method [1] to explicitly specify the key and value type information. As generic erasure will not preseve the <String,String> part of your code.

Thanks,

Rong

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/udfs.html#scalar-functions

On Wed, Mar 27, 2019 at 10:14 AM shkob1 <[hidden email]> wrote:

Im trying to convert a SQL query that has a select map[..] into a pojo with
Map (using tableEnv.toRestractedStream )
It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
GenericType<java.util.Map> while the field type itself is MapTypeInfo with
Map<String,String>


Exception in thread "main" org.apache.flink.table.api.TableException: Result
field does not match requested type. Requested: GenericType<java.util.Map>;
Actual: Map<String, String>

Any suggestion?
Shahar



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[shkob1]

Based on this discussion http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/HashMap-HashSet-Serialization-Issue-td10899.html this seems by design that HashMap/Map are handled as GenericTypes .

However that doesn't work with the query result table schema which generates a Map type.

On Thu, Mar 28, 2019 at 10:32 AM Shahar Cizer Kobrinsky <[hidden email]> wrote:

Hey Rong,

I don't think this is about a UDF, i reproduce the same exception with a simple map['a','b'] where the Pojo has a Map<String,String> property

btw for the UDF i'm already doing it (clazz is based on the specific map im creating):

@Override
public TypeInformation<?> getResultType(Class<?>[] signature) {
    return Types.MAP(Types.STRING, TypeInformation.of(clazz));
}

The table schema looks good but looking at the PojoTypeInfo fields the Map field is a GenericType<Map> - this causes the exception to be thrown on TableEnvironment.generateRowConverterFunction

On Thu, Mar 28, 2019 at 8:56 AM Rong Rong <[hidden email]> wrote:

If your conversion is done using a UDF you need to override the getResultType method [1] to explicitly specify the key and value type information. As generic erasure will not preseve the <String,String> part of your code.

Thanks,

Rong

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/table/udfs.html#scalar-functions

On Wed, Mar 27, 2019 at 10:14 AM shkob1 <[hidden email]> wrote:

Im trying to convert a SQL query that has a select map[..] into a pojo with
Map (using tableEnv.toRestractedStream )
It seems to fail when the field requestedTypeInfo is GenericTypeInfo with
GenericType<java.util.Map> while the field type itself is MapTypeInfo with
Map<String,String>


Exception in thread "main" org.apache.flink.table.api.TableException: Result
field does not match requested type. Requested: GenericType<java.util.Map>;
Actual: Map<String, String>

Any suggestion?
Shahar



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[shkob1]

Apparently the solution is to force map creating using UDF and to have the
UDF return Types.GENERIC(Map.class)
That makes them compatible and treated both as GenericType

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

[Rong Rong]

I think the proper solution should not be Types.GENERIC(Map.class) as you will not be able to do any success processing with the return object. 

For example, Map['k', 'v'].get('k') will not work. 

I think there might be some problem like you suggested that they are handled as GenericType instead of Pojo type, so it is not utilizing the correct serializer. 

It would be great if you can share the complete code that generates the exception.

--

Rong

On Thu, Mar 28, 2019 at 1:56 PM shkob1 <[hidden email]> wrote:

Apparently the solution is to force map creating using UDF and to have the
UDF return Types.GENERIC(Map.class)
That makes them compatible and treated both as GenericType

Thanks!



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/