Re: SQL for Avro GenericRecords on Parquet

Posted by Peter Huang on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/SQL-for-Avro-GenericRecords-on-Parquet-tp31048p31053.html

Hi Hanan,

Thanks for reporting the issue. Would you please attach your test code here? I may help to investigate. 

Best Regards

Peter Huang

On Mon, Nov 18, 2019 at 2:51 AM Hanan Yehudai <[hidden email]> wrote:

I have tried to persist Generic Avro records in a parquet file and then read it via ParquetTablesource – using SQL.
Seems that the SQL I not executed properly !

The persisted records are :
Id  ,  type
3333333,Type1
222222,Type2
3333333,Type1
222222,Type2
3333333,Type1
222222,Type2
3333333,Type1
222222,Type2
3333333,Type1
222222,Type2
3333333,Type1
222222,Type2

While SQL  of SELECT id  ,recordType_  FROM ParquetTable  - return the above ( which is correct)
Running  : "SELECT id  ,recordType_  FROM ParquetTable  where recordType_='Type1' "
Will result in :
3333333,Type1
222222,Type1
3333333,Type1
222222,Type1
3333333,Type1
222222,Type1
3333333,Type1
222222,Type1
3333333,Type1
222222,Type1
3333333,Type1
222222,Type1

As if the equal sign is assignment and not equal …

am I doing something wrong ? is it an issue of Generic record vs SpecificRecords ?