Scala Table API with Java POJO

[Dong-iL, Kim]

I’ve create a program using table API and get an exception like this.
org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
There is an error not in java program, but in scala program.
how can I use java POJO with scala Table API.

[Dong-iL, Kim]

my flink ver is 1.0.3.
thanks.

> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <[hidden email]> wrote:
>
> I’ve create a program using table API and get an exception like this.
> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
> There is an error not in java program, but in scala program.
> how can I use java POJO with scala Table API.
>

[Timo Walther]

Hi Kim,

as the exception says: POJOs have no deterministic field order. You have
to specify the order during the DataSet to Table conversion:

Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a,
pojoField2 as b");

I hope that helps. Otherwise it would help if you could supply a code
snippet of your program.

Timo

Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:

> my flink ver is 1.0.3.
> thanks.
>
>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <[hidden email]> wrote:
>>
>> I’ve create a program using table API and get an exception like this.
>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>> There is an error not in java program, but in scala program.
>> how can I use java POJO with scala Table API.
>>

--
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

[Dong-iL, Kim]

Hi Timo.
I’m using scala API.
There is no error with java API.
my code snippet is this.

dataSet.toTable
                .groupBy(“id")
                .select(‘id, ‘amount.sum as ‘amount)
                .where(‘amount > 0)
                .toDataSet[TestPojo]
                .print()

Thanks.

> On Aug 1, 2016, at 5:50 PM, Timo Walther <[hidden email]> wrote:
>
> Hi Kim,
>
> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>
> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>
> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>
> Timo
>
> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>> my flink ver is 1.0.3.
>> thanks.
>>
>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <[hidden email]> wrote:
>>>
>>> I’ve create a program using table API and get an exception like this.
>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>> There is an error not in java program, but in scala program.
>>> how can I use java POJO with scala Table API.
>>>
>
>
> --
> Freundliche Grüße / Kind Regards
>
> Timo Walther
>
> Follow me: @twalthr
> https://www.linkedin.com/in/twalthr
>

[Timo Walther]

I think you need to use ".as()" instead of "toTable()" to supply the
field order.

Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:

> Hi Timo.
> I’m using scala API.
> There is no error with java API.
> my code snippet is this.
>
> dataSet.toTable
>                  .groupBy(“id")
>                  .select(‘id, ‘amount.sum as ‘amount)
>                  .where(‘amount > 0)
>                  .toDataSet[TestPojo]
>                  .print()
>
> Thanks.
>
>> On Aug 1, 2016, at 5:50 PM, Timo Walther <[hidden email]> wrote:
>>
>> Hi Kim,
>>
>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>
>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>
>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>
>> Timo
>>
>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>> my flink ver is 1.0.3.
>>> thanks.
>>>
>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <[hidden email]> wrote:
>>>>
>>>> I’ve create a program using table API and get an exception like this.
>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>>> There is an error not in java program, but in scala program.
>>>> how can I use java POJO with scala Table API.
>>>>
>>
>> --
>> Freundliche Grüße / Kind Regards
>>
>> Timo Walther
>>
>> Follow me: @twalthr
>> https://www.linkedin.com/in/twalthr
>>

--
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

[Dong-iL, Kim]

I’ve tried like this, but not work.

dataSet.as(‘id as ‘id, ‘amount as ‘amount)

dataSet.as(‘id, ‘amount)

dataSet.as(“id, amount”)

thanks.

> On Aug 1, 2016, at 6:03 PM, Timo Walther <[hidden email]> wrote:
>
> I think you need to use ".as()" instead of "toTable()" to supply the field order.
>
> Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:
>> Hi Timo.
>> I’m using scala API.
>> There is no error with java API.
>> my code snippet is this.
>>
>> dataSet.toTable
>>                 .groupBy(“id")
>>                 .select(‘id, ‘amount.sum as ‘amount)
>>                 .where(‘amount > 0)
>>                 .toDataSet[TestPojo]
>>                 .print()
>>
>> Thanks.
>>
>>> On Aug 1, 2016, at 5:50 PM, Timo Walther <[hidden email]> wrote:
>>>
>>> Hi Kim,
>>>
>>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>>
>>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>>
>>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>>
>>> Timo
>>>
>>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>>> my flink ver is 1.0.3.
>>>> thanks.
>>>>
>>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <[hidden email]> wrote:
>>>>>
>>>>> I’ve create a program using table API and get an exception like this.
>>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>>>> There is an error not in java program, but in scala program.
>>>>> how can I use java POJO with scala Table API.
>>>>>
>>>
>>> --
>>> Freundliche Grüße / Kind Regards
>>>
>>> Timo Walther
>>>
>>> Follow me: @twalthr
>>> https://www.linkedin.com/in/twalthr
>>>
>
>
> --
> Freundliche Grüße / Kind Regards
>
> Timo Walther
>
> Follow me: @twalthr
> https://www.linkedin.com/in/twalthr
>

[Timo Walther]

Ok, then I think I have no better solution than use the Table API of the
upcoming 1.1 release. The Table API has been completely rewritten and
the POJO support is now much better. Maybe you could try the recent 1.1
RC1 release.

Am 01/08/16 um 11:11 schrieb Dong-iL, Kim:

> I’ve tried like this, but not work.
>
> dataSet.as(‘id as ‘id, ‘amount as ‘amount)
>
> dataSet.as(‘id, ‘amount)
>
> dataSet.as(“id, amount”)
>
> thanks.
>
>> On Aug 1, 2016, at 6:03 PM, Timo Walther <[hidden email]> wrote:
>>
>> I think you need to use ".as()" instead of "toTable()" to supply the field order.
>>
>> Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:
>>> Hi Timo.
>>> I’m using scala API.
>>> There is no error with java API.
>>> my code snippet is this.
>>>
>>> dataSet.toTable
>>>                  .groupBy(“id")
>>>                  .select(‘id, ‘amount.sum as ‘amount)
>>>                  .where(‘amount > 0)
>>>                  .toDataSet[TestPojo]
>>>                  .print()
>>>
>>> Thanks.
>>>
>>>> On Aug 1, 2016, at 5:50 PM, Timo Walther <[hidden email]> wrote:
>>>>
>>>> Hi Kim,
>>>>
>>>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>>>
>>>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>>>
>>>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>>>
>>>> Timo
>>>>
>>>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>>>> my flink ver is 1.0.3.
>>>>> thanks.
>>>>>
>>>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <[hidden email]> wrote:
>>>>>>
>>>>>> I’ve create a program using table API and get an exception like this.
>>>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>>>>> There is an error not in java program, but in scala program.
>>>>>> how can I use java POJO with scala Table API.
>>>>>>
>>>> --
>>>> Freundliche Grüße / Kind Regards
>>>>
>>>> Timo Walther
>>>>
>>>> Follow me: @twalthr
>>>> https://www.linkedin.com/in/twalthr
>>>>
>>
>> --
>> Freundliche Grüße / Kind Regards
>>
>> Timo Walther
>>
>> Follow me: @twalthr
>> https://www.linkedin.com/in/twalthr
>>

--
Freundliche Grüße / Kind Regards

Timo Walther

Follow me: @twalthr
https://www.linkedin.com/in/twalthr

[Dong-iL, Kim]

in org.apache.flink.api.table.plan.PlanTranslator.

val inputType = set.getType().asInstanceOf[CompositeType[A]]

if (!inputType.hasDeterministicFieldOrder && checkDeterministicFields) {
      throw new ExpressionException(s"You cannot rename fields upon Table creation: " +
        s"Field order of input type $inputType is not deterministic." )
    }

when A is a PojoType, hasDeterministicFieldOrder always returns false.

what shall I do using Pojo.
Thanks.

> On Aug 1, 2016, at 6:11 PM, Dong-iL, Kim <[hidden email]> wrote:
>
> I’ve tried like this, but not work.
>
> dataSet.as(‘id as ‘id, ‘amount as ‘amount)
>
> dataSet.as(‘id, ‘amount)
>
> dataSet.as(“id, amount”)
>
> thanks.
>
>> On Aug 1, 2016, at 6:03 PM, Timo Walther <[hidden email]> wrote:
>>
>> I think you need to use ".as()" instead of "toTable()" to supply the field order.
>>
>> Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:
>>> Hi Timo.
>>> I’m using scala API.
>>> There is no error with java API.
>>> my code snippet is this.
>>>
>>> dataSet.toTable
>>>                .groupBy(“id")
>>>                .select(‘id, ‘amount.sum as ‘amount)
>>>                .where(‘amount > 0)
>>>                .toDataSet[TestPojo]
>>>                .print()
>>>
>>> Thanks.
>>>
>>>> On Aug 1, 2016, at 5:50 PM, Timo Walther <[hidden email]> wrote:
>>>>
>>>> Hi Kim,
>>>>
>>>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>>>
>>>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>>>
>>>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>>>
>>>> Timo
>>>>
>>>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>>>> my flink ver is 1.0.3.
>>>>> thanks.
>>>>>
>>>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <[hidden email]> wrote:
>>>>>>
>>>>>> I’ve create a program using table API and get an exception like this.
>>>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>>>>> There is an error not in java program, but in scala program.
>>>>>> how can I use java POJO with scala Table API.
>>>>>>
>>>>
>>>> --
>>>> Freundliche Grüße / Kind Regards
>>>>
>>>> Timo Walther
>>>>
>>>> Follow me: @twalthr
>>>> https://www.linkedin.com/in/twalthr
>>>>
>>
>>
>> --
>> Freundliche Grüße / Kind Regards
>>
>> Timo Walther
>>
>> Follow me: @twalthr
>> https://www.linkedin.com/in/twalthr
>>
>