Dot notation not working for accessing case classes nested fields

[Federico D'Ambrosio]

Hi,

I have the following case classes:

case class Event(instantValues: InstantValues)

case class InstantValues(speed: Int, altitude: Int, time: DateTime)

in a DataStream[Event] I'd like to perform a maxBy operation on the field time of instantValue for each event and according to the docs here it would be possible to use the dot notation such the following:

val events = stream
  .keyBy("otherField")
  .window(TumblingEventTimeWindows.of(Time.seconds(5)))
  .maxBy("instantValues.time")




positionToMaxBy - In case of a POJO, Scala case class, or Tuple type, the name of the public) field on which to perform the aggregation. Additionally, a dot can be used to drill down into nested objects, as in "field1.fieldxy" . Furthermore "*" can be specified in case of a basic type (which is considered as having only one field).

Still, I'm getting the following error:

Fields 'instantValues.time' are not valid for 'package.Event(instantValues: package.InstantValues(speed: Integer, altitude: Integer, time: GenericType<org.joda.time.DateTime>))'

whereas if, for instance, use only "instantValues" (while implementing its compareTo method) the aggregation works as usual.

Any idea as to why this isn't working? Am I doing something wrong?

Thanks a lot,

Federico

[Gábor Gévay]

Hi Federico,

Sorry, nested field expressions are not supported in these methods at
the moment. I have created a JIRA issue for this:
https://issues.apache.org/jira/browse/FLINK-7629
I think this should be easy to fix, as all the infrastructure for
supporting this is already in place. I'll try to do it over the
weekend.

Best,
Gábor




On Thu, Sep 14, 2017 at 3:51 PM, Federico D'Ambrosio
<[hidden email]> wrote:

> Hi,
>
> I have the following case classes:
>
> case class Event(instantValues: InstantValues)
> case class InstantValues(speed: Int, altitude: Int, time: DateTime)
>
>
> in a DataStream[Event] I'd like to perform a maxBy operation on the field
> time of instantValue for each event and according to the docs here it would
> be possible to use the dot notation such the following:
>
> val events = stream
>   .keyBy("otherField")
>   .window(TumblingEventTimeWindows.of(Time.seconds(5)))
>   .maxBy("instantValues.time")
>
> positionToMaxBy - In case of a POJO, Scala case class, or Tuple type, the
> name of the public) field on which to perform the aggregation. Additionally,
> a dot can be used to drill down into nested objects, as in "field1.fieldxy"
> . Furthermore "*" can be specified in case of a basic type (which is
> considered as having only one field).
>
>
> Still, I'm getting the following error:
>
> Fields 'instantValues.time' are not valid for 'package.Event(instantValues:
> package.InstantValues(speed: Integer, altitude: Integer, time:
> GenericType<org.joda.time.DateTime>))'
>
> whereas if, for instance, use only "instantValues" (while implementing its
> compareTo method) the aggregation works as usual.
>
> Any idea as to why this isn't working? Am I doing something wrong?
>
> Thanks a lot,
> Federico

[Federico D'Ambrosio]

Great, thanks!

The fact that it's actually written on the documentation is really misleading.

Thank you very much for your response

Il 15 set 2017 13:26, "Gábor Gévay" <[hidden email]> ha scritto:

Hi Federico,

Sorry, nested field expressions are not supported in these methods at
the moment. I have created a JIRA issue for this:
https://issues.apache.org/jira/browse/FLINK-7629
I think this should be easy to fix, as all the infrastructure for
supporting this is already in place. I'll try to do it over the
weekend.

Best,
Gábor




On Thu, Sep 14, 2017 at 3:51 PM, Federico D'Ambrosio
<[hidden email]> wrote:
> Hi,
>
> I have the following case classes:
>
> case class Event(instantValues: InstantValues)
> case class InstantValues(speed: Int, altitude: Int, time: DateTime)
>
>
> in a DataStream[Event] I'd like to perform a maxBy operation on the field
> time of instantValue for each event and according to the docs here it would
> be possible to use the dot notation such the following:
>
> val events = stream
>   .keyBy("otherField")
>   .window(TumblingEventTimeWindows.of(Time.seconds(5)))
>   .maxBy("instantValues.time")
>
> positionToMaxBy - In case of a POJO, Scala case class, or Tuple type, the
> name of the public) field on which to perform the aggregation. Additionally,
> a dot can be used to drill down into nested objects, as in "field1.fieldxy"
> . Furthermore "*" can be specified in case of a basic type (which is
> considered as having only one field).
>
>
> Still, I'm getting the following error:
>
> Fields 'instantValues.time' are not valid for 'package.Event(instantValues:
> package.InstantValues(speed: Integer, altitude: Integer, time:
> GenericType<org.joda.time.DateTime>))'
>
> whereas if, for instance, use only "instantValues" (while implementing its
> compareTo method) the aggregation works as usual.
>
> Any idea as to why this isn't working? Am I doing something wrong?
>
> Thanks a lot,
> Federico