FieldForwarding hints

[Newport, Billy]

For the following map, what would the hint be:

 

@ForwardedField(“f0->f1”) // Correct?

public class Tuplator extends FlatMapBase<GenericRecord, Tuple2<Void, GenericRecord>>

{

     /**

     *

      */

     private static final long serialVersionUID = 4443299154253252672L;

    

     public Tuplator(SerializableAvroRecordBuilder avroRecordBuilder)

     {

           super(avroRecordBuilder);

     }

 

     @Override

     public void flatMap(GenericRecord value, Collector<Tuple2<Void, GenericRecord>> out) throws Exception

     {

           out.collect(new Tuple2<Void, GenericRecord>(null, value));

     }

}

 

Given the input is not a tuple, can I even still use hints?

 

 

Billy Newport

Data Architecture, Goldman Sachs & Co.
30 Hudson | 37th Floor | Jersey City, NJ

Tel:  [phone] |  Cell:  [phone]
Email: [hidden email], KD2DKQ

 

[Fabian Hueske-2]

The correct annotation would be:

@ForwardedField("*->f1")

The asterisk / wildcard addresses the complete input type. The DataSet API also performs a type-based validation. If the types of the fields on the left and right are not correct, it should fail.

Best, Fabian

2017-02-07 23:13 GMT+01:00 Newport, Billy <[hidden email]>:

For the following map, what would the hint be:

 

@ForwardedField(“f0->f1”) // Correct?

public class Tuplator extends FlatMapBase<GenericRecord, Tuple2<Void, GenericRecord>>

{

     /**

     *

      */

     private static final long serialVersionUID = 4443299154253252672L;

    

     public Tuplator(SerializableAvroRecordBuilder avroRecordBuilder)

     {

           super(avroRecordBuilder);

     }

 

     @Override

     public void flatMap(GenericRecord value, Collector<Tuple2<Void, GenericRecord>> out) throws Exception

     {

           out.collect(new Tuple2<Void, GenericRecord>(null, value));

     }

}

 

Given the input is not a tuple, can I even still use hints?

 

 

Billy Newport

Data Architecture, Goldman Sachs & Co.
30 Hudson | 37th Floor | Jersey City, NJ

Tel:  <a href="tel:(212)%20855-7773" value="[phone]" target="_blank">[phone] |  Cell:  <a href="tel:(507)%20254-0134" value="[phone]" target="_blank">[phone]
Email: [hidden email], KD2DKQ