(DEPRECATED) Apache Flink User Mailing List archive.

Problem with Summerization

classic Classic list List threaded Threaded
♦
♦
 5 messages Options
rostami
Reply | Threaded
Open this post in threaded view
♦
♦
|

Problem with Summerization

rostami
Dear All,

I do not understand what the error in the following code can be?

Graph<GradoopId, Long, NullValue> gellyGraph = ...

Graph<GradoopId,Summarization.VertexValue<Long>,  
Summarization.EdgeValue<NullValue>> g =
                             gellyGraph.run(new  
Summarization<GradoopId, Long, NullValue>());

g.getVertices().print(); //this one works fine
g.getEdges().print();    //this one gives the following error

...
org.apache.flink.types.NullFieldException: Field 2 is null, but  
expected to hold a value.
        at  
org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:126)
        at  
org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:30)
        at  
org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:56)
        at  
org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:83)
        at  
org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:85)
        at  
org.apache.flink.runtime.operators.shipping.OutputCollector.collect(OutputCollector.java:65)
        at  
org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
        at  
org.apache.flink.graph.library.Summarization$VertexGroupReducer.reduce(Summarization.java:318)
        at  
org.apache.flink.runtime.operators.GroupReduceDriver.run(GroupReduceDriver.java:131)
        at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:486)
        at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:351)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
        at java.lang.Thread.run(Thread.java:748)

I also tried with different types for EdgeValue. The same error appears.
When I look at the result vertices, everything looks fine.
But the result edges has this problem.

Any idea?

Regards,
Ali

Greg Hogan
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Problem with Summerization

Greg Hogan
Hi Ali,

Could you print and include a gellyGraph which results in this error.

Greg


> On Jun 27, 2017, at 2:48 PM, [hidden email] wrote:
>
> Dear All,
>
> I do not understand what the error in the following code can be?
>
> Graph<GradoopId, Long, NullValue> gellyGraph = ...
>
> Graph<GradoopId,Summarization.VertexValue<Long>, Summarization.EdgeValue<NullValue>> g =
>                            gellyGraph.run(new Summarization<GradoopId, Long, NullValue>());
>
> g.getVertices().print(); //this one works fine
> g.getEdges().print();    //this one gives the following error
>
> ...
> org.apache.flink.types.NullFieldException: Field 2 is null, but expected to hold a value.
> at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:126)
> at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:30)
> at org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:56)
> at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:83)
> at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:85)
> at org.apache.flink.runtime.operators.shipping.OutputCollector.collect(OutputCollector.java:65)
> at org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
> at org.apache.flink.graph.library.Summarization$VertexGroupReducer.reduce(Summarization.java:318)
> at org.apache.flink.runtime.operators.GroupReduceDriver.run(GroupReduceDriver.java:131)
> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:486)
> at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:351)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
> at java.lang.Thread.run(Thread.java:748)
>
> I also tried with different types for EdgeValue. The same error appears.
> When I look at the result vertices, everything looks fine.
> But the result edges has this problem.
>
> Any idea?
>
> Regards,
> Ali
rostami
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Problem with Summerization

rostami
Hi,

these are the outputs for a small test graph:

gellyGraph.getVertices().print();
(000000000000000000000003,9)
(000000000000000000000007,9)
(000000000000000000000005,5)
(000000000000000000000001,9)
(000000000000000000000006,9)
(000000000000000000000000,9)
(000000000000000000000004,9)
(000000000000000000000002,2)
(000000000000000000000008,8)
(000000000000000000000009,9)
(00000000000000000000000a,10)


gellyGraph.getEdges().print();
(000000000000000000000002,000000000000000000000001,(null))
(000000000000000000000006,000000000000000000000003,(null))
(000000000000000000000000,000000000000000000000001,(null))
(000000000000000000000008,000000000000000000000003,(null))
(000000000000000000000005,000000000000000000000006,(null))
(000000000000000000000009,000000000000000000000000,(null))
(000000000000000000000009,00000000000000000000000a,(null))
(000000000000000000000005,00000000000000000000000a,(null))
(000000000000000000000009,000000000000000000000004,(null))
(000000000000000000000005,000000000000000000000006,(null))
(000000000000000000000005,000000000000000000000007,(null))
(000000000000000000000009,000000000000000000000000,(null))
(000000000000000000000006,000000000000000000000007,(null))
(000000000000000000000004,000000000000000000000007,(null))
(000000000000000000000008,000000000000000000000007,(null))
(000000000000000000000002,000000000000000000000000,(null))
(000000000000000000000009,000000000000000000000001,(null))
(000000000000000000000005,000000000000000000000003,(null))
(000000000000000000000008,000000000000000000000006,(null))
(000000000000000000000000,000000000000000000000004,(null))
(000000000000000000000002,000000000000000000000004,(null))
(000000000000000000000007,000000000000000000000004,(null))
(000000000000000000000007,000000000000000000000006,(null))
(000000000000000000000004,000000000000000000000000,(null))

g.getVertices().print(); //correct results
(000000000000000000000005,(5,1))
(000000000000000000000002,(2,1))
(000000000000000000000008,(8,1))
(000000000000000000000004,(9,7))
(00000000000000000000000a,(10,1))

g.getEdges.print(); //error!!

As I said, I also tested when edges have values other than null
and the same problem appears.

Regards,
Ali

Quoting Greg Hogan <[hidden email]>:

> Hi Ali,
>
> Could you print and include a gellyGraph which results in this error.
>
> Greg
>
>
>> On Jun 27, 2017, at 2:48 PM, [hidden email] wrote:
>>
>> Dear All,
>>
>> I do not understand what the error in the following code can be?
>>
>> Graph<GradoopId, Long, NullValue> gellyGraph = ...
>>
>> Graph<GradoopId,Summarization.VertexValue<Long>,  
>> Summarization.EdgeValue<NullValue>> g =
>>                            gellyGraph.run(new  
>> Summarization<GradoopId, Long, NullValue>());
>>
>> g.getVertices().print(); //this one works fine
>> g.getEdges().print();    //this one gives the following error
>>
>> ...
>> org.apache.flink.types.NullFieldException: Field 2 is null, but  
>> expected to hold a value.
>> at  
>> org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:126)
>> at  
>> org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:30)
>> at  
>> org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:56)
>> at  
>> org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:83)
>> at  
>> org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:85)
>> at  
>> org.apache.flink.runtime.operators.shipping.OutputCollector.collect(OutputCollector.java:65)
>> at  
>> org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
>> at  
>> org.apache.flink.graph.library.Summarization$VertexGroupReducer.reduce(Summarization.java:318)
>> at  
>> org.apache.flink.runtime.operators.GroupReduceDriver.run(GroupReduceDriver.java:131)
>> at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:486)
>> at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:351)
>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
>> at java.lang.Thread.run(Thread.java:748)
>>
>> I also tried with different types for EdgeValue. The same error appears.
>> When I look at the result vertices, everything looks fine.
>> But the result edges has this problem.
>>
>> Any idea?
>>
>> Regards,
>> Ali



Greg Hogan
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Problem with Summerization

Greg Hogan
The stacktrace indicates this may be a bug that was fixed for Flink 1.2:

    [FLINK-4624] [gelly] Support null values in Graph Summarization
   
    * Bug was caused by serializers that cannot handle null values (e.g. Long)
    * VertexGroupItem now uses Either<NullValue, VV> instead of VV
    * Generalized test cases
    * Added tests for vertex/edge values of type Long
    * Replaced Guava Lists.newArrayList() with new ArrayList<>()


On Wed, Jun 28, 2017 at 2:22 AM, <[hidden email]> wrote:
Hi,

these are the outputs for a small test graph:

gellyGraph.getVertices().print();
(000000000000000000000003,9)
(000000000000000000000007,9)
(000000000000000000000005,5)
(000000000000000000000001,9)
(000000000000000000000006,9)
(000000000000000000000000,9)
(000000000000000000000004,9)
(000000000000000000000002,2)
(000000000000000000000008,8)
(000000000000000000000009,9)
(00000000000000000000000a,10)


gellyGraph.getEdges().print();
(000000000000000000000002,000000000000000000000001,(null))
(000000000000000000000006,000000000000000000000003,(null))
(000000000000000000000000,000000000000000000000001,(null))
(000000000000000000000008,000000000000000000000003,(null))
(000000000000000000000005,000000000000000000000006,(null))
(000000000000000000000009,000000000000000000000000,(null))
(000000000000000000000009,00000000000000000000000a,(null))
(000000000000000000000005,00000000000000000000000a,(null))
(000000000000000000000009,000000000000000000000004,(null))
(000000000000000000000005,000000000000000000000006,(null))
(000000000000000000000005,000000000000000000000007,(null))
(000000000000000000000009,000000000000000000000000,(null))
(000000000000000000000006,000000000000000000000007,(null))
(000000000000000000000004,000000000000000000000007,(null))
(000000000000000000000008,000000000000000000000007,(null))
(000000000000000000000002,000000000000000000000000,(null))
(000000000000000000000009,000000000000000000000001,(null))
(000000000000000000000005,000000000000000000000003,(null))
(000000000000000000000008,000000000000000000000006,(null))
(000000000000000000000000,000000000000000000000004,(null))
(000000000000000000000002,000000000000000000000004,(null))
(000000000000000000000007,000000000000000000000004,(null))
(000000000000000000000007,000000000000000000000006,(null))
(000000000000000000000004,000000000000000000000000,(null))

g.getVertices().print(); //correct results
(000000000000000000000005,(5,1))
(000000000000000000000002,(2,1))
(000000000000000000000008,(8,1))
(000000000000000000000004,(9,7))
(00000000000000000000000a,(10,1))

g.getEdges.print(); //error!!

As I said, I also tested when edges have values other than null
and the same problem appears.

Regards,
Ali


Quoting Greg Hogan <[hidden email]>:

Hi Ali,

Could you print and include a gellyGraph which results in this error.

Greg


On Jun 27, 2017, at 2:48 PM, [hidden email] wrote:

Dear All,

I do not understand what the error in the following code can be?

Graph<GradoopId, Long, NullValue> gellyGraph = ...

Graph<GradoopId,Summarization.VertexValue<Long>, Summarization.EdgeValue<NullValue>> g =
                           gellyGraph.run(new Summarization<GradoopId, Long, NullValue>());

g.getVertices().print(); //this one works fine
g.getEdges().print();    //this one gives the following error

...
org.apache.flink.types.NullFieldException: Field 2 is null, but expected to hold a value.
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:126)
        at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:30)
        at org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:56)
        at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:83)
        at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:85)
        at org.apache.flink.runtime.operators.shipping.OutputCollector.collect(OutputCollector.java:65)
        at org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
        at org.apache.flink.graph.library.Summarization$VertexGroupReducer.reduce(Summarization.java:318)
        at org.apache.flink.runtime.operators.GroupReduceDriver.run(GroupReduceDriver.java:131)
        at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:486)
        at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:351)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
        at java.lang.Thread.run(Thread.java:748)

I also tried with different types for EdgeValue. The same error appears.
When I look at the result vertices, everything looks fine.
But the result edges has this problem.

Any idea?

Regards,
Ali
rostami
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Problem with Summerization

rostami
Exactly.
Updating flink solved my problem.
Thanks.

Quoting Greg Hogan <[hidden email]>:

> The stacktrace indicates this may be a bug that was fixed for Flink 1.2:
>
>     [FLINK-4624] [gelly] Support null values in Graph Summarization
>
>     * Bug was caused by serializers that cannot handle null values (e.g.
> Long)
>     * VertexGroupItem now uses Either<NullValue, VV> instead of VV
>     * Generalized test cases
>     * Added tests for vertex/edge values of type Long
>     * Replaced Guava Lists.newArrayList() with new ArrayList<>()
>
>
> On Wed, Jun 28, 2017 at 2:22 AM, <[hidden email]> wrote:
>
>> Hi,
>>
>> these are the outputs for a small test graph:
>>
>> gellyGraph.getVertices().print();
>> (000000000000000000000003,9)
>> (000000000000000000000007,9)
>> (000000000000000000000005,5)
>> (000000000000000000000001,9)
>> (000000000000000000000006,9)
>> (000000000000000000000000,9)
>> (000000000000000000000004,9)
>> (000000000000000000000002,2)
>> (000000000000000000000008,8)
>> (000000000000000000000009,9)
>> (00000000000000000000000a,10)
>>
>>
>> gellyGraph.getEdges().print();
>> (000000000000000000000002,000000000000000000000001,(null))
>> (000000000000000000000006,000000000000000000000003,(null))
>> (000000000000000000000000,000000000000000000000001,(null))
>> (000000000000000000000008,000000000000000000000003,(null))
>> (000000000000000000000005,000000000000000000000006,(null))
>> (000000000000000000000009,000000000000000000000000,(null))
>> (000000000000000000000009,00000000000000000000000a,(null))
>> (000000000000000000000005,00000000000000000000000a,(null))
>> (000000000000000000000009,000000000000000000000004,(null))
>> (000000000000000000000005,000000000000000000000006,(null))
>> (000000000000000000000005,000000000000000000000007,(null))
>> (000000000000000000000009,000000000000000000000000,(null))
>> (000000000000000000000006,000000000000000000000007,(null))
>> (000000000000000000000004,000000000000000000000007,(null))
>> (000000000000000000000008,000000000000000000000007,(null))
>> (000000000000000000000002,000000000000000000000000,(null))
>> (000000000000000000000009,000000000000000000000001,(null))
>> (000000000000000000000005,000000000000000000000003,(null))
>> (000000000000000000000008,000000000000000000000006,(null))
>> (000000000000000000000000,000000000000000000000004,(null))
>> (000000000000000000000002,000000000000000000000004,(null))
>> (000000000000000000000007,000000000000000000000004,(null))
>> (000000000000000000000007,000000000000000000000006,(null))
>> (000000000000000000000004,000000000000000000000000,(null))
>>
>> g.getVertices().print(); //correct results
>> (000000000000000000000005,(5,1))
>> (000000000000000000000002,(2,1))
>> (000000000000000000000008,(8,1))
>> (000000000000000000000004,(9,7))
>> (00000000000000000000000a,(10,1))
>>
>> g.getEdges.print(); //error!!
>>
>> As I said, I also tested when edges have values other than null
>> and the same problem appears.
>>
>> Regards,
>> Ali
>>
>>
>> Quoting Greg Hogan <[hidden email]>:
>>
>> Hi Ali,
>>>
>>> Could you print and include a gellyGraph which results in this error.
>>>
>>> Greg
>>>
>>>
>>> On Jun 27, 2017, at 2:48 PM, [hidden email] wrote:
>>>>
>>>> Dear All,
>>>>
>>>> I do not understand what the error in the following code can be?
>>>>
>>>> Graph<GradoopId, Long, NullValue> gellyGraph = ...
>>>>
>>>> Graph<GradoopId,Summarization.VertexValue<Long>,
>>>> Summarization.EdgeValue<NullValue>> g =
>>>>                            gellyGraph.run(new Summarization<GradoopId,
>>>> Long, NullValue>());
>>>>
>>>> g.getVertices().print(); //this one works fine
>>>> g.getEdges().print();    //this one gives the following error
>>>>
>>>> ...
>>>> org.apache.flink.types.NullFieldException: Field 2 is null, but
>>>> expected to hold a value.
>>>>         at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.
>>>> serialize(TupleSerializer.java:126)
>>>>         at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.
>>>> serialize(TupleSerializer.java:30)
>>>>         at org.apache.flink.runtime.plugable.SerializationDelegate.
>>>> write(SerializationDelegate.java:56)
>>>>         at org.apache.flink.runtime.io.network.api.serialization.Spanni
>>>> ngRecordSerializer.addRecord(SpanningRecordSerializer.java:83)
>>>>         at org.apache.flink.runtime.io.network.api.writer.RecordWriter.
>>>> emit(RecordWriter.java:85)
>>>>         at org.apache.flink.runtime.operators.shipping.OutputCollector.
>>>> collect(OutputCollector.java:65)
>>>>         at org.apache.flink.runtime.operators.util.metrics.CountingColl
>>>> ector.collect(CountingCollector.java:35)
>>>>         at org.apache.flink.graph.library.Summarization$VertexGroupRedu
>>>> cer.reduce(Summarization.java:318)
>>>>         at org.apache.flink.runtime.operators.GroupReduceDriver.run(
>>>> GroupReduceDriver.java:131)
>>>>         at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.
>>>> java:486)
>>>>         at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTas
>>>> k.java:351)
>>>>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
>>>>         at java.lang.Thread.run(Thread.java:748)
>>>>
>>>> I also tried with different types for EdgeValue. The same error appears.
>>>> When I look at the result vertices, everything looks fine.
>>>> But the result edges has this problem.
>>>>
>>>> Any idea?
>>>>
>>>> Regards,
>>>> Ali
>>>>
>>>
>>
>>
>>



Free forum by Nabble Edit this page