buffer pool is destroyed

[Chan, Shuang]

Hi Flink community,

 

I have a custom source that emits an user-defined data type, BaseEvent.  The following code works fine when BaseEvent is not POJO.

But, when I changed it to POJO by adding a default constructor, I’m getting “Buffer Pool is destroyed” runtime exception on the Collect method.

 

            DataStream<BaseEvent> eventStream = see.addSource(new AgoraSource(configFile, instance));

 

            DataStream<Tuple4<String, Long, Double, String>> result_order = eventStream

                    .filter(e -> e instanceof OrderEvent)

                    .map(e -> (OrderEvent)e)

                    .map(e -> new Tuple3<>(e.SecurityID, Long.valueOf(1), Double.valueOf(e.OriginalQuantity))).returns(info_tuple3)

                    .keyBy(e -> e.f0)

                    .timeWindow(Time.seconds(5))

                    .reduce((a, b) -> new Tuple3<>(a.f0, a.f1 + b.f1, a.f2 + b.f2))

                    .map(e -> new Tuple4<>(e.f0, e.f1, e.f2, "Order")).returns(info_tuple4);

 

Any idea?

 

Shuang

 

==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================

[Zhijiang(wangzhijiang999)]

Hi Shuang,

Normally this exception you mentioned is not the root cause of failover, and it is mainly caused  by cancel process to make task exit.

You can further check whether there are other failures in job master log to find the root cause.

Best,

Zhijiang

------------------------------------------------------------------

发件人：Chan, Shuang <[hidden email]>

发送时间：2018年12月21日(星期五) 11:12

收件人：[hidden email] <[hidden email]>

主　题：buffer pool is destroyed

Hi Flink community,

 

I have a custom source that emits an user-defined data type, BaseEvent.  The following code works fine when BaseEvent is not POJO.

But, when I changed it to POJO by adding a default constructor, I’m getting “Buffer Pool is destroyed” runtime exception on the Collect method.

 

            DataStream<BaseEvent> eventStream = see.addSource(new AgoraSource(configFile, instance));

 

            DataStream<Tuple4<String, Long, Double, String>> result_order = eventStream

                    .filter(e -> e instanceof OrderEvent)

                    .map(e -> (OrderEvent)e)

                    .map(e -> new Tuple3<>(e.SecurityID, Long.valueOf(1), Double.valueOf(e.OriginalQuantity))).returns(info_tuple3)

                    .keyBy(e -> e.f0)

                    .timeWindow(Time.seconds(5))

                    .reduce((a, b) -> new Tuple3<>(a.f0, a.f1 + b.f1, a.f2 + b.f2))

                    .map(e -> new Tuple4<>(e.f0, e.f1, e.f2, "Order")).returns(info_tuple4);

 

Any idea?

 

Shuang

 

==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================

[Chan, Shuang]

Thank you.  Forgot to mention that the error occurred when I was testing my code in Intellij.

 

From: zhijiang [mailto:[hidden email]]
Sent: Friday, December 21, 2018 4:40 PM
To: [hidden email]; Chan, Shuang (PWTB 291)
Subject: 回复：buffer pool is destroyed

 

Hi Shuang,

 

Normally this exception you mentioned is not the root cause of failover, and it is mainly caused  by cancel process to make task exit.

You can further check whether there are other failures in job master log to find the root cause.

 

Best,

Zhijiang

------------------------------------------------------------------

发件人：Chan, Shuang <[hidden email]>

发送时间：2018年12月21日(星期五) 11:12

收件人：[hidden email] <[hidden email]>

主　题：buffer pool is destroyed

 

Hi Flink community,

 

I have a custom source that emits an user-defined data type, BaseEvent.  The following code works fine when BaseEvent is not POJO.

But, when I changed it to POJO by adding a default constructor, I’m getting “Buffer Pool is destroyed” runtime exception on the Collect method.

 

            DataStream<BaseEvent> eventStream = see.addSource(new AgoraSource(configFile, instance));

 

            DataStream<Tuple4<String, Long, Double, String>> result_order = eventStream

                    .filter(e -> e instanceof OrderEvent)

                    .map(e -> (OrderEvent)e)

                    .map(e -> new Tuple3<>(e.SecurityID, Long.valueOf(1), Double.valueOf(e.OriginalQuantity))).returns(info_tuple3)

                    .keyBy(e -> e.f0)

                    .timeWindow(Time.seconds(5))

                    .reduce((a, b) -> new Tuple3<>(a.f0, a.f1 + b.f1, a.f2 + b.f2))

                    .map(e -> new Tuple4<>(e.f0, e.f1, e.f2, "Order")).returns(info_tuple4);

 

Any idea?

 

Shuang

 

 

==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================

 

==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================