io.netty.channel.epoll.EpollEventLoopGroup cannot be cast to io.netty.channel.MultithreadEventLoopGroup

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

io.netty.channel.epoll.EpollEventLoopGroup cannot be cast to io.netty.channel.MultithreadEventLoopGroup

bat man
Hi,

I am using this library - jasync-postgresql [1] for async calls to postgres in asyncio operator. I am using running Flink 1.9 on EMR.
I am facing this error -

Caused by: java.lang.ClassCastException: io.netty.channel.epoll.EpollEventLoopGroup cannot be cast to io.netty.channel.MultithreadEventLoopGroup
at com.github.jasync.sql.db.util.NettyUtils$DefaultEventLoopGroup$2.invoke(NettyUtils.kt:32)
at com.github.jasync.sql.db.util.NettyUtils$DefaultEventLoopGroup$2.invoke(NettyUtils.kt:19)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
at com.github.jasync.sql.db.util.NettyUtils.getDefaultEventLoopGroup(NettyUtils.kt)
at com.github.jasync.sql.db.Configuration.<init>(Configuration.kt:55)
at com.github.jasync.sql.db.Configuration.<init>(Configuration.kt)
at com.flink.poc.inventory.AsyncJdbcRequest$AsyncDatabaseRequest.open(AsyncJdbcRequest.java:50)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
at org.apache.flink.streaming.api.operators.async.AsyncWaitOperator.open(AsyncWaitOperator.java:167)
at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:529)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:393)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
at java.lang.Thread.run(Thread.java:748)

I tried setting taskmanager.network.netty.transport: epoll still no difference. Is this because of the netty version used by flink and the library, this library uses - 
NETTY_VERSION=4.1.49.Final

[1] - https://github.com/jasync-sql/jasync-sql


Thanks,
Hemant
Reply | Threaded
Open this post in threaded view
|

Re: io.netty.channel.epoll.EpollEventLoopGroup cannot be cast to io.netty.channel.MultithreadEventLoopGroup

Arvid Heise-4
Hi Hemant,

Yes, this looks like an issue with different library versions. You probably have 3 solutions:
* use the netty version of Flink
* shade your netty into your jar with relocations
* ditch jasync and just use jdbc with a custom thread pool (little overhead)

On Wed, Mar 10, 2021 at 2:40 PM bat man <[hidden email]> wrote:
Hi,

I am using this library - jasync-postgresql [1] for async calls to postgres in asyncio operator. I am using running Flink 1.9 on EMR.
I am facing this error -

Caused by: java.lang.ClassCastException: io.netty.channel.epoll.EpollEventLoopGroup cannot be cast to io.netty.channel.MultithreadEventLoopGroup
at com.github.jasync.sql.db.util.NettyUtils$DefaultEventLoopGroup$2.invoke(NettyUtils.kt:32)
at com.github.jasync.sql.db.util.NettyUtils$DefaultEventLoopGroup$2.invoke(NettyUtils.kt:19)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
at com.github.jasync.sql.db.util.NettyUtils.getDefaultEventLoopGroup(NettyUtils.kt)
at com.github.jasync.sql.db.Configuration.<init>(Configuration.kt:55)
at com.github.jasync.sql.db.Configuration.<init>(Configuration.kt)
at com.flink.poc.inventory.AsyncJdbcRequest$AsyncDatabaseRequest.open(AsyncJdbcRequest.java:50)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
at org.apache.flink.streaming.api.operators.async.AsyncWaitOperator.open(AsyncWaitOperator.java:167)
at org.apache.flink.streaming.runtime.tasks.StreamTask.openAllOperators(StreamTask.java:529)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:393)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:530)
at java.lang.Thread.run(Thread.java:748)

I tried setting taskmanager.network.netty.transport: epoll still no difference. Is this because of the netty version used by flink and the library, this library uses - 
NETTY_VERSION=4.1.49.Final

[1] - https://github.com/jasync-sql/jasync-sql


Thanks,
Hemant