Error --GC Cleaner Provider -- Flink 1.11.0
Locked
Error --GC Cleaner Provider -- Flink 1.11.0
 
|
Hi All, I have written simple java code that read data using Hive and transform using Table API (Blink Planner) and Flink 1.11.0 on HDP cluster. I am encountering "java.lang.Error: Failed to find GC Cleaner among available providers” error. Full error stack is at end of the email. Do anyone encounter the same issue or any solutions/suggestions ? Cluster Config: * Hadoop Version: 2.7.3 * Java Version: 1.8.0_40 * Flink 1.11.0 ( built from source) * Hive 1.2.1 Thanks Murali Pusala Caused by: java.lang.Error: Failed to find GC Cleaner among available providers: [Legacy (before Java 9) cleaner provider, New Java 9+ cleaner provider] at org.apache.flink.util.JavaGcCleanerWrapper.findGcCleanerManager(JavaGcCleanerWrapper.java:149) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper.<clinit>(JavaGcCleanerWrapper.java:56) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemoryUtils.createMemoryGcCleaner(MemoryUtils.java:111) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemorySegmentFactory.allocateOffHeapUnsafeMemory(MemorySegmentFactory.java:175) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager.lambda$allocatePages$0(MemoryManager.java:237) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager$$Lambda$188/978816879.apply(Unknown Source) ~[?:?] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853) ~[?:1.8.0_40] at org.apache.flink.runtime.memory.MemoryManager.allocatePages(MemoryManager.java:233) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.util.LazyMemorySegmentPool.nextSegment(LazyMemorySegmentPool.java:82) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.io.disk.SimpleCollectingOutputView.<init>(SimpleCollectingOutputView.java:49) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap$RecordArea.<init>(BytesHashMap.java:522) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:184) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:148) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at LocalHashAggregateWithKeys$100.open(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:473) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask$$Lambda$176/1800425807.run(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:469) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:522) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_40] Caused by: org.apache.flink.util.FlinkRuntimeException: Legacy (before Java 9) cleaner provider: Failed to find Reference#tryHandlePending method at org.apache.flink.util.JavaGcCleanerWrapper$ReflectionUtils.findMethod(JavaGcCleanerWrapper.java:398) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$ReflectionUtils.access$1300(JavaGcCleanerWrapper.java:376) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunnerProvider.createPendingCleanersRunner(JavaGcCleanerWrapper.java:326) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunnerProvider.access$800(JavaGcCleanerWrapper.java:303) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerProvider.createCleanerManager(JavaGcCleanerWrapper.java:180) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerProvider.access$400(JavaGcCleanerWrapper.java:162) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper.findGcCleanerManager(JavaGcCleanerWrapper.java:140) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper.<clinit>(JavaGcCleanerWrapper.java:56) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemoryUtils.createMemoryGcCleaner(MemoryUtils.java:111) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemorySegmentFactory.allocateOffHeapUnsafeMemory(MemorySegmentFactory.java:175) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager.lambda$allocatePages$0(MemoryManager.java:237) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager$$Lambda$188/978816879.apply(Unknown Source) ~[?:?] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853) ~[?:1.8.0_40] at org.apache.flink.runtime.memory.MemoryManager.allocatePages(MemoryManager.java:233) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.util.LazyMemorySegmentPool.nextSegment(LazyMemorySegmentPool.java:82) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.io.disk.SimpleCollectingOutputView.<init>(SimpleCollectingOutputView.java:49) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap$RecordArea.<init>(BytesHashMap.java:522) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:184) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:148) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at LocalHashAggregateWithKeys$100.open(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:473) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask$$Lambda$176/1800425807.run(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:469) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:522) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_40] Suppressed: org.apache.flink.util.FlinkRuntimeException: New Java 9+ cleaner provider: Failed to find java class at org.apache.flink.util.JavaGcCleanerWrapper$ReflectionUtils.findClass(JavaGcCleanerWrapper.java:388) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$ReflectionUtils.access$1200(JavaGcCleanerWrapper.java:376) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerFactoryProvider.createCleanerFactory(JavaGcCleanerWrapper.java:244) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerFactoryProvider.access$700(JavaGcCleanerWrapper.java:217) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerProvider.createCleanerManager(JavaGcCleanerWrapper.java:179) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerProvider.access$400(JavaGcCleanerWrapper.java:162) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper.findGcCleanerManager(JavaGcCleanerWrapper.java:140) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper.<clinit>(JavaGcCleanerWrapper.java:56) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemoryUtils.createMemoryGcCleaner(MemoryUtils.java:111) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemorySegmentFactory.allocateOffHeapUnsafeMemory(MemorySegmentFactory.java:175) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager.lambda$allocatePages$0(MemoryManager.java:237) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager$$Lambda$188/978816879.apply(Unknown Source) ~[?:?] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853) ~[?:1.8.0_40] at org.apache.flink.runtime.memory.MemoryManager.allocatePages(MemoryManager.java:233) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.util.LazyMemorySegmentPool.nextSegment(LazyMemorySegmentPool.java:82) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.io.disk.SimpleCollectingOutputView.<init>(SimpleCollectingOutputView.java:49) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap$RecordArea.<init>(BytesHashMap.java:522) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:184) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:148) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at LocalHashAggregateWithKeys$100.open(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:473) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask$$Lambda$176/1800425807.run(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:469) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:522) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_40] Caused by: java.lang.ClassNotFoundException: java.lang.ref.Cleaner at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_40] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_40] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_40] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_40] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_40] at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_40] at org.apache.flink.util.JavaGcCleanerWrapper$ReflectionUtils.findClass(JavaGcCleanerWrapper.java:385) ~[flink-dist_2.11-1.11.0.jar:1.11.0] ... 28 more Caused by: java.lang.NoSuchMethodException: java.lang.ref.Reference.tryHandlePending(boolean) at java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:1.8.0_40] at org.apache.flink.util.JavaGcCleanerWrapper$ReflectionUtils.findMethod(JavaGcCleanerWrapper.java:395) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$ReflectionUtils.access$1300(JavaGcCleanerWrapper.java:376) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunnerProvider.createPendingCleanersRunner(JavaGcCleanerWrapper.java:326) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$PendingCleanersRunnerProvider.access$800(JavaGcCleanerWrapper.java:303) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerProvider.createCleanerManager(JavaGcCleanerWrapper.java:180) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper$CleanerProvider.access$400(JavaGcCleanerWrapper.java:162) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper.findGcCleanerManager(JavaGcCleanerWrapper.java:140) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.util.JavaGcCleanerWrapper.<clinit>(JavaGcCleanerWrapper.java:56) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemoryUtils.createMemoryGcCleaner(MemoryUtils.java:111) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.core.memory.MemorySegmentFactory.allocateOffHeapUnsafeMemory(MemorySegmentFactory.java:175) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager.lambda$allocatePages$0(MemoryManager.java:237) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.memory.MemoryManager$$Lambda$188/978816879.apply(Unknown Source) ~[?:?] at java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853) ~[?:1.8.0_40] at org.apache.flink.runtime.memory.MemoryManager.allocatePages(MemoryManager.java:233) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.util.LazyMemorySegmentPool.nextSegment(LazyMemorySegmentPool.java:82) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.io.disk.SimpleCollectingOutputView.<init>(SimpleCollectingOutputView.java:49) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap$RecordArea.<init>(BytesHashMap.java:522) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:184) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at org.apache.flink.table.runtime.operators.aggregate.BytesHashMap.<init>(BytesHashMap.java:148) ~[flink-table-uber-blink_2.11-1.11.0.jar:1.11.0] at LocalHashAggregateWithKeys$100.open(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:473) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask$$Lambda$176/1800425807.run(Unknown Source) ~[?:?] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:469) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:522) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) ~[flink-dist_2.11-1.11.0.jar:1.11.0] at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_40] |
Re: Error --GC Cleaner Provider -- Flink 1.11.0
|
|
Hi Murali Pusala, Thanks for reporting this issue. Looks like JavaGcCleanerWrapper should use getDeclaredMethod instead of getMethod to find Reference.tryHandlePending. @Andrey, can you confirm this? Regards,
Roman On Mon, Jul 13, 2020 at 4:42 AM Murali Krishna Pusala <[hidden email]> wrote:
|
Re: Error --GC Cleaner Provider -- Flink 1.11.0
|
|
I think the problem is that the package-private method `Reference.tryHandlePending` does not exist in 1.8.0_40. The method does not exist in OpenJDK 8u40[1], but can be found in the latest AdoptOpenJDK [2]. It seems the method was first introduced in 8u202[3]. Thank you~ Xintong Song On Mon, Jul 13, 2020 at 4:27 PM Khachatryan Roman <[hidden email]> wrote:
|
Re: Error --GC Cleaner Provider -- Flink 1.11.0
|
|
FYI, I've opened FLINK-18581[1] for tracking this. Thank you~ Xintong Song On Mon, Jul 13, 2020 at 4:54 PM Xintong Song <[hidden email]> wrote:
|
Re: Error --GC Cleaner Provider -- Flink 1.11.0
|
|
Thanks for quick reply! On Jul 13, 2020, at 4:14 AM, Xintong Song <[hidden email]> wrote:
|
Re: Error --GC Cleaner Provider -- Flink 1.11.0
|
|
Hi Murali, A proper fix of this problem could take some time. It may or may not catch the next bug-fix release (1.11.1). At the meantime, you can try to workaround this by upgrading your JDK8 to a recent release. E.g., the latest JDK8u252 [1]. Thank you~ Xintong Song On Mon, Jul 13, 2020 at 8:42 PM Murali Krishna Pusala <[hidden email]> wrote:
|
«
Return to (DEPRECATED) Apache Flink User Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |