(DEPRECATED) Apache Flink User Mailing List archive.

Flink 1.8.0 S3 Checkpoints fail with java.security.ProviderException: Could not initialize NSS

classic Classic list List threaded Threaded
♦
♦
 2 messages Options
austin.ce
Reply | Threaded
Open this post in threaded view
♦
♦
|

Flink 1.8.0 S3 Checkpoints fail with java.security.ProviderException: Could not initialize NSS

austin.ce
Hi there,

I'm getting the following error message on a Flink 1.8 cluster deployed on Kubernetes. I've already confirmed that the pod has access to S3 and write permissions to the bucket, but I can't understand what the SSL issue is and if it is related to S3 or not. I have tried both with the default state backend and with rocksdb. It happens immediately upon triggering a savepoint. Has anyone seen errors like this? 

Thank you!
Austin


2019-10-10T22:21:36.496009042Z 2019-10-10 22:21:36,495 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Triggering checkpoint 1 @ 1570746096485 for job 32a9a430038d440cbfee808101dcccd1.
2019-10-10T22:21:36.871364673Z 2019-10-10 22:21:36,858 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Window(GlobalWindows(), DeltaTrigger, TimeEvictor, ComparableAggregator, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (6cb94a0782212895ac1e062b4124e425) switched from RUNNING to FAILED.
2019-10-10T22:21:36.87141613Z java.lang.ExceptionInInitializerError: null
2019-10-10T22:21:36.871422053Z  at sun.security.ssl.SSLSessionImpl.<init>(SSLSessionImpl.java:183) ~[?:1.8.0_181]
2019-10-10T22:21:36.87142623Z   at sun.security.ssl.SSLSessionImpl.<init>(SSLSessionImpl.java:148) ~[?:1.8.0_181]
2019-10-10T22:21:36.871429991Z  at sun.security.ssl.SSLSessionImpl.<clinit>(SSLSessionImpl.java:79) ~[?:1.8.0_181]
2019-10-10T22:21:36.871434248Z  at sun.security.ssl.SSLSocketImpl.init(SSLSocketImpl.java:604) ~[?:1.8.0_181]
2019-10-10T22:21:36.871450145Z  at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:572) ~[?:1.8.0_181]
2019-10-10T22:21:36.871454174Z  at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:110) ~[?:1.8.0_181]
2019-10-10T22:21:36.871457724Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:365) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871461103Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871465705Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:132) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87146981Z   at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871474533Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871494299Z  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.87149796Z   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_181]
2019-10-10T22:21:36.871502637Z  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
2019-10-10T22:21:36.871506464Z  at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
2019-10-10T22:21:36.871510239Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871513871Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.conn.$Proxy19.connect(Unknown Source) ~[?:1.8.0-stream1]
2019-10-10T22:21:36.871516965Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871520624Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87152418Z   at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871527877Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871531559Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871537442Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871540992Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871544703Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1236) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871548864Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871552735Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871569584Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871573374Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871576891Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871580571Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871584153Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871587902Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4325) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871591634Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:5086) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871595388Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:5060) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87159884Z   at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4309) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871602536Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4272) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871606764Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1264) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871610595Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1239) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871616677Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.lambda$getS3ObjectMetadata$2(PrestoS3FileSystem.java:563) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871620511Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.RetryDriver.run(RetryDriver.java:138) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871623898Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.getS3ObjectMetadata(PrestoS3FileSystem.java:560) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871627531Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.getFileStatus(PrestoS3FileSystem.java:311) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871631252Z  at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871647548Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.create(PrestoS3FileSystem.java:356) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871651632Z  at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1169) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871655207Z  at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1149) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87165862Z   at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1038) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871662131Z  at org.apache.flink.fs.s3.common.hadoop.HadoopFileSystem.create(HadoopFileSystem.java:141) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871665978Z  at org.apache.flink.fs.s3.common.hadoop.HadoopFileSystem.create(HadoopFileSystem.java:37) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871669477Z  at org.apache.flink.core.fs.SafetyNetWrapperFileSystem.create(SafetyNetWrapperFileSystem.java:126) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871672862Z  at org.apache.flink.core.fs.EntropyInjector.createEntropyAware(EntropyInjector.java:61) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87167649Z   at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.createStream(FsCheckpointStreamFactory.java:348) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871680186Z  at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.flush(FsCheckpointStreamFactory.java:226) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871683891Z  at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.write(FsCheckpointStreamFactory.java:201) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871689606Z  at org.apache.flink.runtime.state.NonClosingCheckpointOutputStream.write(NonClosingCheckpointOutputStream.java:57) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871693364Z  at java.io.DataOutputStream.write(DataOutputStream.java:107) ~[?:1.8.0_181]
2019-10-10T22:21:36.871696675Z  at java.io.DataOutputStream.writeUTF(DataOutputStream.java:401) ~[?:1.8.0_181]
2019-10-10T22:21:36.871700127Z  at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323) ~[?:1.8.0_181]
2019-10-10T22:21:36.871703449Z  at org.apache.flink.api.java.typeutils.runtime.TupleSerializerSnapshot.writeOuterSnapshot(TupleSerializerSnapshot.java:83) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871706927Z  at org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.internalWriteOuterSnapshot(CompositeTypeSerializerSnapshot.java:287) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871723072Z  at org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.writeSnapshot(CompositeTypeSerializerSnapshot.java:138) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871726906Z  at org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.writeVersionedSnapshot(TypeSerializerSnapshot.java:149) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871731089Z  at org.apache.flink.streaming.api.operators.InternalTimersSnapshotReaderWriters$InternalTimersSnapshotWriterV2.writeKeyAndNamespaceSerializers(InternalTimersSnapshotReaderWriters.java:199) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871735159Z  at org.apache.flink.streaming.api.operators.InternalTimersSnapshotReaderWriters$AbstractInternalTimersSnapshotWriter.writeTimersSnapshot(InternalTimersSnapshotReaderWriters.java:117) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871739042Z  at org.apache.flink.streaming.api.operators.InternalTimerServiceSerializationProxy.write(InternalTimerServiceSerializationProxy.java:101) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871742826Z  at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.snapshotStateForKeyGroup(InternalTimeServiceManager.java:139) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871746473Z  at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:453) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871750167Z  at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:89) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871753793Z  at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:395) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871757361Z  at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.checkpointStreamOperator(StreamTask.java:1113) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871761066Z  at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.executeCheckpointing(StreamTask.java:1055) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871766939Z  at org.apache.flink.streaming.runtime.tasks.StreamTask.checkpointState(StreamTask.java:729) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87177047Z   at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:641) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871774542Z  at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:586) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871778222Z  at org.apache.flink.streaming.runtime.io.BarrierTracker.notifyCheckpoint(BarrierTracker.java:270) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871781806Z  at org.apache.flink.streaming.runtime.io.BarrierTracker.processBarrier(BarrierTracker.java:148) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871795979Z  at org.apache.flink.streaming.runtime.io.BarrierTracker.getNextNonBlocked(BarrierTracker.java:105) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871800298Z  at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:209) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87180383Z   at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871807315Z  at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871810963Z  at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87181459Z   at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
2019-10-10T22:21:36.871834895Z Caused by: java.security.ProviderException: Could not initialize NSS
2019-10-10T22:21:36.871842354Z  at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:223) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871846036Z  at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:103) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871849609Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871853034Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_181]
2019-10-10T22:21:36.871856444Z  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_181]
2019-10-10T22:21:36.871859785Z  at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181]
2019-10-10T22:21:36.871863105Z  at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:224) ~[?:1.8.0_181]
2019-10-10T22:21:36.871866512Z  at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871870812Z  at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871874219Z  at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871880884Z  at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187) ~[?:1.8.0_181]
2019-10-10T22:21:36.871884308Z  at sun.security.jca.ProviderList.getProvider(ProviderList.java:233) ~[?:1.8.0_181]
2019-10-10T22:21:36.871887771Z  at sun.security.jca.ProviderList.getIndex(ProviderList.java:263) ~[?:1.8.0_181]
2019-10-10T22:21:36.871891107Z  at sun.security.jca.ProviderList.getProviderConfig(ProviderList.java:247) ~[?:1.8.0_181]
2019-10-10T22:21:36.87189453Z   at sun.security.jca.ProviderList.getProvider(ProviderList.java:253) ~[?:1.8.0_181]
2019-10-10T22:21:36.871897904Z  at java.security.Security.getProvider(Security.java:501) ~[?:1.8.0_181]
2019-10-10T22:21:36.871913802Z  at sun.security.ssl.SignatureAndHashAlgorithm.<clinit>(SignatureAndHashAlgorithm.java:415) ~[?:1.8.0_181]
2019-10-10T22:21:36.871917562Z  ... 83 more
2019-10-10T22:21:36.871920793Z Caused by: java.io.IOException: NSS is already initialized
2019-10-10T22:21:36.871924033Z  at sun.security.pkcs11.Secmod.initialize(Secmod.java:173) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.87192751Z   at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:218) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871930935Z  at sun.security.pkcs11.SunPKCS11.<init>(SunPKCS11.java:103) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871934517Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871937891Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_181]
2019-10-10T22:21:36.871941341Z  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_181]
2019-10-10T22:21:36.871944937Z  at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181]
2019-10-10T22:21:36.871948443Z  at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:224) ~[?:1.8.0_181]
2019-10-10T22:21:36.8719518Z    at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871955232Z  at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871958353Z  at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871961824Z  at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187) ~[?:1.8.0_181]
2019-10-10T22:21:36.871965181Z  at sun.security.jca.ProviderList.getProvider(ProviderList.java:233) ~[?:1.8.0_181]
2019-10-10T22:21:36.871968623Z  at sun.security.jca.ProviderList.getIndex(ProviderList.java:263) ~[?:1.8.0_181]
2019-10-10T22:21:36.871972723Z  at sun.security.jca.ProviderList.getProviderConfig(ProviderList.java:247) ~[?:1.8.0_181]
2019-10-10T22:21:36.871987354Z  at sun.security.jca.ProviderList.getProvider(ProviderList.java:253) ~[?:1.8.0_181]
2019-10-10T22:21:36.871990887Z  at java.security.Security.getProvider(Security.java:501) ~[?:1.8.0_181]
2019-10-10T22:21:36.871996829Z  at sun.security.ssl.SignatureAndHashAlgorithm.<clinit>(SignatureAndHashAlgorithm.java:415) ~[?:1.8.0_181]
2019-10-10T22:21:36.872000854Z  ... 83 more
Hao Sun
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink 1.8.0 S3 Checkpoints fail with java.security.ProviderException: Could not initialize NSS

Hao Sun
I saw similar issue when using alpine linux.
https://pkgs.alpinelinux.org/package/v3.3/main/x86/nss

Installing this package fixed my problem

Hao Sun


On Thu, Oct 10, 2019 at 3:46 PM Austin Cawley-Edwards <[hidden email]> wrote:
Hi there,

I'm getting the following error message on a Flink 1.8 cluster deployed on Kubernetes. I've already confirmed that the pod has access to S3 and write permissions to the bucket, but I can't understand what the SSL issue is and if it is related to S3 or not. I have tried both with the default state backend and with rocksdb. It happens immediately upon triggering a savepoint. Has anyone seen errors like this? 

Thank you!
Austin


2019-10-10T22:21:36.496009042Z 2019-10-10 22:21:36,495 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator    [] - Triggering checkpoint 1 @ 1570746096485 for job 32a9a430038d440cbfee808101dcccd1.
2019-10-10T22:21:36.871364673Z 2019-10-10 22:21:36,858 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Window(GlobalWindows(), DeltaTrigger, TimeEvictor, ComparableAggregator, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (6cb94a0782212895ac1e062b4124e425) switched from RUNNING to FAILED.
2019-10-10T22:21:36.87141613Z java.lang.ExceptionInInitializerError: null
2019-10-10T22:21:36.871422053Z  at sun.security.ssl.SSLSessionImpl.<init>(http://SSLSessionImpl.java:183) ~[?:1.8.0_181]
2019-10-10T22:21:36.87142623Z   at sun.security.ssl.SSLSessionImpl.<init>(http://SSLSessionImpl.java:148) ~[?:1.8.0_181]
2019-10-10T22:21:36.871429991Z  at sun.security.ssl.SSLSessionImpl.<clinit>(http://SSLSessionImpl.java:79) ~[?:1.8.0_181]
2019-10-10T22:21:36.871434248Z  at sun.security.ssl.SSLSocketImpl.init(SSLSocketImpl.java:604) ~[?:1.8.0_181]
2019-10-10T22:21:36.871450145Z  at sun.security.ssl.SSLSocketImpl.<init>(http://SSLSocketImpl.java:572) ~[?:1.8.0_181]
2019-10-10T22:21:36.871454174Z  at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:110) ~[?:1.8.0_181]
2019-10-10T22:21:36.871457724Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:365) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871461103Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871465705Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:132) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87146981Z   at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871474533Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871494299Z  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.87149796Z   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_181]
2019-10-10T22:21:36.871502637Z  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
2019-10-10T22:21:36.871506464Z  at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
2019-10-10T22:21:36.871510239Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871513871Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.conn.$Proxy19.connect(Unknown Source) ~[?:1.8.0-stream1]
2019-10-10T22:21:36.871516965Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871520624Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87152418Z   at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871527877Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871531559Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871537442Z  at org.apache.flink.fs.s3base.shaded.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871540992Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871544703Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1236) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871548864Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871552735Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871569584Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871573374Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871576891Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871580571Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871584153Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871587902Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4325) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871591634Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:5086) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871595388Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:5060) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87159884Z   at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4309) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871602536Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4272) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871606764Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1264) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871610595Z  at org.apache.flink.fs.s3base.shaded.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1239) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871616677Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.lambda$getS3ObjectMetadata$2(PrestoS3FileSystem.java:563) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871620511Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.RetryDriver.run(RetryDriver.java:138) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871623898Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.getS3ObjectMetadata(PrestoS3FileSystem.java:560) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871627531Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.getFileStatus(PrestoS3FileSystem.java:311) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871631252Z  at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871647548Z  at org.apache.flink.fs.s3presto.shaded.com.facebook.presto.hive.s3.PrestoS3FileSystem.create(PrestoS3FileSystem.java:356) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871651632Z  at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1169) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871655207Z  at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1149) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87165862Z   at org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1038) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871662131Z  at org.apache.flink.fs.s3.common.hadoop.HadoopFileSystem.create(HadoopFileSystem.java:141) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871665978Z  at org.apache.flink.fs.s3.common.hadoop.HadoopFileSystem.create(HadoopFileSystem.java:37) ~[flink-s3-fs-presto-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871669477Z  at org.apache.flink.core.fs.SafetyNetWrapperFileSystem.create(SafetyNetWrapperFileSystem.java:126) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871672862Z  at org.apache.flink.core.fs.EntropyInjector.createEntropyAware(EntropyInjector.java:61) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87167649Z   at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.createStream(FsCheckpointStreamFactory.java:348) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871680186Z  at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.flush(FsCheckpointStreamFactory.java:226) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871683891Z  at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.write(FsCheckpointStreamFactory.java:201) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871689606Z  at org.apache.flink.runtime.state.NonClosingCheckpointOutputStream.write(NonClosingCheckpointOutputStream.java:57) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871693364Z  at java.io.DataOutputStream.write(DataOutputStream.java:107) ~[?:1.8.0_181]
2019-10-10T22:21:36.871696675Z  at java.io.DataOutputStream.writeUTF(DataOutputStream.java:401) ~[?:1.8.0_181]
2019-10-10T22:21:36.871700127Z  at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323) ~[?:1.8.0_181]
2019-10-10T22:21:36.871703449Z  at org.apache.flink.api.java.typeutils.runtime.TupleSerializerSnapshot.writeOuterSnapshot(TupleSerializerSnapshot.java:83) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871706927Z  at org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.internalWriteOuterSnapshot(CompositeTypeSerializerSnapshot.java:287) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871723072Z  at org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.writeSnapshot(CompositeTypeSerializerSnapshot.java:138) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871726906Z  at org.apache.flink.api.common.typeutils.TypeSerializerSnapshot.writeVersionedSnapshot(TypeSerializerSnapshot.java:149) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871731089Z  at org.apache.flink.streaming.api.operators.InternalTimersSnapshotReaderWriters$InternalTimersSnapshotWriterV2.writeKeyAndNamespaceSerializers(InternalTimersSnapshotReaderWriters.java:199) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871735159Z  at org.apache.flink.streaming.api.operators.InternalTimersSnapshotReaderWriters$AbstractInternalTimersSnapshotWriter.writeTimersSnapshot(InternalTimersSnapshotReaderWriters.java:117) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871739042Z  at org.apache.flink.streaming.api.operators.InternalTimerServiceSerializationProxy.write(InternalTimerServiceSerializationProxy.java:101) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871742826Z  at org.apache.flink.streaming.api.operators.InternalTimeServiceManager.snapshotStateForKeyGroup(InternalTimeServiceManager.java:139) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871746473Z  at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:453) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871750167Z  at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:89) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871753793Z  at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:395) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871757361Z  at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.checkpointStreamOperator(StreamTask.java:1113) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871761066Z  at org.apache.flink.streaming.runtime.tasks.StreamTask$CheckpointingOperation.executeCheckpointing(StreamTask.java:1055) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871766939Z  at org.apache.flink.streaming.runtime.tasks.StreamTask.checkpointState(StreamTask.java:729) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87177047Z   at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:641) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871774542Z  at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:586) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871778222Z  at org.apache.flink.streaming.runtime.io.BarrierTracker.notifyCheckpoint(BarrierTracker.java:270) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871781806Z  at org.apache.flink.streaming.runtime.io.BarrierTracker.processBarrier(BarrierTracker.java:148) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871795979Z  at org.apache.flink.streaming.runtime.io.BarrierTracker.getNextNonBlocked(BarrierTracker.java:105) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871800298Z  at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:209) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87180383Z   at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871807315Z  at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.871810963Z  at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) ~[flink-dist_2.11-1.8.0-stream1.jar:1.8.0-stream1]
2019-10-10T22:21:36.87181459Z   at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
2019-10-10T22:21:36.871834895Z Caused by: java.security.ProviderException: Could not initialize NSS
2019-10-10T22:21:36.871842354Z  at sun.security.pkcs11.SunPKCS11.<init>(http://SunPKCS11.java:223) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871846036Z  at sun.security.pkcs11.SunPKCS11.<init>(http://SunPKCS11.java:103) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871849609Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871853034Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_181]
2019-10-10T22:21:36.871856444Z  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_181]
2019-10-10T22:21:36.871859785Z  at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181]
2019-10-10T22:21:36.871863105Z  at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:224) ~[?:1.8.0_181]
2019-10-10T22:21:36.871866512Z  at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871870812Z  at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871874219Z  at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871880884Z  at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187) ~[?:1.8.0_181]
2019-10-10T22:21:36.871884308Z  at sun.security.jca.ProviderList.getProvider(ProviderList.java:233) ~[?:1.8.0_181]
2019-10-10T22:21:36.871887771Z  at sun.security.jca.ProviderList.getIndex(ProviderList.java:263) ~[?:1.8.0_181]
2019-10-10T22:21:36.871891107Z  at sun.security.jca.ProviderList.getProviderConfig(ProviderList.java:247) ~[?:1.8.0_181]
2019-10-10T22:21:36.87189453Z   at sun.security.jca.ProviderList.getProvider(ProviderList.java:253) ~[?:1.8.0_181]
2019-10-10T22:21:36.871897904Z  at java.security.Security.getProvider(Security.java:501) ~[?:1.8.0_181]
2019-10-10T22:21:36.871913802Z  at sun.security.ssl.SignatureAndHashAlgorithm.<clinit>(http://SignatureAndHashAlgorithm.java:415) ~[?:1.8.0_181]
2019-10-10T22:21:36.871917562Z  ... 83 more
2019-10-10T22:21:36.871920793Z Caused by: java.io.IOException: NSS is already initialized
2019-10-10T22:21:36.871924033Z  at sun.security.pkcs11.Secmod.initialize(Secmod.java:173) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.87192751Z   at sun.security.pkcs11.SunPKCS11.<init>(http://SunPKCS11.java:218) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871930935Z  at sun.security.pkcs11.SunPKCS11.<init>(http://SunPKCS11.java:103) ~[sunpkcs11.jar:1.8.0_181]
2019-10-10T22:21:36.871934517Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871937891Z  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_181]
2019-10-10T22:21:36.871941341Z  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_181]
2019-10-10T22:21:36.871944937Z  at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_181]
2019-10-10T22:21:36.871948443Z  at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:224) ~[?:1.8.0_181]
2019-10-10T22:21:36.8719518Z    at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871955232Z  at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
2019-10-10T22:21:36.871958353Z  at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:206) ~[?:1.8.0_181]
2019-10-10T22:21:36.871961824Z  at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:187) ~[?:1.8.0_181]
2019-10-10T22:21:36.871965181Z  at sun.security.jca.ProviderList.getProvider(ProviderList.java:233) ~[?:1.8.0_181]
2019-10-10T22:21:36.871968623Z  at sun.security.jca.ProviderList.getIndex(ProviderList.java:263) ~[?:1.8.0_181]
2019-10-10T22:21:36.871972723Z  at sun.security.jca.ProviderList.getProviderConfig(ProviderList.java:247) ~[?:1.8.0_181]
2019-10-10T22:21:36.871987354Z  at sun.security.jca.ProviderList.getProvider(ProviderList.java:253) ~[?:1.8.0_181]
2019-10-10T22:21:36.871990887Z  at java.security.Security.getProvider(Security.java:501) ~[?:1.8.0_181]
2019-10-10T22:21:36.871996829Z  at sun.security.ssl.SignatureAndHashAlgorithm.<clinit>(http://SignatureAndHashAlgorithm.java:415) ~[?:1.8.0_181]
2019-10-10T22:21:36.872000854Z  ... 83 more
Free forum by Nabble Edit this page