[INFO ] 2021-06-05 10:35:59.009 [flink-akka.actor.default-dispatcher-19] o.a.f.r.e.ExecutionGraph-[transitionState] - 1445 - Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1) (edd6794b02786a8e8022b43ab2a499cc) switched from RUNNING to FAILED on container_1620457281348_0044_01_000002 @ node1 (dataPort=43653). java.lang.Exception: Could not perform checkpoint 6499 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#0. at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:976) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$7(StreamTask.java:936) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:344) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:330) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:202) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:661) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:623) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:776) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) at java.lang.Thread.run(Thread.java:748) Suppressed: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.close(ElasticsearchSinkBase.java:366) at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.dispose(AbstractUdfStreamOperator.java:117) at org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:837) at org.apache.flink.streaming.runtime.tasks.StreamTask.runAndSuppressThrowable(StreamTask.java:816) at org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUpInvoke(StreamTask.java:733) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.cleanUpInvoke(SourceStreamTask.java:186) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:637) ... 3 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete snapshot 6499 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#0. Failure reason: Checkpoint was declined. at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:264) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:169) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:371) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointStreamOperator(SubtaskCheckpointCoordinatorImpl.java:702) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.buildOperatorSnapshotFutures(SubtaskCheckpointCoordinatorImpl.java:623) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:586) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:308) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$8(StreamTask.java:1062) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1046) at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:964) ... 11 common frames omitted Caused by: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkAsyncErrorsAndRequests(ElasticsearchSinkBase.java:432) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.snapshotState(ElasticsearchSinkBase.java:340) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:118) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:89) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:218) ... 21 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted [INFO ] 2021-06-05 10:35:59.016 [flink-akka.actor.default-dispatcher-18] o.a.f.r.r.s.DeclarativeSlotManager-[processResourceRequirements] - 260 - Clearing resource requirements of job 63c270e00b69eb967f59479bb1c84113 [INFO ] 2021-06-05 10:35:59.020 [flink-akka.actor.default-dispatcher-19] o.a.f.r.e.f.f.RestartPipelinedRegionFailoverStrategy-[getTasksNeedingRestart] - 115 - Calculating tasks to restart to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. [INFO ] 2021-06-05 10:35:59.020 [flink-akka.actor.default-dispatcher-19] o.a.f.r.e.f.f.RestartPipelinedRegionFailoverStrategy-[getTasksNeedingRestart] - 152 - 1 tasks should be restarted to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. [INFO ] 2021-06-05 10:35:59.022 [flink-akka.actor.default-dispatcher-19] o.a.f.r.e.ExecutionGraph-[transitionState] - 1039 - Job insert-into_default_catalog.default_database.table3 (63c270e00b69eb967f59479bb1c84113) switched from state RUNNING to RESTARTING. [INFO ] 2021-06-05 10:36:55.583 [flink-akka.actor.default-dispatcher-3] o.a.f.r.j.s.DefaultDeclarativeSlotPool-[releaseSlots] - 440 - Releasing slot [112535b2072d9d19cacc7df3fd267b66]. [INFO ] 2021-06-05 10:36:59.037 [flink-akka.actor.default-dispatcher-3] o.a.f.r.e.ExecutionGraph-[transitionState] - 1039 - Job insert-into_default_catalog.default_database.table3 (63c270e00b69eb967f59479bb1c84113) switched from state RESTARTING to RUNNING. [INFO ] 2021-06-05 10:36:59.040 [flink-akka.actor.default-dispatcher-3] o.a.f.r.c.CheckpointCoordinator-[restoreLatestCheckpointedStateInternal] - 1557 - Restoring job 63c270e00b69eb967f59479bb1c84113 from Checkpoint 6498 @ 1622860524528 for 63c270e00b69eb967f59479bb1c84113 located at hdfs://143.33.161.87:9000/flink-checkpoints/63c270e00b69eb967f59479bb1c84113/chk-6498. [INFO ] 2021-06-05 10:36:59.044 [flink-akka.actor.default-dispatcher-3] o.a.f.r.c.CheckpointCoordinator-[restoreMasterHooks] - 201 - No master state to restore [INFO ] 2021-06-05 10:36:59.044 [flink-akka.actor.default-dispatcher-3] o.a.f.r.e.ExecutionGraph-[transitionState] - 1437 - Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1) (745fb84751e9c366343dced00b08e4dc) switched from CREATED to SCHEDULED. [INFO ] 2021-06-05 10:36:59.045 [flink-akka.actor.default-dispatcher-19] o.a.f.r.r.s.DeclarativeSlotManager-[processResourceRequirements] - 262 - Received resource requirements from job 63c270e00b69eb967f59479bb1c84113: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}] [INFO ] 2021-06-05 10:36:59.065 [flink-akka.actor.default-dispatcher-3] o.a.f.r.e.ExecutionGraph-[transitionState] - 1437 - Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1) (745fb84751e9c366343dced00b08e4dc) switched from SCHEDULED to DEPLOYING. [INFO ] 2021-06-05 10:36:59.065 [flink-akka.actor.default-dispatcher-3] o.a.f.r.e.ExecutionGraph-[deploy] - 571 - Deploying Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1) (attempt #1) with attempt id 745fb84751e9c366343dced00b08e4dc to container_1620457281348_0044_01_000002 @ node1 (dataPort=43653) with allocation id 4d54ea84f546de5d16ba0250206bae74 [INFO ] 2021-06-05 10:36:59.079 [flink-akka.actor.default-dispatcher-3] o.a.f.r.e.ExecutionGraph-[transitionState] - 1437 - Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1) (745fb84751e9c366343dced00b08e4dc) switched from DEPLOYING to INITIALIZING. [INFO ] 2021-06-05 10:36:59.227 [flink-akka.actor.default-dispatcher-4] o.a.f.r.e.ExecutionGraph-[transitionState] - 1437 - Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1) (745fb84751e9c366343dced00b08e4dc) switched from INITIALIZING to RUNNING. [INFO ] 2021-06-05 10:37:03.774 [Checkpoint Timer] o.a.f.r.c.CheckpointCoordinator-[createPendingCheckpoint] - 741 - Triggering checkpoint 6500 (type=CHECKPOINT) @ 1622860623770 for job 63c270e00b69eb967f59479bb1c84113. [INFO ] 2021-06-05 10:37:03.787 [jobmanager-future-thread-1] o.a.f.r.c.CheckpointCoordinator-[completePendingCheckpoint] - 1250 - Completed checkpoint 6500 for job 63c270e00b69eb967f59479bb1c84113 (0 bytes in 15 ms). [INFO ] 2021-06-05 10:37:13.772 [Checkpoint Timer] o.a.f.r.c.CheckpointCoordinator-[createPendingCheckpoint] - 741 - Triggering checkpoint 6501 (type=CHECKPOINT) @ 1622860633770 for job 63c270e00b69eb967f59479bb1c84113. [INFO ] 2021-06-05 10:37:13.783 [jobmanager-future-thread-2] o.a.f.r.c.CheckpointCoordinator-[completePendingCheckpoint] - 1250 - Completed checkpoint 6501 for job 63c270e00b69eb967f59479bb1c84113 (0 bytes in 11 ms). [INFO ] 2021-06-05 10:37:23.771 [Checkpoint Timer] o.a.f.r.c.CheckpointCoordinator-[createPendingCheckpoint] - 741 - Triggering checkpoint 6502 (type=CHECKPOINT) @ 1622860643770 for job 63c270e00b69eb967f59479bb1c84113. [INFO ] 2021-06-05 10:37:23.782 [jobmanager-future-thread-4] o.a.f.r.c.CheckpointCoordinator-[completePendingCheckpoint] - 1250 - Completed checkpoint 6502 for job 63c270e00b69eb967f59479bb1c84113 (0 bytes in 11 ms). [INFO ] 2021-06-05 10:37:33.771 [Checkpoint Timer] o.a.f.r.c.CheckpointCoordinator-[createPendingCheckpoint] - 741 - Triggering checkpoint 6503 (type=CHECKPOINT) @ 1622860653770 for job 63c270e00b69eb967f59479bb1c84113. [INFO ] 2021-06-05 10:37:33.782 [jobmanager-future-thread-5] o.a.f.r.c.CheckpointCoordinator-[completePendingCheckpoint] - 1250 - Completed checkpoint 6503 for job 63c270e00b69eb967f59479bb1c84113 (0 bytes in 11 ms). [INFO ] 2021-06-05 10:37:43.771 [Checkpoint Timer] o.a.f.r.c.CheckpointCoordinator-[createPendingCheckpoint] - 741 - Triggering checkpoint 6504 (type=CHECKPOINT) @ 1622860663770 for job 63c270e00b69eb967f59479bb1c84113. [INFO ] 2021-06-05 10:38:09.410 [flink-akka.actor.default-dispatcher-4] o.a.f.r.e.ExecutionGraph-[transitionState] - 1445 - Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1) (745fb84751e9c366343dced00b08e4dc) switched from RUNNING to FAILED on container_1620457281348_0044_01_000002 @ node1 (dataPort=43653). java.lang.Exception: Could not perform checkpoint 6504 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#1. at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:976) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$7(StreamTask.java:936) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:344) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:330) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:202) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:661) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:623) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:776) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) at java.lang.Thread.run(Thread.java:748) Suppressed: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.close(ElasticsearchSinkBase.java:366) at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.dispose(AbstractUdfStreamOperator.java:117) at org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:837) at org.apache.flink.streaming.runtime.tasks.StreamTask.runAndSuppressThrowable(StreamTask.java:816) at org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUpInvoke(StreamTask.java:733) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.cleanUpInvoke(SourceStreamTask.java:186) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:637) ... 3 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-2 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete snapshot 6504 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#1. Failure reason: Checkpoint was declined. at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:264) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:169) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:371) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointStreamOperator(SubtaskCheckpointCoordinatorImpl.java:702) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.buildOperatorSnapshotFutures(SubtaskCheckpointCoordinatorImpl.java:623) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:586) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:308) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$8(StreamTask.java:1062) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1046) at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:964) ... 11 common frames omitted Caused by: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkAsyncErrorsAndRequests(ElasticsearchSinkBase.java:432) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.snapshotState(ElasticsearchSinkBase.java:340) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:118) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:89) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:218) ... 21 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-2 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted [INFO ] 2021-06-05 10:38:09.412 [flink-akka.actor.default-dispatcher-19] o.a.f.r.r.s.DeclarativeSlotManager-[processResourceRequirements] - 260 - Clearing resource requirements of job 63c270e00b69eb967f59479bb1c84113 [INFO ] 2021-06-05 10:38:09.413 [flink-akka.actor.default-dispatcher-4] o.a.f.r.e.f.f.RestartPipelinedRegionFailoverStrategy-[getTasksNeedingRestart] - 115 - Calculating tasks to restart to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. [INFO ] 2021-06-05 10:38:09.413 [flink-akka.actor.default-dispatcher-4] o.a.f.r.e.f.f.RestartPipelinedRegionFailoverStrategy-[getTasksNeedingRestart] - 152 - 1 tasks should be restarted to recover the failed task cbc357ccb763df2852fee8c4fc7d55f2_0. [INFO ] 2021-06-05 10:38:09.414 [flink-akka.actor.default-dispatcher-4] o.a.f.r.e.ExecutionGraph-[transitionState] - 1039 - Job insert-into_default_catalog.default_database.table3 (63c270e00b69eb967f59479bb1c84113) switched from state RUNNING to FAILING. org.apache.flink.runtime.JobException: Recovery is suppressed by FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=1, backoffTimeMS=60000) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:138) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:82) at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:207) at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:197) at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:188) at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:677) at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:79) at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:435) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at akka.actor.Actor$class.aroundReceive(Actor.scala:517) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) at akka.actor.ActorCell.invoke(ActorCell.scala:561) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) at akka.dispatch.Mailbox.run(Mailbox.scala:225) at akka.dispatch.Mailbox.exec(Mailbox.scala:235) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: java.lang.Exception: Could not perform checkpoint 6504 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#1. at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:976) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$7(StreamTask.java:936) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:344) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:330) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:202) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:661) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:623) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:776) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) at java.lang.Thread.run(Thread.java:748) Suppressed: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.close(ElasticsearchSinkBase.java:366) at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.dispose(AbstractUdfStreamOperator.java:117) at org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:837) at org.apache.flink.streaming.runtime.tasks.StreamTask.runAndSuppressThrowable(StreamTask.java:816) at org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUpInvoke(StreamTask.java:733) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.cleanUpInvoke(SourceStreamTask.java:186) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:637) ... 3 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-2 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete snapshot 6504 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#1. Failure reason: Checkpoint was declined. at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:264) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:169) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:371) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointStreamOperator(SubtaskCheckpointCoordinatorImpl.java:702) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.buildOperatorSnapshotFutures(SubtaskCheckpointCoordinatorImpl.java:623) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:586) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:308) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$8(StreamTask.java:1062) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1046) at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:964) ... 11 common frames omitted Caused by: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkAsyncErrorsAndRequests(ElasticsearchSinkBase.java:432) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.snapshotState(ElasticsearchSinkBase.java:340) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:118) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:89) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:218) ... 21 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-2 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted [INFO ] 2021-06-05 10:38:09.426 [flink-akka.actor.default-dispatcher-4] o.a.f.r.e.ExecutionGraph-[transitionState] - 1039 - Job insert-into_default_catalog.default_database.table3 (63c270e00b69eb967f59479bb1c84113) switched from state FAILING to FAILED. org.apache.flink.runtime.JobException: Recovery is suppressed by FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=1, backoffTimeMS=60000) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:138) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:82) at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:207) at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:197) at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:188) at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:677) at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:79) at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:435) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at akka.actor.Actor$class.aroundReceive(Actor.scala:517) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) at akka.actor.ActorCell.invoke(ActorCell.scala:561) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) at akka.dispatch.Mailbox.run(Mailbox.scala:225) at akka.dispatch.Mailbox.exec(Mailbox.scala:235) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: java.lang.Exception: Could not perform checkpoint 6504 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#1. at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:976) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$7(StreamTask.java:936) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:344) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:330) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:202) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:661) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:623) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:776) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563) at java.lang.Thread.run(Thread.java:748) Suppressed: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.close(ElasticsearchSinkBase.java:366) at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.dispose(AbstractUdfStreamOperator.java:117) at org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:837) at org.apache.flink.streaming.runtime.tasks.StreamTask.runAndSuppressThrowable(StreamTask.java:816) at org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUpInvoke(StreamTask.java:733) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.cleanUpInvoke(SourceStreamTask.java:186) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:637) ... 3 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-2 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete snapshot 6504 for operator Source: TableSourceScan(table=[[default_catalog, default_database, table1]], fields=[MotorVehicleID, InfoKind, SourceID, DeviceID, LocationID, PlaceCode, StorageUrl1, StorageUrl2, StorageUrl3, StorageUrl4, StorageUrl5, LeftTopX, LeftTopY, RightBtmX, RightBtmY, CapDirection, ShotTime, MarkTime, AppearTime, DisappearTime, LaneNo, HasPlate, PlateColor, PlateNo, PlateNo2, LicensePlateStatus, PlateClass, Feature, VehicleTypeID, Speed, Direction, VehicleModel2, VehicleColor, VehicleFrontItem, DescOfFrontItem, VehicleRearItem, DescOfRearItem, Sunvisor, SafetyBelt, Calling, OccupantRight, SunvisorLeft, SeatBeltLeft, CallingLeft, OccupantLeft, RoofRack, Sunroof, Graffiti, ISFace, Faceinfo, Coloridleft, Coloridright, KeyVehiclesClassID, KeyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, Vendor, FeatureData, Version, TaskID, FaceID, hitList, ruleId, score]) -> Calc(select=[CAST(LOCALTIMESTAMP()) AS insertTime, MotorVehicleID AS motorVehicleID, InfoKind AS infoKind, SourceID AS sourceID, DeviceID AS deviceID, LocationID AS locationID, PlaceCode AS placeCode, StorageUrl1 AS storageUrl1, StorageUrl2 AS storageUrl2, StorageUrl3 AS storageUrl3, StorageUrl4 AS storageUrl4, StorageUrl5 AS storageUrl5, LeftTopX AS leftTopX, LeftTopY AS leftTopY, RightBtmX AS rightBtmX, RightBtmY AS rightBtmY, CapDirection AS capDirection, CAST(TO_TIMESTAMP(ShotTime)) AS shotTime, CAST(TO_TIMESTAMP(MarkTime)) AS markTime, CAST(TO_TIMESTAMP(AppearTime)) AS appearTime, CAST(TO_TIMESTAMP(DisappearTime)) AS disappearTime, LaneNo AS laneNo, HasPlate AS hasPlate, PlateColor AS plateColor, PlateNo AS plateNo, PlateNo2 AS plateNo2, LicensePlateStatus AS licensePlateStatus, PlateClass AS plateClass, Feature AS feature, VehicleTypeID AS vehicleTypeID, Speed AS speed, Direction AS direction, VehicleModel2 AS vehicleModel2, VehicleColor AS vehicleColor, VehicleFrontItem AS vehicleFrontItem, DescOfFrontItem AS descOfFrontItem, VehicleRearItem AS vehicleRearItem, DescOfRearItem AS descOfRearItem, Sunvisor AS sunvisor, SafetyBelt AS safetyBelt, Calling AS calling, OccupantRight AS occupantRight, SunvisorLeft AS sunvisorLeft, SeatBeltLeft AS seatBeltLeft, CallingLeft AS callingLeft, OccupantLeft AS occupantLeft, RoofRack AS roofRack, Sunroof AS sunroof, Graffiti AS graffiti, ISFace AS isFace, Faceinfo AS faceinfo, Coloridleft AS coloridleft, Coloridright AS coloridright, KeyVehiclesClassID AS keyVehiclesClassID, KeyVehiclesTypeID AS keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, CAST(TO_TIMESTAMP(uploadTime)) AS uploadTime, longitude, latitude, isRecognition, Vendor AS vendor, FeatureData AS featureData, Version AS version, TaskID AS taskID, FaceID AS faceID, hitList, ruleId, score]) -> Sink: Sink(table=[default_catalog.default_database.table3], fields=[insertTime, motorVehicleID, infoKind, sourceID, deviceID, locationID, placeCode, storageUrl1, storageUrl2, storageUrl3, storageUrl4, storageUrl5, leftTopX, leftTopY, rightBtmX, rightBtmY, capDirection, shotTime, markTime, appearTime, disappearTime, laneNo, hasPlate, plateColor, plateNo, plateNo2, licensePlateStatus, plateClass, feature, vehicleTypeID, speed, direction, vehicleModel2, vehicleColor, vehicleFrontItem, descOfFrontItem, vehicleRearItem, descOfRearItem, sunvisor, safetyBelt, calling, occupantRight, sunvisorLeft, seatBeltLeft, callingLeft, occupantLeft, roofRack, sunroof, graffiti, isFace, faceinfo, coloridleft, coloridright, keyVehiclesClassID, keyVehiclesTypeID, apiversion, manufacturer, dataSource, addrName, snaptype, plateNumZoom, video, uploadType, uploadTime, longitude, latitude, isRecognition, vendor, featureData, version, taskID, faceID, hitList, ruleId, score]) (1/1)#1. Failure reason: Checkpoint was declined. at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:264) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:169) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:371) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointStreamOperator(SubtaskCheckpointCoordinatorImpl.java:702) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.buildOperatorSnapshotFutures(SubtaskCheckpointCoordinatorImpl.java:623) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:586) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:308) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$8(StreamTask.java:1062) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:93) at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1046) at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpoint(StreamTask.java:964) ... 11 common frames omitted Caused by: java.lang.RuntimeException: An error occurred in ElasticsearchSink. at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkAsyncErrorsAndRequests(ElasticsearchSinkBase.java:432) at org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.snapshotState(ElasticsearchSinkBase.java:340) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.trySnapshotFunctionState(StreamingFunctionUtils.java:118) at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.snapshotFunctionState(StreamingFunctionUtils.java:99) at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.snapshotState(AbstractUdfStreamOperator.java:89) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:218) ... 21 common frames omitted Caused by: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-2 [ACTIVE] at org.apache.flink.elasticsearch7.shaded.org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.flink.elasticsearch7.shaded.org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ... 1 common frames omitted [INFO ] 2021-06-05 10:38:09.428 [flink-akka.actor.default-dispatcher-4] o.a.f.r.c.CheckpointCoordinator-[shutdown] - 405 - Stopping checkpoint coordinator for job 63c270e00b69eb967f59479bb1c84113. [INFO ] 2021-06-05 10:38:09.428 [flink-akka.actor.default-dispatcher-4] o.a.f.r.c.StandaloneCompletedCheckpointStore-[shutdown] - 96 - Shutting down [INFO ] 2021-06-05 10:38:09.451 [flink-akka.actor.default-dispatcher-2] o.a.f.r.dispatcher.MiniDispatcher-[jobReachedGloballyTerminalState] - 827 - Job 63c270e00b69eb967f59479bb1c84113 reached globally terminal state FAILED. [INFO ] 2021-06-05 10:38:09.452 [flink-akka.actor.default-dispatcher-2] o.a.f.r.dispatcher.MiniDispatcher-[jobReachedGloballyTerminalState] - 132 - Shutting down cluster with state FAILED, jobCancelled: false, executionMode: DETACHED [INFO ] 2021-06-05 10:38:09.453 [flink-akka.actor.default-dispatcher-2] o.a.f.r.entrypoint.ClusterEntrypoint-[shutDownAsync] - 481 - Shutting YarnJobClusterEntrypoint down with application status FAILED. Diagnostics null. [INFO ] 2021-06-05 10:38:09.453 [flink-akka.actor.default-dispatcher-2] o.a.f.r.j.MiniDispatcherRestEndpoint-[closeAsync] - 309 - Shutting down rest endpoint. [INFO ] 2021-06-05 10:38:09.463 [flink-akka.actor.default-dispatcher-4] o.a.f.runtime.jobmaster.JobMaster-[onStop] - 395 - Stopping the JobMaster for job insert-into_default_catalog.default_database.table3(63c270e00b69eb967f59479bb1c84113). [INFO ] 2021-06-05 10:38:09.472 [flink-akka.actor.default-dispatcher-4] o.a.f.r.j.s.DefaultDeclarativeSlotPool-[releaseSlots] - 440 - Releasing slot [4d54ea84f546de5d16ba0250206bae74]. [INFO ] 2021-06-05 10:38:09.473 [flink-akka.actor.default-dispatcher-4] o.a.f.runtime.jobmaster.JobMaster-[dissolveResourceManagerConnection] - 1124 - Close ResourceManager connection 77a736a0475ef43c2d92a082c47c7a1b: Stopping JobMaster for job insert-into_default_catalog.default_database.table3(63c270e00b69eb967f59479bb1c84113).. [INFO ] 2021-06-05 10:38:09.476 [flink-akka.actor.default-dispatcher-2] o.a.f.r.r.s.DeclarativeSlotManager-[processResourceRequirements] - 260 - Clearing resource requirements of job 63c270e00b69eb967f59479bb1c84113 [INFO ] 2021-06-05 10:38:09.477 [flink-akka.actor.default-dispatcher-2] o.a.f.r.r.a.ActiveResourceManager-[closeJobManagerConnection] - 1047 - Disconnect job manager 00000000000000000000000000000000@akka.tcp://flink@node1:34800/user/rpc/jobmanager_2 for job 63c270e00b69eb967f59479bb1c84113 from the resource manager. [INFO ] 2021-06-05 10:38:09.484 [ForkJoinPool.commonPool-worker-4] o.a.f.r.j.MiniDispatcherRestEndpoint-[lambda$shutDownInternal$5] - 964 - Removing cache directory /tmp/flink-web-027bae9f-9a43-4eb2-8c39-4b5fcdeab64d/flink-web-ui [INFO ] 2021-06-05 10:38:09.487 [ForkJoinPool.commonPool-worker-4] o.a.f.r.j.MiniDispatcherRestEndpoint-[revokeLeadership] - 997 - http://node1:46421 lost leadership [INFO ] 2021-06-05 10:38:09.487 [ForkJoinPool.commonPool-worker-4] o.a.f.r.j.MiniDispatcherRestEndpoint-[lambda$closeAsync$1] - 317 - Shut down complete. [INFO ] 2021-06-05 10:38:09.488 [flink-akka.actor.default-dispatcher-2] o.a.f.r.r.a.ActiveResourceManager-[deregisterApplication] - 629 - Shut down cluster because application is in FAILED, diagnostics null. [INFO ] 2021-06-05 10:38:09.489 [flink-akka.actor.default-dispatcher-2] o.a.f.yarn.YarnResourceManagerDriver-[deregisterApplication] - 221 - Unregister application from the YARN Resource Manager with final status FAILED. [INFO ] 2021-06-05 10:38:09.598 [flink-akka.actor.default-dispatcher-2] o.a.f.r.e.c.DispatcherResourceManagerComponent-[closeAsyncInternal] - 162 - Closing components. [INFO ] 2021-06-05 10:38:09.599 [flink-akka.actor.default-dispatcher-2] o.a.f.r.d.r.JobDispatcherLeaderProcess-[closeInternal] - 134 - Stopping JobDispatcherLeaderProcess. [INFO ] 2021-06-05 10:38:09.599 [flink-akka.actor.default-dispatcher-19] o.a.f.r.dispatcher.MiniDispatcher-[onStop] - 266 - Stopping dispatcher akka.tcp://flink@node1:34800/user/rpc/dispatcher_1. [INFO ] 2021-06-05 10:38:09.599 [flink-akka.actor.default-dispatcher-19] o.a.f.r.dispatcher.MiniDispatcher-[terminateRunningJobs] - 789 - Stopping all currently running jobs of dispatcher akka.tcp://flink@node1:34800/user/rpc/dispatcher_1. [INFO ] 2021-06-05 10:38:09.601 [flink-akka.actor.default-dispatcher-19] o.a.f.r.dispatcher.MiniDispatcher-[lambda$onStop$0] - 278 - Stopped dispatcher akka.tcp://flink@node1:34800/user/rpc/dispatcher_1. [INFO ] 2021-06-05 10:38:09.614 [flink-akka.actor.default-dispatcher-4] o.a.f.r.r.s.DeclarativeSlotManager-[close] - 238 - Closing the slot manager. [INFO ] 2021-06-05 10:38:09.614 [flink-akka.actor.default-dispatcher-4] o.a.f.r.r.s.DeclarativeSlotManager-[suspend] - 212 - Suspending the slot manager. [INFO ] 2021-06-05 10:38:09.616 [AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1] o.a.flink.runtime.blob.BlobServer-[close] - 345 - Stopped BLOB server at 0.0.0.0:35819 [INFO ] 2021-06-05 10:38:09.616 [AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1] o.a.f.r.rpc.akka.AkkaRpcService-[stopService] - 403 - Stopping Akka RPC service. [INFO ] 2021-06-05 10:38:09.619 [AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1] o.a.f.r.rpc.akka.AkkaRpcService-[stopService] - 403 - Stopping Akka RPC service. [INFO ] 2021-06-05 10:38:09.628 [flink-metrics-15] a.r.RemoteActorRefProvider$RemotingTerminator-[apply$mcV$sp] - 83 - Shutting down remote daemon. [INFO ] 2021-06-05 10:38:09.628 [flink-akka.actor.default-dispatcher-19] a.r.RemoteActorRefProvider$RemotingTerminator-[apply$mcV$sp] - 83 - Shutting down remote daemon. [INFO ] 2021-06-05 10:38:09.630 [flink-akka.actor.default-dispatcher-19] a.r.RemoteActorRefProvider$RemotingTerminator-[apply$mcV$sp] - 83 - Remote daemon shut down; proceeding with flushing remote transports. [INFO ] 2021-06-05 10:38:09.631 [flink-metrics-15] a.r.RemoteActorRefProvider$RemotingTerminator-[apply$mcV$sp] - 83 - Remote daemon shut down; proceeding with flushing remote transports. [INFO ] 2021-06-05 10:38:09.643 [flink-akka.actor.default-dispatcher-19] a.r.RemoteActorRefProvider$RemotingTerminator-[apply$mcV$sp] - 83 - Remoting shut down. [INFO ] 2021-06-05 10:38:09.643 [flink-metrics-15] a.r.RemoteActorRefProvider$RemotingTerminator-[apply$mcV$sp] - 83 - Remoting shut down. [INFO ] 2021-06-05 10:38:09.652 [flink-metrics-15] o.a.f.r.rpc.akka.AkkaRpcService-[lambda$stopService$7] - 427 - Stopped Akka RPC service. [INFO ] 2021-06-05 10:38:09.655 [flink-akka.actor.default-dispatcher-2] o.a.f.r.rpc.akka.AkkaRpcService-[lambda$stopService$7] - 427 - Stopped Akka RPC service. [INFO ] 2021-06-05 10:38:09.655 [main] o.a.f.r.entrypoint.ClusterEntrypoint-[runClusterEntrypoint] - 618 - Terminating cluster entrypoint process YarnJobClusterEntrypoint with exit code 1443.