Re: Problem producing to Kinesis

Posted by Lasse Nedergaard on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Problem-producing-to-Kinesis-tp20730p20731.html

Hi. 

We see the same error and to my understanding it’s a known error from Amazon. See. https://github.com/awslabs/amazon-kinesis-producer/issues/39#issuecomment-396219522

We don’t have a workaround and haven’t found the reason for the exception. It is one off the reason why we move to Kafka in the near future. 

Med venlig hilsen / Best regards

Lasse Nedergaard

Den 14. jun. 2018 kl. 20.24 skrev Alexey Tsitkin <[hidden email]>:

  Hi,

I'm trying to run a simple program which consumes from one kinesis stream, does a simple transformation, and produces to another stream.

Running on Flink 1.4.0.

Code can be seen here (if needed I can also paste it directly on this thread):

https://stackoverflow.com/questions/50847164/flink-producing-to-kinesis-not-working

Consuming the source stream works great, but trying to use the producer causes the exception:

org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.DaemonException: The child process has been shutdown and can no longer accept messages.

    at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.Daemon.add(Daemon.java:176)

    at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.KinesisProducer.addUserRecord(KinesisProducer.java:477)

    at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer.invoke(FlinkKinesisProducer.java:248)

    ...

Did anyone have something similar?

Or is there any way to debug the daemon itself, to understand the source of the error?

As you can see, this is a trivial example, which I mostly copy-pasted from the documentation.

Thanks,

Alexey