Prasanna,
The Flink project does not have an SQS connector, and a quick google search hasn't found one. Nor does Flink have an HTTP sink, but with a bit of googling you can find that various folks have implemented this themselves.
As for implementing SQS as a custom sink, if you need exactly once guarantees I believe you will need to limit yourself to using SQS FIFO queues, and you'll need to attach a MessageDeduplicationId to each record. SQS offers to do deduplication for you, but only supports deduplication within a 5 minute window -- which I would find too short to rely on during recovery or redeployment.
Regards,
David
On Fri, Jul 17, 2020 at 9:55 PM Prasanna kumar <
[hidden email]> wrote:
Hi ,
I did not find out of box flink sink connector for http and SQS mechanism.
Has anyone implemented it?
Wanted to know if we are writing a custom sink function , whether it would affect semantic exactly one guarantees ?
Thanks ,
Prasanna
Locked
