Hello,
I’m trying to push some behavior that we’ve currently got in a large, stateful SinkFunction implementation into Flink’s windowing system. The task at hand is similar to what StreamingFileSink provides, but more flexible. I don’t want to re-implement that sink, because it uses the StreamingRuntimeContext.getProcessingTimeService() via a cast - that class is marked as internal, and I’d like to avoid the exposure to an interface that could change. Extending it similarly introduces complexity I would rather not add to our codebase.
WindowedStream.process() provides more or less the pieces I need, but the stream continues on after a ProcessFunction - there’s no way to process() directly into a sink. I could use a ProcessFunction[In, Unit, Key, Window], and follow that immediately with a no-op sink that discards the Unit values, or I could just leave the stream “unfinished," with no sink.
Is there a downside to either of these approaches? Is there anything special about doing sink-like work in a ProcessFunction or FlatMapFunction instead of a SinkFunction?
Thanks,
Andrew
--
*Confidentiality Notice: The information contained in this e-mail and any
attachments may be confidential. If you are not an intended recipient, you
are hereby notified that any dissemination, distribution or copying of this
e-mail is strictly prohibited. If you have received this e-mail in error,
please notify the sender and permanently delete the e-mail and any
attachments immediately. You should not retain, copy or use this e-mail or
any attachment for any purpose, nor disclose all or any part of the
contents to any other person. Thank you.*
Konstantin Knauf | Solutions Architect
+49 160 91394525
Follow us @VervericaData Ververica
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time
--
Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
--
Ververica GmbHFree forum by Nabble | Edit this page |