Adaptive Watermarks Generator

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Adaptive Watermarks Generator

sanmutongzi
Hi , I noticed there is a paper describe about adaptive watermark generator on top of Apache Flink v1.6.2 :https://www.dfki.de/fileadmin/user_upload/import/10260_awad-adaptive-watermarks.pdf

This looks like a more precise generator with much less data drop . Does anybody know more detail about this or does Flink community have any plan about this?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Adaptive Watermarks Generator

rmetzger0
Hi,
I haven't seen this paper before. I'm not aware of any efforts in the community to integrate this watermark generator.
I'm also not sure if we would integrate it directory into the main Flink repository. What I could see is this generator being maintained in a public GitHub repository, linked from flink-packages.org

Best,
Robert

On Sat, May 9, 2020 at 8:47 AM 杨东晓 <[hidden email]> wrote:
Hi , I noticed there is a paper describe about adaptive watermark generator on top of Apache Flink v1.6.2 :https://www.dfki.de/fileadmin/user_upload/import/10260_awad-adaptive-watermarks.pdf

This looks like a more precise generator with much less data drop . Does anybody know more detail about this or does Flink community have any plan about this?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Adaptive Watermarks Generator

sanmutongzi
Thanks Robert! Could you point out the repository name ? I didn't find it from  flink-packages.org  Packages list.

Robert Metzger <[hidden email]> 于2020年5月19日周二 下午5:02写道:
Hi,
I haven't seen this paper before. I'm not aware of any efforts in the community to integrate this watermark generator.
I'm also not sure if we would integrate it directory into the main Flink repository. What I could see is this generator being maintained in a public GitHub repository, linked from flink-packages.org

Best,
Robert

On Sat, May 9, 2020 at 8:47 AM 杨东晓 <[hidden email]> wrote:
Hi , I noticed there is a paper describe about adaptive watermark generator on top of Apache Flink v1.6.2 :https://www.dfki.de/fileadmin/user_upload/import/10260_awad-adaptive-watermarks.pdf

This looks like a more precise generator with much less data drop . Does anybody know more detail about this or does Flink community have any plan about this?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Adaptive Watermarks Generator

Fabian Hueske-2
Hi,

The code of the implementation is linked in the paper: https://github.com/DataSystemsGroupUT/Adaptive-Watermarks
Since this is a prototype for a research paper, I'm doubtful that the project is maintained.
I also didn't find an open-source license attached to the code.
Hence adding the project to flink-packages.org wouldn't help a lot.

One option would be to get permission to fork the project (by adding a permissive open source license) and finding somebody who forks and maintains the code.
Alternatively, it might be possible to create a completely new implementation based on the paper (I guess).
Then it could be added to flink-packages.org.

Best, Fabian

Am Fr., 22. Mai 2020 um 09:01 Uhr schrieb 杨东晓 <[hidden email]>:
Thanks Robert! Could you point out the repository name ? I didn't find it from  flink-packages.org  Packages list.

Robert Metzger <[hidden email]> 于2020年5月19日周二 下午5:02写道:
Hi,
I haven't seen this paper before. I'm not aware of any efforts in the community to integrate this watermark generator.
I'm also not sure if we would integrate it directory into the main Flink repository. What I could see is this generator being maintained in a public GitHub repository, linked from flink-packages.org

Best,
Robert

On Sat, May 9, 2020 at 8:47 AM 杨东晓 <[hidden email]> wrote:
Hi , I noticed there is a paper describe about adaptive watermark generator on top of Apache Flink v1.6.2 :https://www.dfki.de/fileadmin/user_upload/import/10260_awad-adaptive-watermarks.pdf

This looks like a more precise generator with much less data drop . Does anybody know more detail about this or does Flink community have any plan about this?

Thanks!