Timewindow watermarks specific to keys of stream

[madhu phatak]

Hi,

I am trying to analyse the stock market data. In this, for every symbol i want to find max of stock price in last 10 mins. I want to generate watermarks specific to key rather than across the stream. Is this possible in flink?

--

[Jamie Grier]

Hi Madhu,

This is not possible right now but are you sure this is necessary in your application?  Would the timestamps for stock data really be radically different for different symbols that occur close together in the input stream.  The windows themselves are for each key but event time advances at the operator level, not the key level.  For most applications this works very well.  If you explain more about why this won't work for you maybe I can help more.

-Jamie

On Tue, Jul 5, 2016 at 1:14 AM, madhu phatak <[hidden email]> wrote:

Hi,

I am trying to analyse the stock market data. In this, for every symbol i want to find max of stock price in last 10 mins. I want to generate watermarks specific to key rather than across the stream. Is this possible in flink?

--

--

[madhu phatak]

Hi,

It makes sense now. Thank  you.

On Wed, Jul 6, 2016 at 8:05 AM, Jamie Grier <[hidden email]> wrote:

Hi Madhu,

This is not possible right now but are you sure this is necessary in your application?  Would the timestamps for stock data really be radically different for different symbols that occur close together in the input stream.  The windows themselves are for each key but event time advances at the operator level, not the key level.  For most applications this works very well.  If you explain more about why this won't work for you maybe I can help more.

-Jamie

On Tue, Jul 5, 2016 at 1:14 AM, madhu phatak <[hidden email]> wrote:

Hi,

I am trying to analyse the stock market data. In this, for every symbol i want to find max of stock price in last 10 mins. I want to generate watermarks specific to key rather than across the stream. Is this possible in flink?

--

--

--