Flink Streaming ContinuousTimeTriggers

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink Streaming ContinuousTimeTriggers

Ankur Sharma
Hey,

Can you guide me to some example of ContinuousProcessingTimeTrigger?
I want to partition input stream into TimeWindow that should fire at continuous time interval on its on without waiting for a new element to enter the stream.
Could you guide me to it?


Thanks

Best,
Ankur Sharma
Information Systems Group
3.15 E1.1 Universität des Saarlandes
66123, Saarbrücken Germany
Email: [hidden email] 
            [hidden email]

smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Flink Streaming ContinuousTimeTriggers

Gavin Lin
Hi, 

how about this video ? 

Gavin.Lin

2016-02-25 3:55 GMT+08:00 Ankur Sharma <[hidden email]>:
Hey,

Can you guide me to some example of ContinuousProcessingTimeTrigger?
I want to partition input stream into TimeWindow that should fire at continuous time interval on its on without waiting for a new element to enter the stream.
Could you guide me to it?


Thanks

Best,
Ankur Sharma
Information Systems Group
3.15 E1.1 Universität des Saarlandes
66123, Saarbrücken Germany
Email: [hidden email] 
            [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Flink Streaming ContinuousTimeTriggers

Aljoscha Krettek
Hi,
in addition to the video I would also like to point out that the Continuous triggers should only be used with the GlobalWindows assigner. For TimeWindows the right thing to do would be using the EventTimeTrigger (or ProcessingTimeTrigger in case you are doing processing time windows).

Could you maybe go into some detail of what you are trying to achieve? Then we can figure out what a good strategy would be.

Cheers,
Aljoscha

> On 25 Feb 2016, at 04:27, Gavin Lin <[hidden email]> wrote:
>
> Hi,
>
> how about this video ?
> https://www.youtube.com/watch?v=T7hiwcwCXGI
>
> Gavin.Lin
>
> 2016-02-25 3:55 GMT+08:00 Ankur Sharma <[hidden email]>:
> Hey,
>
> Can you guide me to some example of ContinuousProcessingTimeTrigger?
> I want to partition input stream into TimeWindow that should fire at continuous time interval on its on without waiting for a new element to enter the stream.
> Could you guide me to it?
>
>
> Thanks
>
> Best,
> Ankur Sharma
> Information Systems Group
> 3.15 E1.1 Universität des Saarlandes
> 66123, Saarbrücken Germany
> Email: [hidden email]
>             [hidden email]
>

Reply | Threaded
Open this post in threaded view
|

Re: Flink Streaming ContinuousTimeTriggers

Ankur Sharma
Hello,

Thanks for the reply.

Target: I have a stream of tuples(append-only, no updates) that are appended to a file. These records have priority that is changing over time and is calculated as a function of the tuple’s timestamp and current system time. Once any tuple’s priority reaches some MIN, I want to throw it away.
Using this priority I want to select the next job (given by tuple) that is scheduled. These tasks have to be periodically scheduled until they are removed from the queue completely.

Let me know if something is unclear.

Best,
Ankur Sharma
Information Systems Group
3.15 E1.1 Universität des Saarlandes
66123, Saarbrücken Germany
Email: [hidden email] 
            [hidden email]

On 25 Feb 2016, at 11:05, Aljoscha Krettek <[hidden email]> wrote:

Hi,
in addition to the video I would also like to point out that the Continuous triggers should only be used with the GlobalWindows assigner. For TimeWindows the right thing to do would be using the EventTimeTrigger (or ProcessingTimeTrigger in case you are doing processing time windows).

Could you maybe go into some detail of what you are trying to achieve? Then we can figure out what a good strategy would be.

Cheers,
Aljoscha
On 25 Feb 2016, at 04:27, Gavin Lin <[hidden email]> wrote:

Hi,

how about this video ?
https://www.youtube.com/watch?v=T7hiwcwCXGI

Gavin.Lin

2016-02-25 3:55 GMT+08:00 Ankur Sharma <[hidden email]>:
Hey,

Can you guide me to some example of ContinuousProcessingTimeTrigger?
I want to partition input stream into TimeWindow that should fire at continuous time interval on its on without waiting for a new element to enter the stream.
Could you guide me to it?


Thanks

Best,
Ankur Sharma
Information Systems Group
3.15 E1.1 Universität des Saarlandes
66123, Saarbrücken Germany
Email: [hidden email]
           [hidden email]



smime.p7s (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Flink Streaming ContinuousTimeTriggers

Aljoscha Krettek
Hi,
sorry for the long wait, we are currently very busy with finalizing the 1.0 release.

I think I don’t yet completely get your use case. Could you maybe give some example input data and what the expected behavior would be? And maybe some code? If you already have coded something.

Cheers,
Aljoscha

> On 25 Feb 2016, at 13:46, Ankur Sharma <[hidden email]> wrote:
>
> Hello,
>
> Thanks for the reply.
>
> Target: I have a stream of tuples(append-only, no updates) that are appended to a file. These records have priority that is changing over time and is calculated as a function of the tuple’s timestamp and current system time. Once any tuple’s priority reaches some MIN, I want to throw it away.
> Using this priority I want to select the next job (given by tuple) that is scheduled. These tasks have to be periodically scheduled until they are removed from the queue completely.
>
> Let me know if something is unclear.
>
> Best,
> Ankur Sharma
> Information Systems Group
> 3.15 E1.1 Universität des Saarlandes
> 66123, Saarbrücken Germany
> Email: [hidden email]
>             [hidden email]
>
>> On 25 Feb 2016, at 11:05, Aljoscha Krettek <[hidden email]> wrote:
>>
>> Hi,
>> in addition to the video I would also like to point out that the Continuous triggers should only be used with the GlobalWindows assigner. For TimeWindows the right thing to do would be using the EventTimeTrigger (or ProcessingTimeTrigger in case you are doing processing time windows).
>>
>> Could you maybe go into some detail of what you are trying to achieve? Then we can figure out what a good strategy would be.
>>
>> Cheers,
>> Aljoscha
>>> On 25 Feb 2016, at 04:27, Gavin Lin <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> how about this video ?
>>> https://www.youtube.com/watch?v=T7hiwcwCXGI
>>>
>>> Gavin.Lin
>>>
>>> 2016-02-25 3:55 GMT+08:00 Ankur Sharma <[hidden email]>:
>>> Hey,
>>>
>>> Can you guide me to some example of ContinuousProcessingTimeTrigger?
>>> I want to partition input stream into TimeWindow that should fire at continuous time interval on its on without waiting for a new element to enter the stream.
>>> Could you guide me to it?
>>>
>>>
>>> Thanks
>>>
>>> Best,
>>> Ankur Sharma
>>> Information Systems Group
>>> 3.15 E1.1 Universität des Saarlandes
>>> 66123, Saarbrücken Germany
>>> Email: [hidden email]
>>>            [hidden email]
>>>
>