question on sideoutput from ProcessWindow function

Hi,

Can we have a side output from a process window function ?

I am currently genrating a side output from a process function. The main output of the process function is then Windowed and a ProcessWindowFunction is applied on the windows.

Can I add to the SideOutpuStream from the ProcessWindowFunction. I am unable to find any api that enables this functionality.

Thanks,

Prabhu

[Chen Qin]

Hi Prabhu,

That is good question, the short answer is not yet, only ProcessFunction was given flexibility of doing customized sideoutput at the moment.

Window Function wasn't given such flexibility partially due to sideoutput initially targeting late arriving event for window use cases. 

+@Aljoscha might have better picture on this question.

Thanks,

Chen

On Fri, Sep 8, 2017 at 7:19 PM, Prabhu V <[hidden email]> wrote:

Hi,

Can we have a side output from a process window function ?

I am currently genrating a side output from a process function. The main output of the process function is then Windowed and a ProcessWindowFunction is applied on the windows.

Can I add to the SideOutpuStream from the ProcessWindowFunction. I am unable to find any api that enables this functionality.

Thanks,

Prabhu

[Aljoscha Krettek]

Hi,

Chen is correct! I think it would be nice, though, to also add that functionality for ProcessWindowFunction and I think this should be easy to do since the interface is very similar to ProcessFunction and we could also add that to the Context.

Best,

Aljoscha

On 9. Sep 2017, at 06:22, Chen Qin <[hidden email]> wrote:

Hi Prabhu,

That is good question, the short answer is not yet, only ProcessFunction was given flexibility of doing customized sideoutput at the moment.

Window Function wasn't given such flexibility partially due to sideoutput initially targeting late arriving event for window use cases. 

+@Aljoscha might have better picture on this question.

Thanks,

Chen

On Fri, Sep 8, 2017 at 7:19 PM, Prabhu V <[hidden email]> wrote:

Hi,

Can we have a side output from a process window function ?

I am currently genrating a side output from a process function. The main output of the process function is then Windowed and a ProcessWindowFunction is applied on the windows.

Can I add to the SideOutpuStream from the ProcessWindowFunction. I am unable to find any api that enables this functionality.

Thanks,

Prabhu

[Ufuk Celebi]

+1

Created an issue here: https://issues.apache.org/jira/browse/FLINK-7677


On Thu, Sep 14, 2017 at 11:51 AM, Aljoscha Krettek <[hidden email]> wrote:

> Hi,
>
> Chen is correct! I think it would be nice, though, to also add that
> functionality for ProcessWindowFunction and I think this should be easy to
> do since the interface is very similar to ProcessFunction and we could also
> add that to the Context.
>
> Best,
> Aljoscha
>
> On 9. Sep 2017, at 06:22, Chen Qin <[hidden email]> wrote:
>
> Hi Prabhu,
>
> That is good question, the short answer is not yet, only ProcessFunction was
> given flexibility of doing customized sideoutput at the moment.
> Window Function wasn't given such flexibility partially due to sideoutput
> initially targeting late arriving event for window use cases.
>
> +@Aljoscha might have better picture on this question.
>
> Thanks,
> Chen
>
> On Fri, Sep 8, 2017 at 7:19 PM, Prabhu V <[hidden email]> wrote:
>>
>> Hi,
>>
>> Can we have a side output from a process window function ?
>>
>> I am currently genrating a side output from a process function. The main
>> output of the process function is then Windowed and a ProcessWindowFunction
>> is applied on the windows.
>>
>> Can I add to the SideOutpuStream from the ProcessWindowFunction. I am
>> unable to find any api that enables this functionality.
>>
>> Thanks,
>> Prabhu
>
>
>

[Chen Qin]

Be lated update.

actually @phoenixjiangnan is already working on this

​https://issues.apache.org/jira/browse/FLINK-7635

On Sat, Sep 23, 2017 at 8:26 AM, Ufuk Celebi <[hidden email]> wrote:

+1

Created an issue here: https://issues.apache.org/jira/browse/FLINK-7677


On Thu, Sep 14, 2017 at 11:51 AM, Aljoscha Krettek <[hidden email]> wrote:
> Hi,
>
> Chen is correct! I think it would be nice, though, to also add that
> functionality for ProcessWindowFunction and I think this should be easy to
> do since the interface is very similar to ProcessFunction and we could also
> add that to the Context.
>
> Best,
> Aljoscha
>
> On 9. Sep 2017, at 06:22, Chen Qin <[hidden email]> wrote:
>
> Hi Prabhu,
>
> That is good question, the short answer is not yet, only ProcessFunction was
> given flexibility of doing customized sideoutput at the moment.
> Window Function wasn't given such flexibility partially due to sideoutput
> initially targeting late arriving event for window use cases.
>
> +@Aljoscha might have better picture on this question.
>
> Thanks,
> Chen
>
> On Fri, Sep 8, 2017 at 7:19 PM, Prabhu V <[hidden email]> wrote:
>>
>> Hi,
>>
>> Can we have a side output from a process window function ?
>>
>> I am currently genrating a side output from a process function. The main
>> output of the process function is then Windowed and a ProcessWindowFunction
>> is applied on the windows.
>>
>> Can I add to the SideOutpuStream from the ProcessWindowFunction. I am
>> unable to find any api that enables this functionality.
>>
>> Thanks,
>> Prabhu
>
>
>