Specific sink behaviour based on tuple key

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Specific sink behaviour based on tuple key

Alexis Gendronneau
Hi all,

I am looking to customize a sink behaviour based on tuple key.

My goal is to have each sink partition to write to a specific output.

My job will be something like :

input -> keyBy() -> outputSink

This output should be something like './base/path/keyValue/....'.

But I was not able to find where key value is stored, could you help me to go the right way :)

--
Reply | Threaded
Open this post in threaded view
|

Re: Specific sink behaviour based on tuple key

Aljoscha Krettek
Hi,

The Key is not available directly to a user function? You would have to use within that function the same code that you use for your KeySelector.

Best,
Aljoscha

On 26. Aug 2017, at 10:01, Alexis Gendronneau <[hidden email]> wrote:

Hi all,

I am looking to customize a sink behaviour based on tuple key.

My goal is to have each sink partition to write to a specific output.

My job will be something like :

input -> keyBy() -> outputSink

This output should be something like './base/path/keyValue/....'.

But I was not able to find where key value is stored, could you help me to go the right way :)

--