Access the data in a stream after writing to a sink

[Teena Kappen // BPRISE]

Hi,

 

Is it possible to access the data in a stream that was written to a sink? I have a Cassandra Sink in my stream job and I have to access all the records that were written to the Cassandra sink and write it to another sink. Is there any way to do that?

 

Regards,

Teena

[Teena Kappen // BPRISE]

Adding to the previous question, is it possible to check if each record in a stream was written without any exceptions to a Cassandra Sink? I have to write the records to the next sink only if the first write is successful. So, replicating the streams before the write is not an option.

 

From: Teena Kappen // BPRISE <[hidden email]>
Sent: 10 July 2018 12:50
To: [hidden email]
Subject: Access the data in a stream after writing to a sink

 

Hi,

 

Is it possible to access the data in a stream that was written to a sink? I have a Cassandra Sink in my stream job and I have to access all the records that were written to the Cassandra sink and write it to another sink. Is there any way to do that?

 

Regards,

Teena

[Hequn Cheng]

Hi Teena,

It seems that a sink can not output data into another sink. Maybe we can implement a combined user defined sink. In the combined sink, only write to the next sink if the first write is successful.

On Tue, Jul 10, 2018 at 3:23 PM, Teena Kappen // BPRISE <[hidden email]> wrote:

Adding to the previous question, is it possible to check if each record in a stream was written without any exceptions to a Cassandra Sink? I have to write the records to the next sink only if the first write is successful. So, replicating the streams before the write is not an option.

 

From: Teena Kappen // BPRISE <[hidden email]>
Sent: 10 July 2018 12:50
To: [hidden email]
Subject: Access the data in a stream after writing to a sink

 

Hi,

 

Is it possible to access the data in a stream that was written to a sink? I have a Cassandra Sink in my stream job and I have to access all the records that were written to the Cassandra sink and write it to another sink. Is there any way to do that?

 

Regards,

Teena

[Teena Kappen // BPRISE]

Yes.. a custom sink with the required checks seems to be the only option.

 

From: Hequn Cheng <[hidden email]>
Sent: 10 July 2018 18:23
To: Teena Kappen // BPRISE <[hidden email]>
Cc: [hidden email]
Subject: Re: Access the data in a stream after writing to a sink

 

Hi Teena,

 

It seems that a sink can not output data into another sink. Maybe we can implement a combined user defined sink. In the combined sink, only write to the next sink if the first write is successful.

 

On Tue, Jul 10, 2018 at 3:23 PM, Teena Kappen // BPRISE <[hidden email]> wrote:

Adding to the previous question, is it possible to check if each record in a stream was written without any exceptions to a Cassandra Sink? I have to write the records to the next sink only if the first write is successful. So, replicating the streams before the write is not an option.

 

From: Teena Kappen // BPRISE <[hidden email]>
Sent: 10 July 2018 12:50
To: [hidden email]
Subject: Access the data in a stream after writing to a sink

 

Hi,

 

Is it possible to access the data in a stream that was written to a sink? I have a Cassandra Sink in my stream job and I have to access all the records that were written to the Cassandra sink and write it to another sink. Is there any way to do that?

 

Regards,

Teena