Apache Flink - Retries for async processing

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Apache Flink - Retries for async processing

M Singh
Hi Folks:

I am working on a project where I will be using Flink's async processing capabilities.  The job has to make http request using a token.  The token expires periodically and needs to be refreshed.

So, I was looking for patterns for handling async call failures and retries when the token expires.  I found this link Re: Backoff strategies for async IO functions? and it appears that Flink does not support retries and periodically refresh a security token.  I am using 1.6 at the moment but am planning to migrate to 1.9 soon.

 

If there are any patterns on how to deal with this scenario, please let me know.

Thanks

Mans

Reply | Threaded
Open this post in threaded view
|

Re: Apache Flink - Retries for async processing

Jingsong Li
Hi M Singh,

Our internal has this scenario too, as far as I know, Flink does not have this internal mechanism in 1.9 too.
I can share my solution:
- In async function, start a thread factory.
- Send the call to thread factory when this call has failed. Do refresh security token too.
Actually, deal with anything in function. As long as we finally call the relevant methods of ResultFuture.

Best,
Jingsong Lee

On Tue, Dec 10, 2019 at 3:25 AM M Singh <[hidden email]> wrote:
Hi Folks:

I am working on a project where I will be using Flink's async processing capabilities.  The job has to make http request using a token.  The token expires periodically and needs to be refreshed.

So, I was looking for patterns for handling async call failures and retries when the token expires.  I found this link Re: Backoff strategies for async IO functions? and it appears that Flink does not support retries and periodically refresh a security token.  I am using 1.6 at the moment but am planning to migrate to 1.9 soon.

 

If there are any patterns on how to deal with this scenario, please let me know.

Thanks

Mans



--
Best, Jingsong Lee
Reply | Threaded
Open this post in threaded view
|

Re: Apache Flink - Retries for async processing

M Singh
Thanks Jingsong for sharing your solution.

Since both refreshing the token and the actual API request can fail with either recoverable and unrecoverable exceptions, are there any patterns for retrying both and making the code robust to failures.

Thanks again.

On Monday, December 9, 2019, 10:08:39 PM EST, Jingsong Li <[hidden email]> wrote:


Hi M Singh,

Our internal has this scenario too, as far as I know, Flink does not have this internal mechanism in 1.9 too.
I can share my solution:
- In async function, start a thread factory.
- Send the call to thread factory when this call has failed. Do refresh security token too.
Actually, deal with anything in function. As long as we finally call the relevant methods of ResultFuture.

Best,
Jingsong Lee

On Tue, Dec 10, 2019 at 3:25 AM M Singh <[hidden email]> wrote:
Hi Folks:

I am working on a project where I will be using Flink's async processing capabilities.  The job has to make http request using a token.  The token expires periodically and needs to be refreshed.

So, I was looking for patterns for handling async call failures and retries when the token expires.  I found this link Re: Backoff strategies for async IO functions? and it appears that Flink does not support retries and periodically refresh a security token.  I am using 1.6 at the moment but am planning to migrate to 1.9 soon.

 

If there are any patterns on how to deal with this scenario, please let me know.

Thanks

Mans



--
Best, Jingsong Lee
Reply | Threaded
Open this post in threaded view
|

Re: Apache Flink - Retries for async processing

Zhu Zhu
Hi M Singh,

I think you would be able to know the request failure cause and whether it is recoverable or not.
You can handle the error as you like. 
For example, if you think the error is unrecoverable, you can complete the ResultFuture exceptionally to expose this failure to Flink framework. If the error is recoverable, you can just retry (or refresh the token), and only complete the ResultFuture until it succeeds (until timeout).

Thanks,
Zhu Zhu

M Singh <[hidden email]> 于2019年12月10日周二 下午8:51写道:
Thanks Jingsong for sharing your solution.

Since both refreshing the token and the actual API request can fail with either recoverable and unrecoverable exceptions, are there any patterns for retrying both and making the code robust to failures.

Thanks again.

On Monday, December 9, 2019, 10:08:39 PM EST, Jingsong Li <[hidden email]> wrote:


Hi M Singh,

Our internal has this scenario too, as far as I know, Flink does not have this internal mechanism in 1.9 too.
I can share my solution:
- In async function, start a thread factory.
- Send the call to thread factory when this call has failed. Do refresh security token too.
Actually, deal with anything in function. As long as we finally call the relevant methods of ResultFuture.

Best,
Jingsong Lee

On Tue, Dec 10, 2019 at 3:25 AM M Singh <[hidden email]> wrote:
Hi Folks:

I am working on a project where I will be using Flink's async processing capabilities.  The job has to make http request using a token.  The token expires periodically and needs to be refreshed.

So, I was looking for patterns for handling async call failures and retries when the token expires.  I found this link Re: Backoff strategies for async IO functions? and it appears that Flink does not support retries and periodically refresh a security token.  I am using 1.6 at the moment but am planning to migrate to 1.9 soon.

 

If there are any patterns on how to deal with this scenario, please let me know.

Thanks

Mans



--
Best, Jingsong Lee
Reply | Threaded
Open this post in threaded view
|

Re: Apache Flink - Retries for async processing

M Singh
Thanks Zhu for your advice.  Mans

On Tuesday, December 10, 2019, 09:32:01 PM EST, Zhu Zhu <[hidden email]> wrote:


Hi M Singh,

I think you would be able to know the request failure cause and whether it is recoverable or not.
You can handle the error as you like. 
For example, if you think the error is unrecoverable, you can complete the ResultFuture exceptionally to expose this failure to Flink framework. If the error is recoverable, you can just retry (or refresh the token), and only complete the ResultFuture until it succeeds (until timeout).

Thanks,
Zhu Zhu

M Singh <[hidden email]> 于2019年12月10日周二 下午8:51写道:
Thanks Jingsong for sharing your solution.

Since both refreshing the token and the actual API request can fail with either recoverable and unrecoverable exceptions, are there any patterns for retrying both and making the code robust to failures.

Thanks again.

On Monday, December 9, 2019, 10:08:39 PM EST, Jingsong Li <[hidden email]> wrote:


Hi M Singh,

Our internal has this scenario too, as far as I know, Flink does not have this internal mechanism in 1.9 too.
I can share my solution:
- In async function, start a thread factory.
- Send the call to thread factory when this call has failed. Do refresh security token too.
Actually, deal with anything in function. As long as we finally call the relevant methods of ResultFuture.

Best,
Jingsong Lee

On Tue, Dec 10, 2019 at 3:25 AM M Singh <[hidden email]> wrote:
Hi Folks:

I am working on a project where I will be using Flink's async processing capabilities.  The job has to make http request using a token.  The token expires periodically and needs to be refreshed.

So, I was looking for patterns for handling async call failures and retries when the token expires.  I found this link Re: Backoff strategies for async IO functions? and it appears that Flink does not support retries and periodically refresh a security token.  I am using 1.6 at the moment but am planning to migrate to 1.9 soon.

 

If there are any patterns on how to deal with this scenario, please let me know.

Thanks

Mans



--
Best, Jingsong Lee