hmm...
quite interesting question. But I think I don't fully understand
your use case - how are your applications coupled? Through kafka
topics? E.g. output of one is input for other?
Or do they consume from same input?
And why exactly do you want to get back to specific point in all of
them? If there's bug in one then I guess the whole idea of
microservices is to have to fix only one, and don't care about the
others?
I think it can be kind of tricky to do what you describe - if apps
are really decoupled then there's no notion of "same time" for them.
One thing that comes to my mind is having some special "message"
that trigger savepoint - but not sure if it's possible and if it
would work in all cases (like - what if there are many inputs for
stream?)
br,
maciek
On 10/05/2016 09:11, Krzysztof Zarzycki
wrote:
Hi!
I'm thinking about using a great Flink functionality - savepoints . I would like to be
able to stop my streaming application, rollback the state of
it and restart it (for example to update code, to fix a
bug). Let's say I would like travel back in time and
reprocess some data.
But what if I had many streaming applications running,
that's states are correlated, like in microservice
architecture? I would like to travel back in time in all of my
services to a common point in time. Is there a possibility to
somehow manage correlated savepoints? Of course I can trigger
savepointing at approximately same time, but it's just an
approximation, right?
Is there something in Flink that could support this advanced use
case? Maybe someone else hit this issue already and thought
about the solution?
I'll be grateful for any
comments,
Cheers,
Krzysztof
Locked
