Re: Please help, I need to bootstrap keyed state into a stream

Posted by Marco Villalobos-2 on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Please-help-I-need-to-bootstrap-keyed-state-into-a-stream-tp37276p37296.html

First, thank you.

 I want to believe you, I don't see how that is possible.

All of the code is self-contained, and at the bottom of all the code, I print out the non-null values before I attempt to put in the map state.

All of the debug output before and after indicates that there is a null value in there.

I think something else is wrong.  I am going to run this in a distributed environment and see what happens.

> On Aug 9, 2020, at 11:27 PM, Tzu-Li Tai <[hidden email]> wrote:
>
> Hi,
>
> For the NullPointerException, what seems to be happening is that you are
> setting NULL values in your MapState, that is not allowed by the API.
>
> Otherwise, the code that you showed for bootstrapping state seems to be
> fine.
>
>> I have yet to find a working example that shows how to do both
>> (bootstrapping state and start a streaming application with that state)
>
> Not entirely sure what you mean here by "doing both".

By "doing both" I meant, no examples show how to create both Batch and Stream environment in a single job.  Another blog states that is possible.

> The savepoint written using the State Processor API (what you are doing in
> the bootstrap() method) is a savepoint that may be restored from as you
> would with a typical Flink streaming job restore.
> So, usually the bootstrapping part happens as a batch "offline" job, while
> you keep your streaming job as a separate job. What are you trying to
> achieve with having both written within the same job?

Again, thank you.

Marco A. Villalobos