Are configs stored as part of savepoints

[gaurav kulkarni]

Hi, 

I was wondering if configs applied while creating a flink application are also stored as part of savepoint? If yes, an app is restored from a savepoint, does it start with the same configs?

Thanks 

[austin.ce]

Hi Guarav,

Which configs are you referring to? Everything usually stored in `flink-conf.yaml`[1]? The State Processor API[2] is also a good resource to understand what is actually stored, and how you can access it outside of a running job. The SavepointMetadata class[3] is another place to reference, which seems to say that the only stored bit that could be influenced by the config is the `maxParallelism`.

When restoring from a savepoint, you usually want to specify the configuration as well.

Hope that helps,

Austin

[1]: https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/config/#configuration

[2]: https://ci.apache.org/projects/flink/flink-docs-master/docs/libs/state_processor_api/

[3]: https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-libraries/flink-state-processing-api/src/main/java/org/apache/flink/state/api/runtime/metadata/SavepointMetadata.java

On Mon, Apr 19, 2021 at 8:54 PM gaurav kulkarni <[hidden email]> wrote:

Hi, 

I was wondering if configs applied while creating a flink application are also stored as part of savepoint? If yes, an app is restored from a savepoint, does it start with the same configs?

Thanks 

[Arvid Heise-4]

Just to make it explicit: no the configuration is not stored. Only maxParallelism and the state backend choices are implicitly stored.

Thus, you can use the same savepoint to perform some A/B testing based on configuration.

On Tue, Apr 20, 2021 at 6:51 PM Austin Cawley-Edwards <[hidden email]> wrote:

Hi Guarav,

Which configs are you referring to? Everything usually stored in `flink-conf.yaml`[1]? The State Processor API[2] is also a good resource to understand what is actually stored, and how you can access it outside of a running job. The SavepointMetadata class[3] is another place to reference, which seems to say that the only stored bit that could be influenced by the config is the `maxParallelism`.

When restoring from a savepoint, you usually want to specify the configuration as well.

Hope that helps,

Austin

[1]: https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/config/#configuration

[2]: https://ci.apache.org/projects/flink/flink-docs-master/docs/libs/state_processor_api/

[3]: https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-libraries/flink-state-processing-api/src/main/java/org/apache/flink/state/api/runtime/metadata/SavepointMetadata.java

On Mon, Apr 19, 2021 at 8:54 PM gaurav kulkarni <[hidden email]> wrote:

Hi, 

I was wondering if configs applied while creating a flink application are also stored as part of savepoint? If yes, an app is restored from a savepoint, does it start with the same configs?

Thanks