Noisy org.apache.flink.configuration.GlobalConfiguration

[Elias Levy]

Is there a particular reason that GlobalConfiguration is so noisy?

The task manager log is full of "Loading configuration property" messages from GlobalConfiguration each time there is a checkpoint.  Why isn't the configuration read once?

[Till Rohrmann]

Hi Elias,

which version of Flink and which state backend are you running? I tried to reproduce it and wasn't successful so far. 

We recently changed a bit how we load the GlobalConfiguration in combination with dynamic properties [1]. Maybe this has affected what you've reported as well.

[1] https://issues.apache.org/jira/browse/FLINK-7269

Cheers,

Till

On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <[hidden email]> wrote:

Is there a particular reason that GlobalConfiguration is so noisy?

The task manager log is full of "Loading configuration property" messages from GlobalConfiguration each time there is a checkpoint.  Why isn't the configuration read once?

[Ufuk Celebi]

I saw this too recently when using HadoopFileSystem for checkpoints
(HDFS or S3). I thought I had opened an issue for this, but I didn't.
Here it is: https://issues.apache.org/jira/browse/FLINK-7643


On Tue, Sep 19, 2017 at 1:28 PM, Till Rohrmann <[hidden email]> wrote:

> Hi Elias,
>
> which version of Flink and which state backend are you running? I tried to
> reproduce it and wasn't successful so far.
>
> We recently changed a bit how we load the GlobalConfiguration in combination
> with dynamic properties [1]. Maybe this has affected what you've reported as
> well.
>
> [1] https://issues.apache.org/jira/browse/FLINK-7269
>
> Cheers,
> Till
>
> On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <[hidden email]>
> wrote:
>>
>> Is there a particular reason that GlobalConfiguration is so noisy?
>>
>> The task manager log is full of "Loading configuration property" messages
>> from GlobalConfiguration each time there is a checkpoint.  Why isn't the
>> configuration read once?
>
>

[Ufuk Celebi]

PS: To answer the question. No, I think there is no reason for this
and it shouldn't happen. :-(

On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <[hidden email]> wrote:
> Is there a particular reason that GlobalConfiguration is so noisy?
>
> The task manager log is full of "Loading configuration property" messages
> from GlobalConfiguration each time there is a checkpoint.  Why isn't the
> configuration read once?

[Elias Levy]

Till,

Using 1.3.2 and like Ufuk mentioned, using S3 for checkpointing.

On Tue, Sep 19, 2017 at 4:28 AM, Till Rohrmann <[hidden email]> wrote:

Hi Elias,

which version of Flink and which state backend are you running? I tried to reproduce it and wasn't successful so far. 

We recently changed a bit how we load the GlobalConfiguration in combination with dynamic properties [1]. Maybe this has affected what you've reported as well.

[1] https://issues.apache.org/jira/browse/FLINK-7269

Cheers,

Till

On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <[hidden email]> wrote:

Is there a particular reason that GlobalConfiguration is so noisy?

The task manager log is full of "Loading configuration property" messages from GlobalConfiguration each time there is a checkpoint.  Why isn't the configuration read once?

[Stephan Ewen]

Will be opening a PR for file system configuring which should fix that.

On Tue, Sep 19, 2017 at 5:28 PM, Elias Levy <[hidden email]> wrote:

Till,

Using 1.3.2 and like Ufuk mentioned, using S3 for checkpointing.

On Tue, Sep 19, 2017 at 4:28 AM, Till Rohrmann <[hidden email]> wrote:

Hi Elias,

which version of Flink and which state backend are you running? I tried to reproduce it and wasn't successful so far. 

We recently changed a bit how we load the GlobalConfiguration in combination with dynamic properties [1]. Maybe this has affected what you've reported as well.

[1] https://issues.apache.org/jira/browse/FLINK-7269

Cheers,

Till

On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <[hidden email]> wrote:

Is there a particular reason that GlobalConfiguration is so noisy?

The task manager log is full of "Loading configuration property" messages from GlobalConfiguration each time there is a checkpoint.  Why isn't the configuration read once?

[Stephan Ewen]

Hi!

The problem should be fixed in the latest master. 

https://github.com/apache/flink/commit/536675b03a5050fda9c3e1fd403818cb50dcc6ff

Release should come soon...

Best,

Stephan

On Wed, Oct 4, 2017 at 12:50 PM, Stephan Ewen <[hidden email]> wrote:

Will be opening a PR for file system configuring which should fix that.

On Tue, Sep 19, 2017 at 5:28 PM, Elias Levy <[hidden email]> wrote:

Till,

Using 1.3.2 and like Ufuk mentioned, using S3 for checkpointing.

On Tue, Sep 19, 2017 at 4:28 AM, Till Rohrmann <[hidden email]> wrote:

Hi Elias,

which version of Flink and which state backend are you running? I tried to reproduce it and wasn't successful so far. 

We recently changed a bit how we load the GlobalConfiguration in combination with dynamic properties [1]. Maybe this has affected what you've reported as well.

[1] https://issues.apache.org/jira/browse/FLINK-7269

Cheers,

Till

On Tue, Sep 19, 2017 at 2:44 AM, Elias Levy <[hidden email]> wrote:

Is there a particular reason that GlobalConfiguration is so noisy?

The task manager log is full of "Loading configuration property" messages from GlobalConfiguration each time there is a checkpoint.  Why isn't the configuration read once?