Initialization of static variables

[Flavio Pompermaier]

Hi all,

I've a Flink job that initialize a static Map in the main program, before starting any Flink transformation. If I run the job locally that variable is not empty, running the job on the cluster reset that variable..is it a bug or am I doing something wrong?

It only works if I initialize that variable in a static statement before the main, that is:

///////////////// KO EXAMPLE

class ErrorMain {

    private static final Map<String,String> ht = new HashMap<>();

    publis static final main(String[]args){

     ht.put("test","test");

     env.readFile().map(

        ...

       //here ht.get("test") returns null

  }

}

///////////////// OK EXAMPLE

class OkMain {

    private static final Map<String,String> ht = new HashMap<>();

    static{

        ht.put("test","test");

    }

    publis static final main(String[] args){

    

     env.readFile().map(

        ...

       //here ht.get("test") works

  }

}

Best,

Flavio

[Till Rohrmann]

Yes this is normal Flink behaviour. The reason is that static variables are not transferred to the cluster. What happens instead when you first load the class on the cluster is that the static variables are created and possible class initializer are executed. That is also the reason why your second example works whereas the first fails.

Cheers,

Till

On Thu, Jun 23, 2016 at 3:12 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi all,

I've a Flink job that initialize a static Map in the main program, before starting any Flink transformation. If I run the job locally that variable is not empty, running the job on the cluster reset that variable..is it a bug or am I doing something wrong?

It only works if I initialize that variable in a static statement before the main, that is:

///////////////// KO EXAMPLE

class ErrorMain {

    private static final Map<String,String> ht = new HashMap<>();

    publis static final main(String[]args){

     ht.put("test","test");

     env.readFile().map(

        ...

       //here ht.get("test") returns null

  }

}

///////////////// OK EXAMPLE

class OkMain {

    private static final Map<String,String> ht = new HashMap<>();

    static{

        ht.put("test","test");

    }

    publis static final main(String[] args){

    

     env.readFile().map(

        ...

       //here ht.get("test") works

  }

}

Best,

Flavio

[Flavio Pompermaier]

Ok, thanks for the explanation Till!

On Thu, Jun 23, 2016 at 3:19 PM, Till Rohrmann <[hidden email]> wrote:

Yes this is normal Flink behaviour. The reason is that static variables are not transferred to the cluster. What happens instead when you first load the class on the cluster is that the static variables are created and possible class initializer are executed. That is also the reason why your second example works whereas the first fails.

Cheers,

Till

On Thu, Jun 23, 2016 at 3:12 PM, Flavio Pompermaier <[hidden email]> wrote:

Hi all,

I've a Flink job that initialize a static Map in the main program, before starting any Flink transformation. If I run the job locally that variable is not empty, running the job on the cluster reset that variable..is it a bug or am I doing something wrong?

It only works if I initialize that variable in a static statement before the main, that is:

///////////////// KO EXAMPLE

class ErrorMain {

    private static final Map<String,String> ht = new HashMap<>();

    publis static final main(String[]args){

     ht.put("test","test");

     env.readFile().map(

        ...

       //here ht.get("test") returns null

  }

}

///////////////// OK EXAMPLE

class OkMain {

    private static final Map<String,String> ht = new HashMap<>();

    static{

        ht.put("test","test");

    }

    publis static final main(String[] args){

    

     env.readFile().map(

        ...

       //here ht.get("test") works

  }

}

Best,

Flavio