(DEPRECATED) Apache Flink User Mailing List archive.

QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

classic Classic list List threaded Threaded
♦
♦
 7 messages Options
Vishal Santoshi
Reply | Threaded
Open this post in threaded view
♦
♦
|

QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

Vishal Santoshi
Hello folks, 

         Following https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state .  for setting up the Queryable Server and proxy, I have my classpath ( the lib directory ) that has the  required jar, But I do not see the mentioned log and of course am not able to set up the QS server/Proxy . This has worked on 1.7.2 and I think I have everything as advised, see the logs below. I do not  see this log  "Started the Queryable State Proxy Server @ ...".  Any one with this issue...



2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dtaskmanager.numberOfTaskSlots=1

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     --configDir

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     /usr/local/flink/conf

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -  Classpath: /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - --------------------------------------------------------------------------------

2019-04-24 15:54:26,298 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Registered UNIX signal handlers for [TERM, HUP, INT]

2019-04-24 15:54:26,300 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Maximum number of open file descriptors is 65536.

2019-04-24 15:54:26,305 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: state.backend.fs.checkpointdir, hdfs:///flink-checkpoints_k8s_test/prod

2
Vishal Santoshi
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

Vishal Santoshi
Any one ?

On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <[hidden email]> wrote:
Hello folks, 

         Following https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state .  for setting up the Queryable Server and proxy, I have my classpath ( the lib directory ) that has the  required jar, But I do not see the mentioned log and of course am not able to set up the QS server/Proxy . This has worked on 1.7.2 and I think I have everything as advised, see the logs below. I do not  see this log  "Started the Queryable State Proxy Server @ ...".  Any one with this issue...



2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dtaskmanager.numberOfTaskSlots=1

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     --configDir

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     /usr/local/flink/conf

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -  Classpath: /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - --------------------------------------------------------------------------------

2019-04-24 15:54:26,298 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Registered UNIX signal handlers for [TERM, HUP, INT]

2019-04-24 15:54:26,300 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Maximum number of open file descriptors is 65536.

2019-04-24 15:54:26,305 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: state.backend.fs.checkpointdir, hdfs:///flink-checkpoints_k8s_test/prod

2
Guowei Ma
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

Guowei Ma
You could try to set queryable-state.enable to true. And check again. 

Vishal Santoshi <[hidden email]>于2019年4月25日 周四上午1:40写道：
Any one ?

On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <[hidden email]> wrote:
Hello folks, 

         Following https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state .  for setting up the Queryable Server and proxy, I have my classpath ( the lib directory ) that has the  required jar, But I do not see the mentioned log and of course am not able to set up the QS server/Proxy . This has worked on 1.7.2 and I think I have everything as advised, see the logs below. I do not  see this log  "Started the Queryable State Proxy Server @ ...".  Any one with this issue...



2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dtaskmanager.numberOfTaskSlots=1

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     --configDir

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     /usr/local/flink/conf

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -  Classpath: /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - --------------------------------------------------------------------------------

2019-04-24 15:54:26,298 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Registered UNIX signal handlers for [TERM, HUP, INT]

2019-04-24 15:54:26,300 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Maximum number of open file descriptors is 65536.

2019-04-24 15:54:26,305 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: state.backend.fs.checkpointdir, hdfs:///flink-checkpoints_k8s_test/prod

2





--
Best,
Guowei
Dawid Wysakowicz-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

Dawid Wysakowicz-2

Hi Vishal,

As Guowei mentioned you have to enable the Queryable state. The default setting was changed in 1.8.0. There is an open JIRA[1] for changing the documentation accordingly.

Best,

Dawid

[1] https://issues.apache.org/jira/browse/FLINK-12274

On 25/04/2019 03:27, Guowei Ma wrote:
You could try to set queryable-state.enable to true. And check again. 

Vishal Santoshi <[hidden email]>于2019年4月25日 周四上午1:40写道：
Any one ?

On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <[hidden email]> wrote:
Hello folks, 

         Following https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state .  for setting up the Queryable Server and proxy, I have my classpath ( the lib directory ) that has the  required jar, But I do not see the mentioned log and of course am not able to set up the QS server/Proxy . This has worked on 1.7.2 and I think I have everything as advised, see the logs below. I do not  see this log  "Started the Queryable State Proxy Server @ ...".  Any one with this issue...



2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dtaskmanager.numberOfTaskSlots=1

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     --configDir

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     /usr/local/flink/conf

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -  Classpath: /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - --------------------------------------------------------------------------------

2019-04-24 15:54:26,298 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Registered UNIX signal handlers for [TERM, HUP, INT]

2019-04-24 15:54:26,300 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Maximum number of open file descriptors is 65536.

2019-04-24 15:54:26,305 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: state.backend.fs.checkpointdir, hdfs:///flink-checkpoints_k8s_test/prod

2





--
Best,
Guowei

signature.asc (849 bytes) Download Attachment
Vishal Santoshi
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

Vishal Santoshi
Ditto that, queryable-state.enable to true works.

Thanks everyone. 

On Thu, Apr 25, 2019 at 6:28 AM Dawid Wysakowicz <[hidden email]> wrote:

Hi Vishal,

As Guowei mentioned you have to enable the Queryable state. The default setting was changed in 1.8.0. There is an open JIRA[1] for changing the documentation accordingly.

Best,

Dawid

[1] https://issues.apache.org/jira/browse/FLINK-12274

On 25/04/2019 03:27, Guowei Ma wrote:
You could try to set queryable-state.enable to true. And check again. 

Vishal Santoshi <[hidden email]>于2019年4月25日 周四上午1:40写道：
Any one ?

On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <[hidden email]> wrote:
Hello folks, 

         Following https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state .  for setting up the Queryable Server and proxy, I have my classpath ( the lib directory ) that has the  required jar, But I do not see the mentioned log and of course am not able to set up the QS server/Proxy . This has worked on 1.7.2 and I think I have everything as advised, see the logs below. I do not  see this log  "Started the Queryable State Proxy Server @ ...".  Any one with this issue...



2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dtaskmanager.numberOfTaskSlots=1

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     --configDir

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     /usr/local/flink/conf

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -  Classpath: /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - --------------------------------------------------------------------------------

2019-04-24 15:54:26,298 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Registered UNIX signal handlers for [TERM, HUP, INT]

2019-04-24 15:54:26,300 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Maximum number of open file descriptors is 65536.

2019-04-24 15:54:26,305 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: state.backend.fs.checkpointdir, hdfs:///flink-checkpoints_k8s_test/prod

2





--
Best,
Guowei
Ufuk Celebi
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

Ufuk Celebi
I didn't find this as part of the https://flink.apache.org/news/2019/04/09/release-1.8.0.html notes.

I think an update to the Important Changes section would be valuable for users upgrading to 1.8 from earlier releases. Also, logging that the library is on the classpath but the feature flag is set to false would be a helpful.

– Ufuk


On Thu, Apr 25, 2019 at 4:13 PM Vishal Santoshi <[hidden email]> wrote:
Ditto that, queryable-state.enable to true works.

Thanks everyone. 

On Thu, Apr 25, 2019 at 6:28 AM Dawid Wysakowicz <[hidden email]> wrote:

Hi Vishal,

As Guowei mentioned you have to enable the Queryable state. The default setting was changed in 1.8.0. There is an open JIRA[1] for changing the documentation accordingly.

Best,

Dawid

[1] https://issues.apache.org/jira/browse/FLINK-12274

On 25/04/2019 03:27, Guowei Ma wrote:
You could try to set queryable-state.enable to true. And check again. 

Vishal Santoshi <[hidden email]>于2019年4月25日 周四上午1:40写道：
Any one ?

On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <[hidden email]> wrote:
Hello folks, 

         Following https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state .  for setting up the Queryable Server and proxy, I have my classpath ( the lib directory ) that has the  required jar, But I do not see the mentioned log and of course am not able to set up the QS server/Proxy . This has worked on 1.7.2 and I think I have everything as advised, see the logs below. I do not  see this log  "Started the Queryable State Proxy Server @ ...".  Any one with this issue...



2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dtaskmanager.numberOfTaskSlots=1

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     --configDir

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     /usr/local/flink/conf

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -  Classpath: /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - --------------------------------------------------------------------------------

2019-04-24 15:54:26,298 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Registered UNIX signal handlers for [TERM, HUP, INT]

2019-04-24 15:54:26,300 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Maximum number of open file descriptors is 65536.

2019-04-24 15:54:26,305 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: state.backend.fs.checkpointdir, hdfs:///flink-checkpoints_k8s_test/prod

2





--
Best,
Guowei
Ufuk Celebi
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

Ufuk Celebi
Actually, I couldn't even find a mention of this flag in the docs here: https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/state/queryable_state.html

– Ufuk


On Mon, Apr 29, 2019 at 8:45 AM Ufuk Celebi <[hidden email]> wrote:
I didn't find this as part of the https://flink.apache.org/news/2019/04/09/release-1.8.0.html notes.

I think an update to the Important Changes section would be valuable for users upgrading to 1.8 from earlier releases. Also, logging that the library is on the classpath but the feature flag is set to false would be a helpful.

– Ufuk


On Thu, Apr 25, 2019 at 4:13 PM Vishal Santoshi <[hidden email]> wrote:
Ditto that, queryable-state.enable to true works.

Thanks everyone. 

On Thu, Apr 25, 2019 at 6:28 AM Dawid Wysakowicz <[hidden email]> wrote:

Hi Vishal,

As Guowei mentioned you have to enable the Queryable state. The default setting was changed in 1.8.0. There is an open JIRA[1] for changing the documentation accordingly.

Best,

Dawid

[1] https://issues.apache.org/jira/browse/FLINK-12274

On 25/04/2019 03:27, Guowei Ma wrote:
You could try to set queryable-state.enable to true. And check again. 

Vishal Santoshi <[hidden email]>于2019年4月25日 周四上午1:40写道：
Any one ?

On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <[hidden email]> wrote:
Hello folks, 

         Following https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state .  for setting up the Queryable Server and proxy, I have my classpath ( the lib directory ) that has the  required jar, But I do not see the mentioned log and of course am not able to set up the QS server/Proxy . This has worked on 1.7.2 and I think I have everything as advised, see the logs below. I do not  see this log  "Started the Queryable State Proxy Server @ ...".  Any one with this issue...



2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     -Dtaskmanager.numberOfTaskSlots=1

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     --configDir

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -     /usr/local/flink/conf

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       -  Classpath: /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

2019-04-24 15:54:26,296 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - --------------------------------------------------------------------------------

2019-04-24 15:54:26,298 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Registered UNIX signal handlers for [TERM, HUP, INT]

2019-04-24 15:54:26,300 INFO  org.apache.flink.runtime.taskexecutor.TaskManagerRunner       - Maximum number of open file descriptors is 65536.

2019-04-24 15:54:26,305 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: state.backend.fs.checkpointdir, hdfs:///flink-checkpoints_k8s_test/prod

2





--
Best,
Guowei
Free forum by Nabble Edit this page