Hi Tony,
I think your reasoning is correct that this is because of the
fact that rest server runs in the same process as
standalonesession.
It's hard to say if it is an expected behavior or not, but there
is really not much we can do about it. User code can actually call
System.exit from any
place in the code, which will result in the process executing
this code to terminate. In general I would say calling System.exit
is rather discouraged.
Best,
Dawid
On 05/11/2018 06:38, Tony Wei wrote:
Hi,
I used a scala library called scallop[1] to parse
my job’s arguments. When the argument didn’t
exist in the config setting, the default behavior
of scallop would call sys.exit(1).
It is not a problem when I’m using flink
cli to submit job. However, when I used rest api to
submit
job, it seems that sys.exit(1) will
leads to standalonesession daemon closed. Maybe the
reason is
that rest server is also in the same
process as standalonesession daemon. Am I correct?
If this is the root cause, is this an expected
behavior and users should be aware of not using
sys.exit(1) in their jobs?
I tested this on 1.6.0 standalone
session cluster with flip-6 mode. And here are my
testing job
and logs before and after the
submission.
package
com.appier.rt.rt_match
import
org.apache.flink.api.scala.createTypeInformation
import
org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
import org.rogach.scallop.{ScallopConf,
ScallopOption}
object TestMain {
def main(args: Array[String]): Unit = {
object Args extends ScallopConf(args) {
val mode: ScallopOption[String] =
opt[String](default = Some("development"))
verify
}
val env =
StreamExecutionEnvironment.getExecutionEnvironment
env.fromElements(Args.mode()).map(a => a)
env.execute()
}
}
Submit by flink-cli
$ ./bin/flink run
-c com.appier.rt.rt_match.TestMain -p 2 -d
rt-match-assembly-4.5.1-SNAPSHOT.jar --mo xyz
Starting execution of program
[scallop] Error: Unknown option 'mo'
Submit by rest-api
2018-11-05
13:27:58,800 TRACE
org.apache.flink.runtime.webmonitor.handlers.JarListHandler
- Received request /jars/.
2018-11-05 13:27:59,679 TRACE
org.apache.flink.runtime.rest.FileUploadHandler
- Received request. URL:/jobs/overview
Method:GET
2018-11-05 13:27:59,680 TRACE
org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler
- Received request /jobs/overview.
2018-11-05 13:28:01,752 TRACE
org.apache.flink.runtime.rest.FileUploadHandler
- Received request. URL:/jars/ Method:GET
2018-11-05 13:28:01,753 TRACE
org.apache.flink.runtime.webmonitor.handlers.JarListHandler
- Received request /jars/.
2018-11-05 13:28:02,682 TRACE
org.apache.flink.runtime.rest.FileUploadHandler
- Received request. URL:/jobs/overview
Method:GET
2018-11-05 13:28:02,683 TRACE
org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler
- Received request /jobs/overview.
2018-11-05 13:28:03,899 TRACE
org.apache.flink.runtime.rest.FileUploadHandler
- Received request.
URL:/jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
class=com.appier.rt.rt_match.TestMain¶llelism=2&program-args=--mo+xyz
Method:POST
2018-11-05 13:28:03,902 TRACE
org.apache.flink.runtime.webmonitor.handlers.JarRunHandler
- Received request
/jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
class=com.appier.rt.rt_match.TestMain¶llelism=2&program-args=--mo+xyz.
2018-11-05 13:28:04,751 TRACE
org.apache.flink.runtime.rest.FileUploadHandler
- Received request. URL:/jars/ Method:GET
2018-11-05 13:28:04,752 TRACE
org.apache.flink.runtime.webmonitor.handlers.JarListHandler
- Received request /jars/.
2018-11-05 13:28:04,760 INFO
org.apache.flink.runtime.blob.TransientBlobCache
- Shutting down BLOB cache
2018-11-05 13:28:04,761 INFO
org.apache.flink.runtime.blob.BlobServer
- Stopped BLOB server at 0.0.0.0:42075
Best,
Tony Wei.
Locked
signature.asc (849 bytes)