Hi Zhu Zhu, as per my original post. If you have 3 Job nodes and you upload the job through the web UI on a specific node, then only the web UI for that node will show it. If you browse another job node you will not see it.
As per Stevens suggestion, I already have a Gluster FS mounted folder on the Job nodes, I will point web.upload.dir to that folder. So that when I upload from one node all nodes will see it.
John,
I think you are referring to the web upload directory. There is a setting for that folder ‘web.upload.dir’. If you set that to a folder writeable to both masters it will work as desired. I used an NFS mount (AWS EFS).
-Steve
Sent from my iPhone
Hi John,
Not sure why you need to know the location of uploaded job jars?
The job jar will be automatically localized to a taskmanager via BlobService when a task belonging to the job is running on that taskmanager.
The localization dir is blob.storage.directory. If it is not specified, it will be java.io.tmpdir in standalone mode.
Thanks,
Zhu Zhu
And can that folder be shared so that all nodes see it?
Hi John
The jar is not stored in HA path, I think the answer [1] could help you.
Best
Yun Tang
From: John Smith <[hidden email]>
Sent: Friday, October 11, 2019 2:06
To: user <[hidden email]>
Subject: Where are uploaded Job jars stored?
Hi using 1.8.0 running on standalone cluster with Zookeeper HA.
Are job JARs stored at: high-availability.storageDir ???
The thing is when you browse the individual nodes at port 8080 to go submit the job only the node where you uploaded the JAR has it.
- Go to any given node
- Upload a jar
- Browse another node
- Jar is not there.
Locked
