Pyflink 1.10.0 issue on cluster

Posted by rookieCOder on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Pyflink-1-10-0-issue-on-cluster-tp36959.html

'm coding with pyflink 1.10.0 and building cluster with flink 1.10.0
I define the source and the sink as following:
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t2674/%E6%97%A0%E6%A0%87%E9%A2%98.png> 
When I run this code only on master, it's OK. When I run this code on cluster, with 1 master and 1 salve, and I submit the task on master like this:
sudo flink-1.10.0/bin/flink run -py main.py
And error occurs like:
Caused by: java.io.FileNotFoundException: The provided file path
/opt/raw_data/input_json_65adbe54-cfdc-11ea-9d47-020012970011 does not exist.
This file is stored on master's local file system. It seems that the slaves
read their own file system instead of the master's. Or maybe there are other
points I ignored (maybe some configurations in flink when I start the cluster).
The question is how can I avoid the error?



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/