Re: Multiple hdfs

Posted by Kien Truong on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Multiple-hdfs-tp20253p20262.html

Hi,

If your cluster are not high-availability clusters then just use the
full path to the cluster.

For example, to refer to directory /checkpoint on cluster1, use
hdfs://namenode1_ip:port/checkpoint

Like wise, /data on cluster2 will be hdfs://namenode2_ip:port/data


If your cluster is a HA cluster, then you need to modify the
hdfs-site.xml like section 1 of this guide

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.4/bk_administration/content/distcp_between_ha_clusters.html

Then use the full path to the cluster hdfs://cluster1ha/checkpoint &
hdfs://cluster2ha/data

Regards,
Kien

On 5/21/2018 9:19 PM, Raul Valdoleiros wrote:
> Hi,
>
> I want to store my data in one hdfs and the flink checkpoints in
> another hdfs. I didn't find a way to do it, anyone can point me a
> direction?
>
> Thanks in advance,
> Raul