Question on flink and hdfs

[Jerry Peng]

Hello,

Does flink require hdfs to run? I know you can use hdfs to checkpoint and process files in a distributed fashion.  So can flink run standalone without hdfs?

[Márton Balassi]

Hi Jerry,

Yes, you can.

Best,

Marton

On Thu, Sep 3, 2015 at 8:57 PM, Jerry Peng <[hidden email]> wrote:

Hello,

Does flink require hdfs to run? I know you can use hdfs to checkpoint and process files in a distributed fashion.  So can flink run standalone without hdfs?

[Stephan Ewen]

Hi!

Yes, you can run Flink completely without HDFS.

Also the checkpointing can put state into any file system, like S3, or a Unix file system (like a NAS or Amazon EBS), or even Tachyon.

Greetings,

Stephan

On Thu, Sep 3, 2015 at 8:57 PM, Jerry Peng <[hidden email]> wrote:

Hello,

Does flink require hdfs to run? I know you can use hdfs to checkpoint and process files in a distributed fashion.  So can flink run standalone without hdfs?

[tambunanw]

Hi Jerry, 

yes, that's possible. You can download the appropriate version 
https://flink.apache.org/downloads.html

Cheers

On Fri, Sep 4, 2015 at 1:57 AM, Jerry Peng <[hidden email]> wrote:

Hello,

Does flink require hdfs to run? I know you can use hdfs to checkpoint and process files in a distributed fashion.  So can flink run standalone without hdfs?

--

[Maximilian Michels]

Hi Jerry,

If you don't want to use Hadoop, simply pick _any_ Flink version. We recommend the Hadoop 1 version because it contains the least dependencies, i.e. you need to download less and the installation occupies less space. Other than that, it doesn't really matter if you don't use the HDFS functionality, i.e. you don't access hdfs:// paths.

Cheers,

Max

On Fri, Sep 4, 2015 at 10:08 AM, Welly Tambunan <[hidden email]> wrote:

Hi Jerry, 

yes, that's possible. You can download the appropriate version 
https://flink.apache.org/downloads.html

Cheers

On Fri, Sep 4, 2015 at 1:57 AM, Jerry Peng <[hidden email]> wrote:

Hello,

Does flink require hdfs to run? I know you can use hdfs to checkpoint and process files in a distributed fashion.  So can flink run standalone without hdfs?

--

Welly Tambunan
Triplelands 

http://weltam.wordpress.com

http://www.triplelands.com