Flink on Azure HDInsight

[Brig Lamoreaux]

Hi All,

 

Are there any issues with Flink on Azure HDInsight?

 

Thanks,

Brig Lamoreaux
Data Solution Architect US Desert/Mountain Tempe

 

 

[Stephan Ewen]

Hi!

As far as I know, some people have been using Flink together with Azure, and we try and do some release validation on Azure as well.

There is even a section in the docs that describes how to use Hadoop's Azure Table formats with Flink

https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/connectors.html#access-microsoft-azure-table-storage

I am not aware of any Azure specific issues at this point...

Greetings,

Stephan

On Fri, Apr 29, 2016 at 11:18 AM, Brig Lamoreaux <[hidden email]> wrote:

Hi All,

 

Are there any issues with Flink on Azure HDInsight?

 

Thanks,

Brig Lamoreaux
Data Solution Architect US Desert/Mountain Tempe

 

 

[Brig Lamoreaux]

Thanks Stephan,

 

Turns out Azure Table is slightly different than Azure HDInsight. Both use Azure Storage however, HDInsight allows HDFS over Azure Storage.

 

I’d be curious if anyone has tried to use Flink on top of Azure HDInsight.

 

Thanks,

Brig

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Stephan Ewen
Sent: Saturday, April 30, 2016 9:36 PM
To: [hidden email]
Subject: Re: Flink on Azure HDInsight

 

Hi!

 

As far as I know, some people have been using Flink together with Azure, and we try and do some release validation on Azure as well.

 

There is even a section in the docs that describes how to use Hadoop's Azure Table formats with Flink

https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/connectors.html#access-microsoft-azure-table-storage

 

I am not aware of any Azure specific issues at this point...

 

Greetings,

Stephan

 

 

 

On Fri, Apr 29, 2016 at 11:18 AM, Brig Lamoreaux <[hidden email]> wrote:

Hi All,

 

Are there any issues with Flink on Azure HDInsight?

 

Thanks,

Brig Lamoreaux
Data Solution Architect US Desert/Mountain Tempe

 

 

 

[Kostas Tzoumas]

As far as I know, Azure HDInsight is based on Hortonworks HDP, on top of which Flink has been used extensively.

On Mon, May 2, 2016 at 10:42 AM, Brig Lamoreaux <[hidden email]> wrote:

Thanks Stephan,

 

Turns out Azure Table is slightly different than Azure HDInsight. Both use Azure Storage however, HDInsight allows HDFS over Azure Storage.

 

I’d be curious if anyone has tried to use Flink on top of Azure HDInsight.

 

Thanks,

Brig

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Stephan Ewen
Sent: Saturday, April 30, 2016 9:36 PM
To: [hidden email]
Subject: Re: Flink on Azure HDInsight

 

Hi!

 

As far as I know, some people have been using Flink together with Azure, and we try and do some release validation on Azure as well.

 

There is even a section in the docs that describes how to use Hadoop's Azure Table formats with Flink

https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/connectors.html#access-microsoft-azure-table-storage

 

I am not aware of any Azure specific issues at this point...

 

Greetings,

Stephan

 

 

 

On Fri, Apr 29, 2016 at 11:18 AM, Brig Lamoreaux <[hidden email]> wrote:

Hi All,

 

Are there any issues with Flink on Azure HDInsight?

 

Thanks,

Brig Lamoreaux
Data Solution Architect US Desert/Mountain Tempe