Machine Learning on Apache Fink

[Ashutosh Kumar]

I am looking for some study material and examples on machine learning . Are classification , recommendation and clustering libraries available ? What is the timeline for Flink as backend for Mahout? I am building a meta data driven framework over Flink . While building data collection and transformation modules was cool , I am struggling since I started analytics module. Thanks in advance. 

Ashutosh

[Ashutosh Kumar]

I see lot of study materials and even book available for ml on spark. Spark seems to be more mature for analytics related work as of now. Please correct me if I am wrong. As I have already built my collection and data pre processing layers on flink , I want to use Flink for analytics as well. Thanks in advance.

Ashutosh

On Sat, Jan 9, 2016 at 3:32 PM, Ashutosh Kumar <[hidden email]> wrote:

I am looking for some study material and examples on machine learning . Are classification , recommendation and clustering libraries available ? What is the timeline for Flink as backend for Mahout? I am building a meta data driven framework over Flink . While building data collection and transformation modules was cool , I am struggling since I started analytics module. Thanks in advance. 

Ashutosh

[Vasiliki Kalavri]

Hi Ashutosh,

Flink has a Machine Learning library, Flink-ML. You can find more information and examples the documentation [1].

The code is currently in the flink-staging repository. There is also material on Slideshare that you might find interesting [2, 3, 4].

I hope this helps!

-Vasia.

[1]: https://ci.apache.org/projects/flink/flink-docs-master/libs/ml/

[2]: http://www.slideshare.net/tillrohrmann/machine-learning-with-apache-flink

[3]: http://www.slideshare.net/TheodorosVasiloudis/flinkml-large-scale-machine-learning-with-apache-flink

[4]: http://www.slideshare.net/tillrohrmann/computing-recommendations-at-extreme-scale-with-apache-flink-buzzwords-2015-48879155

On 9 January 2016 at 12:46, Ashutosh Kumar <[hidden email]> wrote:

I see lot of study materials and even book available for ml on spark. Spark seems to be more mature for analytics related work as of now. Please correct me if I am wrong. As I have already built my collection and data pre processing layers on flink , I want to use Flink for analytics as well. Thanks in advance.

Ashutosh

On Sat, Jan 9, 2016 at 3:32 PM, Ashutosh Kumar <[hidden email]> wrote:

I am looking for some study material and examples on machine learning . Are classification , recommendation and clustering libraries available ? What is the timeline for Flink as backend for Mahout? I am building a meta data driven framework over Flink . While building data collection and transformation modules was cool , I am struggling since I started analytics module. Thanks in advance. 

Ashutosh

[Till Rohrmann]

Hi Ashutosh,

Flink’s ML library flinkML is maybe not as extensive as Spark’s MLlib. However, Flink has native support for iterations which makes them blazingly fast. Iterations in Flink are a distinct operator so that they don’t have to communicate after each iteration with the client. Furthermore, Flink has support for delta iterations which allow you to iterate over a subset of the elements in DataSet. This can speed appropriate algorithms, such as many graph algorithms, considerably up.

If you have further questions while studying the available material feel free to write me.

Cheers,
Till

​

On Sat, Jan 9, 2016 at 2:20 PM, Vasiliki Kalavri <[hidden email]> wrote:

Hi Ashutosh,

Flink has a Machine Learning library, Flink-ML. You can find more information and examples the documentation [1].

The code is currently in the flink-staging repository. There is also material on Slideshare that you might find interesting [2, 3, 4].

I hope this helps!

-Vasia.

[1]: https://ci.apache.org/projects/flink/flink-docs-master/libs/ml/

[2]: http://www.slideshare.net/tillrohrmann/machine-learning-with-apache-flink

[3]: http://www.slideshare.net/TheodorosVasiloudis/flinkml-large-scale-machine-learning-with-apache-flink

[4]: http://www.slideshare.net/tillrohrmann/computing-recommendations-at-extreme-scale-with-apache-flink-buzzwords-2015-48879155

On 9 January 2016 at 12:46, Ashutosh Kumar <[hidden email]> wrote:

I see lot of study materials and even book available for ml on spark. Spark seems to be more mature for analytics related work as of now. Please correct me if I am wrong. As I have already built my collection and data pre processing layers on flink , I want to use Flink for analytics as well. Thanks in advance.

Ashutosh

On Sat, Jan 9, 2016 at 3:32 PM, Ashutosh Kumar <[hidden email]> wrote:

I am looking for some study material and examples on machine learning . Are classification , recommendation and clustering libraries available ? What is the timeline for Flink as backend for Mahout? I am building a meta data driven framework over Flink . While building data collection and transformation modules was cool , I am struggling since I started analytics module. Thanks in advance. 

Ashutosh

[Dmitriy Lyubimov]

still in the works (for mahout). but soon. 

On Sat, Jan 9, 2016 at 3:46 AM, Ashutosh Kumar <[hidden email]> wrote:

I see lot of study materials and even book available for ml on spark. Spark
seems to be more mature for analytics related work as of now. Please
correct me if I am wrong. As I have already built my collection and data
pre processing layers on flink , I want to use Flink for analytics as well.
Thanks in advance.


Ashutosh

On Sat, Jan 9, 2016 at 3:32 PM, Ashutosh Kumar <[hidden email]>
wrote:

> I am looking for some study material and examples on machine learning .
> Are classification , recommendation and clustering libraries available ?
> What is the timeline for Flink as backend for Mahout? I am building a meta
> data driven framework over Flink . While building data collection and
> transformation modules was cool , I am struggling since I started analytics
> module. Thanks in advance.
> Ashutosh
>