StochasticOutlierSelection

[anissa moussaoui]

Hello,

I would like to use the StochasticOutlierSelection algorithm for anomaly detection in a DataSet in java but the problem the doc is only in scala.

In java doing : 

StochasticOutlierSelection stochasticOutliers= new StochasticOutlierSelection();

the "tranform" operation of this algorithm take three parameters : training dataSet, ParameterMap and TransformOperationDataSet on java but in scala doc take one parameters https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/libs/ml/sos.html#parameters.

It's the same case for other algorithms like KNN un scala take only training dataset un scala but un java takes three parameters !

My question is that i do not know what represent ParameterMap and TransformOperationDataSet and how i initialize them and in an optimised way fr my model?

Do you have dôme exemple for flink ml un java please ?

Thank you in advance !

Best,

Anissa MOUSSAOUI 

      

 Pensez à la planète, imprimer ce papier que si nécessaire 

[Piotr Nowojski-3]

Hi,

I have never used this code, but ml library depends heavily on Scala, so I wouldn’t recommend using it with Java. 

However if you want to go this way (I’m not sure if that’s possible), you would have to pass the implicit parameters manually somehow (I don’t know how to do that from Java). In this case you can take a look at the method’s signature that parameters has a default value and the implicitly passed transformOperation comes from `StochasticOutlierSelection.transformLabeledVectors` or/and (?) `StochasticOutlierSelection.transformVectors`.

Piotrek

On 2 Mar 2019, at 12:21, anissa moussaoui <[hidden email]> wrote:

Hello,

I would like to use the StochasticOutlierSelection algorithm for anomaly detection in a DataSet in java but the problem the doc is only in scala.

In java doing : 

StochasticOutlierSelection stochasticOutliers= new StochasticOutlierSelection();

the "tranform" operation of this algorithm take three parameters : training dataSet, ParameterMap and TransformOperationDataSet on java but in scala doc take one parameters https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/libs/ml/sos.html#parameters.

It's the same case for other algorithms like KNN un scala take only training dataset un scala but un java takes three parameters !

My question is that i do not know what represent ParameterMap and TransformOperationDataSet and how i initialize them and in an optimised way fr my model?

Do you have dôme exemple for flink ml un java please ?

Thank you in advance !

Best,

Anissa MOUSSAOUI 

      

 Pensez à la planète, imprimer ce papier que si nécessaire 

[Piotr Nowojski-3]

(Adding back user mailing list)

Hi Anissa,

Thank you for coming back with the results. I hope this might be helpful for someone else in the future and maybe it will be one more argument for the Flink community to address this issue in some other way.

Piotrek

On 20 Mar 2019, at 17:26, anissa moussaoui <[hidden email]> wrote:

Hi Piotr,

thank you for your return and sorry for my reply so late. I took a little time to look for all the possibilities to use FlinkML in Java, but that's not possible as you told me. After looking in other libraries in Java, I chose the MOA library in combination with flink API for anomaly detection streaming which gives quite satisfactory results.

Best,

Anissa 

Le lun. 4 mars 2019 à 16:08, Piotr Nowojski <[hidden email]> a écrit :

Hi,

I have never used this code, but ml library depends heavily on Scala, so I wouldn’t recommend using it with Java. 

However if you want to go this way (I’m not sure if that’s possible), you would have to pass the implicit parameters manually somehow (I don’t know how to do that from Java). In this case you can take a look at the method’s signature that parameters has a default value and the implicitly passed transformOperation comes from `StochasticOutlierSelection.transformLabeledVectors` or/and (?) `StochasticOutlierSelection.transformVectors`.

Piotrek

On 2 Mar 2019, at 12:21, anissa moussaoui <[hidden email]> wrote:

Hello,

I would like to use the StochasticOutlierSelection algorithm for anomaly detection in a DataSet in java but the problem the doc is only in scala.

In java doing : 

StochasticOutlierSelection stochasticOutliers= new StochasticOutlierSelection();

the "tranform" operation of this algorithm take three parameters : training dataSet, ParameterMap and TransformOperationDataSet on java but in scala doc take one parameters https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/libs/ml/sos.html#parameters.

It's the same case for other algorithms like KNN un scala take only training dataset un scala but un java takes three parameters !

My question is that i do not know what represent ParameterMap and TransformOperationDataSet and how i initialize them and in an optimised way fr my model?

Do you have dôme exemple for flink ml un java please ?

Thank you in advance !

Best,

Anissa MOUSSAOUI 

      

 Pensez à la planète, imprimer ce papier que si nécessaire 

      

 Pensez à la planète, imprimer ce papier que si nécessaire

[anissa moussaoui]

---------- Forwarded message ---------
From: Piotr Nowojski <[hidden email]>
Date: jeu. 21 mars 2019 à 14:09
Subject: Re: StochasticOutlierSelection
To: anissa moussaoui <[hidden email]>, user <[hidden email]>

(Adding back user mailing list)

Hi Anissa,

Thank you for coming back with the results. I hope this might be helpful for someone else in the future and maybe it will be one more argument for the Flink community to address this issue in some other way.

Piotrek

On 20 Mar 2019, at 17:26, anissa moussaoui <[hidden email]> wrote:

Hi Piotr,

thank you for your return and sorry for my reply so late. I took a little time to look for all the possibilities to use FlinkML in Java, but that's not possible as you told me. After looking in other libraries in Java, I chose the MOA library in combination with flink API for anomaly detection streaming which gives quite satisfactory results.

Best,

Anissa 

Le lun. 4 mars 2019 à 16:08, Piotr Nowojski <[hidden email]> a écrit :

Hi,

I have never used this code, but ml library depends heavily on Scala, so I wouldn’t recommend using it with Java. 

However if you want to go this way (I’m not sure if that’s possible), you would have to pass the implicit parameters manually somehow (I don’t know how to do that from Java). In this case you can take a look at the method’s signature that parameters has a default value and the implicitly passed transformOperation comes from `StochasticOutlierSelection.transformLabeledVectors` or/and (?) `StochasticOutlierSelection.transformVectors`.

Piotrek

On 2 Mar 2019, at 12:21, anissa moussaoui <[hidden email]> wrote:

Hello,

I would like to use the StochasticOutlierSelection algorithm for anomaly detection in a DataSet in java but the problem the doc is only in scala.

In java doing : 

StochasticOutlierSelection stochasticOutliers= new StochasticOutlierSelection();

the "tranform" operation of this algorithm take three parameters : training dataSet, ParameterMap and TransformOperationDataSet on java but in scala doc take one parameters https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/libs/ml/sos.html#parameters.

It's the same case for other algorithms like KNN un scala take only training dataset un scala but un java takes three parameters !

My question is that i do not know what represent ParameterMap and TransformOperationDataSet and how i initialize them and in an optimised way fr my model?

Do you have dôme exemple for flink ml un java please ?

Thank you in advance !

Best,

Anissa MOUSSAOUI 

      

 Pensez à la planète, imprimer ce papier que si nécessaire 

      

 Pensez à la planète, imprimer ce papier que si nécessaire

      

 Pensez à la planète, imprimer ce papier que si nécessaire