Naming Scala Operators

[Stefan Bunk]

Hi,

is there a way to give a name to some operators in the Scala API?

So that the log lines

    Reduce(<Unnamed Reducer>) (1/1) switched to READY

are more helpful?

Cheers

Stefan

[Aljoscha Krettek]

Hi Stefan,
right now you have to use something like this:

val words = input.flatMap { ... }

words.contract.setName("the name")

contract is a field of DataSet with which you can access the underlying Operator.

Cheers,

Aljoscha

On Thu, Jul 31, 2014 at 5:28 PM, Stefan Bunk <[hidden email]> wrote:

Hi,

is there a way to give a name to some operators in the Scala API?

So that the log lines

    Reduce(<Unnamed Reducer>) (1/1) switched to READY

are more helpful?

Cheers

Stefan

[Stefan Bunk]

Hi,

thanks for that!

How about adding a name-method to the DataSet, that sets the name and returns itself? That would allow to stay in the fluent interface, and make naming intermediate results easier:

def name(name: String): DataSet[T] = {

contract.setName(name)

this

}

Something like that.

Cheers

Stefan

On Thu, Jul 31, 2014 at 6:32 PM, Aljoscha Krettek <[hidden email]> wrote:

Hi Stefan,
right now you have to use something like this:

val words = input.flatMap { ... }

words.contract.setName("the name")

contract is a field of DataSet with which you can access the underlying Operator.

Cheers,

Aljoscha

On Thu, Jul 31, 2014 at 5:28 PM, Stefan Bunk <[hidden email]> wrote:

Hi,

is there a way to give a name to some operators in the Scala API?

So that the log lines

    Reduce(<Unnamed Reducer>) (1/1) switched to READY

are more helpful?

Cheers

Stefan

... [show rest of quote]

[Stephan Ewen]

We also thought at some point in time to use the textual representation of the lambda / function name for the name.

Through the Macros, we should have access to the AST and shoul dbe able to call "toString()" on it.

For example, teh name could then be:

  - Lambda:  (a: Int, b: Int) => a + b 

  - Referencing a function: "computeSum"

[Aljoscha Krettek]

For a start, we could add the name() method and then later on set the user function code as the default name.

On Mon, Aug 4, 2014 at 2:10 PM, Stephan Ewen <[hidden email]> wrote:

We also thought at some point in time to use the textual representation of the lambda / function name for the name.

Through the Macros, we should have access to the AST and shoul dbe able to call "toString()" on it.

For example, teh name could then be:

  - Lambda:  (a: Int, b: Int) => a + b 

  - Referencing a function: "computeSum"

[Stephan Ewen]

The Scala API needs a lot of refactoring anyways, to bring it in sync with the Java API.

In the Java API, there is a way to define names. When both ought to be in sync, we should follow a similar wa there.

I think the Java API allows to set the name in the fluent method syntax, and takes the "toString()" method of the function object, if nothing else is defined. Can we to the same in the Scala API?