Naming Scala Operators

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Naming Scala Operators

Stefan Bunk
Hi,

is there a way to give a name to some operators in the Scala API?
So that the log lines
    Reduce(<Unnamed Reducer>) (1/1) switched to READY
are more helpful?

Cheers
Stefan
Reply | Threaded
Open this post in threaded view
|

Re: Naming Scala Operators

Aljoscha Krettek
Hi Stefan,
right now you have to use something like this:

val words = input.flatMap { ... }
words.contract.setName("the name")

contract is a field of DataSet with which you can access the underlying Operator.

Cheers,
Aljoscha


On Thu, Jul 31, 2014 at 5:28 PM, Stefan Bunk <[hidden email]> wrote:
Hi,

is there a way to give a name to some operators in the Scala API?
So that the log lines
    Reduce(<Unnamed Reducer>) (1/1) switched to READY
are more helpful?

Cheers
Stefan

Reply | Threaded
Open this post in threaded view
|

Re: Naming Scala Operators

Stefan Bunk
In reply to this post by Stefan Bunk
Hi,

thanks for that!

How about adding a name-method to the DataSet, that sets the name and returns itself? That would allow to stay in the fluent interface, and make naming intermediate results easier:
def name(name: String): DataSet[T] = {
contract.setName(name)
this
}
Something like that.

Cheers
Stefan


On Thu, Jul 31, 2014 at 6:32 PM, Aljoscha Krettek <[hidden email]> wrote:
Hi Stefan,
right now you have to use something like this:

val words = input.flatMap { ... }
words.contract.setName("the name")

contract is a field of DataSet with which you can access the underlying Operator.

Cheers,
Aljoscha


On Thu, Jul 31, 2014 at 5:28 PM, Stefan Bunk <[hidden email]> wrote:
Hi,

is there a way to give a name to some operators in the Scala API?
So that the log lines
    Reduce(<Unnamed Reducer>) (1/1) switched to READY
are more helpful?

Cheers
Stefan


Reply | Threaded
Open this post in threaded view
|

Re: Naming Scala Operators

Stephan Ewen
We also thought at some point in time to use the textual representation of the lambda / function name for the name.

Through the Macros, we should have access to the AST and shoul dbe able to call "toString()" on it.

For example, teh name could then be:
  - Lambda:  (a: Int, b: Int) => a + b 
  - Referencing a function: "computeSum"


Reply | Threaded
Open this post in threaded view
|

Re: Naming Scala Operators

Aljoscha Krettek
For a start, we could add the name() method and then later on set the user function code as the default name.


On Mon, Aug 4, 2014 at 2:10 PM, Stephan Ewen <[hidden email]> wrote:
We also thought at some point in time to use the textual representation of the lambda / function name for the name.

Through the Macros, we should have access to the AST and shoul dbe able to call "toString()" on it.

For example, teh name could then be:
  - Lambda:  (a: Int, b: Int) => a + b 
  - Referencing a function: "computeSum"



Reply | Threaded
Open this post in threaded view
|

Re: Naming Scala Operators

Stephan Ewen
The Scala API needs a lot of refactoring anyways, to bring it in sync with the Java API.

In the Java API, there is a way to define names. When both ought to be in sync, we should follow a similar wa there.

I think the Java API allows to set the name in the fluent method syntax, and takes the "toString()" method of the function object, if nothing else is defined. Can we to the same in the Scala API?