About the native-lambda branch

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
33 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

About the native-lambda branch

Daniel Sun
Hi all,

      I created the native-lambda branch and pushed the first commit to
support very basic native lambda in the static compilation mode[1]. After
native lambda is fully supported in the static compilation mode, I will try
to make native lambda work in the dynamic compilation mode(I wish I would
have enough spare time to complete...)

      Currently some synthetic methods are generated while parsing, so they
are lack of type information. it should be deferred. In addition, I am
thinking about generating another new class with the synthetic methods for
each lambda expression.

      Any help to improve the native lambda is welcome and appreciated!

Cheers,
Daniel.Sun

[1] https://github.com/apache/groovy/tree/native-lambda



--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Graeme Rocher-2
Awesome to see progress on this. Keep it up!

On Thu, Jan 11, 2018 at 9:07 AM, Daniel Sun <[hidden email]> wrote:

> Hi all,
>
>       I created the native-lambda branch and pushed the first commit to
> support very basic native lambda in the static compilation mode[1]. After
> native lambda is fully supported in the static compilation mode, I will try
> to make native lambda work in the dynamic compilation mode(I wish I would
> have enough spare time to complete...)
>
>       Currently some synthetic methods are generated while parsing, so they
> are lack of type information. it should be deferred. In addition, I am
> thinking about generating another new class with the synthetic methods for
> each lambda expression.
>
>       Any help to improve the native lambda is welcome and appreciated!
>
> Cheers,
> Daniel.Sun
>
> [1] https://github.com/apache/groovy/tree/native-lambda
>
>
>
> --
> Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html



--
Graeme Rocher
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Nathan Harvey
In reply to this post by Daniel Sun
Great work, Daniel. What's will the differences be between closures and
lambdas? I love that you can have fine control over closure scoping, but
lambdas play very well with functional interfaces, which are very important!



--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Daniel Sun
In reply to this post by Graeme Rocher-2
Time is the problem, but I'll try to set aside some time to push the progress
;-)

Cheers,
Daniel.Sun



--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Daniel Sun
In reply to this post by Nathan Harvey
Hi Nathan,

> What's will the differences be between closures and lambdas?
The native lambda will have better performance than closure. In addition,
native lambda, which conforms to Java specification,  is less versatile than
closure, which is really powerful...

Cheers,
Daniel.Sun



--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Jochen Theodorou


Am 12.01.2018 um 04:05 schrieb Daniel Sun:
> Hi Nathan,
>
>> What's will the differences be between closures and lambdas?
> The native lambda will have better performance than closure. In addition,
> native lambda, which conforms to Java specification,  is less versatile than
> closure, which is really powerful...

but I think you should then not use the closure inner class mechanism. I
had a bit a look at your code, but there is a ton of things missing. I
did also not understand the innclass attribute visit... I thought you
need that only to enable reflection to find your inner classes.
MethodHandles.Lookup is no such thing. And then I would also question
the early transformation you do where you store the lambda as body in a
method node... which resides in the current class and which may have a
name conflicting with an existing name... and a name check is something
I would not do in that ast builder. In fact I would do this logic just
before or even in ACG. Otherwise you will always have trouble with code
that transforms the lambda expression. And then again we miss totally
the handling of local variables, not in the lambda. And actually there
is a performance related decision to do here. if we follow the closure
logic, there will be a wrapper containing the value of the local
variable, which is then used in the lambda. This costs. Not much, but it
does. Since in Java you have only quasi final variables allowed, you do
not need any wrapper. Not using the wrapper means less Closure
implementation code we can use. But this wrapper also has quite special
handling in many areas, like the VariableScopeVisitor. Oh yes... should
you go with one inner class for all lambdas, then you have to add bridge
methods to be able to access private fields in the outer class.

bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Jesper Steen Møller
Hi Jochen and Daniel

> On 12 Jan 2018, at 12.48, Jochen Theodorou <[hidden email]> wrote:
>
>
>
> Am 12.01.2018 um 04:05 schrieb Daniel Sun:
>> Hi Nathan,
>>> What's will the differences be between closures and lambdas?
>> The native lambda will have better performance than closure. In addition,
>> native lambda, which conforms to Java specification,  is less versatile than
>> closure, which is really powerful...
>
> but I think you should then not use the closure inner class mechanism. I had a bit a look at your code, but there is a ton of things missing. I did also not understand the innclass attribute visit... I thought you need that only to enable reflection to find your inner classes. MethodHandles.Lookup is no such thing. And then I would also question the early transformation you do where you store the lambda as body in a method node... which resides in the current class and which may have a name conflicting with an existing name... and a name check is something I would not do in that ast builder. In fact I would do this logic just before or even in ACG. Otherwise you will always have trouble with code that transforms the lambda expression. And then again we miss totally the handling of local variables, not in the lambda. And actually there is a performance related decision to do here. if we follow the closure logic, there will be a wrapper containing the value of the local variable, which is then used in the lambda. This costs. Not much, but it does. Since in Java you have only quasi final variables allowed, you do not need any wrapper. Not using the wrapper means less Closure implementation code we can use. But this wrapper also has quite special handling in many areas, like the VariableScopeVisitor. Oh yes... should you go with one inner class for all lambdas, then you have to add bridge methods to be able to access private fields in the outer class.
>

I don't think we need to generate bridge classes at all. I'm not sure if you are familiar with how lambdas are implemented by javac, but they also uses a method for each lambda expression, and capture the rest at indy time. Javac alsot differentiates the lambda implementatoin function based on what is captured: The lambda method is an instance method if it captures anything nonstatic, otherwise it can be a static method (and thus possibly recycled).

For capturing locals, it would be optimal to do the same "effectively final" logic similar to what Java does, so we could simply capture the value itself.
If we couldn't guarantee that (which ranges from tricky to impossible), then we'd fallback to using groovy.lang.Reference like closures do.

Am I making sense? Otherwise, it'd be better to talk "in code", I guess.

-Jesper

Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Daniel.Sun
In reply to this post by Jochen Theodorou
Hi Jochen,

    Thanks for your reviewing the code. They are a bit old because I pushed
the latest commit this afternoon :-)

    As you see, the latest implementation is to generate method for lambda
at the class generation stage, in addition new inner classes will be
generated for each lambda.

    I reference the ASM to generate bytecode, maybe some code is not
necessary and can be removed, I will refine them later.

    Currently it is just an initial implementation, there are many things to
complete. Fortunately, we are on the road.

Cheers,
Daniel.Sun




--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Daniel.Sun
In reply to this post by Jesper Steen Møller
Hi Jesper,

      The bytecode generation for lambda of the current implementation
should be somehow same with javac does, because I reference the ASM code,
which is decomplied from bytecode generated by javac.

      If you have some spare time to contribute, glad to work with you again
;-)

Cheers,
Daniel.Sun



--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
Reply | Threaded
Open this post in threaded view
|

Re: About the native-lambda branch

Daniel Sun
In reply to this post by Jochen Theodorou
Hi Jochen,

> but I think you should then not use the closure inner class mechanism.
I remain the implementation for non-native lambda[1] to make lambda work in
legacy code, where closures are widely used. For example, `[1, 2,
3].collect(e -> e + 1)`

> I thought you need that only to enable reflection to find your inner
> classes.
Could you make it a bit detail? To be honest, I can not get your words...

Cheers,
Daniel.Sun

[1]
https://github.com/apache/groovy/blob/8096d278f09343b8acd9d68edf521e03b28b4062/src/main/java/org/codehaus/groovy/classgen/asm/sc/StaticTypesLambdaWriter.java#L84




--
Sent from: http://groovy.329449.n5.nabble.com/Groovy-Dev-f372993.html
1234