Groovy 3 + JDK9 and problems with our Closure

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Groovy 3 + JDK9 and problems with our Closure

Jochen Theodorou
Hi all,

during JCrete I was playing around with removing some of those
setAccessible calls and found one quite problematic case.

AS many here know we have some implementation details for our Closures,
one of them is the following: You can subclass Closure and declare a
doCall method which will the drive part of the logic (accepted arguments
and such) In fact all of our Closures usages in Groovy currently end up
in one or more doCall methods.

Now the problem is, if you declare an anonymous inner class of a Closure
with a doCall method in Java (yes, we do this in our codebase), then it
is quite difficult to give the runtime (or DGM) access to this method.

First I thought the super class Closure could have access if the method
is at least public, but AICs are not public, which automatically
restricts access to methods declared in them, not available otherwise.

I then started to play around with the AIC exhibiting a method handle to
Closure to allow Closure to call the doCall method using the
MethodHandles API. But this will of course add several lines of code to
the subclass, that where not needed before.

And then not to forget about our plan to make our Closures into
something more similar to lambdas in Java.

That did make me think...

(1) maybe Closure should have a factory method that allows a
MethodHandle (or more than one) to be properly registered (ClassValue I
am thinking here), to allow us to call doCall from call

(2) maybe the method should be more convenient and take a Lookup object
and method name instead for the user

(3) even better would be to be able to use a Java lambda for a Closure
but the Java type system does not really allow a nice solution for that,
which means Closure would become:

interface Closure {
   Object call(Object[] args);
   static Closure buildClosure(MethodHandle... h) {...}
   static Closure buildClosure(Lookup lookup, Class<?> baseClass, String
methodName) /*throws ReflectiveOeprationException */{...}
   // plus more here
}

(4) all old code using one of the other call methods or any doCall would
break. Java usage would have to change to either lambdas or one of the
buildClosure methods. Groovy usage could use the MethodHandle based
version and only the MethodHandle based version would include meta
information like the parameter count and types. Accessing this
information lazy could make it a thin wrapper around the handle,
allowing transformations from Closure to SAM cases, while still holding
some kind of reference.

(5) If I think of as lightweight as possible I automatically think of
ValueTypes... Then I wonder... would we rewrite Closure as ValueType of
a MethodHandle at some point (post JDK 11). If yes, should we
investigate more here and maybe go for JKD11+ instead of JDK9 with
Groovy 3? Frankly There is no good reason for me to go with Java 9
anymore, since it is already outdated and not even with LTS.


What do others think about this? Or should I just go an break some arms
and legs... ahm...Groovy code ?

bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: Groovy 3 + JDK9 and problems with our Closure

Guillaume Laforge
Administrator
Just a note on point (5): 
've seen customers and big companies thinking of only targeting JDK 11 as it's indeed gonna be the LTS version, and skipping everything below 11.
So moving from JDK 8 straight to JDK 11!
We might indeed have to rethink our base requirements with the new Java release schedule and LTS support.

Guillaume


On Wed, Aug 8, 2018 at 8:05 PM Jochen Theodorou <[hidden email]> wrote:
Hi all,

during JCrete I was playing around with removing some of those
setAccessible calls and found one quite problematic case.

AS many here know we have some implementation details for our Closures,
one of them is the following: You can subclass Closure and declare a
doCall method which will the drive part of the logic (accepted arguments
and such) In fact all of our Closures usages in Groovy currently end up
in one or more doCall methods.

Now the problem is, if you declare an anonymous inner class of a Closure
with a doCall method in Java (yes, we do this in our codebase), then it
is quite difficult to give the runtime (or DGM) access to this method.

First I thought the super class Closure could have access if the method
is at least public, but AICs are not public, which automatically
restricts access to methods declared in them, not available otherwise.

I then started to play around with the AIC exhibiting a method handle to
Closure to allow Closure to call the doCall method using the
MethodHandles API. But this will of course add several lines of code to
the subclass, that where not needed before.

And then not to forget about our plan to make our Closures into
something more similar to lambdas in Java.

That did make me think...

(1) maybe Closure should have a factory method that allows a
MethodHandle (or more than one) to be properly registered (ClassValue I
am thinking here), to allow us to call doCall from call

(2) maybe the method should be more convenient and take a Lookup object
and method name instead for the user

(3) even better would be to be able to use a Java lambda for a Closure
but the Java type system does not really allow a nice solution for that,
which means Closure would become:

interface Closure {
   Object call(Object[] args);
   static Closure buildClosure(MethodHandle... h) {...}
   static Closure buildClosure(Lookup lookup, Class<?> baseClass, String
methodName) /*throws ReflectiveOeprationException */{...}
   // plus more here
}

(4) all old code using one of the other call methods or any doCall would
break. Java usage would have to change to either lambdas or one of the
buildClosure methods. Groovy usage could use the MethodHandle based
version and only the MethodHandle based version would include meta
information like the parameter count and types. Accessing this
information lazy could make it a thin wrapper around the handle,
allowing transformations from Closure to SAM cases, while still holding
some kind of reference.

(5) If I think of as lightweight as possible I automatically think of
ValueTypes... Then I wonder... would we rewrite Closure as ValueType of
a MethodHandle at some point (post JDK 11). If yes, should we
investigate more here and maybe go for JKD11+ instead of JDK9 with
Groovy 3? Frankly There is no good reason for me to go with Java 9
anymore, since it is already outdated and not even with LTS.


What do others think about this? Or should I just go an break some arms
and legs... ahm...Groovy code ?

bye Jochen


--
Guillaume Laforge
Apache Groovy committer & PMC Vice-President
Developer Advocate @ Google Cloud Platform

Reply | Threaded
Open this post in threaded view
|

Re: Groovy 3 + JDK9 and problems with our Closure

h2gr
In reply to this post by Jochen Theodorou
Why would anyone subclass Closure? Isn't there enough trouble to be
found elsewhere?

H2


Den 2018-08-08 20:05, skrev Jochen Theodorou:

> Hi all,
>
> during JCrete I was playing around with removing some of those
> setAccessible calls and found one quite problematic case.
>
> AS many here know we have some implementation details for our
> Closures, one of them is the following: You can subclass Closure and
> declare a doCall method which will the drive part of the logic
> (accepted arguments and such) In fact all of our Closures usages in
> Groovy currently end up in one or more doCall methods.
>
> Now the problem is, if you declare an anonymous inner class of a
> Closure with a doCall method in Java (yes, we do this in our
> codebase), then it is quite difficult to give the runtime (or DGM)
> access to this method.
>
> First I thought the super class Closure could have access if the
> method is at least public, but AICs are not public, which
> automatically restricts access to methods declared in them, not
> available otherwise.
>
> I then started to play around with the AIC exhibiting a method handle
> to Closure to allow Closure to call the doCall method using the
> MethodHandles API. But this will of course add several lines of code
> to the subclass, that where not needed before.
>
> And then not to forget about our plan to make our Closures into
> something more similar to lambdas in Java.
>
> That did make me think...
>
> (1) maybe Closure should have a factory method that allows a
> MethodHandle (or more than one) to be properly registered (ClassValue
> I am thinking here), to allow us to call doCall from call
>
> (2) maybe the method should be more convenient and take a Lookup
> object and method name instead for the user
>
> (3) even better would be to be able to use a Java lambda for a Closure
> but the Java type system does not really allow a nice solution for
> that, which means Closure would become:
>
> interface Closure {
>   Object call(Object[] args);
>   static Closure buildClosure(MethodHandle... h) {...}
>   static Closure buildClosure(Lookup lookup, Class<?> baseClass,
> String methodName) /*throws ReflectiveOeprationException */{...}
>   // plus more here
> }
>
> (4) all old code using one of the other call methods or any doCall
> would break. Java usage would have to change to either lambdas or one
> of the buildClosure methods. Groovy usage could use the MethodHandle
> based version and only the MethodHandle based version would include
> meta information like the parameter count and types. Accessing this
> information lazy could make it a thin wrapper around the handle,
> allowing transformations from Closure to SAM cases, while still
> holding some kind of reference.
>
> (5) If I think of as lightweight as possible I automatically think of
> ValueTypes... Then I wonder... would we rewrite Closure as ValueType
> of a MethodHandle at some point (post JDK 11). If yes, should we
> investigate more here and maybe go for JKD11+ instead of JDK9 with
> Groovy 3? Frankly There is no good reason for me to go with Java 9
> anymore, since it is already outdated and not even with LTS.
>
>
> What do others think about this? Or should I just go an break some
> arms and legs... ahm...Groovy code ?
>
> bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: Groovy 3 + JDK9 and problems with our Closure

Romain Manni-Bucau



Le jeu. 9 août 2018 à 10:49, <[hidden email]> a écrit :
Why would anyone subclass Closure? Isn't there enough trouble to be
found elsewhere?

It is common in java code integrating with groovy code.
 

H2


Den 2018-08-08 20:05, skrev Jochen Theodorou:
> Hi all,
>
> during JCrete I was playing around with removing some of those
> setAccessible calls and found one quite problematic case.
>
> AS many here know we have some implementation details for our
> Closures, one of them is the following: You can subclass Closure and
> declare a doCall method which will the drive part of the logic
> (accepted arguments and such) In fact all of our Closures usages in
> Groovy currently end up in one or more doCall methods.
>
> Now the problem is, if you declare an anonymous inner class of a
> Closure with a doCall method in Java (yes, we do this in our
> codebase), then it is quite difficult to give the runtime (or DGM)
> access to this method.
>
> First I thought the super class Closure could have access if the
> method is at least public, but AICs are not public, which
> automatically restricts access to methods declared in them, not
> available otherwise.
>
> I then started to play around with the AIC exhibiting a method handle
> to Closure to allow Closure to call the doCall method using the
> MethodHandles API. But this will of course add several lines of code
> to the subclass, that where not needed before.
>
> And then not to forget about our plan to make our Closures into
> something more similar to lambdas in Java.
>
> That did make me think...
>
> (1) maybe Closure should have a factory method that allows a
> MethodHandle (or more than one) to be properly registered (ClassValue
> I am thinking here), to allow us to call doCall from call
>
> (2) maybe the method should be more convenient and take a Lookup
> object and method name instead for the user
>
> (3) even better would be to be able to use a Java lambda for a Closure
> but the Java type system does not really allow a nice solution for
> that, which means Closure would become:
>
> interface Closure {
>   Object call(Object[] args);
>   static Closure buildClosure(MethodHandle... h) {...}
>   static Closure buildClosure(Lookup lookup, Class<?> baseClass,
> String methodName) /*throws ReflectiveOeprationException */{...}
>   // plus more here
> }
>
> (4) all old code using one of the other call methods or any doCall
> would break. Java usage would have to change to either lambdas or one
> of the buildClosure methods. Groovy usage could use the MethodHandle
> based version and only the MethodHandle based version would include
> meta information like the parameter count and types. Accessing this
> information lazy could make it a thin wrapper around the handle,
> allowing transformations from Closure to SAM cases, while still
> holding some kind of reference.
>
> (5) If I think of as lightweight as possible I automatically think of
> ValueTypes... Then I wonder... would we rewrite Closure as ValueType
> of a MethodHandle at some point (post JDK 11). If yes, should we
> investigate more here and maybe go for JKD11+ instead of JDK9 with
> Groovy 3? Frankly There is no good reason for me to go with Java 9
> anymore, since it is already outdated and not even with LTS.
>
>
> What do others think about this? Or should I just go an break some
> arms and legs... ahm...Groovy code ?
>
> bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: Groovy 3 + JDK9 and problems with our Closure

Jochen Theodorou
In reply to this post by h2gr


Am 09.08.2018 um 10:49 schrieb [hidden email]:
> Why would anyone subclass Closure? Isn't there enough trouble to be
> found elsewhere?

hehe, We do it in our Java codebase a lot. And the compiler be default
works with inner classes extending Closure

bye Jochen
Reply | Threaded
Open this post in threaded view
|

Re: Groovy 3 + JDK9 and problems with our Closure

paulk_asert
In reply to this post by Jochen Theodorou
Sounds good. We have JDK8 as minimum for Groovy 3 but can
utilise the plugin system as needed. If we consider value types,
that sounds more like Groovy 4 to me. But in any case, we should just
try to get a spike branch happening and we can merge into wherever
makes sense once things are ready.

Cheers, Paul.


On Thu, Aug 9, 2018 at 4:05 AM Jochen Theodorou <[hidden email]> wrote:
Hi all,

during JCrete I was playing around with removing some of those
setAccessible calls and found one quite problematic case.

AS many here know we have some implementation details for our Closures,
one of them is the following: You can subclass Closure and declare a
doCall method which will the drive part of the logic (accepted arguments
and such) In fact all of our Closures usages in Groovy currently end up
in one or more doCall methods.

Now the problem is, if you declare an anonymous inner class of a Closure
with a doCall method in Java (yes, we do this in our codebase), then it
is quite difficult to give the runtime (or DGM) access to this method.

First I thought the super class Closure could have access if the method
is at least public, but AICs are not public, which automatically
restricts access to methods declared in them, not available otherwise.

I then started to play around with the AIC exhibiting a method handle to
Closure to allow Closure to call the doCall method using the
MethodHandles API. But this will of course add several lines of code to
the subclass, that where not needed before.

And then not to forget about our plan to make our Closures into
something more similar to lambdas in Java.

That did make me think...

(1) maybe Closure should have a factory method that allows a
MethodHandle (or more than one) to be properly registered (ClassValue I
am thinking here), to allow us to call doCall from call

(2) maybe the method should be more convenient and take a Lookup object
and method name instead for the user

(3) even better would be to be able to use a Java lambda for a Closure
but the Java type system does not really allow a nice solution for that,
which means Closure would become:

interface Closure {
   Object call(Object[] args);
   static Closure buildClosure(MethodHandle... h) {...}
   static Closure buildClosure(Lookup lookup, Class<?> baseClass, String
methodName) /*throws ReflectiveOeprationException */{...}
   // plus more here
}

(4) all old code using one of the other call methods or any doCall would
break. Java usage would have to change to either lambdas or one of the
buildClosure methods. Groovy usage could use the MethodHandle based
version and only the MethodHandle based version would include meta
information like the parameter count and types. Accessing this
information lazy could make it a thin wrapper around the handle,
allowing transformations from Closure to SAM cases, while still holding
some kind of reference.

(5) If I think of as lightweight as possible I automatically think of
ValueTypes... Then I wonder... would we rewrite Closure as ValueType of
a MethodHandle at some point (post JDK 11). If yes, should we
investigate more here and maybe go for JKD11+ instead of JDK9 with
Groovy 3? Frankly There is no good reason for me to go with Java 9
anymore, since it is already outdated and not even with LTS.


What do others think about this? Or should I just go an break some arms
and legs... ahm...Groovy code ?

bye Jochen