Forcing dynamic dispatch

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Forcing dynamic dispatch

Dimitar Dimitrov
I just started writing some regression testing code that loads similar classes through different classloaders and I am finding myself ripping out static signatures all over the place, so we can take advantage of Groovy's dynamic dispatch.

It would have been very helpful if I could mark variables methods and classes as @Dynamic, so they would retain the type information for the compiler and IDE usage, but at runtime treat the annotated code as 'def' and always do dynamic dispatch without trying to cast.

I am thinking of writing an AST plug in for that, but I am posting this first to the community to see if it makes sense or if there is already something I can use.
Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

ocs@ocs
Dimitar,

On Feb 13, 2013, at 8:22 AM, Dimitar Dimitrov wrote:

> I just started writing some regression testing code that loads similar
> classes through different classloaders and I am finding myself ripping out
> static signatures all over the place, so we can take advantage of Groovy's
> dynamic dispatch.

Unless I am much wrong and completely misunderstand you, note please that the dynamic dispatch works quite regardless static/dynamic typing in Groovy. Consider e.g.,

class f {
  static def main(av) {
    String.metaClass.toUpperCase={"We are dynamic and $delegate!"}
    def o='typeless'
    String s='typed, no problem'
    println "${o.toUpperCase()}"
    println "${s.toUpperCase()}"
    println "${staticDispatch('we are static (and typed, of course)')}"
  }
  @CompileStatic static staticDispatch(String s) {
    s.toUpperCase()
  }
}

Can you perhaps show an example of a static signature which prevents you taking advantage of the dynamic dispatch, and how ripping the signature out helps?

I might be overlooking something obvious in which case I do apologize, but at the moment, I can see only two possibilities and none of them seems to me to fit your explanation:

(a) the methods/classes you are using were compiled with @CompileStatic -- in which case though removing the static signature would not really help; you would have to recompile the classes without the static annotation;

(b) the problem is not the Groovy static/dynamic dispatch, but rather the terrible design blunder of runtime-checked typecast. Groovy did inherit the abomination from Java; we have been discussing it lately -- see please e.g., my message "Subject: [groovy-user] Java/Groovy typecasts (was: Type checking and type inference)" of February 11, 2013 6:10:22 PM GMT+01:00.

In this case, removing static signatures is not what you want; you'd rather want to remove the checking typecasts -- but I would be rather surprised if that was your problem :)

All the best,
---
Ondra Čada
OCSoftware:     [hidden email]               http://www.ocs.cz
private         [hidden email]             http://www.ocs.cz/oc




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

Andre Steingress
In reply to this post by Dimitar Dimitrov
Hi,

You mean you want to do something like that?

@Dynamic class A {
List<Integer> f(List<String> s) { new B(o: s) }
}

@Dynamic class B {
    BigDecimal o  
}

def a = new A()
a.f(42) // works only with @Dynamic, cast exception otherwise

How should the method dispatch in this case look like? The actual type is 42=java.lang.Integer. Thus you want the code to dispatch the call to f(java.util.List)? What if the dynamic class code has calls to non-dynamic instances? What if there are multiple "f" methods with different signatures in the class hierarchy, does f(List) override f(List) from an non-dynamic upper class? what about co-variant return types? etc. etc.

As you can see from the example above, this just adds an enormous amount of code and runtime complexity - without any obvious advantage (apart from getting code completion you can't trust). I don't think it's worth to support it per se in Groovy. Of course you can add a custom parser plugin/AST transform in your project, but personally I don't see that as a language feature as long as 'def' can be used for dynamic variables.

btw: can you given an example where you had to remove the method type signature? I mean, if the method had a typed signature I would assume that the code was targeted to this type.

Cheers, André
Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

Graeme Rocher-4
In reply to this post by Dimitar Dimitrov
There is already @CompileDynamic in Groovy 2.1

Cheers


On Wed, Feb 13, 2013 at 8:22 AM, Dimitar Dimitrov <[hidden email]> wrote:
I just started writing some regression testing code that loads similar
classes through different classloaders and I am finding myself ripping out
static signatures all over the place, so we can take advantage of Groovy's
dynamic dispatch.

It would have been very helpful if I could mark variables methods and
classes as @Dynamic, so they would retain the type information for the
compiler and IDE usage, but at runtime treat the annotated code as 'def' and
always do dynamic dispatch without trying to cast.

I am thinking of writing an AST plug in for that, but I am posting this
first to the community to see if it makes sense or if there is already
something I can use.




--
View this message in context: http://groovy.329449.n5.nabble.com/Forcing-dynamic-dispatch-tp5713040.html
Sent from the groovy - user mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Graeme Rocher
Grails Project Lead
SpringSource - A Division of VMware
http://www.springsource.com
Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

Stephane Maldini
Also @CompileStatic(TypeCheckingMode.SKIP)  for Groovy 2.0. 

Cheers


On Wed, Feb 13, 2013 at 1:44 PM, Graeme Rocher <[hidden email]> wrote:
There is already @CompileDynamic in Groovy 2.1

Cheers


On Wed, Feb 13, 2013 at 8:22 AM, Dimitar Dimitrov <[hidden email]> wrote:
I just started writing some regression testing code that loads similar
classes through different classloaders and I am finding myself ripping out
static signatures all over the place, so we can take advantage of Groovy's
dynamic dispatch.

It would have been very helpful if I could mark variables methods and
classes as @Dynamic, so they would retain the type information for the
compiler and IDE usage, but at runtime treat the annotated code as 'def' and
always do dynamic dispatch without trying to cast.

I am thinking of writing an AST plug in for that, but I am posting this
first to the community to see if it makes sense or if there is already
something I can use.




--
View this message in context: http://groovy.329449.n5.nabble.com/Forcing-dynamic-dispatch-tp5713040.html
Sent from the groovy - user mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Graeme Rocher
Grails Project Lead
SpringSource - A Division of VMware
http://www.springsource.com



--
Stéphane
--

Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

Dimitar Dimitrov
In reply to this post by Andre Steingress
I mean something like:

ClassLoader cl = ClassLoaderBuilder // this is an internal class, but I hope it is obvious what it does
    .fromClassLoaderOf(MyClass)
    .removeClassPaths(".*\bmylib-.*")
    .prependClassPaths(
        new URL("http://nexus:8081/nexus/content/repositories/releases/my.group/5.0.0.23-rev499/mylib-5.0.0.23.jar"),
        ...
    ).build();

def foo = { MyClass it -> ... }

foo(new MyClass()) // works
foo(cl.loadClass(MyClass).newInstance()) // fails

What I would like is to be able to define foo as:

def foo = { @Dynamic MyClass it -> ... }

and have the code would be interpreted by the JVM as:

def foo = { @Dynamic def it -> ... }

This way I can take advantage of the static checking during development, and dynamic dispatch at runtime. The usefulness of this is limited, as the API of mylib.jar could change between versions, but that is very unlikely scenario (at least in our case) and it will fail at runtime anyway. It can also be useful when hacking OSGI tests, etc.

As far as I see the @CompileDynamic is just a shortcut for @CompileStatic(SKIP) which does not do what I need (i.e. the declared types are still checked at runtime).
Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

Jochen Theodorou
Am 15.02.2013 03:27, schrieb Dimitar Dimitrov:

> I mean something like:
>
> ClassLoader cl = ClassLoaderBuilder // this is an internal class, but I hope
> it is obvious what it does
>      .fromClassLoaderOf(MyClass)
>      .removeClassPaths(".*\bmylib-.*")
>      .prependClassPaths(
>          new
> URL("http://nexus:8081/nexus/content/repositories/releases/my.group/5.0.0.23-rev499/mylib-5.0.0.23.jar"),
>          ...
>      ).build();
>
> def foo = { MyClass it -> ... }
>
> foo(new MyClass()) // works
> foo(cl.loadClass(MyClass).newInstance()) // fails

it fails because the MyClass in this.class is a different one then the
one you load with loadClass. You cannot unload a class by removing the
classpath element it originates from. In Java you would have the same
problem for exampel.

> What I would like is to be able to define foo as:
>
> def foo = { @Dynamic MyClass it -> ... }
>
> and have the code would be interpreted by the JVM as:
>
> def foo = { @Dynamic def it -> ... }
>
> This way I can take advantage of the static checking during development, and
> dynamic dispatch at runtime.The usefulness of this is limited, as the API
> of mylib.jar could change between versions, but that is very unlikely
> scenario (at least in our case) and it will fail at runtime anyway. It can
> also be useful when hacking OSGI tests, etc.

the usual solution for this is to load those classes together with the
library and not putting everything in one class loader.

> As far as I see the @CompileDynamic is just a shortcut for
> @CompileStatic(SKIP) which does not do what I need (i.e. the declared types
> are still checked at runtime).

the declared types do always have a meaning. Method overloading is
surely one example, but they also ensure types by casting and
transformation. In your case you have to remove these so your semantics
is something entirely different.

bye blackdrag

--
Jochen "blackdrag" Theodorou - Groovy Project Tech Lead
blog: http://blackdragsview.blogspot.com/
german groovy discussion newsgroup: de.comp.lang.misc
For Groovy programming sources visit http://groovy-lang.org


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

Dimitar Dimitrov
Jochen, I realize what I am doing and I have (I think) a valid reason to do so.

Essentially I am running acceptance test, where I am bringing up multiple processes in the same JVM for the purpose of whitebox testing (let's not argue whether that is a good idea).

So far, we have been running the test with single classloader, which simulates the scenario where all processes are of the same version. Now we want to test the compatibility *between* versions, so by necessity we want processes of different versions to have separate classpaths, loading different versions of the classes.

Each process still uses a single classloader, and all IPC goes through sockets, so only the testing code needs to use multiple versions of the same class. Of course, we can force all tests to become classloader aware, but that adds unnecesarry layer of complexity. Instead we opted for removing the explicit types whenever some code needs to manipulate different versions of the same class. This was a significant productivity hit, as now we don't get autocompletion, which lead us to my suggestion outlined in the first post.

Anyway, as far as I see, there is nothing existing, so I guess I should roll up my sleeves and see if I can make it work.
Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

ocs@ocs
Dimitar,

On Feb 15, 2013, at 8:55 AM, Dimitar Dimitrov wrote:

> ... Anyway, as far as I see, there is nothing existing, so I guess I should roll
> up my sleeves and see if I can make it work.

I do apologize if I am on a completely wrong track, but as I wrote before, I very strongly suspect

(a) you don't really want to remove static typing;
(b) instead what you in fact want to remove is the abomination of the runtime-checking typecast.

I though am not sure how to do that and whether it is possible in the JVM thing at all :(

Also you need to be wary of the dynamic dispatch with overloaded methods -- presumed there is

class Foo {
  def foo(Object o) { ... }
  def foo(Bar b) { ... } //*
}

have you already ascertained that the dynamic dispatcher recognizes another version of the same class properly, so that

def bar=anotherLoader.get('Bar')
// at this moment, Bar typedbar=bar would crash, for you have got a different version of the class!
new Foo().foo(bar) // does this in fact call //*?

If it does, all right; if it does not, there's a big problem whose solution I can't really see.

All the best,
---
Ondra Čada
OCSoftware:     [hidden email]               http://www.ocs.cz
private         [hidden email]             http://www.ocs.cz/oc




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Forcing dynamic dispatch

Dimitar Dimitrov
Hi Ondřej, indeed, solving the general case would be difficult because of overloads and other stuff mentioned by Jochen. Luckily, my case is simpler. Typical use-case is this:

assert Threads.runnables.find { it.getClass().name==MyProcess.name }
           .streamHandler.eventHandlers.find { it.getClass().name==MyEventHandler.name}
           .store.lokup("blah", 123).account.crmCode = 'foobar'

Except that instead of writing three line assertions, we have a convenient API that given MyProcess and MyEventHandler returns the store with a  nice type, that allows you to get proper autocompletion and documentation.

Right now we have replaced these methods to return 'def' instead and we are achieving our immediate objective, which is telling that the approach is useful even if not semantically equivalent.

Just adding the type information back in a way that will not trigger the Groovy typechecks would be useful productivity boost, although it may lead to seemingly incorrect dispatch in case of overloaded methods.

If we are looking to solve the overloading problem, I would take different approach - i.e. serializing and deserializing the values using something like XStream, though that would have its own hard to trace pitfalls in case when backward compatibility breaks.