Quantcast

help with an AST

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

help with an AST

Ed Clark
Happy New Year All!

As one of my New Year's resolutions, I'm going to try and implement a programming
construct that I've been want to play with for a while - and I think a Groovy AST just
might let me do it.

While the syntax part of this is straightforward (I think), I'm not as clear on the
how of the method resolution logic that will be at the core.

The basic idea is that I'll be able to designate a section of code where binary
operators would be resolved not against the neighbouring object's class, but against
the surrounding context.  Perhaps some code will make my goal clearer....

My understanding of Groovy (and OO in general) is that an expression like
     obj1 + obj2
is interpreted as if it were written
     obj1.plus( obj2)

What I'd like to do is have something like
     withCtx( ctx1) {
         obj1 + obj2
     }
get interpreted as
     ctx1.plus( obj1, obj2)


If I squint at this, it kind of sort of looks to me a bit like how the "with" construct
in Groovy works.  But, the with construct is an enhancement to the existing OO
method resolution; this construct would go in a different direction.

(Actually, to be useful, the interior expression would probably be something
like -- var1 = obj1 + obj2 -- but I want to focus on one step/method at a time.)

With this idea, methods between objects (e.g. binary methods) don't belong to the
class of (one of the) objects involved, but to the context within which the objects
are being used.

And, just to make things more challenging, I'd like to be able to nest and compose
contexts.

Consider
     withCtx( Ctx1) {
         withCtx( ctx2, ctx3) {
             obj1 + obj2
         }
     }

Here, the methods of ctx2 would be checked to see if there's a "plus" that can take
arguments obj1 and obj2.  If not, then ctx3's methods will be check.  If there's no
appropriate "plus" in ctx3, then Ctx1's methods will be checked.

Finally, if there are no appropriate context methods for "plus-ing" these two
objects, then method resolution would fall back to the normal Groovy approach.

To add yet another wrinkle, I'd like to be able to use either a Context class or a context
instance when doing this.  So, in the above, if "Ctx1" is a class name (note the cap C),
its static methods would be used.  But, assuming ctx2 and ctx3 are context instances
(note the lower case c's), the appropriate instance methods would be used.

Hopefully this description makes my goal clearer.

My questions for you are,

     - is an AST the way to go?

     - I'm guessing that I can do the context nesting and composing by building on
       methodMissing (with ctx2 chaining to ctx3 chaining to Ctx1 above), but is there
       a better way?

     - what might I be breaking in Groovy by doing this?

     - what help/hints can I get from the compiler vs. doing this all at runtime?


Thanks for your time,

Ed Clark
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Jochen Theodorou
Hi Ed,

I am wondering if categories would actually do what you want already
without touching the AST. Groovy Categories are actually no syntactic
construct and their scope is not lexical (thread local instead)... but maybe

Let us assume you have two Strings obj1 and obj2, then you can define a
category:

class StringCategory {
   static String plus(String obj1, String obj2) { "from StringCategory
$obj1 $obj2" }
}

and code like this:

use (StringCategory) {
   assert "a"+"b" == "from StringCategory a b"
}

the construct supports nesting from inside out, that means the inner
context wins over the outer context and the rightmost over the leftmost.

What this does not support is an instance based context.

Assuming you still want to do something like this with more... AST
involvement...

> My questions for you are,
>
>      - is an AST the way to go?

first you have to decide what you want to compile to and then we can
think about how the AST for that has to look like. So this is actually
the last step. But assuming you have some kind of handler object you
could compile a+b to handler.invoke("plus",a,b)... in other words you
would replace almost any expression by a MethodCallExpression. Hint:
ExpressonTransformer and helper classes.

>      - I'm guessing that I can do the context nesting and composing by building on
>        methodMissing (with ctx2 chaining to ctx3 chaining to Ctx1 above), but is there
>        a better way?

I guess the code path for categories won´t help you here, since that is
kind of special treated. You could create a special meta class and all
the context methods to it, then use its mechanisms to select an
appropriate method... hint getMetaMethod, init


>      - what might I be breaking in Groovy by doing this?

that depends on how you do it.

>      - what help/hints can I get from the compiler vs. doing this all at runtime?

worry about such things once you have come up with something that works ;)

original mail:
On 30.12.2016 20:02, Ed Clark wrote:

> Happy New Year All!
>
> As one of my New Year's resolutions, I'm going to try and implement a
> programming
> construct that I've been want to play with for a while - and I think a
> Groovy AST just
> might let me do it.
>
> While the syntax part of this is straightforward (I think), I'm not as
> clear on the
> how of the method resolution logic that will be at the core.
>
> The basic idea is that I'll be able to designate a section of code where
> binary
> operators would be resolved not against the neighbouring object's class,
> but against
> the surrounding context.  Perhaps some code will make my goal clearer....
>
> My understanding of Groovy (and OO in general) is that an expression like
>      obj1 + obj2
> is interpreted as if it were written
>      obj1.plus( obj2)
>
> What I'd like to do is have something like
>      withCtx( ctx1) {
>          obj1 + obj2
>      }
> get interpreted as
>      ctx1.plus( obj1, obj2)
>
>
> If I squint at this, it kind of sort of looks to me a bit like how the
> "with" construct
> in Groovy works.  But, the with construct is an enhancement to the
> existing OO
> method resolution; this construct would go in a different direction.
>
> (Actually, to be useful, the interior expression would probably be
> something
> like -- var1 = obj1 + obj2 -- but I want to focus on one step/method at
> a time.)
>
> With this idea, methods between objects (e.g. binary methods) don't
> belong to the
> class of (one of the) objects involved, but to the context within which
> the objects
> are being used.
>
> And, just to make things more challenging, I'd like to be able to nest
> and compose
> contexts.
>
> Consider
>      withCtx( Ctx1) {
>          withCtx( ctx2, ctx3) {
>              obj1 + obj2
>          }
>      }
>
> Here, the methods of ctx2 would be checked to see if there's a "plus"
> that can take
> arguments obj1 and obj2.  If not, then ctx3's methods will be check.  If
> there's no
> appropriate "plus" in ctx3, then Ctx1's methods will be checked.
>
> Finally, if there are no appropriate context methods for "plus-ing"
> these two
> objects, then method resolution would fall back to the normal Groovy
> approach.
>
> To add yet another wrinkle, I'd like to be able to use either a Context
> class or a context
> instance when doing this.  So, in the above, if "Ctx1" is a class name
> (note the cap C),
> its static methods would be used.  But, assuming ctx2 and ctx3 are
> context instances
> (note the lower case c's), the appropriate instance methods would be used.
>
> Hopefully this description makes my goal clearer.
>
> My questions for you are,
>
>      - is an AST the way to go?
>
>      - I'm guessing that I can do the context nesting and composing by
> building on
>        methodMissing (with ctx2 chaining to ctx3 chaining to Ctx1
> above), but is there
>        a better way?
>
>      - what might I be breaking in Groovy by doing this?
>
>      - what help/hints can I get from the compiler vs. doing this all at
> runtime?
>
>
> Thanks for your time,
>
> Ed Clark

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Ed Clark
Hi Jochen,

Thanks for the suggestions.  Replies in line...

On 12/30/2016 03:20 PM, Jochen Theodorou wrote:

> Hi Ed,
>
> I am wondering if categories would actually do what you want already without touching the AST.
> Groovy Categories are actually no syntactic construct and their scope is not lexical (thread local
> instead)... but maybe
>
> Let us assume you have two Strings obj1 and obj2, then you can define a category:
>
> class StringCategory {
>   static String plus(String obj1, String obj2) { "from StringCategory $obj1 $obj2" }
> }
>
> and code like this:
>
> use (StringCategory) {
>   assert "a"+"b" == "from StringCategory a b"
> }
>
> the construct supports nesting from inside out, that means the inner context wins over the outer
> context and the rightmost over the leftmost.
>
> What this does not support is an instance based context.

I don't think categories will meet my wants.  Categories are a way to enhance a specific class,
whereas I'd like to have my "contexts" deal with objects of a variety of class types.

>
> Assuming you still want to do something like this with more... AST involvement...
>
>> My questions for you are,
>>
>>      - is an AST the way to go?
>
> first you have to decide what you want to compile to and then we can think about how the AST for
> that has to look like. So this is actually the last step. But assuming you have some kind of
> handler object you could compile a+b to handler.invoke("plus",a,b)... in other words you would
> replace almost any expression by a MethodCallExpression. Hint: ExpressonTransformer and helper
> classes.

Hmmm, I will have a handler object - sort of - and that's the context object itself.  Well, at
least in the case of instance based contexts.  For class based contexts, there won't be
an instantiated handler, but I could still see having a static "invoke" method.

I hadn't considered the invoke approach; it is interesting.  I think it would provide some
additional flexibility that the context developer could use to add "operators" beyond the
ones built into the Groovy parser/AST.  I'll have to think about this some more.


>
>>      - I'm guessing that I can do the context nesting and composing by building on
>>        methodMissing (with ctx2 chaining to ctx3 chaining to Ctx1 above), but is there
>>        a better way?
>
> I guess the code path for categories won´t help you here, since that is kind of special treated.
> You could create a special meta class and all the context methods to it, then use its mechanisms
> to select an appropriate method... hint getMetaMethod, init

Just to be clear, I don't think there is anything special about the contexts - they are
just POGOs.  Their methods just happen to be focused on manipulating objects that
fall under their scope.  So, I'm not sure how creating a meta class would help.

Remember, I'm trying to get away from the semantics of associating the "plus" in
"obj1 + obj2" back to obj1 (or any potential meta class).  Instead, I'm trying to
associate the "plus" back to the surrounding context (or possibly handler as you
suggest above).

>
>
>>      - what might I be breaking in Groovy by doing this?
>
> that depends on how you do it.
>
>>      - what help/hints can I get from the compiler vs. doing this all at runtime?
>
> worry about such things once you have come up with something that works ;)

Yeah, one of my problems is I think too far ahead ;).  I start worrying about what
it might take to run a marathon when I haven't even learned to walk.  So I don't
even get started.  Part of the this resolution is to just get off my butt and write
some code.




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Jochen Theodorou


On 31.12.2016 17:32, Ed Clark wrote:
[...]

>> Assuming you still want to do something like this with more... AST
>> involvement...
>>
>>> My questions for you are,
>>>
>>>      - is an AST the way to go?
>>
>> first you have to decide what you want to compile to and then we can
>> think about how the AST for that has to look like. So this is actually
>> the last step. But assuming you have some kind of handler object you
>> could compile a+b to handler.invoke("plus",a,b)... in other words you
>> would replace almost any expression by a MethodCallExpression. Hint:
>> ExpressonTransformer and helper classes.
>
> Hmmm, I will have a handler object - sort of - and that's the context
> object itself.  Well, at
> least in the case of instance based contexts.  For class based contexts,
> there won't be
> an instantiated handler, but I could still see having a static "invoke"
> method.
>
> I hadn't considered the invoke approach; it is interesting.  I think it
> would provide some
> additional flexibility that the context developer could use to add
> "operators" beyond the
> ones built into the Groovy parser/AST.  I'll have to think about this
> some more.

the handler object does not have to be the context object itself. It can
be a collection of context objects, be it classes or instances. It could
be also be a collection of methods and instances from the context.
Important for the compiler part is only the calling convention.

>>>      - I'm guessing that I can do the context nesting and composing by building on
>>>        methodMissing (with ctx2 chaining to ctx3 chaining to Ctx1 above), but is there
>>>        a better way?
>>
>> I guess the code path for categories won´t help you here, since that
>> is kind of special treated. You could create a special meta class and
>> all the context methods to it, then use its mechanisms to select an
>> appropriate method... hint getMetaMethod, init
>
> Just to be clear, I don't think there is anything special about the
> contexts - they are
> just POGOs.  Their methods just happen to be focused on manipulating
> objects that
> fall under their scope.  So, I'm not sure how creating a meta class
> would help.

you would not really create a meta class of a specific class, you would
create a meta class in the sense of an enumeration of methods and method
selection mechanism. But well... now need for that at this point I
guess. You can of course simply work with methodMissing.

[...]
> Yeah, one of my problems is I think too far ahead ;).  I start worrying
> about what
> it might take to run a marathon when I haven't even learned to walk.  So
> I don't
> even get started.  Part of the this resolution is to just get off my
> butt and write
> some code.

that's a good resolution for the new year ;)

Let me help you: assuming you use "with"-blocks:

Step 1) write an AST transformation that will recognize the with-block
and store the context in a variable

Step 2) add an expression transformer, that will transform all the
expressions of interest in the with-block to method calls based on the
context variable

Step 3) implement a context

Step 4) transform nested with method calls to calls that allow you the
creation and nesting of a context plus add code to delegate to outer
contexts

Even if it turns out not to be useful in the end (and I think it will be
useful) you will still get some practice about writing such things.
Nothing to really loose here imho ;)

bye Jochen
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Ed Clark
Hi Jochen,

Well, I've been slowly making progress on this; kind of feels like bashing my
head against a wall at times, but I have made a small hole in the wall through
which I can see my goal. ;-)

Unfortunately, I've reached a point where the hole isn't getting any bigger and
my google-fu isn't up to the task of finding what I need to make it bigger.

Specifically, I haven't figured out how to inject a variable into the scope of an outer
with's closure that can be used by code in an interior with's closure.  For example,
consider

   myCtx1.with {
       __outerCtx1 = null            // injected statement
       __currCtx1 = myCtx1       // injected statement
       .... some code ...
       myCtx2.with {
           __outerCtx2 = __currCtx1   // injected statement <---  doesn't work
           __currCtx2 = myCtx2          // injected statement

My problem is the compiler chokes on the reference to __currCtx1 in the inner
with closure, claiming that __currCtx1 isn't a property of myCtx1.

My google-fu only tells me I haven't set up the scoping correctly, but for the life of me
I haven't found the correct way to do this.  And further experimenting, groping in the
dark, and head banging hasn't helped either. :\

Any illuminating insight, or direction pointing, or example would be greatly appreciated!

Thanks

Ed

On 12/31/2016 06:36 PM, Jochen Theodorou wrote:

>
>
> On 31.12.2016 17:32, Ed Clark wrote:
> [...]
>>> Assuming you still want to do something like this with more... AST
>>> involvement...
>>>
>>>> My questions for you are,
>>>>
>>>>      - is an AST the way to go?
>>>
>>> first you have to decide what you want to compile to and then we can
>>> think about how the AST for that has to look like. So this is actually
>>> the last step. But assuming you have some kind of handler object you
>>> could compile a+b to handler.invoke("plus",a,b)... in other words you
>>> would replace almost any expression by a MethodCallExpression. Hint:
>>> ExpressonTransformer and helper classes.
>>
>> Hmmm, I will have a handler object - sort of - and that's the context
>> object itself.  Well, at
>> least in the case of instance based contexts.  For class based contexts,
>> there won't be
>> an instantiated handler, but I could still see having a static "invoke"
>> method.
>>
>> I hadn't considered the invoke approach; it is interesting.  I think it
>> would provide some
>> additional flexibility that the context developer could use to add
>> "operators" beyond the
>> ones built into the Groovy parser/AST.  I'll have to think about this
>> some more.
>
> the handler object does not have to be the context object itself. It can be a collection of
> context objects, be it classes or instances. It could be also be a collection of methods and
> instances from the context. Important for the compiler part is only the calling convention.
>
>>>>      - I'm guessing that I can do the context nesting and composing by building on
>>>>        methodMissing (with ctx2 chaining to ctx3 chaining to Ctx1 above), but is there
>>>>        a better way?
>>>
>>> I guess the code path for categories won´t help you here, since that
>>> is kind of special treated. You could create a special meta class and
>>> all the context methods to it, then use its mechanisms to select an
>>> appropriate method... hint getMetaMethod, init
>>
>> Just to be clear, I don't think there is anything special about the
>> contexts - they are
>> just POGOs.  Their methods just happen to be focused on manipulating
>> objects that
>> fall under their scope.  So, I'm not sure how creating a meta class
>> would help.
>
> you would not really create a meta class of a specific class, you would create a meta class in the
> sense of an enumeration of methods and method selection mechanism. But well... now need for that
> at this point I guess. You can of course simply work with methodMissing.
>
> [...]
>> Yeah, one of my problems is I think too far ahead ;).  I start worrying
>> about what
>> it might take to run a marathon when I haven't even learned to walk.  So
>> I don't
>> even get started.  Part of the this resolution is to just get off my
>> butt and write
>> some code.
>
> that's a good resolution for the new year ;)
>
> Let me help you: assuming you use "with"-blocks:
>
> Step 1) write an AST transformation that will recognize the with-block and store the context in a
> variable
>
> Step 2) add an expression transformer, that will transform all the expressions of interest in the
> with-block to method calls based on the context variable
>
> Step 3) implement a context
>
> Step 4) transform nested with method calls to calls that allow you the creation and nesting of a
> context plus add code to delegate to outer contexts
>
> Even if it turns out not to be useful in the end (and I think it will be useful) you will still
> get some practice about writing such things. Nothing to really loose here imho ;)
>
> bye Jochen


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Jochen Theodorou


On 14.02.2017 12:41, Ed Clark wrote:
> Hi Jochen,
>
> Well, I've been slowly making progress on this; kind of feels like
> bashing my
> head against a wall at times, but I have made a small hole in the wall
> through
> which I can see my goal. ;-)

making holes my lead to brain damage.. then things get really hard to
solve ;)

[...]

> Specifically, I haven't figured out how to inject a variable into the
> scope of an outer
> with's closure that can be used by code in an interior with's closure.
> For example,
> consider
>
>   myCtx1.with {
>       __outerCtx1 = null            // injected statement
>       __currCtx1 = myCtx1       // injected statement
>       .... some code ...
>       myCtx2.with {
>           __outerCtx2 = __currCtx1   // injected statement <---  doesn't
> work
>           __currCtx2 = myCtx2          // injected statement

where is __currCtx1 and __outerCtx1 coming from? Are they supposed to be
new local variables? in that case you will have to add a
DeclarationStatement, not just an assignment to a VariableExpression.
Otherwise the compiler will thnk they are dynamic properties and tries
to resolve them against the context.

as for the logic behind __outerCtxN... with number wouldn't have to be
null I would have said you should think of using getOwner on Closure

  bye Jochen
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Ed Clark
On 02/14/2017 12:16 PM, Jochen Theodorou wrote:

>
>
> On 14.02.2017 12:41, Ed Clark wrote:
>> Hi Jochen,
>>
>> Well, I've been slowly making progress on this; kind of feels like
>> bashing my
>> head against a wall at times, but I have made a small hole in the wall
>> through
>> which I can see my goal. ;-)
>
> making holes my lead to brain damage.. then things get really hard to solve ;)
Maybe I'm too late, and that's why things aren't coming together ;)

>
> [...]
>> Specifically, I haven't figured out how to inject a variable into the
>> scope of an outer
>> with's closure that can be used by code in an interior with's closure.
>> For example,
>> consider
>>
>>   myCtx1.with {
>>       __outerCtx1 = null            // injected statement
>>       __currCtx1 = myCtx1       // injected statement
>>       .... some code ...
>>       myCtx2.with {
>>           __outerCtx2 = __currCtx1   // injected statement <---  doesn't
>> work
>>           __currCtx2 = myCtx2          // injected statement
>
> where is __currCtx1 and __outerCtx1 coming from? Are they supposed to be new local variables? in
> that case you will have to add a DeclarationStatement, not just an assignment to a
> VariableExpression. Otherwise the compiler will thnk they are dynamic properties and tries to
> resolve them against the context.
>
> as for the logic behind __outerCtxN... with number wouldn't have to be null I would have said you
> should think of using getOwner on Closure
>
Hmmm, I'm close to that, but not quite the same.  I was trying

    ExpressionStatement currCtx = new ExpressionStatement(
        new DeclarationExpression (
            new VariableExpression( "__currCtx$levelcnt"),
            new Token( Types.ASSIGNMENT_OPERATOR, "=", -1, -1),
            new MethodCallExpression( new VariableExpression( 'this'), 'getDelegate', new
ArgumentListExpression())
        )
    )

Would a DeclarationStatment act differently from an ExpressionStatement with an embedded
DeclarationExpression?  If my foggy memory is correct, I wrote my code after looking at the
AST displayed for a short test script in groovyConsole.

Plus, if foggy memory serves, I tried using getOwner and it came back with the closure being
owned by the test script, not the object doing the with.  Calling getDelegate gave me the object.
If foggy memory serves.

I also tried playing around with setClosureSharedVariable( true) and putReferencedLocalVariable()
without success.  But, those were somewhat blind stabs in the dark, so I might not have been
using them correctly.

Looking at the AST generated by groovyConsole, my transform looks like it is working, but
the compiler complains about things not being in scope (in the inner closure).  So, I'm doing
something wrong with setting up the scoping.

>  bye Jochen

Thanks,

Ed

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Ed Clark
Ok, a little more headbanging and I seem to have gotten farther (maybe?).
At least the error message has changed.  Now I'm getting:

BUG! exception in phase 'class generation' in source unit 'CtxTest2.groovy' Operation: (<assignment
operator>:  "=" ) not supported

Searching hasn't helped with this one.

The xform-ed code looks good -- in fact, if I type the code into a separate
groovyConsole, it will run.  (The console AST generation works up thru
the Instruction Selection phase, obviously.)

I'm guessing it's still with how I'm setting up the scoping (or how I'm not
setting up the scoping).

Ideas, please?

Thanks,

Ed

On 02/14/2017 12:58 PM, Ed Clark wrote:

> On 02/14/2017 12:16 PM, Jochen Theodorou wrote:
>>
>>
>> On 14.02.2017 12:41, Ed Clark wrote:
>>> Hi Jochen,
>>>
>>> Well, I've been slowly making progress on this; kind of feels like
>>> bashing my
>>> head against a wall at times, but I have made a small hole in the wall
>>> through
>>> which I can see my goal. ;-)
>>
>> making holes my lead to brain damage.. then things get really hard to solve ;)
> Maybe I'm too late, and that's why things aren't coming together ;)
>
>>
>> [...]
>>> Specifically, I haven't figured out how to inject a variable into the
>>> scope of an outer
>>> with's closure that can be used by code in an interior with's closure.
>>> For example,
>>> consider
>>>
>>>   myCtx1.with {
>>>       __outerCtx1 = null            // injected statement
>>>       __currCtx1 = myCtx1       // injected statement
>>>       .... some code ...
>>>       myCtx2.with {
>>>           __outerCtx2 = __currCtx1   // injected statement <---  doesn't
>>> work
>>>           __currCtx2 = myCtx2          // injected statement
>>
>> where is __currCtx1 and __outerCtx1 coming from? Are they supposed to be new local variables? in
>> that case you will have to add a DeclarationStatement, not just an assignment to a
>> VariableExpression. Otherwise the compiler will thnk they are dynamic properties and tries to
>> resolve them against the context.
>>
>> as for the logic behind __outerCtxN... with number wouldn't have to be null I would have said you
>> should think of using getOwner on Closure
>>
> Hmmm, I'm close to that, but not quite the same.  I was trying
>
>    ExpressionStatement currCtx = new ExpressionStatement(
>        new DeclarationExpression (
>            new VariableExpression( "__currCtx$levelcnt"),
>            new Token( Types.ASSIGNMENT_OPERATOR, "=", -1, -1),
>            new MethodCallExpression( new VariableExpression( 'this'), 'getDelegate', new
> ArgumentListExpression())
>        )
>    )
>
> Would a DeclarationStatment act differently from an ExpressionStatement with an embedded
> DeclarationExpression?  If my foggy memory is correct, I wrote my code after looking at the
> AST displayed for a short test script in groovyConsole.
>
> Plus, if foggy memory serves, I tried using getOwner and it came back with the closure being
> owned by the test script, not the object doing the with.  Calling getDelegate gave me the object.
> If foggy memory serves.
>
> I also tried playing around with setClosureSharedVariable( true) and putReferencedLocalVariable()
> without success.  But, those were somewhat blind stabs in the dark, so I might not have been
> using them correctly.
>
> Looking at the AST generated by groovyConsole, my transform looks like it is working, but
> the compiler complains about things not being in scope (in the inner closure).  So, I'm doing
> something wrong with setting up the scoping.
>
>>  bye Jochen
>
> Thanks,
>
> Ed
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

paulk_asert
Do you have something like:

VariableScopeVisitor scopeVisitor = new VariableScopeVisitor(source)
scopeVisitor.visitClass(cNode)

for each of your closure(s)? Where cNode is a closure's classNode?

Cheers, Paul.


On Thu, Feb 16, 2017 at 9:35 PM, Ed Clark <[hidden email]> wrote:

> Ok, a little more headbanging and I seem to have gotten farther (maybe?).
> At least the error message has changed.  Now I'm getting:
>
> BUG! exception in phase 'class generation' in source unit 'CtxTest2.groovy'
> Operation: (<assignment operator>:  "=" ) not supported
>
> Searching hasn't helped with this one.
>
> The xform-ed code looks good -- in fact, if I type the code into a separate
> groovyConsole, it will run.  (The console AST generation works up thru
> the Instruction Selection phase, obviously.)
>
> I'm guessing it's still with how I'm setting up the scoping (or how I'm not
> setting up the scoping).
>
> Ideas, please?
>
> Thanks,
>
> Ed
>
>
> On 02/14/2017 12:58 PM, Ed Clark wrote:
>>
>> On 02/14/2017 12:16 PM, Jochen Theodorou wrote:
>>>
>>>
>>>
>>> On 14.02.2017 12:41, Ed Clark wrote:
>>>>
>>>> Hi Jochen,
>>>>
>>>> Well, I've been slowly making progress on this; kind of feels like
>>>> bashing my
>>>> head against a wall at times, but I have made a small hole in the wall
>>>> through
>>>> which I can see my goal. ;-)
>>>
>>>
>>> making holes my lead to brain damage.. then things get really hard to
>>> solve ;)
>>
>> Maybe I'm too late, and that's why things aren't coming together ;)
>>
>>>
>>> [...]
>>>>
>>>> Specifically, I haven't figured out how to inject a variable into the
>>>> scope of an outer
>>>> with's closure that can be used by code in an interior with's closure.
>>>> For example,
>>>> consider
>>>>
>>>>   myCtx1.with {
>>>>       __outerCtx1 = null            // injected statement
>>>>       __currCtx1 = myCtx1       // injected statement
>>>>       .... some code ...
>>>>       myCtx2.with {
>>>>           __outerCtx2 = __currCtx1   // injected statement <---  doesn't
>>>> work
>>>>           __currCtx2 = myCtx2          // injected statement
>>>
>>>
>>> where is __currCtx1 and __outerCtx1 coming from? Are they supposed to be
>>> new local variables? in that case you will have to add a
>>> DeclarationStatement, not just an assignment to a VariableExpression.
>>> Otherwise the compiler will thnk they are dynamic properties and tries to
>>> resolve them against the context.
>>>
>>> as for the logic behind __outerCtxN... with number wouldn't have to be
>>> null I would have said you should think of using getOwner on Closure
>>>
>> Hmmm, I'm close to that, but not quite the same.  I was trying
>>
>>    ExpressionStatement currCtx = new ExpressionStatement(
>>        new DeclarationExpression (
>>            new VariableExpression( "__currCtx$levelcnt"),
>>            new Token( Types.ASSIGNMENT_OPERATOR, "=", -1, -1),
>>            new MethodCallExpression( new VariableExpression( 'this'),
>> 'getDelegate', new ArgumentListExpression())
>>        )
>>    )
>>
>> Would a DeclarationStatment act differently from an ExpressionStatement
>> with an embedded
>> DeclarationExpression?  If my foggy memory is correct, I wrote my code
>> after looking at the
>> AST displayed for a short test script in groovyConsole.
>>
>> Plus, if foggy memory serves, I tried using getOwner and it came back with
>> the closure being
>> owned by the test script, not the object doing the with.  Calling
>> getDelegate gave me the object.
>> If foggy memory serves.
>>
>> I also tried playing around with setClosureSharedVariable( true) and
>> putReferencedLocalVariable()
>> without success.  But, those were somewhat blind stabs in the dark, so I
>> might not have been
>> using them correctly.
>>
>> Looking at the AST generated by groovyConsole, my transform looks like it
>> is working, but
>> the compiler complains about things not being in scope (in the inner
>> closure).  So, I'm doing
>> something wrong with setting up the scoping.
>>
>>>  bye Jochen
>>
>>
>> Thanks,
>>
>> Ed
>>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: help with an AST

Ed Clark
Hi Paul,

No I don't have that.  Let me play around with it.
Thanks for the pointer.

Ed

On 02/16/2017 04:41 PM, Paul King wrote:

> Do you have something like:
>
> VariableScopeVisitor scopeVisitor = new VariableScopeVisitor(source)
> scopeVisitor.visitClass(cNode)
>
> for each of your closure(s)? Where cNode is a closure's classNode?
>
> Cheers, Paul.
>
>
> On Thu, Feb 16, 2017 at 9:35 PM, Ed Clark <[hidden email]> wrote:
>> Ok, a little more headbanging and I seem to have gotten farther (maybe?).
>> At least the error message has changed.  Now I'm getting:
>>
>> BUG! exception in phase 'class generation' in source unit 'CtxTest2.groovy'
>> Operation: (<assignment operator>:  "=" ) not supported
>>
>> Searching hasn't helped with this one.
>>
>> The xform-ed code looks good -- in fact, if I type the code into a separate
>> groovyConsole, it will run.  (The console AST generation works up thru
>> the Instruction Selection phase, obviously.)
>>
>> I'm guessing it's still with how I'm setting up the scoping (or how I'm not
>> setting up the scoping).
>>
>> Ideas, please?
>>
>> Thanks,
>>
>> Ed
>>
>>
>> On 02/14/2017 12:58 PM, Ed Clark wrote:
>>> On 02/14/2017 12:16 PM, Jochen Theodorou wrote:
>>>>
>>>>
>>>> On 14.02.2017 12:41, Ed Clark wrote:
>>>>> Hi Jochen,
>>>>>
>>>>> Well, I've been slowly making progress on this; kind of feels like
>>>>> bashing my
>>>>> head against a wall at times, but I have made a small hole in the wall
>>>>> through
>>>>> which I can see my goal. ;-)
>>>>
>>>> making holes my lead to brain damage.. then things get really hard to
>>>> solve ;)
>>> Maybe I'm too late, and that's why things aren't coming together ;)
>>>
>>>> [...]
>>>>> Specifically, I haven't figured out how to inject a variable into the
>>>>> scope of an outer
>>>>> with's closure that can be used by code in an interior with's closure.
>>>>> For example,
>>>>> consider
>>>>>
>>>>>    myCtx1.with {
>>>>>        __outerCtx1 = null            // injected statement
>>>>>        __currCtx1 = myCtx1       // injected statement
>>>>>        .... some code ...
>>>>>        myCtx2.with {
>>>>>            __outerCtx2 = __currCtx1   // injected statement <---  doesn't
>>>>> work
>>>>>            __currCtx2 = myCtx2          // injected statement
>>>>
>>>> where is __currCtx1 and __outerCtx1 coming from? Are they supposed to be
>>>> new local variables? in that case you will have to add a
>>>> DeclarationStatement, not just an assignment to a VariableExpression.
>>>> Otherwise the compiler will thnk they are dynamic properties and tries to
>>>> resolve them against the context.
>>>>
>>>> as for the logic behind __outerCtxN... with number wouldn't have to be
>>>> null I would have said you should think of using getOwner on Closure
>>>>
>>> Hmmm, I'm close to that, but not quite the same.  I was trying
>>>
>>>     ExpressionStatement currCtx = new ExpressionStatement(
>>>         new DeclarationExpression (
>>>             new VariableExpression( "__currCtx$levelcnt"),
>>>             new Token( Types.ASSIGNMENT_OPERATOR, "=", -1, -1),
>>>             new MethodCallExpression( new VariableExpression( 'this'),
>>> 'getDelegate', new ArgumentListExpression())
>>>         )
>>>     )
>>>
>>> Would a DeclarationStatment act differently from an ExpressionStatement
>>> with an embedded
>>> DeclarationExpression?  If my foggy memory is correct, I wrote my code
>>> after looking at the
>>> AST displayed for a short test script in groovyConsole.
>>>
>>> Plus, if foggy memory serves, I tried using getOwner and it came back with
>>> the closure being
>>> owned by the test script, not the object doing the with.  Calling
>>> getDelegate gave me the object.
>>> If foggy memory serves.
>>>
>>> I also tried playing around with setClosureSharedVariable( true) and
>>> putReferencedLocalVariable()
>>> without success.  But, those were somewhat blind stabs in the dark, so I
>>> might not have been
>>> using them correctly.
>>>
>>> Looking at the AST generated by groovyConsole, my transform looks like it
>>> is working, but
>>> the compiler complains about things not being in scope (in the inner
>>> closure).  So, I'm doing
>>> something wrong with setting up the scoping.
>>>
>>>>   bye Jochen
>>>
>>> Thanks,
>>>
>>> Ed
>>>

12
Loading...