Quantcast

ConfigSlurper / GroovyClassLoader memory leak

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ConfigSlurper / GroovyClassLoader memory leak

asperkins
The following snippet of code will bring a JVM down by filling up the perm gen heap.

for (int i = 0; i < 10000; i++) {
    new ConfigSlurper().parse("foo = 'bar'");
    println i
}


8078
8079
8080
Caught: java.lang.OutOfMemoryError: PermGen space
        at groovy.util.ConfigSlurper.parse(ConfigSlurper.groovy:123)

A line from jmap -permstat

class_loader classes bytes parent_loader alive? type

0x9527de80 23 96536 0x9527d940 dead  groovy/lang/GroovyClassLoader$InnerLoader@0x91620470

For every iteration there will be an instance like the one above.

Our application using Groovy config files extensively. It also reloads these files at a frequent rate. This problem is causing quite a bit of pain at the moment.

Any ideas?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Jochen Theodorou
asperkins schrieb:
> The following snippet of code will bring a JVM down by filling up the perm
> gen heap.
[...]
> 0x9527de80 23 96536 0x9527d940 dead
> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>
> For every iteration there will be an instance like the one above.
>
> Our application using Groovy config files extensively. It also reloads these
> files at a frequent rate. This problem is causing quite a bit of pain at the
> moment.

the inner loader itself is not bad... it is bad if the outer class
loader still holds a reference to the class in its cache. So that might
be the problem... for example GroovyShell#evaluate uses more or less the
same class loader structure, but there is no such problem...

what version of groovy are you using?

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins

Jochen Theodorou wrote
asperkins schrieb:
> The following snippet of code will bring a JVM down by filling up the perm
> gen heap.
[...]
> 0x9527de80 23 96536 0x9527d940 dead
> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>
> For every iteration there will be an instance like the one above.
>
> Our application using Groovy config files extensively. It also reloads these
> files at a frequent rate. This problem is causing quite a bit of pain at the
> moment.

the inner loader itself is not bad... it is bad if the outer class
loader still holds a reference to the class in its cache. So that might
be the problem... for example GroovyShell#evaluate uses more or less the
same class loader structure, but there is no such problem...

what version of groovy are you using?

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

I've tried 1.5.6, 1.5.7, and 1.6 beta.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Jochen Theodorou
asperkins schrieb:

>
>
> Jochen Theodorou wrote:
>> asperkins schrieb:
>>> The following snippet of code will bring a JVM down by filling up the
>>> perm gen heap.
>> [...]
>>> 0x9527de80 23 96536 0x9527d940 dead
>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>
>>> For every iteration there will be an instance like the one above.
>>>
>>> Our application using Groovy config files extensively. It also reloads
>>> these
>>> files at a frequent rate. This problem is causing quite a bit of pain at
>>> the
>>> moment.
>> the inner loader itself is not bad... it is bad if the outer class
>> loader still holds a reference to the class in its cache. So that might
>> be the problem... for example GroovyShell#evaluate uses more or less the
>> same class loader structure, but there is no such problem...

when looking at the source I see that  there is an ExpandoMetaclass
created for this class... probably one that cannot be collected... in
that case it is no wonder the permgen gets polluted. If that is the
reason, then you could try doing the following... use parsed scripts as
input for ConfigSlurper#parse, after parse set the metaClass in the
registry (GroovySystem.getMetaClassRegistry) to null by using
setMetaClass(Class,MetaClass). After this it should work. Then the class
can be unloaded. Not a nice workaround, but well...

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ExpandoMetaClass Unloading [was Re: ConfigSlurper / GroovyClassLoader memory leak]

Robert Fischer
I what circumstances will an ExpandoMetaClass not get unloaded?

Jochen Theodorou wrote:

> asperkins schrieb:
>>
>>
>> Jochen Theodorou wrote:
>>> asperkins schrieb:
>>>> The following snippet of code will bring a JVM down by filling up the
>>>> perm gen heap.
>>> [...]
>>>> 0x9527de80    23    96536    0x9527d940    dead    
>>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>>
>>>> For every iteration there will be an instance like the one above.
>>>>
>>>> Our application using Groovy config files extensively. It also reloads
>>>> these
>>>> files at a frequent rate. This problem is causing quite a bit of
>>>> pain at
>>>> the
>>>> moment.
>>> the inner loader itself is not bad... it is bad if the outer class
>>> loader still holds a reference to the class in its cache. So that
>>> might be the problem... for example GroovyShell#evaluate uses more or
>>> less the same class loader structure, but there is no such problem...
>
> when looking at the source I see that  there is an ExpandoMetaclass
> created for this class... probably one that cannot be collected... in
> that case it is no wonder the permgen gets polluted. If that is the
> reason, then you could try doing the following... use parsed scripts as
> input for ConfigSlurper#parse, after parse set the metaClass in the
> registry (GroovySystem.getMetaClassRegistry) to null by using
> setMetaClass(Class,MetaClass). After this it should work. Then the class
> can be unloaded. Not a nice workaround, but well...
>
> bye blackdrag
>

--
~~ Robert Fischer.
Smokejumper Consulting  http://smokejumperit.com
Enfranchised Mind Blog  http://enfranchisedmind.com/blog
LinkedIn Profile        http://www.linkedin.com/in/robertfischer
Twitter Feed            http://twitter.com/robertfischer

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins
In reply to this post by Jochen Theodorou

Jochen Theodorou wrote
asperkins schrieb:
>
>
> Jochen Theodorou wrote:
>> asperkins schrieb:
>>> The following snippet of code will bring a JVM down by filling up the
>>> perm gen heap.
>> [...]
>>> 0x9527de80 23 96536 0x9527d940 dead
>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>
>>> For every iteration there will be an instance like the one above.
>>>
>>> Our application using Groovy config files extensively. It also reloads
>>> these
>>> files at a frequent rate. This problem is causing quite a bit of pain at
>>> the
>>> moment.
>> the inner loader itself is not bad... it is bad if the outer class
>> loader still holds a reference to the class in its cache. So that might
>> be the problem... for example GroovyShell#evaluate uses more or less the
>> same class loader structure, but there is no such problem...

when looking at the source I see that  there is an ExpandoMetaclass
created for this class... probably one that cannot be collected... in
that case it is no wonder the permgen gets polluted. If that is the
reason, then you could try doing the following... use parsed scripts as
input for ConfigSlurper#parse, after parse set the metaClass in the
registry (GroovySystem.getMetaClassRegistry) to null by using
setMetaClass(Class,MetaClass). After this it should work. Then the class
can be unloaded. Not a nice workaround, but well...

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Is this what you mean? ( if so, it results in a nullpointer exception )

GroovyClassLoader classLoader = new GroovyClassLoader();
    Script script = (Script) classLoader.parseClass(text).newInstance();
    for (int i = 0; i < count; i++) {
      ConfigObject config = new ConfigSlurper().parse(script);
      GroovySystem.getMetaClassRegistry().setMetaClass(script.getClass(), null);
    }

java.lang.NullPointerException
        at org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put(ConcurrentReaderHashMap.java:488)
        at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.setMetaClass(MetaClassRegistryImpl.java:292)
        at SlurperMemoryLeak.go(SlurperMemoryLeak.java:24)
        at SlurperMemoryLeak.main(SlurperMemoryLeak.java:15)


org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put
   */
  public Object put(Object key, Object value) {
    if (value == null)
      throw new NullPointerException();
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins

asperkins wrote
Jochen Theodorou wrote
asperkins schrieb:
>
>
> Jochen Theodorou wrote:
>> asperkins schrieb:
>>> The following snippet of code will bring a JVM down by filling up the
>>> perm gen heap.
>> [...]
>>> 0x9527de80 23 96536 0x9527d940 dead
>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>
>>> For every iteration there will be an instance like the one above.
>>>
>>> Our application using Groovy config files extensively. It also reloads
>>> these
>>> files at a frequent rate. This problem is causing quite a bit of pain at
>>> the
>>> moment.
>> the inner loader itself is not bad... it is bad if the outer class
>> loader still holds a reference to the class in its cache. So that might
>> be the problem... for example GroovyShell#evaluate uses more or less the
>> same class loader structure, but there is no such problem...

when looking at the source I see that  there is an ExpandoMetaclass
created for this class... probably one that cannot be collected... in
that case it is no wonder the permgen gets polluted. If that is the
reason, then you could try doing the following... use parsed scripts as
input for ConfigSlurper#parse, after parse set the metaClass in the
registry (GroovySystem.getMetaClassRegistry) to null by using
setMetaClass(Class,MetaClass). After this it should work. Then the class
can be unloaded. Not a nice workaround, but well...

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Is this what you mean? ( if so, it results in a nullpointer exception )

GroovyClassLoader classLoader = new GroovyClassLoader();
    Script script = (Script) classLoader.parseClass(text).newInstance();
    for (int i = 0; i < count; i++) {
      ConfigObject config = new ConfigSlurper().parse(script);
      GroovySystem.getMetaClassRegistry().setMetaClass(script.getClass(), null);
    }

java.lang.NullPointerException
        at org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put(ConcurrentReaderHashMap.java:488)
        at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.setMetaClass(MetaClassRegistryImpl.java:292)
        at SlurperMemoryLeak.go(SlurperMemoryLeak.java:24)
        at SlurperMemoryLeak.main(SlurperMemoryLeak.java:15)


org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put
   */
  public Object put(Object key, Object value) {
    if (value == null)
      throw new NullPointerException();
I used the removeMetaClass method instead. Looks like that might be working. I'll keep my fingers crossed.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ExpandoMetaClass Unloading [was Re: ConfigSlurper / GroovyClassLoader memory leak]

Graeme Rocher
In reply to this post by Robert Fischer
When its been modified by the user

Cheers

On Wed, Oct 22, 2008 at 7:43 PM, Robert Fischer
<[hidden email]> wrote:

> I what circumstances will an ExpandoMetaClass not get unloaded?
>
> Jochen Theodorou wrote:
>>
>> asperkins schrieb:
>>>
>>>
>>> Jochen Theodorou wrote:
>>>>
>>>> asperkins schrieb:
>>>>>
>>>>> The following snippet of code will bring a JVM down by filling up the
>>>>> perm gen heap.
>>>>
>>>> [...]
>>>>>
>>>>> 0x9527de80    23    96536    0x9527d940    dead
>>>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>>>
>>>>> For every iteration there will be an instance like the one above.
>>>>>
>>>>> Our application using Groovy config files extensively. It also reloads
>>>>> these
>>>>> files at a frequent rate. This problem is causing quite a bit of pain
>>>>> at
>>>>> the
>>>>> moment.
>>>>
>>>> the inner loader itself is not bad... it is bad if the outer class
>>>> loader still holds a reference to the class in its cache. So that might be
>>>> the problem... for example GroovyShell#evaluate uses more or less the same
>>>> class loader structure, but there is no such problem...
>>
>> when looking at the source I see that  there is an ExpandoMetaclass
>> created for this class... probably one that cannot be collected... in that
>> case it is no wonder the permgen gets polluted. If that is the reason, then
>> you could try doing the following... use parsed scripts as input for
>> ConfigSlurper#parse, after parse set the metaClass in the registry
>> (GroovySystem.getMetaClassRegistry) to null by using
>> setMetaClass(Class,MetaClass). After this it should work. Then the class can
>> be unloaded. Not a nice workaround, but well...
>>
>> bye blackdrag
>>
>
> --
> ~~ Robert Fischer.
> Smokejumper Consulting  http://smokejumperit.com
> Enfranchised Mind Blog  http://enfranchisedmind.com/blog
> LinkedIn Profile        http://www.linkedin.com/in/robertfischer
> Twitter Feed            http://twitter.com/robertfischer
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>   http://xircles.codehaus.org/manage_email
>
>
>



--
Graeme Rocher
Grails Project Lead
G2One, Inc. Chief Technology Officer
http://www.g2one.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Thom Nichols
In reply to this post by Jochen Theodorou
Is this a bug that the class can't be unloaded?  Someone else recently
had the same problem.  If it is in fact a bug, I think it should be
fixed for 1.6

-Tom


On Wed, Oct 22, 2008 at 2:11 PM, Jochen Theodorou <[hidden email]> wrote:

> asperkins schrieb:
>>
>> The following snippet of code will bring a JVM down by filling up the perm
>> gen heap.
>
> [...]
>>
>> 0x9527de80      23      96536   0x9527d940      dead
>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>
>> For every iteration there will be an instance like the one above.
>>
>> Our application using Groovy config files extensively. It also reloads
>> these
>> files at a frequent rate. This problem is causing quite a bit of pain at
>> the
>> moment.
>
> the inner loader itself is not bad... it is bad if the outer class loader
> still holds a reference to the class in its cache. So that might be the
> problem... for example GroovyShell#evaluate uses more or less the same class
> loader structure, but there is no such problem...
>
> what version of groovy are you using?
>
> bye blackdrag
>
> --
> Jochen "blackdrag" Theodorou
> The Groovy Project Tech Lead (http://groovy.codehaus.org)
> http://blackdragsview.blogspot.com/
> http://www.g2one.com/
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>   http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Jochen Theodorou
Tom Nichols schrieb:
> Is this a bug that the class can't be unloaded?  Someone else recently
> had the same problem.  If it is in fact a bug, I think it should be
> fixed for 1.6

that was a different issue. It had nothing to do with constant meta
classes. In case of ConfigSlurper we have a custom meta class and that
meta class is modified, making it into a constant meta class, that can't
be unloaded automatically.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins
Ok, It looks like the removeMetaClass wasn't what fixed my problem.

This leaks...

GroovyClassLoader classLoader  = new GroovyClassLoader(ClassUtils.getDefaultClassLoader());
for (int i = 0; i < 10000; i++) {
    Script script = (Script) classLoader.parseClass("foo = bar").newInstance();
    new ConfigSlurper().parse(script);
}

Change "foo = bar" to a File and NO leak.

GroovyClassLoader classLoader  = new GroovyClassLoader(ClassUtils.getDefaultClassLoader());
for (int i = 0; i < 10000; i++) {
    Script script = (Script) classLoader.parseClass(new File("./someFile.groovy")).newInstance();
    new ConfigSlurper().parse(script);
}

Any idea why this is the case?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

John Prystash
This thread may help provide some more insight into the situation:
http://www.nabble.com/MetaClassRegistryImpl-memory-retainment-and-1.5.7-td19979094.html#a20040179


From: asperkins <[hidden email]>
To: [hidden email]
Sent: Thursday, January 22, 2009 10:57:44 AM
Subject: Re: [groovy-user] ConfigSlurper / GroovyClassLoader memory leak


Ok, It looks like the removeMetaClass wasn't what fixed my problem.

This leaks...

GroovyClassLoader classLoader  = new
GroovyClassLoader(ClassUtils.getDefaultClassLoader());
for (int i = 0; i < 10000; i++) {
    Script script = (Script) classLoader.parseClass("foo =
bar").newInstance();
    new ConfigSlurper().parse(script);
}

Change "foo = bar" to a File and NO leak.

GroovyClassLoader classLoader  = new
GroovyClassLoader(ClassUtils.getDefaultClassLoader());
for (int i = 0; i < 10000; i++) {
    Script script = (Script) classLoader.parseClass(new
File("./someFile.groovy")).newInstance();
    new ConfigSlurper().parse(script);
}

Any idea why this is the case?
--
View this message in context: http://www.nabble.com/ConfigSlurper---GroovyClassLoader-memory-leak-tp20115910p21607195.html
Sent from the groovy - user mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins
In reply to this post by asperkins
I noticed in 1.6  rc2 that the following code still runs out of perm gen. We use the config slurper extensively and this is a rather bothersome issue. I've had to resort to writing the string out to a file, then loading it via the class loader, but even then I have to reuse filenames or the classes still pile up and I run out of perm gen.

Any ideas on what to do if I have an application that is retrieving groovy config objects via a rest call and parsing them?

   for (int i = 0; i < 50000; i++) {
      ConfigObject config = new ConfigSlurper().parse("foo = bar");
      print "  ${i}"
    }

BTW, The removal of the metaclass didn't seem to help this issue.
This didn't work...
GroovyClassLoader classLoader = new GroovyClassLoader();
for (int i = 0; i < 50000; i++) {
Script script = (Script) classLoader.parseClass("foo = bar").newInstance();
  ConfigObject config = new ConfigSlurper().parse(script);
  print "  ${i}"
  GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() );
}
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins
I think I found a work-around.
If I call clearCache on the classLoader everytime, it seems to work. The perm gen gets cleaned up when it hits the max.

GroovyClassLoader classLoader = new GroovyClassLoader();
for (int i = 0; i < 50000; i++) {
Script script = (Script) classLoader.parseClass("foo = bar").newInstance();
  ConfigObject config = new ConfigSlurper().parse(script);
  print "  ${i}"
  GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() );
  classLoader.clearCache();
}

asperkins wrote
I noticed in 1.6  rc2 that the following code still runs out of perm gen. We use the config slurper extensively and this is a rather bothersome issue. I've had to resort to writing the string out to a file, then loading it via the class loader, but even then I have to reuse filenames or the classes still pile up and I run out of perm gen.

Any ideas on what to do if I have an application that is retrieving groovy config objects via a rest call and parsing them?

   for (int i = 0; i < 50000; i++) {
      ConfigObject config = new ConfigSlurper().parse("foo = bar");
      print "  ${i}"
    }

BTW, The removal of the metaclass didn't seem to help this issue.
This didn't work...
GroovyClassLoader classLoader = new GroovyClassLoader();
for (int i = 0; i < 50000; i++) {
Script script = (Script) classLoader.parseClass("foo = bar").newInstance();
  ConfigObject config = new ConfigSlurper().parse(script);
  print "  ${i}"
  GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() );
}
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Jochen Theodorou
In reply to this post by asperkins
asperkins schrieb:

> I noticed in 1.6  rc2 that the following code still runs out of perm gen. We
> use the config slurper extensively and this is a rather bothersome issue.
> I've had to resort to writing the string out to a file, then loading it via
> the class loader, but even then I have to reuse filenames or the classes
> still pile up and I run out of perm gen.
>
> Any ideas on what to do if I have an application that is retrieving groovy
> config objects via a rest call and parsing them?
>
>    for (int i = 0; i < 50000; i++) {
>       ConfigObject config = new ConfigSlurper().parse("foo = bar");
>       print "  ${i}"
>     }
>
> BTW, The removal of the metaclass didn't seem to help this issue.
> This didn't work...
> GroovyClassLoader classLoader = new GroovyClassLoader();
> for (int i = 0; i < 50000; i++) {
> Script script = (Script) classLoader.parseClass("foo = bar").newInstance();
>   ConfigObject config = new ConfigSlurper().parse(script);
>   print "  ${i}"
>   GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() );
> }

Running out of permgen means that the
classes created through classLoader.parseClass("foo = bar") are not
collected. But a class cannot be collected if the class loader is still
alive.

Since you do not discard the class loader in your loop, there is no
chance the loader gets collected. Put the classLoader in the loop and
all should be fine

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins
You are correct!

When I tested that I didn't allow the app to run long enough to see the perm gen clean up in jprofiler.

Thanks!

Tony

Jochen Theodorou wrote
asperkins schrieb:
> I noticed in 1.6  rc2 that the following code still runs out of perm gen. We
> use the config slurper extensively and this is a rather bothersome issue.
> I've had to resort to writing the string out to a file, then loading it via
> the class loader, but even then I have to reuse filenames or the classes
> still pile up and I run out of perm gen.
>
> Any ideas on what to do if I have an application that is retrieving groovy
> config objects via a rest call and parsing them?
>
>    for (int i = 0; i < 50000; i++) {
>       ConfigObject config = new ConfigSlurper().parse("foo = bar");
>       print "  ${i}"
>     }
>
> BTW, The removal of the metaclass didn't seem to help this issue.
> This didn't work...
> GroovyClassLoader classLoader = new GroovyClassLoader();
> for (int i = 0; i < 50000; i++) {
> Script script = (Script) classLoader.parseClass("foo = bar").newInstance();
>   ConfigObject config = new ConfigSlurper().parse(script);
>   print "  ${i}"
>   GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() );
> }

Running out of permgen means that the
classes created through classLoader.parseClass("foo = bar") are not
collected. But a class cannot be collected if the class loader is still
alive.

Since you do not discard the class loader in your loop, there is no
chance the loader gets collected. Put the classLoader in the loop and
all should be fine

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

filirom1
I have exactly the same issue:

I use a ConfigSlurper in order to send emails in a Java Application.

After few hundred emails, I got a PermGen Exception.

Here is the Java code I had before :
ConfigSlurper slurper = new ConfigSlurper();
slurper.setBinding(variables);
ConfigObject conf = slurper.parse(template);

And now, the updated code without permgen, thanks to this mailing list:
GroovyClassLoader classLoader = new GroovyClassLoader();
Script script = (Script) classLoader.parseClass(template).newInstance();
ConfigSlurper slurper = new ConfigSlurper();
slurper.setBinding(variable);
ConfigObject conf = slurper.parse(script);
GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass());
classLoader.clearCache();

Is it possible to write a caution in the ConfigSlurper documentation
http://groovy.codehaus.org/gapi/groovy/util/ConfigSlurper.html ?

It could save some times to other developers, that use the ConfigSlurper inside Java code.



Cheers

Romain
Loading...