Quantcast

ConfigSlurper / GroovyClassLoader memory leak

classic Classic list List threaded Threaded
17 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ConfigSlurper / GroovyClassLoader memory leak

asperkins
The following snippet of code will bring a JVM down by filling up the perm gen heap.

for (int i = 0; i < 10000; i++) {
    new ConfigSlurper().parse("foo = 'bar'");
    println i
}


8078
8079
8080
Caught: java.lang.OutOfMemoryError: PermGen space
        at groovy.util.ConfigSlurper.parse(ConfigSlurper.groovy:123)

A line from jmap -permstat

class_loader classes bytes parent_loader alive? type

0x9527de80 23 96536 0x9527d940 dead  groovy/lang/GroovyClassLoader$InnerLoader@0x91620470

For every iteration there will be an instance like the one above.

Our application using Groovy config files extensively. It also reloads these files at a frequent rate. This problem is causing quite a bit of pain at the moment.

Any ideas?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Jochen Theodorou
asperkins schrieb:
> The following snippet of code will bring a JVM down by filling up the perm
> gen heap.
[...]
> 0x9527de80 23 96536 0x9527d940 dead
> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>
> For every iteration there will be an instance like the one above.
>
> Our application using Groovy config files extensively. It also reloads these
> files at a frequent rate. This problem is causing quite a bit of pain at the
> moment.

the inner loader itself is not bad... it is bad if the outer class
loader still holds a reference to the class in its cache. So that might
be the problem... for example GroovyShell#evaluate uses more or less the
same class loader structure, but there is no such problem...

what version of groovy are you using?

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins

Jochen Theodorou wrote
asperkins schrieb:
> The following snippet of code will bring a JVM down by filling up the perm
> gen heap.
[...]
> 0x9527de80 23 96536 0x9527d940 dead
> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>
> For every iteration there will be an instance like the one above.
>
> Our application using Groovy config files extensively. It also reloads these
> files at a frequent rate. This problem is causing quite a bit of pain at the
> moment.

the inner loader itself is not bad... it is bad if the outer class
loader still holds a reference to the class in its cache. So that might
be the problem... for example GroovyShell#evaluate uses more or less the
same class loader structure, but there is no such problem...

what version of groovy are you using?

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

I've tried 1.5.6, 1.5.7, and 1.6 beta.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Jochen Theodorou
asperkins schrieb:

>
>
> Jochen Theodorou wrote:
>> asperkins schrieb:
>>> The following snippet of code will bring a JVM down by filling up the
>>> perm gen heap.
>> [...]
>>> 0x9527de80 23 96536 0x9527d940 dead
>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>
>>> For every iteration there will be an instance like the one above.
>>>
>>> Our application using Groovy config files extensively. It also reloads
>>> these
>>> files at a frequent rate. This problem is causing quite a bit of pain at
>>> the
>>> moment.
>> the inner loader itself is not bad... it is bad if the outer class
>> loader still holds a reference to the class in its cache. So that might
>> be the problem... for example GroovyShell#evaluate uses more or less the
>> same class loader structure, but there is no such problem...

when looking at the source I see that  there is an ExpandoMetaclass
created for this class... probably one that cannot be collected... in
that case it is no wonder the permgen gets polluted. If that is the
reason, then you could try doing the following... use parsed scripts as
input for ConfigSlurper#parse, after parse set the metaClass in the
registry (GroovySystem.getMetaClassRegistry) to null by using
setMetaClass(Class,MetaClass). After this it should work. Then the class
can be unloaded. Not a nice workaround, but well...

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

ExpandoMetaClass Unloading [was Re: ConfigSlurper / GroovyClassLoader memory leak]

Robert Fischer
I what circumstances will an ExpandoMetaClass not get unloaded?

Jochen Theodorou wrote:

> asperkins schrieb:
>>
>>
>> Jochen Theodorou wrote:
>>> asperkins schrieb:
>>>> The following snippet of code will bring a JVM down by filling up the
>>>> perm gen heap.
>>> [...]
>>>> 0x9527de80    23    96536    0x9527d940    dead    
>>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>>
>>>> For every iteration there will be an instance like the one above.
>>>>
>>>> Our application using Groovy config files extensively. It also reloads
>>>> these
>>>> files at a frequent rate. This problem is causing quite a bit of
>>>> pain at
>>>> the
>>>> moment.
>>> the inner loader itself is not bad... it is bad if the outer class
>>> loader still holds a reference to the class in its cache. So that
>>> might be the problem... for example GroovyShell#evaluate uses more or
>>> less the same class loader structure, but there is no such problem...
>
> when looking at the source I see that  there is an ExpandoMetaclass
> created for this class... probably one that cannot be collected... in
> that case it is no wonder the permgen gets polluted. If that is the
> reason, then you could try doing the following... use parsed scripts as
> input for ConfigSlurper#parse, after parse set the metaClass in the
> registry (GroovySystem.getMetaClassRegistry) to null by using
> setMetaClass(Class,MetaClass). After this it should work. Then the class
> can be unloaded. Not a nice workaround, but well...
>
> bye blackdrag
>

--
~~ Robert Fischer.
Smokejumper Consulting  http://smokejumperit.com
Enfranchised Mind Blog  http://enfranchisedmind.com/blog
LinkedIn Profile        http://www.linkedin.com/in/robertfischer
Twitter Feed            http://twitter.com/robertfischer

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins
In reply to this post by Jochen Theodorou

Jochen Theodorou wrote
asperkins schrieb:
>
>
> Jochen Theodorou wrote:
>> asperkins schrieb:
>>> The following snippet of code will bring a JVM down by filling up the
>>> perm gen heap.
>> [...]
>>> 0x9527de80 23 96536 0x9527d940 dead
>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>
>>> For every iteration there will be an instance like the one above.
>>>
>>> Our application using Groovy config files extensively. It also reloads
>>> these
>>> files at a frequent rate. This problem is causing quite a bit of pain at
>>> the
>>> moment.
>> the inner loader itself is not bad... it is bad if the outer class
>> loader still holds a reference to the class in its cache. So that might
>> be the problem... for example GroovyShell#evaluate uses more or less the
>> same class loader structure, but there is no such problem...

when looking at the source I see that  there is an ExpandoMetaclass
created for this class... probably one that cannot be collected... in
that case it is no wonder the permgen gets polluted. If that is the
reason, then you could try doing the following... use parsed scripts as
input for ConfigSlurper#parse, after parse set the metaClass in the
registry (GroovySystem.getMetaClassRegistry) to null by using
setMetaClass(Class,MetaClass). After this it should work. Then the class
can be unloaded. Not a nice workaround, but well...

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Is this what you mean? ( if so, it results in a nullpointer exception )

GroovyClassLoader classLoader = new GroovyClassLoader();
    Script script = (Script) classLoader.parseClass(text).newInstance();
    for (int i = 0; i < count; i++) {
      ConfigObject config = new ConfigSlurper().parse(script);
      GroovySystem.getMetaClassRegistry().setMetaClass(script.getClass(), null);
    }

java.lang.NullPointerException
        at org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put(ConcurrentReaderHashMap.java:488)
        at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.setMetaClass(MetaClassRegistryImpl.java:292)
        at SlurperMemoryLeak.go(SlurperMemoryLeak.java:24)
        at SlurperMemoryLeak.main(SlurperMemoryLeak.java:15)


org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put
   */
  public Object put(Object key, Object value) {
    if (value == null)
      throw new NullPointerException();
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

asperkins

asperkins wrote
Jochen Theodorou wrote
asperkins schrieb:
>
>
> Jochen Theodorou wrote:
>> asperkins schrieb:
>>> The following snippet of code will bring a JVM down by filling up the
>>> perm gen heap.
>> [...]
>>> 0x9527de80 23 96536 0x9527d940 dead
>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>
>>> For every iteration there will be an instance like the one above.
>>>
>>> Our application using Groovy config files extensively. It also reloads
>>> these
>>> files at a frequent rate. This problem is causing quite a bit of pain at
>>> the
>>> moment.
>> the inner loader itself is not bad... it is bad if the outer class
>> loader still holds a reference to the class in its cache. So that might
>> be the problem... for example GroovyShell#evaluate uses more or less the
>> same class loader structure, but there is no such problem...

when looking at the source I see that  there is an ExpandoMetaclass
created for this class... probably one that cannot be collected... in
that case it is no wonder the permgen gets polluted. If that is the
reason, then you could try doing the following... use parsed scripts as
input for ConfigSlurper#parse, after parse set the metaClass in the
registry (GroovySystem.getMetaClassRegistry) to null by using
setMetaClass(Class,MetaClass). After this it should work. Then the class
can be unloaded. Not a nice workaround, but well...

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Is this what you mean? ( if so, it results in a nullpointer exception )

GroovyClassLoader classLoader = new GroovyClassLoader();
    Script script = (Script) classLoader.parseClass(text).newInstance();
    for (int i = 0; i < count; i++) {
      ConfigObject config = new ConfigSlurper().parse(script);
      GroovySystem.getMetaClassRegistry().setMetaClass(script.getClass(), null);
    }

java.lang.NullPointerException
        at org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put(ConcurrentReaderHashMap.java:488)
        at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.setMetaClass(MetaClassRegistryImpl.java:292)
        at SlurperMemoryLeak.go(SlurperMemoryLeak.java:24)
        at SlurperMemoryLeak.main(SlurperMemoryLeak.java:15)


org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put
   */
  public Object put(Object key, Object value) {
    if (value == null)
      throw new NullPointerException();
I used the removeMetaClass method instead. Looks like that might be working. I'll keep my fingers crossed.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ExpandoMetaClass Unloading [was Re: ConfigSlurper / GroovyClassLoader memory leak]

Graeme Rocher
In reply to this post by Robert Fischer
When its been modified by the user

Cheers

On Wed, Oct 22, 2008 at 7:43 PM, Robert Fischer
<[hidden email]> wrote:

> I what circumstances will an ExpandoMetaClass not get unloaded?
>
> Jochen Theodorou wrote:
>>
>> asperkins schrieb:
>>>
>>>
>>> Jochen Theodorou wrote:
>>>>
>>>> asperkins schrieb:
>>>>>
>>>>> The following snippet of code will bring a JVM down by filling up the
>>>>> perm gen heap.
>>>>
>>>> [...]
>>>>>
>>>>> 0x9527de80    23    96536    0x9527d940    dead
>>>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>>>>
>>>>> For every iteration there will be an instance like the one above.
>>>>>
>>>>> Our application using Groovy config files extensively. It also reloads
>>>>> these
>>>>> files at a frequent rate. This problem is causing quite a bit of pain
>>>>> at
>>>>> the
>>>>> moment.
>>>>
>>>> the inner loader itself is not bad... it is bad if the outer class
>>>> loader still holds a reference to the class in its cache. So that might be
>>>> the problem... for example GroovyShell#evaluate uses more or less the same
>>>> class loader structure, but there is no such problem...
>>
>> when looking at the source I see that  there is an ExpandoMetaclass
>> created for this class... probably one that cannot be collected... in that
>> case it is no wonder the permgen gets polluted. If that is the reason, then
>> you could try doing the following... use parsed scripts as input for
>> ConfigSlurper#parse, after parse set the metaClass in the registry
>> (GroovySystem.getMetaClassRegistry) to null by using
>> setMetaClass(Class,MetaClass). After this it should work. Then the class can
>> be unloaded. Not a nice workaround, but well...
>>
>> bye blackdrag
>>
>
> --
> ~~ Robert Fischer.
> Smokejumper Consulting  http://smokejumperit.com
> Enfranchised Mind Blog  http://enfranchisedmind.com/blog
> LinkedIn Profile        http://www.linkedin.com/in/robertfischer
> Twitter Feed            http://twitter.com/robertfischer
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>   http://xircles.codehaus.org/manage_email
>
>
>



--
Graeme Rocher
Grails Project Lead
G2One, Inc. Chief Technology Officer
http://www.g2one.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Thom Nichols
In reply to this post by Jochen Theodorou
Is this a bug that the class can't be unloaded?  Someone else recently
had the same problem.  If it is in fact a bug, I think it should be
fixed for 1.6

-Tom


On Wed, Oct 22, 2008 at 2:11 PM, Jochen Theodorou <[hidden email]> wrote:

> asperkins schrieb:
>>
>> The following snippet of code will bring a JVM down by filling up the perm
>> gen heap.
>
> [...]
>>
>> 0x9527de80      23      96536   0x9527d940      dead
>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470
>>
>> For every iteration there will be an instance like the one above.
>>
>> Our application using Groovy config files extensively. It also reloads
>> these
>> files at a frequent rate. This problem is causing quite a bit of pain at
>> the
>> moment.
>
> the inner loader itself is not bad... it is bad if the outer class loader
> still holds a reference to the class in its cache. So that might be the
> problem... for example GroovyShell#evaluate uses more or less the same class
> loader structure, but there is no such problem...
>
> what version of groovy are you using?
>
> bye blackdrag
>
> --
> Jochen "blackdrag" Theodorou
> The Groovy Project Tech Lead (http://groovy.codehaus.org)
> http://blackdragsview.blogspot.com/
> http://www.g2one.com/
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>   http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: ConfigSlurper / GroovyClassLoader memory leak

Jochen Theodorou
Tom Nichols schrieb:
> Is this a bug that the class can't be unloaded?  Someone else recently
> had the same problem.  If it is in fact a bug, I think it should be
> fixed for 1.6

that was a different issue. It had nothing to do with constant meta
classes. In case of ConfigSlurper we have a custom meta class and that
meta class is modified, making it into a constant meta class, that can't
be unloaded automatically.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


12
Loading...