|
The following snippet of code will bring a JVM down by filling up the perm gen heap.
for (int i = 0; i < 10000; i++) { new ConfigSlurper().parse("foo = 'bar'"); println i } 8078 8079 8080 Caught: java.lang.OutOfMemoryError: PermGen space at groovy.util.ConfigSlurper.parse(ConfigSlurper.groovy:123) A line from jmap -permstat class_loader classes bytes parent_loader alive? type 0x9527de80 23 96536 0x9527d940 dead groovy/lang/GroovyClassLoader$InnerLoader@0x91620470 For every iteration there will be an instance like the one above. Our application using Groovy config files extensively. It also reloads these files at a frequent rate. This problem is causing quite a bit of pain at the moment. Any ideas? |
|
asperkins schrieb:
> The following snippet of code will bring a JVM down by filling up the perm > gen heap. [...] > 0x9527de80 23 96536 0x9527d940 dead > groovy/lang/GroovyClassLoader$InnerLoader@0x91620470 > > For every iteration there will be an instance like the one above. > > Our application using Groovy config files extensively. It also reloads these > files at a frequent rate. This problem is causing quite a bit of pain at the > moment. the inner loader itself is not bad... it is bad if the outer class loader still holds a reference to the class in its cache. So that might be the problem... for example GroovyShell#evaluate uses more or less the same class loader structure, but there is no such problem... what version of groovy are you using? bye blackdrag -- Jochen "blackdrag" Theodorou The Groovy Project Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ http://www.g2one.com/ --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
I've tried 1.5.6, 1.5.7, and 1.6 beta. |
|
asperkins schrieb:
> > > Jochen Theodorou wrote: >> asperkins schrieb: >>> The following snippet of code will bring a JVM down by filling up the >>> perm gen heap. >> [...] >>> 0x9527de80 23 96536 0x9527d940 dead >>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470 >>> >>> For every iteration there will be an instance like the one above. >>> >>> Our application using Groovy config files extensively. It also reloads >>> these >>> files at a frequent rate. This problem is causing quite a bit of pain at >>> the >>> moment. >> the inner loader itself is not bad... it is bad if the outer class >> loader still holds a reference to the class in its cache. So that might >> be the problem... for example GroovyShell#evaluate uses more or less the >> same class loader structure, but there is no such problem... when looking at the source I see that there is an ExpandoMetaclass created for this class... probably one that cannot be collected... in that case it is no wonder the permgen gets polluted. If that is the reason, then you could try doing the following... use parsed scripts as input for ConfigSlurper#parse, after parse set the metaClass in the registry (GroovySystem.getMetaClassRegistry) to null by using setMetaClass(Class,MetaClass). After this it should work. Then the class can be unloaded. Not a nice workaround, but well... bye blackdrag -- Jochen "blackdrag" Theodorou The Groovy Project Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ http://www.g2one.com/ --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
I what circumstances will an ExpandoMetaClass not get unloaded?
Jochen Theodorou wrote: > asperkins schrieb: >> >> >> Jochen Theodorou wrote: >>> asperkins schrieb: >>>> The following snippet of code will bring a JVM down by filling up the >>>> perm gen heap. >>> [...] >>>> 0x9527de80 23 96536 0x9527d940 dead >>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470 >>>> >>>> For every iteration there will be an instance like the one above. >>>> >>>> Our application using Groovy config files extensively. It also reloads >>>> these >>>> files at a frequent rate. This problem is causing quite a bit of >>>> pain at >>>> the >>>> moment. >>> the inner loader itself is not bad... it is bad if the outer class >>> loader still holds a reference to the class in its cache. So that >>> might be the problem... for example GroovyShell#evaluate uses more or >>> less the same class loader structure, but there is no such problem... > > when looking at the source I see that there is an ExpandoMetaclass > created for this class... probably one that cannot be collected... in > that case it is no wonder the permgen gets polluted. If that is the > reason, then you could try doing the following... use parsed scripts as > input for ConfigSlurper#parse, after parse set the metaClass in the > registry (GroovySystem.getMetaClassRegistry) to null by using > setMetaClass(Class,MetaClass). After this it should work. Then the class > can be unloaded. Not a nice workaround, but well... > > bye blackdrag > -- ~~ Robert Fischer. Smokejumper Consulting http://smokejumperit.com Enfranchised Mind Blog http://enfranchisedmind.com/blog LinkedIn Profile http://www.linkedin.com/in/robertfischer Twitter Feed http://twitter.com/robertfischer --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
In reply to this post by Jochen Theodorou
Is this what you mean? ( if so, it results in a nullpointer exception ) GroovyClassLoader classLoader = new GroovyClassLoader(); Script script = (Script) classLoader.parseClass(text).newInstance(); for (int i = 0; i < count; i++) { ConfigObject config = new ConfigSlurper().parse(script); GroovySystem.getMetaClassRegistry().setMetaClass(script.getClass(), null); } java.lang.NullPointerException at org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put(ConcurrentReaderHashMap.java:488) at org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.setMetaClass(MetaClassRegistryImpl.java:292) at SlurperMemoryLeak.go(SlurperMemoryLeak.java:24) at SlurperMemoryLeak.main(SlurperMemoryLeak.java:15) org.codehaus.groovy.runtime.metaclass.ConcurrentReaderHashMap.put */ public Object put(Object key, Object value) { if (value == null) throw new NullPointerException(); |
I used the removeMetaClass method instead. Looks like that might be working. I'll keep my fingers crossed. |
|
In reply to this post by Robert Fischer
When its been modified by the user
Cheers On Wed, Oct 22, 2008 at 7:43 PM, Robert Fischer <[hidden email]> wrote: > I what circumstances will an ExpandoMetaClass not get unloaded? > > Jochen Theodorou wrote: >> >> asperkins schrieb: >>> >>> >>> Jochen Theodorou wrote: >>>> >>>> asperkins schrieb: >>>>> >>>>> The following snippet of code will bring a JVM down by filling up the >>>>> perm gen heap. >>>> >>>> [...] >>>>> >>>>> 0x9527de80 23 96536 0x9527d940 dead >>>>> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470 >>>>> >>>>> For every iteration there will be an instance like the one above. >>>>> >>>>> Our application using Groovy config files extensively. It also reloads >>>>> these >>>>> files at a frequent rate. This problem is causing quite a bit of pain >>>>> at >>>>> the >>>>> moment. >>>> >>>> the inner loader itself is not bad... it is bad if the outer class >>>> loader still holds a reference to the class in its cache. So that might be >>>> the problem... for example GroovyShell#evaluate uses more or less the same >>>> class loader structure, but there is no such problem... >> >> when looking at the source I see that there is an ExpandoMetaclass >> created for this class... probably one that cannot be collected... in that >> case it is no wonder the permgen gets polluted. If that is the reason, then >> you could try doing the following... use parsed scripts as input for >> ConfigSlurper#parse, after parse set the metaClass in the registry >> (GroovySystem.getMetaClassRegistry) to null by using >> setMetaClass(Class,MetaClass). After this it should work. Then the class can >> be unloaded. Not a nice workaround, but well... >> >> bye blackdrag >> > > -- > ~~ Robert Fischer. > Smokejumper Consulting http://smokejumperit.com > Enfranchised Mind Blog http://enfranchisedmind.com/blog > LinkedIn Profile http://www.linkedin.com/in/robertfischer > Twitter Feed http://twitter.com/robertfischer > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > -- Graeme Rocher Grails Project Lead G2One, Inc. Chief Technology Officer http://www.g2one.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
In reply to this post by Jochen Theodorou
Is this a bug that the class can't be unloaded? Someone else recently
had the same problem. If it is in fact a bug, I think it should be fixed for 1.6 -Tom On Wed, Oct 22, 2008 at 2:11 PM, Jochen Theodorou <[hidden email]> wrote: > asperkins schrieb: >> >> The following snippet of code will bring a JVM down by filling up the perm >> gen heap. > > [...] >> >> 0x9527de80 23 96536 0x9527d940 dead >> groovy/lang/GroovyClassLoader$InnerLoader@0x91620470 >> >> For every iteration there will be an instance like the one above. >> >> Our application using Groovy config files extensively. It also reloads >> these >> files at a frequent rate. This problem is causing quite a bit of pain at >> the >> moment. > > the inner loader itself is not bad... it is bad if the outer class loader > still holds a reference to the class in its cache. So that might be the > problem... for example GroovyShell#evaluate uses more or less the same class > loader structure, but there is no such problem... > > what version of groovy are you using? > > bye blackdrag > > -- > Jochen "blackdrag" Theodorou > The Groovy Project Tech Lead (http://groovy.codehaus.org) > http://blackdragsview.blogspot.com/ > http://www.g2one.com/ > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
Tom Nichols schrieb:
> Is this a bug that the class can't be unloaded? Someone else recently > had the same problem. If it is in fact a bug, I think it should be > fixed for 1.6 that was a different issue. It had nothing to do with constant meta classes. In case of ConfigSlurper we have a custom meta class and that meta class is modified, making it into a constant meta class, that can't be unloaded automatically. bye blackdrag -- Jochen "blackdrag" Theodorou The Groovy Project Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ http://www.g2one.com/ --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
Ok, It looks like the removeMetaClass wasn't what fixed my problem.
This leaks... GroovyClassLoader classLoader = new GroovyClassLoader(ClassUtils.getDefaultClassLoader()); for (int i = 0; i < 10000; i++) { Script script = (Script) classLoader.parseClass("foo = bar").newInstance(); new ConfigSlurper().parse(script); } Change "foo = bar" to a File and NO leak. GroovyClassLoader classLoader = new GroovyClassLoader(ClassUtils.getDefaultClassLoader()); for (int i = 0; i < 10000; i++) { Script script = (Script) classLoader.parseClass(new File("./someFile.groovy")).newInstance(); new ConfigSlurper().parse(script); } Any idea why this is the case? |
|
This thread may help provide some more insight into the situation: http://www.nabble.com/MetaClassRegistryImpl-memory-retainment-and-1.5.7-td19979094.html#a20040179 From: asperkins <[hidden email]> To: [hidden email] Sent: Thursday, January 22, 2009 10:57:44 AM Subject: Re: [groovy-user] ConfigSlurper / GroovyClassLoader memory leak Ok, It looks like the removeMetaClass wasn't what fixed my problem. This leaks... GroovyClassLoader classLoader = new GroovyClassLoader(ClassUtils.getDefaultClassLoader()); for (int i = 0; i < 10000; i++) { Script script = (Script) classLoader.parseClass("foo = bar").newInstance(); new ConfigSlurper().parse(script); } Change "foo = bar" to a File and NO leak. GroovyClassLoader classLoader = new GroovyClassLoader(ClassUtils.getDefaultClassLoader()); for (int i = 0; i < 10000; i++) { Script script = (Script) classLoader.parseClass(new File("./someFile.groovy")).newInstance(); new ConfigSlurper().parse(script); } Any idea why this is the case? -- View this message in context: http://www.nabble.com/ConfigSlurper---GroovyClassLoader-memory-leak-tp20115910p21607195.html Sent from the groovy - user mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
In reply to this post by asperkins
I noticed in 1.6 rc2 that the following code still runs out of perm gen. We use the config slurper extensively and this is a rather bothersome issue. I've had to resort to writing the string out to a file, then loading it via the class loader, but even then I have to reuse filenames or the classes still pile up and I run out of perm gen.
Any ideas on what to do if I have an application that is retrieving groovy config objects via a rest call and parsing them? for (int i = 0; i < 50000; i++) { ConfigObject config = new ConfigSlurper().parse("foo = bar"); print " ${i}" } BTW, The removal of the metaclass didn't seem to help this issue. This didn't work... GroovyClassLoader classLoader = new GroovyClassLoader(); for (int i = 0; i < 50000; i++) { Script script = (Script) classLoader.parseClass("foo = bar").newInstance(); ConfigObject config = new ConfigSlurper().parse(script); print " ${i}" GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() ); } |
|
I think I found a work-around.
If I call clearCache on the classLoader everytime, it seems to work. The perm gen gets cleaned up when it hits the max. GroovyClassLoader classLoader = new GroovyClassLoader(); for (int i = 0; i < 50000; i++) { Script script = (Script) classLoader.parseClass("foo = bar").newInstance(); ConfigObject config = new ConfigSlurper().parse(script); print " ${i}" GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() ); classLoader.clearCache(); }
|
|
In reply to this post by asperkins
asperkins schrieb:
> I noticed in 1.6 rc2 that the following code still runs out of perm gen. We > use the config slurper extensively and this is a rather bothersome issue. > I've had to resort to writing the string out to a file, then loading it via > the class loader, but even then I have to reuse filenames or the classes > still pile up and I run out of perm gen. > > Any ideas on what to do if I have an application that is retrieving groovy > config objects via a rest call and parsing them? > > for (int i = 0; i < 50000; i++) { > ConfigObject config = new ConfigSlurper().parse("foo = bar"); > print " ${i}" > } > > BTW, The removal of the metaclass didn't seem to help this issue. > This didn't work... > GroovyClassLoader classLoader = new GroovyClassLoader(); > for (int i = 0; i < 50000; i++) { > Script script = (Script) classLoader.parseClass("foo = bar").newInstance(); > ConfigObject config = new ConfigSlurper().parse(script); > print " ${i}" > GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass() ); > } Running out of permgen means that the classes created through classLoader.parseClass("foo = bar") are not collected. But a class cannot be collected if the class loader is still alive. Since you do not discard the class loader in your loop, there is no chance the loader gets collected. Put the classLoader in the loop and all should be fine bye blackdrag -- Jochen "blackdrag" Theodorou The Groovy Project Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
You are correct!
When I tested that I didn't allow the app to run long enough to see the perm gen clean up in jprofiler. Thanks! Tony
|
|
I have exactly the same issue:
I use a ConfigSlurper in order to send emails in a Java Application. After few hundred emails, I got a PermGen Exception. Here is the Java code I had before : ConfigSlurper slurper = new ConfigSlurper(); slurper.setBinding(variables); ConfigObject conf = slurper.parse(template); And now, the updated code without permgen, thanks to this mailing list: GroovyClassLoader classLoader = new GroovyClassLoader(); Script script = (Script) classLoader.parseClass(template).newInstance(); ConfigSlurper slurper = new ConfigSlurper(); slurper.setBinding(variable); ConfigObject conf = slurper.parse(script); GroovySystem.getMetaClassRegistry().removeMetaClass(script.getClass()); classLoader.clearCache(); Is it possible to write a caution in the ConfigSlurper documentation http://groovy.codehaus.org/gapi/groovy/util/ConfigSlurper.html ? It could save some times to other developers, that use the ConfigSlurper inside Java code. ![]() Cheers Romain |
| Powered by Nabble | Edit this page |
