[jira] [Comment Edited] (GROOVY-7407) Compilation not thread safe if Grape / Ivy is used in Groovy scripts

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[jira] [Comment Edited] (GROOVY-7407) Compilation not thread safe if Grape / Ivy is used in Groovy scripts

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/GROOVY-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925660#comment-15925660 ]

Jex Jexler edited comment on GROOVY-7407 at 3/15/17 7:22 AM:
-------------------------------------------------------------

I have looked at what GroovyCompileConcurrencyTest calls from GrapeEngine/GrapeIvy. It is always only grab(args, dependencies), with arguments always like this:
{noformat}
grab, args [disableChecksums:false, classLoader:groovy.lang.GroovyClassLoader@a7474bc, autoDownload:true], dependencies [module:guava, version:18.0, group:com.google.guava]
{noformat}
So, I wrote a test that only does such grabs in parallel (and does not use the compiler) and then tries to load the Guava class com.google.common.base.Ascii from the GroovyClassLoader, see the newly attached GrabConcurrencyTest.java. Initialization of args and dependencies in the new test:
{code:java}
final Map<String,Object> args = new HashMap<String,Object>();
final Map<String,Object> dependencies = new HashMap<String,Object>();

args.put("disableChecksums", "false");
//args.put("classLoader", new GroovyClassLoader());
args.put("autoDownload", "true");

dependencies.put("group", "com.google.guava");
dependencies.put("module", "guava");
dependencies.put("version", "1" + (j % 10) + ".0");
{code}
And what each thread does in the new test:
{code:java}
GroovyClassLoader loader = new GroovyClassLoader();
args.put("classLoader", loader);
Object obj = Grape.getInstance().grab(args, dependencies);
if (obj != null) {
    String msg = "grab returned: " + obj;
    throw new RuntimeException(msg);
}
loader.loadClass("com.google.common.base.Ascii");
{code}
This fails reproducibly in the following way: No grabs ever fail and they always return null, but loading the class eventually fails with a stacktrace that I would interpret so far as that the grab simply and silently did not add a File URL for Guava to the GroovyClassLoader:
{noformat}
java.lang.ClassNotFoundException: com.google.common.base.Ascii
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
        at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
        at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
        at GrabConcurrencyTest$1.run(GrabConcurrencyTest.java:56)
        at java.lang.Thread.run(Thread.java:745)
{noformat}
*Edited to add:* Note that you may not be able to reproduce this exactly, unless you also fix the Ivy MessageLoggerHelper concurrency issue mentioned earlier, at least this would also be a possible way for this new test to fail.


was (Author: jexler):
I have looked at what GroovyCompileConcurrencyTest calls from GrapeEngine/GrapeIvy. It is always only grab(args, dependencies), with arguments always like this:
{noformat}
grab, args [disableChecksums:false, classLoader:groovy.lang.GroovyClassLoader@a7474bc, autoDownload:true], dependencies [module:guava, version:18.0, group:com.google.guava]
{noformat}
So, I wrote a test that only does such grabs in parallel (and does not use the compiler) and then tries to load the Guava class com.google.common.base.Ascii from the GroovyClassLoader, see the newly attached GrabConcurrencyTest.java. Initialization of args and dependencies in the new test:
{code:java}
final Map<String,Object> args = new HashMap<String,Object>();
final Map<String,Object> dependencies = new HashMap<String,Object>();

args.put("disableChecksums", "false");
//args.put("classLoader", new GroovyClassLoader());
args.put("autoDownload", "true");

dependencies.put("group", "com.google.guava");
dependencies.put("module", "guava");
dependencies.put("version", "1" + (j % 10) + ".0");
{code}
And what each thread does in the new test:
{code:java}
GroovyClassLoader loader = new GroovyClassLoader();
args.put("classLoader", loader);
Object obj = Grape.getInstance().grab(args, dependencies);
if (obj != null) {
    String msg = "grab returned: " + obj;
    throw new RuntimeException(msg);
}
loader.loadClass("com.google.common.base.Ascii");
{code}
This fails reproducibly in the following way: No grabs ever fail and they always return null, but loading the class eventually fails with a stacktrace that I would interpret so far as that the grab simply and silently did not add a File URL for Guava to the GroovyClassLoader:
{noformat}
java.lang.ClassNotFoundException: com.google.common.base.Ascii
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
        at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
        at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
        at GrabConcurrencyTest$1.run(GrabConcurrencyTest.java:56)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

> Compilation not thread safe if Grape / Ivy is used in Groovy scripts
> --------------------------------------------------------------------
>
>                 Key: GROOVY-7407
>                 URL: https://issues.apache.org/jira/browse/GROOVY-7407
>             Project: Groovy
>          Issue Type: Bug
>          Components: Compiler, Grape
>    Affects Versions: 2.4.3
>         Environment: Essentially independent of the environment, as long as Groovy scripts use Grape; also this bug seems to be present since at least Groovy 1.7.5.
>            Reporter: Jex Jexler
>            Priority: Minor
>              Labels: Compile, Grape, Groovy, Ivy
>         Attachments: GrabConcurrencyTest.java, GrapeAndGroovyShellConcurrencyTest.java, GroovyCompileConcurrencyTest.java, stacktrace-GrapeAndGroovyShellConcurrencyTest-1.txt, stacktrace-GrapeAndGroovyShellConcurrencyTest-2.txt, stacktrace-GroovyCompileConcurrencyTest-1.txt, stacktrace-GroovyCompileConcurrencyTest-2.txt, WorkaroundGroovy7407WrappingGrapeEngine.java
>
>
> If Groovy scripts that import the same libraries via Grape are compiled in separate threads, compilation may fail due to race conditions.
> This does not happen if several threads use the *same* instance of GroovyClassLoader (GCL), because parseClass() uses synchronization.
> But as soon as different GCLs are used in separate threads or if the compiler is used directly (CompilationUnit.compile()), the issue occurs and compilation can fail.
> Two Java unit tests have be attached, which reproduce the issue, although this cannot be guaranteed with 100% certainty, because there is a race condition.
> Two different stacktraces have been observed for each unit test (with origins in Grape and in Ivy), which have also been attached (plus in a different environment (Tomcat webapp CentOS) once a an exception down in Ivy had been observed that seemed to be related to unzipping a JAR file, but no precise record of that exists any more).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Loading...