Groovy versus JRuby on Computer Language Benchmarks Game?

classic Classic list List threaded Threaded
33 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Groovy versus JRuby on Computer Language Benchmarks Game?

John Wells-2
Guys,

I've followed both the Groovy (and Grails) communities for awhile now, and one thing I've heard from time to time is that Groovy is faster (and Grails, consequently) than JRuby/Rails. However, in a comparison of Groovy and JRuby here:

http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=jruby&lang2=groovy

JRuby seems to beat Groovy hands down.

Am I misunderstanding something, or is JRuby truly this much faster?

Thanks,
John

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

paulk_asert

Well, Groovy no doubt has many areas where it can improve and no doubt
there are some areas where JRuby is faster than Groovy but I am dubious
of the times in that shootout.

I just ran the binary trees demo.

Their time for Groovy: 844.99 s
My time locally (using HEAD): 4641 ms

Now, I didn't count JVM startup, but differing by two orders
of magnitude sounds like they don't have something set up properly.

Paul.


John Wells wrote:

> Guys,
>
> I've followed both the Groovy (and Grails) communities for awhile now, and one thing I've heard from time to time is that Groovy is faster (and Grails, consequently) than JRuby/Rails. However, in a comparison of Groovy and JRuby here:
>
> http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=jruby&lang2=groovy
>
> JRuby seems to beat Groovy hands down.
>
> Am I misunderstanding something, or is JRuby truly this much faster?
>
> Thanks,
> John


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

paulk_asert

Oops, I'll take that back. They don't use the default number of iterations
in their published results - though the JRuby program doesn't even check
for defaults. When I use 16 iterations, I now got 420s which is under half
what they publish but now the same order of magnitude. The JRuby supposed
"equivalent" uses lists rather than classes to store the binary tree, so
I am still skeptical that you can call them equivalents.

Paul.

Paul King wrote:

>
> Well, Groovy no doubt has many areas where it can improve and no doubt
> there are some areas where JRuby is faster than Groovy but I am dubious
> of the times in that shootout.
>
> I just ran the binary trees demo.
>
> Their time for Groovy: 844.99 s
> My time locally (using HEAD): 4641 ms
> Now, I didn't count JVM startup, but differing by two orders
> of magnitude sounds like they don't have something set up properly.
>
> Paul.
>
>
> John Wells wrote:
>> Guys,
>>
>> I've followed both the Groovy (and Grails) communities for awhile now,
>> and one thing I've heard from time to time is that Groovy is faster
>> (and Grails, consequently) than JRuby/Rails. However, in a comparison
>> of Groovy and JRuby here:
>>
>> http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=jruby&lang2=groovy 
>>
>>
>> JRuby seems to beat Groovy hands down.
>> Am I misunderstanding something, or is JRuby truly this much faster?
>>
>> Thanks,
>> John

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

paulk_asert
Paul King wrote:
>
> Oops, I'll take that back. They don't use the default number of iterations
> in their published results - though the JRuby program doesn't even check
> for defaults. When I use 16 iterations, I now got 420s which is under half
> what they publish but now the same order of magnitude. The JRuby supposed
> "equivalent" uses lists rather than classes to store the binary tree, so
> I am still skeptical that you can call them equivalents.
>
> Paul.

Running with lists rather than classes and the time is under 300s.
That still makes it 50% slower than the JRuby published result
(haven't run it on my system yet) but definitely not 4.7 times slower.
Both were much slower than the native Ruby version. Of course, with
Groovy, if I was trying to do something computationally intensive
I would write the bottleneck part in Java. This benchmark doesn't seem
to take that into account though.

Paul.

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

Guillaume Laforge
Administrator
In reply to this post by John Wells-2
You may also have a look at this benchmark:
http://grails.org/Grails+vs+Rails+Benchmark

On 8/4/07, John Wells <[hidden email]> wrote:

> Guys,
>
> I've followed both the Groovy (and Grails) communities for awhile now, and one thing I've heard from time to time is that Groovy is faster (and Grails, consequently) than JRuby/Rails. However, in a comparison of Groovy and JRuby here:
>
> http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=jruby&lang2=groovy
>
> JRuby seems to beat Groovy hands down.
>
> Am I misunderstanding something, or is JRuby truly this much faster?
>
> Thanks,
> John
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>


--
Guillaume Laforge
Groovy Project Manager
http://glaforge.free.fr/blog/groovy

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

Alex Popescu
In reply to this post by paulk_asert
On 8/4/07, Paul King <[hidden email]> wrote:

>
> Well, Groovy no doubt has many areas where it can improve and no doubt
> there are some areas where JRuby is faster than Groovy but I am dubious
> of the times in that shootout.
>
> I just ran the binary trees demo.
>
> Their time for Groovy: 844.99 s
> My time locally (using HEAD): 4641 ms
>
> Now, I didn't count JVM startup, but differing by two orders
> of magnitude sounds like they don't have something set up properly.
>
> Paul.
>

Paul have you found a way to download the source files of those benchmarks?

tia,

./alex
--
.w( the_mindstorm )p.

>
> John Wells wrote:
> > Guys,
> >
> > I've followed both the Groovy (and Grails) communities for awhile now, and one thing I've heard from time to time is that Groovy is faster (and Grails, consequently) than JRuby/Rails. However, in a comparison of Groovy and JRuby here:
> >
> > http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=jruby&lang2=groovy
> >
> > JRuby seems to beat Groovy hands down.
> >
> > Am I misunderstanding something, or is JRuby truly this much faster?
> >
> > Thanks,
> > John
>

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

Alex Popescu
On 8/4/07, Alexandru Popescu ☀ <[hidden email]> wrote:

> On 8/4/07, Paul King <[hidden email]> wrote:
> >
> > Well, Groovy no doubt has many areas where it can improve and no doubt
> > there are some areas where JRuby is faster than Groovy but I am dubious
> > of the times in that shootout.
> >
> > I just ran the binary trees demo.
> >
> > Their time for Groovy: 844.99 s
> > My time locally (using HEAD): 4641 ms
> >
> > Now, I didn't count JVM startup, but differing by two orders
> > of magnitude sounds like they don't have something set up properly.
> >
> > Paul.
> >
>
> Paul have you found a way to download the source files of those benchmarks?
>

Finally found it: http://alioth.debian.org/scm/?group_id=30402

./alex
--
.w( the_mindstorm )p.

> tia,
>
> ./alex
> --
> .w( the_mindstorm )p.
>
> >
> > John Wells wrote:
> > > Guys,
> > >
> > > I've followed both the Groovy (and Grails) communities for awhile now, and one thing I've heard from time to time is that Groovy is faster (and Grails, consequently) than JRuby/Rails. However, in a comparison of Groovy and JRuby here:
> > >
> > > http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=jruby&lang2=groovy
> > >
> > > JRuby seems to beat Groovy hands down.
> > >
> > > Am I misunderstanding something, or is JRuby truly this much faster?
> > >
> > > Thanks,
> > > John
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

John Wells-2
In reply to this post by Alex Popescu

----- "Alexandru Popescu ☀" <[hidden email]> wrote:
> Paul have you found a way to download the source files of those
> benchmarks?

If you plan to repeat the comparison (JRuby and Groovy), would you mind posting the results? It'd be interesting to see if the benchmark is flawed by a lack of understanding of language features...

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

paulk_asert
John Wells wrote:
> ----- "Alexandru Popescu ☀" <[hidden email]> wrote:
>> Paul have you found a way to download the source files of those
>> benchmarks?
>
> If you plan to repeat the comparison (JRuby and Groovy), would you mind posting the results? It'd be interesting to see if the benchmark is flawed by a lack of understanding of language features...

The Groovy program below takes 6 mins 20 secs to run.
The equivalent Java program takes 2 sec.
So I guess we still have some work to do.

I didn't take the Groovy program the shootout used, just did the
minimal changes from the Java program - so it isn't Groovified
but should be fairly close to its Java equivalent. I am sure
you could hack it to make it much faster under Groovy but some of
those hacks might violate the benchmark conditions (included
below also). In any case, it might be worthwhile as an example
for us to profile.

Benchmark Notes:
Each program should

    * define a tree node class and methods, a tree node record and procedures, or an algebraic data type and functions, or…
    * allocate a binary tree to 'stretch' memory, check it exists, and deallocate it
    * allocate a long-lived binary tree which will live-on while other trees are allocated and deallocated
    * allocate, walk, and deallocate many bottom-up binary trees
          o allocate a tree
          o walk the tree nodes, checksum node items (and maybe deallocate the node)
          o deallocate the tree
    * check that the long-lived binary tree still exists

Note: this is an adaptation of a benchmark for testing GC so we are interested in the whole tree being allocated before any nodes are GC'd - which probably excludes lazy evaluation.

Note: the left subtrees are heads of the right subtrees, keeping a depth counter in the accessors to avoid duplication is cheating!

Note: the tree should have tree-nodes all the way down, replacing the bottom nodes by some other value is not acceptable; and the bottom nodes should be at depth 0.

There are reference implementations in OCaml, C#, and PHP.

Paul.

------->8---------- Groovy ------------>8----------
long begin = System.currentTimeMillis();
int minDepth = 4
int n = 16
if (args.length > 0) n = Integer.parseInt(args[0])

int maxDepth = (minDepth + 2 > n) ? minDepth + 2 : n
int stretchDepth = maxDepth + 1

int check = (TreeNode.bottomUpTree(0, stretchDepth)).itemCheck()
println "stretch tree of depth " + stretchDepth + "\t check: " + check

TreeNode longLivedTree = TreeNode.bottomUpTree(0, maxDepth)

for (int depth = minDepth; depth <= maxDepth; depth += 2) {
    int iterations = 1 << (maxDepth - depth + minDepth)
    check = 0

    for (int i = 1; i <= iterations; i++) {
        check += (TreeNode.bottomUpTree(i, depth)).itemCheck()
        check += (TreeNode.bottomUpTree(-i, depth)).itemCheck()
    }
    println((iterations * 2) + "\t trees of depth " + depth + "\t check: " + check)
}
println "long lived tree of depth " + maxDepth + "\t check: " + longLivedTree.itemCheck()
println ((System.currentTimeMillis() - begin) + " ms")

class TreeNode {
    private TreeNode left, right
    private int item

    TreeNode(int item) {
        this.item = item
    }

    static TreeNode bottomUpTree(int item, int depth) {
        if (depth > 0)
            return new TreeNode(
                    bottomUpTree(2 * item - 1, depth - 1),
                    bottomUpTree(2 * item, depth - 1), item
                    )
        else return new TreeNode(item)
    }

    TreeNode(TreeNode left, TreeNode right, int item) {
        this.left = left
        this.right = right
        this.item = item
    }

    int itemCheck() {
        // if necessary deallocate here
        if (!left) return item
        else return item + left.itemCheck() - right.itemCheck()
    }
}
------->8---------- Groovy ------------>8----------

------->8----------  Java  ------------>8----------
public class BinTreeMain {

        private final static int minDepth = 4;

        public static void main(String[] args){
        long begin = System.currentTimeMillis();
        int n = 16;
                if (args.length > 0) n = Integer.parseInt(args[0]);

                int maxDepth = (minDepth + 2 > n) ? minDepth + 2 : n;
                int stretchDepth = maxDepth + 1;

                int check = (TreeNode.bottomUpTree(0,stretchDepth)).itemCheck();
                System.out.println("stretch tree of depth "+stretchDepth+"\t check: " + check);

                TreeNode longLivedTree = TreeNode.bottomUpTree(0,maxDepth);

                for (int depth=minDepth; depth<=maxDepth; depth+=2){
                        int iterations = 1 << (maxDepth - depth + minDepth);
                        check = 0;

                        for (int i=1; i<=iterations; i++){
                                check += (TreeNode.bottomUpTree(i,depth)).itemCheck();
                                check += (TreeNode.bottomUpTree(-i,depth)).itemCheck();
                        }
                        System.out.println((iterations*2) + "\t trees of depth " + depth + "\t check: " + check);
                }
                System.out.println("long lived tree of depth " + maxDepth + "\t check: "+ longLivedTree.itemCheck());
        System.out.println((System.currentTimeMillis() - begin) + " ms");
    }


        private static class TreeNode
        {
                private TreeNode left, right;
                private int item;

                TreeNode(int item){
                        this.item = item;
                }

                private static TreeNode bottomUpTree(int item, int depth){
                        if (depth>0){
                                return new TreeNode(
                                                bottomUpTree(2*item-1, depth-1)
                                                , bottomUpTree(2*item, depth-1)
                                                , item
                                );
                        }
                        else {
                                return new TreeNode(item);
                        }
                }

                TreeNode(TreeNode left, TreeNode right, int item){
                        this.left = left;
                        this.right = right;
                        this.item = item;
                }

                private int itemCheck(){
                        // if necessary deallocate here
                        if (left==null) return item;
                        else return item + left.itemCheck() - right.itemCheck();
                }
        }
}
------->8----------  Java  ------------>8----------

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Groovy versus JRuby on Computer Language Benchmarks Game?

Marc Palmer Local

On 4 Aug 2007, at 14:53, Paul King wrote:

> John Wells wrote:
>> ----- "Alexandru Popescu ☀" <[hidden email]>  
>> wrote:
>>> Paul have you found a way to download the source files of those
>>> benchmarks?
>> If you plan to repeat the comparison (JRuby and Groovy), would you  
>> mind posting the results? It'd be interesting to see if the  
>> benchmark is flawed by a lack of understanding of language  
>> features...
>
> The Groovy program below takes 6 mins 20 secs to run.
> The equivalent Java program takes 2 sec.
> So I guess we still have some work to do.
>

did you run with args -server -Xmx512M ?

Marc


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

1234