Built-in JSON support in 1.8

classic Classic list List threaded Threaded
16 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Built-in JSON support in 1.8

Guillaume Laforge
Administrator
Hi all,

As part of the Groovy 1.8 roadmap, we had an item for supporting JSON in Groovy, similarily as to how we support XML, namely by providing a parser (slurper) and a builder.
I've implemented a Groovy JsonSlurper and JsonParser, in the groovy.json package.
It's currently in Trunk.

You can play with those parser and builder by using a snapshot of Groovy 1.8 that you can find on Bamboo, for example:
http://bamboo.ci.codehaus.org/browse/GROOVY-CORE-5493/artifact/deliverables

You can have a look at the JSON GEP that Andres and I started a few months ago (GEP-7):
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support
It shows some of the examples.
Be sure to look at the JavaDocs which show various examples as well.

Although JSON and XML are pretty different, it would be nice if xml builders and json builder would be even closer (which is one of the goals of the built-in JSON support), as for example the following cases are not covered by the current implementation:
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support#GEP7-JSONSupport-Openquestions
I'd be interested in your input as how you'd expect such calls on the builder should serialize the JSON output, or even if we should even support them at all.

Thanks for your attention.

--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one

Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Alex Tkachman
Hi Guillaume!

Looks very intersting. One question - How does performance (both
parser & builder) look like compare with industry standards?

Alex

On Fri, Jan 28, 2011 at 4:26 PM, Guillaume Laforge <[hidden email]> wrote:

> Hi all,
>
> As part of the Groovy 1.8 roadmap, we had an item for supporting JSON in
> Groovy, similarily as to how we support XML, namely by providing a parser
> (slurper) and a builder.
> I've implemented a Groovy JsonSlurper and JsonParser, in the groovy.json
> package.
> It's currently in Trunk.
>
> You can play with those parser and builder by using a snapshot of Groovy 1.8
> that you can find on Bamboo, for example:
> http://bamboo.ci.codehaus.org/browse/GROOVY-CORE-5493/artifact/deliverables
>
> You can have a look at the JSON GEP that Andres and I started a few months
> ago (GEP-7):
> http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support
> It shows some of the examples.
> But you can have a better overview of the various possibilities by looking
> at the test cases here:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/
> In particular the builder tests:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/JsonBuilderTest.groovy
> And the parser tests:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/JsonSlurperTest.groovy
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/RealJsonPayloadsTest.groovy
>
> And you can have a look at the sources too:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/main/groovy/json/
> Be sure to look at the JavaDocs which show various examples as well.
>
> Although JSON and XML are pretty different, it would be nice if xml builders
> and json builder would be even closer (which is one of the goals of the
> built-in JSON support), as for example the following cases are not covered
> by the current implementation:
> http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support#GEP7-JSONSupport-Openquestions
> I'd be interested in your input as how you'd expect such calls on the
> builder should serialize the JSON output, or even if we should even support
> them at all.
>
> Thanks for your attention.
> --
> Guillaume Laforge
> Groovy Project Manager
> Head of Groovy Development at SpringSource
> http://www.springsource.com/g2one
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Guillaume Laforge-4
Hi Alex,

I've not benchmarked it yet.
My gut tells me the builder should be okay (perhaps a few improvements with some serialization were rewritten in Java), but the slurper may be the one I should have a closer look at, wrt how I implemented the lexing. I think there's room for improvements there.

Guillaume

On Fri, Jan 28, 2011 at 17:40, Alex Tkachman <[hidden email]> wrote:
Hi Guillaume!

Looks very intersting. One question - How does performance (both
parser & builder) look like compare with industry standards?

Alex

On Fri, Jan 28, 2011 at 4:26 PM, Guillaume Laforge <[hidden email]> wrote:
> Hi all,
>
> As part of the Groovy 1.8 roadmap, we had an item for supporting JSON in
> Groovy, similarily as to how we support XML, namely by providing a parser
> (slurper) and a builder.
> I've implemented a Groovy JsonSlurper and JsonParser, in the groovy.json
> package.
> It's currently in Trunk.
>
> You can play with those parser and builder by using a snapshot of Groovy 1.8
> that you can find on Bamboo, for example:
> http://bamboo.ci.codehaus.org/browse/GROOVY-CORE-5493/artifact/deliverables
>
> You can have a look at the JSON GEP that Andres and I started a few months
> ago (GEP-7):
> http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support
> It shows some of the examples.
> But you can have a better overview of the various possibilities by looking
> at the test cases here:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/
> In particular the builder tests:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/JsonBuilderTest.groovy
> And the parser tests:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/JsonSlurperTest.groovy
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/test/groovy/json/RealJsonPayloadsTest.groovy
>
> And you can have a look at the sources too:
> http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/main/groovy/json/
> Be sure to look at the JavaDocs which show various examples as well.
>
> Although JSON and XML are pretty different, it would be nice if xml builders
> and json builder would be even closer (which is one of the goals of the
> built-in JSON support), as for example the following cases are not covered
> by the current implementation:
> http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support#GEP7-JSONSupport-Openquestions
> I'd be interested in your input as how you'd expect such calls on the
> builder should serialize the JSON output, or even if we should even support
> them at all.
>
> Thanks for your attention.
> --
> Guillaume Laforge
> Groovy Project Manager
> Head of Groovy Development at SpringSource
> http://www.springsource.com/g2one
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email





--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one
Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

melix
In reply to this post by Guillaume Laforge
Hi Guillaume,

Great work. Having native support for JSON is definitely something good. Grails has support for JSON for ages, and Andres has written a great JSON-lib which integrates with Groovy rather naturally. Is this support supposed to be a replacement for those or a complement ? It's still unclear to me.

Le 28/01/2011 17:26, Guillaume Laforge a écrit :
Hi all,

As part of the Groovy 1.8 roadmap, we had an item for supporting JSON in Groovy, similarily as to how we support XML, namely by providing a parser (slurper) and a builder.
I've implemented a Groovy JsonSlurper and JsonParser, in the groovy.json package.
It's currently in Trunk.

You can play with those parser and builder by using a snapshot of Groovy 1.8 that you can find on Bamboo, for example:
http://bamboo.ci.codehaus.org/browse/GROOVY-CORE-5493/artifact/deliverables

You can have a look at the JSON GEP that Andres and I started a few months ago (GEP-7):
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support
It shows some of the examples.
Be sure to look at the JavaDocs which show various examples as well.

Although JSON and XML are pretty different, it would be nice if xml builders and json builder would be even closer (which is one of the goals of the built-in JSON support), as for example the following cases are not covered by the current implementation:
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support#GEP7-JSONSupport-Openquestions
I'd be interested in your input as how you'd expect such calls on the builder should serialize the JSON output, or even if we should even support them at all.

Thanks for your attention.

--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one


Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Guillaume Laforge-4
Hi Cédric,

Some of the feedback I had along the years (also from a couple customers of mine) was that were interested in built-in json support, and also ones that would be closer to what the syntax is for our existing xml builders -- although we can't really have them totally compatible because of the differences between JSON and XML. 
Both JSON-lib and Grails have some bigger differences in syntax compared to our XML builders, so we tried to have something a bit closer.
But it's not meant as a replacement for either JSON-lib or Grails' support at all.

Guillaume

On Fri, Jan 28, 2011 at 17:49, Cédric CHAMPEAU <[hidden email]> wrote:
Hi Guillaume,

Great work. Having native support for JSON is definitely something good. Grails has support for JSON for ages, and Andres has written a great JSON-lib which integrates with Groovy rather naturally. Is this support supposed to be a replacement for those or a complement ? It's still unclear to me.

Le 28/01/2011 17:26, Guillaume Laforge a écrit :
Hi all,

As part of the Groovy 1.8 roadmap, we had an item for supporting JSON in Groovy, similarily as to how we support XML, namely by providing a parser (slurper) and a builder.
I've implemented a Groovy JsonSlurper and JsonParser, in the groovy.json package.
It's currently in Trunk.

You can play with those parser and builder by using a snapshot of Groovy 1.8 that you can find on Bamboo, for example:
http://bamboo.ci.codehaus.org/browse/GROOVY-CORE-5493/artifact/deliverables

You can have a look at the JSON GEP that Andres and I started a few months ago (GEP-7):
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support
It shows some of the examples.
Be sure to look at the JavaDocs which show various examples as well.

Although JSON and XML are pretty different, it would be nice if xml builders and json builder would be even closer (which is one of the goals of the built-in JSON support), as for example the following cases are not covered by the current implementation:
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support#GEP7-JSONSupport-Openquestions
I'd be interested in your input as how you'd expect such calls on the builder should serialize the JSON output, or even if we should even support them at all.

Thanks for your attention.

--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one





--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one
Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Merlyn Albery-Speyer
In reply to this post by Guillaume Laforge
Very nice!
Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Thom Nichols
In reply to this post by Guillaume Laforge
Couple questions:  Given the proposal, how would I make a JSON object that looks like this?

{ name: 'bob', job: 'fish' }

Point being, your examples all use a Javascript map with a single 'root' key, then the 'object' is really the value.  e.g. { person: { name: 'bob' } }.  AFAIK the JSON spec doesn't require this, yet the builder does since it follows the XMLBuilder model.  XML requires a root element, JSON doesn't -- right?

So the interface should look more like:

builder.build {
  person {
    name "bob"
    job "fish"
  }
}.toString()

// produces:
{person:{name:'bob',job:'fish'}}

builder.build {
  name 'bob'
  job 'fish'
}

// produces
{ name:'bob', job:'fish' }


Also, I'd like a way to incrementally build a javascript array in a builder closure, and I think using dynamic properties (rather than methods) would make this more straightforward, e.g.

def personList = getListOfPople() // returns list of people objects

builder.build { builder ->
  people = []    // this will be a Javascript array
  personList.each { p ->
    // people.add builder.build( name: p.fullName, job: p.job.title )
    // or just:
    people.add [name: p.fullName, job: p.job.title] // map object gets converted automatically
  }
}.toString()

// produces:
{ people: [ {name:'bob',job:'fish'}, {name:'jim',job:'worm'} ] }

I want to say there's a way one could do the same with just method calls, but the best I can think of is something like this:
builder.build {
    people {
        def personList = getListOfPeople()
        personList.each { p ->
          name p.fullName
          job p.job.title
        }
    }
}

However, in the 'people' closure, how do you indicate that the value of 'people' is a JSON array, not a JSON map?  We'd need a special builder method or something.

I think I like properties better because using the current proposal's combination of varargs, named args and closures seems to result in 'too many ways to do it.'  e.g. you can do this (under the current proposal):

builder.person( name:'bob' ) {
  name 'bob'
}

Finally, it's not mentioned, but one should assume the builder should be able to automatically convert a POJO based on their properties as well.  But this should already be supported by most JSON libraries I suppose.  So whatever underlying JSON library is used to implement the groovy builder, support for auto-converting POJOs should be trivial.

-Thom


On Fri, Jan 28, 2011 at 11:26 AM, Guillaume Laforge <[hidden email]> wrote:
Hi all,

As part of the Groovy 1.8 roadmap, we had an item for supporting JSON in Groovy, similarily as to how we support XML, namely by providing a parser (slurper) and a builder.
I've implemented a Groovy JsonSlurper and JsonParser, in the groovy.json package.
It's currently in Trunk.

You can play with those parser and builder by using a snapshot of Groovy 1.8 that you can find on Bamboo, for example:
http://bamboo.ci.codehaus.org/browse/GROOVY-CORE-5493/artifact/deliverables

You can have a look at the JSON GEP that Andres and I started a few months ago (GEP-7):
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support
It shows some of the examples.
Be sure to look at the JavaDocs which show various examples as well.

Although JSON and XML are pretty different, it would be nice if xml builders and json builder would be even closer (which is one of the goals of the built-in JSON support), as for example the following cases are not covered by the current implementation:
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support#GEP7-JSONSupport-Openquestions
I'd be interested in your input as how you'd expect such calls on the builder should serialize the JSON output, or even if we should even support them at all.

Thanks for your attention.

--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one


Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Jochen Theodorou
Am 28.01.2011 20:20, schrieb Thom Nichols:
> Couple questions:  Given the proposal, how would I make a JSON object
> that looks like this?
>
> { name: 'bob', job: 'fish' }

if I know it right, then that would be

builder name:'bob', job:'fish'

[...]
> So whatever underlying JSON library is used to implement the groovy
> builder, support for auto-converting POJOs should be trivial.

there is no underlaying library ;)

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead
http://blackdragsview.blogspot.com/
For Groovy programming sources visit http://groovy.codehaus.org


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Thom Nichols
On Fri, Jan 28, 2011 at 2:53 PM, Jochen Theodorou <[hidden email]> wrote:
if I know it right, then that would be

builder name:'bob', job:'fish'

Ah, ok I see that now in the proposal.  This 'call()' method is fairly uncommon, isn't it?  I don't even know if I was aware of a 'call' method/ operator in Groovy.  I think if this is the first usage of the idiom in the Groovy API, it would be sort of odd.  Why not just stick with a 'build' method like I proposed?  It seems to just further my case of there being 'too many ways to do it' in the JSON builder API.
 
My advice: don't try too hard to make the JSON builder look the same as to the XML builder because the structure of XML and JSON are fundamentally different. 


there is no underlaying library ;)

Really?  Groovy didn't implement its own SAX libraries to provide XML support...   There's no existing JSON library that exists under a compatible license?  Even if you just took the necessary classes and renamed them, I wouldn't expect you to re-write the JSON parsing and serialization from scratch.  Jackson appears to be Apache licensed.  Why not take the parsing and encoding bits from that project?
Reply | Threaded
Open this post in threaded view
|

Re: Built-in JSON support in 1.8

Guillaume Laforge-4
In reply to this post by Thom Nichols
Hi,

Jochen has already answered, but let me add some more details.

On Fri, Jan 28, 2011 at 20:20, Thom Nichols <[hidden email]> wrote:
Couple questions:  Given the proposal, how would I make a JSON object that looks like this?

{ name: 'bob', job: 'fish' }

The idea is to do:

builder {
    name "bob"
    job "fish"
}

But you can also use named arguments:

builder name: "bob", job: "fish"
 
Point being, your examples all use a Javascript map with a single 'root' key, then the 'object' is really the value.  e.g. { person: { name: 'bob' } }.  AFAIK the JSON spec doesn't require this, yet the builder does since it follows the XMLBuilder model.  XML requires a root element, JSON doesn't -- right?

Agreed.
And as the examples above show (and you can see some examples in the javadoc too, for arrays as top level elements too), you can support those JSON objects without a root key.

So the interface should look more like:

builder.build {
  person {
    name "bob"
    job "fish"
  }
}.toString()

// produces:
{person:{name:'bob',job:'fish'}}

builder.build {
  name 'bob'
  job 'fish'
}

// produces
{ name:'bob', job:'fish' }

Well, what bothered me with this approach is that with a builder like MarkupBuilder, this same syntax would create:

<build>
    <name>bob</name>
    <job>fish</job>
</build>

So I wanted to avoid that.
 
Also, I'd like a way to incrementally build a javascript array in a builder closure, and I think using dynamic properties (rather than methods) would make this more straightforward, e.g.

But using properties, it's not really comformant with our "classical" builders anymore (an approach taken by both Grails and JSON-lib).
 
def personList = getListOfPople() // returns list of people objects

builder.build { builder ->
  people = []    // this will be a Javascript array
  personList.each { p ->
    // people.add builder.build( name: p.fullName, job: p.job.title )
    // or just:
    people.add [name: p.fullName, job: p.job.title] // map object gets converted automatically
  }
}.toString()

// produces:
{ people: [ {name:'bob',job:'fish'}, {name:'jim',job:'worm'} ] }

Yeah, actually, since the underlying data structure is just a list of maps or map of lists of maps, etc. (you get the picture), you can very well ammend the data structure afterwards, and add new things (elements, key/value pairs, etc).
I'm too lazy tonight to show an example though.
 
I want to say there's a way one could do the same with just method calls, but the best I can think of is something like this:
builder.build {
    people {
        def personList = getListOfPeople()
        personList.each { p ->
          name p.fullName
          job p.job.title
        }
    }
}

This is indeed a standard builder approach, where you mix groovy code, and that's supported too.
 
However, in the 'people' closure, how do you indicate that the value of 'people' is a JSON array, not a JSON map?  We'd need a special builder method or something.

This is an area where it's obviously not compatible with the xml builders, but you can do something like this:

people elem1, elem2, elem3 // ie. with varargs

Or you can also do

people( [elem1, elem2, elem3] )
 
I think I like properties better because using the current proposal's combination of varargs, named args and closures seems to result in 'too many ways to do it.'  e.g. you can do this (under the current proposal):

builder.person( name:'bob' ) {
  name 'bob'
}

You currently can't, and that's part of my "open questions", as to what we should do with:

builder.person()
builder.person(name: "bob") { age 33 }
builder.person(name: "bob", someValue)

All three notations are supported by "standard" builders, but not implemented with the JSON builder.
I'd be interested in know what you think about these notations and what they should yield.
 
Finally, it's not mentioned, but one should assume the builder should be able to automatically convert a POJO based on their properties as well. 

Just as our XML support doesn't provide a marshalling / unmarshalling framework, the JSON support doesn't.
Full blown solutions like these are really projects on their own, and I feel it's a bit beyond the scope of Groovy.

That said, we can easily get a map of POGO properties and pass that to the builder methods, and you have that serialization easily done, and you could do something similar in the other direction potentially.

At one point in my implementation, I also had the concept of a toJson() method: if in the data structure you create through the builder you have an object (which is not a supported JSON base value), if that object provides a toJson() method, the builder will use that method to serialize that object.
But discussing this with Jochen, we thought it was probably going a bit too far (already in the direction of a full-blown (un)marshalling framework), and if we'd provide a serialization method, we'd need to provide a deserialization one, and this is even trickier (especially without weird naming and structure conventions).
 
But this should already be supported by most JSON libraries I suppose.  So whatever underlying JSON library is used to implement the groovy builder, support for auto-converting POJOs should be trivial.

It's a custom built-in solution, so not relying on any external framework.
All the existing JSON frameworks bring a lot of additional dependencies, and we want to avoid adding too many dependencies to avoid too much JAR versioning hell, etc.

Thanks for your feedback!

Guillaume

 
-Thom



On Fri, Jan 28, 2011 at 11:26 AM, Guillaume Laforge <[hidden email]> wrote:
Hi all,

As part of the Groovy 1.8 roadmap, we had an item for supporting JSON in Groovy, similarily as to how we support XML, namely by providing a parser (slurper) and a builder.
I've implemented a Groovy JsonSlurper and JsonParser, in the groovy.json package.
It's currently in Trunk.

You can play with those parser and builder by using a snapshot of Groovy 1.8 that you can find on Bamboo, for example:
http://bamboo.ci.codehaus.org/browse/GROOVY-CORE-5493/artifact/deliverables

You can have a look at the JSON GEP that Andres and I started a few months ago (GEP-7):
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support
It shows some of the examples.
Be sure to look at the JavaDocs which show various examples as well.

Although JSON and XML are pretty different, it would be nice if xml builders and json builder would be even closer (which is one of the goals of the built-in JSON support), as for example the following cases are not covered by the current implementation:
http://docs.codehaus.org/display/GroovyJSR/GEP+7+-+JSON+Support#GEP7-JSONSupport-Openquestions
I'd be interested in your input as how you'd expect such calls on the builder should serialize the JSON output, or even if we should even support them at all.

Thanks for your attention.

--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one





--
Guillaume Laforge
Groovy Project Manager
Head of Groovy Development at SpringSource
http://www.springsource.com/g2one
12