Quantcast

Multiple concurrent requests with HTTPBuilder

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Multiple concurrent requests with HTTPBuilder

nabblee
I am trying to send multiple requests through HTTPBuilder concurrently.  Should I be able to do that, or do I have to manage my own pool?  Here's what I see when I attempt to make a new request while a long-running request is pending:

<Apr 27, 2009 8:35:34 PM EDT> <Warning> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid use of SingleClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.>
java.lang.IllegalStateException: Adapter is detached.
        at org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab
stractPooledConnAdapter.java:90)
        at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool
edConnAdapter.java:118)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq
uestDirector.java:432)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
ient.java:555)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
ient.java:487)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
ient.java:465)
        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432)
        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405)
        at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet
hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
        at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta
MethodSite.java:52)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi
teArray.java:43)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
llSite.java:116)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
llSite.java:132)

I'm probably misusing it.  Is there a way to accomplish what I'm after?

Thanks!

--Lee
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

Thom Nichols
Use AsyncHTTPBuilder.  The response closure is executed asynchronously
and the request methods return a Future that can be used to determine
the state of the request and pass any information from the response
closure back to the main thread.  See
http://groovy.codehaus.org/modules/http-builder/doc/async.html

On Mon, Apr 27, 2009 at 9:01 PM, nabblee <[hidden email]> wrote:

>
> I am trying to send multiple requests through HTTPBuilder concurrently.
> Should I be able to do that, or do I have to manage my own pool?  Here's
> what I see when I attempt to make a new request while a long-running request
> is pending:
>
> <Apr 27, 2009 8:35:34 PM EDT> <Warning>
> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid
> use of SingleClientConnManager: connection still allocated.
> Make sure to release the connection before allocating another one.>
> java.lang.IllegalStateException: Adapter is detached.
>        at
> org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab
> stractPooledConnAdapter.java:90)
>        at
> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool
> edConnAdapter.java:118)
>        at
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq
> uestDirector.java:432)
>        at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
> ient.java:555)
>        at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
> ient.java:487)
>        at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
> ient.java:465)
>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432)
>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405)
>        at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> java:39)
>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:585)
>        at
> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet
> hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
>        at
> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta
> MethodSite.java:52)
>        at
> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi
> teArray.java:43)
>        at
> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
> llSite.java:116)
>        at
> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
> llSite.java:132)
>
> I'm probably misusing it.  Is there a way to accomplish what I'm after?
>
> Thanks!
>
> --Lee
> --
> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23268224.html
> Sent from the groovy - user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

Marc Guillemot
an alternative could perhaps be to use a
MultiThreadedHttpConnectionManager for the underlying HttpClient.

Cheers,
Marc.
--
Web: http://www.efficient-webtesting.com
Blog: http://mguillem.wordpress.com

Tom Nichols wrote:

> Use AsyncHTTPBuilder.  The response closure is executed asynchronously
> and the request methods return a Future that can be used to determine
> the state of the request and pass any information from the response
> closure back to the main thread.  See
> http://groovy.codehaus.org/modules/http-builder/doc/async.html
>
> On Mon, Apr 27, 2009 at 9:01 PM, nabblee <[hidden email]> wrote:
>> I am trying to send multiple requests through HTTPBuilder concurrently.
>> Should I be able to do that, or do I have to manage my own pool?  Here's
>> what I see when I attempt to make a new request while a long-running request
>> is pending:
>>
>> <Apr 27, 2009 8:35:34 PM EDT> <Warning>
>> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid
>> use of SingleClientConnManager: connection still allocated.
>> Make sure to release the connection before allocating another one.>
>> java.lang.IllegalStateException: Adapter is detached.
>>        at
>> org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab
>> stractPooledConnAdapter.java:90)
>>        at
>> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool
>> edConnAdapter.java:118)
>>        at
>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq
>> uestDirector.java:432)
>>        at
>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>> ient.java:555)
>>        at
>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>> ient.java:487)
>>        at
>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>> ient.java:465)
>>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432)
>>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405)
>>        at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
>> java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
>> sorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:585)
>>        at
>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet
>> hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
>>        at
>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta
>> MethodSite.java:52)
>>        at
>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi
>> teArray.java:43)
>>        at
>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
>> llSite.java:116)
>>        at
>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
>> llSite.java:132)
>>
>> I'm probably misusing it.  Is there a way to accomplish what I'm after?
>>
>> Thanks!
>>
>> --Lee
>> --
>> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23268224.html
>> Sent from the groovy - user mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

Thom Nichols
That's exactly what AsyncHttpBuilder does.  But there's a little more
to it than that - it also wraps a ThreadPoolExecutor so that requests
return a Future that can be used to retrieve the result.

On Thu, Apr 30, 2009 at 2:47 AM, Marc Guillemot <[hidden email]> wrote:

> an alternative could perhaps be to use a
> MultiThreadedHttpConnectionManager for the underlying HttpClient.
>
> Cheers,
> Marc.
> --
> Web: http://www.efficient-webtesting.com
> Blog: http://mguillem.wordpress.com
>
> Tom Nichols wrote:
>> Use AsyncHTTPBuilder.  The response closure is executed asynchronously
>> and the request methods return a Future that can be used to determine
>> the state of the request and pass any information from the response
>> closure back to the main thread.  See
>> http://groovy.codehaus.org/modules/http-builder/doc/async.html
>>
>> On Mon, Apr 27, 2009 at 9:01 PM, nabblee <[hidden email]> wrote:
>>> I am trying to send multiple requests through HTTPBuilder concurrently.
>>> Should I be able to do that, or do I have to manage my own pool?  Here's
>>> what I see when I attempt to make a new request while a long-running request
>>> is pending:
>>>
>>> <Apr 27, 2009 8:35:34 PM EDT> <Warning>
>>> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid
>>> use of SingleClientConnManager: connection still allocated.
>>> Make sure to release the connection before allocating another one.>
>>> java.lang.IllegalStateException: Adapter is detached.
>>>        at
>>> org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab
>>> stractPooledConnAdapter.java:90)
>>>        at
>>> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool
>>> edConnAdapter.java:118)
>>>        at
>>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq
>>> uestDirector.java:432)
>>>        at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>>> ient.java:555)
>>>        at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>>> ient.java:487)
>>>        at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>>> ient.java:465)
>>>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432)
>>>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405)
>>>        at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
>>> java:39)
>>>        at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
>>> sorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:585)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet
>>> hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta
>>> MethodSite.java:52)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi
>>> teArray.java:43)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
>>> llSite.java:116)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
>>> llSite.java:132)
>>>
>>> I'm probably misusing it.  Is there a way to accomplish what I'm after?
>>>
>>> Thanks!
>>>
>>> --Lee
>>> --
>>> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23268224.html
>>> Sent from the groovy - user mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

nabblee
I'm having a lot of trouble making AsyncHTTPBuilder work.

When I was using HTTPBuilder, my requests looked like this (from one of the examples):

                http.URL = 'https://...'
                def httpResult = http.request(POST,TEXT) { req ->
                  send URLENC, [sSessionID:binding.sessionid, lOrderID:id]
                         
                  response.success = { resp, reader ->
                       assert resp.statusLine.statusCode == 200
                  def xml = new XmlSlurper().parse(reader)
                          ...
                }
                         
                response.'404' = { resp ->
                    println "Retrieval failed: $resp"
                }


I'm trying to turn these into AsyncHTTPBuilder requests, but I keep getting a NullPointerException that I don't understand.  Here's what I've got:

                def httpResult = http.post( path:'/',
                                body:[sUserName:username, sPassword:password],
                                contentType:URLENC) {resp,html ->
                }
                               
                while(!httpResult.done) {
                        Thread.sleep(1000)
                }

                def xml = httpResult.post()
                // do something with response


The stacktrace originates at the very first line (def httpResult = ...), but here is the rest of it:

Exception in thread "Thread-1" java.lang.NullPointerException
        at groovyx.net.http.URIBuilder.convertToURI(URIBuilder.java:85)
        at groovyx.net.http.HTTPBuilder$SendDelegate.setPropertiesFromMap(HTTPBuilder.java:831)
        at groovyx.net.http.HTTPBuilder.post(HTTPBuilder.java:323)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
        at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)

I need to use a different URL for each request, so I tried to put in a base uri when instantiating AsyncHTTPBuilder:

        http = new AsyncHTTPBuilder(
                        poolSize : 20,
                        uri : baseUri,
                        contentType : URLENC )

I've tried using path:'/xyz' in the actual call.  I've tried uri, url, uri.path, etc.  I have yet to find the magic incantation that works.

All I want to do is allow multiple HTTP POSTs to occur in parallel.  I can't figure out the proper syntax for a post, and I'm not sure why I'm getting an NPE.  Nothing seems to be null, but the stacktrace indicates a problem with the URI.

Thanks,
Lee


Tom Nichols wrote
That's exactly what AsyncHttpBuilder does.  But there's a little more
to it than that - it also wraps a ThreadPoolExecutor so that requests
return a Future that can be used to retrieve the result.

On Thu, Apr 30, 2009 at 2:47 AM, Marc Guillemot <mguillemot@yahoo.fr> wrote:
> an alternative could perhaps be to use a
> MultiThreadedHttpConnectionManager for the underlying HttpClient.
>
> Cheers,
> Marc.
> --
> Web: http://www.efficient-webtesting.com
> Blog: http://mguillem.wordpress.com
>
> Tom Nichols wrote:
>> Use AsyncHTTPBuilder.  The response closure is executed asynchronously
>> and the request methods return a Future that can be used to determine
>> the state of the request and pass any information from the response
>> closure back to the main thread.  See
>> http://groovy.codehaus.org/modules/http-builder/doc/async.html
>>
>> On Mon, Apr 27, 2009 at 9:01 PM, nabblee <nabble@leegrey.com> wrote:
>>> I am trying to send multiple requests through HTTPBuilder concurrently.
>>> Should I be able to do that, or do I have to manage my own pool?  Here's
>>> what I see when I attempt to make a new request while a long-running request
>>> is pending:
>>>
>>> <Apr 27, 2009 8:35:34 PM EDT> <Warning>
>>> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid
>>> use of SingleClientConnManager: connection still allocated.
>>> Make sure to release the connection before allocating another one.>
>>> java.lang.IllegalStateException: Adapter is detached.
>>>        at
>>> org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab
>>> stractPooledConnAdapter.java:90)
>>>        at
>>> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool
>>> edConnAdapter.java:118)
>>>        at
>>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq
>>> uestDirector.java:432)
>>>        at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>>> ient.java:555)
>>>        at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>>> ient.java:487)
>>>        at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
>>> ient.java:465)
>>>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432)
>>>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405)
>>>        at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
>>> java:39)
>>>        at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
>>> sorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:585)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet
>>> hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta
>>> MethodSite.java:52)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi
>>> teArray.java:43)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
>>> llSite.java:116)
>>>        at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa
>>> llSite.java:132)
>>>
>>> I'm probably misusing it.  Is there a way to accomplish what I'm after?
>>>
>>> Thanks!
>>>
>>> --Lee
>>> --
>>> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23268224.html
>>> Sent from the groovy - user mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

nabblee
Okay, my bad.  I was running version 0.4.  I just upgraded to the latest 0.5-SNAPSHOT (http-builder-0.5.0-20090502.024916-2-all.zip).  So, I'm not getting the same NPE problem as before, but I'm not out of the woods, either.

        http = new AsyncHTTPBuilder(
                        poolSize : 20,
                        uri : 'https://oxbranch.optionsxpress.com/',
                        contentType : URLENC )

                def httpResult = http.post( path:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource',
                                body:[sUserName:username, sPassword:password],
                                contentType:URLENC) {resp,html ->
                                println "resp = ${resp.dump()}"
                                println "html = $html"
                }
                               
                while(!httpResult.done) {
                        println "Outside login..."
                        Thread.sleep(1000)
                }

println httpResult.dump()
                def xml = httpResult.get()
                println "httpResult.get() returns $xml"



Here's the output:
Outside login...
Outside login...
Outside login...
Outside login...
resp = <groovyx.net.http.HttpResponseDecorator@ef4504 headers=null responseBase=org.apache.http.message.BasicHttpResponse@9c176c responseData=null>
html = [:]
<java.util.concurrent.FutureTask@82d603 sync=java.util.concurrent.FutureTask$Sync@1b09282[State = 2, empty queue]>
httpResult.get() returns null

Why does get() return null?  Am I supposed to be doing something else to retrieve the response?

Thanks again!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

Thom Nichols
Since you are defining your own response handler closure, needs to
return whatever you want Future.get() to return.  If you do not pass a
closure as the last argument to post(), it will use the default
response handler which does just this.

So your example should look like this:
              def httpResult = http.post(...) { resp, html ->
                               println "resp = ${resp.dump()}"
                               println "html = $html"
                               return html
              }


On Sun, May 3, 2009 at 6:54 PM, nabblee <[hidden email]> wrote:

>
> Okay, my bad.  I was running version 0.4.  I just upgraded to the latest
> 0.5-SNAPSHOT (http-builder-0.5.0-20090502.024916-2-all.zip).  So, I'm not
> getting the same NPE problem as before, but I'm not out of the woods,
> either.
>
>        http = new AsyncHTTPBuilder(
>                        poolSize : 20,
>                        uri : 'https://oxbranch.optionsxpress.com/',
>                        contentType : URLENC )
>
>                def httpResult = http.post(
> path:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource',
>                                body:[sUserName:username, sPassword:password],
>                                contentType:URLENC) {resp,html ->
>                                println "resp = ${resp.dump()}"
>                                println "html = $html"
>                }
>
>                while(!httpResult.done) {
>                        println "Outside login..."
>                        Thread.sleep(1000)
>                }
>
> println httpResult.dump()
>                def xml = httpResult.get()
>                println "httpResult.get() returns $xml"
>
>
>
> Here's the output:
> Outside login...
> Outside login...
> Outside login...
> Outside login...
> resp = <groovyx.net.http.HttpResponseDecorator@ef4504 headers=null
> responseBase=org.apache.http.message.BasicHttpResponse@9c176c
> responseData=null>
> html = [:]
> <java.util.concurrent.FutureTask@82d603
> sync=java.util.concurrent.FutureTask$Sync@1b09282[State = 2, empty queue]>
> httpResult.get() returns null
>
> Why does get() return null?  Am I supposed to be doing something else to
> retrieve the response?
>
> Thanks again!
>
> --
> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23360779.html
> Sent from the groovy - user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

nabblee
Thanks for your response, Tom, but it's still not working.  :-(  I'm getting an empty map back...

                def httpResult = http.post(
                        uri:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource',
                        body:[sUserName:username, sPassword:password, sSource:binding.sourceid, sSessionID:''],
                        contentType:URLENC
                        ) { resp, xml ->
                                println "resp = ${resp.dump()}"
                                println "xml = $xml"
                                return xml
                }

        while(!httpResult.done) {
                                println "Outside login..."
                                Thread.sleep(1000)
                        }

println "httpResult: ${httpResult.dump()}"
                        def xml = httpResult.get()
                        println "httpResult.get() returns $xml"


Output:

Outside login...
Outside login...
Outside login...
Outside login...
resp = <groovyx.net.http.HttpResponseDecorator@9c176c headers=null responseBase=org.apache.http.message.BasicHttpResponse@1c5ddd3 responseData=null>
xml = [:]
httpResult: <java.util.concurrent.FutureTask@82d603 sync=java.util.concurrent.FutureTask$Sync@1b09282[State = 2, empty queue]>
httpResult.get() returns [:]


Maybe I'm not structuring the arguments to post() correctly?  It takes about the amount of time I expect, but I don't see a usable reply.

Thanks for any insights,
Lee


Tom Nichols wrote
Since you are defining your own response handler closure, needs to
return whatever you want Future.get() to return.  If you do not pass a
closure as the last argument to post(), it will use the default
response handler which does just this.

So your example should look like this:
              def httpResult = http.post(...) { resp, html ->
                               println "resp = ${resp.dump()}"
                               println "html = $html"
                               return html
              }


On Sun, May 3, 2009 at 6:54 PM, nabblee <nabble@leegrey.com> wrote:
>
> Okay, my bad.  I was running version 0.4.  I just upgraded to the latest
> 0.5-SNAPSHOT (http-builder-0.5.0-20090502.024916-2-all.zip).  So, I'm not
> getting the same NPE problem as before, but I'm not out of the woods,
> either.
>
>        http = new AsyncHTTPBuilder(
>                        poolSize : 20,
>                        uri : 'https://oxbranch.optionsxpress.com/',
>                        contentType : URLENC )
>
>                def httpResult = http.post(
> path:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource',
>                                body:[sUserName:username, sPassword:password],
>                                contentType:URLENC) {resp,html ->
>                                println "resp = ${resp.dump()}"
>                                println "html = $html"
>                }
>
>                while(!httpResult.done) {
>                        println "Outside login..."
>                        Thread.sleep(1000)
>                }
>
> println httpResult.dump()
>                def xml = httpResult.get()
>                println "httpResult.get() returns $xml"
>
>
>
> Here's the output:
> Outside login...
> Outside login...
> Outside login...
> Outside login...
> resp = <groovyx.net.http.HttpResponseDecorator@ef4504 headers=null
> responseBase=org.apache.http.message.BasicHttpResponse@9c176c
> responseData=null>
> html = [:]
> <java.util.concurrent.FutureTask@82d603
> sync=java.util.concurrent.FutureTask$Sync@1b09282[State = 2, empty queue]>
> httpResult.get() returns null
>
> Why does get() return null?  Am I supposed to be doing something else to
> retrieve the response?
>
> Thanks again!
>
> --
> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23360779.html
> Sent from the groovy - user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

nabblee
Okay, I've got it now.  The problem was the contentType.  It needed to be XML, since that's what comes back.  I had URLENC, because that's what I'm sending to them, but I guess maybe that's implied with a POST?  Anyway, I guess that's why I was getting an empty map back, since it had no success finding key/value pairs in the XML response.  Once I changed it to XML, I started seeing the contents I expected.

Not to look a gift horse in the mouth, because I do appreciate HTTPBuilder, but it really needs more docs in order to be understood.  The few examples that are provided don't give enough data points to allow extrapolation.  And POST examples are always more useful than GET examples.  ;-)

Sincere thanks for the library, which I make lots of use of.

Cheers,
Lee
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

nabblee
In reply to this post by Thom Nichols
I do have another question about not passing a closure to post().  When I try that, I get an error:


                def httpResult = http.post(
                        uri:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource',
                        body:[sUserName:username, sPassword:password, sSource:binding.sourceid, sSessionID:''],
                        contentType:XML
                        )


May 3, 2009 7:48:39 PM groovyx.net.http.AsyncHTTPBuilder$1 call
SEVERE: Exception thrown from request delegate: groovyx.net.http.HTTPBuilder$RequestConfigDelegate@13a0212
java.lang.IllegalArgumentException: Response closure must accept one or two parameters
        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:459)
        at groovyx.net.http.AsyncHTTPBuilder.doRequestSuper(AsyncHTTPBuilder.java:119)
        at groovyx.net.http.AsyncHTTPBuilder.access$000(AsyncHTTPBuilder.java:55)
        at groovyx.net.http.AsyncHTTPBuilder$1.call(AsyncHTTPBuilder.java:103)
        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

So I seem to have no choice but to create a closure that simply returns the second parameter.  Why is that different from what you said would be the default?

Thanks,
Lee


Tom Nichols wrote
Since you are defining your own response handler closure, needs to
return whatever you want Future.get() to return.  If you do not pass a
closure as the last argument to post(), it will use the default
response handler which does just this.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

Thom Nichols
In reply to this post by nabblee
Yes, in this case the post method assumes the request body is
url-encoded.  I suppose I should have mentioned it in the
documentation example, but it is mentioned in the JavaDoc:
http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/HTTPBuilder.html#post(java.util.Map,%20groovy.lang.Closure)

If you use RESTClient, it does not automatically assume a url-encoded
request, since you could i.e. be POSTing XML data.  HTTPBuilder.post
is meant as a convenience method since HTTP form POSTs are a very
common task.  It's the only method that explicitly assumes a different
request content type.  I suppose I should have made that clear in the
website documentation.  Thanks for pointing that out.

Normally, in any case where contentType is specified, it defines the
request _and_ response contentType.  If they're different, there's a
'requestContentType' that you can specify as well.  The 'post'
method's javadoc also links to this method:
http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/HTTPBuilder.RequestConfigDelegate.html#setPropertiesFromMap(java.util.Map)
which defines all named parameters for all of the convenience methods
used in HTTPBuilder and child classes (RESTClient and
AsyncHTTPBuilder).

FYI, if you were to do the same using the request() method, you would
do it like so in order to define both the request and response
content-type:
http.request( 'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource',
XML ) {
  send URLENC, [sUserName:username, sPassword:password,
sSource:binding.sourceid, sSessionID:'']
  response.success =  { resp, xml ->
                               println "resp = ${resp.dump()}"
                               println "xml = $xml"
                               return xml
  }
}

I tend to prefer this form as it's easier to read:  "I'm requesting
this URL to send me XML, and sending a url-encoded map as the body.
Then on a successful response do (xyz)...

'send' is defined here:
http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/HTTPBuilder.RequestConfigDelegate.html#send(java.lang.Object,%20java.lang.Object)
and it's what defines the extra methods in the request closure's scope.


Hope this helps explain things.
-Tom


On Sun, May 3, 2009 at 7:47 PM, nabblee <[hidden email]> wrote:

>
> Okay, I've got it now.  The problem was the contentType.  It needed to be
> XML, since that's what comes back.  I had URLENC, because that's what I'm
> sending to them, but I guess maybe that's implied with a POST?  Anyway, I
> guess that's why I was getting an empty map back, since it had no success
> finding key/value pairs in the XML response.  Once I changed it to XML, I
> started seeing the contents I expected.
>
> Not to look a gift horse in the mouth, because I do appreciate HTTPBuilder,
> but it really needs more docs in order to be understood.  The few examples
> that are provided don't give enough data points to allow extrapolation.  And
> POST examples are always more useful than GET examples.  ;-)
>
> Sincere thanks for the library, which I make lots of use of.
>
> Cheers,
> Lee
> --
> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23361126.html
> Sent from the groovy - user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Multiple concurrent requests with HTTPBuilder

Thom Nichols
In reply to this post by nabblee
This appears to be a bug related to my use of MethodClosure from a
sub-class.  I'm looking into it now.  But yes, a work-around is to
define your own response handler for now.


On Sun, May 3, 2009 at 7:53 PM, nabblee <[hidden email]> wrote:

>
> I do have another question about not passing a closure to post().  When I try
> that, I get an error:
>
>
>                def httpResult = http.post(
>
> uri:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource',
>                        body:[sUserName:username, sPassword:password, sSource:binding.sourceid,
> sSessionID:''],
>                        contentType:XML
>                        )
>
>
> May 3, 2009 7:48:39 PM groovyx.net.http.AsyncHTTPBuilder$1 call
> SEVERE: Exception thrown from request delegate:
> groovyx.net.http.HTTPBuilder$RequestConfigDelegate@13a0212
> java.lang.IllegalArgumentException: Response closure must accept one or two
> parameters
>        at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:459)
>        at
> groovyx.net.http.AsyncHTTPBuilder.doRequestSuper(AsyncHTTPBuilder.java:119)
>        at groovyx.net.http.AsyncHTTPBuilder.access$000(AsyncHTTPBuilder.java:55)
>        at groovyx.net.http.AsyncHTTPBuilder$1.call(AsyncHTTPBuilder.java:103)
>        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>        at java.util.concurrent.FutureTask.run(Unknown Source)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>        at java.lang.Thread.run(Unknown Source)
>
> So I seem to have no choice but to create a closure that simply returns the
> second parameter.  Why is that different from what you said would be the
> default?
>
> Thanks,
> Lee
>
>
>
> Tom Nichols wrote:
>>
>> Since you are defining your own response handler closure, needs to
>> return whatever you want Future.get() to return.  If you do not pass a
>> closure as the last argument to post(), it will use the default
>> response handler which does just this.
>>
>
> --
> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23361185.html
> Sent from the groovy - user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

HTTPBuilder warning "Cannot find parser for content-type"

David R
In reply to this post by Thom Nichols
I am using HttpBuilder to download files -- mp3, etc.

I get the following warning:
WARNING: Cannot find parser for content-type: audio/mpeg -- using
default parser.

The default parser is fine for me since I am just saving the file
locally.
Perhaps this should be an INFO log and not WARNING?


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HTTPBuilder warning "Cannot find parser for content-type"

Thom Nichols
Well, that's just it -- there is no 'default parser' -- you'll
actually get null for the second argument in your response handler
closure.  I suppose I could make a default parser that just passes the
InputStream to the response handler.  But in your case, you could do
something like this:

def mp3Dir = new File( 'mp3s/' )
mp3Dir.mkdirs()

def http = new HTTPBuilder( 'http://www.somesite.com' )
http.handler.'audio/mpeg' = { resp ->
  def filename =
resp.getFirstHeader('Content-Disposition').elements[0].getParameterByName('filename').value
  def destFile = new File( mp3Dir, filename )
  destFile << resp.entity.content
}

Then just call:
http.get( path : 'dir/song.mp3' )

and it will save the file directly to your folder or throw an
exception if the request failed.  Note that the above code is
untested.

I'm reluctant to change the log level for that message, because I
imagine most people will expect the automatic parsing, and I suspect
I'd start getting a lot of "why am I not getting any parsed data"
questions if that log message was hidden.

-Tom


On Wed, May 6, 2009 at 6:37 AM, David Rosenstark
<[hidden email]> wrote:

> I am using HttpBuilder to download files -- mp3, etc.
>
> I get the following warning:
> WARNING: Cannot find parser for content-type: audio/mpeg -- using
> default parser.
>
> The default parser is fine for me since I am just saving the file
> locally.
> Perhaps this should be an INFO log and not WARNING?
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: HTTPBuilder warning "Cannot find parser for content-type"

David R
Thanks, that seems pretty simple. All I have to do is move my current code.

My point was that if I have a default parser in my code, like this:
http.get(path: strUrl) {resp, reader ->
.....
}
Then, there is no reason for the warning since I am handling all cases the same.
In my case, the code is generic -- it can read any binary data and write to the file so why define a specific handler?

I need to look to confirm that the " destFile << resp.entity.content" really works. I think when I started this project I tried that and in the end had to read the data using eachByte so it would be read correctly (binary).




-----Original Message-----
From: Tom Nichols [mailto:[hidden email]]
Sent: Wednesday, May 06, 2009 4:01 PM
To: [hidden email]
Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"

Well, that's just it -- there is no 'default parser' -- you'll
actually get null for the second argument in your response handler
closure.  I suppose I could make a default parser that just passes the
InputStream to the response handler.  But in your case, you could do
something like this:

def mp3Dir = new File( 'mp3s/' )
mp3Dir.mkdirs()

def http = new HTTPBuilder( 'http://www.somesite.com' )
http.handler.'audio/mpeg' = { resp ->
  def filename =
resp.getFirstHeader('Content-Disposition').elements[0].getParameterByName('filename').value
  def destFile = new File( mp3Dir, filename )
  destFile << resp.entity.content
}

Then just call:
http.get( path : 'dir/song.mp3' )

and it will save the file directly to your folder or throw an
exception if the request failed.  Note that the above code is
untested.

I'm reluctant to change the log level for that message, because I
imagine most people will expect the automatic parsing, and I suspect
I'd start getting a lot of "why am I not getting any parsed data"
questions if that log message was hidden.

-Tom


On Wed, May 6, 2009 at 6:37 AM, David Rosenstark
<[hidden email]> wrote:

> I am using HttpBuilder to download files -- mp3, etc.
>
> I get the following warning:
> WARNING: Cannot find parser for content-type: audio/mpeg -- using
> default parser.
>
> The default parser is fine for me since I am just saving the file
> locally.
> Perhaps this should be an INFO log and not WARNING?
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: HTTPBuilder warning "Cannot find parser for content-type"

David R
In reply to this post by Thom Nichols
Tom, sorry but one more question about your response.

My response closure takes 2 parameters currently -- the resp and the reader, so I have defined my own catch all parser.
Is there a way to tell the class that this parser is the one to use for all content types?

-----Original Message-----
From: Tom Nichols [mailto:[hidden email]]
Sent: Wednesday, May 06, 2009 4:01 PM
To: [hidden email]
Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"

Well, that's just it -- there is no 'default parser' -- you'll
actually get null for the second argument in your response handler
closure.  I suppose I could make a default parser that just passes the
InputStream to the response handler.  But in your case, you could do
something like this:

def mp3Dir = new File( 'mp3s/' )
mp3Dir.mkdirs()

def http = new HTTPBuilder( 'http://www.somesite.com' )
http.handler.'audio/mpeg' = { resp ->
  def filename =
resp.getFirstHeader('Content-Disposition').elements[0].getParameterByName('filename').value
  def destFile = new File( mp3Dir, filename )
  destFile << resp.entity.content
}

Then just call:
http.get( path : 'dir/song.mp3' )

and it will save the file directly to your folder or throw an
exception if the request failed.  Note that the above code is
untested.

I'm reluctant to change the log level for that message, because I
imagine most people will expect the automatic parsing, and I suspect
I'd start getting a lot of "why am I not getting any parsed data"
questions if that log message was hidden.

-Tom


On Wed, May 6, 2009 at 6:37 AM, David Rosenstark
<[hidden email]> wrote:

> I am using HttpBuilder to download files -- mp3, etc.
>
> I get the following warning:
> WARNING: Cannot find parser for content-type: audio/mpeg -- using
> default parser.
>
> The default parser is fine for me since I am just saving the file
> locally.
> Perhaps this should be an INFO log and not WARNING?
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HTTPBuilder warning "Cannot find parser for content-type"

Thom Nichols
I realized in my previous code example I have a
http.handler.'audio/mpeg' -- which should be http.parser.'audio/mpeg'
= { ... }

I just remembered what will actually do what you want --
http.contentType = ContentType.BINARY

This will cause all responses to be parsed using
http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/ParserRegistry.html#parseStream(org.apache.http.HttpResponse)
which really just passes the underlying
HttpResponse.getEntity().getContent() as the second parameter to the
response closure.  And the stream will automatically be closed after
your response closure returns.


2009/5/6 David Rosenstark <[hidden email]>:

> Tom, sorry but one more question about your response.
>
> My response closure takes 2 parameters currently -- the resp and the reader, so I have defined my own catch all parser.
> Is there a way to tell the class that this parser is the one to use for all content types?
>
> -----Original Message-----
> From: Tom Nichols [mailto:[hidden email]]
> Sent: Wednesday, May 06, 2009 4:01 PM
> To: [hidden email]
> Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"
>
> Well, that's just it -- there is no 'default parser' -- you'll
> actually get null for the second argument in your response handler
> closure.  I suppose I could make a default parser that just passes the
> InputStream to the response handler.  But in your case, you could do
> something like this:
>
> def mp3Dir = new File( 'mp3s/' )
> mp3Dir.mkdirs()
>
> def http = new HTTPBuilder( 'http://www.somesite.com' )
> http.handler.'audio/mpeg' = { resp ->
>  def filename =
> resp.getFirstHeader('Content-Disposition').elements[0].getParameterByName('filename').value
>  def destFile = new File( mp3Dir, filename )
>  destFile << resp.entity.content
> }
>
> Then just call:
> http.get( path : 'dir/song.mp3' )
>
> and it will save the file directly to your folder or throw an
> exception if the request failed.  Note that the above code is
> untested.
>
> I'm reluctant to change the log level for that message, because I
> imagine most people will expect the automatic parsing, and I suspect
> I'd start getting a lot of "why am I not getting any parsed data"
> questions if that log message was hidden.
>
> -Tom
>
>
> On Wed, May 6, 2009 at 6:37 AM, David Rosenstark
> <[hidden email]> wrote:
>> I am using HttpBuilder to download files -- mp3, etc.
>>
>> I get the following warning:
>> WARNING: Cannot find parser for content-type: audio/mpeg -- using
>> default parser.
>>
>> The default parser is fine for me since I am just saving the file
>> locally.
>> Perhaps this should be an INFO log and not WARNING?
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: HTTPBuilder warning "Cannot find parser for content-type"

David R
Two questions:
1. When am I setting the contentType after creating my HttpBuilder?
2. All this returns is an outputstream which does not seem that different from my current code

http.get(path: strUrl)  {resp, reader ->
                                reader.eachByte {
                                        outFile.write(it)
                                }
                                outFile.close()
}

(By the way, I had decided to suppress the log for this class alone in log4j config to solve the issue)

 

-----Original Message-----
From: Tom Nichols [mailto:[hidden email]]
Sent: Wednesday, May 06, 2009 8:00 PM
To: [hidden email]
Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"

I realized in my previous code example I have a http.handler.'audio/mpeg' -- which should be http.parser.'audio/mpeg'
= { ... }

I just remembered what will actually do what you want -- http.contentType = ContentType.BINARY

This will cause all responses to be parsed using
http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/ParserRegistry.html#parseStream(org.apache.http.HttpResponse)
which really just passes the underlying
HttpResponse.getEntity().getContent() as the second parameter to the response closure.  And the stream will automatically be closed after your response closure returns.


2009/5/6 David Rosenstark <[hidden email]>:

> Tom, sorry but one more question about your response.
>
> My response closure takes 2 parameters currently -- the resp and the reader, so I have defined my own catch all parser.
> Is there a way to tell the class that this parser is the one to use for all content types?
>
> -----Original Message-----
> From: Tom Nichols [mailto:[hidden email]]
> Sent: Wednesday, May 06, 2009 4:01 PM
> To: [hidden email]
> Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"
>
> Well, that's just it -- there is no 'default parser' -- you'll
> actually get null for the second argument in your response handler
> closure.  I suppose I could make a default parser that just passes the
> InputStream to the response handler.  But in your case, you could do
> something like this:
>
> def mp3Dir = new File( 'mp3s/' )
> mp3Dir.mkdirs()
>
> def http = new HTTPBuilder( 'http://www.somesite.com' )
> http.handler.'audio/mpeg' = { resp ->
>  def filename =
> resp.getFirstHeader('Content-Disposition').elements[0].getParameterByN
> ame('filename').value
>  def destFile = new File( mp3Dir, filename )
>  destFile << resp.entity.content
> }
>
> Then just call:
> http.get( path : 'dir/song.mp3' )
>
> and it will save the file directly to your folder or throw an
> exception if the request failed.  Note that the above code is
> untested.
>
> I'm reluctant to change the log level for that message, because I
> imagine most people will expect the automatic parsing, and I suspect
> I'd start getting a lot of "why am I not getting any parsed data"
> questions if that log message was hidden.
>
> -Tom
>
>
> On Wed, May 6, 2009 at 6:37 AM, David Rosenstark
> <[hidden email]> wrote:
>> I am using HttpBuilder to download files -- mp3, etc.
>>
>> I get the following warning:
>> WARNING: Cannot find parser for content-type: audio/mpeg -- using
>> default parser.
>>
>> The default parser is fine for me since I am just saving the file
>> locally.
>> Perhaps this should be an INFO log and not WARNING?
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: HTTPBuilder warning "Cannot find parser for content-type"

Thom Nichols
You know, I was tired this morning when I read your initial question
and if I had woken up a little bit first I would have realized that
you're already doing what I was trying to suggest.  I also forgot that
you do in fact get an InputStream passed to the response handler by a
default parser (the parseStream method); I was thinking the parsed
data would be null if it couldn't find a suitable parser.  (BTW -
looking at your code example, just remember it is a stream, not a
reader - reader suggests a conversion from bytes to characters based
on a certain character set).

The reason I suggested passing a BINARY contentType parameter is
because this would force the HTTPBuilder to use
ParserRegistry.parseStream, which is what the default parser does
anyway (but you wouldn't get that warning).

Anyway, if all I've done is serve to confuse you, I realized it is
probably worthwhile to be able to manipulate the default parser
behavior, so that's what I've done in the ParserRegistry class.  And
assuming one did set a default parser, it would be annoying to see a
warning message for an expected behavior.  So I will in fact change
that log level.

But after all that, and reading your emails a couple more times, it
looks like you already had the solution; changing the log level for
ParserRegistry was probably the easiest thing to do.  The one thing
I'd suggest is that iterating over each byte and writing a byte at a
time will probably be slow, but if it works for you then great :)

-Tom


2009/5/6 David Rosenstark <[hidden email]>:

> Two questions:
> 1. When am I setting the contentType after creating my HttpBuilder?
> 2. All this returns is an outputstream which does not seem that different from my current code
>
> http.get(path: strUrl)  {resp, reader ->
>                                reader.eachByte {
>                                        outFile.write(it)
>                                }
>                                outFile.close()
> }
>
> (By the way, I had decided to suppress the log for this class alone in log4j config to solve the issue)
>
>
>
> -----Original Message-----
> From: Tom Nichols [mailto:[hidden email]]
> Sent: Wednesday, May 06, 2009 8:00 PM
> To: [hidden email]
> Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"
>
> I realized in my previous code example I have a http.handler.'audio/mpeg' -- which should be http.parser.'audio/mpeg'
> = { ... }
>
> I just remembered what will actually do what you want -- http.contentType = ContentType.BINARY
>
> This will cause all responses to be parsed using
> http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/ParserRegistry.html#parseStream(org.apache.http.HttpResponse)
> which really just passes the underlying
> HttpResponse.getEntity().getContent() as the second parameter to the response closure.  And the stream will automatically be closed after your response closure returns.
>
>
> 2009/5/6 David Rosenstark <[hidden email]>:
>> Tom, sorry but one more question about your response.
>>
>> My response closure takes 2 parameters currently -- the resp and the reader, so I have defined my own catch all parser.
>> Is there a way to tell the class that this parser is the one to use for all content types?
>>
>> -----Original Message-----
>> From: Tom Nichols [mailto:[hidden email]]
>> Sent: Wednesday, May 06, 2009 4:01 PM
>> To: [hidden email]
>> Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"
>>
>> Well, that's just it -- there is no 'default parser' -- you'll
>> actually get null for the second argument in your response handler
>> closure.  I suppose I could make a default parser that just passes the
>> InputStream to the response handler.  But in your case, you could do
>> something like this:
>>
>> def mp3Dir = new File( 'mp3s/' )
>> mp3Dir.mkdirs()
>>
>> def http = new HTTPBuilder( 'http://www.somesite.com' )
>> http.handler.'audio/mpeg' = { resp ->
>>  def filename =
>> resp.getFirstHeader('Content-Disposition').elements[0].getParameterByN
>> ame('filename').value
>>  def destFile = new File( mp3Dir, filename )
>>  destFile << resp.entity.content
>> }
>>
>> Then just call:
>> http.get( path : 'dir/song.mp3' )
>>
>> and it will save the file directly to your folder or throw an
>> exception if the request failed.  Note that the above code is
>> untested.
>>
>> I'm reluctant to change the log level for that message, because I
>> imagine most people will expect the automatic parsing, and I suspect
>> I'd start getting a lot of "why am I not getting any parsed data"
>> questions if that log message was hidden.
>>
>> -Tom
>>
>>
>> On Wed, May 6, 2009 at 6:37 AM, David Rosenstark
>> <[hidden email]> wrote:
>>> I am using HttpBuilder to download files -- mp3, etc.
>>>
>>> I get the following warning:
>>> WARNING: Cannot find parser for content-type: audio/mpeg -- using
>>> default parser.
>>>
>>> The default parser is fine for me since I am just saving the file
>>> locally.
>>> Perhaps this should be an INFO log and not WARNING?
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: HTTPBuilder warning "Cannot find parser for content-type"

David R
If what you are saying is correct, that it is a stream, why does the code work as a reader?
And what method can I use to iterate over it in groovy -- I know of eachByte and eachLine, what would be my alternative?
As for efficiency, the main problem was output and by using a bufferedwriter, performance improved. If I could read blocks in groovy that would be great and not have to lose my each... Closure solution.

Thanks,
David
 

-----Original Message-----
From: Tom Nichols [mailto:[hidden email]]
Sent: Thursday, May 07, 2009 1:15 AM
To: user
Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"

You know, I was tired this morning when I read your initial question and if I had woken up a little bit first I would have realized that you're already doing what I was trying to suggest.  I also forgot that you do in fact get an InputStream passed to the response handler by a default parser (the parseStream method); I was thinking the parsed data would be null if it couldn't find a suitable parser.  (BTW - looking at your code example, just remember it is a stream, not a reader - reader suggests a conversion from bytes to characters based on a certain character set).

The reason I suggested passing a BINARY contentType parameter is because this would force the HTTPBuilder to use ParserRegistry.parseStream, which is what the default parser does anyway (but you wouldn't get that warning).

Anyway, if all I've done is serve to confuse you, I realized it is probably worthwhile to be able to manipulate the default parser behavior, so that's what I've done in the ParserRegistry class.  And assuming one did set a default parser, it would be annoying to see a warning message for an expected behavior.  So I will in fact change that log level.

But after all that, and reading your emails a couple more times, it looks like you already had the solution; changing the log level for ParserRegistry was probably the easiest thing to do.  The one thing I'd suggest is that iterating over each byte and writing a byte at a time will probably be slow, but if it works for you then great :)

-Tom


2009/5/6 David Rosenstark <[hidden email]>:

> Two questions:
> 1. When am I setting the contentType after creating my HttpBuilder?
> 2. All this returns is an outputstream which does not seem that
> different from my current code
>
> http.get(path: strUrl)  {resp, reader ->
>                                reader.eachByte {
>                                        outFile.write(it)
>                                }
>                                outFile.close() }
>
> (By the way, I had decided to suppress the log for this class alone in
> log4j config to solve the issue)
>
>
>
> -----Original Message-----
> From: Tom Nichols [mailto:[hidden email]]
> Sent: Wednesday, May 06, 2009 8:00 PM
> To: [hidden email]
> Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"
>
> I realized in my previous code example I have a http.handler.'audio/mpeg' -- which should be http.parser.'audio/mpeg'
> = { ... }
>
> I just remembered what will actually do what you want --
> http.contentType = ContentType.BINARY
>
> This will cause all responses to be parsed using
> http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/ht
> tp/ParserRegistry.html#parseStream(org.apache.http.HttpResponse)
> which really just passes the underlying
> HttpResponse.getEntity().getContent() as the second parameter to the response closure.  And the stream will automatically be closed after your response closure returns.
>
>
> 2009/5/6 David Rosenstark <[hidden email]>:
>> Tom, sorry but one more question about your response.
>>
>> My response closure takes 2 parameters currently -- the resp and the reader, so I have defined my own catch all parser.
>> Is there a way to tell the class that this parser is the one to use for all content types?
>>
>> -----Original Message-----
>> From: Tom Nichols [mailto:[hidden email]]
>> Sent: Wednesday, May 06, 2009 4:01 PM
>> To: [hidden email]
>> Subject: Re: [groovy-user] HTTPBuilder warning "Cannot find parser for content-type"
>>
>> Well, that's just it -- there is no 'default parser' -- you'll
>> actually get null for the second argument in your response handler
>> closure.  I suppose I could make a default parser that just passes
>> the InputStream to the response handler.  But in your case, you could
>> do something like this:
>>
>> def mp3Dir = new File( 'mp3s/' )
>> mp3Dir.mkdirs()
>>
>> def http = new HTTPBuilder( 'http://www.somesite.com' )
>> http.handler.'audio/mpeg' = { resp ->
>>  def filename =
>> resp.getFirstHeader('Content-Disposition').elements[0].getParameterBy
>> N
>> ame('filename').value
>>  def destFile = new File( mp3Dir, filename )
>>  destFile << resp.entity.content
>> }
>>
>> Then just call:
>> http.get( path : 'dir/song.mp3' )
>>
>> and it will save the file directly to your folder or throw an
>> exception if the request failed.  Note that the above code is
>> untested.
>>
>> I'm reluctant to change the log level for that message, because I
>> imagine most people will expect the automatic parsing, and I suspect
>> I'd start getting a lot of "why am I not getting any parsed data"
>> questions if that log message was hidden.
>>
>> -Tom
>>
>>
>> On Wed, May 6, 2009 at 6:37 AM, David Rosenstark
>> <[hidden email]> wrote:
>>> I am using HttpBuilder to download files -- mp3, etc.
>>>
>>> I get the following warning:
>>> WARNING: Cannot find parser for content-type: audio/mpeg -- using
>>> default parser.
>>>
>>> The default parser is fine for me since I am just saving the file
>>> locally.
>>> Perhaps this should be an INFO log and not WARNING?
>>>
>>>
>>> --------------------------------------------------------------------
>>> - To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


12
Loading...