Multiple concurrent requests with HTTPBuilder
Locked
123
123
 
|
I am trying to send multiple requests through HTTPBuilder concurrently. Should I be able to do that, or do I have to manage my own pool? Here's what I see when I attempt to make a new request while a long-running request is pending:
<Apr 27, 2009 8:35:34 PM EDT> <Warning> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid use of SingleClientConnManager: connection still allocated. Make sure to release the connection before allocating another one.> java.lang.IllegalStateException: Adapter is detached. at org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab stractPooledConnAdapter.java:90) at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool edConnAdapter.java:118) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq uestDirector.java:432) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl ient.java:555) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl ient.java:487) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl ient.java:465) at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432) at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405) at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta MethodSite.java:52) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi teArray.java:43) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa llSite.java:116) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa llSite.java:132) I'm probably misusing it. Is there a way to accomplish what I'm after? Thanks! --Lee |
Re: Multiple concurrent requests with HTTPBuilder
|
|
Use AsyncHTTPBuilder. The response closure is executed asynchronously
and the request methods return a Future that can be used to determine the state of the request and pass any information from the response closure back to the main thread. See http://groovy.codehaus.org/modules/http-builder/doc/async.html On Mon, Apr 27, 2009 at 9:01 PM, nabblee <[hidden email]> wrote: > > I am trying to send multiple requests through HTTPBuilder concurrently. > Should I be able to do that, or do I have to manage my own pool? Here's > what I see when I attempt to make a new request while a long-running request > is pending: > > <Apr 27, 2009 8:35:34 PM EDT> <Warning> > <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid > use of SingleClientConnManager: connection still allocated. > Make sure to release the connection before allocating another one.> > java.lang.IllegalStateException: Adapter is detached. > at > org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab > stractPooledConnAdapter.java:90) > at > org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool > edConnAdapter.java:118) > at > org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq > uestDirector.java:432) > at > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl > ient.java:555) > at > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl > ient.java:487) > at > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl > ient.java:465) > at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432) > at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405) > at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. > java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces > sorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at > org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet > hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229) > at > org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta > MethodSite.java:52) > at > org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi > teArray.java:43) > at > org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa > llSite.java:116) > at > org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa > llSite.java:132) > > I'm probably misusing it. Is there a way to accomplish what I'm after? > > Thanks! > > --Lee > -- > View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23268224.html > Sent from the groovy - user mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
Re: Multiple concurrent requests with HTTPBuilder
|
|
an alternative could perhaps be to use a
MultiThreadedHttpConnectionManager for the underlying HttpClient. Cheers, Marc. -- Web: http://www.efficient-webtesting.com Blog: http://mguillem.wordpress.com Tom Nichols wrote: > Use AsyncHTTPBuilder. The response closure is executed asynchronously > and the request methods return a Future that can be used to determine > the state of the request and pass any information from the response > closure back to the main thread. See > http://groovy.codehaus.org/modules/http-builder/doc/async.html > > On Mon, Apr 27, 2009 at 9:01 PM, nabblee <[hidden email]> wrote: >> I am trying to send multiple requests through HTTPBuilder concurrently. >> Should I be able to do that, or do I have to manage my own pool? Here's >> what I see when I attempt to make a new request while a long-running request >> is pending: >> >> <Apr 27, 2009 8:35:34 PM EDT> <Warning> >> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid >> use of SingleClientConnManager: connection still allocated. >> Make sure to release the connection before allocating another one.> >> java.lang.IllegalStateException: Adapter is detached. >> at >> org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab >> stractPooledConnAdapter.java:90) >> at >> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool >> edConnAdapter.java:118) >> at >> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq >> uestDirector.java:432) >> at >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl >> ient.java:555) >> at >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl >> ient.java:487) >> at >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl >> ient.java:465) >> at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432) >> at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405) >> at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. >> java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces >> sorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:585) >> at >> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet >> hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229) >> at >> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta >> MethodSite.java:52) >> at >> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi >> teArray.java:43) >> at >> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa >> llSite.java:116) >> at >> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa >> llSite.java:132) >> >> I'm probably misusing it. Is there a way to accomplish what I'm after? >> >> Thanks! >> >> --Lee >> -- >> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23268224.html >> Sent from the groovy - user mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
Re: Multiple concurrent requests with HTTPBuilder
|
|
That's exactly what AsyncHttpBuilder does. But there's a little more
to it than that - it also wraps a ThreadPoolExecutor so that requests return a Future that can be used to retrieve the result. On Thu, Apr 30, 2009 at 2:47 AM, Marc Guillemot <[hidden email]> wrote: > an alternative could perhaps be to use a > MultiThreadedHttpConnectionManager for the underlying HttpClient. > > Cheers, > Marc. > -- > Web: http://www.efficient-webtesting.com > Blog: http://mguillem.wordpress.com > > Tom Nichols wrote: >> Use AsyncHTTPBuilder. The response closure is executed asynchronously >> and the request methods return a Future that can be used to determine >> the state of the request and pass any information from the response >> closure back to the main thread. See >> http://groovy.codehaus.org/modules/http-builder/doc/async.html >> >> On Mon, Apr 27, 2009 at 9:01 PM, nabblee <[hidden email]> wrote: >>> I am trying to send multiple requests through HTTPBuilder concurrently. >>> Should I be able to do that, or do I have to manage my own pool? Here's >>> what I see when I attempt to make a new request while a long-running request >>> is pending: >>> >>> <Apr 27, 2009 8:35:34 PM EDT> <Warning> >>> <org.apache.http.impl.conn.SingleClientConnManager> <BEA-000000> <Invalid >>> use of SingleClientConnManager: connection still allocated. >>> Make sure to release the connection before allocating another one.> >>> java.lang.IllegalStateException: Adapter is detached. >>> at >>> org.apache.http.impl.conn.AbstractPooledConnAdapter.assertAttached(Ab >>> stractPooledConnAdapter.java:90) >>> at >>> org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPool >>> edConnAdapter.java:118) >>> at >>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq >>> uestDirector.java:432) >>> at >>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl >>> ient.java:555) >>> at >>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl >>> ient.java:487) >>> at >>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl >>> ient.java:465) >>> at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:432) >>> at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:405) >>> at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:358) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. >>> java:39) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces >>> sorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:585) >>> at >>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMet >>> hodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229) >>> at >>> org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMeta >>> MethodSite.java:52) >>> at >>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSi >>> teArray.java:43) >>> at >>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa >>> llSite.java:116) >>> at >>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCa >>> llSite.java:132) >>> >>> I'm probably misusing it. Is there a way to accomplish what I'm after? >>> >>> Thanks! >>> >>> --Lee >>> -- >>> View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23268224.html >>> Sent from the groovy - user mailing list archive at Nabble.com. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
I'm having a lot of trouble making AsyncHTTPBuilder work.
When I was using HTTPBuilder, my requests looked like this (from one of the examples): http.URL = 'https://...' def httpResult = http.request(POST,TEXT) { req -> send URLENC, [sSessionID:binding.sessionid, lOrderID:id] response.success = { resp, reader -> assert resp.statusLine.statusCode == 200 def xml = new XmlSlurper().parse(reader) ... } response.'404' = { resp -> println "Retrieval failed: $resp" } I'm trying to turn these into AsyncHTTPBuilder requests, but I keep getting a NullPointerException that I don't understand. Here's what I've got: def httpResult = http.post( path:'/', body:[sUserName:username, sPassword:password], contentType:URLENC) {resp,html -> } while(!httpResult.done) { Thread.sleep(1000) } def xml = httpResult.post() // do something with response The stacktrace originates at the very first line (def httpResult = ...), but here is the rest of it: Exception in thread "Thread-1" java.lang.NullPointerException at groovyx.net.http.URIBuilder.convertToURI(URIBuilder.java:85) at groovyx.net.http.HTTPBuilder$SendDelegate.setPropertiesFromMap(HTTPBuilder.java:831) at groovyx.net.http.HTTPBuilder.post(HTTPBuilder.java:323) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoCachedMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:229) at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128) I need to use a different URL for each request, so I tried to put in a base uri when instantiating AsyncHTTPBuilder: http = new AsyncHTTPBuilder( poolSize : 20, uri : baseUri, contentType : URLENC ) I've tried using path:'/xyz' in the actual call. I've tried uri, url, uri.path, etc. I have yet to find the magic incantation that works. All I want to do is allow multiple HTTP POSTs to occur in parallel. I can't figure out the proper syntax for a post, and I'm not sure why I'm getting an NPE. Nothing seems to be null, but the stacktrace indicates a problem with the URI. Thanks, Lee
|
|
|
Okay, my bad. I was running version 0.4. I just upgraded to the latest 0.5-SNAPSHOT (http-builder-0.5.0-20090502.024916-2-all.zip). So, I'm not getting the same NPE problem as before, but I'm not out of the woods, either.
http = new AsyncHTTPBuilder( poolSize : 20, uri : 'https://oxbranch.optionsxpress.com/', contentType : URLENC ) def httpResult = http.post( path:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource', body:[sUserName:username, sPassword:password], contentType:URLENC) {resp,html -> println "resp = ${resp.dump()}" println "html = $html" } while(!httpResult.done) { println "Outside login..." Thread.sleep(1000) } println httpResult.dump() def xml = httpResult.get() println "httpResult.get() returns $xml" Here's the output: Outside login... Outside login... Outside login... Outside login... resp = <groovyx.net.http.HttpResponseDecorator@ef4504 headers=null responseBase=org.apache.http.message.BasicHttpResponse@9c176c responseData=null> html = [:] <java.util.concurrent.FutureTask@82d603 sync=java.util.concurrent.FutureTask$Sync@1b09282[State = 2, empty queue]> httpResult.get() returns null Why does get() return null? Am I supposed to be doing something else to retrieve the response? Thanks again! |
Re: Multiple concurrent requests with HTTPBuilder
|
|
Since you are defining your own response handler closure, needs to
return whatever you want Future.get() to return. If you do not pass a closure as the last argument to post(), it will use the default response handler which does just this. So your example should look like this: def httpResult = http.post(...) { resp, html -> println "resp = ${resp.dump()}" println "html = $html" return html } On Sun, May 3, 2009 at 6:54 PM, nabblee <[hidden email]> wrote: > > Okay, my bad. I was running version 0.4. I just upgraded to the latest > 0.5-SNAPSHOT (http-builder-0.5.0-20090502.024916-2-all.zip). So, I'm not > getting the same NPE problem as before, but I'm not out of the woods, > either. > > http = new AsyncHTTPBuilder( > poolSize : 20, > uri : 'https://oxbranch.optionsxpress.com/', > contentType : URLENC ) > > def httpResult = http.post( > path:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource', > body:[sUserName:username, sPassword:password], > contentType:URLENC) {resp,html -> > println "resp = ${resp.dump()}" > println "html = $html" > } > > while(!httpResult.done) { > println "Outside login..." > Thread.sleep(1000) > } > > println httpResult.dump() > def xml = httpResult.get() > println "httpResult.get() returns $xml" > > > > Here's the output: > Outside login... > Outside login... > Outside login... > Outside login... > resp = <groovyx.net.http.HttpResponseDecorator@ef4504 headers=null > responseBase=org.apache.http.message.BasicHttpResponse@9c176c > responseData=null> > html = [:] > <java.util.concurrent.FutureTask@82d603 > sync=java.util.concurrent.FutureTask$Sync@1b09282[State = 2, empty queue]> > httpResult.get() returns null > > Why does get() return null? Am I supposed to be doing something else to > retrieve the response? > > Thanks again! > > -- > View this message in context: http://www.nabble.com/Multiple-concurrent-requests-with-HTTPBuilder-tp23268224p23360779.html > Sent from the groovy - user mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Thanks for your response, Tom, but it's still not working. :-( I'm getting an empty map back...
def httpResult = http.post( uri:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource', body:[sUserName:username, sPassword:password, sSource:binding.sourceid, sSessionID:''], contentType:URLENC ) { resp, xml -> println "resp = ${resp.dump()}" println "xml = $xml" return xml } while(!httpResult.done) { println "Outside login..." Thread.sleep(1000) } println "httpResult: ${httpResult.dump()}" def xml = httpResult.get() println "httpResult.get() returns $xml" Output: Outside login... Outside login... Outside login... Outside login... resp = <groovyx.net.http.HttpResponseDecorator@9c176c headers=null responseBase=org.apache.http.message.BasicHttpResponse@1c5ddd3 responseData=null> xml = [:] httpResult: <java.util.concurrent.FutureTask@82d603 sync=java.util.concurrent.FutureTask$Sync@1b09282[State = 2, empty queue]> httpResult.get() returns [:] Maybe I'm not structuring the arguments to post() correctly? It takes about the amount of time I expect, but I don't see a usable reply. Thanks for any insights, Lee
|
|
|
Okay, I've got it now. The problem was the contentType. It needed to be XML, since that's what comes back. I had URLENC, because that's what I'm sending to them, but I guess maybe that's implied with a POST? Anyway, I guess that's why I was getting an empty map back, since it had no success finding key/value pairs in the XML response. Once I changed it to XML, I started seeing the contents I expected.
Not to look a gift horse in the mouth, because I do appreciate HTTPBuilder, but it really needs more docs in order to be understood. The few examples that are provided don't give enough data points to allow extrapolation. And POST examples are always more useful than GET examples. ;-) Sincere thanks for the library, which I make lots of use of. Cheers, Lee |
|
|
In reply to this post by Thom Nichols
I do have another question about not passing a closure to post(). When I try that, I get an error:
def httpResult = http.post( uri:'https://oxbranch.optionsxpress.com/accountservice/account.asmx/GetOxSessionWithSource', body:[sUserName:username, sPassword:password, sSource:binding.sourceid, sSessionID:''], contentType:XML ) May 3, 2009 7:48:39 PM groovyx.net.http.AsyncHTTPBuilder$1 call SEVERE: Exception thrown from request delegate: groovyx.net.http.HTTPBuilder$RequestConfigDelegate@13a0212 java.lang.IllegalArgumentException: Response closure must accept one or two parameters at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:459) at groovyx.net.http.AsyncHTTPBuilder.doRequestSuper(AsyncHTTPBuilder.java:119) at groovyx.net.http.AsyncHTTPBuilder.access$000(AsyncHTTPBuilder.java:55) at groovyx.net.http.AsyncHTTPBuilder$1.call(AsyncHTTPBuilder.java:103) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) So I seem to have no choice but to create a closure that simply returns the second parameter. Why is that different from what you said would be the default? Thanks, Lee
|
«
Return to Groovy Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |