Perlbal - Sending a HTTP request from within a plugin

Author

Sending a HTTP request from within a plugin

Robby Dermody

2007-07-25, 1:11 pm

Hi guys,

I'm writing a perlbal plugin where when I receive a certain event
(specifically, end_proxy_request), I need to make a HTTP request out to an
external resource. Basically this is because the I have a reproxy setup
going with perlbal and need to keep state in my environment around when a
particular resource request starts and stops for each given client. The HTTP
request will be very simple with only 1 or 2 GET parameters (no POST data),
and I will just need to receive a single equally basic HTTP response, which
should be a 200 OK.

I was looking at the source and it looks like I could use Danga::Socket
directly for this, however given the fact that these HTTP requests could be
going out quite frequently (and all to the same host), I'd probably want to
make the requests on a persistent connection (using KeepAlives).

It's been a long time since I've worked in PERL though, so I was wondering
if there was a built in mechanism in perlbal or somewhere else I could use
to make these requests, or some code I could start with to implement this.

Thanks,

Robby

Brad Fitzpatrick

2007-07-25, 7:12 pm

There's no Danga::Socket-compatible HTTP client, and our POE/Danga:Socket
integration work isn't yet done, so at this point the only easy (and
recommended) option for you is to use Gearman. You can use
Gearman::Client::Async to use external gearmand servers/workers (which can
then do HTTP requests if needed), or you can embed a Gearman::Server in
the perlbal server process and have child processes under the perlbal
doing their work speaking Gearman protocol over pipes between parent/child
processes. Either way, you get persistent connections too, so rate of
requests isn't a problem.

On Wed, 25 Jul 2007, Robby Dermody wrote:

> Hi guys,
>
> I'm writing a perlbal plugin where when I receive a certain event
> (specifically, end_proxy_request), I need to make a HTTP request out to an
> external resource. Basically this is because the I have a reproxy setup
> going with perlbal and need to keep state in my environment around when a
> particular resource request starts and stops for each given client. The HTTP
> request will be very simple with only 1 or 2 GET parameters (no POST data),
> and I will just need to receive a single equally basic HTTP response, which
> should be a 200 OK.
>
> I was looking at the source and it looks like I could use Danga::Socket
> directly for this, however given the fact that these HTTP requests could be
> going out quite frequently (and all to the same host), I'd probably want to
> make the requests on a persistent connection (using KeepAlives).
>
> It's been a long time since I've worked in PERL though, so I was wondering
> if there was a built in mechanism in perlbal or somewhere else I could use
> to make these requests, or some code I could start with to implement this.
>
> Thanks,
>
> Robby
>
>
>

Robby Dermody

2007-07-26, 1:11 am

Thanks Brad,

What may simplify things is that perlbal and the component that would be
getting the HTTP request are always going to be on the same box (and this is
the only thing I will need to speak HTTP with). Since that is the case, I
have other possibilities to make a request besides sockets/HTTP such as
named pipes. Really all I need to do is to pass some data from perlbal to my
app during an end_proxy_request situation. Does the change the nature of the
beast any, or is Gearman still the recommended way to go?

Robby

-----Original Message-----
From: Brad Fitzpatrick [mailto:brad@danga.com]
Sent: Wednesday, July 25, 2007 5:34 PM
To: Robby Dermody
Cc: perlbal@lists.danga.com
Subject: Re: Sending a HTTP request from within a plugin

There's no Danga::Socket-compatible HTTP client, and our POE/Danga:Socket
integration work isn't yet done, so at this point the only easy (and
recommended) option for you is to use Gearman. You can use
Gearman::Client::Async to use external gearmand servers/workers (which can
then do HTTP requests if needed), or you can embed a Gearman::Server in
the perlbal server process and have child processes under the perlbal
doing their work speaking Gearman protocol over pipes between parent/child
processes. Either way, you get persistent connections too, so rate of
requests isn't a problem.

On Wed, 25 Jul 2007, Robby Dermody wrote:

> Hi guys,
>
> I'm writing a perlbal plugin where when I receive a certain event
> (specifically, end_proxy_request), I need to make a HTTP request out to an
> external resource. Basically this is because the I have a reproxy setup
> going with perlbal and need to keep state in my environment around when a
> particular resource request starts and stops for each given client. The
HTTP
> request will be very simple with only 1 or 2 GET parameters (no POST
data),
> and I will just need to receive a single equally basic HTTP response,
which
> should be a 200 OK.
>
> I was looking at the source and it looks like I could use Danga::Socket
> directly for this, however given the fact that these HTTP requests could
be
> going out quite frequently (and all to the same host), I'd probably want
to
> make the requests on a persistent connection (using KeepAlives).
>
> It's been a long time since I've worked in PERL though, so I was wondering
> if there was a built in mechanism in perlbal or somewhere else I could use
> to make these requests, or some code I could start with to implement this.
>
> Thanks,
>
> Robby
>
>
>

Jeremy James

2007-07-26, 7:11 am

Robby Dermody wrote:
> I was looking at the source and it looks like I could use Danga::Socket
> directly for this, however given the fact that these HTTP requests could be
> going out quite frequently (and all to the same host), I'd probably want to
> make the requests on a persistent connection (using KeepAlives).

Depending on the precise application design and what your error
conditions are (what happens if the other host doesn't return a 200?),
I'd personally consider sending UDP packets instead - should be faster
to send, and less coding required! (*)

-jeremy

---
(*) My PERL was never very good, but perhaps something like - this
probably uses both IO::Socket and Danga::Socket:

$udp_sock ||= IO::Socket::INET->new(Proto=>'udp');
$udp_sock->send("SOME DATA", 0, \
Socket::sockaddr_in("1234",Socket::inet_aton("192.168.5.64")));
---

Brad Fitzpatrick

2007-07-27, 7:11 pm

Gearman still easiest, as things stand now.

On Wed, 25 Jul 2007, Robby Dermody wrote:

> Thanks Brad,
>
> What may simplify things is that perlbal and the component that would be
> getting the HTTP request are always going to be on the same box (and this is
> the only thing I will need to speak HTTP with). Since that is the case, I
> have other possibilities to make a request besides sockets/HTTP such as
> named pipes. Really all I need to do is to pass some data from perlbal to my
> app during an end_proxy_request situation. Does the change the nature of the
> beast any, or is Gearman still the recommended way to go?
>
> Robby
>
> -----Original Message-----
> From: Brad Fitzpatrick [mailto:brad@danga.com]
> Sent: Wednesday, July 25, 2007 5:34 PM
> To: Robby Dermody
> Cc: perlbal@lists.danga.com
> Subject: Re: Sending a HTTP request from within a plugin
>
> There's no Danga::Socket-compatible HTTP client, and our POE/Danga:Socket
> integration work isn't yet done, so at this point the only easy (and
> recommended) option for you is to use Gearman. You can use
> Gearman::Client::Async to use external gearmand servers/workers (which can
> then do HTTP requests if needed), or you can embed a Gearman::Server in
> the perlbal server process and have child processes under the perlbal
> doing their work speaking Gearman protocol over pipes between parent/child
> processes. Either way, you get persistent connections too, so rate of
> requests isn't a problem.
>
>
> On Wed, 25 Jul 2007, Robby Dermody wrote:
>
> HTTP
> data),
> which
> be
> to
>
>