|
Home > Archive > Unix Programming > November 2006 > HTTP Request from a shell script - Pinging Google
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
HTTP Request from a shell script - Pinging Google
|
|
| mrbowes@gmail.com 2006-11-13, 9:01 am |
| I am trying to automate the process of pinging Google to let them know
that we have updated our Google sitemap. They require an HTTP request
be sent to them.
How can I do this in a shell script?
Thanks so much.
Regards,
Josh
| |
| Mark Warburton 2006-11-13, 9:01 am |
| Hi Josh. What you need is "wget" or "curl". Just use whichever is
available on your system -- both can send HTTP requests from the
command line (or a shell script).
Cheers,
Mark.
| |
| mrbowes@gmail.com 2006-11-13, 9:01 am |
| Mark,
Thanks for the reply.
man wget and man curl return "No manual entry for xxx" so I don't
think I have those modules installed.
Is there any other way? Or should I look at installing these modules?
There's always a fair amount of process getting anything installed
around here, so if there's another way, it would be ideal.
-Josh
| |
| Rainer Temme 2006-11-13, 9:01 am |
| mrbowes@gmail.com wrote:
> man wget and man curl return "No manual entry for xxx" so I don't
> think I have those modules installed.
Well, either install wget, or try with netcat ...
you need to construct the HTTP request yourself then,
nevertheless, it can be done with netcat.
Rainer
| |
| Pascal Bourguignon 2006-11-13, 9:01 am |
| mrbowes@gmail.com writes:
> Mark,
>
> Thanks for the reply.
>
> man wget and man curl return "No manual entry for xxx" so I don't
> think I have those modules installed.
>
> Is there any other way? Or should I look at installing these modules?
> There's always a fair amount of process getting anything installed
> around here, so if there's another way, it would be ideal.
You don't need them you can directly write network stuff with bash:
------------------------------------------------------------------------
#!/bin/bash
url="$1"
case "$url" in
http://*)
true
;;
*)
echo Usage:
echo " $(basename $0) http://... > resource"
exit 1
;;
esac
str="${url:7}"
str="${str/\/*}"
case "$str" in
*@*)
lpart="${str/@*}"
rpart="${str#*@}"
;;
*)
lpart=""
rpart="$str"
;;
esac
case "$lpart" in
*:*)
user=${lpart/:*}
pass=${lpart#*:}
;;
*)
user="$lpart"
pass=""
;;
esac
case "$rpart" in
*:*)
host=${rpart/:*}
port=${rpart#*:}
;;
*)
host="$rpart"
port=80
esac
(
printf "GET %s HTTP/1.0\r\n\r\n" "$url" 1>&3
cat 0<&3
) 3<>/dev/tcp/$host/$port
exit 0
------------------------------------------------------------------------
--
__Pascal Bourguignon__ http://www.informatimago.com/
THIS IS A 100% MATTER PRODUCT: In the unlikely event that this
merchandise should contact antimatter in any form, a catastrophic
explosion will result.
| |
| Logan Shaw 2006-11-13, 9:01 am |
| Pascal Bourguignon wrote:
> mrbowes@gmail.com writes:
>
>
> You don't need them you can directly write network stuff with bash:
>
> ------------------------------------------------------------------------
> #!/bin/bash
> (
> printf "GET %s HTTP/1.0\r\n\r\n" "$url" 1>&3
> cat 0<&3
> ) 3<>/dev/tcp/$host/$port
I checked two Unix systems (OS X and Solaris), and neither of them
seemed to have a /dev/tcp that could be used in this manner. So
I guess that works only on certain systems.
- Logan
| |
| Logan Shaw 2006-11-13, 9:01 am |
| mrbowes@gmail.com wrote:
> man wget and man curl return "No manual entry for xxx" so I don't
> think I have those modules installed.
>
> Is there any other way? Or should I look at installing these modules?
It sounds like you're in an environment where administrative restraints
may be more of a difficulty than technical ones. Which is unfortunate,
because it often makes things into more work than necessary. (And wget
and curl are hardly exotic software, incidentally.)
One other possibility is to do what you want to do with perl. PERL is
often installed on systems as a standard thing, so it might be there.
With any luck, it is, and libwww-perl is also installed. If so, it
should be just a matter of using the HTTP::Request class. There is
some documentation at http://search.cpan.org/~gaas/libwww-perl-5.805/ .
You can test if you have HTTP::Request by doing this:
perl -M'HTTP::Request' -e 1
Another possibility, if you don't have it and can't get it installed,
but you still have perl, is to do it directly in perl. "perldoc perlipc"
should have some documentation about how to use Perl's socket interface.
Doubtless similar scripts could be whipped up fairly easily in other
languages, like Python or Ruby, for example.
- Logan
| |
| Pascal Bourguignon 2006-11-13, 9:01 am |
| Logan Shaw <lshaw-usenet@austin.rr.com> writes:
> Pascal Bourguignon wrote:
>
>
> I checked two Unix systems (OS X and Solaris), and neither of them
> seemed to have a /dev/tcp that could be used in this manner. So
> I guess that works only on certain systems.
It works on systems that have bash (perhaps some modern version of bash).
/dev/tcp/$h/$p is a virtual device simulated by bash itself.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -- Georges W. Bush
| |
| Theo v. Werkhoven 2006-11-13, 9:01 am |
| The carbonbased lifeform Pascal Bourguignon inspired comp.unix.programmer with:
> Logan Shaw <lshaw-usenet@austin.rr.com> writes:
>
>
> It works on systems that have bash (perhaps some modern version of bash).
>
> /dev/tcp/$h/$p is a virtual device simulated by bash itself.
I have no idea how you got that notion, but Bash knows nothing about
devices (not even when run with UID 0 permissions), only about files.
Use 'netcat' for those applications.
Theo
--
theo at van-werkhoven.nl ICQ:277217131 SuSE Linux
linuxcounter.org: 99872 Jabber:muadib at jabber.xs4all.nl AMD XP3000+ 1024MB
"ik _heb_ niets tegen Microsoft, ik heb iets tegen
de uitwassen *van* Microsoft"
| |
| Stefaan A Eeckels 2006-11-13, 9:01 am |
| On Sat, 11 Nov 2006 07:17:01 +0100
Pascal Bourguignon <pjb@informatimago.com> wrote:
> /dev/tcp/$h/$p is a virtual device simulated by bash itself.
Yikes, what an abomination.
First we had Emacs with OS pretensions, now we have bash.
--
Stefaan A Eeckels
--
Sometimes I wonder whether the world is run by smart people who are
putting us on or by imbeciles who really mean it. --Mark Twain
| |
| Stefaan A Eeckels 2006-11-13, 9:01 am |
| On Sat, 11 Nov 2006 14:12:55 +0100
"Theo v. Werkhoven" <theo@van-werkhoven.nl.invalid> wrote:
>
> I have no idea how you got that notion, but Bash knows nothing about
> devices (not even when run with UID 0 permissions), only about files.
> Use 'netcat' for those applications.
Well, Pascal's script works on my Solaris system (which happens to
have /dev/tcp) with bash-3.1. Bash interprets
/dev/tcp/www.sun.com/80
specially, even when the device file /dev/tcp is present.
From the bash man page:
Bash handles several filenames specially when they are used
in redirections, as described in the following table:
/dev/fd/fd
If fd is a valid integer, file descriptor fd is
duplicated.
/dev/stdin
File descriptor 0 is duplicated.
/dev/stdout
File descriptor 1 is duplicated.
/dev/stderr
File descriptor 2 is duplicated.
/dev/tcp/host/port
If host is a valid hostname or Internet address,
and port is an integer port number or service
name, bash attempts to open a TCP connection to
the corresponding socket.
/dev/udp/host/port
If host is a valid hostname or Internet address,
and port is an integer port number or service
name, bash attempts to open a UDP connection to
the corresponding socket.
A failure to open or create a file causes the redirection to
fail.
Scary.
--
Stefaan A Eeckels
--
"The most common of all follies is to believe passionately in
the palpably not true. It is the chief occupation of mankind."
--H. L. Mencken
| |
| racaille 2006-11-13, 9:01 am |
| Logan Shaw wrote:
> Pascal Bourguignon wrote:
[vbcol=seagreen]
> I checked two Unix systems (OS X and Solaris), and neither of them
> seemed to have a /dev/tcp that could be used in this manner. So
> I guess that works only on certain systems.
bash emulates itself /dev/tcp/* - of course that directory doesn't
exist,
since it's only a bash artifact.
a really bad idea btw - the user may be tempted to pass
/dev/tcp/host/port
as arguments to other programs, etc.
| |
| Theo v. Werkhoven 2006-11-13, 9:01 am |
| The carbonbased lifeform Stefaan A Eeckels inspired comp.unix.programmer with:
> On Sat, 11 Nov 2006 14:12:55 +0100
> "Theo v. Werkhoven" <theo@van-werkhoven.nl.invalid> wrote:
>
>
>
> Well, Pascal's script works on my Solaris system (which happens to
> have /dev/tcp) with bash-3.1. Bash interprets
>
> /dev/tcp/www.sun.com/80
>
> specially, even when the device file /dev/tcp is present.
>
> From the bash man page:
[..]
>
> Scary.
Ack.
I never took the time to read that bit carefully. To my surprise it does
actually work too.
I'd still rather use netcat though.
Tnx,
Theo
--
theo at van-werkhoven.nl ICQ:277217131 SuSE Linux
linuxcounter.org: 99872 Jabber:muadib at jabber.xs4all.nl AMD XP3000+ 1024MB
"ik _heb_ niets tegen Microsoft, ik heb iets tegen
de uitwassen *van* Microsoft"
| |
| Bill Marcum 2006-11-16, 7:20 pm |
| On Sat, 11 Nov 2006 07:17:01 +0100, Pascal Bourguignon
<pjb@informatimago.com> wrote:
> Logan Shaw <lshaw-usenet@austin.rr.com> writes:
>
>
> It works on systems that have bash (perhaps some modern version of bash).
>
/dev/tcp is an optional feature that is not always compiled into bash.
--
And if sometime, somewhere, someone asketh thee,
"Who kilt thee?", tell them it 'twas the Doones of Bagworthy!
| |
| rk.girish@gmail.com 2006-11-28, 1:33 am |
| Hi Josh
I am working AIX. Here I have to send http request google
Regards
Girish
On Nov 9, 2:24 pm, mrbo...@gmail.com wrote:
> I am trying to automate the process of pinging Google to let them know
> that we have updated our Google sitemap. They require an HTTP request
> be sent to them.
>
> How can I do this in a shell script?
>
> Thanks so much.
>
> Regards,
>
> Josh
|
|
|
|
|