|
Home > Archive > Unix Programming > July 2005 > interconnection between two processes
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
interconnection between two processes
|
|
| Roman Mashak 2005-07-18, 7:51 am |
| Hello, All!
I'm considering now what mechanisms to use for interconnection between 2
processes: one is daemon, the other one is CGI-scipt. Both are running on
the same host.
What would be more comfortable and efficient to use:
1) pipes/FIFO
2) UNIX sockets
3) something else..
Thanks.
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
| |
| Ulrich Hobelmann 2005-07-18, 5:53 pm |
| Roman Mashak wrote:
> Hello, All!
>
> I'm considering now what mechanisms to use for interconnection between 2
> processes: one is daemon, the other one is CGI-scipt. Both are running on
> the same host.
>
> What would be more comfortable and efficient to use:
> 1) pipes/FIFO
> 2) UNIX sockets
> 3) something else..
Really depends how they're supposed to communicate. If one
process creates the other one, then pipes are probably the easiest
(create pipe, then fork and exec). If you need two-way
communication, you need two pipe()s.
Closing the pipe can be the signal for the other process to exit.
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
| |
| Gordon Burditt 2005-07-18, 5:53 pm |
| >I'm considering now what mechanisms to use for interconnection between 2
>processes: one is daemon, the other one is CGI-scipt. Both are running on
>the same host.
>
>What would be more comfortable and efficient to use:
>1) pipes/FIFO
>2) UNIX sockets
>3) something else..
Are the processes related (one invokes the other)? Probably not.
That lets pipes out. FIFOs are a bit clumsy and are not guaranteed
to provide two-way communication.
UNIX-domain sockets or Internet-domain sockets would work. If
you are sure the server will stay on the same machine as the CGI
client, use UNIX-domain sockets, as they are generally more efficient.
Gordon L. Burditt
| |
|
|
The simplest way to communicate values between processes is via a file. One
process writes to the file and another process reads from it. This is a
surprisingly simple mechanism which has several advantages:
a.. It will allow any pair of processes with access permission to the file
to use it for communication.
b.. It can allow the transfer of very large amounts of data between
processes.
Despite these advantages, the use of files as a major IPC mechanism has two
big drawbacks:
a.. In order to make sure that the reader has the opportunity to see all
the data sent to it, the writer must only add new data to the end of the
file. This means the file can grow very large for long lived processes.
b.. If the reader performs its task faster than the writer then it will
frequently catch up with the writer and have nothing to read. This means
that the reader will constantly be reading end-of-file and not be able to
tell whether this is because the writer has finished and closed the file, or
the writer is still computing values to pass to the reader and there will be
more to follow.
The pipe overcomes both the problems of using files. First, a pipe is a
fixed-size buffer (4096 bytes in Linux) so that its size can't grow
unchecked like a file.The consequence of using a single fixed-sized buffer
is that, on writing, the pipe can become full. When this happens subsequent
write() calls to the pipe will block by default, waiting for some data to be
read to make enough room for the write() to take place. Notice that reading
data from a pipe is a one-off operation and that the data, once read, is
discarded from the pipe to free up the space for more data to be written.
Second, it is also possible, if the reader is working faster than the
writer, for the pipe to become empty when all the current data has been
read. When this happens a subsequent read() call will block by default,
waiting for more data to be written. This solves the read() returning
end-of-file problem.
Anonymous pipes overcome the main problems with using files for
inter-process communication. However, pipes have one drawback not shared by
files they can only be used by processes that have a common ancestor which
created the pipe for them and passed on its file descriptors to them.
The use of pipes and FIFOs still suffers from one disadvantage and that is
the amount of data copying and processor context switching that goes on to
transfer the data.
Shared memory overcomes this problem by making the same block of memory
simultaneously visible in the address spaces of two or more processes.
"Roman Mashak" <mrv@tusur.ru> wrote in message
news:dbg6oo$2rq9$1@relay.tomsk.ru...
> Hello, All!
>
> I'm considering now what mechanisms to use for interconnection between 2
> processes: one is daemon, the other one is CGI-scipt. Both are running on
> the same host.
>
> What would be more comfortable and efficient to use:
> 1) pipes/FIFO
> 2) UNIX sockets
> 3) something else..
>
> Thanks.
>
> With best regards, Roman Mashak. E-mail: mrv@tusur.ru
>
>
| |
| Konstantin Sorokin 2005-07-18, 5:53 pm |
| Ulrich Hobelmann <u.hobelmann@web.de> wrote:
> Roman Mashak wrote:
>
> Really depends how they're supposed to communicate. If one
> process creates the other one, then pipes are probably the easiest
> (create pipe, then fork and exec). If you need two-way
> communication, you need two pipe()s.
socketpair(2) is more apropriate for this task
>
> Closing the pipe can be the signal for the other process to exit.
>
--
Konstantin Sorokin
| |
| Ulrich Hobelmann 2005-07-18, 5:53 pm |
| Konstantin Sorokin wrote:
>
> socketpair(2) is more apropriate for this task
Hm, I've had my problems with that manpage, and for that reason
always chose pipe()s for my own use.
"The socketpair() call creates an unnamed pair of connected
sockets in the
specified domain d, of the specified type, and using the
optionally spec-
ified protocol. The descriptors used in referencing the new
sockets are
returned in sv[0] and sv[1]. The two sockets are
indistinguishable."
Hm, what do I care, in what domain the sockets exist; I only want
to fork anyway...
And why are the sockets indistinguishable? Does that mean I can
communicate over them like over a pipe, or not?
"BUGS
This call is currently implemented only for the UNIX domain."
Well, ok, so maybe it's like a pipe. But still, the call requires
for too much information for my taste, so I always went with
pipe() in the past.
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
| |
| Konstantin Sorokin 2005-07-18, 5:53 pm |
| Ulrich Hobelmann <u.hobelmann@web.de> wrote:
> Konstantin Sorokin wrote:
>
> Hm, I've had my problems with that manpage, and for that reason
> always chose pipe()s for my own use.
>
> "The socketpair() call creates an unnamed pair of connected
> sockets in the
> specified domain d, of the specified type, and using the
> optionally spec-
> ified protocol. The descriptors used in referencing the new
> sockets are
> returned in sv[0] and sv[1]. The two sockets are
> indistinguishable."
>
> Hm, what do I care, in what domain the sockets exist; I only want
> to fork anyway...
No problem. Just do something like (modulo error checking):
int s[2];
socketpair(AF_LOCAL, SOCK_STREAM, 0, s);
then fork() and you now you have bidirectional communication channel
between parent and child.
> And why are the sockets indistinguishable? Does that mean I can
> communicate over them like over a pipe, or not?
yes
>
> "BUGS
> This call is currently implemented only for the UNIX domain."
>
> Well, ok, so maybe it's like a pipe. But still, the call requires
> for too much information for my taste, so I always went with
> pipe() in the past.
It seems to me more convenient then you need bidirectional channel.
--
Konstantin Sorokin
| |
| Roman Mashak 2005-07-19, 2:48 am |
| Thanks to everyone for answers.
Here I specify some details.
The processes are supposed to communicate like this:
1) process 'A' (CGI) gets data request from client and sends these data
(about 80 bytes max.) to process 'B'
2) process 'B' receives it, takes some time processing and sends some
response code to process 'A' (like SUCESS or FAILED)
3) depending on process 'B' response, 'A' will make further decision
Issue I concern about: I didn't decide yet whether to use 'wget' or 'curl'
on client side (it will be embedded environment) to generate HTTP request or
implement it by myselft, because client application should wait for response
from CGI and I assume 'wget' or any other software kind of that can't do it
for me (if I'm wrong, please correct me).
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
| |
| Pascal Bourguignon 2005-07-19, 2:48 am |
| "Roman Mashak" <mrv@tusur.ru> writes:
> Thanks to everyone for answers.
>
> Here I specify some details.
> The processes are supposed to communicate like this:
> 1) process 'A' (CGI) gets data request from client and sends these data
> (about 80 bytes max.) to process 'B'
> 2) process 'B' receives it, takes some time processing and sends some
> response code to process 'A' (like SUCESS or FAILED)
> 3) depending on process 'B' response, 'A' will make further decision
The HTTP server will fork a new CGI process for each request.
I assume you want to kee the server process 'B' running continuously.
So this eliminates pipes.
Remains:
- named pipes
- SYSV IPC (message queues)
- unix sockets
- network sockets
Named pipes and unix sockets are often implemented the same way by the kernel.
Named pipes, unix sockets, or message queues can be selected when both
the CGI process 'A' and the server process 'B' are running on the same
kernel.
Message queues might be faster, but they're more limited resourcewise
than the others. For small requests and answers, they may be
interesting, but they'd be more inconvenient if the server downloads
web pages (10K-100K).
In the case of named pipes, bi-directional communication is more
complicated (you'd need two pipes, more if there are several CGI
processes 'A').
Network sockets are nice when you want to be able to migrate the
server process to another computer than the HTTP server.
I would use network sockets, unless very high performance was needed
(in which case SYSV IPC or shared memory would be in order).
> Issue I concern about : I didn't decide yet whether to use 'wget' or 'curl'
> on client side (it will be embedded environment) to generate HTTP request or
> implement it by myselft, because client application should wait for response
> from CGI and I assume 'wget' or any other software kind of that can't do it
> for me (if I'm wrong, please correct me).
There are (user settable) time outs, but wget can perfectly manage
with server delays. How else could it work?
--
__Pascal Bourguignon__ http://www.informatimago.com/
Cats meow out of angst
"Thumbs! If only we had thumbs!
We could break so much!"
| |
| David Schwartz 2005-07-19, 2:48 am |
|
"Roman Mashak" <mrv@tusur.ru> wrote in message
news:dbhp9v$tp0$1@relay.tomsk.ru...
> Issue I concern about : I didn't decide yet whether to use 'wget' or 'curl'
> on client side (it will be embedded environment) to generate HTTP request
> or implement it by myselft, because client application should wait for
> response from CGI and I assume 'wget' or any other software kind of that
> can't do it for me (if I'm wrong, please correct me).
All of these programs wait for the web server to send its reply to their
query or they would have no point.
DS
| |
| Roman Mashak 2005-07-19, 2:48 am |
| Hello, Pascal!
You wrote on Tue, 19 Jul 2005 05:31:44 +0200:
PB> The HTTP server will fork a new CGI process for each request.
Then it might be efficient to fork() process for every request. In my case
probably 'A' and 'B' processes will run on the same host with the same
kernel, so I don't expect much overhead.
PB> I assume you want to kee the server process 'B' running continuously.
PB> So this eliminates pipes.
PB> Remains:
PB> - named pipes
PB> - SYSV IPC (message queues)
PB> - unix sockets
PB> - network sockets
[skip]
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
| |
| Roman Mashak 2005-07-19, 2:48 am |
| Hello, Pascal!
You wrote on Tue, 19 Jul 2005 05:31:44 +0200:
PB> The HTTP server will fork a new CGI process for each request.
PB> I assume you want to kee the server process 'B' running continuously.
PB> So this eliminates pipes.
PB> Remains:
PB> - named pipes
PB> - SYSV IPC (message queues)
PB> - unix sockets
PB> - network sockets
By the way, what about shared memory? Is it heavy method, what drawbacks?
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
| |
| Ulrich Hobelmann 2005-07-19, 2:48 am |
| Roman Mashak wrote:
> By the way, what about shared memory? Is it heavy method, what drawbacks?
It's not the cleanest thing to do. I consider it a performance
hack, so you don't have to transfer large data blocks over pipes
or sockets.
The main problems are synchronization, and that you have to
manually release the shared memory segments, IIRC (i.e. even when
you program exits, they remain).
For usual data, I'd just try the fork and (pipe or socket)
approach, and when that's getting too slow, you might consider
avoiding a fork each time, and maybe use shared memory.
--
XML is a prime example of retarded innovation.
-- Erik Meijer and Peter Drayton, Microsoft Corporation
| |
| Roman Mashak 2005-07-19, 7:49 am |
| Hello, David!
You wrote on Mon, 18 Jul 2005 20:32:44 -0700:
??>> should wait for response from CGI and I assume 'wget' or any other
??>> software kind of that can't do it for me (if I'm wrong, please correct
??>> me).
DS> All of these programs wait for the web server to send its reply to
DS> their query or they would have no point.
The problem is client application (e.g. 'wet' or 'curl') will receive some
response code generated by CGI. They don't have support to look for some
specific tags in HTML, as far as I know.
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
| |
| Pascal Bourguignon 2005-07-19, 7:49 am |
| "Roman Mashak" <mrv@tusur.ru> writes:
> Hello, David!
> You wrote on Mon, 18 Jul 2005 20:32:44 -0700:
>
> ??>> should wait for response from CGI and I assume 'wget' or any other
> ??>> software kind of that can't do it for me (if I'm wrong, please correct
> ??>> me).
>
> DS> All of these programs wait for the web server to send its reply to
> DS> their query or they would have no point.
> The problem is client application (e.g. 'wet' or 'curl') will receive some
> response code generated by CGI. They don't have support to look for some
> specific tags in HTML, as far as I know.
You could use lynx or lincs to watch for some specific tags in HTML ;-)
Otherwise perhaps you want to run:
URL=http://www.google.com/
SPECIFICTAG=HTML
if ( wget $URL -O - 2>/dev/null | grep -q -s -i "< *${SPECIFICTAG}[^>]*>" )
then
echo Found the specific tag.
else
echo Not found the specific tag.
fi
Well, something more complex than grep would be needed actually, to
handle HTML complexities...
--
__Pascal_Bourguignon__ _ Software patents are endangering
() ASCII ribbon against html email (o_ the computer industry all around
/\ 1962:DO20I=1.100 //\ the world http://lpf.ai.mit.edu/
2001:my($f)=`fortune`; V_/ http://petition.eurolinux.org/
| |
| David Schwartz 2005-07-19, 7:49 am |
|
"Roman Mashak" <mrv@tusur.ru> wrote in message
news:dbiems$1d46$1@relay.tomsk.ru...
> Hello, David!
> You wrote on Mon, 18 Jul 2005 20:32:44 -0700:
> ??>> should wait for response from CGI and I assume 'wget' or any other
> ??>> software kind of that can't do it for me (if I'm wrong, please
> correct
> ??>> me).
> DS> All of these programs wait for the web server to send its reply to
> DS> their query or they would have no point.
> The problem is client application (e.g. 'wet' or 'curl') will receive some
> response code generated by CGI. They don't have support to look for some
> specific tags in HTML, as far as I know.
So what? Why does that matter? They will wait for the reply to be
complete.
DS
| |
| Matthias Buelow 2005-07-19, 7:49 am |
| Ulrich Hobelmann <u.hobelmann@web.de> writes:
>Hm, I've had my problems with that manpage, and for that reason always
>chose pipe()s for my own use.
You can use socketpair() like a bidirectional pipe.
For some time (4.4BSD), pipe() was implemented on top of socketpair().
One generally can't rely on a pipe() being bidirectional although it
often appears to be, like on System V with STREAMS-based pipes.
mkb.
| |
| Roman Mashak 2005-07-19, 8:48 pm |
| Hello, Pascal!
You wrote on Tue, 19 Jul 2005 11:55:50 +0200:
PB> Otherwise perhaps you want to run:
PB> URL=http://www.google.com/
PB> SPECIFICTAG=HTML
PB> if ( wget $URL -O - 2>/dev/null | grep -q -s -i "<
PB> *${SPECIFICTAG}[^>]*>" ) then
PB> echo Found the specific tag.
PB> else
PB> echo Not found the specific tag.
PB> fi
Thanks for a tip 
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
|
|
|
|
|