|
Home > Archive > Unix Programming > February 2006 > File that behaves like a socket
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
File that behaves like a socket
|
|
| Phil Endecott 2006-02-19, 8:29 am |
| Dear Experts,
I am looking for a good way to make something that looks like a file to
the reader, but behaves more like a socket to the writer.
More specifically, I want just about any program to be able to open and
read from it without realising that it is special. When they open it, a
server program starts to supply the data. More than one connection
should be possible simultaneously.
This has similarities to the way that character-special devices (serial
ports etc) work: they can be read like normal files, but their "server
process" has to be in the kernel.
I had been hoping that Unix-domain sockets would be a solution, but as
far as I can see a Unix-domain socket can't be opened like a regular
file with open(), but rather needs to be opened much like an IP socket
with socket() and bind(). (Is this true?)
Here's the application: I have a hardware MPEG encoder that can supply
data to no more than one process at a time through a
character-special-device, /dev/video0. I have written a simple
distributor program that reads this and serves it via IP sockets to as
many programs as want to listen. Unfortunately these programs generally
want to be passed a filename to read from. The best I have been able to
do so far is to use a named pipe, but that is not ideal since the server
can't listen() for the named pipe being opened.
Any ideas?
I'm doing this on Linux, but a generic solution would be best.
Cheers,
--Phil.
| |
| Nils O. Selåsdal 2006-02-19, 8:29 am |
| Phil Endecott wrote:
> Dear Experts,
>
> I am looking for a good way to make something that looks like a file to
> the reader, but behaves more like a socket to the writer.
>
> More specifically, I want just about any program to be able to open and
> read from it without realising that it is special. When they open it, a
> server program starts to supply the data. More than one connection
> should be possible simultaneously.
>
> This has similarities to the way that character-special devices (serial
> ports etc) work: they can be read like normal files, but their "server
> process" has to be in the kernel.
>
> I had been hoping that Unix-domain sockets would be a solution, but as
> far as I can see a Unix-domain socket can't be opened like a regular
> file with open(), but rather needs to be opened much like an IP socket
> with socket() and bind(). (Is this true?)
Yes.
> Here's the application: I have a hardware MPEG encoder that can supply
> data to no more than one process at a time through a
> character-special-device, /dev/video0. I have written a simple
> distributor program that reads this and serves it via IP sockets to as
> many programs as want to listen. Unfortunately these programs generally
> want to be passed a filename to read from. The best I have been able to
> do so far is to use a named pipe, but that is not ideal since the server
> can't listen() for the named pipe being opened.
>
> Any ideas?
Try a named pipe (mkfif0)
| |
| Barry Margolin 2006-02-19, 8:29 am |
| In article <43f739f9$1@news.wineasy.se>,
"Nils O. Selåsdal" <noselasd@asgaard.removethis.homelinux.org> wrote:
> Phil Endecott wrote:
> The best I have been able to
>
> Try a named pipe (mkfif0)
Huh? Did you read the sentence quoted above?
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
| |
| Heiner Steven 2006-02-19, 8:29 am |
| Phil Endecott wrote:
[...]
> I am looking for a good way to make something that looks like a file to
> the reader, but behaves more like a socket to the writer.
>
> More specifically, I want just about any program to be able to open and
> read from it without realising that it is special. When they open it, a
> server program starts to supply the data. More than one connection
> should be possible simultaneously.
>
> This has similarities to the way that character-special devices (serial
> ports etc) work: they can be read like normal files, but their "server
> process" has to be in the kernel.
>
> I had been hoping that Unix-domain sockets would be a solution, but as
> far as I can see a Unix-domain socket can't be opened like a regular
> file with open(), but rather needs to be opened much like an IP socket
> with socket() and bind(). (Is this true?)
>
> Here's the application: I have a hardware MPEG encoder that can supply
> data to no more than one process at a time through a
> character-special-device, /dev/video0. I have written a simple
> distributor program that reads this and serves it via IP sockets to as
> many programs as want to listen. Unfortunately these programs generally
> want to be passed a filename to read from. The best I have been able to
> do so far is to use a named pipe, but that is not ideal since the server
> can't listen() for the named pipe being opened.
You cannot listen(2) on the named pipe, but the kernel does
the synchronization for you: you could just wrte(2) to the
pipe, and your program would block until the first reader read
the data.
But there's still the question of how to serve multiple
streams simultaneously. Using a named pipe you'll only be
able to communicate with one client on the other side. As far
as I know there's no way of using a single named pipe to "multicast"
data to multiple processes *without them having to be changed*.
If your system supported STREAMS (e.g. Solaris; Linux does not
support them), you could also use the fattach(3) library call,
which also associates a path name with a file descriptor.
Maybe there even is a way to create several endpoints for different
clients in this case.
Heiner
--
___ _
/ __| |_ _____ _____ _ _ Heiner STEVEN <heiner.steven@nexgo.de>
\__ \ _/ -_) V / -_) ' \ Shell Script Programmers: visit
|___/\__\___|\_/\___|_||_| http://www.shelldorado.com/
| |
| Alex Fraser 2006-02-19, 8:29 am |
| "Phil Endecott" <phil_nn05@chezphil.org> wrote in message
news:i6EJf.1431$58.1170@newsfe3-win.ntli.net...
> I am looking for a good way to make something that looks like a file to
> the reader, but behaves more like a socket to the writer.
>
> More specifically, I want just about any program to be able to open and
> read from it without realising that it is special. When they open it, a
> server program starts to supply the data. More than one connection
> should be possible simultaneously.
[snip]
> Here's the application: I have a hardware MPEG encoder that can supply
> data to no more than one process at a time through a
> character-special-device, /dev/video0. I have written a simple
> distributor program that reads this and serves it via IP sockets to as
> many programs as want to listen. Unfortunately these programs generally
> want to be passed a filename to read from. The best I have been able to
> do so far is to use a named pipe, but that is not ideal since the server
> can't listen() for the named pipe being opened.
You can fix the latter by having a thread or process block on open() on the
named pipe. When it returns, communicate the fact to the distributor, which
can then (if necessary, open and) write to the named pipe. (I've glossed
over some details I have in mind, mainly because if you don't like the idea
it would be a waste of time going into them.)
A similar approach would be to have the threads or processes block on open()
as above, but then connect to your distributor and relay data through the
named pipe. This is simpler than the above (uses your existing distributor,
and requires no special IPC) but has the obvious disadvantage of a
performance hit from passing the data through the extra thread or process.
Both of these approaches need one named pipe per potential data sink. I
don't know of any way to meet the "can open with open()" requirement without
using named pipes. You could unlink() then mkfifo() for each "connection";
AFAICS you can't avoid the window where the named pipe doesn't exist.
Another snag with named pipes is the unspecified chaos that results if
multiple processes read from the same one with no coordination.
Alex
| |
| Nils O. Selåsdal 2006-02-19, 8:29 am |
| Barry Margolin wrote:
> In article <43f739f9$1@news.wineasy.se>,
> "Nils O. Selåsdal" <noselasd@asgaard.removethis.homelinux.org> wrote:
>
>
> Huh? Did you read the sentence quoted above?
*sigh* Sorry.
Nevertheless - several pipes and a multiplexor (select or poll) might
do.
There are linux specific solutions though, implement a fileserver using
FUSE or V9FS.
| |
| Ben Bacarisse 2006-02-19, 8:29 am |
| On Sat, 18 Feb 2006 12:07:42 +0000, Phil Endecott wrote:
> Here's the application: I have a hardware MPEG encoder that can supply
> data to no more than one process at a time through a
> character-special-device, /dev/video0. I have written a simple
> distributor program that reads this and serves it via IP sockets to as
> many programs as want to listen. Unfortunately these programs generally
> want to be passed a filename to read from. The best I have been able to
> do so far is to use a named pipe, but that is not ideal since the server
> can't listen() for the named pipe being opened.
If the protocol used for the data is TCP or UDP, you could use a program
like nc (netcat) at the client end to open the stream and send it
synchronously to stdout or to a named temporary fifo whose name is given
to the client program.
--
Ben.
| |
| Phil Endecott 2006-02-19, 10:32 am |
| > I am looking for a good way to make something that looks like a file to
> the reader, but behaves more like a socket to the writer.
Thanks for the suggestions. I am currently using "netcat | prog" when
the program can accept its input on stdin, and "mknod p p; netcat > p &&
prog p" when it can't. I may investigate automatic creation of named
pipes, with unlinking once they are open at both ends, if that seems
necessary; Does anyone know at what point the writer will block if there
is no reader; on open(), on write(), or only once the kernel buffer is full?
One other idea that occured to me was to use an LD_PRELOAD trick to make
it possible for a reader to open() a unix-domain socket as if it were a
regular file, i.e. intercept open() calls and substitute socket();bind()
(or whatever it is) with the path is "special".
Of course if we were all using Plan 9 this would be easy!
Cheers,
--Phil.
| |
| Alex Fraser 2006-02-19, 5:50 pm |
| "Phil Endecott" <phil_nn05@chezphil.org> wrote in message
news:u%%Jf.2882$58.1025@newsfe3-win.ntli.net...
>
> Thanks for the suggestions. I am currently using "netcat | prog" when
> the program can accept its input on stdin, and "mknod p p; netcat > p &&
> prog p" when it can't. I may investigate automatic creation of named
> pipes, with unlinking once they are open at both ends, if that seems
> necessary; Does anyone know at what point the writer will block if there
> is no reader; on open(), on write(), or only once the kernel buffer is
> full?
On open(), unless O_NONBLOCK is specified (in which case the call fails).
> One other idea that occured to me was to use an LD_PRELOAD trick to make
> it possible for a reader to open() a unix-domain socket as if it were a
> regular file, i.e. intercept open() calls and substitute socket();bind()
> (or whatever it is) with the path is "special".
Perhaps first call the real open(), and if that fails then maybe (depending
on the arguments) try socket()/connect() to the same name. Then the actual
path won't affect the LD_PRELOADed .so.
Alex
|
|
|
|
|