|
Home > Archive > Unix Programming > January 2008 > conveyor non-stop
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
| Andrew Wingorodov 2007-12-28, 1:26 pm |
| Hi there
I wish to make the conveyor, but i can't receive the answer
before I shall close a pipe.
For example:
int
main ()
{
int rpipe [2];
int wpipe [2];
::pipe (rpipe);
::pipe (wpipe);
pid_t _pid;
enum { RD, WR };
if (0 == (_pid=::fork()))
{
::close (rpipe [WR]);
::dup2 (rpipe [RD], STDIN_FILENO); // read rpipe through stdin
::close (rpipe [RD]);
::close (wpipe [RD]);
::dup2 (wpipe [WR], STDOUT_FILENO); // write stdout to wpipe
::close (wpipe [WR]);
::execl ("/usr/bin/tr", "tr", "A-Z", "a-z", 0);
cout << ::strerror (errno) << endl;
return errno;
} else {
::close (rpipe [RD]);
::close (wpipe [WR]);
std::string line;
while ( std::getline (cin, line)) {
::write ( rpipe [WR], line.c_str(), line.size () );
::write ( rpipe [WR], "\n" , sizeof("\n") );
///::close ( rpipe [WR]);
/// With close() I read; without close() I blocked
/// But close() destroy my cool conveyor!
/// How to me to avoid it?
char buf [1024];
int n = ::read ( wpipe [RD], buf, 1024 );
buf [n] = '\0';
cout << buf << endl;
}
}
}
--
www.andr.mobi
| |
| Syren Baran 2007-12-28, 1:26 pm |
| Andrew Wingorodov schrieb:
> Hi there
>
> I wish to make the conveyor, but i can't receive the answer
> before I shall close a pipe.
>
> ::execl ("/usr/bin/tr", "tr", "A-Z", "a-z", 0);
>
Not 100% sure here. But doesnt tr read until it gets en EOF?
close() triggers that EOF. Basicly you get the same behavior if you
create a pipe in a shell, e.g.
$mknod testpipe p
then type
$tr "A-Z" "a-z" <pipe
and in a different shell type
$echo Whatsoever>testpipe
Sending further lines into testpipe wont get read by tr.
| |
| Syren Baran 2007-12-28, 1:26 pm |
| Syren Baran schrieb:
> Andrew Wingorodov schrieb:
> Not 100% sure here. But doesnt tr read until it gets en EOF?
Ok, send that comment to /dev/null.
A simple
$tr "A-Z" "a-z"
works as expected, reading one line from from stdin at a time.
| |
| fjblurt@yahoo.com 2007-12-28, 7:23 pm |
| On Dec 28, 8:52 am, Syren Baran <sy...@gmx.de> wrote:
> Syren Baran schrieb:> Andrew Wingorodov schrieb:
>
>
>
>
> Ok, send that comment to /dev/null.
> A simple
> $tr "A-Z" "a-z"
> works as expected, reading one line from from stdin at a time.
Most programs use line buffering when reading/writing to a terminal,
but full buffering when reading/writing anything else. This is the
default behavior of the C library's stdio functions. tr apparently
does the same. So when you talk to it through a pipe, it will not
write its output back to the pipe until its buffer is full, usually
several Kbytes. This of course is not what you want.
Some programs have an option to force line buffering, though I kind of
doubt tr would include this. Another thing you can do is to use a
pseudo-tty instead of a pipe, which emulates a terminal and should get
you back to line buffering. This is a little more work, but you can
probably find some examples online.
In this case, you could also close the pipe after the first line, and
start over with a new instance of tr for each new one.
| |
| Andrew Wingorodov 2007-12-29, 1:56 am |
| fjblurt@yahoo.com wrote:
> but full buffering when reading/writing anything else. This is the
> default behavior of the C library's stdio functions. tr apparently
> does the same. So when you talk to it through a pipe, it will not
> write its output back to the pipe until its buffer is full, usually
> several Kbytes. This of course is not what you want.
i am disappointed
well-known UNIX-concept as is 'conveyor' actually is a myth
fork demands very much greater resources, it cannot be used
for a plenty of files
why to not use special symbol NOF (next of file  ,
which finishes a file, but doesnt close IO and begins following?!
--
www.andr.mobi
| |
| Syren Baran 2007-12-30, 7:37 am |
| Andrew Wingorodov schrieb:
> fjblurt@yahoo.com wrote:
>
> i am disappointed
> well-known UNIX-concept as is 'conveyor' actually is a myth
> fork demands very much greater resources, it cannot be used
> for a plenty of files
>
> why to not use special symbol NOF (next of file ,
> which finishes a file, but doesnt close IO and begins following?!
>
Why not just set the reaceiving end of the pipe to nonblocking mode via
fcntl and use select to check for new input?
| |
| Rainer Weikusat 2007-12-30, 7:37 am |
| Andrew Wingorodov <mail@andr.mobi> writes:
> fjblurt@yahoo.com wrote:
>
> i am disappointed
> well-known UNIX-concept as is 'conveyor' actually is a myth
> fork demands very much greater resources, it cannot be used
> for a plenty of files
This is a belief you are presumably sharing with a lot of other
people who have never really even worked with a 'slow' PC (let's say
25Mhz 486) let alone a computer that was actually 'slow'. Yet people
have, surprisingly, managed to use shell created pipelines (this being
the usual term) on such 'slow' computers for actual work.
Obviously, the universe must work different in some places ...
> why to not use special symbol NOF (next of file , which finishes a
> file, but doesnt close IO and begins following?!
'tr' is probably using the stdio-library and stdio streams not
referring to terminal devices are by default 'fully buffered', meaning
output is accumulated until the program using them either terminates
(eg, because it got an EOF on the input side) or the buffer is filled
completely, and then, a single system call is used to write it.
Your specific problem is that you are using the same file descriptor
for input and output, so, you cannot force an EOF in the input side
without losing the accumulated output. You could, for instance, be
using a socket pair (=> socketpair) instead and use shutdown (=>
shutdown) to cause an input EOF without invalidating the descriptor
for reading from it as well. The other common solution would be to use
separate pipes for input and output.
| |
| fjblurt@yahoo.com 2007-12-30, 7:24 pm |
| On Dec 30, 4:42 am, Rainer Weikusat <rweiku...@mssgmbh.com> wrote:
> Andrew Wingorodov <m...@andr.mobi> writes:
>
>
> This is a belief you are presumably sharing with a lot of other
> people who have never really even worked with a 'slow' PC (let's say
> 25Mhz 486) let alone a computer that was actually 'slow'. Yet people
> have, surprisingly, managed to use shell created pipelines (this being
> the usual term) on such 'slow' computers for actual work.
I think he means he doesn't want to fork/exec a new instance of tr for
every single line.
> Obviously, the universe must work different in some places ...
>
>
> 'tr' is probably using the stdio-library and stdio streams not
> referring to terminal devices are by default 'fully buffered', meaning
> output is accumulated until the program using them either terminates
> (eg, because it got an EOF on the input side) or the buffer is filled
> completely, and then, a single system call is used to write it.
>
> Your specific problem is that you are using the same file descriptor
> for input and output, so, you cannot force an EOF in the input side
> without losing the accumulated output. You could, for instance, be
> using a socket pair (=> socketpair) instead and use shutdown (=>
> shutdown) to cause an input EOF without invalidating the descriptor
> for reading from it as well. The other common solution would be to use
> separate pipes for input and output.
The real problem is that tr will exit when it encounters EOF.
| |
| Rainer Weikusat 2007-12-31, 7:34 am |
| fjblurt@yahoo.com writes:
> On Dec 30, 4:42 am, Rainer Weikusat <rweiku...@mssgmbh.com> wrote:
[...]
>
> The real problem is that tr will exit when it encounters EOF.
This was a trash-posting, anyway, because the OP is using two
different pipes. That's why I canceled it. Certain poplular
sub-standard NNTP-gateways just apparently cannot process control
messages properly :->.
Anway, a solution for the problem is to either force the 'tr stdio' to
line buffering by means of a PTY (as you had already written) or to
just continue to feed input into tr and process output from tr as it
becomes available. If there is no more input, what else should the
program do except terminate?
| |
|
| Rainer Weikusat wrote:
> Certain poplular sub-standard NNTP-gateways just apparently cannot
> process control messages properly :->.
Many Usenet providers now ignore cancel messages.
|
|
|
|
|