|
Home > Archive > Unix Programming > October 2006 > accomodating an unknown number of pipes in a shell program
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
accomodating an unknown number of pipes in a shell program
|
|
| djneill@iwebworks.com 2006-10-12, 1:36 am |
| Hi all,
This is my first attempt to use pipes. My shell has to take in a
command line containing a number of commands separated by the '|'
character representing a pipe.
So /bin/ls | /bin/sort would list the files in the current working
directory in sorted order.
Each command is exec'd in a forked child process. The parent only
creates the pipes and waits for the children to finish.
Does this plan sound right?
1. Make exactly the number of pipes needed. Store as array of arrays
of 2 ints.
e.g. declare as int mypipes[num_commands-1][2]; // a pipe between each
command
2. Fork n processes.
3. Within each child process.
a. if this is the first command, direct stdout to the first pipe's
write end with dup2(pipes[0][1], 1). Close all existing pipe file
descriptors (and I mean all!): with close(pipe[i][0]) and close
(pipe[i][1]).
b. if this is the last command, direct stdin to the last pipe's
read end with dup2(pipes[LAST][0]). Close all pipe file descriptors:
as above.
c. else, for the commands in the middle with incoming and outgoing
pipes, direct stdin to the incoming pipe's read end, and stdout to the
outgoing pipe's write end. Close all pipe file descriptors: as above.
d. exec the appropriate command in the child process.
4. The parent process just closes all pipes' file descriptors and
waits for children to finish.
For any info, I would greatly appreciate it.
Sincerely,
Dan
| |
| Logan Shaw 2006-10-12, 1:36 am |
| djneill@iwebworks.com wrote:
> This is my first attempt to use pipes. My shell has to take in a
> command line containing a number of commands separated by the '|'
> character representing a pipe.
>
> So /bin/ls | /bin/sort would list the files in the current working
> directory in sorted order.
>
> Each command is exec'd in a forked child process. The parent only
> creates the pipes and waits for the children to finish.
>
> Does this plan sound right?
>
> 1. Make exactly the number of pipes needed. Store as array of arrays
> of 2 ints.
> e.g. declare as int mypipes[num_commands-1][2]; // a pipe between each
> command
> 2. Fork n processes.
> 3. Within each child process.
> a. if this is the first command, direct stdout to the first pipe's
> write end with dup2(pipes[0][1], 1). Close all existing pipe file
> descriptors (and I mean all!): with close(pipe[i][0]) and close
> (pipe[i][1]).
> b. if this is the last command, direct stdin to the last pipe's
> read end with dup2(pipes[LAST][0]). Close all pipe file descriptors:
> as above.
> c. else, for the commands in the middle with incoming and outgoing
> pipes, direct stdin to the incoming pipe's read end, and stdout to the
> outgoing pipe's write end. Close all pipe file descriptors: as above.
> d. exec the appropriate command in the child process.
> 4. The parent process just closes all pipes' file descriptors and
> waits for children to finish.
That plan should work.
However, you could also do it in a way that's a little bit like
recursion: take the last command, build a pipe, start the command
in a child, then recurse and pass the remainder (everything before
the last pipe character) on to another child to repeat the process.
When you finally have a command with no pipe character, just fork
and exec that command. (That's the base case.)
Either would work fine, but the recursive process makes things a
little cleaner. You don't need to dynamically allocate an array
of pipes because each time you fork, you are creating a new
context that only needs to hold one pipe. More importantly, you
can probably make each piece in the pipe wait on the piece that
is feeding it data. Then each process only needs to wait on one
other process, which makes the wait() logic much simpler.
- Logan
| |
| SM Ryan 2006-10-13, 1:40 am |
| djneill@iwebworks.com wrote:
# Hi all,
# This is my first attempt to use pipes. My shell has to take in a
# command line containing a number of commands separated by the '|'
# character representing a pipe.
#
# So /bin/ls | /bin/sort would list the files in the current working
# directory in sorted order.
#
# Each command is exec'd in a forked child process. The parent only
# creates the pipes and waits for the children to finish.
#
# Does this plan sound right?
#
# 1. Make exactly the number of pipes needed. Store as array of arrays
# of 2 ints.
In some shells when you have the pipeline p1|p2|p3|...|pn,
the shell treats it as P|Q where P=(p1|...|pn-1), Q=pn
or P=p1, Q=(p2|...|pn). It then creates one pipe, one fork
and either the parent or child evaluate P and the child or
parent evaluate Q. A pipeline then expands into a multigeneration
lineage of processes.
The complication with doing only one fork per shell process is
that you can lose track of orphans.
# write end with dup2(pipes[0][1], 1). Close all existing pipe file
# descriptors (and I mean all!): with close(pipe[i][0]) and close
# (pipe[i][1]).
This the complication that can bite you when you create all pipes
before all forks, if you get it wrong.
--
SM Ryan http://www.rawbw.com/~wyrmwif/
Elvis was an artist. But that didn't stop him from joining the service
in time of war. That's why he is the king, and you're a shmuck.
| |
| djneill@iwebworks.com 2006-10-13, 1:40 am |
| SM Ryan wrote:
> djneill@iwebworks.com wrote:
> # Hi all,
> # This is my first attempt to use pipes. My shell has to take in a
> # command line containing a number of commands separated by the '|'
> # character representing a pipe.
> #
> # So /bin/ls | /bin/sort would list the files in the current working
> # directory in sorted order.
> #
> # Each command is exec'd in a forked child process. The parent only
> # creates the pipes and waits for the children to finish.
> #
> # Does this plan sound right?
> #
> # 1. Make exactly the number of pipes needed. Store as array of arrays
> # of 2 ints.
>
> In some shells when you have the pipeline p1|p2|p3|...|pn,
> the shell treats it as P|Q where P=(p1|...|pn-1), Q=pn
> or P=p1, Q=(p2|...|pn). It then creates one pipe, one fork
> and either the parent or child evaluate P and the child or
> parent evaluate Q. A pipeline then expands into a multigeneration
> lineage of processes.
>
> The complication with doing only one fork per shell process is
> that you can lose track of orphans.
>
> # write end with dup2(pipes[0][1], 1). Close all existing pipe file
> # descriptors (and I mean all!): with close(pipe[i][0]) and close
> # (pipe[i][1]).
>
> This the complication that can bite you when you create all pipes
> before all forks, if you get it wrong.
>
> --
> SM Ryan http://www.rawbw.com/~wyrmwif/
> Elvis was an artist. But that didn't stop him from joining the service
> in time of war. That's why he is the king, and you're a shmuck.
Hi all,
Thanks you two for your replies. I fumbled and got mine to work, but
say I wanted to do it more correctly and elegantly with recursion.
Would it be something like this?
info_type RecursePipes (int numCommandsToLink)
info_type input;
int fd[2];
pipe(fd);
if (--numCommandsToLink) {
input = RecursePipes(numCommandsToLink);
}
pid = fork();
if (pid=0) { // child
dup2(fd[1], 1);
close(fd[0]); close(fd[1]);
return (output from exec(commands.pop()) );
_exit();
}
if (pid > 0) { // parent
close(fd[0]); close(fd[1]);
}
else { } //handle error
} // end RecursePipes function
Please slash, mock and edit as needed. It's all for a good cause.
Thanks,
Dan
| |
| SM Ryan 2006-10-13, 7:46 pm |
| # Hi all,
# Thanks you two for your replies. I fumbled and got mine to work, but
# say I wanted to do it more correctly and elegantly with recursion.
# Would it be something like this?
It's not more correct. Whatever works is correct. It's an alternate
approach.
#
# info_type RecursePipes (int numCommandsToLink)
# info_type input;
# int fd[2];
# pipe(fd);
# if (--numCommandsToLink) {
# input = RecursePipes(numCommandsToLink);
# }
# pid = fork();
# if (pid=0) { // child
# dup2(fd[1], 1);
# close(fd[0]); close(fd[1]);
# return (output from exec(commands.pop()) );
# _exit();
# }
# if (pid > 0) { // parent
# close(fd[0]); close(fd[1]);
# }
# else { } //handle error
# } // end RecursePipes function
#
# Please slash, mock and edit as needed. It's all for a good cause.
If you want to do a bisecting pipeline it might look like
void recursivePipes(int numcommands,Command *command) {
if (ncommands>1) {
enum {R,W,NP}; int P[NP],pid;
if (pipe(P)<0) {
...
}
pid = fork();
if (pid<0) {
...
}else if (pid==0) { /*beginning of pipeline*/
close(1); dup(P[W]); close(P[R]); close(P[W]);
recursivePipes(numcommands-1,command);
}else { /*connect last command to pipeline*/
close(0); dup(P[R]); close(P[R]); close(P[W]);
}
}
exec command[ncommands-1]; /*ending of pipeline*/
... /*unless error each call ends in a nonreturning exec*/
}
int pipes(int numcommands,Command *command) {
int pid = fork();
if (pid<0) {
...
}else if (pid==0) { /*all child processes of pipeline*/
recursivePipes(numcommands,command);
}else {
return /*to the original shell parent*/ pid;
}
If the pipeline is &-detached, the shell can continue, otherwise
it can wait on the returned pid.
You can also do things like put the pipeline children in a process
group.
--
SM Ryan http://www.rawbw.com/~wyrmwif/
We found a loophole; they can't keep us out anymore.
|
|
|
|
|