Unix Shell - Bourne Shell: scope of variables in while loop

Author

Bourne Shell: scope of variables in while loop

Doug Ritschel

2006-08-26, 1:24 pm

For some reason, I am not able to access variables that were changed in a
while loop, after the while loop finishes. For example:

i=0
echo "this is a test" | while read line
do
i = `expr $i + 1`
done
echo $i

-------------------------------
The echo statement prints 0 rather than 1. Does anyone know why?

Michal Nazarewicz

2006-08-26, 1:24 pm

"Doug Ritschel" <ritschel@optonline.net> writes:

> i=0
> echo "this is a test" | while read line; do
> i = `expr $i + 1`
> done
> echo $i

The while loop is run in a subshell and it has it's own local
variables so changing them won't change the value of oryginal
variable.

In fact AFAIK the question was answered in FAQ.

--
Best regards, _ _
.o. | Liege of Serenly Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michal "mina86" Nazarewicz (o o)
ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--

Doug Ritschel

2006-08-26, 1:24 pm

"Michal Nazarewicz" <mina86@tlen.pl> wrote in message
news:87r6z3d27r.fsf@erwin.piotrekn...
> "Doug Ritschel" <ritschel@optonline.net> writes:
>
>
> The while loop is run in a subshell and it has it's own local
> variables so changing them won't change the value of oryginal
> variable.
>
> In fact AFAIK the question was answered in FAQ.
>
>

Why, then , if I run the following, the echo prints 3, which is the number
of lines in the hosts file.

i=0
while read line; do
i = `expr $i + 1`
done < /etc/hosts
echo $i

Doug Ritschel

2006-08-26, 1:24 pm

"Doug Ritschel" <ritschel@optonline.net> wrote in message
news:8sZHg.206$Y43.108@newsfe09.lga...
>
> "Michal Nazarewicz" <mina86@tlen.pl> wrote in message
> news:87r6z3d27r.fsf@erwin.piotrekn...
>
> Why, then , if I run the following, the echo prints 3, which is the number
> of lines in the hosts file.
>
> i=0
> while read line; do
> i = `expr $i + 1`
> done < /etc/hosts
> echo $i
>
>

Found the below answer:

The disadvantage of the pipe-to-while method is that each element in the
pipeline is run in a subshell, so variables set inside the while loop
(for example) won't be set after the loop is complete[*].

find . | {
while read line
do
: whatever
word=${line%% *}
done

: variables set within the loop are still available here, e.g.:
printf "%s\n" "$word"

}

Michal Nazarewicz

2006-08-26, 1:24 pm

"Doug Ritschel" <ritschel@optonline.net> writes:

> "Michal Nazarewicz" <mina86@tlen.pl> wrote in message
> news:87r6z3d27r.fsf@erwin.piotrekn...
>
> Why, then , if I run the following, the echo prints 3, which is the
> number of lines in the hosts file.
>
> i=0
> while read line; do i=`expr $i + 1`; done < /etc/hosts
> echo $i

Because there is no need for subshell. In the first while loop you've
used a pipe so shell had to run another process. In the second while
loop you didn't use a pipe but only redirect stdout to be read from
/etc/hosts.

--
Best regards, _ _
.o. | Liege of Serenly Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michal "mina86" Nazarewicz (o o)
ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--

Chris F.A. Johnson

2006-08-26, 7:22 pm

On 2006-08-26, Doug Ritschel wrote:
>
> "Michal Nazarewicz" <mina86@tlen.pl> wrote in message
> news:87r6z3d27r.fsf@erwin.piotrekn...
>
> Why, then , if I run the following, the echo prints 3, which is the number
> of lines in the hosts file.
>
> i=0
> while read line; do
> i = `expr $i + 1`
> done < /etc/hosts
> echo $i

First, you did not get any output from that script unless you have
an executable named 'i'. You are asking the shell to execute a
command named 'i', with arguments of '=' and '1'; you are not
incrementing a variable. You meant: i=`expr $i + 1` (no spaces
around the equals sign).

Second, you were not running it (with the error fixed) in a Bourne
shell. If you were, you would still have seen '0'. In a Bourne
shell, the redirecion alone is enough to force a subshell.

--
Chris F.A. Johnson, author <http://cfaj.freeshell.org>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence

Michal Nazarewicz

2006-08-27, 7:25 am

"Chris F.A. Johnson" <cfajohnson@gmail.com> writes:

> On 2006-08-26, Doug Ritschel wrote:
>
> Second, you were not running it (with the error fixed) in a Bourne
> shell. If you were, you would still have seen '0'. In a Bourne
> shell, the redirecion alone is enough to force a subshell.

Are you sure?

$ cat test.sh
i=0
while read line; do i=$(($i + 1)); done </etc/hosts
echo $i
$ ash test.sh
24
$ bash test.sh
24
$ zsh test.sh
24

Any pointers to standards which support your claim?

--
Best regards, _ _
.o. | Liege of Serenly Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michal "mina86" Nazarewicz (o o)
ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--

Stephane CHAZELAS

2006-08-27, 1:20 pm

2006-08-27, 13:57(+02), Michal Nazarewicz:
> "Chris F.A. Johnson" <cfajohnson@gmail.com> writes:
>
>
> Are you sure?
>
> $ cat test.sh
> i=0
> while read line; do i=$(($i + 1)); done </etc/hosts
> echo $i
> $ ash test.sh
> 24
> $ bash test.sh
> 24
> $ zsh test.sh
> 24
>
>
> Any pointers to standards which support your claim?
[...]

Neither ash, bash or zsh are the Bourne shell. Your ash is not
the original ash as it supports $((...)). You can find a Bourne
shell as the /bin/sh of Solaris or some very old commercial
Unices.

The Bourne shell doesn't have $((...)), and there's no standard
specifying the Bourne shell.

In the Bourne shell, redirection meant fork. Newer shells use
some trick to work around that limitation (they save the
redirected fds (for restauration afterwards) by dupplicating
them on other fds before doing the redirections, which you could
do manually with the Bourne shell this way:
exec 3<&0 < /etc/hosts
while read line
do i=`expr "$i" + 1 3<&-`
done
exec <&3 3<&-
)

--
Stéphane

Michal Nazarewicz

2006-08-27, 1:20 pm

Stephane CHAZELAS <this.address@is.invalid> writes:

> Neither ash, bash or zsh are the Bourne shell. [...] You can find
> a Bourne shell as the /bin/sh of Solaris [...]

You're right; I've checked that on SunOS 5.10 and it've printed 0.

> The Bourne shell doesn't have $((...)), and there's no standard
> specifying the Bourne shell.

I get more and more confused as far as standards are concerned. Some
time ago I've downloaded "The Open Group Base Specifications Issue 6
IEEE Std 1003.1, 2004 Edition" and checked all my scripts whether the
conform to that standard and now it appears they will all crush and
burn on some unix platforms.

--
Best regards, _ _
.o. | Liege of Serenly Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michal "mina86" Nazarewicz (o o)
ooo +--<mina86*tlen.pl>--<jid:mina86*jabber.org>--ooO--(_)--Ooo--

Stachu 'Dozzie' K.

2006-08-27, 1:20 pm

On 27.08.2006, Michal Nazarewicz <mina86@tlen.pl> wrote:
> Stephane CHAZELAS <this.address@is.invalid> writes:
>
>
> You're right; I've checked that on SunOS 5.10 and it've printed 0.
>
>
> I get more and more confused as far as standards are concerned. Some
> time ago I've downloaded "The Open Group Base Specifications Issue 6
> IEEE Std 1003.1, 2004 Edition" and checked all my scripts whether the
> conform to that standard and now it appears they will all crush and
> burn on some unix platforms.

Yeap. But they will crush on platforms which doesn't conform to
standard (by default). On Solaris POSIX/SUS conformant shell exists as
/usr/xpg4/bin/sh.

--
<Kosma> Niektórzy lubi± dozziego...
<Kosma> Oczywi¶cie szanujemy ich.
Stanislaw Klekot

Stephane CHAZELAS

2006-08-27, 1:20 pm

2006-08-27, 15:29(+02), Michal Nazarewicz:
> Stephane CHAZELAS <this.address@is.invalid> writes:
>
>
> You're right; I've checked that on SunOS 5.10 and it've printed 0.
>
>
> I get more and more confused as far as standards are concerned. Some
> time ago I've downloaded "The Open Group Base Specifications Issue 6
> IEEE Std 1003.1, 2004 Edition" and checked all my scripts whether the
> conform to that standard and now it appears they will all crush and
> burn on some unix platforms.
[...]

Those standards specify the POSIX sh which is based on a subset
of ksh which is itself based on the Bourne shell.

sh over the time was a Thomson shell, then a Bourne shell, then
a POSIX shell. Where before they were implementations, now it is
a standard, and several sh implementations (bash, ksh, new
ashes...) implement that standard.

--
Stéphane

Jon LaBadie

2006-08-27, 1:20 pm

Stephane CHAZELAS wrote:
> 2006-08-27, 15:29(+02), Michal Nazarewicz:
> [...]
>
> Those standards specify the POSIX sh which is based on a subset
> of ksh which is itself based on the Bourne shell.
>
> sh over the time was a Thomson shell, then a Bourne shell, then
> a POSIX shell. Where before they were implementations, now it is
> a standard, and several sh implementations (bash, ksh, new
> ashes...) implement that standard.
>

Gee, no Mashey shell in there

What does POSIX/OpenGroup have to say about redirection of shell
control structures and special commands? Probably nothing or
'implementation defined'.

bash seems to standout among the three "major" Bourne evolutions,
bash, zsh, and ksh, in treating redirection caused by the redirection
operators differently than that caused by the pipe operator.

$ cat testscript
i=0
while read line; do
i=`expr $i + 1`
done < /etc/hosts
echo $i

s=foobar
read s < /etc/hosts
echo $s

i=0
cat /etc/hosts |
while read line; do
i=`expr $i + 1`
done
echo $i

s=foobar
cat /etc/hosts | read s
echo $s

$ ksh ./testscript
33
# Internet host table
33
# Internet host table

$ zsh ./testscript
33
# Internet host table
33
# Internet host table

$ bash ./testscript
33
# Internet host table
0
foobar

Sven Mascheck

2006-08-27, 1:20 pm

Stephane CHAZELAS wrote:

> You can find a Bourne shell as the /bin/sh of Solaris
> or some very old commercial Unices.

or in SCO OpenServer or Tru64 or UnixWare or in the Heirloom project.

Stephane CHAZELAS

2006-08-27, 1:20 pm

2006-08-27, 11:15(-04), Jon LaBadie:
[...]
>
> Gee, no Mashey shell in there

That was simplified, you may want to insert as well the
different evolutions of the Bourne shell (addition of functions,
different variants on different OSes...)

> What does POSIX/OpenGroup have to say about redirection of shell
> control structures and special commands? Probably nothing or
> 'implementation defined'.

It's specified. It should behave the bash/ksh way, not the
Bourne way.

> bash seems to standout among the three "major" Bourne evolutions,
> bash, zsh, and ksh, in treating redirection caused by the redirection
> operators differently than that caused by the pipe operator.

That's something different. parts of a pipeline are run
concurrently, so have to be run in different processes. zsh,
AT&T ksh run the rightmost part in the current shell process,
contrary to bash, ash or pdksh or the Bourne shell where every
part is in children processes. POSIX says that one shouldn't
assume that either part is run on a subshell or not, so that
both bash and AT&T ksh are conformant.

--
Stéphane

Jon LaBadie

2006-08-27, 7:25 pm

Stephane CHAZELAS wrote:
> 2006-08-27, 11:15(-04), Jon LaBadie:
[...]
>
>
> That's something different. parts of a pipeline are run
> concurrently, so have to be run in different processes. zsh,
> AT&T ksh run the rightmost part in the current shell process,
> contrary to bash, ash or pdksh or the Bourne shell where every
> part is in children processes. POSIX says that one shouldn't
> assume that either part is run on a subshell or not, so that
> both bash and AT&T ksh are conformant.
>

Though I've certainly had scripts using pipes on the output of a
control structure, apparently I never attempted to use any of the
control structure's local environment later in the script. Good
to know that limitation.

Not that I didn't believe you ;), but I tested to confirm.