filenames with spaces and list in a for loop
Web Server forum
Back To The Forum Home!Search!Private Messaging System

Web Server Talk Web Server Talk > Unix and Linux reviews > Free Unix support > Unix Shell > filenames with spaces and list in a for loop




  Last Thread   Next Thread Next
  Show Printable Version Email this Page Subscribe to this Thread      Post New Thread    Post A Reply      

    filenames with spaces and list in a for loop  
SiKing


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-24-07 06:16 PM

Hi all,

I need to process files in a loop, and I am having problems with filename
spaces, big time.

Everything is in bash 3.1.
The script in question is supposed to build a link-forest: replicate a direc
tory
structure, but replace each file with a link pointing to the original. I dec
ided
to do gro of the work with a recursive function:

function make_links
{
for each_item in $LIST; do
if [ -L "$1/$each_item" ]; then
# link
rm -f "$each_item"  # mercilessly overwrite existing stuff
ln -s `readlink "$1/$each_item"` "$each_item"
elif [ -d "$1/$each_item" ]; then
# directory
mkdir -p "$each_item"
cd "$each_item"
make_links "$1/$each_item" &
cd ..
else
# file
rm -f "$each_item"  # mercilessly overwrite existing stuff
ln -s "$1/$each_item" "$each_item"
fi
done
return 0
}

I am having significant issues with spaces in filenames, and I figure everyt
hing
revolves around the way I generate the $LIST in the above function.

My first attempt was to _replace_ the $LIST with `ls -1A $1`. I wanted this 
to
handle dot-files as well. Unfortunately, any file with a space would be hand
led
as two items. No good!
Second attempt was to double quote the "`ls -1A $1`", which generates ONE hu
ge
argument to the for loop. Again, no good!

In the next round I experimented with `ls -mA $1`. This generates a comma
separated list, but it also brings several new problems. To break up the com
ma
separated list, I used IFS=','. However, this left each_item with a leading
extra space. I corrected this with a each_item=`eval echo $each_item` inside
 the
loop. Unfortunately, the list also generates a newline every 80 characters (
or
sooner) in the $LIST, so once in a while I was left with each_item being set
 to
something like "\nfilename". At the first occurrence of $each_item in the
function I would get something like <filename> not found. 'man ls' hinted th
at
the secret is in the variable COLUMNS. Seems export COLUMNS=0 does nothing.
Right now I got COLUMNS=65000, but what if I have a directory that is longer
?
This seems like the best option out of the ones I tried. However, what if I 
have
a directory with more than 65000 characters worth? Or, what is a better way 
of
getting rid of the newlines?

I also tried to use expansion. I generated the list with simple $1/*, which 
also
gives me the complete path, and no dot-files. The path can be corrected with
each_item=`basename "$each_item"`. However, trying to fix the dot-files with
adding $1/.* to the list, has "very interesting" effects -> DO NOT TRY THIS 
AT
HOME! After having to reboot the machine and purge the disk of a _lot_ of li
nks,
the sys.admin. warned me against doing any more stupid things.

Anyone has any other _workable_ solution?

TIA, SK.


--
It may be that your sole purpose in life
is simply to serve as a warning to others.
-----
Candy for spammers:
http://members.shaw.ca/grubb/spamthis.html





[ Post a follow-up to this message ]



    Re: filenames with spaces and list in a for loop  
Eric Moors


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-24-07 06:16 PM

SiKing wrote:

> Hi all,
>
> I need to process files in a loop, and I am having problems with filename
> spaces, big time.
>
> Everything is in bash 3.1.

shopt -s dotglob
for each_item in $1/*; do
..
done

Eric





[ Post a follow-up to this message ]



    Re: filenames with spaces and list in a for loop  
Stephane CHAZELAS


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-24-07 06:16 PM

2007-01-24, 16:47(+01), Eric Moors:
> SiKing wrote:
> 
>
> shopt -s dotglob
> for each_item in $1/*; do
>    ...
> done
[...]

for each_item in "$1"/*; do

unless you want to consider the content of $1 as a globbing pattern as well.

--
Stéphane





[ Post a follow-up to this message ]



    Re: filenames with spaces and list in a for loop  
Bill Marcum


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-24-07 06:16 PM

On Wed, 24 Jan 2007 15:30:20 +0000, SiKing
<nospam@noway.invalid> wrote:
>
>
> Hi all,
>
>
> I am having significant issues with spaces in filenames, and I figure
> everything revolves around the way I generate the $LIST in the above
> function.
>
Instead of 'for each_item in $LIST', try
while read each_item; do ... ; done < $FILE
or
ls -A "$1" | while read each_item ...


--
Q:	Why do the police always travel in threes?
A:	One to do the reading, one to do the writing, and the other keeps
an eye on the two intellectuals.





[ Post a follow-up to this message ]



    Re: filenames with spaces and list in a for loop  
Stephane CHAZELAS


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-24-07 06:16 PM

2007-01-24, 10:57(-05), Bill Marcum:
> On Wed, 24 Jan 2007 15:30:20 +0000, SiKing
>   <nospam@noway.invalid> wrote: 
> Instead of 'for each_item in $LIST', try
>    while read each_item; do ... ; done < $FILE
> or
>   ls -A "$1" | while read each_item ...
[...]

To read a line of input (which is already an approximation for
reading an item from a list, given that a filename doesn't
necessarily map to one line), it is not

read each_item

it is

IFS= read -r each_item

"read each_item" means a different thing.

And to loop over the elements of a ksh/bash/zsh list type
variable, it is not

for each_item in $LIST

(except in zsh) it's is:

for each_item in "${LIST[@]}"

And if $LIST is not a list-type variable but a string type
variable containing a newline separated list of items, it is
not

for each_item in $LIST

But:

set -f
IFS='
'
for each_item in $LIST

--
Stéphane





[ Post a follow-up to this message ]



    Re: filenames with spaces and list in a for loop  
SiKing


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-25-07 12:19 PM

Stephane CHAZELAS wrote:[vbcol=seagreen]
> 2007-01-24, 16:47(+01), Eric Moors: 
> [...]
>
> for each_item in "$1"/*; do
>
> unless you want to consider the content of $1 as a globbing pattern as well.[/vbco
l]

This one seems to be the cleanest. I did not even know about 'shopt'. Thanx!

In case anyone is interested, the final^Wworking solution is:
function make_links
{
echo Processing $1

shopt -s dotglob    # reveal dotfiles
shopt -s nullglob   # skip empty directories
for each_item in "$1"/*; do
each_item=`basename "$each_item"`   # extract just the filename

if [ -L "$1/$each_item" ]; then
# link
link_name=`readlink "$1/$each_item"`
rm -f "$each_item"  # mercilessly overwrite existing stuff
ln -s "$link_name" "$each_item"
elif [ -d "$1/$each_item" ]; then
# directory
mkdir -p "$each_item"
cd "$each_item"
make_links "$1/$each_item" &
cd ..
else
# file
rm -f "$each_item"  # mercilessly overwrite existing stuff
ln -s "$1/$each_item" "$each_item"
fi

done

return 0
}

I am sure it can still be tweaked, but I am worried about tweaking things th
at
already work.

SK.


--
It may be that your sole purpose in life
is simply to serve as a warning to others.
-----
Candy for spammers:
http://members.shaw.ca/grubb/spamthis.html





[ Post a follow-up to this message ]



    Sponsored Links  




 





   All times are GMT. The time now is 02:24 AM.      Post New Thread    Post A Reply      
  Last Thread   Next Thread Next


Most Popular forums 

Forum Jump:
Rate This Thread:

Forum Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is OFF
vB code is ON
Smilies are ON
[IMG] code is OFF
 
Medical and Health forum | Computer Games Reviews | Graphics design forum

Back To The Top
Home | Usercp | Faq | Register