|
Home > Archive > Unix Shell > November 2006 > list of filenames with spaces -> var
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
list of filenames with spaces -> var
|
|
|
| I often split my scripts into functions like this:
#!/bin/sh
VERBOSE=0; # this is -v
.....
PROCESS_ARGV "$@" # sets $VERBOSE, and $FILES
PROCESS_FILES # process files $FILES
This works, except when filenames in $FILES contains spaces
or somesuch other funny char. this breaks. I could invoke
'PROCESS_FILES' from PROCESS_ARGV (like PROCESS_FILES "$@"),
but I do not much like this solution; I'd like to keep invocations of
PROCESS_ARGV PROCESS_FILES one after the other on the
same level. Which options do I have to solve the problem with spaces
in $FILES ?
Thanks
Yakov
| |
| Janis Papanagnou 2006-11-22, 7:27 am |
| Yakov wrote:
> I often split my scripts into functions like this:
>
> #!/bin/sh
> VERBOSE=0; # this is -v
> ....
> PROCESS_ARGV "$@" # sets $VERBOSE, and $FILES
> PROCESS_FILES # process files $FILES
What shall that do? A function/command without arguments? How do you
pass the filename list to the function/command?
>
> This works, except when filenames in $FILES contains spaces
> or somesuch other funny char.
Your problem seems to be elsewhere. You don't pass filenames in the
second function/command.
> this breaks. I could invoke
> 'PROCESS_FILES' from PROCESS_ARGV (like PROCESS_FILES "$@"),
> but I do not much like this solution; I'd like to keep invocations of
> PROCESS_ARGV PROCESS_FILES one after the other on the
> same level. Which options do I have to solve the problem with spaces
> in $FILES ?
You can generally avoid problems with spaces in filenames by double
quoting all the relevant variable expansions as you've done with "$@".
You seem to want the function PROCESS_ARGV to return/print the number
of arguments that are to be passed further to PROCESS_FILES, so that
you can shift $nof_options and call it as PROCESS_FILES "$@".
nof_options=$( PROCESS_ARGV "$@" )
shift $nof_options
PROCESS_FILES "$@"
To handle any errors you should return/print some value that you can
check before calling the second function.
Janis
>
> Thanks
> Yakov
>
| |
|
| Janis Papanagnou wrote:
> Yakov wrote:
>
> What shall that do? A function/command without arguments? How do you
> pass the filename list to the function/command?
These functions communicate via [global] variables. PROCESS_ARGV
sets global variable $FILES. PROCESS_FILES uses global variable $FILES.
Yakov
| |
| Janis Papanagnou 2006-11-22, 1:17 pm |
| Yakov wrote:
> Janis Papanagnou wrote:
>
>
>
>
> These functions communicate via [global] variables. PROCESS_ARGV
> sets global variable $FILES. PROCESS_FILES uses global variable $FILES.
Okay, so you're actually doing something like...
FILES=""
PROCESS_ARGV ()
{
: process N, say 3, options using getopts (or whatever)
: Whatever with "$1" "$2" "$3"
shift 3
FILES="$@"
}
PROCESS_FILES ()
{
printf "File: %s\n" $FILES
}
PROCESS_ARGV "$@"
PROCESS_FILES
The problem is that "$@" is handled specifically containing structured
data, and you make that structure flat by assigning it to an ordinary
variable.
What I had suggested upthread was...
PROCESS_ARGV ()
{
: process N, say 3, options using getopts (or whatever)
: Whatever with "$1" "$2" "$3"
printf "%s\n" 3
}
PROCESS_FILES ()
{
printf "File: %s\n" "$@"
}
nof_files=$( PROCESS_ARGV "$@" )
shift $nof_files
PROCESS_FILES "$@"
....which preserves the whitespace.
Janis
>
> Yakov
>
| |
| Janis Papanagnou 2006-11-22, 1:17 pm |
| Janis Papanagnou wrote:
> Yakov wrote:
>
>
>
> Okay, so you're actually doing something like...
>
> FILES=""
>
> PROCESS_ARGV ()
> {
> : process N, say 3, options using getopts (or whatever)
> : Whatever with "$1" "$2" "$3"
> shift 3
> FILES="$@"
> }
>
> PROCESS_FILES ()
> {
> printf "File: %s\n" $FILES
> }
>
> PROCESS_ARGV "$@"
> PROCESS_FILES
>
>
> The problem is that "$@" is handled specifically containing structured
> data, and you make that structure flat by assigning it to an ordinary
> variable.
>
> What I had suggested upthread was...
>
> PROCESS_ARGV ()
> {
> : process N, say 3, options using getopts (or whatever)
> : Whatever with "$1" "$2" "$3"
> printf "%s\n" 3
> }
>
> PROCESS_FILES ()
> {
> printf "File: %s\n" "$@"
> }
>
> nof_files=$( PROCESS_ARGV "$@" )
> shift $nof_files
> PROCESS_FILES "$@"
>
>
> ...which preserves the whitespace.
Another possibility is to pass the shift size through a global variable...
SHIFT=0
PROCESS_ARGV ()
{
: process N, say 3, options
printf "opt1=%s opt2=%s opt3=%s\n" "$1" "$2" "$3"
SHIFT=3
}
PROCESS_FILES ()
{
printf "File: %s\n" "$@"
}
PROCESS_ARGV "$@"
shift $SHIFT
PROCESS_FILES "$@"
....which might be closer to what you already have.
Janis
[vbcol=seagreen]
>
> Janis
>
| |
| Stephane CHAZELAS 2006-11-22, 1:17 pm |
| 2006-11-22, 03:12(-08), Yakov:
> I often split my scripts into functions like this:
>
> #!/bin/sh
> VERBOSE=0; # this is -v
> ....
> PROCESS_ARGV "$@" # sets $VERBOSE, and $FILES
> PROCESS_FILES # process files $FILES
>
> This works, except when filenames in $FILES contains spaces
> or somesuch other funny char. this breaks. I could invoke
> 'PROCESS_FILES' from PROCESS_ARGV (like PROCESS_FILES "$@"),
> but I do not much like this solution; I'd like to keep invocations of
> PROCESS_ARGV PROCESS_FILES one after the other on the
> same level. Which options do I have to solve the problem with spaces
> in $FILES ?
[...]
The only list-type variable Bourne or standard sh has is "$@".
The only other reasonable option is to use a string type
variable and split it whenever you want to use it.
Then, you need to use a separator that is not likely to be
present in a filename. The NUL characters is not allowed in a
file name but unfortunately, not in a variable either (except
with zsh). The newline character comes to mind.
As in:
IFS='
'
FILES="$*"
then, you can do
set -f;
for f in $FILES; do...
or:
set -f
cmd -- $FILES
But best is to stick with the "$@" variable*s* (remember there's
one per function).
--
Stéphane
| |
| osiris@abydos.kmt 2006-11-24, 1:31 am |
|
On Wed, 22 Nov 2006 15:01:42 +0000, Stephane CHAZELAS <this.address@is.invalid> wrote:
>
>2006-11-22, 03:12(-08), Yakov:
>[...]
>
>The only list-type variable Bourne or standard sh has is "$@".
>
>The only other reasonable option is to use a string type
>variable and split it whenever you want to use it.
>
>Then, you need to use a separator that is not likely to be
>present in a filename. The NUL characters is not allowed in a
>file name but unfortunately, not in a variable either (except
>with zsh). The newline character comes to mind.
>
>As in:
>
>IFS='
>'
>FILES="$*"
>
>then, you can do
>
>set -f;
>for f in $FILES; do...
>
>or:
>
>set -f
>cmd -- $FILES
>
>But best is to stick with the "$@" variable*s* (remember there's
>one per function).
>
>--
>Stéphane
I am using Bash. I have a list of filenames in one variable ( $FILES1 )
and another list of filenames in a second variable ( $FILES2 ).
Only the filenames in $FILES2 contain spaces.
I need to count the number of filenames and my attempts
are not working. If I set IFS="\n" (actually as you did above)
before I assign the 2nd variable, is it possible to count the filenames
of each variable and then add them by:
COUNT1=`echo -n "$FILES1" |wc -w`
COUNT2=`echo "$FILES2" |wc -l`
COUNT=$(( $COUNT1 + $COUNT2 ))
Am I understanding this correctly? Thanks for any help.
| |
|
| osiris@abydos.kmt wrote:
[snip]
>
> I am using Bash. I have a list of filenames in one variable ( $FILES1 )
> and another list of filenames in a second variable ( $FILES2 ).
> Only the filenames in $FILES2 contain spaces.
Are these, FILES[12], array variables or plain variables?
>
> I need to count the number of filenames and my attempts
> are not working. If I set IFS="\n" (actually as you did above)
> before I assign the 2nd variable, is it possible to count the filenames
> of each variable and then add them by:
>
> COUNT1=`echo -n "$FILES1" |wc -w`
> COUNT2=`echo "$FILES2" |wc -l`
> COUNT=$(( $COUNT1 + $COUNT2 ))
>
> Am I understanding this correctly? Thanks for any help.
In case it's a plain variable you cannot simply distinguish whether
a space in a filename is part of the file or the delimiter between
files.
If it is an array variable you can directly interrogate the number
of array elements (files) using builtins...
$ F1=( a $'b c' d )
$ printf "%s\n" "${F1[@]}"
a
b c
d
$ printf "%s\n" "${#F1[@]}"
3
Janis
| |
| osiris@abydos.kmt 2006-11-24, 7:19 pm |
|
On 24 Nov 2006 02:09:36 -0800, "Janis" <janis_papanagnou@hotmail.com> wrote:
>
>osiris@abydos.kmt wrote:
>[snip]
>
>Are these, FILES[12], array variables or plain variables?
They were plain variables but they're now array elements.
Thanks for knocking me on the head. 
That helped a lot!
>
>In case it's a plain variable you cannot simply distinguish whether
>a space in a filename is part of the file or the delimiter between
>files.
>
>If it is an array variable you can directly interrogate the number
>of array elements (files) using builtins...
>
> $ F1=( a $'b c' d )
>
> $ printf "%s\n" "${F1[@]}"
> a
> b c
> d
>
> $ printf "%s\n" "${#F1[@]}"
> 3
I have the count correct but I'm still not getting spaces
in the filenames to delineate correctly when used in commands that follow.
My array of filenames (where "$CWD" could contain spaces) looks like:
ARRAY=( "${CWD}" "${CWD}.html" "${CWD}.png" )
Below is the for loop which does not delineate the filenames
containing spaces properly. All the loop is doing is
constructing the righthand side of the find command
which is later to be used in my script.
BOOLE="-and"
for (( J = 0 ; J < ${#ARRAY[@]} ; J++ )) ; do
if [ $(( $J + 1 )) -eq "${#ARRAY[@]}" ] ; then
BOOLE="-print"
fi
eval s${J}="\\!\ \ -name\ ${ARRAY[$J]}\ $BOOLE"
done
Can you see what's wrong?
Below is where the find command is completely constructed
CMD='find . -maxdepth 1 '
OPT=""
J=1
while [ "$J" -le ${#ARRAY[@]} ] ; do
OPT="${OPT} \$s${J}"
J=$(( $J + 1 ))
done
Later, I use the find command in this way:
eval ${CMD}\ ${OPT} |
{
# do the main processing excluding the files in $ARRAY
}
Thanks much.
You guys are the most helpful and most knowledgeable resource on the Internet!
Almost everything I wrote above is what I've learned from all of you.
| |
| Chris F.A. Johnson 2006-11-24, 7:19 pm |
| On 2006-11-24, osiris@abydos.kmt wrote:
>
> On 24 Nov 2006 02:09:36 -0800, "Janis" <janis_papanagnou@hotmail.com> wrote:
>
> They were plain variables but they're now array elements.
> Thanks for knocking me on the head.
> That helped a lot!
>
>
> I have the count correct but I'm still not getting spaces
> in the filenames to delineate correctly when used in commands that follow.
>
> My array of filenames (where "$CWD" could contain spaces) looks like:
>
> ARRAY=( "${CWD}" "${CWD}.html" "${CWD}.png" )
>
> Below is the for loop which does not delineate the filenames
> containing spaces properly. All the loop is doing is
> constructing the righthand side of the find command
> which is later to be used in my script.
>
> BOOLE="-and"
> for (( J = 0 ; J < ${#ARRAY[@]} ; J++ )) ; do
Even when using non-portable features, such as arrays, I use
portable syntax whenever possible:
J=0
while [ "$J" -lt "${#ARRAY[@]}" ]
do
...
J=$(( $J + 1 ))
done
It helps to prevent bad habits, and make the script easier to
modify for a POSIX shell if it should ever be necessary. You don't
gain anything by the non-portable syntax.
> if [ $(( $J + 1 )) -eq "${#ARRAY[@]}" ] ; then
> BOOLE="-print"
> fi
> eval s${J}="\\!\ \ -name\ ${ARRAY[$J]}\ $BOOLE"
Do you intend s$J as a scalar variable, or do you mean s[$J]?
> done
>
> Can you see what's wrong?
eval "s${J}=\"\\! -name '${ARRAY[$J]}' $BOOLE"
> Below is where the find command is completely constructed
>
> CMD='find . -maxdepth 1 '
> OPT=""
> J=1
> while [ "$J" -le ${#ARRAY[@]} ] ; do
> OPT="${OPT} \$s${J}"
> J=$(( $J + 1 ))
> done
>
> Later, I use the find command in this way:
>
> eval ${CMD}\ ${OPT} |
You are stripping the quotes.
eval "${CMD}\ ${OPT}" |
Without going into it further, I cannot tell whether that is enough
to do what you want. I suggest you examine exactly what is being
executed (especially by eval). Use 'set -x', for example, or echo
the command.
> {
> # do the main processing excluding the files in $ARRAY
> }
>
>
> Thanks much.
> You guys are the most helpful and most knowledgeable resource on the Internet!
> Almost everything I wrote above is what I've learned from all of you.
--
Chris F.A. Johnson, author <http://cfaj.freeshell.org/shell>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence
|
|
|
|
|