Unix Shell - working with paths

Author

working with paths

rickm@galaxy.nsc.com

2007-12-07, 1:37 am

I need a script that will look for specific portion of a path and
print all if it does not find it but if it does, then stop on
directory prior......like that make sense! :^)

Here an example:

look for modify :
----------------------------------------------------------------
/path/to/the/file/that/I_need/to/dont_modify

not found so print the entire line:

/path/to/the/file/that/I_need/to/dont_modify

----------------------------------------------------------------

/path/to/the/file/that/I_need/to/modify

modify is found so it needs to print:

/path/to/the/file/that/I_need/to

Whats the best why to do this?

THANKS!!!!

Steffen Schuler

2007-12-07, 1:37 am

On Thu, 06 Dec 2007 22:31:41 -0800, rickm wrote:

> I need a script that will look for specific portion of a path and print
> all if it does not find it but if it does, then stop on directory
> prior......like that make sense! :^)
>
> Here an example:
>
> look for modify :
> ----------------------------------------------------------------
> /path/to/the/file/that/I_need/to/dont_modify
>
> not found so print the entire line:
>
> /path/to/the/file/that/I_need/to/dont_modify
>
> ----------------------------------------------------------------
>
> /path/to/the/file/that/I_need/to/modify
>
> modify is found so it needs to print:
>
> /path/to/the/file/that/I_need/to
>
>
> Whats the best why to do this?
>
> THANKS!!!!

A Bourne shell script:

#!/bin/sh
for path
do
IFS=/
export IFS
set -- $path
found=0
for comp in "$@"
do
if [ "$comp" = modify ]
then
found=1
break
fi
done
if [ "$found" -ne 0 ]
then
path="${path%/*}"
if [ -z "$path" ]
then
printf "/\n"
else
printf "%s\n" "$path"
fi
else
printf "%s\n" "$path"
fi
done

Regards,

Steffen "goedel" Schuler

Steffen Schuler

2007-12-07, 7:31 am

On Thu, 06 Dec 2007 22:31:41 -0800, rickm wrote:

> I need a script that will look for specific portion of a path and print
> all if it does not find it but if it does, then stop on directory
> prior......like that make sense! :^)
>
> Here an example:
>
> look for modify :
> ----------------------------------------------------------------
> /path/to/the/file/that/I_need/to/dont_modify
>
> not found so print the entire line:
>
> /path/to/the/file/that/I_need/to/dont_modify
>
> ----------------------------------------------------------------
>
> /path/to/the/file/that/I_need/to/modify
>
> modify is found so it needs to print:
>
> /path/to/the/file/that/I_need/to
>
>
> Whats the best why to do this?
>
> THANKS!!!!

An AWK script:

BEGIN { OFS = FS = "/" }
{ for (i = 1; i <= NF; ++i)
if ($i == "modify") {
sub(/\/[^\/]+\/?$/, "")
break
} }
$0 == "" { $0 = "/" }
1

The script may be called e.g.

awk -f script <<EOT
/modify
/dont_modify
/path/to/modify/
/path/to/dont_modify/
EOT

and returns:

/
/dont_modify
/path/to
/path/to/dont_modify/

Regards,

Steffen "goedel" Schuler

Stephane Chazelas

2007-12-07, 7:31 am

On Thu, 6 Dec 2007 22:31:41 -0800 (PST), rickm@galaxy.nsc.com wrote:
> I need a script that will look for specific portion of a path and
> print all if it does not find it but if it does, then stop on
> directory prior......like that make sense! :^)
>
> Here an example:
>
> look for modify :
> ----------------------------------------------------------------
> /path/to/the/file/that/I_need/to/dont_modify
>
> not found so print the entire line:
>
> /path/to/the/file/that/I_need/to/dont_modify
>
> ----------------------------------------------------------------
>
> /path/to/the/file/that/I_need/to/modify
>
> modify is found so it needs to print:
>
> /path/to/the/file/that/I_need/to
[...]

file=/path/to/the/file/that/I_need/to/modify
search=modify

IFS=/
set -f

set -- $file
result=
for i do
[ "$i" = "$search" ] && break
[ -n "$i" ] || continue
result=$result/$i
done
result=${result:-/}
printf '%s\n' "$result"

could be one way. It assumes "$file" is not a relative path.

--
Stephane

Stephane Chazelas

2007-12-07, 7:31 am

On 7 Dec 2007 07:15:04 GMT, Steffen Schuler wrote:
[...]
> A Bourne shell script:
>
> #!/bin/sh

Note that that shebang line generally doesn't get you a Bourne
shell, nowadays, it gives you a POSIX sh. POSIX sh behaves
differently from the old Bourne shell, so your Bourne script
will be interpreted differently. Not all Unix still have a
Bourne shell nowadays, and even fewer have it as "/bin/sh".

> for path
> do
> IFS=/
> export IFS

You don't need to export IFS, and actually you shouldn't.
You export a variable when you want that commands you execute
(as in the exec() family of system calls) inherit that variable.
If you export IFS that means that it could affect the behavior
of every shell script you execute! Fortunately, except for some
old Bourne shells, most shells will ignore the $IFS from their
environment.

> set -- $path

When you leave a variable unquoted, you need to disable filename
generation unless you want globbing to be performed. So you
need:

set -f

before doing that.

Also word splitting when IFS is not blank is one area where the
Bourne shell behaves differently from nowadays standard shs.

The Bourne shell will split "/foo//bar/" into "foo" and "bar"
while modern shs will split it into "", "foo", "" and "bar"
(some will add another "").

Yet another difference: if $path is empty or contains only
blanks, in the Bourne shell (and in the Bourne shell only),

set --

will leave the positional parameters untouched. So you should
insert a

shift "$#"

before, to make sure the list of positional parameters is empty
before starting.

Also note that $path is a special variable in zsh (an array tied
to $PATH) so it's a good idea to avoid using it.

> found=0
> for comp in "$@"

Another difference between the Bourne shell and modern shs is
that if $# is 0, "$@" will expand to one empty element instead
of no element at all as in modern sh. So, it's better to use the
more portable:

for i do
...

instead.

--
Stephane

rickm@galaxy.nsc.com

2007-12-07, 1:29 pm

Sorry for the confusion, I was not clear in my need which is a file of
paths not the variable $paths.
Could you please take another look

Thanks and again my appologies

Rick

Stephane Chazelas

2007-12-07, 1:29 pm

On Fri, 7 Dec 2007 07:42:10 -0800 (PST), rickm@galaxy.nsc.com wrote:
> Sorry for the confusion, I was not clear in my need which is a file of
> paths not the variable $paths.
> Could you please take another look
[...]

sed 's,/modify$/.*$\{0,1\}$,,'

--
Stephane

rickm@galaxy.nsc.com

2007-12-07, 1:29 pm

The must be a problem with the syntax, heres the error message:

sed: command garbled: s,/modify$/.*$\{0,1\}$,,

Thanks

On Dec 7, 8:49 am, Stephane Chazelas <stephane_chaze...@yahoo.fr>
wrote:
> On Fri, 7 Dec 2007 07:42:10 -0800 (PST), ri...@galaxy.nsc.com wrote:
>
> [...]
>
> sed 's,/modify$/.*$\{0,1\}$,,'
>
> --
> Stephane

Icarus Sparry

2007-12-07, 1:29 pm

On Thu, 06 Dec 2007 22:31:41 -0800, rickm wrote:

> I need a script that will look for specific portion of a path and print
> all if it does not find it but if it does, then stop on directory
> prior......like that make sense! :^)
>
> Here an example:
>
> look for modify :
> ----------------------------------------------------------------
> /path/to/the/file/that/I_need/to/dont_modify
>
> not found so print the entire line:
>
> /path/to/the/file/that/I_need/to/dont_modify
>
> ----------------------------------------------------------------
>
> /path/to/the/file/that/I_need/to/modify
>
> modify is found so it needs to print:
>
> /path/to/the/file/that/I_need/to
>
>
> Whats the best why to do this?
>
> THANKS!!!!

This looks to be just a text manipulation problem, rather than anything
involving files. You want to remove "/modify" from the end of the string.

path="/path/to/the/file/that/I_need/to/modify"
result=${path%/modify}

You might want to remove as well /modify/ to the end of the string. To do
this add

result=${result%%/modify/*}

Then you can do what you want with the result, e.g.

echo "$result"

rickm@galaxy.nsc.com

2007-12-07, 1:29 pm

What I have is a file that has lots of paths and I need to create a
copy script but I need
it to either pass the entire path if the search result is false.
Heres a different example
the file contents:

/dirA/dirB/dirC/dirE/dirF/dirG/dirH
/dir1/dir2/dir3/dir4/dir5/dir6/dir7

If "dirG" exists, then print:

/dirA/dirB/dirC/dirE/dirF

if dirG does not exit, then print: the entire line:

/dir1/dir2/dir3/dir4/dir5/dir6/dir7

On Dec 7, 10:17 am, Icarus Sparry <use...@icarus.freeuk.com> wrote:
> On Thu, 06 Dec 2007 22:31:41 -0800, rickm wrote:
>
>
>
>
>
>
>
>
>
>
>
> This looks to be just a text manipulation problem, rather than anything
> involving files. You want to remove "/modify" from the end of the string.
>
> path="/path/to/the/file/that/I_need/to/modify"
> result=${path%/modify}
>
> You might want to remove as well /modify/ to the end of the string. To do
> this add
>
> result=${result%%/modify/*}
>
> Then you can do what you want with the result, e.g.
>
> echo "$result"

John W. Krahn

2007-12-07, 7:23 pm

rickm@galaxy.nsc.com wrote:
>
> What I have is a file that has lots of paths and I need to create a
> copy script but I need
> it to either pass the entire path if the search result is false.
> Heres a different example
> the file contents:
>
> /dirA/dirB/dirC/dirE/dirF/dirG/dirH
> /dir1/dir2/dir3/dir4/dir5/dir6/dir7
>
> If "dirG" exists, then print:
>
> /dirA/dirB/dirC/dirE/dirF
>
> if dirG does not exit, then print: the entire line:
>
> /dir1/dir2/dir3/dir4/dir5/dir6/dir7

Do you mean exists in the file system or exists in the string?

John
--
use Perl;
program
fulfillment

rickm@galaxy.nsc.com

2007-12-07, 7:23 pm

On Dec 7, 1:12 pm, "John W. Krahn" <kra...@telus.net> wrote:
> ri...@galaxy.nsc.com wrote:
>
>
>
>
>
>
>
> Do you mean exists in the file system or exists in the string?
>
> John
> --
> use Perl;
> program
> fulfillment

in a string that are lines in a file. The file has lines of file
pointers with paths, that s what Im trying to parse

Stephane Chazelas

2007-12-07, 7:23 pm

On Fri, 7 Dec 2007 09:08:33 -0800 (PST), rickm@galaxy.nsc.com wrote:
> The must be a problem with the syntax, heres the error message:
>
> sed: command garbled: s,/modify$/.*$\{0,1\}$,,
[...]

Sorry, I should have used another character than "," for the
separator as "," is used as well in the pattern, try with : for
instance:

sed 's:/modify$/.*$\{0,1\}$::'

--
Stephane

John W. Krahn

2007-12-07, 7:23 pm

rickm@galaxy.nsc.com wrote:
>
> On Dec 7, 1:12 pm, "John W. Krahn" <kra...@telus.net> wrote:
>
> in a string that are lines in a file. The file has lines of file
> pointers with paths, that s what Im trying to parse

$ echo "/dirA/dirB/dirC/dirE/dirF/dirG/dirH
/dir1/dir2/dir3/dir4/dir5/dir6/dir7" | sed -e's!/dirG/.*!!'
/dirA/dirB/dirC/dirE/dirF
/dir1/dir2/dir3/dir4/dir5/dir6/dir7

John
--
use Perl;
program
fulfillment

Steffen Schuler

2007-12-08, 7:42 pm

On Fri, 07 Dec 2007 08:28:21 +0000, Stephane Chazelas wrote:

> On 7 Dec 2007 07:15:04 GMT, Steffen Schuler wrote: [...]
>
> Note that that shebang line generally doesn't get you a Bourne shell,
> nowadays, it gives you a POSIX sh. POSIX sh behaves differently from the
> old Bourne shell, so your Bourne script will be interpreted differently.
> Not all Unix still have a Bourne shell nowadays, and even fewer have it
> as "/bin/sh".
>
>
> You don't need to export IFS, and actually you shouldn't. You export a
> variable when you want that commands you execute (as in the exec()
> family of system calls) inherit that variable. If you export IFS that
> means that it could affect the behavior of every shell script you
> execute! Fortunately, except for some old Bourne shells, most shells
> will ignore the $IFS from their environment.
>
>
> When you leave a variable unquoted, you need to disable filename
> generation unless you want globbing to be performed. So you need:
>
> set -f
>
> before doing that.
>
> Also word splitting when IFS is not blank is one area where the Bourne
> shell behaves differently from nowadays standard shs.
>
> The Bourne shell will split "/foo//bar/" into "foo" and "bar" while
> modern shs will split it into "", "foo", "" and "bar" (some will add
> another "").
>
> Yet another difference: if $path is empty or contains only blanks, in
> the Bourne shell (and in the Bourne shell only),
>
> set --
>
> will leave the positional parameters untouched. So you should insert a
>
> shift "$#"
>
> before, to make sure the list of positional parameters is empty before
> starting.
>
> Also note that $path is a special variable in zsh (an array tied to
> $PATH) so it's a good idea to avoid using it.
>
>
> Another difference between the Bourne shell and modern shs is that if $#
> is 0, "$@" will expand to one empty element instead of no element at all
> as in modern sh. So, it's better to use the more portable:
>
> for i do
> ...
>
> instead.

Thanks for teaching me, Stephane!

Can you recommend a good online guide or paper book to learn advanced
POSIX shell scripting in the UNIX and GNU/Linux environment? (Perhaps
Chris F.A. Johnson's book or O'Reilly's "Classic Shell Scripting"?)

I use SUSv3 and test each "POSIX" shell script with ash (and bash) under
Debian GNU/Linux as recommended by the Debian developers and I'm able to
test them under Solaris 10 and in the future under some free BSD
variants. Is this a good way to ensure POSIX compliance?

How do you have learned it besides by a lot of scripting and testing?

--
Steffen

Stephane Chazelas

2007-12-08, 7:42 pm

On 8 Dec 2007 16:20:13 GMT, Steffen Schuler wrote:
[...]
> Can you recommend a good online guide or paper book to learn advanced
> POSIX shell scripting in the UNIX and GNU/Linux environment? (Perhaps
> Chris F.A. Johnson's book or O'Reilly's "Classic Shell Scripting"?)
>
> I use SUSv3 and test each "POSIX" shell script with ash (and bash) under
> Debian GNU/Linux as recommended by the Debian developers and I'm able to
> test them under Solaris 10 and in the future under some free BSD
> variants. Is this a good way to ensure POSIX compliance?

You may want to consider "posh" on debian which is meant to be a
pedantic shell with regards to standard compliance. However note
that the standard posh wants to comply to is not POSIX, it's
the debian standard which is mostly POSIX plus some extensions
(for instance, it differs on "echo" and on allowing the "local"
builtin).

But that only checks the shell syntax, not the utilities syntax
and is not fool-proof.

> How do you have learned it besides by a lot of scripting and testing?
[...]

Mostly through reading man pages (initially on Solaris), this
newsgroup (and other Unix related ones) and trying out. For
"POSIX" scripting, the only way is to refer to the standard.
I've not read many Unix shell scripting books, but most of them
either predate POSIX or don't give much consideration to
standards, and many times unfortunately teach bad practice.
I've never really seen a book about shell scripting in a POSIX
environment, but yet again, I've not looked hard. You find a lot
of bash or ksh or Bourne shell resources, but oddly enough not
many on the Unix sh and utilities programming. It's as if there
were only resources on gcc or the Borland C compile, and nothing
on the C language.

Sorry for not being very helpful here.

--
Stephane

rickm@galaxy.nsc.com

2007-12-10, 1:20 pm

I no longer get the "garbled" error but it passes everyting so
something is still not right

Rick

On Dec 7, 2:34 pm, Stephane Chazelas <stephane_chaze...@yahoo.fr>
wrote:
> On Fri, 7 Dec 2007 09:08:33 -0800 (PST), ri...@galaxy.nsc.com wrote:
>
>
> [...]
>
> Sorry, I should have used another character than "," for the
> separator as "," is used as well in the pattern, try with : for
> instance:
>
> sed 's:/modify$/.*$\{0,1\}$::'
>
> --
> Stephane

Stephane Chazelas

2007-12-10, 1:20 pm

On Mon, 10 Dec 2007 07:35:56 -0800 (PST), rickm@galaxy.nsc.com wrote:
[...correct order restored...]
[...][vbcol=seagreen]
> I no longer get the "garbled" error but it passes everyting so
> something is still not right

Are you sure you're using a POSIX/Unix conformant sed?

sed 's:/modify$/.*$\{0,1\}$::' << EOF
/path/to/modify/foo
/foo/modify
/foo/bar_modify
/bar/modify_foo
EOF

Gives me:

/path/to
/foo
/foo/bar_modify
/bar/modify_foo

Maybe you need "command -p sed" instead of "sed" to get the
portable (POSIX/Unix) version of the "sed" utility, like on
Solaris.

And if you do are on Solaris, you probably want to make sure
that you use a Unix shell interpreter instead of the Bourne
shell interpreter confusingly called "sh" and found in /bin.
(IOW, on Solaris, /bin/sh is a non-standard shell from another
era called the Bourne shell).

So

#! /usr/xpg4/bin/sh -
PATH=$(command -p getconf PATH;

export PATH

sed ...

--
Stephane

rickm@galaxy.nsc.com

2007-12-10, 1:20 pm

I tried it on a linux box and it does work as a line command. For
Solaris, if I create an executable
with the lines you sent and the command:

#! /usr/xpg4/bin/sh -
PATH=$(command -p getconf PATH;

export PATH
sed ...

then it also works. I was able to run as a line command on Solaris if
I use:

/usr/xpg4/bin/sed 's:/opus$/.*$\{0,1\}$::' cds.lib

THANKS!!!!!!

Rick

On Dec 10, 8:16 am, Stephane Chazelas <stephane_chaze...@yahoo.fr>
wrote:
> On Mon, 10 Dec 2007 07:35:56 -0800 (PST), ri...@galaxy.nsc.com wrote:
>
> [...correct order restored...]
>
> [...]
>
> Are you sure you're using a POSIX/Unix conformant sed?
>
> sed 's:/modify$/.*$\{0,1\}$::' << EOF
> /path/to/modify/foo
> /foo/modify
> /foo/bar_modify
> /bar/modify_foo
> EOF
>
> Gives me:
>
> /path/to
> /foo
> /foo/bar_modify
> /bar/modify_foo
>
> Maybe you need "command -p sed" instead of "sed" to get the
> portable (POSIX/Unix) version of the "sed" utility, like on
> Solaris.
>
> And if you do are on Solaris, you probably want to make sure
> that you use a Unix shell interpreter instead of the Bourne
> shell interpreter confusingly called "sh" and found in /bin.
> (IOW, on Solaris, /bin/sh is a non-standard shell from another
> era called the Bourne shell).
>
> So
>
> #! /usr/xpg4/bin/sh -
> PATH=$(command -p getconf PATH;

> export PATH
>
> sed ...
>
> --
> Stephane