Unix Shell - zsh pattern-matching makes no sense...

Author

zsh pattern-matching makes no sense...

2007-05-19, 1:20 pm

I posted the following query recently:

In <f2j0dj$r26$1@reader2.panix.com> kj <socyl@987jk.com.invalid> writes:

>Assuming that both the EXTENDED_GLOB and GLOB_DOTS options are
>enabled, what's the most concise zsh pattern that will match a
>directory foo/ and, recursively, all the files and directories
>under it.

>According to the zsh documentation, the pattern foo(/*)# *should*
>work, but...
>if one tries this pattern, zsh responds with a cryptic "bad pattern"
>error.

I learned from a reply by Jean-Rene David that this expression

foo/{,**/*}

accomplishes the task I described, but I remain completely baffled
by "patterns".

If I understand correctly, the last expression that Jean-Rene
proposed is not really a single "file generation pattern", but
rather a a shorthand for

foo/ foo/**/*

Since I'm trying to understand zsh patterns once and for all, and
eliminate all the crazy guessing and trial-and-error I have to do
when I attempt to use them, I'd like solve the problem using a
single pattern.

The documentation (man zshall) is wrong or, at best, very incomplete.
For example, it says that * "matches any string, including the null
string", but the pattern foo/* does NOT match foo/ (and yet the
pattern foo* does match foo).

There's a huge number of patterns that I can come up with that
should be valid according to these docs, but that zsh rejects as
invalid. These include:

foo(/*)#
foo/(*/*)#
foo/(|**/*)
foo/(|(*/)#*)

....and a bazillion more I won't bore you with.

Also, I can't find anything in the documentation that would explain
why foo(/*)# is invalid, but foo/(*/)# is OK.

Is there a more complete and accurate description of the zsh pattern
syntax than what one can find in the zshall man page? Alternatively,
is there a way to get more informative error messages from zsh?

Thanks!

kj

--
NOTE: In my address everything before the first period is backwards;
and the last period, and everything after it, should be discarded.

Stephane CHAZELAS

2007-05-19, 7:17 pm

Hi, what you need to understand is that globbing matches file
names.

* is the pattern that matches any sequence of characters. But
when it comes to globbing, there are two steps involves:

1- come up with a list of file names
2- apply the pattern to that list and only retain those that
are matched.

With "*", the file list is built as the list of the names of the
files in the current directory.

"/" is special in globbing patterns, that is well described in
the manual, and it's the same for every shell.

"*" doesn't match "foo/bar", the shell doesn't even have a look
at the content of "foo" to look for files matching "*". The
manual will say that "/" must be matched explicitely, but it
helps if you also understand that the "/"s in the pattern will
also tell the shell, whose directory to read the content to make
a list of file paths to apply the pattern against.

In:

*foo*/*bar*, the shell will first find the directories (and only
directories) that match "*foo*" and then list their content and
apply the other pattern.

You can't do

(foo|bar/baz)/*.txt, that's described in the manual.

The "/", with its special status of telling the shell where to
look for files can't be put anywhere.

To allow for recursive globbing, zsh (and only zsh) has a well
defined exception, it's:

(<pattern>/)#foo
or
(<pattern>/)##<glob>

this also is described in the manual.

Above that tells zsh to recursively look for directories
matching the pattern and apply the <glob> on their content.

foo(*/)# is not of that special kind, it's not supported by zsh.

You could do:

**/*~^foo(|/*)

as what's after ~ is considered as a pattern that is applied to
the final list, it's not globbing, but it won't be efficient in
that every subdirectory of the current will be searched, not
only "foo".

If you read the manual again, you'll see that it's explained,
I'm not inventing it.

--
Stéphane

Stephane CHAZELAS

2007-05-19, 7:17 pm

2007-05-19, 17:39(+00), kj:
[...]
> Also, I can't find anything in the documentation that would explain
> why foo(/*)# is invalid, but foo/(*/)# is OK.
[...]

(...)
Matches the enclosed pattern. This is used for grouping. If the
KSH_GLOB option is set, then a `@', `*', `+', `?' or `!'
immediately preceding the `(' is treated specially, as detailed
below. The option SH_GLOB prevents bare parentheses from being
used in this way, though the KSH_GLOB option is still available.

Note that grouping cannot extend over multiple directories: it is
an error to have a `/' within a group (this only applies for
patterns used in filename generation). There is one exception: a
group of the form (PAT/)# appearing as a complete path segment can
match a sequence of directories. For example, foo/(a*/)#bar
matches foo/bar, foo/any/bar, foo/any/anyother/bar, and so on.

> Is there a more complete and accurate description of the zsh pattern
> syntax than what one can find in the zshall man page? Alternatively,
> is there a way to get more informative error messages from zsh?
[...]

man is not really appropriate for a manual this size, I would
recommand you learn how to use "info" efficiently (look at its
"i" and "g" commands combined with completion), or read the
manual online in HTML with the help of some search engine.

--
Stéphane