Unix Shell - Reintroducing fish, the friendly interactive shell

Author

Reintroducing fish, the friendly interactive shell

liljencrantz@gmail.com

2006-03-06, 5:54 pm

Hi,

A little more than a year ago, I posted a message to this group about
fish, a new shell I'd written. (See
http://groups.google.se/group/comp....ba776d029e871dd)

At the time, fish was what I would describe as a program with some nice
interactive features, and a quite lousy language syntax. Much of the
focus of the following discussion on this list focused on the syntax,
not on the new interactive features, such as syntax highlighting,
advanced tab completions, etc.. The new UI features had been the main
focus of my efforts so far, which is why the shell syntax was in the
state it was. In retrospect, I did not make it clear that while there
where some interesting new syntax ideas, the syntax itself should have
been considered a placeholder. It should also be noted that during the
course of the original conversation, it was demonstrated to me that zsh
already contained several of the features that I thought where specific
to fish, only they where not enabled by default and the way to enable
them was not obvious to me. Fish still has plenty of unique UI
features, though.

Since then, I've spent a large amount of time hacking on fish, and many
others have also helped out with patches and opinions and counter
arguments, and the fish shell is very different from how it was one
year ago. Most of these changes are related to the fish syntax. When
rewriting the fish syntax, I had the choice of either using a
Posix-like syntax, using the somewhat less broken rc syntax or doing
something completely new. Given the number of bike sheds in the world,
and the well known NIH-syndrome, it should not surprise anyone that I
chose the latter option, though I have tried to use the Posix syntax in
all places where I do not think it is completely broken. There are a
small number of changes that mostly change something that is not
really, broken, where I simply thought the new syntax to be a bit
better. Perhaps these where mistakes, I am unsure. The most noticable
such change is the use of () instead of $() for command substitutions.
Still, I hope that when reading up on the fish syntax, people will
agree with me that overall going for a new syntax was the right thing
to do. I am very curious about what the people in this group think
about fish the way it works now. Here are the main syntax features of
fish compared to Posix-like shells like bash:

No Macro language pretence:

Two of the most common issues with Posix shells I've seen are caused by
the roots of shells as macro expansion languages, even though at least
bash is not internally implemented as a macro expansion language. The
first of these is the ever present requirement of using quotes around
variables to avoid argument separation on spaces. This is just silly,
it is almost never what you want, and in the few rare cases where it
is, it can be achived with a command substitution and a pipe through
tr. So fish never separates arguments on spaces in variable values.

The second such common example is when doing something like:

bash> FOO=BAR
bash> FOO=BAZ; echo $FOO
BAR

This makes a lot of sense if you consider the shell as a macro
expansion language that works on a line-by-line basis, but that does
not make it a good syntax.

The fact that bash has fixed the second of these two misfeatures, but
not the first just makes things worse in my opinion, since it makes it
clear that bash is not a macro expansion language, it is simply a
language that pretendes to be one. In my opinion, it makes the language
unpredictable, since you never know without testing if bash will behave
like a macro exapnder or not without trying.

If I'm not mistaken (I often am when it comes to zsh, I'm sorry in
advance for all the misrepresentations of zsh that I'm sure exist in
this post), zsh allows you to choose how it behaves in both these
cases. This is a recuring theme with zsh, at least it seems that way to
me. The philosophy of always answering the question 'Which of these two
paths should we choose?' with 'Both!' is suboptimal as I see it. I
don't think that it is a good idea to make the syntax of programming
languages configurable. If you do, you can't safely run a piece of
shellscript without checking that the scripter used the same settings
as you do. Also, the zsh approach means that everyone has to take the
time to tweak every little detail of the shell to their liking (since
the defaults are rarely very good), and that if they are stuck on a
foreign host, they will be lost since everything will be tweaked all
wrong. In my opinion, it is often better to choose one way of doing
something and sticking with it. If it turns out to be the wrong way (A
conclusion that should _never_ be reached without a great deal of
thought), implement a new one and eventually _remove_ the old, bad way
of doing it.

Scoping:

Posix shells have no notion of local variables, and instead use
subshells to get something more or less like read-only variables in
some situations. Unless I'm mistaken, bash allows you to create local
variables as an extension, but the main way of providing variable scope
is still to use subshells, either through () or using command
substitution or simply by using a pipeline. It should also be noted
that Posix does not define the behaviour of pipelines with regards to
forking of subshells, and that bash and zsh do this differently.
Specifically, the following works in zsh but is meaningless in bash:

cat file.txt|read contents

In fish all variables have scope. By default, when creating a new
variable the scope is local to the currently running function, or if no
function is running, the scope is global. You can explicitly set the
scope of a variable using -g (global) or -l (local to the current block
of commands). There are _no_ subshells in fish, i.e. fish never forks
of a subprocess other than to execute a command in it. This means that
you can freely change variable values inside command substitutions and
pipes, and these changes will be visible in the shell proper.

I think that while some people may be used to them, the traditional
shell scoping rules are both error prone (since large shellscripts will
often have problems with variable name clashes because most variables
end up as globals) and limiting (since you often can not rely on
variable writes being permanent - if you do you will limit your
function to never be used in pipelines or in command substitutions),
and that the type of scoping provided in fish, as well as nearly every
other procedural or object oriented language on the planet, is often
both safer and more powerful. As an extra bonus, many people use other
languages with similar scoping rules to fish, meaning that they will
already be familiar with the fish scoping rules.

Block commands:

Fish changes the way blocks of code are defined. Examples:

if foo; then bar; fi -> if foo; bar; end
case $foo; a) bar;; *) baz;; esac -> switch $foo; case a; bar;
case '*'; baz; end
foo(){ bar; } -> function foo; bar; end

You will notice that 'fi', 'esac', 'done', '}' and all the other block
end commands have been replaced by 'end'. The updated block syntax is
inspired by Matlab and Lua. The really major benefit, in my opinion, is
with function definitions. Quickly now, which of the following are
legal function definitions?

hello () {echo hello }
hello () {echo hello;}
hello () {;echo hello }
hello () {;echo hello;}
hello () { echo hello }
hello () { echo hello;}
hello () { ;echo hello }
hello () { ;echo hello;}

The answer is - only number six. At least in my book, that is proof of
a completely broken syntax.

Fish also drops words like 'then' and 'do', which make the code look
less like code and more like english at the expense of brevity and
consistence. I have never found these extra keywords to be more helpful
than 'please' is in Intercal.

Everything is a command:

Here are a few more fish syntax changes:

foo=bar -> set foo bar
make && make install -> make; and make install

These changes exist to make the fish syntax more consistent.
Specifically, in fish pretty much everything is a command. Variable
assignemnts, loops and conditionals are all regular builtin commands. I
firmly belive that by making as much of the syntax as possible obey the
same rules, the language becomes more predictable and easier to learn.
It could be argued that this is trying to fit a round peg in a square
hole, bit I don't think that is the case, I've found that all these
tasks are very suitable for builtin commands. Also, it is much easier
to get help on a command than on some weird piece of syntactic sugar.
Want to know how to use the 'set' builtin? type 'set --help' and you'll
get it.

Nicer array variables:

In fish, all variables are really arrays. To define a variable 'foo'
with the elements 'a', 'b' and 'c', simply use

set foo a b c

$foo is expanded to all the elements of the array as separate elements.
Use [] to acces an element in the array, e.g. $foo[1]. You can specify
multiple elements, e.g. $foo[1 3], and you can use the seq command
inside a command substitution to slice the array, e.g. $foo[(seq 2)].
You can also set and erase parts of the array. For example, the
following loops over the $argv vector until it is empty

while count $argv >/dev/null
switch $argv[1]
...
end
set -e foo[1]
end

Compare this with the bash syntax where you use syntax that is
non-obviously (to me, at least) different from the regular variable
syntax ro reference and assign arrays, and it is hardly surprising that
very few people seem to use arrays in bash.

An extra bonus in fish is that all variables inherited from the parent
process are turned into arrays using ':' as the array separator. That
means that PATH, CDPATH, LS_COLORS and various other lists are treated
as arrays by fish. When exported to subcommands, all arrays are of
course concatenaded together using ':' as the join character.

Universal variables:

Since the dawn of time, clueless users have asked questions like 'how
can I change an environment varible in another running process?' The
answer has always been variations of 'You don't.' No longer so in fish.
Fish supports universal variables, which are variables whose value is
shared between all running fish instances with the specified user on
the specified machine. Universal variables are automatically saved
between reboots and shutdowns, so you don't need to put their values in
an init file. Universal variables have the outermost scope, meaning the
will never get used in preference of shell-specific variables, which
should minimize security implications.

Universal variables make it much more practical to use environment
variables for configuration options. Youy simply change an environemnt
variable in one shell, and the change will propagate to all sunning
shells, and it will be saved so that the new value is used after a
reboot as well. One example of environemnt variables in action can be
had by launching two fish instances in separate terminals side-by-side.
The issue the command 'set fish_color_cwd blue' and the color of the
current working directory element of the prompt will change color to
blue in both shells. Using universal variables makes it much more
convenient to set configuration options like $BROWSER, $PAGER and
$CDPATH.

Events:

Fish allows you to trigger a shellscript function at a specific time,
such as at the completion of a specific job or process, when a specific
variable changes value or when a specific signal is recieved. The
syntax for this is:

function winch_handler --on-signal WINCH; echo WINCH; end
function browser_handler --on-variable BROWSER; echo new browser is
$BROWSER; end
function process_exit_handler --on-process 123; echo process 123 died;
end

For example, bash process substitution is not natively supported by
fish, but a workalike, implemented in shellscript, is included with
fish:

function psub
# By setting the variables here, we set their scope to function local.
# This means they won't overwrite any existing global variables with
# the same name.
set -l filename
set -l funcname

# Find unique file name for writing output to
while true
set filename /tmp/.psub.(echo %self).(random);
if not test -e $filename
break;
end
end

mkfifo $filename
cat >$filename &
echo $filename

# Find unique function name
while true
set funcname __fish_psub_(random);
if not functions $funcname >/dev/null ^/dev/null
break;
end
end

# Make sure we remove fifo when caller exits
eval "function $funcname --on-job-exit caller; rm $filename; functions
--erase $funcname; end"

end

The above allows you to replace this

diff <(sort foo.txt) <(sort bar.txt)

with

diff (sort foo.txt|psub) (sort bar.txt|psub)

This is very slightly less efficient, since the output is filtered
through the cat command, but it never touches disk since a fifo is
used, and all commands can run concurrently, so the eficcency is still
more than acceptable.

A workalike of the Posix 'trap' builtin is also implemented as a
shellscript wrapper around event handlers. Being able to trigger
function calls on diverse types of events is, in my opinion, a rather
powerful language feature, and it is one I hope to extend with new
trigger types in the future.

Error reporting:

Fish tries to help the user by being verbose and specific in it's error
messages. Here are a few examples:

Trying to use Posix-style variable assignment tells you how to use fish
style variable assignments:

fish> foo=bar
fish: Unknown command 'foo=bar'. Did you mean 'set VARIABLE VALUE'?
For information on setting variable values, see the help section on
the set command by typing 'help set'.

Using Posix short circut operators, Posix style command substitution
and various other features that use a different syntax in fish will
also give such error messages with pointers to how to do the same thing
in fish.

Fish also provides you with stack traces on errors:

fish> . foo.fish
fish: Unknown command 'abc123'
/home/axel/code/c/fish_current/foo.fish (line 5): abc123
^
in function 'do_something',
called on line 7 of file
'/home/axel/code/c/fish_current/foo.fish',

in . (source) call of file '/home/axel/code/c/fish_current/foo.fish',
called on standard input,

Autoloaded functions:

Fish functions and command specific completions can be placed in
special directories, specified using the environment variables
$fish_function_path and $fish_complete_path. Such files are autoloaded
on the first invocation of completions for the command, or the first
invocation of the function. The modification time of each such file is
also tracked, so that if the file is changed, or a new one is added in
a directory with higher priority in the path, the relevant file is
reloaded. This allows one to write a huge amount of code in shellscript
without increasing either memory usage or startup time and more
importantly, without forcing the user to turn on specific features
manually. Features that aren't used will not take up memory or
processing power. Fish ships with many thousand lines of shellscript
code, but only a few hundered lines are run on startup.

Dropped features:

Fish tries to consolidate multiple strongly related Posix features into
one. One such consolidation is dropping dollar-quotes like $'\n' in
favor of allowing backslash escapes in regular strings, e.g. \n and
\x20 both work as you would expect. Other dropped features include
subshells (use fish scoping or a block of commands), math mode (use the
bc or expr commands), here documents (Use quotes or, in order to paste
large amounts of data possibly containing quotes and other illegal
characters into the shell, use ^Y to paste from the X clipboard) and
process substitution (Use the psub workalike described above).

Failiures:

While implementing new syntax features for fish, a great many different
ideas where tried out. Most of the ideas that I tried out turned out to
be bad ones, and they where dropped. I have often heard the argument
against chaing the Posix syntax that the original designers knew what
they where doing, and that chaning things will only make them worse.
This is true in the sense that if one would simply try out every gret
new idea one gets, and then make it stick around forever, even if it
turns out that the idea was a bad one, things will degenerate quickly.
But one _can_ to some extend separate the wheat from the chaff. I have
presented some of what I concluded to be wheat above. For those that
are interested, here follows some of the chaff:

Originally, when fish encountered a wildcard that had no matches, it
would silently remove that argument. This makes a huge amount of sense
sometimes, eg. when doing things like 'for i in *.txt', but does not do
what you would like when using e.g. 'ls *.txt'. I do not like the bash
method of leaving such arguments unexpanded, since it relies on the
called commands to detect the wildcard and try to report a meaningful
error. This can lead to very confusing behaviour. Instead, I've opted
to follow the same path chosen by csh, namely if all waldcards to a
specific command fail to match, the command is not executed. In
interactive mode, a warning is printed as well. This is just one of the
many things I feel csh got right. Too bad that the number of things csh
got horribly wrong is even greater...

Originally, fish made no difference between single quotes and double
quotes, they both turned of all types of argument expansion. I argued
that to embed strings into a quoted string, one quold simply use
printf. I aslo rargued that mising the two mixes code with text, which
is bad. I argued that it is confusing to have two types of quotes,
which are similar but not identical. In the end, I allowed variable
expansion in double quotes, because it saves keystrokes, because it
allwos you to make sure array variables get expanded into exactly one
token, no matter how many elements the array contains, and it allows me
get much more time for hacking instead of answering the same question
over and over again.

Originally, fish did not have any blocks at all. Instead, code was
given as arguments to other commands; e.g. the last argument to a for
loop was the list of commands to run in the loop. This is easy to
implement, but completely useless for longer scripts.

Closing comments:

There are a great many features in fish that have little to do with
syntax, like syntax highlighting, advanced tab completion, X clipboard
intergration, etc.. But this post is only mean to discuss the design
and implications of the changes made to regular shell syntax in fish.
Specifically, I'd be interested in opinions on security considerations,
regressions and further possible changes to the syntax. To try out
fish, visit http://roo.no-ip.org/fish/ or use the prepackaged version
avaialable for many systems including Debian. Fish is GPL:ed, and it
works on most Linux versions, NetBSD, FreeBSD, OS X, Solaris and
possibly Cygwin.

--
Axel

Jordan Abel

2006-03-06, 5:54 pm

On 2006-03-06, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
> The second such common example is when doing something like:
>
> bash> FOO=BAR
> bash> FOO=BAZ; echo $FOO
> BAR

On what shell does this happen? Even on the original unix v7 bourne
shell, this isn't true. The unix v6 shell didn't support variables at
all.

liljencrantz@gmail.com

2006-03-06, 5:54 pm

Jordan Abel skrev:

> On 2006-03-06, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> On what shell does this happen? Even on the original unix v7 bourne
> shell, this isn't true. The unix v6 shell didn't support variables at
> all.

You are of course completely right, I got confused. I meant:

bash> FOO=BAR
bash> FOO=BAZ echo $FOO
BAR

Which is something different. And I notice that bash does indeed _not_
stray from the traditional output of BAR in the above situation. My
bad.

My point about macro expanding languages having unintuitive behaviour
still stands, though.

--
Axel

Jordan Abel

2006-03-06, 8:48 pm

On 2006-03-06, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> Jordan Abel skrev:
>
>
> You are of course completely right, I got confused. I meant:
>
> bash> FOO=BAR
> bash> FOO=BAZ echo $FOO
> BAR
>
> Which is something different. And I notice that bash does indeed _not_
> stray from the traditional output of BAR in the above situation. My
> bad.

That's because you misunderstand what the VAR=value cmd syntax is for -
it doesn't set a shell variable at all - it puts a variable in the
command's environment. The real problem is in the half-assed equivalence
between shell variables and environment variables.

try

FOO=BAR
FOO=BAZ env | grep FOO

> My point about macro expanding languages having unintuitive behaviour
> still stands, though.

It's only unintuitive when you misread the intent of a construct.

liljencrantz@gmail.com

2006-03-06, 8:48 pm

Jordan Abel skrev:

> On 2006-03-06, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> That's because you misunderstand what the VAR=value cmd syntax is for -
> it doesn't set a shell variable at all - it puts a variable in the
> command's environment. The real problem is in the half-assed equivalence
> between shell variables and environment variables.
>
> try
>
> FOO=BAR
> FOO=BAZ env | grep FOO
>

I understand why that happens. I got a bit confused before since I
don't use Posix shells enough these days, but I know _why_ the above
code does what it does. I just don't think it's a good syntax.

>
> It's only unintuitive when you misread the intent of a construct.

You yourself stated, 'The real problem is in the half-assed equivalence
between shell variables and environment variables'. My point isn't that
the Posix syntax here is somehow wrong, given the abstraction of a
macro language, it makes perfect sense. But that does not make it a
_good_ syntax. Macro languages simply aren't very useful when compared
to regular procedural languages, in my experience.

As an extreme example of the same phenomenon, consider the Intercal
notion of reversing the order of the bits in a byte on output. It makes
perfect sense once you accept that Intercal uses the abstraction of
loking at IO as passing messages on notes and throwing them backwards.
If you turn your head and look at the things you've written, they will
be in reverse order because you've turned your head 180 degrees. But
from the outlook of a programmer actually wanting to write code, this
is a counter productive abstraction which makes Intercal a horrible
language to use.

--
Axel

bsh

2006-03-07, 2:49 am

liljencrantz@gmail.com wrote:
> ...

Thank you for another shell resource; I'm updating my record
of "fish" since the last posting to C.U.S. which I remember
well. Your points are well taken, even if bourne-like shell
scripting has only acceptably well "weathered" the subsequent
syntactic supersets applied to it over the many years.

However, a few quibbles:

> Posix shells have no notion of local variables ...

I want to have noted that the modus operandi of POSIX is to
not mandate features, but minimal functionality. The distinction
for this instance is that local variable scopes can be provided
by a superset of the POSIX 1003.2 that does not break required
standards.

> Quickly now, which of the following are legal function definitions?
> hello () {echo hello }
> hello () {echo hello;}
> hello () {;echo hello }
> hello () {;echo hello;}
> hello () { echo hello }
> hello () { echo hello;}
> hello () { ;echo hello }
> hello () { ;echo hello;}
> ... that is proof of a completely broken syntax.

(Number 4 should also work....)

This is completely untrue. Although the explanation is beyond
the scope of the discussion (and our patience...), the grouping
keyword ("}") is commensurate with the other terminating
keywords "esac", "done", and "fi", and as such require being on
their own line or after another keyword.

For instance, I've written scripts that end with one line:
"} esac done fi" which is documented syntax back to Sys6 sh(1).

Granted, the syntactic hoops that must be jumped through are
more than found in shells rc/akanga/es, because these are written
in lex/yacc. k/sh is written as a triple-pass recursive-descent
parser, with _separate_ parsing rules for variable-lists and
redirection-operators; however, the syntax is regular enough
that I have written a parser for k/sh(1) that accepts _all_
legal constructs, whatever the contextual quoting rules
(such as whether quote-removal occurs or not within parameter
expansion).

Incidentally, the fact that bash(1) allows
"[function] hello [()] { echo hello[;] }" is provided for the
programmer's convenience.

> Fish supports universal variables....

Potential unmaintable code and intermittent bug alert!
Speaking for myself, I've found and used two methods
that allow the "migration" of the environment to-and-from
different processes/jobs, which is to say that the
attractiveness of scripting is that any given solution
might be had, even if it is not elegant.

=Brian

liljencrantz@gmail.com

2006-03-07, 2:49 am

bsh wrote:

> liljencrantz@gmail.com wrote:
>
> Thank you for another shell resource; I'm updating my record
> of "fish" since the last posting to C.U.S. which I remember
> well. Your points are well taken, even if bourne-like shell
> scripting has only acceptably well "weathered" the subsequent
> syntactic supersets applied to it over the many years.

Than you for your kind words. I'd say that many of the extensions
provided over the years have been implemented in rather inconsistent
ways. If functions, indirect variable expansion, arrays, local
variables and other new features had been better integrated into the
shell, the language would, in my opinion, have been in a much better
condition today.

>
> However, a few quibbles:
>
>
> I want to have noted that the modus operandi of POSIX is to
> not mandate features, but minimal functionality. The distinction
> for this instance is that local variable scopes can be provided
> by a superset of the POSIX 1003.2 that does not break required
> standards.

I belive I mentioned that bash provides local variables in my original
post. But the fact that you have to explicitly declare each variable as
such increases the likelyhood slipups which lead to bugs. To support
this argument, one simply needs to look at some real-world scripts. I
can find lots of places where the 'local' builtin is not used when it
should be in the startup scripts of my fedora machine.

>
>
> (Number 4 should also work....)

It doesn't in bash:

bash$ hello () {;echo hello;}
bash: syntax error near unexpected token `;'

>
> This is completely untrue. Although the explanation is beyond
> the scope of the discussion (and our patience...), the grouping
> keyword ("}") is commensurate with the other terminating
> keywords "esac", "done", and "fi", and as such require being on
> their own line or after another keyword.

If I understand you correctly, your point is that '}' should be viewed
as a command-like keyword, like 'esac' and 'fi'. And if you do so, it
will indeed seem more intuitive. But that is beside the point for me,
since one would not expect '}' to behave that way because:

* The syntax is clearly borrowed/inspired by from C-like languages
where } does not behave like a function or reserved word.
* One does not expect alphabetical and non-alphabetical keywords to
follow the same rules. Notice that none of '&', '&&', '|', ';', ';;',
'`' and ')' require a preceeding newline or other such niceties while
'fi', 'done', 'esac' and friends do. The only exceptions I can think of
are '{' and '}'.
* The syntax strongly resembles () and to some degree also $(), i.e.
subshells and command substitutions, neither of which use ')' as a
regular command.

To me, the latter is a _very_ solid reson to expect '}' to behave in a
very different way. This signals that the syntax is tacked on as an
afterthought, rather than a well thought out design. And unless I'm
mistaken, this syntax was borrowed from rc, so this impression is
indeed to some extent correct.

>
> For instance, I've written scripts that end with one line:
> "} esac done fi" which is documented syntax back to Sys6 sh(1).

Yes it seems bash accepts sequences like that. As I would expect it to.

>
> Granted, the syntactic hoops that must be jumped through are
> more than found in shells rc/akanga/es, because these are written
> in lex/yacc. k/sh is written as a triple-pass recursive-descent
> parser, with _separate_ parsing rules for variable-lists and
> redirection-operators; however, the syntax is regular enough
> that I have written a parser for k/sh(1) that accepts _all_
> legal constructs, whatever the contextual quoting rules
> (such as whether quote-removal occurs or not within parameter
> expansion).

I did not know ksh needs three parsing passes. The fish parser uses
only a single pass, and the parser is <3000 lines long, with another
<1000 for the tokenizer. I have tried to make the fish syntax easy to
parse, partly based on the assumption that if it easy for the computer
to parse something, it is probably easy for a human. While this
assumption is not always correct, I think it applies here. Fish
provides much fewer features than other shells, but the features are
more orthogonal and designed to still give you the same expressive
power.

>
> Incidentally, the fact that bash(1) allows
> "[function] hello [()] { echo hello[;] }" is provided for the
> programmer's convenience.

I'm not following you here.

>
>
> Potential unmaintable code and intermittent bug alert!
> Speaking for myself, I've found and used two methods
> that allow the "migration" of the environment to-and-from
> different processes/jobs, which is to say that the
> attractiveness of scripting is that any given solution
> might be had, even if it is not elegant.

I was actually surprised at how little code was required for this, less
than 2000 lines of code, with a fair number of comments in them. And
the code does not use anything more advanced than standard Unix
sockets, either. Hopefully this removes most of your fears about
unmaintable code and intermittent bugs.

I interpret the rest of your comment as a preference towards allowing
multiple unelegant solutions rather than a single, elegent genreal
purpose one. Not that I agree, but if that is your opinion, there is
nothing preventing you from using some other method as well for sharing
variable values. That fact that fish has universal variables does not
force you to use them, and nor does it make it impossible to implement
alternative schemes for doing the same thing.

>
> =Brian

--
Axel

Jordan Abel

2006-03-07, 2:49 am

On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> Jordan Abel skrev:
>
>
> I understand why that happens. I got a bit confused before since I
> don't use Posix shells enough these days, but I know _why_ the above
> code does what it does. I just don't think it's a good syntax.
>
>
> You yourself stated, 'The real problem is in the half-assed equivalence
> between shell variables and environment variables'. My point isn't that
> the Posix syntax here is somehow wrong, given the abstraction of a
> macro language, it makes perfect sense. But that does not make it a
> _good_ syntax. Macro languages simply aren't very useful when compared
> to regular procedural languages, in my experience.

They're good when the main purpose is to put together a bunch of
routines whose argument set is a list of strings.

Jordan Abel

2006-03-07, 2:49 am

On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
> If I understand you correctly, your point is that '}' should be viewed
> as a command-like keyword, like 'esac' and 'fi'. And if you do so, it
> will indeed seem more intuitive. But that is beside the point for me,
> since one would not expect '}' to behave that way because:
>
> * The syntax is clearly borrowed/inspired by from C-like languages
> where } does not behave like a function or reserved word.

It might have been better, yes, for something like "begin" and "end" to
be used, particularly given how enamored Bourne otherwise was with
ALGOL (when you #define BEGIN { #define END ;} in the shell's _source_,
you ought to be using those keywords for the language defined by the
shell)

Stephane CHAZELAS

2006-03-07, 2:49 am

2006-03-6, 17:26(-08), liljencrantz@gmail.com:
[...]
>
> I understand why that happens. I got a bit confused before since I
> don't use Posix shells enough these days, but I know _why_ the above
> code does what it does. I just don't think it's a good syntax.

It's a direct mapping to the execve system call. It couldn't be
more intuitive if you think of it that way. And had it output
BAR above, I would have found it counter-intuitive.

[...]
> You yourself stated, 'The real problem is in the half-assed equivalence
> between shell variables and environment variables'. My point isn't that
> the Posix syntax here is somehow wrong, given the abstraction of a
> macro language, it makes perfect sense. But that does not make it a
> _good_ syntax. Macro languages simply aren't very useful when compared
> to regular procedural languages, in my experience.

I don't think shells ever claimed to be macro languages. How do
you define a macro language yourself.

You would have had a point if you had chose as an example:

alias a='echo a'
alias a='echo b'; b

--
Stéphane

Stephane CHAZELAS

2006-03-07, 2:49 am

2006-03-6, 18:55(-08), bsh:
[...]
>
> I want to have noted that the modus operandi of POSIX is to
> not mandate features, but minimal functionality. The distinction
> for this instance is that local variable scopes can be provided
> by a superset of the POSIX 1003.2 that does not break required
> standards.

But then a POSIX script can't use them as the behavior is
unspecified. There's no need to have a POSIX conformant shell if
it's to write non-POSIX conformant scripts. That doesn't help
with portability.

I find that a shell can't be at the same time a good shell and a
good programming language. There are conflicting purposes and
means between both (the fact that a shell has to be human
oriented, for instance). So, I find it acceptable that it has
limited programming capabilities, but I find it important that
there exists a standard that still allows to write portable
shell programs as those can reveal useful sometimes.

>
> (Number 4 should also work....)

Well "{" is supposed to be followed by a command there, just as
"then".

> This is completely untrue. Although the explanation is beyond
> the scope of the discussion (and our patience...), the grouping
> keyword ("}") is commensurate with the other terminating
> keywords "esac", "done", and "fi", and as such require being on
> their own line or after another keyword.

Note that it's not the case of zsh, where

hello() {echo hello}

is valid.

[...]
> Granted, the syntactic hoops that must be jumped through are
> more than found in shells rc/akanga/es, because these are written
> in lex/yacc. k/sh is written as a triple-pass recursive-descent
> parser, with _separate_ parsing rules for variable-lists and
> redirection-operators; however, the syntax is regular enough
> that I have written a parser for k/sh(1) that accepts _all_
> legal constructs, whatever the contextual quoting rules
> (such as whether quote-removal occurs or not within parameter
> expansion).

That sounds great, have you made it available?

--
Stéphane

Stephane CHAZELAS

2006-03-07, 2:49 am

2006-03-6, 20:16(-08), liljencrantz@gmail.com:
[...]
> Than you for your kind words. I'd say that many of the extensions
> provided over the years have been implemented in rather inconsistent
> ways. If functions, indirect variable expansion, arrays, local
> variables and other new features had been better integrated into the
> shell, the language would, in my opinion, have been in a much better
> condition today.

I couldn't agree more.

[...]
>
> If I understand you correctly, your point is that '}' should be viewed
> as a command-like keyword, like 'esac' and 'fi'. And if you do so, it
> will indeed seem more intuitive. But that is beside the point for me,
> since one would not expect '}' to behave that way because:
>
> * The syntax is clearly borrowed/inspired by from C-like languages
> where } does not behave like a function or reserved word.
[...]

That could be a backward portability issue.

echo }

outputs "}" (it's not the case in zsh except in sh/ksh modes).

--
Stéphane

liljencrantz@gmail.com

2006-03-07, 7:49 am

Jordan Abel skrev:

> On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> It might have been better, yes, for something like "begin" and "end" to
> be used, particularly given how enamored Bourne otherwise was with
> ALGOL (when you #define BEGIN { #define END ;} in the shell's _source_,
> you ought to be using those keywords for the language defined by the
> shell)

I was under the impression that the function syntax was not part of the
original Bourne shell, but rather an addition originating from rc. But
I may be wrong.

And as you say, a keyword like 'end' might be more suitable, which is
exactly why fish uses the keyword 'end'.

--
Axel

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Stephane CHAZELAS skrev:

> 2006-03-6, 17:26(-08), liljencrantz@gmail.com:
> [...]
t_[vbcol=seagreen]
r -[vbcol=seagreen]
ce[vbcol=seagreen]
>
> It's a direct mapping to the execve system call. It couldn't be
> more intuitive if you think of it that way. And had it output
> BAR above, I would have found it counter-intuitive.

The problem I have with this is that a direct mapping to a low-level
syscall is not a suitable abstraction to use as the main invocation of
commands in a high-level scripting language. Instead of trying to
provide the user with an thin execve wrapper, it might be better (in my
opinion) to focus on how to design a good _language_. A language where
variable scope makes sense from a language point of view and not an
implementors point of view. Specifically, from a language point of
view, the command

FOO=3DBAR echo $FOO

is intuitively viewed as being evaluated in the scope of that single
command. The parsing of the command, with substitutions lof variables
like $FOO are intuitively done in the scope of that command. In
reality, the parsing is done by the shell and only after the parsing is
done is a child forked of, but that's just implementation.

>
> [...]
>
> I don't think shells ever claimed to be macro languages. How do
> you define a macro language yourself.

Well, that is a bit of antropomorphisation on my part. The shells don't
really talk to me. But according to Wikipedia "A macro language is a
programming language in which all or most computation is done by
expanding macros", which is very much true for shells. Variable
substitution, wildcard substitution, alias substitution and many other
language features work through macro expansion or an emulation of macro
expansion.

>
> You would have had a point if you had chose as an example:
>
> alias a=3D'echo a'
> alias a=3D'echo b'; b

Typo. I think you meant

alias a=3D'echo a'
alias a=3D'echo b'; a

which will output 'a', and not 'b'. But you are completely right. That
is a _much_ better example of the macro-like properties of shells, and
one that I really feel should be removed. I noticed that this is one is
not fixed in zsh either, at least no by default. It is also one which
is removed in fish, where writing

function a; echo a; end; a
function a; echo b; end; a

will output 'a', not 'b'.

Fish only has functions, no aliases. One could of course add aliases by
doing something like:

function alias --description "A wrapper for providing function
definitions using a weaker, but simpler syntax"
set -l name $argv[1]
set -l body $argv[2]
eval "function $name; $body \$argv; end"
end

alias ll "ls -l"

but that is beside the point.

I chose bad examples in my original post, I obviously haven't been
using enough shellscripts recently to clearly remember the things I
dislike the most. Thank you for the correction.

>=20
> --=20
> St=E9phane

--=20
Axel

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Stephane CHAZELAS skrev:

> 2006-03-6, 20:16(-08), liljencrantz@gmail.com:
> [...]
>
> I couldn't agree more.

Glad to hear we agree here. So, aside from the 'FOO=3DBAR echo
$FOO'-question, where we seem to see things differently, what are your
feelings on e.g. the fish array syntax?

>
> [...]
> [...]
>
> That could be a backward portability issue.
>
> echo }
>
> outputs "}" (it's not the case in zsh except in sh/ksh modes).

As you mentioned in another post, zsh is much more clever in it's
parsing of ';' and '}'. It may be that there are some awkward edge
cases where there is a clash between braces used to denote blocks and
braces used fo brace expansion, but overall the zsh syntax makes _much_
more sense to me. That said, there is of course still the issue of
using all these keywords for denoting end-of-block:

fi, esac, done, }, ;;

When testing around in zsh, the only thing that I felt should be
allowed which wasn't was the use of multiple ';' without any whitespace
between them. It seems that ';;' is interpreted as the case-end
keyword, and as such will lead to a parse error. That means that one
can legally write an empty command like this:

;

But one can't write two empty commands on a single line like this:

;;

which is of course a very minor detail all things considered. I mention
this mostly since I think that it makes a lot of sense from a language
point of view to minimize the difference between a ';' and a newline.
It is legal to have any number of newlines with no whitespace, so from
that perspective it makes sense to do the same with ';'. This is an
exact parallel to how spaces, tabs and newlines are exactly equivalent
outside of string literals in C. In fish ';' and newlines are
syntactically equivalent in none-quoted contexts, so you can write
';;;;;;;;;' and fish will gladly do nothing at all.

>=20
> --=20
> St=E9phane

--=20
Axel

Stephane Chazelas

2006-03-07, 5:54 pm

On 7 Mar 2006 05:58:29 -0800, liljencrantz@gmail.com wrote:
[...]
>
> Typo. I think you meant
>
> alias a='echo a'
> alias a='echo b'; a
>
> which will output 'a', and not 'b'. But you are completely right. That
> is a _much_ better example of the macro-like properties of shells, and
> one that I really feel should be removed. I noticed that this is one is
> not fixed in zsh either, at least no by default. It is also one which
> is removed in fish, where writing
>
> function a; echo a; end; a
> function a; echo b; end; a

Those are not aliases. aliases are meant to be aliases. It's
important that they are expanded very early in the parsing
process. It is meant to modify the input. If you don't want that
use functions.

f() { echo a;}
f() { echo b;}; f

will output b

You can do in POSIX shells as silly things as:

alias a='echo $(( 1'

a + 1))

it's a feature.

aliases are aliases, not functions.

zsh even has global aliases (every token can be replaced)

alias -g ...=../..

cd ...

[...]
> function alias --description "A wrapper for providing function
> definitions using a weaker, but simpler syntax"
> set -l name $argv[1]
> set -l body $argv[2]
> eval "function $name; $body \$argv; end"
> end

that is not the same.

> alias ll "ls -l"

alias ls 'ls -F'

will probably create an infinite loop.

--
Stephane

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Stephane Chazelas skrev:

> On 7 Mar 2006 05:58:29 -0800, liljencrantz@gmail.com wrote:
> [...]
>
> Those are not aliases. aliases are meant to be aliases. It's
> important that they are expanded very early in the parsing
> process. It is meant to modify the input. If you don't want that
> use functions.
>
> f() { echo a;}
> f() { echo b;}; f
>
> will output b
>
> You can do in POSIX shells as silly things as:
>
> alias a='echo $(( 1'
>
> a + 1))
>
> it's a feature.
>
> aliases are aliases, not functions.

Absolutley. Aliases are a form of macro expansion, something that, as
you show above, can be used to do many rather strange things. I
consider this a _bad_ thing in that the number of sane uses for this is
small in comparison to the number of typos and minor misunderstandings
that lead to subtle but painful bugs. I don't mean to make the tired
old argument that dangerous features are bad, what I mean to say is
that the usefullness of dangerous features has to be weighed against
the potential to introduce subtle, evil bugs. Your original example is
a perfect example of a situation where the macro expanding properties
of aliases will cause such havoc.

I simply think that a real function syntax is much more suited for
programming than simple alias substitution.

>
> zsh even has global aliases (every token can be replaced)
>
> alias -g ...=../..
>
> cd ...

In fish, if you enter a directory name instead of a command, it is
assumed that you want to cd to that directory, so you can simply write
'../..' to do the same thing, which is one character shorter. ;-)

>
> [...]
>
> that is not the same.
>
>
> alias ls 'ls -F'
>
> will probably create an infinite loop.

Actually it won't, since fish explicitly checks for unconditional
recursion. These checks can of course be fooled, but your example above
will call the ls command, not the ls function.

>
> --
> Stephane

--
Axel

Stephane Chazelas

2006-03-07, 5:54 pm

On 7 Mar 2006 06:54:45 -0800, liljencrantz@gmail.com wrote:
[...]
> I simply think that a real function syntax is much more suited for
> programming than simple alias substitution.

Yes, that's why you use functions for programming and aliases
for aliasing.

There are some times where aliases are useful for programming.
But some shells disable them.

Like:

alias die='{ echo >&2 ERROR; return 1; }'

that can't be implemented as a function.

> In fish, if you enter a directory name instead of a command, it is
> assumed that you want to cd to that directory, so you can simply write
> '../..' to do the same thing, which is one character shorter. ;-)

Same for zsh, then you can enter "..." which is 2 characters
shorter ;)

--
Stephane

Stephane Chazelas

2006-03-07, 5:54 pm

On 7 Mar 2006 06:18:17 -0800, liljencrantz@gmail.com wrote:
[...]
> As you mentioned in another post, zsh is much more clever in it's
> parsing of ';' and '}'. It may be that there are some awkward edge
> cases where there is a clash between braces used to denote blocks and
> braces used fo brace expansion, but overall the zsh syntax makes _much_
> more sense to me. That said, there is of course still the issue of
> using all these keywords for denoting end-of-block:
>
> fi, esac, done, }, ;;

You can use the shorter forms if you don't like the longer ones.
I use the shorter forms at the prompt:

if [[ a = b ]] {echo yes} else {echo no}

for f (1 2 3) echo $f

while ((i++ < 5)) echo $i

repeat 10 echo $((++i))

case a {(b) echo yes;; (*) echo no;;}

--
Stephane

Stephane Chazelas

2006-03-07, 5:54 pm

On 7 Mar 2006 04:48:55 -0800, liljencrantz@gmail.com wrote:
[...]
>
> I was under the impression that the function syntax was not part of the
> original Bourne shell, but rather an addition originating from rc. But
> I may be wrong.
[...]

Not from rc. I think rc dates back to the early nineties while
the Bourne shell has had functions since the early eighties. ksh
had function before the Bourne shell, though but using a
different syntax (function foo { ...; }).

--
Stephane

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Stephane Chazelas skrev:

> On 7 Mar 2006 06:54:45 -0800, liljencrantz@gmail.com wrote:
> [...]
>
> Yes, that's why you use functions for programming and aliases
> for aliasing.
>
> There are some times where aliases are useful for programming.
> But some shells disable them.
>
> Like:
>
> alias die='{ echo >&2 ERROR; return 1; }'
>
> that can't be implemented as a function.

Which is a _good_ thing, in my opinion. This is like the difference
between Basic and C; in Basic you can jump to any line, but in C you
can only jump to specific code entry points, called functions. This is
the same thing, only with code exit points instead. If you hide the
code exit point like that, it's impossible to know by simply scanning
the code for 'exit', 'return' and 'break' where the exit points are,
meaning it's much harder to see the code flow.

This is a tradeoff between supporting 'cool hacks' and supporting
maintainable coding practices, and to me the latter is more important.

>
>
> Same for zsh, then you can enter "..." which is 2 characters
> shorter ;)

Just tried it on my computer, using zsh 4.2.1, and this doesn't work.
Writing '..' in the commandline doesn't transport me to the parent
directory.

>
> --
> Stephane

--
Axel

Jordan Abel

2006-03-07, 5:54 pm

On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> Stephane CHAZELAS skrev:
>
>
> The problem I have with this is that a direct mapping to a low-level
> syscall is not a suitable abstraction to use as the main invocation of
> commands in a high-level scripting language.

A shell may incidentally be a scripting language, but its primary
purpose is to run other programs. Everything else is a way to determine
what files those programs have open, choose what programs get executed
in what order, choose what the contents of the argument list and
environment list for those programs are, and choose whether/when to wait
for those programs to finish.

> Instead of trying to provide the user with an thin execve wrapper, it
> might be better (in my opinion) to focus on how to design a good
> _language_.

What you consider a good language, and I may even agree, would NOT make
a good shell.

> Well, that is a bit of antropomorphisation on my part. The shells
> don't really talk to me. But according to Wikipedia "A macro language
> is a programming language in which all or most computation is done by
> expanding macros", which is very much true for shells. Variable
> substitution, wildcard substitution, alias substitution and many other
> language features work through macro expansion or an emulation of
> macro expansion.

The shell doesn't do computation. it runs programs. I would say that
"all or most computation" is done not by expanding macros, but by
calling external programs.

Jordan Abel

2006-03-07, 5:54 pm

On 2006-03-07, Stephane Chazelas <stephane_chazelas@yahoo.fr> wrote:
> On 7 Mar 2006 06:54:45 -0800, liljencrantz@gmail.com wrote:
> [...]
>
> Yes, that's why you use functions for programming and aliases
> for aliasing.
>
> There are some times where aliases are useful for programming.
> But some shells disable them.
>
> Like:
>
> alias die='{ echo >&2 ERROR; return 1; }'
>
> that can't be implemented as a function.

die() {
echo >&2 ERROR;
exit 1;
}

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Stephane Chazelas skrev:

> On 7 Mar 2006 10:20:10 -0800, liljencrantz@gmail.com wrote:
> [...]
>
> Well, it depends. A shell is mostly a tool for interactive use.
> You don't want to bridle the user there by adding constraints
> that he doesn't care about. At the prompt, I don't care if what
> I type is maintainable or nice code. That's one reason I think
> it difficult to have a good shell that is also a good
> programming language.

Sure, the reuirements are different. But I have found that a lot of the
things that you can get through ugly hacks can be done in a non-hackish
way as well, if you just give it a bit of thought. Saying 'this is only
for interactive mode, so it's ok that the syntax is a horrible mess' is
a cop out in my opinion.

As an example of rhis, in zsh it is not uncommon to use e.g. 'L' as a
global alias for '|less;'. In fish, I have added the following
keybinding instead:

"\M-p": if commandline -j|grep -v 'less *$' >/dev/null; commandline -aj
"|less;"; end

What it does is if you press Meta-p, check if the current job
definition ends with 'less' and if not, append the string '|less;'.

Advantages:

* Readable. You can see on the commandline what the code does.
* No rare problems when you actually _want_ to use L as an argument.

>
> BTW, if we take that comparison further with C, aliases are like
> preprocessor macros.
>
> [...]
>
> Sorry, you need:
>
> setopt autocd
>
> I never use that feature, I should disable that option. I prefer
> to use cd so that the completion only completes directories (or
> ~user or +<dirstack-number> )

*meh*
Yet again zsh implements a huge number of features, but not in a well
thought out way, and hence the feature is off by default. In fish, tab
completions understand both implicit cd and CDPATH.

>
> --
> Stephane

--
Axel

Jordan Abel

2006-03-07, 5:54 pm

On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> Stephane Chazelas skrev:
>
>
> Sure, the reuirements are different. But I have found that a lot of the
> things that you can get through ugly hacks can be done in a non-hackish
> way as well, if you just give it a bit of thought. Saying 'this is only
> for interactive mode, so it's ok that the syntax is a horrible mess' is
> a cop out in my opinion.
>
> As an example of rhis, in zsh it is not uncommon to use e.g. 'L' as a
> global alias for '|less;'. In fish, I have added the following
> keybinding instead:
>
> "\M-p": if commandline -j|grep -v 'less *$' >/dev/null; commandline -aj
> "|less;"; end
>
> What it does is if you press Meta-p, check if the current job
> definition ends with 'less' and if not, append the string '|less;'.

It'd be simpler to append it unconditionally. Processes are cheap these
days.

Bruce Barnett

2006-03-07, 5:54 pm

Stephane Chazelas <stephane_chazelas@yahoo.fr> writes:

> Not from rc. I think rc dates back to the early nineties while
> the Bourne shell has had functions since the early eighties. ksh
> had function before the Bourne shell, though but using a
> different syntax (function foo { ...; }).

My System V manual (from DEC 1984) describes sh functions.
My 4.3bsd manual (1986) doesn't. (7th Edition Unix).

--
Sending unsolicited commercial e-mail to this account incurs a fee of
$500 per message, and acknowledges the legality of this contract.

Chris F.A. Johnson

2006-03-07, 5:54 pm

On 2006-03-07, bsh wrote:
> liljencrantz@gmail.com wrote:
....
>
> (Number 4 should also work....)

Not in any shell I've tried it: bash[123], ksh93, ash, pdksh.

--
Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
Shell Scripting Recipes: | My code in this post, if any,
A Problem-Solution Approach | is released under the
2005, Apress | GNU General Public Licence

bsh

2006-03-07, 5:54 pm

Stephane CHAZELAS wrote:
> 2006-03-6, 18:55(-08), bsh:
> [...]
>
> But then a POSIX script can't use them as the behavior is
> unspecified. There's no need to have a POSIX conformant shell if
> it's to write non-POSIX conformant scripts. That doesn't help
> with portability.

There's a POSIX-specified difference between POSIX _compliant_
and POSIX _conformant_ -- are you aware of this? Compliance
indicates a strict accommodation to the standard; comformance
presumably allows for a superset of functionality.

> Note that it's not the case of zsh, where
> hello() {echo hello}

zsh(1) is kind of an oddball, with the impression that the author
was removing the goofier elements of k/sh syntax, without realising
the historical context for why they existed in the first place. For
at least the casual zsh(1) scripter, this has not served to hurt
the language at all -- just the opposite. But with a background of
language design, and having laboriously delved into the bowels
of the k/sh(1) syntax, I can say that the ultimate salvation of
any language is consistency, even if in the case of ksh(1)
consistency has brought about some decisions which seem
odd (like the issue of "{" and "}" above) but MUST be so to
maintain the subtler semantics of the language and still
remain a superset (that is, ksh(1) runs 95% of sh(1) scripts).
(Of course for a new language, such decisions do not apply;
but then I am wondering, what does the language have to offer
more or better than existing scripting languages?)

Because I have studied the grammar so extensively, it does
neither confuse me or dismay me; _however_, the hardest
thing to understand well is the fact that parsing and quoting
rules vary by the shell _context_. For instance:

$ print ${var:=this & that} # param expansion is intrinsically quoted
$ (( var+=1 )) # arithmetric "context" within "let" statement. (no
"$var")
$ shift var # shift parses argument as arithmetric expression

There is a table in B&K for the selection and order(!) of the
various parsing contexts for the many kinds constructs, and
there are _many_ footnotes....

sh(1) only had one parsing context that you would have needed
to read the documentation to know about : the argument to
"case" NEVER needs to be quoted. It is intrinsically quoted:

var='text with spaces'
case $var in ... # a convenience, but still needing memorization

> That sounds great, have you made it available?

I accidentally omitted the fact that the scanner/parser is
written in (old) awk(1) and (old) sed(1), which was to indicate
that any language that is possible to parse in such languages
must be at least fairly regular!

I have made it selectively available to beta testers over the
years, but as Zdenek Sekera will attest, my code base is
currently unavailable....

It is rather a small subsystem of a much more ambitious
IDE and function library for k/sh, which I will call SIDE and
publish eventually as open-source code.

=Brian

bsh

2006-03-07, 5:54 pm

Chris F.A. Johnson wrote:
> On 2006-03-07, bsh wrote:
> Not in any shell I've tried it: bash[123], ksh93, ash, pdksh.

Perhaps I shoud instead have said, "Number 4 _should_ also work...."
and append a smiley just in case.

It reminds me of the broken parsing for (IIRC):

for var; do; ...; done

Apparently semicolons are not quite the same as newlines.

;)

=Brian

bsh

2006-03-07, 5:54 pm

Chris F.A. Johnson

2006-03-07, 5:54 pm

On 2006-03-07, bsh wrote:
....
> Because I have studied the grammar so extensively, it does
> neither confuse me or dismay me; _however_, the hardest
> thing to understand well is the fact that parsing and quoting
> rules vary by the shell _context_. For instance:
>
> $ print ${var:=this & that} # param expansion is intrinsically quoted

It is? Try this:

printf "%s\n" ${var:=this & that}

> $ (( var+=1 )) # arithmetric "context" within "let" statement. (no
> "$var")

That is not POSIX.

> $ shift var # shift parses argument as arithmetric expression

Is that a ksh peculiarity?

$ shift 1+2
bash: shift: 1+2: numeric argument required

--
Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
Shell Scripting Recipes: | My code in this post, if any,
A Problem-Solution Approach | is released under the
2005, Apress | GNU General Public Licence

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Jordan Abel skrev:

> On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
[...][vbcol=seagreen]
>
> It'd be simpler to append it unconditionally. Processes are cheap these
> days.

I'm sure it would. It's just me trying to be clever, not wanting to
turn the prompt into 'foo|less|less|less|less|less' on repeated
keypresses. Perhaps this is overdesigning things a bit.

The part that I do like, however is that if places the |less at the
correct place even if there are multiple commans on the prompt. For
example, given

echo foo; whoami
^
Cursor is here

pressing Meta-p will result in

echo foo|less; whoami

Which I think is nice.

--
Axel

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Jordan Abel skrev:

> On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> A shell may incidentally be a scripting language, but its primary
> purpose is to run other programs. Everything else is a way to determine
> what files those programs have open, choose what programs get executed
> in what order, choose what the contents of the argument list and
> environment list for those programs are, and choose whether/when to wait
> for those programs to finish.

Replace 'program' with 'function' and you have a pretty good
description of e.g. C. I agree that different languages have different
requirements. Shells are languages designed to run other programs which
is a rather special domain, much like Matlab is a language for
numerical calculations. Domain specific languages have very special
requirements, different from general purpose languages like Java C++ or
Python. I think the fish design fulfills the requirements of a shell
language very well, though.

>
>
> What you consider a good language, and I may even agree, would NOT make
> a good shell.

I'm not trying to argue that the shell should be a general purpose
language. Quite the contrary, I want it to solve a very specific domain
of problems. I don't think that a thin wrapper around execve is a good
design for any language, including a shell.

>
>
> The shell doesn't do computation. it runs programs. I would say that
> "all or most computation" is done not by expanding macros, but by
> calling external programs.

>From the CPU:s point of view, pretty much the only thing a shell ever
does is call fork. That is by far the most costly operation. But the
implementation of fork is provided by the OS, and is only one line of
code. From a shell limplementors point of view, a traditional shell is
a macro expander, since that is what most of the code you write does.

--
Axel

liljencrantz@gmail.com

2006-03-07, 5:54 pm

bsh skrev:

> Stephane CHAZELAS wrote:
>
> There's a POSIX-specified difference between POSIX _compliant_
> and POSIX _conformant_ -- are you aware of this? Compliance
> indicates a strict accommodation to the standard; comformance
> presumably allows for a superset of functionality.
>
>
> zsh(1) is kind of an oddball, with the impression that the author
> was removing the goofier elements of k/sh syntax, without realising
> the historical context for why they existed in the first place. For
> at least the casual zsh(1) scripter, this has not served to hurt
> the language at all -- just the opposite. But with a background of
> language design, and having laboriously delved into the bowels
> of the k/sh(1) syntax, I can say that the ultimate salvation of
> any language is consistency, even if in the case of ksh(1)
> consistency has brought about some decisions which seem
> odd (like the issue of "{" and "}" above) but MUST be so to
> maintain the subtler semantics of the language and still
> remain a superset (that is, ksh(1) runs 95% of sh(1) scripts).
> (Of course for a new language, such decisions do not apply;
> but then I am wondering, what does the language have to offer
> more or better than existing scripting languages?)

In the case of fish, these are a few of the things offered:

* Support for more useful variable scoping rules as described in my
original post.
* Universal variables as described in my original post.
* A generic event syntax as described in my original post.
* Better error reporting as described in my original post
* Drop the cruft you describe. No more strange annoying quirks.
* Syntax highlighting when in interactive mode
* Advanced tab completions, including useful descriptions. For example,
type 'man wcs<TAB>' and you'll be presented with a listing of all
manual pages beginning with 'wcs' and a short summary of what each
page contains.
* X clipboard integration. ^K moves the rest of the line to the X
clipboard, ^Y pastes from the clipboard, etc. (If X is not running, an
internal fallback is used)
* Simple integrated history search - type in a search string into the
prompt and press the up arrow to search for the specified string in the
history. Use Meta-up to search for a token and not a whole line.

There are many, many other changes. A large portion of all fish
features is user interface polish. I meant for this discussion to be
about shell syntax, but I'd be happy to discuss user interface issues
as well.

[...]
> =Brian

--
Axel

liljencrantz@gmail.com

2006-03-07, 5:54 pm

Bruce Barnett skrev:

> Stephane Chazelas <stephane_chazelas@yahoo.fr> writes:
>
>
> My System V manual (from DEC 1984) describes sh functions.
> My 4.3bsd manual (1986) doesn't. (7th Edition Unix).

Thanks for taking the time to look this up. One can guess then that
functions are a SysV invention, though there are of course other
possibilities. The lack of functions in 4.3bsd would at least imply
that they are not a part of the original Bourne shell, which was my
main point.

>
> --
> Sending unsolicited commercial e-mail to this account incurs a fee of
> $500 per message, and acknowledges the legality of this contract.

--
Axel

Jordan Abel

2006-03-07, 8:48 pm

On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> Jordan Abel skrev:
>
> [...]
>
> I'm sure it would. It's just me trying to be clever, not wanting to
> turn the prompt into 'foo|less|less|less|less|less' on repeated
> keypresses. Perhaps this is overdesigning things a bit.

Have the keypress in question both append the pipe to less and dispatch
the command line for execution.

> The part that I do like, however is that if places the |less at the
> correct place even if there are multiple commans on the prompt. For
> example, given
>
> echo foo; whoami
> ^
> Cursor is here
>
> pressing Meta-p will result in
>
> echo foo|less; whoami
>
> Which I think is nice.

A key-binding to "execute the command piped to less", all in one action,
would be even nicer.

[hell, even i might buy your shell then - or at least download it, since
it's [i hope] open-source]

Jordan Abel

2006-03-07, 8:48 pm

On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> bsh skrev:
>
>
> In the case of fish, these are a few of the things offered:
>
> * Support for more useful variable scoping rules as described in my
> original post.
> * Universal variables as described in my original post.

zsh has dynamic scope. lexical scope can be overly restrictive.

> * A generic event syntax as described in my original post.

I didn't read the original post all the way through. what are "events",
though?

> * Better error reporting as described in my original post
> * Drop the cruft you describe. No more strange annoying quirks.

It's only strange and annoying when you don't know the syntax

> * Syntax highlighting when in interactive mode

Nice.

> * Advanced tab completions, including useful descriptions. For example,
> type 'man wcs<TAB>' and you'll be presented with a listing of all
> manual pages beginning with 'wcs' and a short summary of what each
> page contains.

Zsh does this.

> * X clipboard integration. ^K moves the rest of the line to the X
> clipboard, ^Y pastes from the clipboard, etc. (If X is not running, an
> internal fallback is used)

You haven't said whether you do this - are multiple cuts without cursor
movement treated as one? [most other applications do this, i.e.]

word1 word2_word3 word4

^K^W will put "word2 word3 word4" in the cut buffer. the cursor location
is underlined

What would be nice would be integration with the _screen_ clipboard -
send \e]83;paste .\a to paste, and \e]83;readbuf [tmpfile]\a to copy.
you could use the screen-exchange file, too.

> * Simple integrated history search - type in a search string into the
> prompt and press the up arrow to search for the specified string in the
> history. Use Meta-up to search for a token and not a whole line.

that sounds like vim's way - anything wrong with the i-search feature in
other shells?

> There are many, many other changes. A large portion of all fish
> features is user interface polish. I meant for this discussion to be
> about shell syntax, but I'd be happy to discuss user interface issues
> as well.

Sven Mascheck

2006-03-07, 8:48 pm

liljencrantz@gmail.com wrote:
> Bruce Barnett skrev:
[vbcol=seagreen]
>
> Thanks for taking the time to look this up. One can guess then that
> functions are a SysV invention,

No need to guess. The Bourne shell introduced functions with
SVR2 ('84). The Korn shell might have been earlier but AFAIK
it was not widely available until ksh86 or even ksh88.
--
<http://www.in-ulm.de/~mascheck/bourne/>

Stephane CHAZELAS

2006-03-08, 2:49 am

2006-03-07, 10:40(-08), Kurt Swanson:
> Stephane Chazelas <stephane_chazelas@yahoo.fr> writes:
>
>
>
>
>
> % setopt autocd
> % ...
> zsh: correct '...' to '..' [nyae]? n
> zsh: command not found: ...
> %

See earlier messages. It supposes an
alias -g ...=../..

--
Stéphane

Stephane CHAZELAS

2006-03-08, 2:49 am

2006-03-7, 11:06(-08), liljencrantz@gmail.com:
[...]
>
> *meh*
> Yet again zsh implements a huge number of features, but not in a well
> thought out way, and hence the feature is off by default. In fish, tab
> completions understand both implicit cd and CDPATH.
[...]

Of course, zsh does as well, but on the first word, obviously,
it also completes command names and variable names (for VAR=val
value), while after "cd" it only completes directories.

--
Stéphane

Stephane CHAZELAS

2006-03-08, 2:49 am

2006-03-7, 18:58(+00), Jordan Abel:
> On 2006-03-07, Stephane Chazelas <stephane_chazelas@yahoo.fr> wrote:
>
> die() {
> echo >&2 ERROR;
> exit 1;
> }

That exits the script, not the current function.

--
Stéphane

Stephane CHAZELAS

2006-03-08, 2:49 am

2006-03-7, 18:03(-05), Chris F.A. Johnson:
> On 2006-03-07, bsh wrote:
> ...
>
> It is? Try this:
>
> printf "%s\n" ${var:=this & that}

That's word splitting involved here.

Try:

IFS=; printf "%s\n" ${var:=this & that}

>
> That is not POSIX.
>
>
> Is that a ksh peculiarity?

Yes, I think Brian was speaking of ksh in particular there.

>
> $ shift 1+2
> bash: shift: 1+2: numeric argument required

--
Stéphane

Jordan Abel

2006-03-08, 2:49 am

On 2006-03-08, Stephane CHAZELAS <this.address@is.invalid> wrote:
> 2006-03-7, 18:58(+00), Jordan Abel:
>
> That exits the script, not the current function.

oh. I wasn't aware that was the effect you were going for.

What exactly do you want? multiple breakout 'return'? _no_ language has
that - the closest thing is probably longjmp in c.

Stephane CHAZELAS

2006-03-08, 2:49 am

2006-03-8, 02:28(+01), Sven Mascheck:
> liljencrantz@gmail.com wrote:
>
>
> No need to guess. The Bourne shell introduced functions with
> SVR2 ('84). The Korn shell might have been earlier but AFAIK
> it was not widely available until ksh86 or even ksh88.

There's that David Korn article on your web page that talks
about it:

http://www.in-ulm.de/~mascheck/bourne/korn.html

He says functions were added to the Bourne shell in 1982.

--
Stéphane

Stephane CHAZELAS

2006-03-08, 2:49 am

2006-03-8, 08:15(+00), Jordan Abel:
[...]
>
> oh. I wasn't aware that was the effect you were going for.
>
> What exactly do you want? multiple breakout 'return'? _no_ language has
> that - the closest thing is probably longjmp in c.

No I just want the equivalent of:

#define RETURN_WITH_ERROR { fputs("ERROR\n", stderr); return 1; }

so that I can use it as:

void f() {
if (blah) RETURN_WITH_ERROR;

if (foo) RETURN_WITH_ERROR;

if (bar) RETURN_WITH_ERROR;
}

(yes I know, I could have written it otherwise or with a goto,
that was just to show the point).

--
Stéphane

liljencrantz@gmail.com

2006-03-08, 7:52 am

Jordan Abel skrev:

> On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> Have the keypress in question both append the pipe to less and dispatch
> the command line for execution.
>
>
> A key-binding to "execute the command piped to less", all in one action,
> would be even nicer.
>
> [hell, even i might buy your shell then - or at least download it, since
> it's [i hope] open-source]

Download and install fish, and add the following to ~/.fish_inputrc:

"\M-l": commandline -aj "|less;"; eval (commandline); commandline ""

The first command appends'|less;' to job under the cursor on the
commandline, the second calls eval for the current contents of the
commandline and the third one clears the commandline.

I'll probably add this to the next fish release, it does seem like a
better thing to do.

Fish is GPL, btw.

--
Axel

Markus Gyger

2006-03-08, 7:52 am

liljencrantz@gmail.com writes:
> I mention
> this mostly since I think that it makes a lot of sense from a language
> point of view to minimize the difference between a ';' and a newline.

Although it doesn't have to be defined this way -- see e.g.
JavaScript[1], where most semicolons can be omitted.

[1] Standard ECMA-262: ECMAScript Language Specification
7.9.1 Rules of Automatic Semicolon Insertion
http://www.ecma-international.org/p...ds/Ecma-262.htm
[I have to admit that some things in the standard are not
too obvious at first -- like e.g. that property accessors
can be used to mimic associative arrays (hashes).]

Markus

Stephane Chazelas

2006-03-08, 7:52 am

On 8 Mar 2006 02:07:08 -0800, liljencrantz@gmail.com wrote:
[...]
>
> Download and install fish, and add the following to ~/.fish_inputrc:
>
> "\M-l": commandline -aj "|less;"; eval (commandline); commandline ""
>
> The first command appends'|less;' to job under the cursor on the
> commandline, the second calls eval for the current contents of the
> commandline and the third one clears the commandline.
>
> I'll probably add this to the next fish release, it does seem like a
> better thing to do.
[...]

With zsh:

append-pipe-pager-and-accept-line() {
emulate -L zsh
setopt extendedglob

[[ $BUFFER = *\|[[:blank:]]#less[[:blank:]]# ]] ||
BUFFER="{$BUFFER} | ${PAGER:-more}"

zle accept-line
}

zle -N append-pipe-pager-and-accept-line

bindkey '\el' append-pipe-pager-and-accept-line

--
Stephane

liljencrantz@gmail.com

2006-03-08, 7:52 am

Jordan Abel skrev:

> On 2006-03-07, liljencrantz@gmail.com <liljencrantz@gmail.com> wrote:
>
> zsh has dynamic scope. lexical scope can be overly restrictive.

But if you simply create a new variable, it will be global by default,
which is the pert the has me worried.

>
>
> I didn't read the original post all the way through. what are "events",
> though?

How about reading that bit of my original post to find out?

>
>
> It's only strange and annoying when you don't know the syntax
>

Look at the subconversation in this thread about 'printf "%s\n"
${var:=this & that}'. I would not say that bsh, Chris F.A. Johnson and
Stephane Chazelas don't know the syntax, would you? But the fact is,
there are so many subtle ambiguties that even they occasionally get
the details wrong. (Or at least one of them, since they don't agree)

>
> Nice.
>
>
> Zsh does this.

At least zsh 4.2.1 doesn't. If you complete 'ls -' it will show you
switches to ls with descriptions, just like fish. But at least my
version of zsh (using compinit) doesn't do any of the following:

* Provide the whatis entry as the description for manual page
completions
* Provide the whatis entry as the description for command name
completions
* Provide the full name as the description when completing a username
* Correctly complete strings containing braces, e.g. 'rm
{.,backup}/INST<tab>' will not work
* Correctly complete switches when multiple short switches are grouped
together, like 'tar -zx<TAB>'
* Correctly support gnu-style switches that accept arguments but don't
require them, like ls --color. It always completes with '--color=',
even though '--color' would also be a correct completion

>
>
> You haven't said whether you do this - are multiple cuts without cursor
> movement treated as one? [most other applications do this, i.e.]
>
> word1 word2_word3 word4
>
> ^K^W will put "word2 word3 word4" in the cut buffer. the cursor location
> is underlined

Not yet. Hadn't though of that one. Will add it though.

>
> What would be nice would be integration with the _screen_ clipboard -
> send \e]83;paste .\a to paste, and \e]83;readbuf [tmpfile]\a to copy.
> you could use the screen-exchange file, too.

That sounds very useful. Is there some place I can read up on how the
screen clipboard works?

>
>
> that sounds like vim's way - anything wrong with the i-search feature in
> other shells?

Yes.

* If you've typed something and realise that you may have already
written that before, you can search on what you've already typed
* Fish combines the history moving with the history search, so there
are fewer things to learn
* Fish highlights the search match, making it easier to tell if you've
found the right command
* Fish allwos you to search for a specific token instead of a whole
command

--
Axel

liljencrantz@gmail.com

2006-03-08, 7:52 am

Stephane Chazelas skrev:

> On 8 Mar 2006 02:07:08 -0800, liljencrantz@gmail.com wrote:
> [...]
> [...]
>
> With zsh:
>
> append-pipe-pager-and-accept-line() {
> emulate -L zsh
> setopt extendedglob
>
> [[ $BUFFER = *\|[[:blank:]]#less[[:blank:]]# ]] ||
> BUFFER="{$BUFFER} | ${PAGER:-more}"
>
> zle accept-line
> }
>
> zle -N append-pipe-pager-and-accept-line
>
> bindkey '\el' append-pipe-pager-and-accept-line

Sure. I wasn't trying to say that this wasn't possible in other shells,
only that ugly hacks, like global aliases, generally are possible to
implement in a non-ugly, non-hackish way that is comparatively
convenient.

>
> --
> Stephane

--
Axel

liljencrantz@gmail.com

2006-03-08, 7:52 am

Markus Gyger skrev:

> liljencrantz@gmail.com writes:
>
> Although it doesn't have to be defined this way -- see e.g.
> JavaScript[1], where most semicolons can be omitted.

I wouldn't use JavaScript as an example if a very good language. Most
sane languages try to make as little difference as possible between
diffferent types of whitespace and different types of command
delimiters.

>
> [1] Standard ECMA-262: ECMAScript Language Specification
> 7.9.1 Rules of Automatic Semicolon Insertion
> http://www.ecma-international.org/p...ds/Ecma-262.htm
> [I have to admit that some things in the standard are not
> too obvious at first -- like e.g. that property accessors
> can be used to mimic associative arrays (hashes).]
>
>
> Markus

--
Axel

liljencrantz@gmail.com

2006-03-08, 7:52 am

Stephane CHAZELAS skrev:

> 2006-03-8, 08:15(+00), Jordan Abel:
> [...]
>
> No I just want the equivalent of:
>
> #define RETURN_WITH_ERROR { fputs("ERROR\n", stderr); return 1; }
>
> so that I can use it as:
>
> void f() {
> if (blah) RETURN_WITH_ERROR;
>
> if (foo) RETURN_WITH_ERROR;
>
> if (bar) RETURN_WITH_ERROR;
> }
>
> (yes I know, I could have written it otherwise or with a goto,
> that was just to show the point).

Which is my point exactly. You can do the same things witout resorting
to code obfuscation using aliases; I have yet to see an example of
where aliases help more than they hurt.

The '...' global alias could be replaced by a keybinding, e.g. Meta-g,
that inserts '../..' at the current cursor position. This means that
your history file will always be in plan, sane, shellscript, and it
means that you won't accidentally trigger the script when you didn't
mean to.

The return with error described above could be implemented in any
number of ways, including

function f_internal
blah; or return 1
foo; or return 1
bar; or return 1
end

function f
if not f_internal
echo ERROR >&2
return 1
end
end

Notice that you can actually _see_ all the exit points of the function,
making it much easier to predict the code flow.

> --=20
> St=E9phane

--=20
Axel

Stephane Chazelas

2006-03-08, 7:52 am

On 8 Mar 2006 02:37:47 -0800, liljencrantz@gmail.com wrote:
[...]
>
> But if you simply create a new variable, it will be global by default,
> which is the pert the has me worried.

Same in awk, PERL and many languages. It looks more intuitive
(familiar) to me to have to tell it when I want to limit the
scope.

[...]
>
> At least zsh 4.2.1 doesn't. If you complete 'ls -' it will show you
> switches to ls with descriptions, just like fish. But at least my
> version of zsh (using compinit) doesn't do any of the following:
>
> * Provide the whatis entry as the description for manual page
> completions
> * Provide the whatis entry as the description for command name
> completions

These are nice features. Note that zsh has all the necessary
framework to enable you to do the same without having to modify
zsh's code. I would bet zsh supports many more such features
than fish given that new command supports have been added to it
for about 10 years, though.

One of the very few critics of zsh I've heard of is that when
you use all of its features, it can get very bloated and
resource consuming. If you hold a cache of all the whatis or
user database in memory, you'll run into similar problems.

> * Provide the full name as the description when completing a username
> * Correctly complete strings containing braces, e.g. 'rm
> {.,backup}/INST<tab>' will not work

You can't expect that to work with zsh, the way it is. zsh
completes one thing at a time, {..,..} is not some sort of
globbing, it is expanded very early (before any other things),
it doesn't make much sense to complete there as you can't do
anything reliable.

> * Correctly complete switches when multiple short switches are grouped
> together, like 'tar -zx<TAB>'

What do you mean? That's not the correct tar syntax BTW.

> * Correctly support gnu-style switches that accept arguments but don't
> require them, like ls --color. It always completes with '--color=',
> even though '--color' would also be a correct completion

zsh supports them

If you type:

ls --co<tab><space>

or

ls --co<tab><enter>

zsh will remove the "="

[...]
>
> That sounds very useful. Is there some place I can read up on how the
> screen clipboard works?

info screen

All that is easily done with zsh (see
http://stchaz.free.fr/mouse.zsh for the X clipboard support)

see also the "predict" mode in zsh.
[vbcol=seagreen]
>
> Yes.
>
> * If you've typed something and realise that you may have already
> written that before, you can search on what you've already typed

<Esc-p> in zsh?

> * Fish combines the history moving with the history search, so there
> are fewer things to learn
> * Fish highlights the search match, making it easier to tell if you've
> found the right command

That's neat.

--
Stephane

liljencrantz@gmail.com

2006-03-08, 7:52 am

Stephane Chazelas skrev:

> On 8 Mar 2006 02:37:47 -0800, liljencrantz@gmail.com wrote:
> [...]
>
> Same in awk, PERL and many languages. It looks more intuitive
> (familiar) to me to have to tell it when I want to limit the
> scope.

I disagree. Most variables that you use inside functions should be
function local in my experience. And most of those that shouldn't,
should be exported as well, anyway.

>
> [...]
>
> These are nice features. Note that zsh has all the necessary
> framework to enable you to do the same without having to modify
> zsh's code. I would bet zsh supports many more such features
> than fish given that new command supports have been added to it
> for about 10 years, though.

Probably. Fish has command specific completions for ~150 commands. I
tried out a few commands and noticed that zsh lacks completions for
darcs, su, yum and most subcommands of apt, while fish lacks
completions for telnet, rcp and finger. It would seem to me that zsh
has completions for many old-school commands that fish lacks, while
fish has completion support for a few newer commands the zsh lacks. It
is possible that there is a new zsh release featuring support for many
more commands, I'm using 4.2.1.

>
> One of the very few critics of zsh I've heard of is that when
> you use all of its features, it can get very bloated and
> resource consuming. If you hold a cache of all the whatis or
> user database in memory, you'll run into similar problems.

Fish autoloads everything on first use. The first time you use a
shellscript function, that function is loaded. The first time you
complete a command, the completions for that command are loaded. The
first time you access the commandline history, the history file is
loaded. These incremental loads are so small that they aren't noticable
on my 300MHz Speed deamon unless the disk is spinned down. There is an
issue with memory use, however, since nothing ever gets unloaded. This
issue is amplified by the fact that fish internally uses wide character
strings, so most text is quadrupled in size. I just used massigf to
tell me the fish memory usage, and it claims fish uses ~100 kB on
startup and ~1.6 MB if you manually load _everything_. I think massif
is doing something wrng however, the memory usage profile looks funny.
My guess would be that fish uses ~150 kB on startup. I find that my
fish sessions usually pan out at 300-400 kB of memory, which seems to
be what zsh uses on startup.

I plan to implement unloading of functions and completions that haven't
been used in a long time to make this problem go away.

Fish is also a bit slower than other shells on some types of scripts,
since it doesn't implement a huge number of commands as builtins. E.g.
time, pwd, kill, printf and echo are use standard unix commands, not
builtins.

>
>
> You can't expect that to work with zsh, the way it is. zsh
> completes one thing at a time, {..,..} is not some sort of
> globbing, it is expanded very early (before any other things),
> it doesn't make much sense to complete there as you can't do
> anything reliable.

I can expect it and I do. Zsh supports every known feature on the
planet, so why not this? ;-)

The fish globbing code can be run in a 'completion-mode', where instead
of expanding things on wildcards, etc. all possible completions are
added. This way, one can in fish complete strings like 'grep foo
/proc/*/cmd<TAB>' without replacing the * with all found matches, like
in zsh.

>
>
> What do you mean? That's not the correct tar syntax BTW.

tar -zx

is synonymous with

tar -z -x

One can group single-character switches on a single hyphen. Fish
supports this in it's completion code.

>
>
> zsh supports them
>
> If you type:
>
> ls --co<tab><space>
>
> or
>
> ls --co<tab&