Given:  The config file for an application (written in C) includes a
directive for a command to be invoked via one of the "exec()" calls
(following a "fork()").

Problem: What is the cleanest way to parse that command:
cmd arg1 arg2 arg3

so that such an "exec()" can then be done?

It would also be nice if it could understand:
cmd "arg1a arg1b" arg2 arg3

(that is, allowing an argument to include whitespace).

Is there already a portable subroutine for this sort of quasi-shell
parsing?  In my years of programming, I don't recall seeing any such
subroutine.  Or do I need to write my own using something like "strtok()"?


(Yes, I'm alert to the potentially lethal possibilities of interference
leading to something like "/bin/rm -rf /" ...!)


--

David Lee

[ Post a follow-up to this message ]

David Lee <t.d.lee@durham.ac.uk> writes:

> Given:  The config file for an application (written in C) includes a
> directive for a command to be invoked via one of the "exec()" calls
> (following a "fork()").
>
> Problem: What is the cleanest way to parse that command:
>    cmd arg1 arg2 arg3
>
> so that such an "exec()" can then be done?
>
> It would also be nice if it could understand:
>    cmd "arg1a arg1b" arg2 arg3
>
> (that is, allowing an argument to include whitespace).
>
> Is there already a portable subroutine for this sort of quasi-shell
> parsing?  In my years of programming, I don't recall seeing any such
> subroutine.  Or do I need to write my own using something like "strtok()"?

Maybe wordexp() is what you've been looking for all these years.

--
Måns Rullgård
mans@mansr.com

[ Post a follow-up to this message ]

David Lee wrote:
> Given:  The config file for an application (written in C) includes a
> directive for a command to be invoked via one of the "exec()" calls
> (following a "fork()").
>
> Problem: What is the cleanest way to parse that command:
>    cmd arg1 arg2 arg3
>
> so that such an "exec()" can then be done?
>
> It would also be nice if it could understand:
>    cmd "arg1a arg1b" arg2 arg3
>
> (that is, allowing an argument to include whitespace).
>
> Is there already a portable subroutine for this sort of quasi-shell
> parsing?  In my years of programming, I don't recall seeing any such
> subroutine.

Perl, I believe, handles this issue internally with the following
rules:

(1) if there are no metacharacters (backslash, quotes, etc.),
split on whitespace and pass the resulting list of tokens
to exec().

(2) if there are metacharacters, pass the list
( "sh", "-c", string_with_metacharacters ) to exec() instead.

In this way, you get all the efficiency of doing it directly in a
lot of cases, and in the other cases, you get the full flexibility
of the shell.  It also keeps the complexity of parsing out of
your code.

The disadvantage is that the logic is different for different cases.
This makes it slightly harder to test and slightly harder to understand.
It also makes the process tree shaped differently depending on the
string, with a child in one case where there is a grandchild in another.
I suppose it probably has implications for error handling as well
(although "sh -c" will propagate the exit code, I think).

- Logan

[ Post a follow-up to this message ]

On Mon, 14 May 2007, Logan Shaw wrote:

> Perl, I believe, handles this issue internally with the following
> rules:
>
> (1) if there are no metacharacters (backslash, quotes, etc.),
>      split on whitespace and pass the resulting list of tokens
>      to exec().
>
> (2) if there are metacharacters, pass the list
>      ( "sh", "-c", string_with_metacharacters ) to exec() instead.
>
> In this way, you get all the efficiency of doing it directly in a
> lot of cases, and in the other cases, you get the full flexibility
> of the shell.  It also keeps the complexity of parsing out of
> your code.
>
> The disadvantage is that the logic is different for different cases.
> This makes it slightly harder to test and slightly harder to understand.
> It also makes the process tree shaped differently depending on the
> string, with a child in one case where there is a grandchild in another.
> I suppose it probably has implications for error handling as well
> (although "sh -c" will propagate the exit code, I think).

Many thanks.  Actually it was this "sh -c" process-treee behaviour that
lay in the background to my question.  The application already does:

fork();
if (child) {
execl("/bin/sh", "sh", "-c", command, NULL);
}
else { /* parent*/
/* subsequently exercise control over child "command" */
}

But this seems to be causing portability issues.  On Linux, that "/bin/sh"
(bash) appears to replace itself with "command" (that is, "command"
appears still to be the child itself).  But on OSes (Solaris) where
"/bin/sh" is Bourne, that shell seems to do a further fork before doing
the command (that is, "command" seems to end up the grandchild).

(I'm reporting the above second hand, so feel free to confirm/refute!)


Hence our wanting to go directly to:
exec...(command, ...)

so that we will know exactly what we're getting.


--

:  David Lee                                I.T. Service          :
:  Senior Systems Programmer                Computer Centre       :
:  UNIX Team Leader                         Durham university     :
:                                           South Road            :
:  http://www.dur.ac.uk/t.d.lee/            Durham DH1 3LE        :
:  Phone: +44 191 334 2752                  U.K.                  :

[ Post a follow-up to this message ]

On Mon, 14 May 2007, [iso-8859-1] M=E5ns Rullg=E5rd wrote:

> David Lee <t.d.lee@durham.ac.uk> writes:
> 
)"?[vbcol=seagreen]
>
> Maybe wordexp() is what you've been looking for all these years.

Great!  At a quick glance that looks promising.  And it seems to be
present on three sample systems I've just checked (Linux, Solaris,
FreeBSD).  Thanks.

(I wonder how I never noticed it before?



--=20

:  David Lee                                I.T. Service          :
:  Senior Systems Programmer                Computer Centre       :
:  UNIX Team Leader                         Durham university     :
:                                           South Road            :
:  http://www.dur.ac.uk/t.d.lee/            Durham DH1 3LE        :
:  Phone: +44 191 334 2752                  U.K.                  :

[ Post a follow-up to this message ]

David Lee wrote:

> The application already does:
>
>    fork();
>    if (child) {
>       execl("/bin/sh", "sh", "-c", command, NULL);
>    }
>    else { /* parent*/
>       /* subsequently exercise control over child "command" */
>    }
>
> But this seems to be causing portability issues.  On Linux, that "/bin/sh"
> (bash) appears to replace itself with "command" (that is, "command"
> appears still to be the child itself).  But on OSes (Solaris) where
> "/bin/sh" is Bourne, that shell seems to do a further fork before doing
> the command (that is, "command" seems to end up the grandchild).

Maybe what you need is:

execl("/bin/sh", "sh", "-c", "exec command ...", (char *)NULL);

Of course this will only work if "command" is something that can be
exec-ed by the shell.

(Note that the (char *) cast I added is required for portability.)

--
Geoff Clare <netnews@gclare.org.uk>

[ Post a follow-up to this message ]

Geoff Clare <geoff@clare.See-My-Signature.invalid> writes:
> David Lee wrote:
> 
>
> Maybe what you need is:
>
>         execl("/bin/sh", "sh", "-c", "exec command ...", (char *)NULL);
>
> Of course this will only work if "command" is something that can be
> exec-ed by the shell.
>
> (Note that the (char *) cast I added is required for portability.)

That hasn't become more true since the last time. The last argument of
the execl-routine needs to be 'a null pointer'. NULL is an
implementation defined 'null pointer constant'. A 'null pointer
constant' is converted to 'a null pointer' whenever it is used in some
place where an expression of some pointer type is required. While the
type of the expansion of null may be a pointer type, it is not
specified that it must be a pointer type (eg it could be a plain 0 of
type int). That makes NULL unsuitable for use in places where
automatic conversions do not take place (like in arguments to variadic
routines) and casting it (or another null pointer constant, eg int 0
again) to a suitable pointer type is required for correctness. What
exactly a suitable pointer type is supposed to be is not specified by
SUS.

[ Post a follow-up to this message ]

Rainer Weikusat wrote:

> That hasn't become more true since the last time. The last argument of
> the execl-routine needs to be 'a null pointer'. NULL is an
> implementation defined 'null pointer constant'. A 'null pointer
> constant' is converted to 'a null pointer' whenever it is used in some
> place where an expression of some pointer type is required. While the
> type of the expansion of null may be a pointer type, it is not
> specified that it must be a pointer type (eg it could be a plain 0 of
> type int). That makes NULL unsuitable for use in places where
> automatic conversions do not take place (like in arguments to variadic
> routines) and casting it (or another null pointer constant, eg int 0
> again) to a suitable pointer type is required for correctness. What
> exactly a suitable pointer type is supposed to be is not specified by
> SUS.

Isn't it obvious that the suitable pointer is "char *" (with or without
the const qualifier), given that arg1 to argN-1 have to be "const char
*"? Is there any way to implement execl() so that arg1 to argN-1 are
"char *" and argN is an incompatible pointer?

--
mail1dotstofanetdotdk

[ Post a follow-up to this message ]

Bjorn Reese <breese@see.signature> writes:
> Rainer Weikusat wrote: 
>
> Isn't it obvious that the suitable pointer is "char *" (with or without
> the const qualifier), given that arg1 to argN-1 have to be "const char
> *"?

It is a possible interpretation of the standard text. But this implies
that its content is not part of the standard text. This appears to be
an accidental omission.

[ Post a follow-up to this message ]

Rainer Weikusat wrote:

> It is a possible interpretation of the standard text. But this implies
> that its content is not part of the standard text. This appears to be
> an accidental omission.

You are asking for a completely unnecessary addition. Show me an example
of execl() that reads arg1 to argN-1 as "char *" and argN as an
unsuitable pointer.

--
mail1dotstofanetdotdk

[ Post a follow-up to this message ]