autoconf: Shell Substitutions
1
1 11.8 Shell Substitutions
1 ========================
1
1 Contrary to a persistent urban legend, the Bourne shell does not
1 systematically split variables and back-quoted expressions, in
1 particular on the right-hand side of assignments and in the argument of
1 `case'. For instance, the following code:
1
1 case "$given_srcdir" in
1 .) top_srcdir="`echo "$dots" | sed 's|/$||'`" ;;
1 *) top_srcdir="$dots$given_srcdir" ;;
1 esac
1
1 is more readable when written as:
1
1 case $given_srcdir in
1 .) top_srcdir=`echo "$dots" | sed 's|/$||'` ;;
1 *) top_srcdir=$dots$given_srcdir ;;
1 esac
1
1 and in fact it is even _more_ portable: in the first case of the first
1 attempt, the computation of `top_srcdir' is not portable, since not all
1 shells properly understand `"`..."..."...`"', for example Solaris 10
1 ksh:
1
1 $ foo="`echo " bar" | sed 's, ,,'`"
1 ksh: : cannot execute
1 ksh: bar | sed 's, ,,': cannot execute
1
1 Posix does not specify behavior for this sequence. On the other hand,
1 behavior for `"`...\"...\"...`"' is specified by Posix, but in
1 practice, not all shells understand it the same way: pdksh 5.2.14
1 prints spurious quotes when in Posix mode:
1
1 $ echo "`echo \"hello\"`"
1 hello
1 $ set -o posix
1 $ echo "`echo \"hello\"`"
1 "hello"
1
1 There is just no portable way to use double-quoted strings inside
1 double-quoted back-quoted expressions (pfew!).
1
1 Bash 4.1 has a bug where quoted empty strings adjacent to unquoted
1 parameter expansions are elided during word splitting. Meanwhile, zsh
1 does not perform word splitting except when in Bourne compatibility
1 mode. In the example below, the correct behavior is to have five
1 arguments to the function, and exactly two spaces on either side of the
1 middle `-', since word splitting collapses multiple spaces in `$f' but
1 leaves empty arguments intact.
1
1 $ bash -c 'n() { echo "$#$@"; }; f=" - "; n - ""$f"" -'
1 3- - -
1 $ ksh -c 'n() { echo "$#$@"; }; f=" - "; n - ""$f"" -'
1 5- - -
1 $ zsh -c 'n() { echo "$#$@"; }; f=" - "; n - ""$f"" -'
1 3- - -
1 $ zsh -c 'emulate sh;
1 > n() { echo "$#$@"; }; f=" - "; n - ""$f"" -'
1 5- - -
1
1 You can work around this by doing manual word splitting, such as using
1 `"$str" $list' rather than `"$str"$list'.
1
1 There are also portability pitfalls with particular expansions:
1
1 `$@'
1 One of the most famous shell-portability issues is related to
1 `"$@"'. When there are no positional arguments, Posix says that
1 `"$@"' is supposed to be equivalent to nothing, but the original
1 Unix version 7 Bourne shell treated it as equivalent to `""'
1 instead, and this behavior survives in later implementations like
1 Digital Unix 5.0.
1
1 The traditional way to work around this portability problem is to
1 use `${1+"$@"}'. Unfortunately this method does not work with Zsh
1 (3.x and 4.x), which is used on Mac OS X. When emulating the
1 Bourne shell, Zsh performs word splitting on `${1+"$@"}':
1
1 zsh $ emulate sh
1 zsh $ for i in "$@"; do echo $i; done
1 Hello World
1 !
1 zsh $ for i in ${1+"$@"}; do echo $i; done
1 Hello
1 World
1 !
1
1 Zsh handles plain `"$@"' properly, but we can't use plain `"$@"'
1 because of the portability problems mentioned above. One
1 workaround relies on Zsh's "global aliases" to convert `${1+"$@"}'
1 into `"$@"' by itself:
1
1 test "${ZSH_VERSION+set}" = set && alias -g '${1+"$@"}'='"$@"'
1
1 Zsh only recognizes this alias when a shell word matches it
1 exactly; `"foo"${1+"$@"}' remains subject to word splitting.
1 Since this case always yields at least one shell word, use plain
1 `"$@"'.
1
1 A more conservative workaround is to avoid `"$@"' if it is
1 possible that there may be no positional arguments. For example,
1 instead of:
1
1 cat conftest.c "$@"
1
1 you can use this instead:
1
1 case $# in
1 0) cat conftest.c;;
1 *) cat conftest.c "$@";;
1 esac
1
1 Autoconf macros often use the `set' command to update `$@', so if
1 you are writing shell code intended for `configure' you should not
1 assume that the value of `$@' persists for any length of time.
1
1 `${10}'
1 The 10th, 11th, ... positional parameters can be accessed only
1 after a `shift'. The 7th Edition shell reported an error if given
1 `${10}', and Solaris 10 `/bin/sh' still acts that way:
1
1 $ set 1 2 3 4 5 6 7 8 9 10
1 $ echo ${10}
1 bad substitution
1
1 Conversely, not all shells obey the Posix rule that when braces are
1 omitted, multiple digits beyond a `$' imply the single-digit
1 positional parameter expansion concatenated with the remaining
1 literal digits. To work around the issue, you must use braces.
1
1 $ bash -c 'set a b c d e f g h i j; echo $10 ${1}0'
1 a0 a0
1 $ dash -c 'set a b c d e f g h i j; echo $10 ${1}0'
1 j a0
1
1 `${VAR:-VALUE}'
1 Old BSD shells, including the Ultrix `sh', don't accept the colon
1 for any shell substitution, and complain and die. Similarly for
1 ${VAR:=VALUE}, ${VAR:?VALUE}, etc. However, all shells that
1 support functions allow the use of colon in shell substitution,
1 and since m4sh requires functions, you can portably use null
1 variable substitution patterns in configure scripts.
1
1 `${VAR+VALUE}'
1 When using `${VAR-VALUE}' or `${VAR-VALUE}' for providing
1 alternate substitutions, VALUE must either be a single shell word,
1 quoted, or in the context of an unquoted here-document. Solaris
1 `/bin/sh' complains otherwise.
1
1 $ /bin/sh -c 'echo ${a-b c}'
1 /bin/sh: bad substitution
1 $ /bin/sh -c 'echo ${a-'\''b c'\''}'
1 b c
1 $ /bin/sh -c 'echo "${a-b c}"'
1 b c
1 $ /bin/sh -c 'cat <<EOF
1 ${a-b c}
1 EOF
1 b c
1
1 According to Posix, if an expansion occurs inside double quotes,
1 then the use of unquoted double quotes within VALUE is
1 unspecified, and any single quotes become literal characters; in
1 that case, escaping must be done with backslash. Likewise, the
1 use of unquoted here-documents is a case where double quotes have
1 unspecified results:
1
1 $ /bin/sh -c 'echo "${a-"b c"}"'
1 /bin/sh: bad substitution
1 $ ksh -c 'echo "${a-"b c"}"'
1 b c
1 $ bash -c 'echo "${a-"b c"}"'
1 b c
1 $ /bin/sh -c 'a=; echo ${a+'\''b c'\''}'
1 b c
1 $ /bin/sh -c 'a=; echo "${a+'\''b c'\''}"'
1 'b c'
1 $ /bin/sh -c 'a=; echo "${a+\"b c\"}"'
1 "b c"
1 $ /bin/sh -c 'a=; echo "${a+b c}"'
1 b c
1 $ /bin/sh -c 'cat <<EOF
1 ${a-"b c"}
1 EOF'
1 "b c"
1 $ /bin/sh -c 'cat <<EOF
1 ${a-'b c'}
1 EOF'
1 'b c'
1 $ bash -c 'cat <<EOF
1 ${a-"b c"}
1 EOF'
1 b c
1 $ bash -c 'cat <<EOF
1 ${a-'b c'}
1 EOF'
1 'b c'
1
1 Perhaps the easiest way to work around quoting issues in a manner
1 portable to all shells is to place the results in a temporary
1 variable, then use `$t' as the VALUE, rather than trying to inline
1 the expression needing quoting.
1
1 $ /bin/sh -c 't="b c\"'\''}\\"; echo "${a-$t}"'
1 b c"'}\
1 $ ksh -c 't="b c\"'\''}\\"; echo "${a-$t}"'
1 b c"'}\
1 $ bash -c 't="b c\"'\''}\\"; echo "${a-$t}"'
1 b c"'}\
1
1 `${VAR=VALUE}'
1 When using `${VAR=VALUE}' to assign a default value to VAR,
1 remember that even though the assignment to VAR does not undergo
1 file name expansion, the result of the variable expansion does
1 unless the expansion occurred within double quotes. In particular,
1 when using `:' followed by unquoted variable expansion for the
1 side effect of setting a default value, if the final value of
1 `$var' contains any globbing characters (either from VALUE or from
1 prior contents), the shell has to spend time performing file name
1 expansion and field splitting even though those results will not be
1 used. Therefore, it is a good idea to consider double quotes when
1 performing default initialization; while remembering how this
1 impacts any quoting characters appearing in VALUE.
1
1 $ time bash -c ': "${a=/usr/bin/*}"; echo "$a"'
1 /usr/bin/*
1
1 real 0m0.005s
1 user 0m0.002s
1 sys 0m0.003s
1 $ time bash -c ': ${a=/usr/bin/*}; echo "$a"'
1 /usr/bin/*
1
1 real 0m0.039s
1 user 0m0.026s
1 sys 0m0.009s
1 $ time bash -c 'a=/usr/bin/*; : ${a=noglob}; echo "$a"'
1 /usr/bin/*
1
1 real 0m0.031s
1 user 0m0.020s
1 sys 0m0.010s
1
1 $ time bash -c 'a=/usr/bin/*; : "${a=noglob}"; echo "$a"'
1 /usr/bin/*
1
1 real 0m0.006s
1 user 0m0.002s
1 sys 0m0.003s
1
1 As with `+' and `-', you must use quotes when using `=' if the
1 VALUE contains more than one shell word; either single quotes for
1 just the VALUE, or double quotes around the entire expansion:
1
1 $ : ${var1='Some words'}
1 $ : "${var2=like this}"
1 $ echo $var1 $var2
1 Some words like this
1
1 otherwise some shells, such as Solaris `/bin/sh' or on Digital
1 Unix V 5.0, die because of a "bad substitution". Meanwhile, Posix
1 requires that with `=', quote removal happens prior to the
1 assignment, and the expansion be the final contents of VAR without
1 quoting (and thus subject to field splitting), in contrast to the
1 behavior with `-' passing the quoting through to the final
1 expansion. However, `bash' 4.1 does not obey this rule.
1
1 $ ksh -c 'echo ${var-a\ \ b}'
1 a b
1 $ ksh -c 'echo ${var=a\ \ b}'
1 a b
1 $ bash -c 'echo ${var=a\ \ b}'
1 a b
1
1 Finally, Posix states that when mixing `${a=b}' with regular
1 commands, it is unspecified whether the assignments affect the
1 parent shell environment. It is best to perform assignments
1 independently from commands, to avoid the problems demonstrated in
1 this example:
1
1 $ bash -c 'x= y=${x:=b} sh -c "echo +\$x+\$y+";echo -$x-'
1 +b+b+
1 -b-
1 $ /bin/sh -c 'x= y=${x:=b} sh -c "echo +\$x+\$y+";echo -$x-'
1 ++b+
1 --
1 $ ksh -c 'x= y=${x:=b} sh -c "echo +\$x+\$y+";echo -$x-'
1 +b+b+
1 --
1
1 `${VAR=VALUE}'
1 Solaris `/bin/sh' has a frightening bug in its handling of literal
1 assignments. Imagine you need set a variable to a string
1 containing `}'. This `}' character confuses Solaris `/bin/sh'
1 when the affected variable was already set. This bug can be
1 exercised by running:
1
1 $ unset foo
1 $ foo=${foo='}'}
1 $ echo $foo
1 }
1 $ foo=${foo='}' # no error; this hints to what the bug is
1 $ echo $foo
1 }
1 $ foo=${foo='}'}
1 $ echo $foo
1 }}
1 ^ ugh!
1
1 It seems that `}' is interpreted as matching `${', even though it
1 is enclosed in single quotes. The problem doesn't happen using
1 double quotes, or when using a temporary variable holding the
1 problematic string.
1
1 `${VAR=EXPANDED-VALUE}'
1 On Ultrix, running
1
1 default="yu,yaa"
1 : ${var="$default"}
1
1 sets VAR to `M-yM-uM-,M-yM-aM-a', i.e., the 8th bit of each char
1 is set. You don't observe the phenomenon using a simple `echo
1 $var' since apparently the shell resets the 8th bit when it
1 expands $var. Here are two means to make this shell confess its
1 sins:
1
1 $ cat -v <<EOF
1 $var
1 EOF
1
1 and
1
1 $ set | grep '^var=' | cat -v
1
1 One classic incarnation of this bug is:
1
1 default="a b c"
1 : ${list="$default"}
1 for c in $list; do
1 echo $c
1 done
1
1 You'll get `a b c' on a single line. Why? Because there are no
1 spaces in `$list': there are `M- ', i.e., spaces with the 8th bit
1 set, hence no IFS splitting is performed!!!
1
1 One piece of good news is that Ultrix works fine with `:
1 ${list=$default}'; i.e., if you _don't_ quote. The bad news is
1 then that QNX 4.25 then sets LIST to the _last_ item of DEFAULT!
1
1 The portable way out consists in using a double assignment, to
1 switch the 8th bit twice on Ultrix:
1
1 list=${list="$default"}
1
1 ...but beware of the `}' bug from Solaris (see above). For safety,
1 use:
1
1 test "${var+set}" = set || var={VALUE}
1
1 `${#VAR}'
1 `${VAR%WORD}'
1 `${VAR%%WORD}'
1 `${VAR#WORD}'
1 `${VAR##WORD}'
1 Posix requires support for these usages, but they do not work with
1 many traditional shells, e.g., Solaris 10 `/bin/sh'.
1
1 Also, `pdksh' 5.2.14 mishandles some WORD forms. For example if
1 `$1' is `a/b' and `$2' is `a', then `${1#$2}' should yield `/b',
1 but with `pdksh' it yields the empty string.
1
1 ``COMMANDS`'
1 Posix requires shells to trim all trailing newlines from command
1 output before substituting it, so assignments like `dir=`echo
1 "$file" | tr a A`' do not work as expected if `$file' ends in a
1 newline.
1
1 While in general it makes no sense, do not substitute a single
1 builtin with side effects, because Ash 0.2, trying to optimize,
1 does not fork a subshell to perform the command.
1
1 For instance, if you wanted to check that `cd' is silent, do not
1 use `test -z "`cd /`"' because the following can happen:
1
1 $ pwd
1 /tmp
1 $ test -z "`cd /`" && pwd
1 /
1
1 The result of `foo=`exit 1`' is left as an exercise to the reader.
1
1 The MSYS shell leaves a stray byte in the expansion of a
1 double-quoted command substitution of a native program, if the end
1 of the substitution is not aligned with the end of the double
1 quote. This may be worked around by inserting another pair of
1 quotes:
1
1 $ echo "`printf 'foo\r\n'` bar" > broken
1 $ echo "`printf 'foo\r\n'`"" bar" | cmp - broken
1 - broken differ: char 4, line 1
1
1 Upon interrupt or SIGTERM, some shells may abort a command
1 substitution, replace it with a null string, and wrongly evaluate
1 the enclosing command before entering the trap or ending the
1 script. This can lead to spurious errors:
1
1 $ sh -c 'if test `sleep 5; echo hi` = hi; then echo yes; fi'
1 $ ^C
1 sh: test: hi: unexpected operator/operand
1
1 You can avoid this by assigning the command substitution to a
1 temporary variable:
1
1 $ sh -c 'res=`sleep 5; echo hi`
1 if test "x$res" = xhi; then echo yes; fi'
1 $ ^C
1
1 `$(COMMANDS)'
1 This construct is meant to replace ``COMMANDS`', and it has most
1 of the problems listed under ``COMMANDS`'.
1
1 This construct can be nested while this is impossible to do
1 portably with back quotes. Unfortunately it is not yet
1 universally supported. Most notably, even recent releases of
1 Solaris don't support it:
1
1 $ showrev -c /bin/sh | grep version
1 Command version: SunOS 5.10 Generic 121005-03 Oct 2006
1 $ echo $(echo blah)
1 syntax error: `(' unexpected
1
1 nor does IRIX 6.5's Bourne shell:
1 $ uname -a
1 IRIX firebird-image 6.5 07151432 IP22
1 $ echo $(echo blah)
1 $(echo blah)
1
1 If you do use `$(COMMANDS)', make sure that the commands do not
1 start with a parenthesis, as that would cause confusion with a
1 different notation `$((EXPRESSION))' that in modern shells is an
1 arithmetic expression not a command. To avoid the confusion,
1 insert a space between the two opening parentheses.
1
1 Avoid COMMANDS that contain unbalanced parentheses in
1 here-documents, comments, or case statement patterns, as many
1 shells mishandle them. For example, Bash 3.1, `ksh88', `pdksh'
1 5.2.14, and Zsh 4.2.6 all mishandle the following valid command:
1
1 echo $(case x in x) echo hello;; esac)
1
1 `$((EXPRESSION))'
1 Arithmetic expansion is not portable as some shells (most notably
1 Solaris 10 `/bin/sh') don't support it.
1
1 Among shells that do support `$(( ))', not all of them obey the
1 Posix rule that octal and hexadecimal constants must be recognized:
1
1 $ bash -c 'echo $(( 010 + 0x10 ))'
1 24
1 $ zsh -c 'echo $(( 010 + 0x10 ))'
1 26
1 $ zsh -c 'emulate sh; echo $(( 010 + 0x10 ))'
1 24
1 $ pdksh -c 'echo $(( 010 + 0x10 ))'
1 pdksh: 010 + 0x10 : bad number `0x10'
1 $ pdksh -c 'echo $(( 010 ))'
1 10
1
1 When it is available, using arithmetic expansion provides a
1 noticeable speedup in script execution; but testing for support
1 requires `eval' to avoid syntax errors. The following construct
1 is used by `AS_VAR_ARITH' to provide arithmetic computation when
1 all arguments are provided in decimal and without a leading zero,
1 and all operators are properly quoted and appear as distinct
1 arguments:
1
1 if ( eval 'test $(( 1 + 1 )) = 2' ) 2>/dev/null; then
1 eval 'func_arith ()
1 {
1 func_arith_result=$(( $* ))
1 }'
1 else
1 func_arith ()
1 {
1 func_arith_result=`expr "$@"`
1 }
1 fi
1 func_arith 1 + 1
1 foo=$func_arith_result
1
1 `^'
1 Always quote `^', otherwise traditional shells such as `/bin/sh'
1 on Solaris 10 treat this like `|'.
1
1