Minutes of the 18 December 2014 Teleconference Austin-686 Page 1 of 1
Submitted by Andrew Josey, The Open Group. 19 December 2014
Attendees:
Mark Ziegast, SHware Systems
Don Cragun, IEEE PASC OR
Joerg Schilling FOKUS Fraunhofer
Geoff Clare, The Open Group
Andrew Josey, The Open Group
Martin Rehak, Oracle, The Open Group OR
Eric Blake, Red Hat
Richard Hansen, BBN
Apologies
Nick Stoughton, USENIX, ISO/IEC JTC 1/SC 22 OR
David Clissold, IBM
Mark Brown, Canonical
* General news
On the PASC PAR approval progress - still open. Don currently has
the action to progress.
* Outstanding actions
+Bug 0000251: Forbid newline, or even bytes 1 through 31 (inclusive), in filenames OPEN
http://austingroupbugs.net/view.php?id=251
Don has an action to produce a proposal.
+Bug 0000561: NUL-termination of sun_path in Unix sockets OPEN
http://austingroupbugs.net/view.php?id=561
Eric has an action to update the proposal.
+Bug 0000573: Please add '+' to the portable filename character set OPEN
http://austingroupbugs.net/view.php?id=573
Joerg has an action to prepare a proposed change.
+Bug 0000592: consistent use of struct timespec OPEN
http://austingroupbugs.net/view.php?id=592
Jim had provided additional information in bugnote 1627.
This was discussed and Jim took an action to provide further information.
+Bug 0000598: OH shading and new interfaces OPEN
http://austingroupbugs.net/view.php?id=598
Eric has an action to propose a new solution with self-contained headers.
+Bug 0000517: EBNF support OPEN
http://austingroupbugs.net/view.php?id=517
Action on Joerg to look at this.
+Bug 0000633: SIGEV_THREAD delivery renders many signal interfaces unsafe
OPEN
http://austingroupbugs.net/view.php?id=633
We noted that feedback has settled down on the mailing list, and
will discuss next session.
+Bug 0000657: Conditions under which fmemopen() write a NUL to the buffer
are insufficiently specified OPEN
http://austingroupbugs.net/view.php?id=657
Eric has an action to propose wording to clarify the behavior for
fmemopen(), and also to contact the glibc developers to get their
feedback.
+Bug 0000658: Undefined/unspecified behavior clauses in description of
open have race conditions OPEN
http://austingroupbugs.net/view.php?id=658
It was noted that there is some overlap with changes in TC1. Eric took an
action to update the proposal to resolve the overlaps appropriately.
+Bug 0000615: pthread_setcancelstate should be async-signal-safe OPEN
http://austingroupbugs.net/view.php?id=615
We now have reports on AIX and Apple. Jim to report back on whether
pthread_cancelstate() is async-signal-safe on Solaris. Andrew to ask
HP whether pthread_cancelstate() is async-signal-safe on HP-UX.
+Bug 0000672: Necessary step(s) to synchronize filename operations on disk
OPEN
http://austingroupbugs.net/view.php?id=672
Geoff has a new proposed resolution in note 1618. Decided to solicit input
from FS developers. Eric to go to Linux, David to AIX and Jim to Solaris.
Jim has completed his action (see bugnote 1691).
Andrew should chase HP and Apple for input.
+Bug 0000663: Specification of str[n]casecmp is ambiguous reopened
http://austingroupbugs.net/view.php?id=663
Action on David to follow up with the IBM developers about the EBCDIC
collation sequence.
Bug 696 either NAME_MAX shouldn't be optional, or readdir_r() needs clarification
http://www.austingroupbugs.net/view.php?id=696
Don has an action to propose a resolution.
Bug 0000721: Internal storage vs static storage OPEN
http://austingroupbugs.net/view.php?id=721
This item is still open.
Bug 0000375: Extend test/[...] conditionals: ==, <, >, -nt, -ot, -ef OPEN
http://austingroupbugs.net/view.php?id=375
This is still left open due to discussions pending on the reflector.
Bug 0000789: Add set -o pipefail OPEN
http://austingroupbugs.net/view.php?id=789
* Current Business
- Bug processing
Bug #854: requirement for additional built-in utilities to be searched for via $PATH was not and is not existing practice Accepted as marked
http://austingroupbugs.net/view.php?id=854
The issue is tagged for Issue 8.
An interpretation is required.
Interpretation response:
The standard states that if a PATH search for a utility is successful,
the built-in version of that utility (if present) will always be
executed, and if the PATH search is not successful then attempts
to execute the utility must fail with exit status 127. Conforming
implementations must conform to this. However, concerns have been
raised about this which are being referred to the sponsor.
Rationale:
Many existing implementations execute a regular built-in without
performing a PATH search. This behavior does not match the normative
text, and it does not allow script authors to override regular
built-in utilities via a specially crafted PATH. In addition, the
rationale explains that the intention is to allow authors to override
built-ins by modifying PATH, but this is not what the normative
text says.
Notes to the Editor (not part of this interpretation):
On page 46 lines 1440-1443 (XBD 3.83 definition of Built-In Utility), change:
A utility implemented within a shell. The utilities referred
to as special built-ins have special qualities. Unless qualified,
the term "built-in" includes the special built-in utilities.
Regular built-ins are not required to be actually built into
the shell on the implementation, but they do have special
command-search qualities.
Note: Special Built-In Utilities are defined in detail in XCU
Section 2.14 (on page 2356). Regular Built-In Utilities are
defined in detail in XCU Section 2.9.1.1 (on page 2339).
to:
A utility implemented within a shell. There are two main types
of built-in utilities: special built-ins and regular built-ins.
Unless qualified, the term "built-in" includes both types. The
utilities referred to as special built-ins have special qualities.
Regular built-ins are not required to be actually built into
the shell on the implementation, but they usually have special
command-search qualities or affect the current execution
environment.
Note: Special Built-In Utilities are defined in detail in XCU
Section 2.14 (on page 2356). Regular Built-In Utilities are
defined in detail in XCU Section 1.6 (on page 2318).
On page 65 after line 1901 insert a new section:
3.201 Intrinsic Utility
A utility that is not subject to a PATH search during command
search, usually implemented as a regular built-in utility.
Note: Intrinsic Utilities are defined in detail in XCU Section
1.7 (on page 2318).
On page 84 after line 2374 insert a new section:
3.318 Regular Built-In Utility (or Regular Built-In)
See Built-In Utility in Section 3.83 (on page 46).
Change the description of PATH in XBD section 8, P178, L5711-5729 from:
This variable shall represent the sequence of path prefixes
that certain functions and utilities apply in searching for an
executable file known only by a filename. The prefixes shall
be separated by a <colon> (’:’). When a nonzero-length prefix
is applied to this filename, a <slash> shall be inserted between
the prefix and the filename if the prefix did not end in <slash>.
A zero-length prefix is a legacy feature that indicates the
current working directory. It appears as two adjacent <colon>
characters ("::"), as an initial <colon> preceding the rest of
the list, or as a trailing <colon> following the rest of the
list. A strictly conforming application shall use an actual
pathname (such as .) to represent the current working directory
in PATH. The list shall be searched from beginning to end,
applying the filename to each prefix, until an executable file
with the specified name and appropriate execution permissions
is found. If the pathname being sought contains a <slash>, the
search through the path prefixes shall not be performed. If the
pathname begins with a <slash>, the specified path is resolved
(see Section 4.12, on page 111). If PATH is unset or is set to
null, the path search is implementation-defined.
Since <colon> is a separator in this context, directory names
that might be used in PATH should not include a <colon> character.
to:
This variable shall represent the sequence of path prefixes
that certain functions and utilities apply in searching for
built-ins and executable files known only by a filename. The
prefixes shall be separated by a <colon> (’:’). When a
nonzero-length prefix is applied to this filename, a <slash>
shall be inserted between the prefix and the filename if the
prefix did not end in <slash>. A zero-length prefix is a legacy
feature that indicates the current working directory. It appears
as two adjacent <colon> characters ("::"), as an initial <colon>
preceding the rest of the list, or as a trailing <colon> following
the rest of the list. A strictly conforming application shall
use an actual pathname (such as .) to represent the current
working directory in PATH. The list shall be searched from
beginning to end, applying the filename to each prefix, until
either an executable file with the specified name and appropriate
execution permissions is found or, if the shell is performing
the search, a built-in associated with that prefix and filename
is found. If the pathname being sought contains a <slash>, the
search through the path prefixes shall not be performed. If the
pathname begins with a <slash>, the specified path shall be
resolved (see Section 4.12, on page 111). If PATH is unset or
is set to null, or if a path prefix in PATH contains a
<percent-sign> character ('%'), the path search is
implementation-defined.
Since <colon> is a separator in this context, directory names
that might be used in PATH should not include a <colon> character.
Since <percent-sign> may have an implementation-defined meaning
when searching for built-in utilities, directory names in PATH
that will be used to search for non-built-in utilities should
not contain a <percent-sign> character.
On page 2318 lines 73544-73553 (XCU 1.6 Built-In Utilities), change:
The utilities named in Table 1-5 are frequently provided in
built-in form. All of the utilities named in the table have
special properties in terms of command search order within the
shell, as described in Section 2.9.1.1 (on page 2339).
Table 1-5 Regular Built-In Utilities
alias
bg
cd
command
false
fc
fg
getopts
jobs
kill
newgrp
pwd
read
true
umask
unalias
wait
However, all of the standard utilities, including the regular
built-ins in the table, but not the special built-ins [...]
to:
The intrinsic utilities described in Section 1.7 (on page 2318)
are frequently provided as regular built-ins.
However, all of the standard utilities, including the regular
built-ins but not the special built-ins [...]
On page 2318 after line 73556 insert a new section:
1.7 Intrinsic Utilities
As described in Section 2.9.1.1 (on page 2339), intrinsic
utilities are not subject to a PATH search during command search
and execution. The utilities in Table 1-5 shall be intrinsic
utilities.
Table 1-5 Intrinsic Utilities
alias
bg
cd
command
fc
fg
getopts
hash
jobs
kill
read
[XSI]type[/XSI]
[XSI]ulimit[/XSI]
umask
unalias
wait
Whether any additional utility is considered an intrinsic utility
is implementation-defined. Because applications are unable to
override an intrinsic utility with a utility from PATH,
implementations should not make any utility an intrinsic utility
beyond the utilities in Table 1-5.
On page 2339 lines 74388-74389 (XCU 2.9.1.1 bullet 1.c.), change:
If the command name matches the name of a utility listed in the
following table, that utility shall be invoked.
alias
bg
cd
command
false
fc
fg
getopts
jobs
kill
newgrp
pwd
read
true
umask
unalias
wait
to:
If the command name matches the name of an intrinsic utility
(see Section 1.17 on page 2318), that utility shall be invoked.
On page 2340 lines 74397-74398 (XCU 2.9.1.1 bullet 1.d.i.a.), change:
If the system has implemented the utility as a regular built-in
or as a shell function, it shall be invoked at this point in
the path search.
to:
If the system has implemented the utility as a built-in or as
a shell function, and the built-in or function is associated
with the directory that was most recently tested during the
successful PATH search, that built-in or function shall be
invoked.
Add the following paragraphs to the end of the rationale for the
PATH environment variable in XRAT subclause A.8.3 after P3499,
L118222:
The standard specifies that (when no <slash> character is
included in a command pathname) special built-in utilities and
intrinsic utilities are not subject to a search using PATH. All
other standard utilities, even if implemented as shell built-ins,
are required to be found by searching PATH. This means that if
a shell includes a built-in for a standard utility that is not
intrinsic, a user can write a utility that will override that
built-in. The standard also requires that all standard utilities
can be executed by commands like:
find . -type d -exec printf 'Found directory: %s\n' '{}' +
So, other than differences caused by using different shell
execution environments, a standard utility that is implemented
as a built-in and the non-built-in version of that standard
utility are both required to behave as the standard specifies.
But, if a non-standard utility is found in PATH before the
standard utility's location in PATH, the non-standard utility
must be invoked rather than the built-in. For instance, if the
shell includes a built-in printf utility (which most shells
do), PATH is initialized using:
PATH="$HOME/bin:$(command -p getconf PATH)"
and <tt>$HOME/bin/printf</tt> is an executable file containing:
command -p printf 'In %s with args:\n' "${0##*/}" >&2
command -p printf '\t%s\n' "$@" >&2
command -V printf >&2
command -Vp printf >&2
command -p printf "$@"
then the command:
printf '%s %s\n' HOME "$HOME" PATH "$PATH"
should produce output similar to:
In printf with args:
%s %s\n
HOME
/Users/dwc
PATH
/Users/dwc/bin:/usr/bin:/bin:/usr/sbin:/sbin
printf is a tracked alias for /Users/dwc/bin/printf
printf is a shell builtin
HOME /Users/dwc
PATH /Users/dwc/bin:/usr/bin:/bin:/usr/sbin:/sbin
The current version of the Korn shell installs built-ins into
the shell using a builtin utility that allows the built-in to
be associated with the pathname of the non-built-in version of
that utility. (Unfortunately, some implementations that use
ksh93 as their standard sh utility do not make use of this
feature and install built-ins for standard utilities that are
not associated with a PATH search. And, most other shells
incorrectly always use a built-in utility if one is installed,
even when it should be overridden by a PATH search that should
find the non-standard version of a utility with the name of
that built-in.) Some other shells use a <percent-sign> character
in a directory pathname in PATH to indicate one or more directories
that should be used when processing PATH to determine when
non-intrinsic standard utilities should be found. The POSIX.1-201x
revision of the standard allows either of these methods to be
used to install built-ins that meet the requirements stated in
XCU subclause 2.9.1.1 by making the behavior of the built-in
path search implementation-defined when a <percent-sign> character
is found in PATH.
On page 3674 lines 125546-125552 (XRAT C.1.7 rationale for XCU 1.6 Built-In Utilities), change:
All of these utilities can be exec-ed. There is no requirement
that these utilities are actually built into the shell itself,
but many shells need the capability to do so because XCU Section
2.9.1.1 (on page 2339) requires that they be found prior to the
PATH search. The shell could satisfy its requirements by keeping
a list of the names and directly accessing the file-system
versions regardless of PATH. Providing all of the required
functionality for those such as cd or read would be more
difficult.
There were originally three justifications for allowing the
omission of exec-able versions:
to:
Other than the special built-in utilities, there is no requirement
to build utilities into the shell itself. However, many shells
implement certain utilities as regular built-ins for the following
reasons:
To improve performance, especially for frequently used
lightweight utilities (such as test, true, and false).
To eliminate the need for some sort of interprocess
communication between the shell and those utilities that
read or modifiy the shell's execution environment (such as
cd).
To make it easier to satisfy the command search and execution
requirements in XCU Section 2.9.1.1 (on page 2339) for
intrinsic utilities. Intrinsic utilities must be found prior
to the PATH search. The shell could satisfy this requirement
by keeping a list of the intrinsic utility pathnames and
directly accessing the file-system versions regardless of
PATH, but these utilities usually need to read or modify
the shell's execution environment anyway.
With the exception of the intrinsic utilities, all regular
built-in utilities are subject to the PATH search and can be
overridden by a specially crafted PATH environment variable.
All of the regular built-in utilities can be exec-ed. There
were originally three justifications for allowing the omission
of exec-able versions:
On page 3675 between lines 125572 and 125573, insert a new section heading:
C.1.8 Intrinsic Utilities
On page 3675 line 125573, change:
There were varying reasons for including utilities in the table
of built-ins:
to:
There were varying reasons for including utilities in the table
of intrinsic utilities:
On page 3675 line 125580, change:
cd, getopts, newgrp, read, umask, wait
to:
cd, getopts, hash, read, [XSI]type, ulimit,[/XSI] umask, wait
Remove page 3675 lines 125599-125602 (true, false).
On page 3675 after line 125602 insert a new paragraph:
The following utilities are frequently implemented as intrinsic
(and built-in) utilities. Future versions of this standard might
not allow these utilities, or any other standard utility not
in Table 1-5 in XCU 1.7 on page 2318, to be intrinsic;
implementations are encouraged to implement these as non-intrinsic
utilities instead (but still built-in if they were previously
built-in).
[
echo
false
newgrp
printf
pwd
test
true
On page 3694 lines 126360-126366 (rationale for XCU 2.9.1.1), change:
The sequence selected for the Shell and Utilities volume of
POSIX.1-2008 acknowledges that special built-ins cannot be
overridden, but gives the programmer full control over which
versions of other utilities are executed. It provides a means
of suppressing function lookup (via the command utility) for
the user's own functions and ensures that any regular built-ins
or functions provided by the implementation are under the control
of the path search. The mechanisms for associating built-ins
or functions with executable files in the path are not specified
by the Shell and Utilities volume of POSIX.1-2008, [...]
to:
The sequence selected for the Shell and Utilities volume of
POSIX.1-2008 acknowledges that special built-ins cannot be
overridden, but gives the programmer full control over which
versions of other utilities are executed (with some exceptions).
It provides a means of suppressing function lookup (via the
command utility) for the user's own functions and, with the
exception of the intrinsic utilities (see XCU Section 1.7),
ensures that any regular built-ins or functions provided by the
implementation are under the control of the path search. The
mechanisms for associating non-intrinsic built-ins or functions
with executable files in the path are not specified by the Shell
and Utilities volume of POSIX.1-2008, [...]
Bug #879: strptime is missing conversion specifiers described in strftime OPEN
http://austingroupbugs.net/view.php?id=879
This was discussed during the call and notes made in the etherpad.
http://posix.rhansen.org:9001/p/2014-12-18
We will continue this item on the next call.
Next Steps
----------
The next call is on January 8, 2014 (a Thursday)
The timetable for upcoming calls is Dec 18, and then Jan 8 2015
Calls are anchored on US time. (8am Pacific)
This call will be for the regular 90 minutes.
http://austingroupbugs.net
An IRC channel will be available for the meeting
irc://irc.freenode.net/austingroupbugs
An etherpad is usually up for the meeting, with a URL using the date format as below:
http://posix@posix.rhansen.org:9001/p/201x-mm-dd
password=2115756#