Unapproved draft Last Updated 11 Jan 2008 ____________________________________________________________________________ EDITORIAL Enhancement Request Number 1 e.landry:opengroup.org Defect in XBD 13 Headers (rdvk# 1) {redundancy} Thu, 11 Sep 2003 07:49:08 +0100 (BST) __________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject____ Rationale for rejected or partial changes: This is fixed in the 2004 edition Accept and add to the errata since editorial (note that this will be post tc2 errata) ___________________________________________________________________________ Page: 341 Line: 12119,12122 Section: 13 Problem: Edition of Specification (Year): 2003 Defect code : 1. Error Line 12122 is redundant with line 12119 in the same context. This can also be found in the 2001 edition on page 339 lines 12060,12063. For reference the lines content are: "[TYM] The Typed Memory Objects option" Action: Delete line 12122 (or update it to refer to the right option whatever that is). _______________________________________________________________________________ COMMENT Enhancement Request Number 2 glenn.lagasse:sun.com Defect in XBD arpa/inet.h (rdvk# 2) {gl-00001} Fri, 3 Oct 2003 20:08:52 +0100 (BST) ______________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: No change is necessary since arpa/inet.h is allowed to make netinet/in.h visible, and netinet/in.h is allowed to make sys/socket.h visible. _______________________________________________________________________________ Page: 258 Line: 0 Section: arpa/inet.h Problem: Edition of Specification (Year): 2003 Defect code : 3. Clarification required The arpa/inet.h header definition does not provide a provision for specifically including the sys/socket.h header file. Since the function prototype for inet_ntop() has the last parameter type defined as socklen_t (which is TYPEDEF'd in sys/socket.h). This would appear to be an ommision on the part of the specification that needs to be addressed to prevent cries of name space pollution. Interestingly, the current testsuite does not point out a possible name space pollution when sys/socket.h is included in arpa/inet.h which lends more credence to this being an oversight in the specification imho. Action: Specifically mention/allow the inclusion of sys/socket.h in the arpa/inet.h header. _____________________________________________________________________________ OBJECTION Enhancement Request Number 3 don.cragun:sun.com Defect in XBD 4.11 (rdvk# 1) {dwc:PathnameResolution-1} Tue, 11 Nov 2003 22:40:54 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Note that an interpretation has been filed on this item AI-016 ERN 3(->interps) AI-016 _____________________________________________________________________________ Page: 102-103 Line: 3161-3163 Section: 4.11 Problem: Edition of Specification (Year): 2003 Defect code : 1. Error I am requesting a formal interpretation of the meaning of trailing slashes in the general concept Pathname Resolution in subclause 4.11 in IEEE Std 1003.1-2001 (2003 edition, pages 102-103, lines 3161-3163). I do not believe the normative text matches the intent specified in the rationale. I also believe the normative text places unneeded restrictions on applications and implementations. The current POSIX standard finally implemented some changes in the definition of pathname resolution based on concerns that were discussed years ago at POSIX meetings about trailing slash characters in pathnames. I have discussed this issue with several people that were involved in the original discussions that lead to the text in the P1003.1a drafts that were incorporated into the definition of the Pathname Resolution general concept in XBD6 subclause 4.11. All have agreed that the current wording would have serious side effects if implementations were written to the letter of these requirements. (Specifically, things like rmdir("dir/") are required to fail even though the intent of the changes introduced into P1003.1a drafts was intended to allow this to succeed in cases where "dir" names an existing directory. The normative text of the standard is silent on what happens with rename("dir/", "dir2") when "dir2" names an existing empty directory or does not name an existing file, but the rationale clearly states that "dir/." and "dir/.." are not allowed. The intent of the working group that drafted these changes was to allow a trailing slash here. We have also discussed whether mkdir("new/") and rename("dir/", "new/") should succeed or fail when "new" does not name an existing file.) The way that this requirement is worded in the current POSIX standard (POSIX.1, 2003 edition, Base Definitions volume, page 102-103, lines 3161-3163 is: "A pathname that contains at least one non-slash character and that ends with one or more trailing slash characters shall be resolved as if a single dot character ('.') were appended to the pathname." This wording requires failure for these cases: 1. rmdir("dir/.") is required to fail with EINVAL, 2. mkdir("new/.") is required to fail with ENOTDIR, and 3. rename(..., "new/.") is required to fail with ENOTDIR. As mentioned above, rename() does not explicitly mention rename("dir/.", ...) in normative text, but the rationale says: "Renaming dot or dot-dot is prohibited in order to prevent cyclical file system paths." (See XSH6, P1216, L37967). I believe the intent of the working group was that if a pathname has a trailing slash character, it can only be used to refer to a directory. The original SVID3 wording (from Volume 1, page 4-11, top two paragraphs) in the definition of "pathname and path prefix" includes: "A pathname is used to identify a file. It consists of at most, {PATH_MAX} bytes, including a terminating null character. It has an optional beginning slash, followed by zero or more filenames separated by *** slashes. If the pathname refers to a directory, it *** may also have one or more trailing slashes. Multiple consecutive slashes may be interpreted in an implementation-defined manner, although more than two leading slashes are treated as a single slash. If a pathname begins with a slash, the path search begins at the root directory. Otherwise, the search *** begins from the current working directory. If a *** pathname refers to a directory, it may also have one *** or more trailing slashes. Multiple consecutive slashes are considered the same as a single slash." (Note that SVID3 was one of the reference documents used frequently in the development of POSIX.1-1990.) I believe the two complete sentences on the line marked with *** above cover the same intent, but System V did not enforce the part about trailing slashes being allowed only after directories and always threw away trailing slash characters. (In fact, SVID3 never specified what is supposed to happen if a pathname that does not refer to a directory has one or more trailing slashes.) Unfortunately, the wording from SVID3 is ambiguous. "If a pathname refers to a directory" could mean "If a pathname refers to aN EXISTING directory" or could mean "If a pathname refers to SOMETHING THAT WILL BE USED AS a directory". The unanimous consensus of the people I have discussed this with so far is that the desired meaning is the latter, except for a special case for the last argument to rename(). A short summary of the discussions leading to this conclusion follows: In these examples "dir" and "dir2" refer to existing files of type directory and "new" refers to a non-existent file. mkdir(): We all believe that mkdir("new") should be equivalent to mkdir("new/") although there were dissenting opinions at the start of the discussion. There are several uses of mkdir("new/") in third party applications and this is clearly in line with one interpretation of the SVID3 wording. Many of these applications are creating a temporary directory and saving the directory name (with the trailing slash) for later use to create pathnames for files to be created in that directory. These applications save a step by including the trailing slash when the temporary directory is created and did not believe they were breaking the intended requirements of any standard. rename(): We all believe that rename("dir", "new") and rename("dir/", "new") should be equivalent. We all believe that rename("dir", "new") should not be treated the same as rename("dir", "new/") although there were dissenting opinions at the start of the discussion. We believe that in this case, "new/" should fail if new does not name an existing directory. The rename() function is very similar to the mv utility when given two operands that are on the same file system. For the mv utility, the command: mv dir dir2/ should always rename dir to be dir2/dir or fail (matching historic BSD mv behavior). To match this, rename("dir", "dir2") and rename("dir", "dir2/") should rmdir("dir2") and rename("dir", "dir2") as an atomic operation, but rename("dir", "new/") should fail (since "new" is not an existing directory). rmdir(): We all believe that rmdir("dir") and rmdir("dir/") should be equivalent. Action: Assuming that the interpretation is that the standard needs to be changed to match the intent of the working group, I propose the following changes in the next revision of the standard: Change XBD6, P102, L3161-3163 from (definition of Pathname Resolution general concept): ``A pathname that contains at least one non-slash character and that ends with one or more trailing slash characters shall be resolved as if a single dot character ('.') were appended to the pathname.'' to: ``If a pathname contains at least one non-slash character and ends with one or more trailing slashes after all symbolic links have been processed, and if the final component of the pathname before the trailing slashes names a non-existant file in an operation that is not attempting to create a directory or refers to an existing file that is not a directory, the implementation shall consider this an error. Otherwise, if a pathname contains at least one non-slash character and ends with one or more trailing slashes after all symbolic links have been processed, the pathname shall be resolved as though all trailing slashes after the last non-slash character were removed. Interfaces using pathname resolution may specify additional constraints when a pathname contains one or more trailing slashes and does not name an existing directory.* __________________ * The only interfaces that further constrain pathnames that do not name existing directories in this standard are rename() in the System Interfaces volume of this standard and the rm utility in the Shell and Utilities volume of this standard.'' and move this changed text to follow P103, L3180. Change XRAT6, P37, L1481-1482 from (rationale for Pathname Resolution general concept): ``IEEE Std 1003.1-2003 requires that a pathname with a trailing slash character be treated as if it had a trailing "/." everywhere.'' to: ``An earlier version of this standard required that a pathname with a trailing slash character be treated as if it had a trailing "/." everywhere. This unintentionally broke applications that used mkdir("newdir/") and rmdir("existing-dir/"). This standard requires that a pathname with a trailing slash be rejected unless it refers to a file that is a directory or to a file that is to be created as a directory. The rename() function and the mv utility further specify that a trailing slash cannot be used on a pathname naming a file that that does not exist when used as the last argument to rename() or the as the last operand to mv.'' Add the new sentence (CX shaded): ``If the new argument does not resolve to an existing file of type directory and the new argument contains at least one non-slash character and ends with one or more trailing slashes after all symbolic links have been processed, rename() shall fail.'' to the end of the paragraph on XSH6, P1214, L37861-37862 (2nd paragraph of the description of rename()). Change XSH6, P1215, L37922-37933 (shall fail ENOTDIR error in rename()) from: ``[ENOTDIR] A component of either path prefix is not a directory; or the old argument names a directory and the new argument names a non-directory file.'' to: ``[ENOTDIR] A component of either path prefix is not a directory; the old argument names a directory and the new argument names a non-directory file; or the new argument names a non-existant file and ends with a trailing slash character.'' all CX shaded. Add the new sentence: ``In this case, if target_file ends with a trailing slash character, mv shall treat this as an error and no source_file operands will be processed.'' to the end of the paragraph on XCU6, P655, L25352-25355 (1st paragraph of the description of mv). (Note that a separate aardvark report has already been filed to change "target_file" in the second synopsis form on P655, L25350 to "target_dir".) To also fix the inconsistency between the normative text and the rationale in rename()'s handling of pathnames whose last component is dot or dot-dot, I also suggest changing the EINVAL shall fail error condition description on XSH6, P1215, L37907-37908 from: ``[EINVAL] The new directory pathname contains a path prefix that names the old directory.'' to: ``[EINVAL] The new directory pathname contains a path prefix that names the old directory, or the final pathname component of old or new is dot or dot-dot.'' If accepted, these changes would allow applications to perform the following operations successfully (assuming non-existing does not name any existing file and assuming existing-dir and another-existing-dir resolve to existing directories upon which the process has permission to operate): mkdir("non-existing/", ...) mkdir("non-existing", ...) rmdir("existing-dir/") rmdir("existing-dir") rename("existing-dir/", "another-existing-dir/") rename("existing-dir", "another-existing-dir/") rename("existing-dir/", "another-existing-dir") rename("existing-dir", "another-existing-dir") And all of the following would fail (assuming non-dir names an existing file that is not a directory): open("non-dir/", ..., ...) mkdir("non-dir/") unlink("non-dir/") rename("non-dir/", ...) rename(..., "non-dir/") rename(..., "non-existing/") _____________________________________________________________________________ OBJECTION Enhancement Request Number 4 gwc:opengroup.org Defect in XBD limits.h (rdvk# 2) [gwc OPEN_MAX] Tue, 4 Nov 2003 18:20:19 +0000 _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD/5 Suggested change for next revision Go with option 1, do the [EMFILE] change as suggested, note that the exact changes still need enumeration. Also see Austin/280 Editors notes arising from SD/5 _____________________________________________________________________________ Page: 248 Line: 8724 Section: limits.h Problem: Defect code : 1. Error The description of OPEN_MAX on the limits.h page is: {OPEN_MAX} Maximum number of files that one process can have open at any one time. This is fine if OPEN_MAX is a constant defined in limits.h, but if the limit varies then I believe the description does not match current behaviour in the case where a file descriptor >= OPEN_MAX is already open. I do not think current systems keep an independent count of open file descriptors. They search for the first available file descriptor and if they reach OPEN_MAX without finding one then they give an EMFILE error. The standard does not currently allow this behaviour - it requires an EMFILE error when OPEN_MAX file descriptors are open, even if some file descriptors numbered less than OPEN_MAX are still available. The problem can be solved either by changing the description of OPEN_MAX, the description of EMFILE in the "Error Numbers" section of XSH6, and the descriptions of EMFILE for individual functions in XSH6; or by forbidding the situation where a fd >= OPEN_MAX is open. Note that I proposed the latter in March 2002 (along with the change that allowed sysconf(_SC_OPEN_MAX) to return different values before and after a call to setrlimit() that changes RLIMIT_NOFILE), and it was rejected then. However, this new aspect of the "fd >= OPEN_MAX" problem has come to light since then and so was not taken into account. Action: Either 1. Change "Maximum number of files that one process can have open at any one time" to "A value one greater than the maximum value that the system may assign to a newly-created file descriptor". On XSH6 page 24 lines 964-965 change: "[EMFILE] Too many open files. An attempt was made to open more than the maximum number of file descriptors allowed in this process." to: "[EMFILE] File descriptor too large. An attempt was made to open a file descriptor with a value greater than or equal to {OPEN_MAX}, [XSI] or greater than or equal to the soft RLIMIT_NOFILE limit for the process (if smaller than {OPEN_MAX}).[/XSI]" and make suitable changes to the EMFILE descriptions on individual function pages. Or 2. Insert before XSH6 page 1469 line 45443, "The value returned by sysconf(_SC_OPEN_MAX) shall be either -1 or a value greater than the highest currently open file descriptor." _____________________________________________________________________________ EDITORIAL Enhancement Request Number 5 heukelum:freemail.nl Defect in XBD 0 (rdvk# 2) {0} Mon, 8 Mar 2004 16:45:26 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 This response is annotated as per the original request. (1a) This is a duplicate report XBD ERN 1 which has already been added as an errata item. No further action is necessary. (1b) Page 341 (2003 Ed). Accept, an additional list item "o [MR] |X> Range Memory Locking #include (2a) Page 308, line 10983 (2003 Ed) Change to "The following shall be declared as functions and may also be defined as macros. Function prototypes shall be provided." (2b) Page 373, line 13196 (2003 Ed) Change to: "The following shall be declared as a function and may also be defined as a macro. A function prototype shall be provided." (3a) Accept. Page 325, line 11640 (2003 Ed) Errata item: The function prototype of the function `vsnprintf' misses a closing bracket. (3b) This item only affects the html translation where two end option markers ( (4a) Page 238, line 8348 (2003 Ed) Errata, extra white space not needed in the function prototype for globfree, a space is printed in between the function name `globfree' and the following opening bracket. The space should be removed. (4b) Page 264, line 9349 (2003 Ed) Errata, In the list of function prototypes, a space is printed in between the function name `localeconv' and the following opening bracket. The space should be removed. Html help file (note that this is not part of the normative standard but an addition for useability of the html translation) (5a) In the section "Codes" of the help file (at least in the on-line version), an inconsistancy exists between the paragraphs describing the short-hand notations [MC1] and [MC2], and the short-hand notation [MC3]. The paragraphs [MC1] and [MC2] end with "Refer to _Margin_Code_Notation_", followed by "See help on _Margin_Code_Notation_.", while [MC3] has only the line "Refer to _Margin_Code_Notation_". Add the line "See help on _Margin_Code_Notation_." to the paragraph of [MC3]. (5b) Add the new MC4 notation to XBD 1.5.1 Codes, and also to the html help file. I Add after the paragraph describing [MC3]: ---start--- [MC4] |X> Memory Mapped Files, Shared Memory Objects, Process Memory Locking, Memory Protection, Typed Memory Objects, Synchronized Input and Output, Advisory Information, or Range Memory Locking The MC4 margin code is shorthand for MF|SHM|ML|MPR|TYM|SIO|ADV|MR. Features which are shaded with this margin code require support of either the Memory Mapped Files, Shared Memory Objects, Process Memory Locking, Memory Protection, Typed Memory Objects, Synchronized Input and Output, Advisory Information, or Range Memory Locking options. Range Memory Locking #include Memory Mapped Files, Shared Memory Objects, Process Memory Locking, Memory Protection, Typed Memory Objects, Synchronized Input and Output, Advisory Information, or Range Memory Locking The MC4 margin code is shorthand for MF|SHM|ML|MPR|TYM|SIO|ADV|MR. Features which are shaded with this margin code require support of either the Memory Mapped Files, Shared Memory Objects, Process Memory Locking, Memory Protection, Typed Memory Objects, Synchronized Input and Output, Advisory Information, or Range Memory Locking options. Page 347 2003 (ed) Problem [A] The layout of the header should be reformatted as follows: --changes start at p347 line 12338 new layout shown below-- The header shall define the fd_set type as a structure. The header shall define the following macro: FD_SETSIZE Maximum number of file descriptors in an fd_set structure. The following shall be declared as functions, defined as macros, or both. If functions are declared, function prototypes shall be provided." (ed note:code segment) void FD_CLR(int fd, fd_set *fdset); int FD_ISSET(int fd, fd_set *fdset); void FD_SET(int fd, fd_set *fdset); void FD_ZERO(fd_set *fdset); (ed note: end code segment) If implemented as macros, these may evaluate their arguments more than once, so applications should ensure that the arguments they supply are never expressions with side effects. The following shall be declared as functions and may also be defined as macros. Function prototypes shall be provided. --end-- (1a) No change required (1b) Page 398 line 14039 Change comma to full stop in: The following shall be declared as functions and may also be defined as macros, Function prototypes shall be provided. (1c) A duplicate request of XBD ERN 5 2(a) and 2(b) (2a) No Change required (2b) The group felt that the wording is fine and that no change is required. (2c) page 426 l 15170 Remove "name" from the end of the sentence: The header shall define the following macro name: p214 line 7615 (2003 Ed) Change The following are defined as macros: to The header shall define the following as macros: (3a) Page 208 l 7355 (2003 Ed) Change: "The following shall either be declared as functions, defined as macros, or both." To: "The following shall be declared as functions, or defined as macros, or both." (3b) Page 302 l 10748 (2003 Ed) Change from: "The following may be declared as a function, or defined as a macro, or both. Function prototypes shall be provided." To: "The following shall be declared as functions, or defined as macros, or both. If functions are declared, function prototypes shall be provided.". 302 line 10748 (3c) Already fixed in Item A. _____________________________________________________________________________ Page: 0 Line: 0 Section: 0 Problem: Edition of Specification (Year): 2003 Defect code : 3. Clarification required http://www.opengroup.org/onlinepubs/007904975/basedefs/sys/select.h.html Problem [A] In the section describing the header file sys/select, the symbols FD_CLR, FD_ISSET, FD_SET, and FD_ZERO are explained. This is superfluous, because they are also explained in the XSH/"System Interfaces"-part of the standard. Moreover, the typesetting of these functions and the placing of the list are inconsistent with other parts of the standard. AJ>Ok, should be more like ntohl in arpa/inet.h Problem [B] Below, I sum up the wordings that are used as a caption of a table of function declarations and/or macro definitions. If I interpret them correctly, there are only three possible meanings: -1- The interface must be declared as a real function, (the C-library must export the symbol), but may _additionaly_ be defined as a macro. -2- The interface shall not be declared as a function, (the C-library shall not export the symbol), but it must be available as a macro. -3- The implementation may choose between 1 and 2. However, the wordings are not used consistently, which can (and by Murphy's law will) lead to some confusion. I propose to unify the wordings. FUNCTION -------- http://www.opengroup.org/onlinepubs/007904975/basedefs/aio.h.html (1a) "The following shall be declared as functions and may also be defined as macros. Function prototypes shall be provided." in the header files aio, arpa/inet, complex, ctype, dirent, dlfcn, fcntl, fenv, ftw, glob, grp, iconv, inttypes, libgen, locale, math, mqueue, ndbm, net/if, netdb, nl_types, pthread, pwd, regex, sched, search, semaphore, setjmp, spawn, stdio, stdlib, string, strings, stropts, sys/mman, sys/msg, sys/resource, sys/select, sys/sem, sys/shm, sys/socket, sys/stat, sys/statvfs, sys/time, sys/uio, sys/wait, syslog, termios, time, trace, unistd, utmpx, wchar, wctype, and wordexp. - or (if only item) - http://www.opengroup.org/onlinepubs/007904975/basedefs/fnmatch.h.html "The following shall be declared as a function and may also be defined as a macro. A function prototype shall be provided." in the header files fmtmsg, fnmatch, langinfo, monetary, poll, sys/ipc, sys/timeb, sys/times, ulimit and utime. http://www.opengroup.org/onlinepubs/007904975/basedefs/ucontext.h.html (1b) "The following shall be declared as functions and may also be defined as macros, Function prototypes shall be provided." in the header file ucontext. http://www.opengroup.org/onlinepubs/007904975/basedefs/signal.h.html (1c) "The following shall be declared as functions and may also be defined as macros:" in the header file signal. - or (if only item) - http://www.opengroup.org/onlinepubs/007904975/basedefs/sys/utsname.h.html "The following shall be declared as a function and may also be defined as a macro:" in the header file sys/utsname. MACRO ----- http://www.opengroup.org/onlinepubs/007904975/basedefs/complex.h.html (2a) "The header shall define the following macros..." (sometimes followed by a short further explanation) in the header files complex, fmtmsg, math, net/if, netdb (4x), netinet/in (6x), netinet/tcp, stdbool, stddef, stdio, stdlib, sys/socket (7x), sys/wait, and wchar. http://www.opengroup.org/onlinepubs/007904975/basedefs/fmtmsg.h.html (2b) "The header shall also define the following macros for use as return values for fmtmsg():" in the header file fmtmsg. http://www.opengroup.org/onlinepubs/007904975/basedefs/wctype.h.html (2c) "The header shall define the following macro name:" in the header file wctype. http://www.opengroup.org/onlinepubs/007904975/basedefs/sys/select.h.html (2d) "The following shall be defined as a macro:" in the header file sys/select. http://www.opengroup.org/onlinepubs/007904975/basedefs/ctype.h.html (2e) "The following are defined as macros:" in the header file ctype. YOUR CHOICE ----------- http://www.opengroup.org/onlinepubs/007904975/basedefs/arpa/inet.h.html (3a) "The following shall either be declared as functions, defined as macros, or both. If functions are declared, function prototypes shall be provided." in the header file arpa/inet. http://www.opengroup.org/onlinepubs/007904975/basedefs/setjmp.h.html (3b) "The following may be declared as a function, or defined as a macro, or both. Function prototypes shall be provided." in the header file setjmp. http://www.opengroup.org/onlinepubs/007904975/basedefs/sys/select.h.html (3c) "Each of the following may be declared as a function, or defined as a macro, or both:" in the header file sys/select. Action: Problem [A] The section describing the header file sys/select should be altered in the following ways (the first and last paragraphs are unchanged, included here for orientation; comments are marked as {{ this is a comment }}): --start-- The header shall define the fd_set type as a structure. {{ A part of the text has been moved down... }} The following shall be defined as a macro: {{ OR, if the relevant change of [B] is accepted: "The header shall define the following macro:" }} FD_SETSIZE Maximum number of file descriptors in an fd_set structure. {{ ... to this position... }} Each of the following may be declared as a function, or defined as a macro, or both: {{ OR, if the relevant change of [B] is accepted: "The following shall be declared as functions, defined as macros, or both. If functions are declared, function prototypes shall be provided." }} {{ The following list should be typeset in the same way as the pselect and select function declarations below. }} void FD_CLR(int fd, fd_set *fdset); {{ note: semicolon 4x }} int FD_ISSET(int fd, fd_set *fdset); void FD_SET(int fd, fd_set *fdset); void FD_ZERO(fd_set *fdset); If implemented as macros, these may evaluate their arguments more than once, so applications should ensure that the arguments they supply are never expressions with side effects. {{ ... end of moved part of the text. }} The following shall be declared as functions and may also be defined as macros. Function prototypes shall be provided. --end-- Problem [B] Proposed changes: FUNCTION -------- (1a) (reference version) http://www.opengroup.org/onlinepubs/007904975/basedefs/ucontext.h.html (1b) correct the typo (comma instead of full stop) in ucontext.h. Ok http://www.opengroup.org/onlinepubs/007904975/basedefs/signal.h.html http://www.opengroup.org/onlinepubs/007904975/basedefs/sys/utsname.h.html (1c) change the wording in signal.h and sys/utsname.h to add the obligation to provide a prototype. I filed a bug report earlier for (1c), but I include it here again for completeness. MACRO ----- (2a) (reference version) http://www.opengroup.org/onlinepubs/007904975/basedefs/fmtmsg.h.html (2b) The (superfluous) word "also" makes it unclear that the macros in the following list are the only possible return values. It should be removed. http://www.opengroup.org/onlinepubs/007904975/basedefs/wctype.h.html (2c) The word "name" is used incorrectly here. It should be removed. http://www.opengroup.org/onlinepubs/007904975/basedefs/sys/select.h.html http://www.opengroup.org/onlinepubs/007904975/basedefs/ctype.h.html (2d) and (2e) These are the only other inconsistancies I could find. Changing them to "The header shall define the following macro:" and "The header shall define the following macros:" would make me happier. YOUR CHOICE ----------- http://www.opengroup.org/onlinepubs/007904975/basedefs/arpa/inet.h.html (3a) The word either implies a choice of 2 possiblities. However, there are three possibilities. Remove the word "either", and add "or" after the first comma (so it reads "... as functions, or defined as macros, or both.") http://www.opengroup.org/onlinepubs/007904975/basedefs/setjmp.h.html (3b) The word "may" implies that the interface may in fact be absent. It should be replaced by "shall". The first sentence, then, still allows a macro-only interface, but the second sentence implies that the function should be available too, which seems to be incorrect. The second sentence should be changed to "If functions are declared, function prototypes shall be provided.". http://www.opengroup.org/onlinepubs/007904975/basedefs/sys/select.h.html (3c) The word "may" implies that the interface may in fact be absent. For consistancy, this sentence should be replaced by "The following shall be declared as functions, defined as macros, or both. If functions are declared, function prototypes shall be provided." Also, the list that follows this caption is typeset differently than usual (see Problem [A]). _____________________________________________________________________________ COMMENT Enhancement Request Number 7 gwc:opengroup.org Defect in XBD 9.3.6 (rdvk# 1) [gwc BRE nested subpatterns] Mon, 1 Mar 2004 15:15:55 +0000 _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 7(->interps) AI-036 (see AI-036) This is an Interpretation The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Notes to the Editor for a future revision (not part of this interpretation): Proposed changes: 9.3.6 BREs Matching Multiple Characters [* Current description (page 174 lines 6129-6135): *] For example, the expression "\(.*\)\1$" matches a line consisting of two adjacent appearances of the same string, and the expression "\(a\)*\1" fails to match 'a' . When the referenced subexpression matched more than one string, the back-referenced expression shall refer to the last matched string. If the subexpression referenced by the back-reference matches more than one string because of an asterisk ( '*' ) or an interval expression (see item (5)), the back-reference shall match the last (rightmost) of these strings. [* Change to: *] The string matched by a contained subexpression shall be within the string matched by the containing subexpression. If the containing subexpression does not match, or if there is no match for the contained subexpression within the string matched by the containing subexpression then back-reference expressions corresponding to the contained subexpression shall not match. When a subexpression matches more than one string, a back-reference expression corresponding to the subexpression shall refer to the last matched string. For example, the expression "^\(.*\)\1$" matches lines consisting of two adjacent appearances of the same string, the expression "\(a\)*\1" fails to match 'a', the expression "\(a\(b\)*\)*\2" fails to match 'abab', and the expression "^\(ab*\)*\1$" matches 'ababbabb' but fails to match 'ababbab'. ed [* Current description (page 345 lines 13312-13313): *] As a more general feature, the characters '\n' , where n is a digit, shall be replaced by the text matched by the corresponding back-reference expression. [* Add this: *] If the corresponding back-reference expression does not match then the characters '\n' shall be replaced by the empty string. ex [* Current description (page391 lines 15168-15170): *] The sequence '\n' , where n is an integer, shall be replaced by the text matched by the pattern enclosed in the nth set of parentheses '\(' and '\)'. [* Change to: *] The sequence '\n' , where n is an integer, shall be replaced by the text matched by the corresponding back-reference expression. If the corresponding back-reference expression does not match then the characters '\n' shall be replaced by the empty string. expr [* Current description (page 431 lines 16738-16739): *] Alternatively, if the pattern contains at least one regular expression subexpression "[\(...\)]", the string corresponding to "\1" shall be returned. [* Change to: *] Alternatively, if the pattern contains at least one regular expression subexpression "[\(...\)]", the string matched by the back-reference expression "\1" shall be returned. If the back-reference expression "\1" does not match then the null string shall be returned. pax [* Current description (page 707 line 27455): *] '\n' (where n is a digit) backreferences, ... [* Change to: *] '\n' (where n is a digit) back-references, ... sed [* Current description (page 846 lines 32803-32804): *] The characters "\n", where n is a digit, shall be replaced by the text matched by the corresponding backreference expression. [* Change to: *] The characters "\n", where n is a digit, shall be replaced by the text matched by the corresponding back-reference expression. If the corresponding back-reference expression does not match then the characters "\n" shall be replaced by the empty string. _____________________________________________________________________________ Page: 172 Line: 6105-6109 Section: 9.3.6 Defect code : 3. Clarification required Problem: There seems to be a discrepancy between the description of BREs in XBD6 and the description of regcomp() in XSH6 as regards the treatment of nested subpatterns with a following * repeater. The example that prompted this is whether: echo aba | sed 's/\(a\(b\)*\)*/<\1|\2>/' should output or . According to the description of BREs: "If the subexpression referenced by the back-reference matches more than one string because of an asterisk ( '*' ) or an interval expression (see item (5)), the back-reference shall match the last (rightmost) of these strings." which seems to imply that the correct output is since the last string that the second subpattern matches is the b that it got from the first iteration of the first subpattern. (In the second iteration of the first subpattern the second subpattern doesn't match anything.) Several sed implementations do output . However, the description of regcomp() says: "3. If subexpression i is contained within another subexpression j, and i is not contained within any other subexpression that is contained within j, and a match of subexpression j is reported in pmatch[j], then the match or non-match of subexpression i reported in pmatch[i] shall be as described in 1. and 2. above, but within the substring reported in pmatch[j] rather than the whole string. The offsets in pmatch[i] are still relative to the start of string." Since the second subpattern in the example does not match anything within the last string matched by the first subpattern, this implies that regcomp() would report the second subpattern as a non-match. Although there is no requirement for sed to be implemented using regcomp(), presumably this is an unintentional difference between the descriptions of regcomp() and BREs. The text quoted above on BREs was an addition in POSIX.1-2001 intended to clarify what happens for repeated subpatterns. Possibly the added text is too simplistic and should have treated nested subpatterns in the way that regcomp() does. On the other hand, since several current sed implementations do not behave as the regcomp() description implies, perhaps it is the regcomp() description that needs alteration. (So far I am only aware of one, historical, sed implementation which outputs for the above example, however I have not tested many.) Action: Issue an official interpretation of the current requirement. Depending on the outcome of the interpretation, implement a clarification or correction in the next revision. _____________________________________________________________________________ COMMENT Enhancement Request Number 8 konrad.schwarz:siemens.com Defect in XBD aio_suspend (rdvk# 1) {0} Thu, 25 Mar 2004 11:34:32 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: This proposed change is not necessary. Since the arguments are const, they can not be affected by aliases. _____________________________________________________________________________ Page: 0 Line: 0 Section: aio_suspend Problem: Edition of Specification (Year): 2003 Defect code : 2. Omission Comparing the prototypes of aio_suspend() and lio_listio(), aio_suspend() seems to be missing a number of ``restrict'' qualifiers. Action: Change the prototype of aio_suspend() to int aio_suspend(const struct aiocb *restrict const list[restrict], int nent, const struct timespec *restrict timeout); ------------------------------------------------------------------------------ OBJECTION Enhancement Request Number 9 gwc:opengroup.org Defect in XBD unistd.h (rdvk# 1) [gwc option constants 1] Mon, 26 Apr 2004 15:19:35 +0100 ------------------------------------------------------------------------------ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 9(->interps) AI-026 See AI-026 This ERN was re-opened after some discussion on the mailing list since the last meeting. The intent of the current wording regarding options was that implementations should not need to change anything to show that they do not support a new feature [added to this standard (note not possible) or a companion standard that builds upon this standard] -- this was a reaction against the rolling amendments in previous times. It was envisaged that applications could using a runtime call (at least on UNIX systems) to getconf in a Makefile build optional code, and/or possibly use dlopen() at runtime to link in objects built using the option. There was agreement that this is a perhaps a bit convoluted and that we should forward concerns to the sponsor about the current wording for option constants. This should be forwarded down the interpretations track, with the recommendation that the standard is unclear in this area, and that a future direction should define the case when symbolic constants are undefined to be the same as when the symbolic constants set to -1. ------------------------------------------------------------------------------ Page: 26-27 Line: 1062-1070 Section: 2.1.6 Problem: Defect code : 1. Error Section 2.1.6 "Options" has not been properly updated to account for the new meaning of option constants. It only talks about options being supported or not supported; it doesn’t distinguish between support indications made at compile time and at runtime. Action: Change: "The symbolic constants defined in , Constants for Options and Option Groups reflect implementation options for IEEE Std 1003.1-2001. These symbols can be used by the application to determine which optional facilities are present on the implementation. The sysconf() function defined in the System Interfaces volume of IEEE Std 1003.1-2001 or the getconf utility defined in the Shell and Utilities volume of IEEE Std 1003.1-2001 can be used to retrieve the value of each symbol on each specific implementation to determine whether the option is supported. Where an option is not supported, the associated utilities, functions, or facilities need not be present." to: "The symbolic constants defined in , Constants for Options and Option Groups reflect implementation options for IEEE Std 1003.1-2001. These symbols can be used by the application to determine which of three categories of support for optional facilities are provided by the implementation: 1. Option never supported. The implementation advertises at compile time that the option will never be supported. In this case, the headers, data types, function interfaces and utilities required only for the option need not be present. 2. Option always supported. The implementation advertises at compile time that the option will always be supported. In this case, all headers, data types, function interfaces and utilities required only for the option shall be available and shall operate as specified. 3. Option might or might not be supported. The implementation does not advertise at compile time whether or not the option is supported. In this case, the fpathconf(), pathconf(), or sysconf() functions defined in the System Interfaces volume of IEEE Std 1003.1-2001 or the getconf utility defined in the Shell and Utilities volume of IEEE Std 1003.1-2001 can be used to retrieve the value of each symbol on each specific implementation to determine whether the option is supported. All headers, data types, and function interfaces required to compile and execute applications which use the option at runtime (after checking at runtime that the option is supported) shall be provided, but if the option is not supported they need not operate as specified. Utilities or other facilities required only for the option, but not needed to compile and execute such applications, need not be present." ------------------------------------------------------------------------------ EDITORIAL Enhancement Request Number 10 tjr:freebsd.org Defect in XBD Character Set Description File (rdvk# 3) {tjr2} Thu, 29 Apr 2004 03:11:35 +0100 (BST) ------------------------------------------------------------------------------ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 In XBD Section 6.4 Change from: shall be of the same type, and interpreted in byte order from first to last with the least significant byte of the multi-byte character specified by the last constant. To: shall be of the same type, and interpreted in sequence from first to last with the first byte of the multibyte character specified by the first byte in the sequence. ------------------------------------------------------------------------------ Page: 121 Line: 3852 Section: Character Problem: Edition of Specification (Year): 2003 Defect code : 1. Error The standard states (XBD 3851-3854): "When constants are concatenated for multi-byte character values, they shall be of the same type, and interpreted in byte order from first to last with the least significant byte of the multi-byte character specified by the last constant. The manner in which these constants are represented in the character stored in the system is implementation-defined." Multi-byte characters do not have a least significant byte or a byte order. Since "character" and "multi-byte character" are equivalent (XBD 1608-1610), the manner in which multi-byte characters are represented in characters is perfectly well defined: they are identical. This error also occurs at XBD 4055-4056, XBD 4061-4063, XBD 4068-4070 (and in ISO/IEC TR 14652:2002 -- Information technology -- Specification method for cultural conventions.) Action: The meaning of the statement in question is not clear to me and should be reworded paying careful attention to the difference between characters, wide characters, and multi-byte characters. It should either say: When constants are concatenated, they shall be of the same type, and shall be interpreted from first to last as the bytes making up a multi-byte character. (Second sentence deleted.) -or- When constants are concatenated, they shall be of the same type, and shall be interpreted in byte order from first to last as the octets of a wide character, with the least significant octet of the wide character specified by the last constant. The manner in which these constants are represented in the wide character stored in the system is implementation-defined. ------------------------------------------------------------------------------ COMMENT Enhancement Request Number 11 Michael T Kerrisk Bug in TC2-XBD (key_t) (rdvk# 4) Thu, 6 May 2004 15:26:30 +0200 (MEST) ------------------------------------------------------------------------------ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 11(->interps) AI-033 See AI-033 The group agreed with the proposal that key_t be changed in to just be an arithmetic type to be consistent with XSH section 2.12 Data Types, and noted that any implementation that has key_t as a pointer would be broken by this change. It was agreed to put this down the interpretations track. The standard is inconsistent and no conformance distinction can be made for the current standard. The interpretation should include the recommendation that in a future revision we include the change as in the ERN. The proposed change would correct an inconsistency which has been around since at least XPG4. ------------------------------------------------------------------------------ Page: 372 Line: 13070 Section: unk Problem: In XBD we find the following the statment that key_t need not be an arithmetic type: 13068 All of the types shall be defined as arithmetic types of an appropriate length, with the following 13069 exceptions: 13070 XSI key_t But in the description of "Data Types" in XSH (Section 2.12), we find the following contradictory statement: 3363 key_t Arithmetic type used for XSI interprocess communication. Geoff Clare privatetly noted to me: > Since ftok() returns (key_t)-1 on error, I think that if it is not an > arithmetic type then the only other thing it could be is a pointer. Action: Remove line 13070 "XSI key_t" from TC2-XBD ------------------------------------------------------------------------------ COMMENT Enhancement Request Number 12 gwc:opengroup.org Defect in XBD unistd.h (rdvk# 2) [gwc option constants 2] Mon, 26 Apr 2004 15:19:35 +0100 ------------------------------------------------------------------------------ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 12(->interps) AI-026 See AI-026 This should be treated consistently with XBD ERN 9 and go down the interpretations track as part of the same interpretation. Arising recommendations for a future revision are as follows: In section: Constants for Options and Option Groups Delete in paragraph 1: "If these are undefined, the fpathconf(), pathconf(), or sysconf() functions can be used to determine whether the option is provided for a particular invocation of the application." Change in paragraph 2 from: If a symbolic constant is defined with the value -1, the option is not supported. To: If a symbolic constant is not defined or is defined with the value -1, the option is not supported. Change in paragraph 4 from: The application can check at runtime to see whether the option is supported by calling fpathconf (),pathconf (),orsysconf( ) with the indicated name parameter. To: The application can check at runtime to see whether the option is supported for a particular invocation of the application by calling fpathconf (),pathconf (),orsysconf( ) with the indicated name parameter. ------------------------------------------------------------------------------ Page: 398 Line: 14023-14034 Section: unistd.h Problem: Defect code : 3. Clarification required The descriptions on the page under the heading "Constants for Options and Option Groups" of the "undefined" case and the "defined with value zero" case are separate and slightly different. This can mislead some readers into thinking that there must be some subtle difference between the two cases, when in fact the intention was for the requirements to be exactly the same. (This intention is clear from the explanation of the new options mechanism in the rationale on the page.) I think the text ended up the way it is purely by historical accident. The text for the "undefined" case was carried over from SUSv2 with some minor adjustment, and the text describing the -1, >0 and 0 cases was added after it. The descriptions of the "undefined" case and the "defined with value zero" case need to be merged in order to remove any doubt about their equivalence. Action: On lines 14023-14025 delete: "If these are undefined, the fpathconf(), pathconf(), or sysconf() functions can be used to determine whether the option is provided for a particular invocation of the application." On line 14033 change "is defined with the value zero" to "is undefined or is defined with the value zero". On line 14034 change "whether the option is supported" to "whether the option is supported for a particular invocation of the application". _____________________________________________________________________________ OBJECTION Enhancement Request Number 13 ajosey:opengroup.org Defect in XBD sched.h (rdvk# 1) {optiion-constants-defined} Fri, 14 May 2004 11:19:32 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 It was agreed to go with the option 1 in the proposal which is to remove the lead in so that the following sentence would start "The sched_param structure...." : "In addition, if _POSIX_SPORADIC_SERVER or _POSIX_THREAD_SPORADIC_SERVER is defined, the sched_param structure defined in shall contain the following members in addition to those specified above:" ______________________________________________________________________________ ______________________________________________________________________________ Page: 298 Line: 10597 Section: sched.h Problem: Edition of Specification (Year): 2004 Defect code : 1. Error sched.h has the following text in the DESCRIPTION: "In addition, if _POSIX_SPORADIC_SERVER or _POSIX_THREAD_SPORADIC_SERVER is defined, the sched_param structure defined in shall contain the following members in addition to those specified above:" Since symbolic constants for options can be defined to -1, (or not defined and sysconf() report support for the option) this language is incorrect . Action: Either remove the lead in so the section says "The sched_param structure...." (the option is thus noted by the shading and margin marker notation) or restate in terms of the option names "In addition, if the Process Sporadic Server or Thread Sporadic Server options are supported, the sched_param structure..." ___________________________________________________________________________ OBJECTION Enhancement Request Number 14 gwc:opengroup.org Defect in XBD netdb.h (rdvk# 1) [gwc netdb s_port] Thu, 3 Jun 2004 21:20:55 +0100 ______________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 ______________________________________________________________________________ ______________________________________________________________________________ Page: 278 Line: 9871 Section: netdb.h Defect code : 1. Error Problem: The description of the s_port member of struct servent is: int s_port The port number at which the service resides, in network byte order. Since int is typically 32-bit, this implies (wrongly) that s_port is a network-byte-order 32-bit integer and thus should be converted to host byte order using ntohl(). Of course, it is actually converted with ntohs() (or assigned directly to the sin_port member of a struct sockaddr_in, which is a network-byte-order 16-bit integer). There is a similar problem with the description of the port argument of getservbyport() in XSH6. Action: Change: The port number at which the service resides, in network byte order. To: A value which, when converted to uint16_t, yields the port number in network byte order at which the service resides. Similarly in XSH6 at page 287 line 9366 section endservent objection Change: The port argument shall be in network byte order. To: The port argument shall be a value obtained by converting a uint16_t in network byte order to int. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 15 tauvan:starpower.net Defect in XBD System Interfaces (rdvk# 1) {none} Tue, 10 Aug 2004 16:14:25 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 _____________________________________________________________________________ Page: 0 Line: 0 Section: System Problem: Edition of Specification (Year): 2004 Defect code : 1. Error In the prototype for strerror_r() is: int *strerror_r(int, char *, size_t); For the definition of strerror_r() you have: int strerror_r(int errnum, char *strerrbuf, size_t buflen); with the definition stating a return of int. I believe your prototype in is in error. Action: In the prototype for strerror_r() should read: int strerror_r(int, char *, size_t); _____________________________________________________________________________ COMMENT Enhancement Request Number 16 ajosey:opengroup.org Defect in XBD 12.2 (rdvk# 1) {ag-ob-synopses} Tue, 24 Aug 2004 10:12:42 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 16(->interps) AI-027 See AI-027 It was agreed that an interpretation be made , that the standard is clear and no change is required. The standard permits implementations to have extensions that violate the Utility Syntax Guidelines so long as when the utility is used in line with the forms defined by the standard that it follows the Utility Syntax Guidelines. Thus head -42 file and ls --help are permitted as extensions. _____________________________________________________________________________ Page: 204 Line: 7237-7240 Section: 12.2 Problem: Edition of Specification (Year): 2004 Defect code : 3. Clarification required This is a request for interpretation of the standard. Is an implementation of a standard utility permitted to support extensions beyond the standard in violation of the Utility Syntax Guidelines when the description of the utility says it shall support them? An example would be an implementation of the head utility supporting the previously obsolescent synopsis , head -42 file rather than head -n 42 file Action: Confirmation is sought that an implementation is permitted to have extensions so long as the standard form is also accepted and that the extension is for backwards-compatibility. The proposed rationale is as follows: The last paragraph of 1003.2-1992 2.10.2 and XBD 1003.1-2001 2.12 allows utilities to accept as extensions other forms that do not match the guidelines so long as the standard form is also accepted. Looking at head I would interpret the situation as follow: 1. head -n 42 filename An implementation of the head utility that only accepts this form would be a strictly conforming implementation. This form must be used by conforming applications. 2. head -42 filename An implementation of the head utility that accepts this form ( and the form in 1 above) is a conforming implementation with extensions. The extension violates the guidelines as allowed by 2.12. but is clearly for backwards compatibility since its a form that was in previous version of the standard. The intent of removing the obsolescent forms of the synopses was not to disallow them to be supported by implementations but to downgrade their status of their use in applications from conforming application using an obsolescent feature to non-conforming application. This form must not be used by conforming application. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 17 bug1:iinet.net.au Defect in XBD 2.1.6 Options (rdvk# 17) {1} Sat, 28 Aug 2004 05:31:20 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 _____________________________________________________________________________ Page: 0 Line: 0 Section: 2.1.6 Problem: Edition of Specification (Year): 2004 Defect code : 1. Error In the conformance section of the base definition, 'section 2.1.6; shell and utilities; POSIX2_UPE' there is a list of 'User Portability Utilities'. The list seems to be alphapetical in columns, the first column is alias at batch bg crontab split ctags df du ex the third column is nm patch ps renice split strings tabs talk time tput Notice that 'split' is mentioned in both columns, and in the first column it isnt in alphebetical order. The command 'csplit' isnt mentioned in any columns at all, even though its description says it should be in there. I believe 'split' in the first column is actually an editorial error, and it should be 'split' Action: Replace 'split' with 'csplit' in the first column of the conformance section of the base definition, section 2.1.6; shell and utilities; POSIX2_UPE [Proposed resolution: Accept, this should be an errata item] _____________________________________________________________________________ EDITORIAL Enhancement Request Number 18 simon:comsys.ntu-kpi.kiev.ua Defect in XBD 0 (rdvk# 18) {0} Tue, 7 Sep 2004 10:42:52 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject____ Rationale for rejected or partial changes: (this is only in the html indexing) _____________________________________________________________________________ Page: 0 Line: 0 Section: 0 Problem: Edition of Specification (Year): 2004 Defect code : 1. Error Comment for write() function is "write to another user" in Alphabetical->W, it should be "write on a file". Action: Fix it. _____________________________________________________________________________ COMMENT Enhancement Request Number 19 eggert:cs.ucla.edu Defect in XBD 8.1 Environment Variable Defn (rdvk# 19) {20040918a} Sun, 19 Sep 2004 00:14:27 +0100 (BST) ______________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 19(->interps) AI-047 This is an interpretation. The standard is clear.... concerns are being fwd'd to the sponsor Notes to the editor not part of this interpretation: (Line numbers based on latest PDF, the one that includes TC2) ACTION for XBD ERN 19: (1) In XBD Section Section 13 page 410 (unistd.h) add the following after line 14460: _CS_V6_ENV This is the value that provides the environment variable information (other than that provided by _CS_PATH) that is required by the implementation to create a conforming environment, as described in the implementation's conformance documentation. {Proposal Note: "V6_ENV" obviously refers to the Issue 6 document, but implementations are free (via namespace rules) to also create "_CS_UNIX95_ENV", "_CS_POSIX1990", or "_CS_FOO" to cover the other environments that they may simultaneously support.} (2) In XSH Chapter 3, page 219 (confstr()), insert the following before line 7316: _CS_V6_ENV (3) In XSH Chapter 3, page 220 (confstr()) insert the following before line 7355: After a call to: confstr(_CS_V6_ENV, buf, sizeof(buf)) the string stored in buf will contain the space-separated list of variable=value environment variable pairs required by the implementation to create a conforming environment, as described in the implementation's conformance documentation. (4) In XSH Chapter 3, page 307 (exec() family), insert the following text after line 9991: The new process might be invoked in a nonconforming environment if the "envp" array does not contain implementation-defined variables required by the implementation to provide a conforming environment. See the _CS_V6_ENV entry in unistd.h, and confstr(), for details. (5) In XCU Chapter 4, page 354 (env), insert the following at the end of line 13663: It is possible to create a non-conforming environment using the -i option, as it may remove environment variables required by the implementation for conformance. The following will preserve these environment variables as well as preserve the PATH for conforming utilities: env -i $(getconf V6_ENV) $(getconf PATH) command ______________________________________________________________________________ Page: 161 Line: 5584 Section: 8.1 Problem: Edition of Specification (Year): 2004 Defect code : 3. Clarification required Can conforming implementations require all-caps environment variables to be set to particular values, as part of the configuration instructions? The standard says "Environment variable names used by the utilities ... consist solely of uppercase letters, digits, and the '_' (underscore) ... and do not begin with a digit." Let's call these names "all-caps" for short. The question is whether an implementation can insist on a particular all-caps variable (or variables) being set to particular values, in order to establish a conforming environment. Clearly variables whose names contain lower-case letters are reserved for applications and cannot affect the behavior of utilities, and it's also clear from confstr() that an implementation can require PATH to be set to the output of the command "getconf PATH" in order to get conforming behavior, but it's not clear from the standard whether an implementation can require any other all-caps variable to be set. On one side of the issue, requiring an all-caps variable to be set is common implementation practice. AIX requires XPG_SUS_ENV='ON'. HP-UX requires UNIX95 to be set. IRIX requires _XPG to be set to a numeric value greater than 0. UnixWare requires POSIX2='on'. GNU utilities require POSIXLY_CORRECT to be set. In general these environment settings can affect the behavior of utilities, functions, and other facilities. On the other side, we have this opinion, quoted from Don Cragun in : If an application defines an environment variable not specified by the standard or adds a nonstandard option to a command line, an implementation is free to do whatever it wants. But, an implementation that requires an application to define a nonstandard environment variable to get standard behavior is not a conforming implementation. He further gave this example. If "env" and "getconf" conform to the standard, then env -i PATH=$(getconf PATH) command must invoke "command" in a POSIX-conforming environment in which only the PATH variable is set; if an implementation requires (say) POSIXLY_CORRECT to be set as well for conformance, it will not execute the command correctly in general. This question came up recently in the austin-group-l while discussing draft AI-027, but it is a different issue and deserves a separate Aardvark. My own feeling is that the standard is unclear, and that widespread existing practice indicates that conforming implementations can require certain all-caps environment variables to be set. Action: In XBD section 8.1 page 161 line 5584, change: ... and do not begin with a digit. to: ... and do not begin with a digit; setting or unsetting any variable in this name space results in unspecified behavior unless the variable is defined in this standard. Append the following after XCU section "env" page 354 line 13663: The utility might be invoked in a nonconforming environment if "env" is directed to set (or with -i, unset) a variable whose name falls in the space reserved to the implementation (see XBD section 8.1 page 161). _____________________________________________________________________________ OBJECTION Enhancement Request Number 20 gwc:opengroup.org Defect in XBD 4.6 Filenames (rdvk# 20) [gwc hyphen in filenames] Sun, 26 Sep 2004 20:09:22 +0100 _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: Withdrawn by the originator _____________________________________________________________________________ Page: 98 Line: 3071 Section: 4.6 Problem: Defect code : 1. Error I have seen Paul Eggert's defect report concerning "pathchk -p" (austin-review-l 1842) and I agree that the description of the pathchk -p option is inconsistent with XBD 4.6 Filenames. However, I propose that this inconsistency should be resolved by changing XBD 4.6, not by changing the pathchk page in XCU. The restriction that portable filenames should not begin with a hyphen has always been something of an oddity. I believe that the only reason it appeared in POSIX originally was because of the difficulties in using such filenames as utility arguments. This is not really a problem with filenames, but with utility syntax, and should be addressed not by restricting filenames, thus unnecessarily affecting their use in function calls, but by advising application writers how to correctly employ the "--" option delimiter whenever filenames beginning with a hyphen might accidentally appear to a utility to be an argument containing options instead of being an operand. Compare the filename case to that of regular expressions. Specifying that regular expressions beginning with a hyphen are not portable, because grep "$regex" infile would mistake "$regex" as containing options, would be absurd. With hindsight, specifying that filenames beginning with a hyphen are not portable was just as absurd back in 1988, as there is no reason for creat("-name", mode) not to be portable. The only issue with filenames beginning with a hyphen is one of utility syntax and it should be addressed as such, not as a filename portability issue. The fact that, when the description of pathchk -p was written, a bullet item for filenames beginning with hyphen was overlooked demonstrates that this is not really a filename portability issue. By omitting this case for the -p option the author(s) of the pathchk page, perhaps unconsciously, got filename portability right. Action: On line 3071 delete "The hyphen character shall not be used as the first character of a portable filename." Add the following to the end of XCU 2.6 Word Expansions: Whenever utility arguments are the result of word expansions, and the arguments are intended to be passed as operands, applications should ensure that the "--" option delimiter is used when necessary to prevent the utility from treating an argument beginning with a hyphen as containing options. Examples rm -f -- *.bak In this example, the "--" is needed because if a filename beginning with a hyphen and ending ".bak" exists in the current directory it could cause the expansion of "*.bak" to begin with a hyphen. grep -- "$regex" filetosearch Here the "--" is needed because the variable regex could contain a regular expression that begins with a hyphen. set -- $(cat onelinefile) field2="$2" In this example, the first non-IFS byte in onelinefile could be a hyphen, which would cause problems without the "--". _____________________________________________________________________________ OBJECTION Enhancement Request Number 21 eggert:cs.ucla.edu Defect in XBD Utility Syntax Guidelines (rdvk# 21) {20040928a} Tue, 28 Sep 2004 21:02:12 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 Change from: Guideline 10: The argument -- should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the '-' character. The -- argument should not be used as an option or as an operand. To: Guideline 10: The first -- argument that is not an option argument should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the '-' character. Add the following text to the rationale for Guideline 7 (after XRAT page 75 line 3062): Guideline 7 allows any string to be an option-argument: an option-argument can begin with any character, can be "-" or "--", and can be an empty string. For example, the commands "pr -h -", "pr -h --", "pr -h -d", "pr -h +2", and "pr -h ''" contain the option-arguments "-", "--", "-d", "+2" and "", respectively. Conversely, the command "pr -h -- -d" treats "-d" as an option, not as an argument, because the "--" is an option-argument here, not a delimiter. _____________________________________________________________________________ Page: 204 Line: 7223 Section: Utility Problem: Edition of Specification (Year): 2004 Defect code : 1. Error The utility syntax guidelines prohibit the use of "--" as an operand in all cases, even in examples like "printf -- --" where it was clearly intended that "--" be allowed as an operand. Also, it's not entirely clear from the standard or rationale that an "--" argument is not a delimiter if it occurs as an option-argument. Action: Change the following text in Guideline 10 (XBD page 204 lines 7223-7224) from: The -- argument should not be used as an option or as an operand. to: The -- argument should not be used as an option, or as an operand unless an earlier argument is the -- delimiter. Add the following text to the rationale for Guideline 7 (after XRAT page 75 line 3062): Guideline 7 allows any string to be an option-argument: an option-argument can begin with any character, can be "-" or "--", and can be an empty string. For example, the commands "pr -h -", "pr -h --", "pr -h -d", "pr -h +2", and "pr -h ''" contain the option-arguments "-", "--", "-d", "+2" and "", respectively. Conversely, the command "pr -h -- -d" treats "-d" as an option, not as an argument, because the "--" is an option-argument here, not a delimiter. _____________________________________________________________________________ COMMENT Enhancement Request Number 22 eggert:cs.ucla.edu Defect in XBD Utility Syntax Guidelines (rdvk# 22) {20040928c} Wed, 29 Sep 2004 00:47:40 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 Add the following text to the Utility Syntax Guidelines (after XBD page 204 line 7236): Guideline 14: If an argument can be identified according to guidelines 3 through 10 as an option, or as a group of options without option-arguments behind one '-' delimiter, then it should be treated as such. Add the following text to the Utility Syntax Guidelines rationale (after XRAT page 76 line 3075): Guideline 14 is intended to prohibit implementations that would treat the command "ls -l -d" as if it were "ls -- -l -d" or "ls -l -- -d". _____________________________________________________________________________ Page: 204 Line: 7326 Section: Utility Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission Nowhere do the Utility Syntax Guidelines explicitly say that the first operand cannot begin with "-" unless it is the single character "-". Obviously it was intended that option-parsing be "greedy", so "ls -l -d" is supposed to treat both "-l" and "-d" as options. But this intent isn't written down explicitly. In theory, this omission might lead to confusion, as an implementation could treat both arguments as operands, or treat "-l" as an option and "-d" as an operand, without violating the standard. The arguments "-l" and "-d" are allowed as operands in some cases (e.g., "ls -- -l -d"), and the standard doesn't make it clear why they must be treated as options in this particular case. One might object that there is no possible ambiguity since a naive implementation of "ls" using getopt() must treat both "-l" and "-d" as options. However, this objection is flawed, as will be discussed in a later Aardvark on getopt(). And even if the objection were sustained, the guidelines would still be made clearer by explicitly addressing the potential ambiguity. Action: Add the following text to the Utility Syntax Guidelines (after XBD page 204 line 7236): Guideline 14: If an argument can be identified according to guidelines 3 through 10 as an option, or as a group of options without option-arguments behind one '-' delimiter, then it should be treated as such. Add the following text to the Utility Syntax Guidelines rationale (after XRAT page 76 line 3075): Guideline 14 is intended to prohibit perverse implementations that would treat the command "ls -l -d" as if it were "ls -- -l -d" or "ls -l -- -d". _____________________________________________________________________________ OBJECTION Enhancement Request Number 23 karen:apple.com Defect in XBD signal.h (rdvk# 23) {3714764} Thu, 21 Oct 2004 23:34:19 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 23(->interps) AI-034 See AI-034 Interpretation: ----------------- The standards states the requirements for SIGPOLL to be supported as part of the XSI option, and conforming implementations must conform to this. However, concerns have been raised about this which are being referred to the sponsor. Rationale: ----------- The semantics of SIGPOLL are only specified with functionality in the XSR option. Notes to the Editor for a future revision (not part of this interpretation): --------------------------------------------------------------------------- In XBD, signal.h In the DESCRIPTION Change from: "[XSI] SIGPOLL T Pollable event." To: "[XSR] SIGPOLL T Pollable event." and start XSI shading on the line below. In the definition of siginfo_t XSR mark and shade the following: "long si_band Band event for SIGPOLL. " The following should be XSR marked and shaded in the table headed Signals/Codes: " SIGPOLL POLL_IN Data input available. POLL_OUT Output buffers available. POLL_MSG Input message available. POLL_ERR I/O error. POLL_PRI High priority input available. POLL_HUP Device disconnected. " And in the table headed Signal/Member/Value XSR mark and shade: "SIGPOLL long si_band Band event for POLL_IN, POLL_OUT, or POLL_MSG." _____________________________________________________________________________ Page: 0 Line: 0 Section: signal.h Problem: Edition of Specification (Year): 2003 Defect code : 1. Error STREAMS support is optional and interfaces directly related to streams are shaded [XSR] throughout the standard. For example, stropts.h, isastream, getmsg, getpmsg, ioctl, putmsg, putpmsg, fattach, fdetach are all shaded [XSR]. SIGPOLL is defined in signal.h Aside from signal.h, SIGPOLL is only ever discussed in XSR related text. Since the SIGPOLL signal is only used by the above STREAMS interfaces, we think it should be shaded [XSR] in signal.h. Currently SIGPOLL is [XSI]. Action: SIGPOLL references in signal.h should be shaded [XSR]. _____________________________________________________________________________ COMMENT Enhancement Request Number 24 mstrbill:us.ibm.com Defect in XBD limits.h (rdvk# 24) {WLT-XBD-5} Fri, 22 Oct 2004 14:17:00 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: No change required This is presently covered in B.2.8 Realtime, Advisory Information Advisory Information POSIX.1b contains an Informative Annex with proposed interfaces for "realtime files". These interfaces could determine groups of the exact parameters required to do "direct I/O" or "extents". These interfaces were objected to by a significant portion of the balloting group as too complex. A conforming application had little chance of correctly navigating the large parameter space to match its desires to the system. In addition, they only applied to a new type of file (realtime files) and they told the implementation exactly what to do as opposed to advising the implementation on application behavior and letting it optimize for the system the (portable) application was running on. For example, it was not clear how a system that had a disk array should set its parameters. There seemed to be several overall goals: * Optimizing sequential access * Optimizing caching behavior * Optimizing I/O data transfer * Preallocation The advisory interfaces, posix_fadvise() and posix_madvise(), satisfy the first two goals. The POSIX_FADV_SEQUENTIAL and POSIX_MADV_SEQUENTIAL advice tells the implementation to expect serial access. Typically the system will prefetch the next several serial accesses in order to overlap I/O. It may also free previously accessed serial data if memory is tight. If the application is not doing serial access it can use POSIX_FADV_WILLNEED and POSIX_MADV_WILLNEED to accomplish I/O overlap, as required. When the application advises POSIX_FADV_RANDOM or POSIX_MADV_RANDOM behavior, the implementation usually tries to fetch a minimum amount of data with each request and it does not expect much locality. POSIX_FADV_DONTNEED and POSIX_MADV_DONTNEED allow the system to free up caching resources as the data will not be required in the near future. POSIX_FADV_NOREUSE tells the system that caching the specified data is not optimal. For file I/O, the transfer should go directly to the user buffer instead of being cached internally by the implementation. To portably perform direct disk I/O on all systems, the application must perform its I/O transfers according to the following rules: 1. The user buffer should be aligned according to the {POSIX_REC_XFER_ALIGN} pathconf() variable. 2. The number of bytes transferred in an I/O operation should be a multiple of the {POSIX_ALLOC_SIZE_MIN} pathconf() variable. 3. The offset into the file at the start of an I/O operation should be a multiple of the {POSIX_ALLOC_SIZE_MIN} pathconf() variable. 4. The application should ensure that all threads which open a given file specify POSIX_FADV_NOREUSE to be sure that there is no unexpected interaction between threads using buffered I/O and threads using direct I/O to the same file. In some cases, a user buffer must be properly aligned in order to be transferred directly to/from the device. The {POSIX_REC_XFER_ALIGN} pathconf() variable tells the application the proper alignment. The preallocation goal is met by the space control function, posix_fallocate(). The application can use posix_fallocate() to guarantee no [ENOSPC] errors and to improve performance by prepaying any overhead required for block allocation. Implementations may use information conveyed by a previous posix_fadvise() call to influence the manner in which allocation is performed. For example, if an application did the following calls: fd = open("file"); posix_fadvise(fd, offset, len, POSIX_FADV_SEQUENTIAL); posix_fallocate(fd, len, size); an implementation might allocate the file contiguously on disk. Finally, the pathconf() variables {POSIX_REC_MIN_XFER_SIZE}, {POSIX_REC_MAX_XFER_SIZE}, and {POSIX_REC_INCR_XFER_SIZE} tell the application a range of transfer sizes that are recommended for best I/O performance. Where bounded response time is required, the vendor can supply the appropriate settings of the advisories to achieve a guaranteed performance level. The interfaces meet the goals while allowing applications using regular files to take advantage of performance optimizations. The interfaces tell the implementation expected application behavior which the implementation can use to optimize performance on a particular system with a particular dynamic load. The posix_memalign() function was added to allow for the allocation of specifically aligned buffers; for example, for {POSIX_REC_XFER_ALIGN}. The working group also considered the alternative of adding a function which would return an aligned pointer to memory within a user-supplied buffer. This was not considered to be the best method, because it potentially wastes large amounts of memory when buffers need to be aligned on large alignment boundaries. _____________________________________________________________________________ Page: 253 Line: 8871 Section: limits.h Problem: Edition of Specification (Year): 2004 Defect code : 1. Error The following are defined for the POSIX Advisory Information option. ADV {POSIX_ALLOC_SIZE_MIN} 8871 Minimum number of bytes of storage actually allocated for any portion of a file. 8872 Minimum Acceptable Value: Not specified. 8873 ADV {POSIX_REC_INCR_XFER_SIZE} 8874 Recommended increment for file transfer sizes between the 8875 {POSIX_REC_MIN_XFER_SIZE} and {POSIX_REC_MAX_XFER_SIZE} values. 8876 Minimum Acceptable Value: Not specified. 8877 ADV {POSIX_REC_MAX_XFER_SIZE} 8878 Maximum recommended file transfer size. 8879 Minimum Acceptable Value: Not specified. 8880 ADV {POSIX_REC_MIN_XFER_SIZE} 8881 Minimum recommended file transfer size. 8882 Minimum Acceptable Value: Not specified. 8883 ADV {POSIX_REC_XFER_ALIGN} 8884 Recommended file transfer buffer alignment. 8885 Minimum Acceptable Value: Not specified. This information doesn't seem to be sufficient to determine exactly what values an implementation should supply. None of the Advisory system calls descriptions describe using these variables, so it is not clear what effect or what use they might have in an application. Action: Add more information. And possibly, add examples on how to use these in an application. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 25 terekhov:de.ibm.com de.ibm.com (rdvk# 25 ) {alt-msync-2004-11-02} Tue, 2 Nov 2004 12:34:02 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 on XBD page 100 add at line 3132 The pthread_mutex_lock() function need not synchronize memory if the mutex type is PTHREAD_MUTEX_RECURSIVE and the calling thread already owns the mutex. The pthread_mutex_unlock() function need not synchronize memory if the mutex type is PTHREAD_MUTEX_RECURSIVE and the mutex has a lock count greater than one. Add to the SEE ALSO section, "XBD 4.10 Memory Synchronization" for each of the following functions listed in the table in XBD 4.10: fork () pthread_barrier_wait() pthread_cond_broadcast() pthread_cond_signal () pthread_cond_timedwait() pthread_cond_wait() pthread_create() pthread_join () pthread_mutex_lock() pthread_mutex_timedlock() pthread_mutex_trylock() pthread_mutex_unlock() pthread_spin_lock() pthread_spin_trylock() pthread_spin_unlock() pthread_rwlock_rdlock() pthread_rwlock_timedrdlock() pthread_rwlock_timedwrlock() pthread_rwlock_tryrdlock() pthread_rwlock_trywrlock() pthread_rwlock_unlock() pthread_rwlock_wrlock() sem_post() sem_trywait() sem_wait() wait() waitpid ()) _____________________________________________________________________________ Page: 0 Line: 0 Section: 4.10 Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission Application programmers shall make no mistake with respect to "memory synchronization" effect (I mean that they shall not expect and rely on it) of recursive operations on PTHREAD_MUTEX_RECURSIVE mutexes and pthread_rwlock_t locks. Implementors shall be allowed/encouraged to take advantage of it. Action: Please add after "The pthread_once() function shall synchronize memory for the first call in each thread for a given pthread_once_t object." something along the lines of: If the mutex type of the specified mutex is PTHREAD_MUTEX_RECURSIVE, pthread_mutex_lock() function may have no memory synchronization effect when the calling thread merely relocks the specified mutex." and If the mutex type of the specified mutex is PTHREAD_MUTEX_RECURSIVE, pthread_mutex_unlock() function may have no memory synchronization effect when the calling thread has more than one lock on the specified mutex (before calling this function). Please also add similar wordings for recursive operations on pthread_rwlock_t locks. TIA. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 26 terekhov:de.ibm.com Defect in XBD 4.10 Memory Synchronization (rdvk# 26) {cv-msync-28.10.2004} Thu, 28 Oct 2004 17:12:41 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: R5: Reject; The review team disagrees with the problem statement and action . 4.10 states "Such access is restricted using functions that synchronize thread execution and also synchronize memory with respect to other threads." The application is responsible for ensuring that two threads cannot access the same "memory location" at the same time. There are many ways to accomplish this without explicit "thread execution" synchronization (e.g., mutexes, semaphores, etc.) It is sufficient to ensure that only one thread can call a function that modifies some memory location. What it doesn't ensure is that any other thread will be able to read the modified value, because the specification does not (by intent) require any form of implicit memory synchronization. The specification, however, makes it clear that calling any of the listed functions provides memory synchronization. Therefore, the standard currently provides for pthread_cond_signal and pthread_cond_broadcast to be used essentially as memory barriers. The group is actually not particularly happy with the implications of this; but that's what the standard says all the same. The issue here isn't whether the aardvark is "theoretically correct", but whether the change proposed is justifiable ("safe and of sufficient value") at this point in the life of the standard. Previous attempts at this revision have been rejected because the working group felt the risk of breaking existing applications depending on this inadvertent "memory barrier" behavior was too great to justify the benefit of slightly simpler or more efficient implementation code. The group does not see how that has changed. _____________________________________________________________________________ Page: 0 Line: 0 Section: 4.10 Problem: Edition of Specification (Year): 2004 Defect code : 1. Error XBD 4.10 lists pthread_cond_signal() and pthread_cond_broadcast() among functions that "synchronize memory". This is a thinko, I believe. cond_wait: mutex::release_guard guard(mutex); sleep(random()); cond_signal: nop cond_broadcast: nop Is clearly conforming (apart from realtime scheduling, of course). And nops don't synchronize anything. Action: Remove pthread_cond_signal() and pthread_cond_broadcast() from the XBD 4.10. The idea is to allow compilers to take advantage of "#pragma isolated_call"-like semantics (see http://tinyurl.com/68jav) for pthread_cond_signal() and pthread_cond_broadcast(). _____________________________________________________________________________ EDITORIAL Enhancement Request Number 27 terekhov:de.ibm.com de.ibm.com (rdvk# 27) {alt-barrier-2004-11-03} Wed, 3 Nov 2004 20:21:29 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: R6: Reject: The review team believes that accepting the proposed change would decrease consensus. The standard developers did not agree that adding the suggested wording was justified. (the submitter also indicated after the working group had expended review time that this item should have been withdrawn) _____________________________________________________________________________ Page: 0 Line: 0 Section: 4.10 Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission Not really a big problem, but given two other XBD 4.10 DRs (and common sense ;-) )... Action: Please add after "The pthread_once() function shall synchronize memory for the first call in each thread for a given pthread_once_t object." something along the lines of: If the specified pthread_barrie_t object is initialized with count (number of threads that must call pthread_barrier_wait() before any of them successfully return from the call) equal 1, pthread_barrier_wait() may have no memory syncronization effect. TIA. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 28 terekhov:de.ibm.com de.ibm.com (rdvk# 28) {alt-msync-sem_timedwait-2004-11-04} Thu, 4 Nov 2004 13:36:47 GMT _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 _____________________________________________________________________________ Page: 0 Line: 0 Section: 4.10 Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission The list of functions that "synchronize memory with respect to other threads" contains sem_trywait(), sem_wait() and sem_post() but doesn't contain sem_timedwait(). An oversight, I think. Action: Add sem_timedwait() to the list of functions that "synchronize memory with respect to other threads". _____________________________________________________________________________ COMMENT Enhancement Request Number 29 terekhov:de.ibm.com de.ibm.com (rdvk# 29) {alt-msync-sem_getvalue-2004-11-04} Thu, 4 Nov 2004 17:19:03 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: R6: Reject: The review team believes that accepting the proposed change would decrease consensus. There are no atomic operations, and therefore memory barriers, necessary to implement this function. The proposed change could therefore break currently conforming implementations. At least one existing implementation does not use atomic operations and thus does not have memory synchronization for sem_getvalue(). _____________________________________________________________________________ Page: 0 Line: 0 Section: 4.10 Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission It appears that semantics of sem_getvalue() allow the use of that function to synchronize threads execution in order to "ensure that access to any memory location by more than one thread of control (threads or processes) is restricted such that no thread of control can read or modify a memory location while another thread of control may be modifying it" (XBD 4.10). For example, a single published/producer may simply post a semaphore (using sem_post() on a sema initialized with zero semaphore value) to signal to multiple consumers (they would periodically call sem_getvalue() that the data published by the producer can now be accessed (read) concurrently. Here's an illustration using infamous DCSI (DCL) "pattern". http://www.aristeia.com/Papers/DDJ_Jul_Aug_2004_revised.pdf http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html DCSI-SEMAS: (double-checked serialized initialization with semas) Given: sema sema_flag(0) // sem_init(&m_sem, pshared, 0) sema sema_lock(1) // sem_init(&m_sem, pshared, 1) Solution: once_init() { // 1st check (sem_getvalue(&m_sem, &return_value)) if (!sema_flag.getvalue()) { // Lock (ctor: sem_wait(&m_sem), dtor: sem_post(&m_sem)) sema::lock_guard guard(sema_lock); // 2nd check (sem_getvalue(&m_sem, &return_value)) if (!sema_flag.getvalue()) { sema_flag.post(); // sem_post(&m_sem) } } } This solution is clearly conforming as far as XBD 4.10's first statement is concerned. sem_post() is already on the list of functions that "synchronize memory with respect to other threads". However, this solution relies on ("acquire") memory synchronization effect of sem_getvalue(), not only ("release") memory synchronization effect of sem_post(). Action: Add sem_getvalue() to the list of functions that "synchronize memory with respect to other threads". _____________________________________________________________________________ EDITORIAL Enhancement Request Number 30 terekhov:de.ibm.com de.ibm.com (rdvk# 30) {alt-msync-XSI-semas-2004-11-04} Thu, 4 Nov 2004 20:25:18 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 Add semop() and semctl() to the list of functions that "synchronize memory with respect to other threads" in the table in 4-10. _____________________________________________________________________________ Page: 0 Line: 0 Section: 4.10 Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission The Net is full of code implementing -like interfaces using XSI IPC semas. Such code relies on memory synchronization effects for semop() and semctl(), but the list of functions that "synchronize memory with respect to other threads" doesn't contain these functions. See alt-msync-sem_getvalue-2004-11-04 DR for illustration on sem_getvalue (GETVAL) msync "problem". Here (belatedly, sorry for that) is some code. Suppose that is implemented (to the extent of just making that illustration work) using XSI IPC semas. #include #include #include //*** ES aside for a moment... //*** POSIX semaphore wrapper (just an illustration) class sema { //*** unimplemented since it's non-copyable/non-copy-constructible sema(const sema &); sema & operator=(const sema &); sem_t m_sem; public: //*** RAII scoped locking guard class lock_guard { //*** unimplemented since it's non-copyable/non-copy-constructible lock_guard(const lock_guard &); lock_guard & operator=(const lock_guard &); sema & m_sem; public: lock_guard(sema & sem) : m_sem(sem) { m_sem.wait(); } ~lock_guard() { m_sem.post(); } }; //*** class sema::guard sema(unsigned value) { int status = sem_init(&m_sem, 0, value); assert(!status); } ~sema() { int status = sem_destroy(&m_sem); assert(!status); } void wait() { int status; do status = sem_wait(&m_sem); while (status == EINTR); assert(!status); } void post() { int status = sem_post(&m_sem); assert(!status); } int getvalue() { int return_value, status = sem_getvalue(&m_sem, &return_value); assert(!status); return return_value; } }; //*** class sema #include #include std::auto_ptr expensive; sema sema_flag(0); sema sema_lock(1); const std::string & DCSI_SEMAS() { // 1st check (sem_getvalue(&m_sem, &return_value)) if (!sema_flag.getvalue()) { // Lock (ctor: sem_wait(&m_sem), dtor: sem_post(&m_sem) sema::lock_guard guard(sema_lock); // 2nd check (sem_getvalue(&m_sem, &return_value)) if (!sema_flag.getvalue()) { expensive.reset(new std::string("DCSI_SEMAS")); // sema_flag.post(); // sem_post(&m_sem) } } return *expensive; } Action: Add semop() and semctl() to the list of functions that "synchronize memory with respect to other threads". And, perhaps, say that semctrl() operations "unrelated" to semahore value may have no "memory syncronization" effect. Or something like that but more specific. _____________________________________________________________________________ COMMENT Enhancement Request Number 31 terekhov:de.ibm.com Defect in XBD XBD 4.10 (rdvk# 31) {alt-memory-model-2004-11-06} Sat, 6 Nov 2004 18:03:58 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: R3. Reject: we cannot see the problem at the referenced lines, as such this comment is non-responsive. R6: Reject: The review team believes that accepting the proposed change would decrease consensus. The problem statement was not sufficient to convince the standard developers that change was needed to this section. Another aardvark following the aardvark guidelines would need to be filed if this matter is to be reconsidered. As with all problem reports the working group can summarily reject aardvark that are not well formed, the working group is not required to give consulting advice, the problem report should be self contained and understandable by those not familiar with the problem. _____________________________________________________________________________ Page: 0 Line: 0 Section: XBD Problem: Edition of Specification (Year): 2004 Defect code : 3. Clarification required XBD 4.10 is hopelessly broken. It needs (almost) complete overhaul. Here's an attempt... just a sketch. Action: Replace XBD 4.10 by something stating that ---- Applications shall ensure that access to any memory location by more than one thread is restricted such that no thread can read or modify a memory location while another thread of control may be modifying it. Applications shall use functions listed below that synchronize thread execution and ensure that modifications to locations in memory are ordered with respect to accesses to the same memory locations in other threads. There is a happens-before inter-thread ordering with respect to preceding (in program and inter-thread order) modifications of memory locations in a thread calling pthreads_create() and accesses to modified memory locations in the created thread; a thread calling fork() and accesses to modified memory locations in the initial thread of the created process; a terminated thread and accesses to modified memory locations in another thread in the same process after another thread returns from pthread_join() joining the terminated thread, or in another thread in a parent process after another thread returns from wait() or waitpid() observing terminated status of a child process; a thread calling sem_post() on a specified semaphore and accesses to modified memory locations in another thread after another thread calls sem_wait() or sem_trywait() or sem_timedwait() and locks the same semaphore unlocked by the calling thread, or another thread observes the incremented semaphore value of the same semaphore by calling sem_getvalue(); a thread calling sem_wait() or sem_trywait() or sem_timedwait() decrementing semaphore value of a specified semaphore and accesses to modified memory locations in another thread after another thread observes the decremented semaphore of the same semaphore using sem_getvalue(); a thread calling pthread_spin_unlock() on a specified spin lock and accesses to modified memory locations in another thread after another thread calls pthread_spin_lock() or pthread_spin_trylock() and locks the same spin lock unlocked by the calling thread; a thread calling pthread_rwlock_unlock() on a specified read- write lock releasing a write lock and accesses to modified memory locations in another thread after another thread calls pthread_rwlock_rdlock() or pthread_rwlock_timedrdlock() or pthread_rwlock_timedwrlock() or pthread_rwlock_tryrdlock() or pthread_rwlock_trywrlock() or pthread_rwlock_wrlock() and acquires a read or a write lock on the same read-write lock unlocked by the calling thread; a thread calling pthread_mutex_unlock() on a specified mutex and accesses to modified memory locations in another thread after another thread calls pthread_mutex_lock() or pthread_mutex_timedlock() or pthread_mutex_trylock() and locks the same mutex unlocked by the calling thread or when another thread locks the same mutex unlocked by the calling thread and returns from a call to pthread_cond_wait() or pthread_cond_timedwait(); a thread calling pthread_cond_wait() or pthread_cond_timedwait() and accesses to modified memory locations in another thread after another thread calls pthread_mutex_lock() or pthread_mutex_timedlock() or pthread_mutex_trylock() and locks the same mutex unlocked by the calling thread or when another thread locks the same mutex unlocked by the calling thread and returns from a call to pthread_cond_wait() or pthread_cond_timedwait(); a thread calling pthread_barrier_wait() on a specified barrier and accesses to modified memory locations in another thread after another thread returns from pthread_barrier_wait() call on the same specified barrier on the same or later barrier cycle; Functions listed above and pthread_cond_signal() and pthread_cond_broadcast() are all side effects. Conforming applications are said to be data race-free. ---- or something like that. Get rid of "memory synchronization" term. ____________________________________________________________________________ OBJECTION Enhancement Request Number 32 gwc:opengroup.org Defect in XBD Definitions (connected socket) (rdvk# 32) [gwc connected socket def] Mon, 1 Nov 2004 11:26:17 +0000 _______________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: ERN 32(->interps) AI-035 See AI-035 This is an interpretation. The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Notes to the editor for a future revision (not part of this interpretation) Insert a new definition before "3.108 Connection": Connected Socket A connection-mode socket for which a connection has been established, or a connectionless-mode socket for which a peer address has been set. See also Sections 3.108, 3.109, 3.110 and 3.348 (on page 83). Note cross volume changes to XSH page 223 line 7458 section connect Change: "The connect() function shall attempt to make a connection on a socket." to: "The connect() function shall attempt to make a connection on a connection-mode socket or to set or reset the peer address of a connectionless-mode socket." page 223 line 7471 section connect Append: "Note that despite no connection being made, the term `connected' is used to describe a connectionless-mode socket for which a peer address has been set." page 1295 line 40598 section send Change: "(including when the peer of a connectionless socket has been set via connect())." to: "If the socket is connectionless-mode, the message shall be sent to the pre-specified peer address." page 1296 line 40635 section send Change: "The socket is not connected or otherwise has not had the peer pre-specified." to: "The socket is not connected." page 1302 line 40853 section sendto Delete "This error may or may not be returned for connection mode sockets." ______________________________________________________________________________ Page: 50 Line: 1714 Section: Definitions Problem: Defect code : 3. Clarification required The standard uses the term "connected" for sockets without defining what it means. For example, in the description of the sendmsg() EISCONN error: "A destination address was specified and the socket is already connected." it is unclear whether this applies only to connection-mode sockets for which a connection has been established, or also to connectionless-mode sockets for which a peer address has been set using connect(). On the one hand there is this statement on the connect() page: "If the initiating socket is not connection-mode, then connect() shall set the socket's peer address, and no connection is made. For SOCK_DGRAM sockets, the peer address identifies where all datagrams are sent on subsequent send() functions, and limits the remote sender for subsequent recv() functions." Since it says "no connection is made" and does not say anything about the socket now being considered "connected", this would seem to point towards such sockets not being "connected". There is also this description of the send() ENOTCONN error: "The socket is not connected or otherwise has not had the peer pre-specified." The use of "otherwise" here implies that "is connected" and "has had the peer pre-specified" are different things. However, on the other hand there are some uses of "connected" elsewhere which suggest the opposite. E.g. in section 2.10.16 Use of Options: "SO_ERROR may be used to check for asynchronous errors on connected connectionless-mode sockets" There are other places which just add to the confusion, such as the sendto() EISCONN description: "A destination address was specified and the socket is already connected. This error may or may not be returned for connection mode sockets." This is in the "may error" part of the ERRORS section, so the second sentence seems to be completely redundant. What, if anything, the presence of an explicit note about connection-mode sockets implies about connectionless-mode sockets is anyone's guess. Then there is this statement on the send() page which could be taken either way: "The send() function shall send a message only when the socket is connected (including when the peer of a connectionless socket has been set via connect())." The need for the remark in parentheses to be present seems to imply that without it the word "connected" would not be considered to apply, but its use of "including" suggests the opposite. Action: Insert a new definition before "3.108 Connection": Connected Socket A connection-mode socket for which a connection has been established, or a connectionless-mode socket for which a peer address has been set. See also Sections 3.108, 3.109, 3.110 and 3.348 (on page 83). The following changes to XSH would also help to remove some of the confusion: page 223 line 7458 section connect Change: "The connect() function shall attempt to make a connection on a socket." to: "The connect() function shall attempt to make a connection on a connection-mode socket or to set or reset the peer address of a connectionless-mode socket." page 223 line 7471 section connect Append: "Note that despite no connection being made, the term `connected' is used to describe a connectionless-mode socket for which a peer address has been set." page 1295 line 40598 section send Change: "(including when the peer of a connectionless socket has been set via connect())." to: "If the socket is connectionless-mode, the message shall be sent to the pre-specified peer address." page 1296 line 40635 section send Change: "The socket is not connected or otherwise has not had the peer pre-specified." to: "The socket is not connected." page 1302 line 40853 section sendto Delete "This error may or may not be returned for connection mode sockets." _____________________________________________________________________________ EDITORIAL Enhancement Request Number 33 konrad.schwarz:siemens.com Defect in XBD nanosleep (rdvk# 33) {1} Tue, 9 Nov 2004 08:17:49 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 It was deliberate not to put restrict in the arguments to nanosleep , and they can point to the same object. Strictly speaking no change is required. It was agreed that as an an editorial improvement we could add the words as suggested. _____________________________________________________________________________ Page: 0 Line: 0 Section: nanosleep Problem: Edition of Specification (Year): 2004 Defect code : 3. Clarification required http://www.opengroup.org/onlinepubs/000095399/functions/nanosleep.html The wording in the Description section does not make it clear whether the rqtp and the rmtp arguments may point to the same object. Although the RETURN VALUE section says the rmtp is updated by the routine, it does not explicitly say this is done only on the return path, after rqtp has been read. Action: Add rqtp and rmtp may point to the same object. after If the rmtp argument is non-NULL, the timespec structure referenced by it is updated to contain the amount of time remaining in the interval (the requested time minus the time actually slept). in the Return Value section, unless this is not true or this can be implied from other parts of the standard. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 34 terekhov:de.ibm.com de.ibm.com (rdvk# 34) {alt-msync-rwlocks-2004-11-08} Mon, 8 Nov 2004 11:10:53 GMT _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_X___ Rationale for rejected or partial changes: R6: Reject: The review team believes that accepting the proposed change would decrease consensus. Memory synchronization does not apply to applications that ignore write locks that had been obtained by other threads. If an application misuses locks the standard cannot prevent data corruption and should not be expected to do so _____________________________________________________________________________ Page: 0 Line: 0 Section: 4.10 Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission Just to sorta stress the point even further (I mean the need for something along the lines of alt-memory-model-20