Unapproved draft Last update 5 Mar 2008 ___________________________________________________________________________ COMMENT Enhancement Request Number 1 gwc:opengroup.org Defect in XRAT B.2.9.5 (rdvk# 1) [gwc cancellation points] Mon, 24 May 2004 15:40:27 +0100 ______________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 (incorporated in D2R) _____________________________________________________________________________ Page: 175 Line: 7396-7399 Section: B.2.9.5 Problem: Defect code : 1. Error The first paragraph of the rationale for cancellation points contains some historical left-over text which contradicts the normative text. It should be updated (at the next opportunity) to reflect the current intentions. Action: Change "Cancellation points are points inside of certain functions where a thread has to act on any pending cancellation request when cancelability is enabled, if the function would block. As with checking for signals, operations need only check for pending cancellation requests when the operation is about to block indefinitely." to "Cancellation points are points inside of certain functions where a thread has to act on any pending cancellation request when cancelability is enabled. For functions in the "shall occur" list, a cancellation check must be done on every call regardless of whether, absent the cancellation, the call would have blocked. For functions in the "may occur" list, a cancellation check may be done on some calls but not others; i.e. whether or not a cancellation point occurs when one of these functions is being executed can depend on current conditions." _____________________________________________________________________________ OBJECTION Enhancement Request Number 2 eggert:cs.ucla.edu Defect in XRAT Utility Syntax Guidelines (rdvk# 2) {20040905a} Sun, 5 Sep 2004 22:08:56 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 for D2R Insert the following before XRAT page 75 line 3057: Unless explicitly stated otherwise in the utility description guideline 9 requires applications to put options before operands, and requires utilities to accept any such usage without misinterpreting operands as options. For example, if an implementation of the "printf" utility supports an "-e" option as an extension, the command "printf %s -e" must output the string "-e" without interpreting the "-e" as an option. _____________________________________________________________________________ Page: 75 Line: 3057 Section: Utility Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission This is a followup to the September 1 draft of Austin Group Interpretation reference 1003.1-2001 #027. Can utilities support, as an extension to POSIX, usages like "sort input -o output" and "ls file -l" where options follow operands? Such usages clearly violate the Utility Syntax Guidelines, but it is not clear from the standard or the rationale whether a conforming implementation must attempt to access files named "-o" and "-l" when given these two commands, or whether it can interpret the "-o" and "-l" as options. Common practice is to treat "sort input -o output" as if it were "sort -o output input", as this was required by earlier editions of POSIX. Common practice on GNU hosts is to treat "ls file -l" as if it were "ls -l file". The standard no longer distinguishes between "sort input -o output" and "ls file -l", so presumably both forms fall into the same category, as far as the standard is concerned. The motivation for Interpretation reference 1003.1-2001 #027 is to allow support for common practice, so the proposed action assumes this is the case here too. A downside of the proposed action is that portable applications that deal with arbitrary file names must prepend "./" to names that begin with "-", even if the file name is not the first operand. But this problem is already clearly the case for first operands, and for any file whose name is the single character "-". Since leading "./" works in general, it is suggested in the proposed action. Action: Insert the following before XRAT page 75 line 3057: Guideline 9 requires applications to put options before operands, and requires utilities to accept any such usage without misinterpreting operands as options. For example, even if an implementation of the "printf" utility supports an "-e" option as an extension, the command "printf %s -e" must output the string "-e" without interpreting the "-e" as an option. However, since portable file names cannot begin with "-", utilities that use operands to represent files can support trailing options as an extension. For example, an implementation can support the obsolescent command "sort input -o output" and the GNU-style command "ls file -l" as extensions to POSIX. Portable applications should therefore prepend "./" to any operands that represent files and that begin with "-". _____________________________________________________________________________ OBJECTION Enhancement Request Number 3 eggert:cs.ucla.edu Defect in XRAT Utility Syntax Guidelines (rdvk# 3) {20040903a} Sat, 4 Sep 2004 08:22:36 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_of_XCU35 Reject_____ Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 76 Line: 3075 Section: Utility Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission This section of the rationale does not sufficiently explain Utility Syntax Guideline 13 (XBD page 204 lines 7233-7236), and this has led to problems of interpretation discussed on austin-group-l in August 2004. The point requiring further clarification is usages like "od -" and "cksum -", where Guideline 13 seems to say that "-" must stand for standard input, whereas the rationale clearly says that utilities like "od" and "cksum" need not accept "-" to mean standard input. There is a similar issue with "tee -" and standard output. Current practice is wildly inconsistent in this area, as Gunnar Ritter showed . My understanding is that the correct interpretation of Guideline 13 is that specifications claiming conformance to the Utility Syntax Guidelines must not use "-" for any reason other than standard input (or output); however, such specifications need not use "-", and when they do not use "-" the standard does not specify the behavior of "-". If my understanding is correct, the only real question is whether usages like "od -" have undefined or unspecified behavior. The action proposed below uses the word "undefined", but "unspecified" would be fine with me too. One other minor point that tripped me up was that "-" does not have a special meaning when it is an option-argument, e.g., "sort -o -" must write to a file named "-". The action proposed below addresses this minor problem as well. Action: Append the following text after XBD page 76 line 3075: If an individual utility section does not describe the treatment of the '-' operand, its behavior is undefined. For example, commands like "cksum -", "od -", and "tee -" have undefined behavior. Historical practice in these cases is to treat '-' as standard input (or output), or as a file named "-", but there is little consistency within or across implementations and portable applications cannot rely on either behavior. '-' has its special meaning only when it is an operand. For example, the command "sort -o -" must write to a file named "-", not to standard output, because '-' is an option-argument here, not an operand. _____________________________________________________________________________ OBJECTION Enhancement Request Number 4 eggert:cs.ucla.edu Defect in XRAT Utility Syntax Guidelines (rdvk# 4) {20040905b} Sun, 5 Sep 2004 22:51:36 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: (the suggested text has been incorporated in AI-027) _____________________________________________________________________________ Page: 76 Line: 3076 Section: Utility Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission This is a followup to the September 1 draft of Austin Group Interpretation reference 1003.1-2001 #027. That interpretation says "the standard is clear", but my experience has been otherwise. I interpreted Utility Syntax Guideline 3 "Multi-digit options should not be allowed" as a prohibition on support for multi-digit options, and contributed implementations along those lines to the GNU core utilities in February 2002, so that programs like "head" did not support multi-digit options when configured to conform to POSIX 1003.1-2001. Since then, several users have expressed dismay that support for the old usage was withdrawn from suitably-configured coreutils installations, but when I pointed them to the text of the standard they didn't see any other way to read it either. Only recently, when one of these users found the time to take the matter up with the standardization committee, did I discover that we had misinterpreted the standard. Action: Add the text of the interpretation to the rationale, by inserting the following before XRAT page 76 line 3076: The standard permits implementations to have extensions that violate the Utility Syntax Guidelines so long as when the utility is used in line with the forms defined by the standard that it follows the Utility Syntax Guidelines. Thus "head -42 file" and "ls --help" are permitted as extensions. The intent is to allow extensions so long as the standard form is accepted and follows the guidelines. _____________________________________________________________________________ COMMENT Enhancement Request Number 5 eggert:cs.ucla.edu Defect in XRAT Utility Syntax Guidelines (rdvk# 5) {20040928b} Tue, 28 Sep 2004 21:07:37 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_of_3 Reject_____ Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 76 Line: 3075 Section: Utility Problem: Edition of Specification (Year): 2004 Defect code : 3. Clarification required It's not clear from the standard what implementations are allowed to do with commands like "cksum -" where the standard does not require "-" to be treated as standard input (or output). Utility Syntax Guideline 13 (XBD page 204 lines 7233-7236) says that for such utilities, "the '-' operand should be used only to mean standard input", which would seem to imply that "cksum -" must read standard input. However, the rationale (XRAT page 75 lines 3070-3074) countermands this, saying that standard utilities like cksum need not accept "-" to mean standard input, and that the standard says only "that, by default, '-' operands are not used for other purposes". Left unsaid in the rationale is what implementations of commands like "cksum -" are allowed to do: is the behavior undefined, or completely unspecified, or only partially specified? Action: Add the following text to the Utility Syntax Guidelines rationale (after XRAT page 76 line 3075): For file-reading (or -writing) utilities like "cksum", "od", and "tee" where the standard does not require "-" to be treated as standard input (or output), it is unspecified whether a "-" operand is treated as standard input (or output) or as a pathname. _____________________________________________________________________________ OBJECTION Enhancement Request Number 6 eggert:cs.ucla.edu Defect in XRAT Utility Syntax Guideline 9 (rdvk# 6) {20041007a} Thu, 7 Oct 2004 08:46:06 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_X___ Duplicate_____ Reject_____ Rationale for rejected or partial changes: The sort utility has been addressed in the response to AI-027. It appears to be a bug for some implementations to allow pr to intersperse args with operands i.e pr filename +2 filename2 It should issue a diagnostic or print the file named "+2" We believe split, touch and uniq should behave the same way as pr should. For utilities in the CD and SD options and the m4 utility guideline 9 may not apply. ** CROSS VOLUME CHANGE TO XCU *** Add to SD5 for D2R Modify the first paragraph in OPTIONS for utility XXXXX to add "except for guideline 9") So for an example with no exceptions (make), this becomes: The XXXXX utility shall conform to the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines, except for guideline 9. where XXXXX is lex, yacc, ar, make For c99, add a bullet point ~8290 Options may be interspersed with operands. for m4, add to the end of the para (page 579, line 22354): and options may be interspersed with operands (note specific edits may be different where existing exceptions exist) _____________________________________________________________________________ Page: 75 Line: 9 Section: Utility Problem: Edition of Specification (Year): 2004 Defect code : 2. Omission The standard does not currently address the issue of what can happen when a standard utility is given an operand followed by an option. Such a usage clearly violates Utility Syntax Guideline 9, and portable applications cannot invoke utilities that way, but the issue is whether a conforming implementation can support such a usage as an extension. Draft AI-027 says that conforming utilities are allowed to support extensions that violate the guidelines so long as the extensions do not conflict with standard usage. So the issue here is whether the standard prohibits support for options after operands. Here are some examples that illustrate the issue. (This list is not exhaustive.) c99 src.c -Dfieldlist=a.b.c m4 file1 -Dx=y file2 make target -f makefile pr file1 +2 file2 sort file1 -T /var/tmp file2 sort input -o output split file -5 touch file1 -c file2 uniq file -c All these examples work in many modern implementations that claim POSIX conformance. For example, they all work in Solaris 9 in POSIX-conforming mode (substituting "c89" for "c99"). And this is not surprising, since these examples all date back to 7th Edition Unix (substituting "cc" for "c99"). Only the "sort input -o output" example was standardized in a previous POSIX edition (as an obsolescent form, since withdrawn); the other examples have always been common behaviors that are not required by any standard. GNU implementations take the above tradition further: they support options-after-operands for all (or almost all) utilities that accept pathname operands (or other operands where leading "-" is not allowed by POSIX 1003.1-2004, e.g., "make" targets). This extension is typically disabled by setting the POSIXLY_CORRECT environment variable, but for various reasons this environment variable is rarely used in practice. If the standard is changed to prohibit support for options after operands, most current implementations will not conform. On the other hand, assuming support is permitted, application writers should be notified about potential portability problems in this area, and it might be wise to prohibit support in contexts where operands with leading "-" have always been allowed (e.g., "printf" operands). One might object that there is no possible ambiguity since a naive implementation of "make target -f makefile" using getopt() must treat both the "-f" as an operand. However, this objection is flawed, as will be discussed in a later Aardvark on getopt(). And even if the objection were sustained, the standard would still be made clearer by explicitly addressing the potential ambiguity. One might also object that if XBD ERN 20 is accepted as-is, then portable pathnames can begin with "-" so the traditional utility behavior is no longer tenable. But (as XBD ERN 20 itself notes) these are separable issues: traditionally, pathnames with leading "-" have always been supported by function implementations, but not necessarily by utility implementations, and it is the utilities that are at issue here. This Aardvark proposes a relatively permissive approach: a proposed clarification that allows common existing practice, but attempts to draw the line in roughly the same place that the line is drawn for the "-" argument: namely, support for the options-after-operands is allowed only in contexts where leading "-" has traditionally been problematic. Action: Proposed response: For commands that contain the "--" delimiter, the standard is clear: all arguments after the "--" delimiter must be accepted as operands, not options. Otherwise, the standard does not speak to this issue, and as such no conformance distinction can be made between alternative implementations based on their behavior with commands like "make target -f makefile". This is being referred to the sponsor. Notes to the Editor (not part of this interpretation): ----------------------------------------------------- A future revision should consider adding the following text to the Utility Syntax Guidelines rationale (after XRAT page 75 line 3056): Guideline 9 constrains standard usage, but implementations may provide extensions that violate Guideline 9. Previous editions of this standard, historic practice, and some modern implementations support commands where options-after-operands can occur in pathname and some other contexts. Examples of such usages include "sort input -o output", "make target -f makefile", "pr file1 +2 file2", and "ls file -l". Such usages are not portable, but implementations can support them as obsolescent extensions. However, implementations should not support options-after-operands for commands like "env" and "printf" where operands are not necessarily pathnames (or other constructs, such as make targets and user names, that have the same restrictions as portable filenames). Applications can prevent trailing operands from being treated as options by using the "--" delimiter. They can also prevent pathname operands from having a special meaning by prefixing "./". The latter technique works even for the special operand "-". ____________________________________________________________________________ EDITORIAL Enhancement Request Number 7 mstrbill:us.ibm.com Defect in XRAT B.2.11.10 Rationale (rdvk# 7) {WLT-XRAT-1} Mon, 26 Sep 2005 20:41:11 +0100 (BST) ___________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject____ Rationale for rejected or partial changes: Add to SD5 for D2R _____________________________________________________________________________ Page: 205 Line: 8598 Section: B.2.11.10 Problem: Edition of Specification (Year): 2004 Defect code : 1. Error In XRAT section B.2.11.10 the following is found. Overrun in Dumping Trace Streams into Trace Logs The user lets the trace mechanism dump the trace stream initialized with POSIX_TRACE_FLUSH policy automatically into a trace log. If the dump operation is slower than the recording of trace events, the trace stream can overrun. This condition is detected by getting the status of the active trace stream (function posix_trace_get_status( )) and looking at the member posix_log_overrun_status of the read posix_stream_status structure. This overrun indicates that the trace mechanism is not able to operate in this mode at this speed. It is the responsibility of the user to modify one of the trace parameters (the stream size or the trace event type filter, for instance) to avoid such overrun conditions, if overruns are to be prevented. The same already predefined trace event types (see Overrun in Trace Streams Initialized with POSIX_TRACE_LOOP Policy) are used to detect and to know the duration of an overflow. the wording states that posix_log_overrun_status is checked for trace stream overruns. But according to XSH 2.11.1.1 Structures, second table, posix_log_overrun_status is only required if POSIX TRACE LOG is supported. The first table states that posix_stream_overrun_status is defined as "Indicates whether trace events were lost in the trace stream." Seems to be a simple substitution error. Action: replace posix_log_overrun_status with posix_stream_overrun_status in above text. _____________________________________________________________________________ COMMENT Enhancement Request Number 8 gwc@opengroup.org Defect in XRAT B.2.1 (rdvk# 1) [gwc macro functions] Fri, 29 Jun 2007 10:12:16 +0100 _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 for D4R _____________________________________________________________________________ Page: 86 Line: 3407 Section: B.2.1 Problem: Defect code : 2. Omission As part of a discussion on the Austin Group mailing list in Nov 2006, a problem came to light concerning the use of standard function names in macro values defined in standard headers. It was felt that since this is a trap that several implementations fall into, it is worth mentioning in the POSIX rationale. Action: Add to the end of B.2.1 Use and Implementation of Functions: Implementors should note that since applications can #undef a macro in order to ensure that the function is used, this means that it is not safe for implementations to use the names of any standard functions in macro values, since the application could use #undef to ensure that no macro exists and then use the same name for an identifier with local scope. For example, historically it was common for a getchar() macro to be defined in as: #define getchar() getc(stdin) This definition does not conform, because an application is allowed to use the identifier getc with local scope, and the expansion of the getchar() macro would then pick up the local getc. The following is conforming code, but would not compile with the above definition of getchar(): #include #undef getc int main(void) { int getc; getc = getchar(); return getc; } This does not only affect function-like macros. For example the following definition does not conform because there could be a local sysconf variable in scope when SIGRTMIN is expanded: #define SIGRTMIN ((int)sysconf(_SC_SIGRT_MIN)) Implementors can avoid the problem by using aliases for standard functions instead of the actual function, with names that conforming applications cannot use for local variables. For example: #define SIGRTMIN ((int)__sysconf(_SC_SIGRT_MIN)) _____________________________________________________________________________ OBJECTION Enhancement Request Number 9 mh-austin@osj.us Defect in XRAT B.2.12 (rdvk# 1) {mh200402} Thu, 24 Jan 2008 06:03:50 GMT _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: Add to SD5 for D5R _____________________________________________________________________________ Page: 0 Line: 0 Section: B.2.12 Problem: Edition of Specification (Year): 2004 Defect code : 1. Error The off_t data type rationale in XRAT B.2.12 http://www.opengroup.org/onlinepubs/009695399/xrat/xsh_chap02.html#tag_03_02_12 contains obsolete information which is no longer correct. In fact, all 5 statements are arguably incorrect. The off_t data type is now used in many additional system interfaces, and on modern systems off_t is a 64-bit type in at least one programming environment. Action: Change: off_t This type is used only in lseek(), fcntl(), and . Many implementations would have difficulties if it were defined as anything other than long. Requiring an integer type limits the capabilities of lseek() to four gigabytes. The ISO C standard supplies routines that use larger types; see fgetpos() and fsetpos(). XSI-conformant systems provide the fseeko() and ftello() functions that use larger types. To: off_t This type is used to represent a file offset or file size. On systems supporting large files, off_t is larger than 32 bits in at least one programming environment. Other programming environments may use different sizes for off_t, for compatibility or other reasons. --last-no-9