Minutes of the 19 April 2012 Teleconference Austin-555 Page 1 of 1 Submitted by Andrew Josey, The Open Group. 20 April, 2012 Attendees Joerg Schilling, Fraunhofer Society Don Cragun, PASC OR Nick Stoughton, USENIX, ISO/IEC OR Geoff Clare, The Open Group Eric Blake, Red Hat Andrew Josey, The Open Group Jim Pugsley, Oracle Apologies Mark Brown, IBM, TOG OR - TC1-2008 development The changes to the TC1 draft as a result of the last round of review have been applied, as have the Mandatory Editorial Coordination comments. We will be able to start progressing this towards its ballot soon. Andrew will contact IEEE to check that we are ok to proceed (given the change to IEEE ballot groups). The next step is to draft the letter to accompany the ballot. It was agreed that we need to be clear in the ballot that the document under ballot is the TC1 document, and not the merged document which will be provided as an aid to reviewers. - Update to Maintenance Procedures Andrew had completed the change to the maintenance procedures including the addition of the new standing document SD/6. He should ensure that the document register entries for Austin/112 and the home page reference SD/6. (now completed) * Old Business Bug 0000529: fildes unspecified on close()'s [EINTR] Accept as Marked http://austingroupbugs.net/view.php?id=529 Eric has completed his action in bugnote 1200. Geoff had posted some proposed wording changes in austin-group-l:archive/latest/17412. The changes were applied to bug note 1200 and accepted as marked. This item is tagged for Issue 8. An interpretation is required. Interpretation response: The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: Currently implementations can choose whether to close the file descriptor when close() returns an [EINTR] error. This is unsatisfactory since multithreaded applications need to know whether the file descriptor has been closed in order to know whether it is safe to call close() again. Notes to the Editor (not part of this interpretation): After line 15033 [XBD Constants for Functions], add: The header shall define the following symbolic constant as a value for the flag used by posix_close( ): POSIX_CLOSE_RESTART Allows restarts if a signal interrupts a close operation. This constant shall not be 0 unless posix_close( ) never returns -1 with errno set to EINTR. After line 15094 [XBD Declarations], add: int posix_close(int, int); At line 23862 [XSH close NAME], change: close - close a file descriptor to: close, posix_close - close a file descriptor After line 22865 [XSH close SYNOPSIS], add: int posix_close(int fildes, int flag); At line 22871 [XSH close DESCRIPTION], change: If close( ) is interrupted by a signal that is to be caught, it shall return -1 with errno set to [EINTR] and the state of fildes is unspecified. If an I/O error occurred while reading from or writing to the file system during close( ), it may return -1 with errno set to [EIO]; if this error is returned, the state of fildes is unspecified. to: If close( ) is interrupted by a signal that is to be caught, then it is unspecified whether it returns -1 with errno set to [EINTR] with fildes remaining open, or returns -1 with errno set to [EINPROGRESS] with fildes being closed, or returns 0 to indicate successful completion; except that if POSIX_CLOSE_RESTART is defined as 0, then the option of returning -1 with errno set to [EINTR] and fildes remaining open shall not occur. If close() returns -1 with errno set to [EINTR], it is unspecified whether fildes can subsequently be passed to any function except close( ) or posix_close( ) without error. For all other error situations (except for [EBADF] where fildes was invalid), fildes shall be closed. If fildes was closed even though the close operation is incomplete, the close operation shall continue asynchronously and the process shall have no further ability to track the completion or final status of the close operation. After line 22915 [XSH close DESCRIPTION], add: The posix_close( ) function shall be equivalent to the close( ) function, except with the modifications based on the flag argument as described below. If flag is 0, then posix_close shall not return -1 with errno set to [EINTR], which implies that fildes will always be closed (except for [EBADF], where fildes was invalid). If flag includes POSIX_CLOSE_RESTART and POSIX_CLOSE_RESTART is defined as a non-zero value, and posix_close( ) is interrupted by a signal that is to be caught, then posix_close( ) may return -1 with errno set to [EINTR], in which case fildes shall be left open; however, it is unspecified whether fildes can subsequently be passed to any function except close( ) or posix_close( ) without error. If flag is invalid, posix_close( ) may fail with errno set to [EINVAL], but shall otherwise behave as if flag had been 0 and close fildes. At line 22920 [XSH close ERRORS], change: The close( ) function shall fail if: [EBADF] The fildes argument is not a valid file descriptor. [EINTR] The close( ) function was interrupted by a signal. The close( ) function may fail if: [EIO] An I/O error occurred while reading from or writing to the file system. to: The close( ) and posix_close( ) functions shall fail if: [EBADF] The fildes argument is not a valid file descriptor. [EINPROGRESS] The function was interrupted by a signal and fildes was closed but the close operation is continuing asynchronously. The close( ) and posix_close( ) functions may fail if: [EINTR] The function was interrupted by a signal, POSIX_CLOSE_RESTART is defined as non-zero, and (in the case of posix_close( )) the flag argument included POSIX_CLOSE_RESTART, in which case fildes is still open. [EIO] An I/O error occurred while reading from or writing to the file system. The posix_close( ) function may fail if: [EINVAL] The value of the flag argument is invalid. The close( ) and posix_close( ) functions shall not return an [EAGAIN] or [EWOULDBLOCK] error. If POSIX_CLOSE_RESTART is zero, the close( ) function shall not return an [EINTR] error. The posix_close( ) function shall not return an [EINTR] error unless flag includes a non-zero POSIX_CLOSE_RESTART. At line 22964 [XSH close RATIONALE], change: The use of interruptible device close routines should be discouraged to avoid problems with the implicit closes of file descriptors by exec and exit( ). This volume of POSIX.1-2008 only intends to permit such behavior by specifying the [EINTR] error condition. to: The use of interruptible device close routines should be discouraged to avoid problems with the implicit closes of file descriptors, such as by exec, process termination, or dup2( ). This volume of only intends to permit such behavior by specifying the [EINTR] error condition for close( ) and posix_close( ) with non-zero POSIX_CLOSE_RESTART, to allow applications a portable way to resume waiting for an event associated with the close operation (for example, a tape drive rewinding) after receiving an interrupt. This standard also permits implementations to define POSIX_CLOSE_RESTART to 0 if they do not choose to provide a way to restart an interrupted close action. Although the file descriptor is left open on [EINTR], it might no longer be usable; that is, passing it to any function except close( ) or posix_close( ) might result in an error such as [EIO]. If an application must guarantee that data will not be lost, it is recommended that the application use fsync( ) or fdatasync( ) prior to the close operation, rather than leaving the close operation to deal with pending I/O and risk an interrupt. Earlier versions of this standard left the state of fildes unspecified after errors such as [EINTR] and [EIO]; and implementations differed on whether close( ) left fildes open after [EINTR]. This was unsatisfactory once threads were introduced, since multithreaded applications need to know whether fildes has been closed. Applications cannot blindly call close( ) again, because if fildes was closed by the first call another thread could have been allocated a file descriptor with the same value as fildes, which must not be closed by the first thread. On the other hand, the alternative of never retrying close( ) would lead to a file descriptor leak in implementations where close( ) did not close fildes, although such a leak may be harmless if the process is about to exit or the file descriptor is marked FD_CLOEXEC and the process is about to be replaced by exec. This standard introduced posix_close( ) with a flag argument that allows a choice between the two possible error behaviors, and leaves it unspecified which of the two behaviors is implemented by close( ) (although it is guaranteed to be one of the two behaviors of posix_close( ), rather than leaving things completely unspecified as in earlier versions of the standard). Note that the standard requires that close( ) and posix_close( ) must leave fildes open after [EINTR] (in the cases where [EINTR] is permitted) and must close the file descriptor regardless of all other errors (except [EBADF], where fildes is already invalid). In general, portable applications should only retry a close( ) after checking for [EINTR] (and on implementations where POSIX_CLOSE_RESTART is defined to be zero, this retry loop will be dead code), and risk a file descriptor leak if a retry loop is not attempted. It should also be noted that [EINTR] is only possible if close( ) can be interrupted; if no signal handlers are installed, then close( ) will not be interrupted. Conversely, if a single-threaded application can guarantee that no file descriptors are opened or closed in signal handlers, then a retry loop without checking for [EINTR] will be harmless (since the retry will fail with [EBADF]), but guaranteeing that no external libraries introduce the use of threading can be difficult. There are additional guarantees for applications which will only ever be used on systems where POSIX_CLOSE_RESTART is defined as 0. These observations should help in determining whether an application needs to have its close( ) calls audited for replacement with posix_close( ). It should also be noted that the requirement for posix_close( ) with a flag of 0 to always close fildes, even if an error is reported, is similar to the requirements on fclose( ) to always release the stream, even if an error is encountered while flushing data. Implementations that previously always closed fildes can meet the new requirements by translating [EINTR] to [EINPROGRESS] in close( ); and may define POSIX_CLOSE_RESTART to 0 rather than having to add restart semantics. On the other hand, implementations that previously left fildes open on [EINTR] can map that to posix_close( ) with POSIX_CLOSE_RESTART, and must add the semantics of posix_close( ) when flag is 0; one possibility is by calling the original close( ) implementation, checking for failure, and on [EINTR], using actions similar to dup2( ) to replace the incomplete close operation with another file descriptor that can be closed immediately by another call to the original close( ), all before returning to the application. Either way, close( ) should always map to one of the two behaviors of posix_close( ), and implementations are encouraged to keep the behavior of close( ) unchanged so as not to break implementation-specific expectations of older applications that were relying on behavior not specified by older versions of this standard. The standard developers considered introducing a thread-local variable that close( ) would set to indicate whether it had closed fildes when returning -1. However, this was rejected in favor of the simpler solution of tightening close( ) to guarantee that fildes is closed except for [EINTR], and exposing a choice of whether to expect [EINTR] by adding posix_close( ). Additionally, while the name posix_close( ) is new to this standard, it is reminiscent of at least one implementation that introduced an alternate system call named close_nocancel( ) in order to allow an application to choose whether restart semantics were desired. Another consideration was whether implementations might return [EAGAIN] as an extension and whether close( ) should be required to leave the file descriptor open in this case, since [EAGAIN] normally implies an operation should be retried. It seemed very unlikely that any implementation would have a legitimate reason to return [EAGAIN] or [EWOULDBLOCK], and therefore this requirement would mean applications have to include code for an error case that will never be used. Therefore close( ) is now forbidden from returning [EAGAIN] and [EWOULDBLOCK] errors. Before page 1410 [XSH posix_fadvise], add a new redirect page: NAME posix_close - close a file descriptor SYNOPSIS #include int posix_close(int fildes, int flag); DESCRIPTION Refer to close( ). *New business Bug 0000519: Add to make macro variable pattern substitution (expansion), e.g., $(foo:%.o=%.c) Accepted as Marked http://austingroupbugs.net/view.php?id=519 This item is tagged for Issue 8 After the paragraph: Macro expansions in string1 of macro definition lines shall be evaluated when read. Macro expansions in string2 of macro definition lines shall be performed when the macro identified by string1 is expanded in a rule or command. add the following new paragraphs: Macro expansions using the forms $(string1:[op]%[os]=[np][%][ns]) or ${string1:[op]%[os]=[np][%][ns]} are called pattern macro expansions, where "op" is the old prefix, "os" is the old suffix, "np" is the new prefix and "ns" is the new suffix. Any item inside square brackets is optional. With this form, when the macro string1 is expanded each whitespace separated word that completely matches the "[op]%[os]" pattern on the left hand side of the ('='), where the ('%') character matches zero or more characters, shall be replaced by the right hand side of the and shall then be further modified according to the use of characters as described below. Any words that do not match shall be unmodified in the expansion. If more than one character appears on the left hand side of the ('='), the second and subsequent characters shall be treated as literal characters in "os". If no character appears on the right hand side of the , no further modification of the word shall be performed. If a single character appears on the right hand side, the character in the word shall be replaced with the characters matched by the on the left hand side. If more than one character appears on the right hand side, it is unspecified whether the first character in the word is replaced with the characters matched by the on the left hand side and all remaining characters are left unchanged, or each character is replaced with the characters matched by the on the left hand side. In both macro expansion forms, any macro expansions on the right hand side of the colon shall be recursively expanded before further examination. If this results in more than one after the , the first one shall be considered to be the separator. If the original value of the variable named by string1 is an empty string, the final result shall be an empty string. Matching shall be case-sensitive. Bug 0000505: Blank and empty lines in makefiles Accepted as Marked http://austingroupbugs.net/view.php?id=505 Don had submitted a proposal in bugnote 1208. An Interpretation is required. Tagged for TC2-2008 Interpretation response: The standard is unclear on this issue, and no conformance distinction can be made between alternative implementations based on this. This is being referred to the sponsor. Rationale: The standard as currently written doesn't allow blank lines and empty lines in places where all known implementations have allowed them to appear. These changes more accurately reflect existing practice. Notes to the Editor (not part of this interpretation): Change the last sentence on P2911, L95644-95645 from: comment Comments start with a (’#’) and continue until an unescaped is reached. to: There are three kinds of comments: blank lines, empty lines, and a ('#') and all following characters up to the first unescaped character. Blank lines, empty lines, and lines with ('#') as the first character on the line are also known as comment lines. Change "empty lines, and comments" on P2911, L95656 to "and comments". Change: The trailing and any comment shall be discarded. on P2912, L95671 to: The trailing , any characters immediately preceeding a comment, and any comment shall be discarded. Change: An empty or blank line, or a line beginning with ’#’, may begin a new entry. on P2913, L95722 to: Any comment line may begin a new entry. Change: The algorithm for determining a new entry for target rules is partially unspecified. Some historical makes allow blank, empty, or comment lines within the collection of commands marked by leading characters. A conforming makefile must ensure that each command starts with a , but implementations are free to ignore blank, empty, and comment lines without triggering the start of a new entry. on P2926, L96289-96293 to: The algorithm for determining a new entry for target rules is partially unspecified. Some historical makes allow comment lines (including blank and empty lines) within the collection of commands marked by leading characters. A conforming makefile must ensure that each command starts with a , but implementations are free to ignore comments without triggering the start of a new entry. Add a new paragraph to RATIONALE after P2928, L96368: Earlier versions of this standard defined comment lines only as lines with '#' as the first character. Many places then talked about comments, blank lines, and empty lines; but some places inadvertently only mentioned comments when blank lines and empty lines had also been accepted in all known implementations. The standard now defines comment lines to be blank lines, empty lines, and lines starting with a '#' character and explictily lists cases where blank lines and empty lines are not acceptable. The rest of the call was spent discussing the sun_path issue raised on the reflector on sequence austin-group-l:archive/latest/17413 This item needs to be filed as a Mantis defect report. We need to determine what Solaris, HP-UX and AIX do in this area. We will continue the discussion on the next call. Next Steps ---------- The next call is on May 3 and will continue processing defect reports. This call will be for the regular 90 minutes. http://austingroupbugs.net See the calendar for the list of dialup numbers. An IRC channel will be available for the meeting irc://irc.freenode.org/austingroupbugs