Minutes of the 21st September 2023 Teleconference Austin-1345 Page 1 of 1 Submitted by Andrew Josey, The Open Group. 22 September 2023 Attendees: Nick Stoughton, Logitech/USENIX, ISO/IEC JTC 1/SC 22 OR Don Cragun, IEEE PASC OR Eric Ackermann, HPI, University of Potsdam Eric Blake, Red Hat, The Open Group OR Mark Ziegast, SHware Systems Dev. Apologies Geoff Clare, The Open Group Tom Thompson, IEEE Andrew Josey, The Open Group * General news * Current Business Note for issue resolution all items are tagged for Issue 8 unless noted otherwise or disposition is reject or duplicate. Bug 1649: Field splitting is woefully under specified, and in places, simply wrong https://austingroupbugs.net/view.php?id=1649 Accepted as Marked Replace XCU section 2.6.5 on issue 8 draft 3 P2476, L80478-80504 with: After parameter expansion (Section 2.6.2), command substitution (Section 2.6.3), and arithmetic expansion (Section 2.6.4), and if the shell variable IFS [xref XCU 2.5.3] is set and its value is not empty, or if the IFS variable is unset, the shell shall scan each field containing results of expansions and substitutions that did not occur in double-quotes for field splitting; zero, one or multiple fields can result. For the remainder of this section, any reference to the results of an expansion, or results of expansions, shall be interpreted to mean the results from one or more unquoted variable or arithmetic expansions, or unquoted command substitutions. If the IFS variable is set and has an empty string as its value, no field splitting occurs. However if an input field which contained the results of an expansion is entirely empty, it shall be removed. Note that this occurs before quote removal, any input field that contains any quoting characters can never be empty at this point. After the removal of any such fields from the input, the possibly modified input field list becomes the output. Each input field is considered in sequence, first to last, with the results of the algorithm described in this section causing output fields to be generated, which remain in the same order as the input fields from which they originated. Fields which contain no results from expansions shall not be affected by field splitting, and shall remain unaltered, simply moving from the list of input fields to be next in the list of output fields. In the remainder of this description, it is assumed that there is present in the field at least one expansion result, this assumption will not be restated. Field splitting only ever alters those parts of the field. For the purposes of this section, the term "IFS white space" shall mean any of the single byte characters , or from the Portable Character Set [xref XBD 6.1] which are present in the value of the IFS variable, and perhaps other White Space characters. It is implementation defined whether other White Space characters which appear in the value of IFS are also considered as "IFS white space". The three characters above specified as IFS white space characters are always IFS white space, when they occur in the value of IFS, regardless of whether they are White Space characters in any relevant locale. For other locale specific White Space characters allowed by the implementation it is unspecified whether the character is considered as IFS white space if it is White Space at the time it is assigned to the IFS variable, or if it is White Space at the time field splitting occurs (the locale may have altered between those events). If the IFS variable is unset, then for the purposes of this section, but without altering the value of the variable, its value shall be considered to contain the three single byte characters , and from the Portable Character Set, all of which are IFS white space characters. The shell shall use the byte sequences that form the characters in the value of the IFS variable as delimiters. Each of the characters and which appears in the value of IFS shall be a single byte delimiter. The shell shall use these delimiters as field terminators to split the results of expansions, along with other adjacent bytes, into separate fields, as described below. Note that these delimiters terminate a field, they do not, of themselves, cause a new field to start, subsequent data bytes that are not from the results of an expansion, or that do not form IFS white space characters are required for a new field to begin. Note that the shell processes arbitrary bytes from the input fields, there is no requirement that those bytes form valid characters. If results of the algorithm are that no fields are delimited, that is, if the input field is wholly empty or consists entirely of IFS white space, the result shall be zero fields (rather than an empty field). For the purposes of this section, when a field is said to be delimited, the the candidate field, as generated below shall become an output field. When the algorithm transforms a candidate into an output field it shall be appended to the current list of output fields. Each field containing the results from an expansion shall be processed in order, intermixed with fields not containing the results of expansions, processed as described above, as if as follows, examining bytes in the input field, beginning to end: Begin with an empty candidate field. While the input is not empty... When instructed to perform the next iteration of the loop, start again with the test here, with the input as modified below. Consider the leading remaining byte or byte sequence of the input. No such byte sequence shall contain data such that some bytes in the sequence resulted from an expansion, and others did not. If the byte or sequence of bytes is:
    A byte (or sequence of bytes) in the input that did not result from an expansion: Append this byte (or sequence) to the candidate, and remove it from the input. Next iteration of the loop. A byte sequence in the input which resulted from an expansion that does not form a character in IFS: Append the first byte of the sequence to the candidate, and remove that byte from the input. Next iteration of the loop. A byte sequence in the input which resulted from an expansion that forms an IFS white space character: Remove that byte sequence from the input, consider the new leading input byte sequence, and repeat this step. A byte sequence in the input that resulted from an expansion that forms an IFS character, which is not IFS white space: Remove that byte sequence from the input, but note it was observed. At this point, if the candidate is not empty, or if a sequence of bytes representing an IFS character that is not IFS white space was seen at the final step, then a field is said to have been delimited, and the candidate becomes an output field. Empty (clear) the candidate, and perform the next iteration of the loop. Once the input is empty, the candidate becomes an output field if and only if it is not empty. The ordered list of output fields so produced, which may be empty, replaces the list of input fields. Next Steps ---------- We will pick up on 251 (if Don's proposal is ready) The next call is on: Mon 2023-09-25 (Zoom meeting - general bugs/ballot resolution) Thu 2023-09-28 (Zoom meeting - general bugs/ballot resolution) Apologies in advance: Geoff Clare 2023-09-11 through 2023-09-28, maybe 2023-10-02 The calls are for 90 minutes Calls are anchored on US time. (8am Pacific) Please check the calendar invites for dial in details. Bugs are at: https://austingroupbugs.net An etherpad is usually up for the meeting, with a URL using the date format as below: https://posix.rhansen.org/p/20xx-mm-dd (For write access this uses The Open Group single sign on, for those individuals with gitlab.opengroup.org accounts. Please contact Andrew if you need to be setup)