Document Number: AUSTIN/126 Title: XCU/ft/3 Aardvark Change Request Report Revision Date: 2002-08-02 Source: Andrew Josey, Chair Action: for review This report contains the draft dispositions of the aardvark comments submitted against the XCU/ft/3 (3rd batch of FT aardvark) text. Aardvark Summary Table ______________________ ERN 1 Accept ERN 2 Accept ERN 3 Accept ERN 4 OPEN ERN 5 Accept ERN 6 Accept _____________________________________________________________________________ COMMENT Enhancement Request Number 1 bjh21@netbsd.org Defect in XCU ar (rdvk# 1) {bjh21:ar trailer} Tue, 2 Jul 2002 13:51:14 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 139 Line: 5246 Section: ar Problem: The description of the 4.4BSD archive format claims that each header ends with the two-byte trailer "". This is incorrect -- the trailer is a backquote followed by a . IEEE Std 1003.2-1992 got this right. Action: Replace "" with "`". [Ed recommendation: TC1 candidate] _____________________________________________________________________________ OBJECTION Enhancement Request Number 2 ajosey@opengroup.org Defect in XCU chmod (rdvk# 6) {bwg2001-010r} Thu, 1 Aug 2002 07:06:19 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 238 Line: 9245 Section: chmod Problem: Defect code : 1. Error This is from the Base Working group, who have found that the original resolution bwg2001-010 was in error and have revised it, resulting in this defect report. The original ERN and bwg2001-010 resulted in specifying behavior that diverges from the historical behavior on UNIX systems. The intent in adding the specification was to document existing behavior. This error may have been caused by a mix up with the who symbols u and o where its possible to read "o" out of context and think "owner" instead of "other". We believe where "o" is used below "u" ("user") should be specified and where u is used o should be specified. It also may be that the original ERN based its text on the gnu chmod o+t behavior for causing an executable to remain allocated on swap. For reference this is in section 3.2.3 of the gnu chmod page. This is not about the behavior of the "restriction deletion flag" for a directory. We have no objection to gnu chmod continuing supporting o+t behavior on an executable, nor the ignore u+t and g+t on non-directory files. We believe the revised changes allow both historic and the gnu behavior. Action: Insert the following replacement text for XCU6, P238, L9245-9248: The perm symbol t shall specify the S_ISVTX bit. When used with a file of type directory, it can be used with the who symbol a, or with no who symbol. It shall not be an error to specify a who symbol of u, g, or o in conjunction with the perm symbol t, but the meaning of these combinations is unspecified. The effect when using the perm symbol t with any file type other than directory is unspecified. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 3 lennox@cs.columbia.edu Defect in XCU ex (rdvk# 2) {1} Thu, 27 Jun 2002 21:49:51 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 398 Line: 15344 Section: ex Problem: Defect code : 1. Error The Rationale section for ex includes a reference to nvi, as a reference implementation for POSIX ex/vi. Unfortunately, the URL provided, , isn't a valid URL. It doesn't even refer to a valid top-level domain, but even if that is fixed (assuming opengroup should be opengroup.org) the referenced host doesn't exist. One host that does exist, ftp.opengroup.org, doesn't seem to host any mirrored software. Action: Replace the URL with a correct URL to a mirror of the nvi source distribution, ideally one hosted by opengroup.org. [Ed recommendation: TC1 candidate] _____________________________________________________________________________ OBJECTION Enhancement Request Number 4 don.cragun@sun.com Defect in XCU file utility (rdvk# 5) {DWC-file:1} Wed, 31 Jul 2002 00:43:11 +0100 (BST) _____________________________________________________________________________ Accept_____ Accept as marked below_____ Duplicate_____ Reject_____OPEN Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 442-447 Line: 17024-17121 Section: file Problem: Defect code : 1. Error *** Background Information *** The file utility has typically been implemented with: T1. a bunch of built-in tests to check for the existence and ability to get file status for the file, T2. a bunch of built-in tests to report the file type for non-regular files, T3. a check for an empty regular file, T4a. tests provided by the implementation that could be specified in a magic file (/etc/magic on many implementations) based on certain values being found at certain locations in a file, T4b. if the implementation supported a -m option, any tests specified by the magic files specified by the -m option (on some historic implementations, step T4b just replaced processing of /etc/magic in step T4a instead of being an additional step), T4c. perform any tests specified by -M options (this is new; it was invented for P1003.2b and didn't exist on any historic implementations as far as I know), T5. more built-in tests that look for certain strings in context, and T6. if none of the above identified a probable file type, a statement that the file contains "data". Note that many tests that could be implemented using a magic file are now frequently built into the file utility so it can provide more information than magic file formats allow (examples: a magic file can identify that a file is a coredump file, but can't print the name of the executable file that dropped core; a magic file can identify that a text file starting with #! is an executable shell script, but can't print the name of the interpreter used, except for a specific list of known interpreters). The tests listed in T5 above try to determine if a file is a Shell script, C-language source, FORTRAN source and other file types that are determined by examining the file contents looking for various strings or patterns that don't appear at specific locations. Note that if these tests are applied before tests in magic files specified by the -m or -M options and if these tests report that a file is ascii text (containing no binary data), no tests from user specified magic files following a -d option will ever be applied to non-binary files. Therefore these T5 tests need to be applied after any user supplied magic files specified by -m and -M options; not at the time a -d option is processed. P1003.2b and XCU6 gave users the ability to specify the -M and -m options to replace or add to "default system tests" and added the -d option to specify when "default system tests" were to be applied, but never defined the term "default system tests". I believe that the intent of the changes from P1003.2b were to allow users to specify additional tests (-m) to be performed in step T4b above or alternative tests (-M) to be performed instead of steps T4a and T5 above. (One might also argue that test T3 should be skipped if -M is specified, but I don't believe that was the original intent.) The description of file lists four set of tests on P442, L17028-17037. The text on lines 17028-17030 correspond to step T2 above (tests here checked for files of type directory, FIFO, socket, block special, character special, and when appropriate symbolic links as specified in Table 4-8). The text on line 17032 corresponds to step T3 above (this tests for an empty regular file as specified in Table 4-8). The text on lines 17033-17035 corresponds to steps T4a (including tests for executable binary, ar archive libraries, cpio archives and ustar archives as specified in Table 4-8) and T5 (including tests for shell scripts (not already identified as executable binaries), C-language source, and FORTRAN source specified in Table 4-8) above. The text on lines 17036-17037 corresponds to step T1 above. Step T6 above isn't mentioned in the description, but is specified in the stdout section on P444, lines 17113-17115. *** Detailed Interpretation Issues *** Issue 1:In the last paragraph of the options section (P442, L17057-17059) it says: "If a -M option is specified, no tests other than those specified using the -d, -M, and -m options shall be applied to the file.". Taken literally, this means that steps T1, T2, T3 and T6 have to be skipped. It clearly makes no sense to try to apply magic file tests to directories and other special files, to non-existent files, nor to empty regular files. I believe the intent was that steps T4a, T4b, T4c, and T5 would only be executed if explicitly requested by -d, -m, -M, and -d options, respectively, but that the other tests would continue to be executed. Is this correct? Issue 2:When the description of -d says that "default system tests" are to be applied, the description of -M says that "default system tests" shall not be applied, and the text at the end of the options sections talks about when "default system tests" are to be applied, what does "default system tests" mean? And, does it mean the same thing everywhere it is used? I believe that the intent was that they are the same. But, the fact that there are two distinct sets of default system tests (steps T4a and T5) causes confusion later. I believe that the intent was as follows: line 17044: description of -d option: I believe "Apply any default system tests to the file." meant that tests in steps T4a and T5 were to be applied. Is this correct? line 17052: description of -M option: I believe "No default system tests shall be applied." meant that tests in steps T4a and T5 were not to be applied unless a -d option was also specified on the command line. Is this correct? line 17056-17057 "concatenation of tests specified" in description of interactions between multiple instances of -m, -d, or -M options and line 17059-17060: description of what happens if neither -d nor -M options are specified: I believe that "If multiple instances of the -m, -d, and -M options are specified, the concatenation of the tests specified, in the order specified, shall be the set of tests that are applied." and "If neither the -d nor the -M options are specified, any default system tests shall be applied after any tests specified using the -m option." really means that steps T4b, T4a, and T4c should be applied, respectively, in sequence for each occurrence of a -m, -d, or -M option. Then, if no -M option and no -d option is specified step T4a shall be applied, And, finally, if no -M option is specified or if both a -M option and a -d option are specified, step T5 should be applied after all step T4a, T4b, and T4c tests are applied. Is this correct? Note that if this if any of these are not correct, the rationale on line 17246 concerning the -d option and on line 17248 concerning the -M option may need to be changed. See also Issue 5 below. Issue 3:In the stdout section, certain strings are required to be written whenever files of these types are found when running in the POSIX locale. Taken literally, this may require the file utility to modify message fields found in magic files named by -M and -m options unless a -d option is specified before -M or -m options that specify magic files containing tests for any of these types of files. I believe the intent was that if these file types were detected in steps T2, T3, T4a, T5, and T6 the type field would contain these strings when run in the POSIX locale; not that these strings would be required if detected in steps T4b and T4c (by tests in user supplied magic files). Is this correct? Issue 4:The -i option essentially specifies that no -d, -M, or -m options are to be processed. But there is no indication that these options are mutually exclusive and no indication that the last specified of these groups of conlicting options wins. I believe that the intent was that the use of the -i option is mutually exclusive with the use of the -d, -M, and -m options. Is this correct? Issue 5:The SYNOPSIS section indicates that only one -m option and only one -M option are allowed on a command line (no ... after the [-M file] and [-m file] entries). But, the OPTIONS section implies that multiple -M options are allowed and maybe multiple -d options and -m options as well. (It says "If a -M option" instead of "If the -M option" on P442, L17057 implying that multiple -M options can be specified. When it says "If multiple instances of the -m, -d, or -M options are specified," on P442, L17056 it is not clear whether this means that each option can be specified more than one time or if it means that -d, -M, and -m are not mutually exclusive, but each should only be specified once.) Note that specifying -d more than once can never identify any more files than specifying -d once, unless the files are changing while the file command is running.. I believe that the intent was that no more than one occurrence each of the -d, -M, and -m options needs to be accepted, but they are not mutually exclusive. Is this correct? Issue 6:The alternative output format for displaying symbolic link information is specified to be used in all locales. Similarly, the string "cannot open" and "data" are specified to be used in all locales to indicate file operands that don't exist or can't be read, or if the type can't be determined. And, the string "regular file" is specified to be used in all locales in the description of the -i option.. I believe that the intent was that this output format and these strings should apply only in the POSIX locale. Is this correct? Action: If the answers to all of my "Is this correct?" questions is "Yes.", the following changes could be applied to XCU6 to make the specification unambiguously describe what I believe was intended. SYNOPSIS Change P442, L17024 (the entire SYNOPSIS section) to: file [-dh][-M file][-m file] file... file -i [-h] file... with both lines shaded with a UP marking. DESCRIPTION Replace P442, L17027-17039 (the entire DESCRIPTION section) with: The file utility shall perform a series of tests in sequence on each specified file in an attempt to classify it: 1. If file does not exist, cannot be read, or its file status could not be determined, the output shall indicate that the file was processed, but that its type could not be determined. 2. If the file is not a regular file, its file type shall be identified. The file types directory, FIFO, socket, block special, and character special shall be identified as such. Other implementation-defined file types may also be identified. If file is a symbolic link, by default the link shall be resolved and file shall test the type of file referenced by the symbolic link. (See the -h and -i options below.) 3. If the length of file is zero, it shall be identified as an empty file. 4. The file utility shall examine an initial segment of file and shall make a guess at identifying its contents based on position sensitive tests. (The answer is not guaranteed to be correct. See the -d, -M and -m options below.) 5. The file utility shall shall examine an initial segment of file and small make a guess at identifying its contents based on context sensitive default system tests. (The answer is not guaranteed to be correct.) 6. The file shall be identified as a data file. OPTIONS Change the description of the -d option on P442, L17044 to: -d Apply any position sensitive default system tests and context sensitive default system tests to the file. This is the default if no -M nor -m option is specified. Change the description of the -i, -M, and -m options, and the interactions between options on P442, L17048-17060 to: -i If a file is a regular file, do not attempt to classify the type of the file further, but identify the file as specified in the STDOUT section." -M file Specify the name of a file containing position sensitive tests that shall be applied to a file in order to classify it (see the EXTENDED DESCRIPTION). No position sensitive default system tests nor context sensitive default system tests shall be applied unless the -d option is also specified. -m file Specify the name of a file containing position sensitive tests that shall be applied to a file in order to classify it (see the EXTENDED DESCRIPTION). If the -m option is specified without specifying the -d option or the -M option, position sensitive default system tests shall be applied after the position sensitive tests specified by the -m option. If the -M option is specified with the -d option, the -m option, or both; or the -m option is specified with the -d option; the concatenation of the position sensitive tests specified by these options shall be applied in the order specified by the appearance of these options. STDOUT Change the text on P443-444, L17089-17115 to: The values for are unspecified, except that in the POSIX locale, if file is identified as one of the types listed in the following table, shall contain (but is not limited to) the corresponding string, unless the file is identified by a position sensitive test specified by a -M or -m option. Each space shown in the strings shall be exactly one . Table 4-8 File Utility Output Strings _____________________________________________________________________________ If file is: shall contain Notes the string: _____________________________________________________________________________ nonexistent cannot open Block special block special 1 Character special character special 1 Directory directory 1 FIFO fifo 1 Socket socket 1 Symbolic link symbolic link to 1 Regular file regular file 1,2 Empty regular file empty 3 regular file that cannot be read cannot open 3 Executable binary executable 4,6 ar archive library (see ar) archive 4,6 Extended cpio format (see pax) cpio archive 4,6 Extended tar format (see ustar in pax) tar archive 4,6 Shell script commands text 5,6 C-language source c program text 5,6 FORTRAN source fortran program text 5,6 regular file whose type cannot be determined data _____________________________________________________________________________ Notes: 1. This is a file type test. 2. This test is applied only if the -i option is specified. 3. This test is applied only if the -i option is not specified. 4. This is a position sensitive default system test. 5. This is a context sensitive default system test. 6. Position sensitive default system tests and context sensitive default system tests are not applied if the -M option is specified unless the -d option is also specified. In the POSIX locale, if file is identified as a symbolic link (see the -h option), the following alternative output format shall be used: "%s: %s %s\n", , , " If the file named by the file operand does not exist, cannot be read, or the type of the file named by the file operand cannot be determined, this shall not be considered an error that affects the exit status. EXTENDED DESCRIPTION Change "one test per line" on P444, L17121 to "one position sensitive test per line". APPLICATION USAGE Add to end of paragraph on P446, L17223-17225: Note also that on systems that recognize shell script files starting with "#!" as executable files, these may be identified as executable binary files rather than as shell scripts. RATIONALE Add new paragraphs after P447, L17248: The IEEE Std 1003.1-2001 description of default system tests and the interaction between the -d, -M, and -m options didn't clearly indicate that there were two types of "default system tests". The "position sensitive tests" determine file types by looking for certain string or binary values at specific offsets in the file being examined. These position sensitive tests were implemented in historical systems using the magic file described above. Some of these tests are now built into the file utility itself on some implementations so the output can provide more detail than can be provided by magic files. For example, a magic file can easily identify a core file on most implementations, but can't name the program file that dropped the core. A magic file could produce output like: /home/dwc/core: ELF 32-bit MSB core file SPARC Version 1 but by building the test into the file utility, you could get output like: /home/dwc/core: ELF 32-bit MSB core file SPARC Version 1, from 'testprog' These extended built-in tests are still to be treated as position sensitive default system tests even if they are not listed in /etc/magic or any other magic file. The context sensitive default system tests were always built into the file utility. These tests looked for language constructs in text files trying to identify shell scripts, C, FORTRAN, and other computer language source files, and even plain text files. With the addition of the -m and -M options the distinction between position sensitive and context sensitive default system tests became important becuase the order of testing is important. The context sensitive system default tests should never be applied before any position sensitive tests even if the -d option is specified before a -m option or -M option due to the high probability that the context sensitive system default tests will incorrectly identify arbitrary text files as text files before position sensitive tests specified by the -m or -M option would be applied to give a more accurate identification. _____________________________________________________________________________ EDITORIAL Enhancement Request Number 5 joannaf@sun.com Defect in XCU vi (rdvk# 4) {Sun-jf-ac-vi01} Wed, 31 Jul 2002 12:04:38 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 997 Line: 38217 Section: vi Problem: Defect code : 2. Omission Count missing from Synopsis. Text on lines 38218-38219 mention "count": "Move the cursor backward through the edit buffer to the first character of the previous section boundary, count times." Action: On page 997, line 38217: change Synopsis: [[ to Synopsis: [count] [[ [Ed recommendation: TC1 candidate] _____________________________________________________________________________ EDITORIAL Enhancement Request Number 6 joannaf@sun.com Defect in XCU vi (rdvk# 3) {Sun-jf-ac-vi02} Wed, 31 Jul 2002 12:08:41 +0100 (BST) _____________________________________________________________________________ Accept_X___ Accept as marked below_____ Duplicate_____ Reject_____ Rationale for rejected or partial changes: _____________________________________________________________________________ Page: 998 Line: 38237 Section: vi Problem: Defect code : 2. Omission Count missing from Synopsis. Text on lines 38238-38239 mention "count": "Move the cursor forward through the edit buffer to the first character of the previous section boundary, count times." Action: On page 998, line 38237: change Synopsis: ]] to Synopsis: [count] ]] [Ed recommendation: TC1 candidate]