The XPG4 Core Operating System Calls and Libraries Test Suite
- Verifies the core operating system calls and libraries.
- Built upon the proven VSXgen harness, yielding
a configurable and productive tool.
- Using the VSXgen architecture, this suite is provided as a VSXgen
Testset Package.
- Over 10,000 tests.
- Test Coverage: IEEE POSIX 1003.1-1990 (ISO 9945-1:1990), FIPS 151-2,
XPG3, XPG4, UNIX 98.
- Test Coverage also available for POSIX 1003.1,2003 and UNIX 03 in the Westwood version.
- Mandatory for testing XPG4 Base, XPG4 Base 95 and the Single UNIX®
Specification (including Versions 1, 2 and 3).
-
An embedded test capability is also available in the form
of the derivative VSX4-PSE test suite
-
Derivative test suites providing a subset of the full VSX4 test suite are:
VSX-PCTS1999, VSX-PCTS2003 and the LSB-VSX.
This test suite is one of the most widely used test suites in the industry.
If your organization is developing an operating system supporting at least
POSIX.1 then this test suite gives significant test coverage and is recommended
for in-house quality-assurance as well as part of your branding campaign.
Top
VSX4 is the descendant of X/Open Company's first test suite, VSX2. Developed
continuously since 1988 to test the System Call interfaces, VSX4 is a
mature and dependable test suite that verifies the core operating system
calls and libraries.
It comprises over 10,000 tests (approx 750,000 lines of source code),
giving coverage of the IEEE POSIX 2003.1 test assertions as well as providing
extensive tests for the X/Open extensions beyond POSIX, including internationalization,
and header file namespace tests. It also includes tests for traditional
``Common Usage'' C language compiler syntax and tests for the IEEE POSIX
1003.2 C Language Binding interfaces. VSX4 is mandatory for testing XPG4
Base, XPG4 Base 95, UNIX 95, UNIX 98 and UNIX 03.
It has the following test modes:
- IEEE POSIX 1003.1-1990 (also known as ISO/IEC 9945-1)
- X/Open Portability Guide Issue 3 (XPG3)
- X/Open Portability Guide Issue 4 (XPG4)
- UNIX 98
Additionally the Westwood version provides:
- IEEE POSIX 1003.1, 2003 Edition(also known as ISO/IEC 9945-1:2003)
- UNIX 03
In POSIX mode VSX4 can be run as a POSIX Conformance Test Suite (PCTS),
and as such is the recognized certification test tool in Europe. For the
Open Brand, the test suite is run in one of the X/Open modes (XPG3 , XPG4
, UNIX 98 or UNIX 03) which are supersets of the POSIX test mode (the X/Open XPG and UNIX Specifications defer to POSIX.1).
An enhanced version known as VSX4 +XNFS is available for testing the
Network File System (NFS) specification, including tests for system interfaces
over NFS, as well as the NFS protocol itself.
VSX4 uses the robust VSXgen test harness, which in turn uses the industry
standard Test Environment Toolkit (TET) . The benefits of using VSXgen,
which is derived from an earlier version of VSX4, is that the harness
has proven portability and robustness characteristics and it allows the
VSX4 tests to be integrated with other testsets that run under VSXgen,
making the test suites easily useable for in-house quality assurance and
automated regresssion testing.
The TET is described on the TETworks
web site.
Top
Disk space requirements on a typical CISC based system are as follows:
- Source code - 25 Mbytes
- Source and tests built - 100 Mbytes (on a CISC system)
Top
- POSIX.1-1990, POSIX.1-2003 mode - over 6000 tests
- XPG4/UNIX 98/UNIX 03 Mode - over 10,000 tests
A typical test run from start to finish takes approximately 15 hours on
an Intel Pentium based system.
Top
The VSX test suite verifies compliance of systems to The Open Group
Base Specifications for core kernel and library
interfaces. VSX features include:
- Complete coverage of the IEEE POSIX 2003.1 test assertions.
- Extensive tests for the XSI extensions over POSIX, including internationalisation,
and header file namespace tests.
- Extensive tests for traditional C language compiler syntax (K&R C).
- Test coverage for the IEEE POSIX 1003.2 (1003.1-2003) C Language Binding interfaces.
The source tree and the executable tree are organised in a hierarchical
design, being divided into sections corresponding approximately but not
exactly to boundaries between various standards that is POSIX, ANSI C
[ISO C], and X/Open. The testsets may be classified as follows:
- System Interface Tests
- Header File Tests
- Commands Tests
- Programming Language Tests
Top
VSX4 is modular, both at the implementation level and at the user level.
The testing process is divided into clear stages -
Each stage is clearly documented in the VSX User and Installation Guide,
which accompanies the product.
This stage includes setting up user-ids and groups for testing, and
reading the tape. This stage includes configuring and installing the TET
harness, which is used to control building, execution and cleanup of VSX
tests.
VSX provides a configuration script that interrogates the user and the
system, checking for availability of certain resources. If these resources
are not present then stub routines are supplied. This enables a non-compliant
system to be tested.
After configuration, the installation stage builds the test harness
and associated libraries.
VSX separates the building of the test harness and tools from the testsets
themselves. The test suite may be built completely or in sub-sections, which
decrease in granularity from test sections and areas down to individual
testsets.
Once built, the test suite may be run as a whole in one pass, or in sub-sections
which decrease in granularity from test areas down to individual tests within
testsets. If you are producing an X/Open branding application the test suite
must run in a single uninterrupted pass.
VSX generates an intermediate level journal file in a TET compliant
format. This is sufficient for debugging purposes. When a formal report
is required, such as for X/Open branding then the formal report writers
provided with VSX4 are used to process the journal file.
The source tree and the executable tree are organised in a hierarchical
design, being divided into sections corresponding approximately but not
exactly to boundaries between standards. There are ten sections, the name
suffix giving an indication of the class of tests contained therein. Within
the test section, there is a further subdivision into areas, which then
contain related testsets.
For example the section that tests the IEEE POSIX assertions is denoted
POSIX.os.
The testsets may be classified as follows:
- System Interface Tests
- Header File Tests
- Commands Tests
- Programming Language Tests
For each of the classifications the sections of the test suite are described
together with a list of testsets for each section. Testsets denoted _X are
for testing X/Open extensions beyond POSIX 1003.1-1990. VSX4 contains tests
for macro versions of interfaces. In this case the subsection begins with
a capital M, for example: POSIX.os/MCsupport/tzset tests the macro version
of tzset.
System Interface Tests
The sections to test system call and library interfaces have the
suffix os.
There are four os sections within VSX4:
- POSIX.os
- XOPEN.os
- ANSI.os
- XPG4.os
These are described in detail in the following sections.
POSIX.os
This includes all the POSIX 2003.1 assertions excluding the majority
of the P2003.1 chapter Language-Specific Services for the C Programming
Language assertions which are located in the ANSI.os section.
There are additional assertions that test X/Open extensions to POSIX
(denoted by the suffix _X) and these are automatically excluded if
POSIX only testing is required.
The POSIX tests are divided into the following areas:
- POSIX.os/Csupport
This contains some of the assertions in the P2003.1 chapter
Language-Specific Services for the C Programming Language.
Assertions which are in both in this chapter and the ISO C standard
are included in the ANSI.os section. The testsets in this
area follow:
| fdopen
| fileno
| siglongjmp
| sigsetjmp
| tzset
|
| tzset_X
|
- POSIX.os/dataform
This contains the assertions in the P2003.1 chapter, Data
Interchange Format, which includes tests for the following
functions:
- POSIX.os/devclass
This contains the assertions in the P2003.1 chapter, Device-
and Class-Specific Functions which includes the following
testsets.
| c_cc
| c_cflag
| c_iflag
| c_iflag_X
| c_lflag
|
| c_oflag
| c_oflag_X
| cfgetispee
| cfgetospee
| cfsetispee
|
| cfsetospee
| i_access
| i_canon
| i_close
| i_ctty
|
| i_inproc
| i_modem
| i_noncanon
| i_spchars
| tcdrain
|
| tcflow
| tcflush
| tcgetattr
| tcgetpgrp
| tcsendbrea
|
| tcsetattr
| tcsetpgrp
|
- POSIX.os/files
This contains the assertions in the P2003.1 chapter, Files
and Directories which includes the following testsets:
| access
| access_X
| chdir
| chmod
| chown
|
| chown_X
| closedir
| creat
| creat_X
| fpathconf
|
| fstat
| getcwd
| link
| mkdir
| mkfifo
|
| open
| open_X
| opendir
| pathconf
| readdir
|
| rename
| rename_X
| rewinddir
| rmdir
| stat
|
| stat_X
| umask
| unlink
| unlink_X
| utime
|
- POSIX.os/ioprim
This contains the assertions in the P2003.1 chapter, Input
and Output Primitives, which includes the following testsets
:
| close
| dup
| dup2
| fcntl
| fcntl_X
|
| lseek
| pipe
| read
| write
|
- POSIX.os/procenv
This contains the assertions in the P2003.1 chapter, Process
Environment, which includes the following testsets:
| ctermid
| getegid
| getenv
| geteuid
| getgid
|
| getgroups
| getlogin
| getpgrp
| getpid
| getppid
|
| getuid
| isatty
| isatty_X
| setgid
| setpgid
|
| setsid
| setuid
| sysconf
| sysconf_X
| time
|
| times
| ttyname
| uname
|
- POSIX.os/procprim
This contains the assertions in the P2003.1 chapter, Process
Primitives, which includes the following testsets:
| _exit
| alarm
| exec
| exec_X
| fork
|
| fork_X
| kill
| kill_X
| pause
| sigaction
|
| sigaddset
| sigconcept
| sigdelset
| sigemptyse
| sigfillset
|
| sigismembe
| sigpending
| sigprocmas
| sigsuspend
| sleep
|
| wait
| waitpid
|
- POSIX.os/sysdb
This contains the assertions in the P2003.1 chapter, System
Databases, which includes tests for the following functions:
| getgrgid
| getgrnam
| getpwnam
| getpwuid
|
ANSI.os
This includes all the C Languagage specific P2003.1 assertions plus
many additional assertions based on the text of the XPG and ISO C
(was ANSI C).
The ANSI tests are divided into the following areas:
- ANSI.os/charhandle
This area contains tests for character handling routines and
has the following testsets:
| isalnum
| isalnum_X
| isalpha
| isalpha_X
| iscntrl
|
| iscntrl_X
| isdigit
| isgraph
| isgraph_X
| islower
|
| islower_X
| isprint
| isprint_X
| ispunct
| ispunct_X
|
| isspace
| isspace_X
| isupper
| isupper_X
| isxdigit
|
| tolower
| tolower_X
| toupper
| toupper_X
|
- ANSI.os/diagnostics
This contains a single testset.
assert
- ANSI.os/genuts
This sections contains tests for general functionality and has
the following testsets:
| abort
| abs
| atof
| atoi
| atol
|
| bsearch
| calloc
| exit
| free
| malloc
|
| qsort
| rand
| realloc
| srand
| strtod_X
|
| strtol_X
| system_X
|
- ANSI.os/jump
This section contains "jmp" related functionality.
- ANSI.os/locale
This section contains a testset for the setlocale function.
setlocale
- ANSI.os/maths
This contains testsets for the mathematical functions.
| acos
| asin
| atan
| atan2
| ceil
|
| cos
| cosh
| exp
| fabs
| floor
|
| fmod
| frexp
| ldexp
| log
| log10
|
| modf
| pow
| sin
| sinh
| sqrt
|
| tan
| tanh
|
- ANSI.os/signal
signal_X
- ANSI.os/streamio
This contains testsets for the stdio-streams related functionality.
| clearerr
| fclose
| feof
| ferror
| fflush
|
| fgets
| fopen
| fopen_X
| fputs
| fread
|
| freopen
| freopen_X
| fseek
| ftell
| fwrite
|
| getc
| gets
| perror
| printf
| printf_X
|
| putc
| puts
| remove
| remove_X
| rewind
|
| scanf
| scanf_X
| setbuf
| setvbuf_X
| tmpfile
|
| tmpnam
| ungetc
| vprintf
| vprintf_X
|
- ANSI.os/string
This contains testsets for string related functionality.
| memchr_X
| memcmp_X
| memcpy_X
| memset_X
| strcat
|
| strchr
| strcmp
| strcoll_X
| strcpy
| strcspn
|
| strerror_X
| strlen
| strncat
| strncmp
| strncpy
|
| strpbrk
| strrchr
| strspn
| strstr
| strtok
|
| strxfrm_X
|
- ANSI.os/time
This contains testsets for time related functionality.
| asctime
| ctime
| ctime_X
| gmtime
| localtim_X
|
| localtime
| mktime
| mktime_X
| strftime
| strftime_X
|
XOPEN.os
This includes interfaces that are not contained in the POSIX or
ANSI standards. For example, popen, ftw, getpass, getopt. Where groups
of functions have similar functionality to areas within POSIX.os
and ANSI.os, then the areas have been given a similar name,
e.g. XOPEN.os/files. This section is for XPG3 based funtionality,
there also exists an XPG4.os.
- XOPEN.os/charhandle
| _tolower
| _toupper
| isascii
| toascii
|
- XOPEN.os/files
| dev_null
| dev_tty
| fsync
| seekdir
| telldir
|
- XOPEN.os/genuts
| crypt
| drand48
| encrypt
| ftw
| getopt
|
| getpass
| hsearch
| lsearch
| pclose
| popen
|
| regexp
| setkey
| swab
| tsearch
|
- XOPEN.os/locale
| catclose
| catgets
| catopen
| nl_langinf
|
- XOPEN.os/maths
| erf
| gamma
| hypot
| isnan
| j0
|
| lgamma
| y0
|
- XOPEN.os/procenv
| chroot
| cuserid
| nice
| putenv
| ulimit
|
- XOPEN.os/streamio
- XOPEN.os/string
memccpy
- XOPEN.os/time
clock
XPG4.os
This section contains tests for XPG4 specific features. This section
is not run in XPG3 mode.
The XPG4 specific system interface tests are divided into the following
areas:
- XPG4.os/genuts
| atexit
| div
| fnmatch
| getopt
| glob
|
| labs
| ldiv
| regcomp
| strfmon
| strtoul
|
| wordexp
|
- XPG4.os/ipc
| msgctl
| msgget
| msgrcv
| msgsnd
| semctl
|
| semget
| semop
| shmat
| shmctl
| shmdt
|
| shmget
|
- XPG4.os/locale
localeconv
- XPG4.os/procenv
confstr
- XPG4.os/signal
raise
- XPG4.os/streamio
- XPG4.os/string
| mblen
| mbstowcs
| mbtowc
| memmove
| wcstombs
|
| wctomb
|
- XPG4.os/time
- XPG4.os/wcharhandle
| iswalnum
| iswalpha
| iswcntrl
| iswctype
| iswdigit
|
| iswgraph
| iswlower
| iswprint
| iswpunct
| iswspace
|
| iswupper
| iswxdigit
| towlower
| towupper
| wctype
|
- XPG4.os/wgenuts
| iconv
| iconv_clos
| iconv_open
| wcstod
| wcstol
|
| wcstoul
|
- XPG4.os/wprintpos
- XPG4.os/wstreamio
| fgetws
| fputws
| getwc
| putwc
| ungetwc
|
- XPG4.os/wstring
| wcscat
| wcschr
| wcscmp
| wcscoll
| wcscpy
|
| wcscspn
| wcslen
| wcsncat
| wcsncmp
| wcsncpy
|
| wcspbrk
| wcsrchr
| wcsspn
| wcstok
| wcswcs
|
| wcsxfrm
|
- XPG4.os/wtime
Header file tests.
The sections that test the header files have the suffix hdr.
There are four header file test sections, which are organised in
a similar manner to the os tests, except that each section
may test the same header file to check for definitions mandated by
the different standards.
The sections are:
- POSIX.hdr
This contains testsets for the POSIX header files.
| dirent
| dirent_X
| errno
| errno_X
| fcntl
|
| fcntl_X
| general
| grp
| limits
| limits_X
|
| pwd
| signal
| signal_X
| stat
| stat_X
|
| tar
| termios
| termios_X
| times
| types
|
| unistd_1
| unistd_2
| unistd_3
| unistd_X
| utime
|
| utsname
| wait
|
- ANSI.hdr
This contains testsets for the ANSI header files.
| assert
| ctype
| ctype_X
| locale
| locale_X
|
| math
| math_X
| setjmp
| stdio_1
| stdio_2
|
| stdio_X
| stdlib
| stdlib_X
| string
| string_X
|
| time
| time_X
|
- XOPEN.hdr
This contains testsets for header files that are defined by
XPG3 only.
| cpio
| ftw
| langinfo
| nl_types
| regexp
|
| search
| ulimit
| varargs
|
- XPG4.hdr
This section contains tests for XPG4 header files. This test
section is not run in XPG3 mode.
The header files are:
| float
| fnmatch
| glob
| iconv
| ipc
|
| monetary
| msg
| regex
| sem
| shm
|
| stdarg
| stddef
| wchar
| wordexp
|
Commands Tests
The sections that test command interfaces have the suffix cmd. Presently
only a single command is tested.
XOPEN.cmd
This section tests the gencat command.
Programming Language Tests
The sections for language specific tests have the prefix lang.
Presently there is only one language tested, the "C" language.
lang.C
This section contains the C language syntax tests (Volume 4 of XPG3).
This does not test for ISO C syntax but Common Usage C (K&R).
There are no tests for ISO C, since X/Open recognises NIST validated
ISO C test suites.
Top
Top
|
