A condition of Product Certification is that a product has not only been
designed to be conformant to a Product Standard, but also that
conformance has been tested in practice. This applies regardless of
whether or not a designated test suite (Indicator of Compliance) is
identified in the Product Standard. In all cases the precise
hardware/software environment in which conformance has been established
must be identified in the Conformance Statement, and it must be
sufficiently detailed to enable conformance to be re-demonstrated and
test results reproduced.
Many Product Standards identify, in the Indicator of Compliance
section, that a specific test suite must be used during conformance
testing. In these cases a test report which shows that there were no
unresolved issues or failures (except for those covered by appropriate
Interpretations, Test Suite
Designated test suites are given in generic terms (for example, a
"VSX4 test report"), rather than by identifying a specific version of
each suite and the detailed testing requirements, to avoid the need to
re-issue a Product Standard every time the situation changes. In
practice, specific versions of test suites are mandated at any given
point in time. Conformance Administration can advise, although details
of the current test suites and other criteria are available at
New versions of test suites replace current versions with a six-month overlap period, during which time either will be accepted as the valid Indicator of Compliance.
New test suites will normally become mandatory for Product Certification after a nine-month notice period, and thereafter a test suite report will be required at the first annual renewal of all relevant Product Certifications.
Conformance test suites typically test for the presence or absence of
functionality, and the behavior of functionality when present. The
Conformance Statement is the means by which an applicant declares which
optional functionality is supported, and therefore a test report for a
product should match the functionality claimed for it in the
Conformance Statement. For example, if the Conformance Statement states
that the implementation provides POSIX-2 C-language binding, then the
conformance tests should have verified that the implementation provides
these facilities according to the Technical Standards.
In performing test report audits on Product Certifications, The Open Group carries out extensive checks to determine the consistency of what is claimed for a product, comparing the information in the Conformance Statement against the test suite reports.
Some test suites are made available under license from The Open Group. In other cases-for instance, in the programming language Product Standards where the specifications conform to formal standards-the Product Standard references the test suites that have been developed to support the existing formal certification programs for these languages. Full details of the test suites and related matters can be found at www.opengroup.org/testing.
In support of a Product Certification application, The Open Group
requires, among other things, a formal test report from the supplier in
respect of Product Standards for which an Indicator of Compliance is