COE Platform Certification Frequently Asked Questions
June 6 2003
Issues during Certification
We started certification testing for a product and while testing was taking place, we found there are product problems requiring some bug-fixing modifications. Do we have to rerun the entire tests in the campaign?
The COE Platform Certification program is a voluntary program of the The Open Group, open to any product meeting the conformance requirements.
COE Platform Certification is a formal process built around the COE Platform Certification Policy document and The Open Brand Trademark License Agreement. Suppliers of certified products, warrant and represent that the product meets all the conformance requirements applicable to the class of COE Platform Certification being certified.
The key documents describing the obligations, terms and conditions of COE Platform certification are:
the COE Platform Certification Policy
the Open Brand Trademark License Agreement
the COE Platform Certification Agreement
The technical requirements for a conforming system are described in detail in the Product Standards.
Once a supplier has achieved certification for a product, they are permitted to use the Open Brand trademark in connection with that product.
COE Platform certification is widely supported within the DOD community and certification is widely accepted by government and other major procurers of IT @ T devices. To compete with the major suppliers whose products are certified you may wish to consult your intended customers to ascertain their needs. Given the low cost of certification after you have built a product that meets the specifications and passes the tests, we believe the return on investment is substantial.
To become familiar with the COE Platform Certification program, you should read the following program documents:
The COE Platform Certification Policy
This document should be read first. It is the foundation of the program, providing information on the types of products that can be certified in the program, what it means to be certified, what is required to get a product certified and how to make sure that a product remains certified.
The COE Platform Trademark License Agreement
The COE Platform Trademark License Agreement is between you and the The Open Group and requires signature. It is recommended that you commence the process to complete the Trademark License Agreement as soon as possible to save delays later in the certification process. A trademark license must be completed before a product can complete the certification process and be entered onto the Certification Register.
The Trademark License agreement only needs to be entered into once per organization. When it is in place multiple products can be certified.
The COE Platform Product Standards
The COE Platform Product Standards should then be read to understand the detailed conformance requirements against which a product can be certified. Product Standards provide a mapping between certification, the specifications required by COE and the test suites needed to demonstrate conformance.
The COE Platform Certification Agreement
The COE Platform Certification Agreement is between you and the Certification Authority. It defines the certification service and the legal commitment to the conditions of the service. It applies to each product being certified.
The Guide to COE Platform Certification
This document is a guide for suppliers who want to certify products and provides detailed information on the steps you are required to take to achieve certification including:
Understanding the certification program and process
Testing the product informally
Applying for certification
Formal testing and submission of results
The list of test suites currently authorized for use in the COE Platform Certification program, including information on where they can be obtained, is available here. The exact test suites required depends on the test campaign selected, for example you may choose at your option to reference existing certifications to meet the requirements of the COE Platform Standards Product Standard. Please see the Guide to COE Platform Certification for information about the test campaign definition form which must be completed as part of the initial submission information.
The Open Group offers support contracts for all its test suites referenced in the program, which includes email support provided through the web at http://www.opengroup.org/testing/support/ .
The COE Platform policy allows for an existing certified product to be renamed without the need for further certification or testing. You will be required to provide a written statement to the CA indicating that there have been no material changes to the certified product. If there are changes then it depends on the nature of the changes, see the COE Platform Certification Policy, Section 7.
Certification may take place at any stage of the product's life cycle. The product declared on the register needs to be commercially available for procurement. The software version of the product should be clearly identifiable. If the R&D name and version is very different from the commercial release then we have to be assured that it is indeed a maintenance release. Please communicate with the Certification Authority regarding any uncertainties. We will respond to you regarding the scope of testing if, for any reason, the situation is not clear. Please refer to the COE Platform Certification Policy, Section 7 on Scope, Re-Test and Re-Certification covers the requirements, particularly the section for maintenance releases.
Timescales and preparations for formal testing depend on each situation, so it is not possible for us to provide an estimate. From the time we receive your initial submission,including conformance statement and test campaign definition form, the normal timeframe is up to 10 working days for the test campaign definition to be approved. Once you have completed your test campaign and submitted full test results, the normal timeframe is up to 25 working days for certification to be confirmed. This assumes there are no queries or issues.
This depends on the test campaign required. For example if you require full testing of all the items in the COE Platform Standards Product Standard rather than referencing existing certificates there are aproximately 18000 automated tests, and thirteen sets of manual validation procedures (which total approximately 1000 pages).
The automated test runs typically take 3 days to run. The manual validation procedures typically take up to 10 days to complete.
Once you have tested and your results have been approved as conforming to the current COE Platform Specification, you will be certified for a period of three years. If you are given permission to OEM or Re-Badge another companies product then your product will have the same renewal date as the original certified product on which it is based.
COE Platform Certification Authority
The Open Group
Apex Plaza, Forbury Road
Reading, Berks RG1 1AX
Tel: +44 0118 950 8311
Fax: +1 415 276 3760
Email: COE Platform Certification Authory
Any questions about the TMLA should be addressed to: the COE Platform Cerification Authority as above.
Please see the fee schedule by clicking here.
Issues during Certification
To access and submit a Conformance statement and a COE Platform Certification Agreement, please go to The Open Group Web for COE Platform Certification at http://www.opengroup.org/openbrand/coe/. You are able to view the COE Platform Certification Agreement at this URL and all other documents required for COE Platform Certification.
Please check the Problem Report Database first, which can be accessed from the test suite web pages. If your fail results are covered by and agreed test suite deficiencies (TSD's) or agreed interpretations for the version of the test suite used, then certification may be able to proceed. If you have submitted your test results for certification and you have unresolved fail results, your certification fee covers two attempts to certify.
If you believe there is a bug in the test suite, or the Certification system, or a specification, please raise a Problem Report at http://www.opengroup.org/openbrand/PR/ giving enough detail for the problem to be reproduced. This will enable any fault with the test suite and the particular version to be dealt with satisfactorily. If you are having difficulty running tests or operating the test suite, and these are maintained by The Open Group please report the problem to the Test Suite Maintenance Authority (see http://www.opengroup.org/testing/support/)
The COE Platform Certification Authority does not provide support for the test suite and is unable to provide help directly in debugging products. Please contact the Test Suite Maintenance Authority (as noted above), please provide sufficient information for then to determine what the problem is ( for example, specific syntax or logic error in the content and the journal output).
For certification we need clear runs of the test suite. If your product requires bug fixing during the test cycle, the fixes should be made and the product re-tested from the beginning if you want to certify the latest release of your product. After certification, if more product changes are required, the directions in Section 7 of the COE Platform Certification Policy apply. This describes the scope of re-testing already-certified products.
For FIP results (further information provided) you need to sign off that the manual resolution of the test is a pass result.
An explanation of the result codes from the automated test suites which use TET follows below:
PASS - a test result belonging to this group is considered to be a pass for certification purposes and does not need reference to any granted interpretations or waivers:Pass - the test has been executed correctly and to completion without any kind of problemWarning - the functionality is acceptable, but you should be aware that later revisions of the relevant standards or specification may change the requirements in this area.FIP - additional information is provided which needs to be checked manually.Unsupported - an optional feature is not available or not supported in the implementation under test.Not in Use - some tests may not be required in certain test modes or when an interface can be implemented by a macro or function and there are two versions of the test only one is used.Untested - no test written to check a particular feature or an optional facility needed to perform a test is not available on the system.
FAIL - a test result belonging to this group is considered to be a fail for certification purposes (unless the failure has been waived by an agreed interpretation or waiver in the Interpretations database):Fail - the interface did not behave as expected.Uninitiated - the particular test in question did not start to execute.Unresolved - the test started but did not reach the point where the test was able to report success or failure.Unreported - a major error occurred during the testset execution.
If the bug is preventing you from certifying a product it should be reported into the COE Platform Certification Authority Problem Reporting system at http://www.opengroup.org/openbrand/PR/ . From the Problem Reporting entry page select "Search/Submit", and then on the next screen "Submit an Interpretation (right hand column)". You are then presented with a screen where you must select the name of the Product Standard -- please select the lowest level product standard, for example if you are having a problem with the NFS Manual Validation Procedure select the COE Platform Standards Product Standard. When entering problems against the Manual Validation Procedures, select COE-VP as the name of the test suite, and give the chapter name, and version number, as well as the procedure reference (for example, A.1.1).
If the bug does not concern COE Platform Certification, it should be reported into the test suite support team directly at http://www.opengroup.org/testing/support/
Please check that the status of the Problem Report is either a test suite deficiency (TSD), certification system deficiency (CSD) or an interpretation (PIN or TIN). REJ means it was not agreed and therefore the PR may not be referenced. Please also check that the PR applies to the certification release of the test suite, which you are using. Please note that TSDs are test suite specific.
You can track problem report status on the Open Brand PR web site at http://www.opengroup.org/openbrand/PR/