Open Group White Paper: Certifying that Applications Work With LDAP

30 March 2000

Introduction

As the directory market matures, customers are thinking less about selection of LDAP servers, and more about selection and deployment of applications. They need reliable information about which applications will work with their servers, and under what circumstances.

At the same time, application vendors need to be able first to ensure that their products will work with all standard servers, and then to prove it.

The Open Group "Works With LDAP" certificate will provide the proof that customers want to see and vendors want to provide.

This White Paper sets out the specific form of certification that The Open Group proposes to implement for "Works With LDAP".

Summary

This White Paper describes

the definition of "Works With LDAP" certification
the actions that must be taken to put the certification process in place
the rationale underlying the definition.

Appendices contain

a draft application form
descriptions of some existing application certification programs, for comparison.

For the form of certification described in this White Paper, an application vendor wishing to obtain a certificate for a product must say

which of the product functions use LDAP
under what circumstances they work with LDAP
what tests have been run to verify that they work with LDAP.

There will be a web application form incorporating a checklist to help vendors make these statements. It will ensure that they consider common circumstances under which products may or may not work with LDAP, and will ensure that the tests are reasonably thorough.

The certificate will be awarded if the vendor makes the application as required by the web form and warrants that the product works with any standard LDAP server. The vendor's warranty will be the essence of the certificate. The tests will be supporting evidence. The test results will not be audited as part of the application process.

If a vendor's claim is found to be incorrect, the vendor will have to fix the product or fix the claim (for example, by changing the statement of the conditions under which the product works). If the vendor does not do so the certificate will be withdrawn.

Operation of the certification process will be funded by registration fees paid by vendors. The process is sufficiently lightweight that these fees can be kept low.

The actions needed to put "Works With LDAP" certification in place are to finalize the legal documents, develop the application form, implement the web site, and raise awareness. In addition, further work is needed to link the certification with server vendors' application certification programs.

The requirement for The Open Group to develop certification for LDAP applications was initially put forward by the Directory Interoperability Forum (DIF), and has been discussed by the DIF and the Open Group Directory Program. The rationale leading to the form of certification chosen as a result of those discussions is given in the "Rationale" section of this White Paper.

"Works With LDAP" Certification

This section looks at how "Works With LDAP" certification will operate from the points of view of customers and vendors of directory-enabled applications. It then describes the underlying legal framework that gives force to the certification.

Customers' Perspective

The "Works With LDAP" web site will list all the products that work with LDAP. For each product, there will be a description of how it uses LDAP, and under what conditions.

The description of the conditions will help customers to understand how easy it will be to integrate the product into their environments. For example, it will say whether there are any access control requirements that their servers might not be able to meet. And it will say whether there are any schema requirements that might conflict with those of other applications.

There will be a logo associated with the certificate. Vendors that have the certificate will be able to use the logo on their registered products.

The use of the logo on a product, and the appearance of the product on the "Works With LDAP" web site, will mean that the customer knows that the product will work with standard LDAP servers.

Vendors' Perspective

There will be a web form that vendors will use to apply for certification. Using the form, they will enter the information about their products that will be displayed to customers on the "Works With LDAP" web site, and will complete a checklist to support their application.

The information that will be displayed to customers is

product name
functions that use LDAP
conditions under which the product works with LDAP.

The form will contain questions and instructions designed to help the vendor provide accurate and useful information to be displayed to customers. It will also require the vendor to say how the product has been tested to verify that it works with LDAP servers.

The level of testing that will be expected will be no more than what any prudent vendor would carry out as a minimum as part of product development. For example, testing of a similar nature to that of BLITS will be perfectly acceptable.

The vendor will complete the form and submit it. The Open Group will review the submitted application, and will discuss any incomplete or unsatisfactory answers with the vendor. The Open Group will not perform testing or auditing of test results.

The Open Group will prepare a legal agreement using the submitted information, and send it to the vendor for signature. When the agreement has been signed, and a registration fee paid, The Open Group will countersign it and place the product information on the "Works With LDAP" web site. The vendor will then be entitled to use the logo in conjunction with the product.

The registration fee will be low, just sufficient to cover administration costs. The target fee is $500.

The main expense for the vendor will be in providing the information and in doing the testing. However, generating this information, and performing the tests, is something that vendors need to do to develop and market products that work with LDAP, regardless of the existence of the certificate. Completing the checklist should in itself be valuable for them.

The agreement may be terminated by the vendor or by The Open Group at any time. The Open Group will do this if, and only if, it is clear that the product in question does not work with LDAP as claimed.

Legal Framework

"Works With LDAP" will be one of a range of Open Group "Works With" certificates.

The legal basis for these certificates is trademark law. The vendor enters into an agreement with The Open Group. Under this agreement they are licensed to use an Open Group trademark in conjunction with their products. As part of the agreement, the licensee represents and warrants that the product meets the applicable Quality Standards set forth in a schedule to the agreement.

That schedule is specific to the particular certificate concerned. The Quality Standards schedule for "Works With LDAP" will be as follows.

The Products interoperate with LDAP servers in order to perform the functions set forth in Schedule X. Each of The Products will correctly perform these functions when interoperating with any LDAP Server that conforms to the Open Group LDAP 2000 Product Standard provided that the environmental conditions set forth in Schedule Y are satisfied.

Schedules X and Y are specific to the products in question. The contents of them are supplied by the vendor on the web form when applying for certification.

Actions Required to Implement "Works With LDAP" Certification

Legal Agreement and Logo Usage Guidelines

The "Works With" agreement and guidelines for the use of the logo have been drafted. The drafts must be reviewed by The Open Group product development and legal departments, and finalized.

Application Form

A preliminary version of the application form is contained in an appendix to this White Paper. This must be developed and completed by the Directory Program Group.

Web Site

The "Works With LDAP" web site must be designed and developed. This will be done by The Open Group IT department, with input from the Program Group.

Awareness

Awareness of "Works With LDAP" certification must be raised among directory application developers. Server vendors, and especially the DIF, can play a major role in this. A marketing Plan must be developed.

Links to Vendors' Programs

An aim of "Works With LDAP" certification is that it should have links to server vendors' application certification programs. In principle, the flexible design of "Works With LDAP" certification should make this possible.

Linkages that might be made include:

Tests performed for vendors' certificates could be quoted as evidence for "Works With LDAP" certification.
The "Works With LDAP" certificate could be accepted as a partial qualification for a vendors' certificate.

Further work, including detailed discussions with server vendors that operate application certification, is needed to make progress in this area.

Rationale for the Definition

Background

At the July 99 Open Group Conference in Montreal, members of the Directory Interoperability Forum (DIF) put forward a strong and clear requirement for The Open Group to design and manage certification schemes for Directories and for Directory-Enabled Applications.

The Open Group responded by defining the LDAP 2000 Open Brand for Directories. Products that conform to the LDAP 2000 Product Standard are eligible for certification under this scheme.

The companion scheme for Directory-Enabled Applications - the "Works With LDAP" scheme - has taken longer to define because there is no widely accepted pre-existing model. At the January 2000 Open Group Conference in San Diego, The Open Group presented a framework for this scheme, and the discussion of how to apply that framework was started.

Following the San Diego meeting, The Open Group circulated two alternative proposals which were discussed by a teleconference of interested parties. As agreed at that teleconference, this White Paper is based on the second of those alternatives.

Value of the Scheme

The value of the scheme is that it

meets the needs of Customers for proof that the products they buy will interoperate
meets the needs of Vendors to provide that proof to Customers
by increasing Customer confidence, enables Vendors to sell more products, earlier.

Requirements

The principal requirements are:

Customers must find certification easy to understand.
Vendors that deserve certification must find it easy to acquire.
It must be inexpensive. The costs will be covered by application vendors' registration fees. These must be kept low.
It should have links to server vendors' programs for certifying applications.

There are a host of applications that can benefit from being directory-enabled. They include e-mail, virtual private networks (VPNs), distributed systems management, human resources management, and others. From the point of view of these applications, directory is not the main purpose - it is a useful tool that helps that purpose to be attained more easily. Certification must be usable and valuable in this context.

Implementation Framework

A framework within which the scheme should be implemented was agreed at the San Diego Open Group Conference in January 2000. This framework covers delivery and legal aspects of the scheme.

The principal means of delivery will be the World-Wide Web. This will provide:

a repository for documents, logos, etc.
a register of certified products
a list of Vendors that are in the program
information for Customers.

The legal basis of the scheme will be as follows. There will be a logo associated with the scheme. The Open Group will own the logo, and will register it as a trademark as widely as is practical. Vendors will be licensed to use the logo on products that are certified under the scheme. The license will require them to "warrant and represent" that the products conform to certain quality standards. The license will provide for the right to use the logo to be withdrawn if a product does not conform.

Quality Standards

The quality standards state what precisely a product must do to be certified. (The quality standards chosen for Works With LDAP are presented in the Legal Framework section of this White Paper.)

There are three aspects to the Quality Standards:

What must the Application Vendors state about their products?
How do the Application Vendors satisfy themselves that their products do what they say?
How do the Application Vendors satisfy The Open Group that their products do what they say?

Criteria for Selecting the Quality Standards

The following criteria, in order of importance, should be used to define the Quality Standards.

Achievability - the certification must be realistic.
Effort from server vendors - certification should not rely on effort from server vendors.
Value to Customers - the more valuable it is to Customers of Directory products, applications and services, the better.
Administration cost - must be covered by the fees charged, and the lower the better.
Effort from application vendors - the less required, the better.

What the Vendor Guarantees

With traditional Open Group certification, including the LDAP 2000 Brand for servers, the vendor guarantees that the product conforms to an independent standard. (In the case of LDAP 2000, this is the LDAP 2000 Product Standard which incorporates by reference IETF RFC 2251 and other published standards.)

The problem with applying this principle to "Works With LDAP" is that there are currently no standards for Directory applications, other than the LDAP protocol standards. The Application Vendor should of course guarantee that the product conforms to these. However, this guarantee will not in practice provide a high degree of assurance that the products will really interoperate with LDAP servers. The reasons for this include:

an application may operate the standard LDAP protocol correctly but also rely on non-standard features
an application may have particular schema requirements that some servers can not meet
an application may assume a particular access-control model that some servers do not support.

The options for defining what the vendor guarantees beyond basic LDAP conformance are as follows.

Independent Standard: Standards for application behavior could be created. This might be possible for a few specific applications (Eg "E-mail client that Works With LDAP") but is not realistic for generic "Works With LDAP" certification..
Vendor-Defined Specification: Each participating vendor could state how their products Work With LDAP. This would require effort on the Vendors' part, but would not necessarily mean significantly higher administration costs. The Vendor statement could include information such as how functionally they use LDAP, what optional features of the LDAP 2000 Brand they rely on, what protocol features not included in LDAP 2000 they use, what schema they require, and what access control model they assume.
Basic LDAP Conformance: The scheme could simply require conformance to the basic LDAP protocol requirements. This would be the easiest option. The question is whether it would give the scheme sufficient value in the eyes of Customers.

The principle of the Vendor-Defined Specification was selected because it gives the certificate more value than Basic LDAP Conformance and is realistic.

How the Vendors Know They Work With LDAP

In traditional Open Group certification, there is normally a test suite. The stated purpose of this is to be an indicator of conformance - that is, to be part of what tells Customers and others that the product conforms. However the test suite also provides an equally valuable - perhaps more valuable - function. It gives the vendor assurance that the product conforms, and that the guarantee that certification requires can safely be given.

The following options might be used to provide this function for "Works With LDAP".

Independent Test Suite: Defining a test suite for applications in general - even to cover only basic protocol requirements - is not possible. Creating independent application-specific tests for all applications that vendors might wish to certify would be impractical.
Vendor-Defined Tests: Each participating vendor could define their own tests to prove that their products Work With LDAP. This would require effort on the Vendors' part, although they would presumably define tests for their products even if not participating in the scheme, and the requirement to document such tests for the scheme need not be too heavy an additional burden. The tests they use could be publicly available via the "Works With LDAP" scheme Web pages. This option would not necessarily mean significantly higher administration costs.
Interoperability Test Bed: Server Vendors could make LDAP-2000 Servers available for interoperability testing by application vendors. This would be a valuable means of assurance for the Application Vendors. It would also be a valuable addition to the Vendor-Defined Tests option as it would provide a neutral platform to run such tests on.
DirConnects: While not maintaining a permanent test bed, Server Vendors could offer Application Vendors the opportunity to test against their products at regular DirConnect interoperability testing events. Again, such testing should not be required, but should be encouraged.
Use of a Standard SDK: It would be possible to define standard tests - and even a certification scheme - for LDAP Client Software Development Kits (SDKs). Use of a certified or tested SDK would at least give the Application Vendor assurance that the product conformed to basic protocol requirements. This would be a simple option but would exclude vendors that write their own LDAP-handling software. It would not require high administration costs per se, but would mean establishing a separate scheme for SDKs, thus introducing an extra dimension of complexity.
Source Checker: If use of a standard SDK is a criterion, the Open Group Source Checker might be used to verify that the SDK is used correctly and that no APIs outside the SDK are used by the application to communicate with the LDAP server.
No Assurance: It would be possible to operate the scheme even if it did not include any way by which vendors could check product conformance. This would be the easiest option. It would however result in lower numbers of certified products or lower quality of certified products (or perhaps both).

Defining an independent test suite was rejected as impractical. Making an Interoperability Test Bed or DirConnects a formal part of the scheme would require significant effort from the server vendors, and probably a legal agreement between them and The Open Group, and were both therefore rejected. (As a separate exercise, vendors should be encouraged to participate in such a test bed and in DirConnects.) Making the Source Checker usable for all applications (Java as well as C, for example) would not be practical.

The principle of Vendor-Defined Tests was selected because, of the remaining options, it gives the highest value while being practical to implement.

How The Open Group Knows the Vendors Work With LDAP

Evidence of the Application Vendors' claims could be given in various ways.

Independent Testing: Testing could be performed by The Open Group or by an independent third party. This would provide a high degree of assurance but would have very high administration costs.
Independent Audit: Testing or other procedures performed by the Application Vendors to provide assurance that their products Work With LDAP could be audited by The Open Group or by an independent third party. This would be less expensive than independent testing and could provide almost the same degree of assurance. It would however impact the registration fee to some extent - perhaps by around $1K per registration.
Spot Checks: Rather than auditing every registration, there could be spot-checks on (say) one in ten, chosen at random. One tenth of the cost of auditing, but a lower degree of assurance.
Vendor Statement: The Application Vendors could provide statements of what tests or other procedures they had carried out. This would mean some work for the Application Vendors, but no impact on the registration fee. The quality of these statements would probably be highly variable, but could be made clearly visible to the Customers.
Vendor Assurance: The Application Vendors could simply give the assurance that their products Work With LDAP, without giving any further information to back this claim. This would be simplest and cheapest but would give the scheme little added value - most vendors will presumably make such claims in any case.

Independent testing and independent audit were both rejected as giving too high a cost of administration. Vendor assurance does not give sufficient value.

The Vendor Statement principle was selected.

It would be possible to add Spot Checks to this, but it is doubtful whether the additional assurance gained would be worth the added cost and complication.