Legacy Evolution to SOA – Case Study: Large Telecommunications Company
This case study provides a historical view of a journey that an enterprise has taken to evolve legacy applications to SOA. The objectives of the case study are to:
- Provide a sample of how enterprises are evolving their legacy application to SOA
- Identify gaps and learn from their journey to establish guidelines and best practices
- Describe the viewpoint on the current state for the evolution
- Provide recommendations to improve the journey by adapting the approach and best practices outlined above
This initiative was started with a pilot in 2008 and the first project in the second half of 2009. The objectives of this initiative comprise long-term improvements of the application landscape, particularly better agility, improved data quality, improved end-user experience (e.g., SSO), and sustainable cost reduction. It has selected Service-Oriented Architecture (SOA) as the underlying architecture framework to build upon. The telecommunications organization that is the subject of this case study selected a particular vendor's products as their strategic platform, including operations environment, database platform, and integration platform. In parallel to the decision on the platform product, mandatory architecture guidelines were also documented.
Note that this was a discrete, application-focused project with the intention of incrementally adopting some fundamental capabilities of SOA for rapid business benefits.
High operating costs:
- The legacy environment consists of more than 40 applications. Most of the applications, including the application architecture and underlying technology infrastructures, are out-of-date.
- The same functionality may have to be implemented in more than one application, creating redundancy.
- Data is shared between applications mostly through replication, creating redundancy and the risk of non-actual data.
- Data is inconsistent since one data entity can be modified in more than one application.
- Business logic, including functions and rules, is “hard-coded” into applications and difficult to maintain and re-use.
- All integrations are point-to-point, creating a web of direct dependencies.
- Those redundancies and dependencies in the landscape make changes difficult, risky, and costly.
- As the result of mobile technology evolution, the mobile network must adjust to the new technology generation; for example, the next mobile generation Long-Term Evolution (LTE). This requires frequent changes in the existing applications and new development.
- Because of the IT constraints mentioned above, changes and new development became increasingly costly to the business.
Poor Business Process Management (BPM) support:
- Business processes are mostly not automated as such. Changes in process would lead to changes in code.
Poor user experience:
- There is no SSO support. The user has to log on to different applications using different credentials, in order to perform one task.
The SOA initiative was managed as a program consisting of many projects. Examples of two of the core projects are:
- Project I: SOA Framework: The major goal of this project is the delivery of preparatory work that is necessary before business-driven IT projects can start. The preparatory work comprises SOA infrastructure (web services framework, registry, security, keys, development environment, and monitoring). It is crucial that the results of this project can be used by the organization and external vendors to deliver results in the future. Furthermore, this project must take into consideration that the current IT landscape of the organization is heterogeneous. Current applications are building upon a number of different types of technologies. This fact implies that this project must deliver results that are equally useful for all those technologies. For example, this project must enable third-party developers to deliver web services that can be consumed by applications that are delivered by other vendors.
- Project II: Address Management: The proposed project delivers a new database for storing geometry data, subsequently referred to as a GEO database. This new GEO database will be the central master for storing objects related to geographic locations including addresses. The GEO database will be “spatially enabled” which means that it supports spatial data types and spatial functions as specified by the Open Geospatial Consortium, Inc. (OGC) and ISO. The main advantage of this new technology is the support of complex and time-consuming spatial operations. Comparable technologies are currently not in the field and are unsupported by the current database technology used. Additionally, the proposed project delivers a set of re-usable SOA services to access the objects which are stored in the GEO database or for the validation of addresses.
This section illustrates the high-level migration plan, as well as the Transition Architectures communicated with relevant stakeholders.
The simplified Baseline Architecture is shown below. The address management system is built on a legacy database. It consists of nine distributed, redundant databases. The changes that occurred in one database are replicated through a central hub database. The client applications access the database through proprietary APIs.
The first work package aims to migrate data from the legacy database to the targeted new database system (see Step 1: Data Migration).
Two important activities during the data migration are:
- Implementing New Information Model: The data model is redesigned according to the (new developed) information model NSS SID. This model is defined based on TM Forum Shared Information and Data (SID), which provides the telecommunications industry with a common vocabulary and set of information and data definitions and relationships used in the definition of NGOSS architectures. In addition, the company has identified the need to further extend the SID model, so-called “SID refinement levels”. The extensions include additional attributes and references to the existing data entities, as well as new data entities.
- Data Validation and Data Cleansing: The migrated data will be validated using predefined rules. This ensures the data quality for the new start.
In the second step, the new database becomes the master of address data. This is achieved by the following activities:
- All write access to the legacy address database is disabled; e.g., application “Start” (see Step 2a: Shutdown of Write Access). This is done by database-level security.
- The application Start is “re-interfaced” with the new capability to make changes through web service calls (see Step 2b: Data Replication).
- Address Data Service (“BasicAddressService”) provides CRUD operations that access the new data sources.
- The change data is synchronized to the legacy database through database-level replication.
In the next step, all other applications are migrated one by one to access the new data source. Each migration contains the following activities:
- First, the access to the legacy data resource is shut down (see Step 3a: Migration of One Read-Only Application).
- The application is then “re-interfaced” to retrieve data by consuming BasicAddressService (see Step 3b. Migration of the Remaining Applications).
Finally, after all applications are switched to new data source, the legacy database will be retired.
The following sections describe how the applications are “re-interfaced”.
Consumer Layered Architecture shows high-level layers of a web service consumer:
- The legacy application could be START or STOV that doesn’t have built-in web service capability.
- The web service adapter is a component that exposes remote web service provider functionalities to legacy applications as if the provider is local to the legacy application. The main activities of an adapter involve protocol transformation and data transformation. It can be further decomposed into three sub-systems: WS wrapper, WS proxy, and message post-processing.
WS wrapper provides the following functionalities:
- Data Transformation: The input parameters will be transformed into NSS SID entity object instances. The result will be transformed back to the required format. In most cases, this part should be implemented programmatically. If the transformation logic is not stable, both proprietary data type and SID data type must be first transformed into XML documents. The transformation logic would be implemented through XSLT programming.
- Endpoint Look-up: The provider endpoint URL can be looked up in the UDDI Registry through direct routing.
- Invocation of operations provided by web service proxy.
- WS proxy implements the WS*-Standard. It is sometimes also referred to as the Service Endpoint Interface (SEI). Basically, it contains the standard web sub-systems. In case of contract first approach, WS proxy will be generated by the WSDL compiler.
Message post-processing/SOAP interceptor: The post-processing refers to interceptors that can process SOAP before it is sent to the provider. The following standard interceptors are implemented:
- Security: Enables Kerberos-based SSO; enables web service security using SAML token.
- Logging: It logs the messages in the central log store for debugging purposes.
- Message ID: It adds a unique message ID to the SOAP header.
Client-Side Interactions depicts possible interactions between all consumer site parties.
“Re-Interface” on Consumer Legacy Applications: General Approach: Technology Platform-Specific Scenarios
This section demonstrates how the web service framework fits in the required (legacy) technology stacks (a non-Java and a Java stack).
START as Web Service Consumer shows an integration scenario with application START. The web service adapter will be implemented as .NET components/assemblies, since the START is built with a language version that does not support WS*. START will utilize a COM object that bridges the procedure call from START to web service adapter. In addition, START will provide an additional layer as controller that manages distributed transactions.
STOV is a Java swing application that currently manages direct read access to the database though Data Transfer Objects (DTO) and Data Access Objects (DAO). Instead of implemented database access methods, STOV will call the web service adapter directly, and pass the DTO (in legacy format) as a parameter (see STOV as Web Service Consumer).
- Various Operating Systems
- Databases from different vendors
- Web Service Container
- Enterprise Service Bus
- Service Repository
- Service Registry
- Business Process Management
- Single Sign-On
- SOA Monitoring
- Software Configuration Management
Challenges (to date)
- Requirement Management: Because of paradigm change, the requirements are not clearly stated.
- IT Operations: These SOA projects will add (a lot of) new parts (both software and hardware) in the existing infrastructure. This requires tight collaboration with the existing operations team. The acceptance of the program within the operations team is low, so a change program is needed.
- Skills: New technology change requires new skills. Management is hesitating with the training, because they assume all development will be taken over off-shore anyway.
- Technology: Interoperability has to be ensured between different technology stacks. Also, performance may suffer because of network hops and XML overhead.
- An SOA framework (or platform) needs to be delivered and proofed before various IT transformation projects (scoped on an application) start.
- Tight collaboration between the existing operations team and the team that designs and installs the new SOA-based software and (additional) hardware is essential.
- When migrating data as part of the program (in SOA-related projects mostly consolidation towards data services), validation and cleaning is of utmost importance.
- Use of standardized (industry) information models is advisable.
- Instead of only focusing on SOA frameworks for specific solutions, also ensure that enterprise SOA principles are accounted for.
- The major business case was to decrease time-to-market. Define metrics, baseline them, and then measure after implementation. Some potential metrics are named in the case; e.g., frequency of changes in existing applications, frequency of new developments.
- Implementing SOA components like proxy, registry, policy manager, interceptors, etc. can be complex. It is important to execute a proof-of-concept addressing the riskiest areas of the solution architecture before starting a project.
- Managing change using a formal approach running parallel to SOA project execution is crucial in gaining acceptance of new technologies and processes used in projects.
- Various testing activities need to be planned as part of the project planning (and as early as possible), especially interoperability and performance testing.