SOA in the Digital Age


The series of webinars SOA in the Digital Age explores the role and value of Service-Oriented Architecture (SOA) in the context of digital technologies that are new in 2015.

The first webinar, on Developments in SOA, was held on Tuesday March 10, 2015. It discussed how SOA has changed to meet the needs of the digital age. A recording is available from The Open Group Bookstore and, with other relevant information, on the web.

The other webinars in the series are yet to take place. They are planned to address:

  • How SOA now enables enterprises to conduct business more effectively, and the role of the business technologist
  • Whether microservices are an evolution of SOA, and how they should be used in Enterprise Architecture
  • How SOA can enable enterprise agility, and whether SOA governance is adapting to meet the needs of the digital age
  • The new International Standard SOA reference architecture (currently in draft) and how it will help enterprises use SOA

Developments in SOA

Service-Oriented Architecture (SOA) first came to prominence in 2005, although some companies had been experimenting with it before that. At first, it was like the Wright brothers’ airplane, brilliant in concept but lacking in technical maturity. Now, in 2015, it is ten years old. Ten years is an age in information technology evolutionary timescales. How has it developed since the early days, and what is its role in the new era of digital technologies such as cloud computing, big data analysis, and the Internet of Things?

Initial Evolution of SOA

The original challenge was integration between multiple independent applications and technologies with proprietarily interfaces. The service-oriented approach gave developers the ability to create applications with interoperable interfaces and with the underlying technology encapsulated, so that they could focus on the business aspects.

In a typical early project, information was stored on a collection of mainframes and minicomputers speaking different protocols. The World-Wide Web had become popular, and every system had an HTTP server. The project team wrote scripts that extracted the data to be exchanged, converted it into HTML in an agreed format, and transferred it between systems, effectively creating web services.

As web services evolved, HTML was replaced by XML. This enabled distributed data exchange between systems. Architects were able to think in terms of a networked distributed processing model, which not only helped integration, but also led to better software architectures.

At that point, SOA was about defining XML documents and creating services so that applications could query for the documents and fetch them. This was effective, but had limitations when the information was processed in infrastructure that was not web-oriented. This is why the initial XML-over-HTTP web services evolved to use SOAP.

The architectural dimension was crucial. In the early stages, many vendors offered products for web services, but deploying web services did not in itself create an SOA. It took time for enterprises to understand the need for architecture around the services. Only after the initial deployment did it dawn on them that they needed web services design and management, including policy enforcement, provisioning, and tracking metrics.

In time, SOA and Enterprise Architecture came to work hand-in-hand to ensure that goals are realized within the broader enterprise context.

Business Value

The ability to deploy information in a standardized manner means that the cost of integration for the providers and users of the information is greatly reduced. In particular, SOA facilitates supply chain integration, with members of a chain leveraging services from each other. With SOA, integration becomes standard. Also, it allows for simplified implementation using low-cost tools, and this makes a big difference.

The hotel industry provides a good example of business value delivered by SOA. Hotels need to exchange information with online travel agencies. They do this through services that transfer XML documents. The format of these documents is often defined by industry bodies such as the Open Travel Alliance (OTA) and the Hotel-NextGeneration Consortium (HTNG). The service-oriented approach lowers costs for the hotels and for their business partners.

Tooling and Infrastructure

SOA started off being very cheap. As it developed, in particular with the addition of SOAP, it grew in complexity. This gave value but at a big additional cost. It was great for solving enterprise problems, but the virtues of the original simplicity were lost.

Tooling and infrastructure evolved to meet these needs. Full-featured languages such as C++ and Java have powerful libraries that handle much of the complexity, and the Enterprise Services Bus (ESB) is a powerful piece of infrastructure to support SOA.

Just as deploying web services does not necessarily mean that you have SOA, neither does using an ESB. You still need the appropriate architecture and governance structure to use the ESB correctly. An ESB is an enabler that helps you to enforce the right levels of control and structure, but it is not a solution to every problem. If you have a problem that an ESB solves, then it is right to use one, otherwise it is a waste of money.

A user application often involves orchestrating multiple core business services. Within an enterprise, an ESB with an orchestration capability is a great way of doing this. Also, it de-couples the core services from the application so that they do not have to be integrated in a point-to-point fashion. Message routing can be changed without changing the message clients; the ESB simply redirects the messages to new targets. This is important because, while the core business services are typically long-lived, application services are short-lived, reflecting what must be exposed to the end user. Finally, an ESB can be a common security control point.

In a hotel chain, for example, an ESB can provide an excellent way of integrating the website with the central reservations and folio management systems, with flexible message routing, and easier security implementation.

A Return to Simplicity?

Today, we are seeing a pushback against the increased complexity and cost of SOA, and a move back to simplification. It includes use of an underlying REST-ful model for data access, and the use of JSON rather than XML to represent information.

This is partly due to the languages and tools used by developers today. Many of the newer languages, such as PHP5, JavaScript, Ruby, and Python, do not have the depth and richness of the C++ and Java libraries. Very few developers leave college today with training in SOAP and other technologies that were popular five years ago. They prefer a setup that is easier for them to understand, and they are now looking at REST, JSON, JavaScript, etc.

REST is not a protocol, it is a philosophy, and it is possible to use SOAP in a REST-ful way. But most developers now understand REST as the application of the philosophy using HTTP directly, without SOAP.

MicroServices Architecture (MSA) has evolved as a simplified subset of full SOA with certain design constraints. With MSA, rather than taking a monolithic application and exposing its capabilities as services, the idea is to construct the application as a collection of small services. MSA can be defined as SOA done right, on a small scale.

Full SOA, using ESBs where appropriate, remains an excellent approach to handling internal complexity within an enterprise. REST and MSA are an excellent way of exposing APIs externally. The two approaches work well in combination.

SOA and Digital Technologies

SOA works well in conjunction with the new digital technologies, and is in fact an essential basis for cloud computing.

Cloud computing is about utilization of under-utilized resources and service provision across firewalls. It is built on web services, together with other technologies such as virtualization. The fundamental principles of SOA, although rooted in business and applications, apply also to infrastructure. The availability of infrastructure as a service is a key effect of SOA. Cloud computing would not exist today if we had not created SOA.

The Open Group defined the first technical standard for Service-Oriented Cloud Computing Infrastructure (SOCCI).

The Internet of Things is a classic case where SOA can be a big winner. For example, a typical hotel has hundreds of thermostats, and wants to manage them centrally. Services give the ability to create an architecture that fits the solution, by polling the devices or with devices as sources of messages.

SOA can be used with big data, but not at a granular level. For example, services can be defined to set up and direct video streams and other data streams, or to manage bulk file transfers. These services work at stream level, not at byte level. Large databases expose services for data retrieval and for slicing and dicing.

SOA Today

SOA has been commoditized, becoming more available and prevalent. Also, it is more mature, with better governance, architecture, policy enforcement, security, and portfolio management.

It has had a major impact on the way that development teams think. For some time, the key to success has been to ensure that what is built can integrate with the next partner’s application. There is a proactive urge to be compliant with standards, if not to define them, and to address interoperability from the start. Development teams no longer just look at what it takes to build and deploy their applications, they look at how to factor in design considerations early on, to take advantage of available services in a more effective manner.

SOA has grown up. Initial SOA had all the essential components, but undeveloped, just as a baby has all the fingers and toes of an adult. Now that it has grown up, it can be used with its original simplicity, but also for more complex tasks. This complexity brings a need for more sophisticated management and governance. The manager of a service-based system needs to understand who exposes the services, who consumes them, who monitors them, and how they are combined, just as the manager of a website needs to look at the logs.

The core concepts of request/response or request only, with standard messages agreed between consumer and provider, remain unaltered. Fundamentally, SOA has not changed, but it certainly has grown in maturity. It has evolved, just as the Boeing 747 evolved from the Wright brothers’ airplane, to be the robust, useful architecture style that we have today.