SOA Reference Architecture – Basic Concepts
A number of key and fundamental concepts recur throughout the SOA Reference Architecture (SOA RA). We will summarize them here.
The distinction between logical/design-time and physical/runtime elements of the SOA are described below:
- All runtime elements that are part of the physical or operational or deployment architecture are actually running in the Operational Systems Layer of the SOA RA.
- We will provide a logical abstraction of the runtime at a moment in time and expand that view in terms of a set of layers that are depicted by horizontal/functional layers and a set of supporting or cross-cutting layers (backdrop) of the SOA RA. The five horizontal layers deal with enabling the business capabilities required by the application running on the architecture. The four supporting layers support the functionality that is provided by the horizontal layers. The horizontal or functional layers include the Service Component Layer, the Services Layer, the Business Process Layer, and the Consumer Layer.
- Each aspect of the runtime is abstracted into a layer whose responsibility is significantly distinct from that of the other layers. For example, the Consumer Layer is responsible for interaction with consumers of the service, whereas the Service Component Layer is responsible for the implementation of the service in a Service Component. That Service Component in turn will be running in a container within the Operational Systems Layer, at runtime.
This section explains some of the concepts, principles, and reasoning of the SOA RA and sets the tone for the rest of the document. We recommend you read this before you read the SOA RA details, but it can be referred to at any time. Some of the terms in this section are defined later in the document. These foundational concepts will be discussed in a Question and Answer (Q&A) format.
Does the Consumer Layer consist primarily of things like a Graphical User Interface (GUI), or is it a programmatic means of access to the services, or both?
It is both. The Consumer Layer provides access to services. It allows consumers to consume services. The means of consumption of services can be a GUI or an API-like connection to the services.
Can the Integration Layer access services?
Yes. In fact, the Integration Layer is a layer of choice through which services are invoked, but it is not the only means to invoke services. Services can, in a less mature SOA, be invoked directly by the consumer without the level of indirection associated with an Integration Layer.
Which layer is responsible for invoking the service end-point?
The Consumer Layer is responsible for invoking the service end-point. It accesses the services via the Integration Layer. Or it can access the service directly if the given architecture does not wish to implement an Integration Layer. Invocation is done by opening up access to the Consumer Layer.
Who provisions the service?
Provisioning means doing all of the tasks necessary to make the service available for consumers to be able to invoke. Part of the responsibility of provisioning is updating the registry (in the Governance Layer) which contains the service metadata that consumers need to find, bind, and invoke services.
Are all the Architecture Building Blocks (ABBs) of a layer required for every SOA implementation? Or can we pick and choose from among the ABBs based on the needs of individual projects?
Not all ABBs are required for every single implementation of an SOA. Instead every project will choose from the list of building blocks within each layer of the SOA RA and select those that are appropriate for the particular project. In the case where SOA RA is applied for the enterprise architecture standard within an organization, there will be patterns of ABBs within the layers that will be selected. Some will be mandatory and some will be optional, and projects will be chosen to conform to a fixed set of patterns and configurations of the ABBs along with product selections and implementations.
Registries and repositories may need to exist in multiple layers during a physical implementation in my projects. Where are they considered to be in the SOA RA?
We have chosen to organize the registry and repository location in the Governance Layer of the SOA RA. In this way, we can manage, monitor, and administer the registry and/or repository from a single logical location, even though physically we may have federation or distribution.
Can you tell us about how the nine layers would be utilized in practice for realization of the architecture?
The four vertical or cross-cutting layers – namely Integration Layer, Quality of Service Layer, Information Layer, and Governance Layer – are basically supporting capabilities realized through implementation and vendor products. The five functional or horizontal layers – namely Operational Systems Layer, Service Component Layer, Services Layer, Business Process Layer, and Consumer Layer – will support the functional capabilities of the architecture. The Operational Systems Layer will provide the actual runtime for all layers, vertical or horizontal.
What are the ABBs in each layer?
They are the key building blocks of a particular layer. You can think of them as the components providing key capabilities that are expected from that layer. For example, the Integration Layer is expected to provide mediation, routing, and protocol transformation capabilities. Therefore, these capabilities are realized by a set of building blocks, each of which provides exactly that atomic unit of architectural capability you are seeking.
What kind of ABBs do we have?
We have some ABBs that are related to the functionality within an application such as:
- Consumer Layer: Portal to loan processing application.
- Business Process Layer: Initiation of a loan processing application.
- Services Layer: Services that are required to support the loan processing application.
- Component Layer: Software components that need to be built that support the realization and implementation of the services.
- Operational Systems Layer: Actual runtime environment in which the components, legacy systems, all back-end applications, and packaged applications reside and run.
Then we have the cross-cutting layers which include the Integration Layer that serves as the central point where integration occurs across the enterprise and consolidates the design decisions into a set of software components that facilitate and enable the actual integration of applications.
The Information Layer includes all data and information needed to support and instantiate each of the layers. Note that it is a cross-cutting layer indicating that each of the horizontal layers may have and will have data associated with their functioning and they will draw upon this data from the Information Layer via metadata, actual data, or analytics.
The Quality of Service Layer ensures Quality of Service (QoS) by serving as a collection point for the administration and control, or monitoring and management of most if not all of the Non-Functional Requirements (NFRs). This includes security, availability, configuration, monitoring, and management capabilities, to name a few.
Lastly, the Governance Layer provides a central point in which policies are set into registries and repositories. In general, governance capabilities and processes are administered and run centrally via this layer. Note again that this layer is the underlying layer for all of the other layers within the architecture and it relates to and touches upon all other functional and cross-cutting layers of the SOA RA.
Where are the policies defined and where are they enforced?
Policy definition is the responsibility of the Governance Layer. The definition of Policy Enforcement Points (PEPs) and policy enforcers is the responsibility of the individual horizontal/functional layers, such as Services Layer, Business Process Layer, etc. The enforcement of those policies is then the responsibility of a cross-cutting layer, and we have chosen to consolidate that within the Quality of Service Layer. Therefore, monitoring and enforcement of the policies are the responsibility of the Quality of Service Layer, while the administration of the policies remains the responsibility of the Governance Layer.
Where are the business rules defined?
Business rules are also a cross-cutting architectural concern. They need to be consistently applied across the multiple layers in the SOA or, over time, there is a tendency to develop rule divergence and lose the consistency that SOA brings. Therefore, business rule definition and management is the responsibility of the Governance Layer and its validation and enforcement is the responsibility of the Quality of Service Layer.
Are ABBs in the SOA RA and artifacts that we create as part of methods the same?
Not in all cases. For example, a process definition or a process model artifact is used by the Business Process ABB in the Business Process Layer to describe the underlying business process.
Events are cross-cutting. Where do they reside and where are they handled?
We have agreed to treat the cross-cutting nature of events within the Integration Layer, Quality of Service Layer, and Information Layer.
How and where are complex event processing capabilities addressed? Which layer is responsible for the events?
No single layer is responsible for events. Many layers and corresponding ABBs collaborate to produce a composite capability related to events such as Complex Event Processing (CEP), Business Activity Monitoring (BAM), etc.
Business events occur during the course of business process execution. An example of this is when we want to monitor fraud. Fraud management occurs when during the execution of a business process certain data items’ sequences and patterns are spotted that trigger a set of rules relating to possible fraud. This will trigger a notification and subsequent action. A set of data access or update patterns can also be a reason to trigger events in the Information Layer.
Thus, CEP is addressed across multiple layers of the SOA RA; it may start from the Business Process Layer or Information Layer. But the building blocks for producing and listening for events reside within the Integration Layer. The Quality of Service Layer is responsible for monitoring and managing the events. The Information Layer is used to define the events.
If someone asserts they have an SOA that conforms to the SOA RA, what would have to be true for that statement to be true? Is the SOA RA normative? How should the SOA RA be used?
The SOA RA is intended to be a set of best practices and guidance on creating successful architectures using the SOA paradigm. It is not intended to be mandatory or normative.
It is a qualitative standard in that users of the SOA RA may choose to deviate from the standard in certain areas. Based on the maturity of the organization implementing SOA across the multiple domains (business, governance, methods and processes, application, architecture, information, and infrastructure), an organization may choose among the various ABBs provided by the SOA RA for conducting assessments, designing, or implementing architectures.
Some of the ABBs in the SOA RA are foundational and may be essential or mandatory, and others are advanced or optional and may be required for higher maturity in SOA.
If we claim we have an SOA then we should compare it to the reference architecture. If the SOA solution does not conform to the partial layer architecture and if there are certain things that are logically missing, then we should identify the missing building blocks that are key for the specific instance of the architecture. The corresponding Solution Building Blocks should be present.
There are implications in terms of resource requirements. We will see in RFPs compliance with the SOA RA. There will be a spectrum of implementation ranging from point-to-point to physically brokered via one or more Enterprise Service Buses (ESBs). Customers’ architectures can be different but conformant. It is also a quality and compliance checking mechanism.
What is the difference between an ABB, Service Component, Deployment Unit, Solution Component, and a Solution Building Block?
An ABB is a logical entity. Each layer is composed of ABBs with which it implements its capabilities.
We can separate ABBs into those that provide the infrastructure to support the SOA, and the services themselves. An example of an ABB which provides infrastructure support could be a Service Container ABB in the Services Layer, or a Mediator ABB in the Integration Layer. An example of an ABB related to implementing or providing a service is the Service Component ABB.
Let us understand the ABBs which form a service first. Each service has a Service Component – which is an ABB in the Service Component Layer. A Service Component is composed of a Functional and a Technical Component. The Functional Component provides the functional capability that the service implements. This could be a legacy system invocation, a Java POJO (Plain Old Java Object), a COBOL subroutine, etc., or one or more other Functional Components. The Technical Component encapsulates the technical capabilities to support standards compliance and technical support that a service must provide. However, the Service Component and its associated Technical and Functional Components are logical, design-time assets.
Now let us look at the creation of a service and its conversion into a runtime entity. The service is created using an Integrated Development Environment (IDE) of some sort, with resulting Service, Technical, and Functional Components. Next the Service, Functional, and Technical Components are bundled up into a Deployment Unit. Finally the Deployment Unit is deployed into a runtime environment, becoming a Solution Component. Structural Relationship between Service-Related ABBs shows the structural relationship between the different service-related ABBs.
The Dynamic/Temporal Relationship between Service Components, Deployment Units, and Solution Components shows the “lifecycle”, or the dynamic relationship between the design and development time Service, Functional, and Technical Components, the bundled Deployment Unit, and the runtime Solution Component.
To understand this in more detail, let us consider an example. In this example we have a CreditCheck Service, which is a SOAP doc-style Web Service, which checks credit by invoking other services – CheckInternalCredit, CheckCreditAcmeRatingAgency1, and CheckCreditAcmeRatingAgency2. CheckInternalCredit is a service which invokes a JEE component which encapsulates a mainframe-based relational database.
Let us consider CheckCredit first. We develop a Service Component which deploys on a .NET environment, whose Technical Component encapsulates the SOAP binding with the .NET stack and integration with the QoS implementation interfaces. The Functional Component for CheckCredit is a POCO (Plain Old CLR Object), which contains the logic to invoke the services CheckInternalCredit, CheckCreditAcmeRatingAgency1, and CheckCreditAcmeRatingAgency2, as well as to compose a response (the data) which is going to be sent back to the service consumer of CheckCredit. The build process compiles and bundles all the resources and parts of the Service Component together to form the .zip file and .dll component which is then placed in the runtime environment. When placed in the runtime environment (in the appropriate directory), the .dll component becomes the Solution Component for the CheckCredit service.
In practice, SOA typically involves multiple platforms and environments, so we will consider CheckInternalCredit next. This service is a SOAP doc-style web service using JAX-WS as the binding, a POJO (Plain Old Java Object) to wrap invocations to the database and build (compose) the response. In this case the Service Component includes the Technical Component which manages the SOAP stack binding and support, and the POJO, database components, and other elements which form the Functional Component. When the build script compiles and puts all the parts of the Service Component together it creates a Deployment Unit (let us call it a servlet for this discussion), which is then deployed into the appropriate directory to be used at runtime – becoming a Solution Component.
So what is a Solution Building Block? In both services considered in the example above, the service gets deployed into a Solution Building Block which provides the runtime implementation of a Service Container. In the case of the .NET service (CheckCredit), the Solution Building Block is (for the sake of our example) an IIS/WAS (Internet Information Services (IIS)/Windows Activation Service (WAS)), and in the case of the JEE service (CheckInternalCredit), the Solution Building Block is (for the sake of our example) an Acme JEE Server. In both cases, the corresponding ABB for the Solution Building Block is the Service Container ABB in the Services Layer.
As can be seen, the Solution Building Block is the runtime instance of an ABB that provides the infrastructure to run services. Solution Components on the other hand are ABBs which provide the runtime instances of the services themselves.