COTS Applications and the Service Boundary

Most commercial off the shelf (COTS) applications these days expose Web service endpoints so they can interoperate in a platform neutral way with other applications in your enterprise. These COTS applications are not services in their own right.

Although they expose Web service endpoints, they are not services in the context of SOA. The endpoints they expose should be considered integration points to be leveraged inside the service boundary.

The contract of a business service must be expressed in terms of your business. The contract should abstract away the implementation details of the service (such as which COTS application is in use), expressing the business functionality in the language of your business.

The same off the shelf package will fulfil different roles in different services in different organisations. For instance in my last post, we explored an example where we had a CRM system being used to support both a Sales service and a Marketing service.

Now this could be two instances of the same CRM package, or as in the example could be a shared instance. Regardless, the functions being exposed to the enterprise are two distinct services that based on their service contracts perform Sales and Marketing functions.

If we directly exposed the CRM application's service interface as both the Sales and Marketing service contracts, then both services would have the same interface! This would imply they perform the same business function. Obviously as they are distinct services, they do not.

There needs to be an abstraction away from the service interfaces offered by COTS applications to the intended business function to be performed by the service.

So in summary, do not directly expose a COTS application's Web service endpoints to the other services in your organisation. Put a layer of abstraction in between.

Sharing Applications Between Services

A reader posted a good question regarding my Service Boundary Example post. He asked:

Let's say as a result of our enterprise architecture planning we've decided that we're going to have a Sales Service (Leads, Opportunities, etc) and a Marketing Service (Campaigns, Promotions, etc).

However let's say we currently have one legacy CRM system that supports the marketing department and the sales department.

Which service boundary will contain that system?

The answer to that is that different parts of the legacy CRM would sit behind each service boundary. There is no issue in sharing infrastructure between services. You just need to be aware of the dependencies on that infrastructure from a risk management perspective.

Service boundaries are drawn up around data. There must be no data shared between services except through explicit message exchanges.

At the very least I would suggest that we have process isolation between services, so that data isolation is enforced at the OS level. To be fair however, you don't really require process isolation between two services in order for them to function as services. It is just obviously a good design choice.

So in this case, as long as the lead and opportunity data are never used by the Marketing service, and the campaigns and promotions data are never used by the Sales service, then we have data isolation and we haven't offended the SOA gods too deeply.

When transitioning legacy systems into an SOA, we sometimes have to compromise further on this because we have a single legacy system used by two services that doesn't enforce this data isolation. We can't control the design of this system. And to be fair, it wasn't designed to be used in this way.

The issue is we now have a message sent to one service, affecting the state of the second. This is obviously very bad and not ideal, and I wouldn't consider these services as true isolated services in the context of SOA. However it can be a useful half way point on the way to true SOA.

Why a Single Service?

In my last post, I illustrated an example where two legacy applications were bundled up to be included in a single CRM service. The question was raised as to why these were not just simply made to be separate CRM services.

The answer comes down to aligning the service definitions with the business. In the majority of businesses, the handling of a VIP customer is similar to that of a regular customer. The two processes have a lot in common. In order to have loose coupling and high cohesion, both of these concerns should then be placed in the same service.

Moreover, the decision as to whether or not a customer is VIP is a CRM concern. We do not want this logic in other services that send requests to the CRM service. If we had two separate CRM services, then each service sending them requests would need to decide which request to send - the VIP CRM service request, or the regular CRM service request. This decision should not be a concern of the requesting service.

Service Boundary Example

After my last post, I thought I'd give an example to clarify the point I was making about service boundary definition. Let's say we've got a couple of legacy custom built CRM applications that we want to leverage in support of a new CRM service. Let's say one application is used for VIP customers, and the other for regular customers.

We have identified that we want a single CRM service to support the Sales department. The development team decides that they will expose Web service endpoints on each of the legacy CRM applications in order to provide integration points for our new CRM service.

We define our CRM service contract based on the business function we want our service to perform. Note that we now have 3 service contracts, but only one service. We have the public CRM service contract, which is what defines our service boundary; and then we have a service contract for each of the legacy CRM applications, which sit behind the service boundary.

These legacy CRM applications are not services in the context of SOA, despite accepting messages over a WS-* stack defined by a WSDL document. They are an implementation detail of our single CRM service, and should not be directly accessed by anything outside the CRM service's boundary.

We now need to implement the the public CRM service operations, leveraging the private operations exposed on the legacy CRM applications. We might employ a tool such as BizTalk for this job.

At a future point in time, if our enterprise decided to replace the legacy CRM applications, we could do so without updating the CRM service contract - assuming of course that the required functionality offered by the CRM service to the enterprise did not change.

Defining the Service Boundary

I think maybe we need a break from our lengthy thread of discussion on data and its influence on SOA. So I've decided to start off a discussion on service definition and the service boundary.

Perhaps one of the reasons why there has been so much confusion over the definition of SOA, is that we haven't been very clear on the definition of a service itself. A service in any given architecture is defined by its boundary. The boundary tells us what parts make up the service (what is inside the boundary), and how the service interacts with the outside world (determined by the service contract).

One problem is that the WS-* stack has become pretty much ubiquitous, and is used today for pretty much any type of communication between distributed components. WCF as Microsoft's unified communications stack perpetuates this. When we have WCF artifacts such as service hosts, service contracts, endpoints, etc floating around everywhere, people tend to align their definition of a service with these constructs.

As a result of this WS-* proliferation, it is extremely likely that we will have a lot of WS-* communications going on behind the boundary of a distributed service.

A distributed service is one where communications over a network occur between distributed components behind the service boundary. Although these communications are message based and defined by a contract described in WSDL, these components do not sit on the service boundary. Whether the parts of a service are distributed or not is an implementation detail of the service - it does not affect its contract, and as such does not affect its boundary.

So with all these endpoints and WSDL documents flying around, how does one identify the service boundary? Well, we need to classify all distributed components exposed in a service as public or private. The public components define the service boundary. The private components are an implementation concern of a distributed service and must be hidden from the service's consumers.

No system outside the service boundary shall communicate directly with any system behind the service boundary. Data cannot be directly shared between services. It can be shared only by way of message exchange through the explicitly defined public endpoints at the service boundary.

Enforcing this is really a matter of IT governance. Service boundaries must be defined, documented and published by the architect. Only the public endpoints should be known to service consumers. This in turn defines the intended service boundaries.

Fear the DSP

Udi Dahan has just made a post which brought to my attention a new TLA (Three Letter Acronym) in the SOA space - DSP (Data Services Platform). I agree with Udi - this is all bad. Hopefully it will go away on its own.

Udi made a previous post citing similar criticism of Microsoft's then named Astoria product, which has now been re-badged as ADO.NET Data Services. I agree that this product has the potential to encourage people to expose CRUD interfaces at the service boundary, which for reasons I've previously explained are bad news.

The space in which this product should play is exposing data for consumption by Web clients in AJAX or Silverlight applications. In this case, the CRUD interface is used only inside the service boundary, which is fine.

If it is unclear to you as to why the Web client is inside the service boundary, stay tuned for my next series of posts which should clear the fog a bit.

SOA and Business Intelligence

This post follows on from my post on SOA and the "Universal Truth". The "Universal Truth" as it was described, referred to the ability (or inability as it turned out) to find the complete truth about a given business entity (such as a customer or an order) at any given point in time. What was not addressed was the ability to see a wholistic view of a number of entities distributed across a number of services.

For this, we create a BI service which sits on the service bus, subscribed to all relevant business events. When these events occur, the BI service stores the relevant information extracted from the notification messages into a local data mart, which is then used to populate one or more OLAP cubes. The business can then use a standard BI package to query and present the data.

Note that this BI service does not constitute centralisation of data in the context of my previous post on Centralised vs. Decentralised Data. Although we have a large part of enterprise data stored in a single location, that data is also still duplicated in all the various business services throughout the enterprise. As such, data has in fact become more decentralised.

There is also some debate as to whether a BI service would constitute a service in the strict sense of the term in the context of SOA. A service should have both behaviour and data. A BI service that has only a reporting capability would probably be considered a shared component of all services whose data is duplicated within the data warehouse.

In this regard it is similar to an enterprise portal (where portlets can be exposed in a central portal although actually being hosted in other portals elsewhere in the enterprise). The central enterprise portal is just an aggregation of portlets in a central location and doesn't itself constitute a service. I'll be doing a series of posts on defining services and service boundaries next, so stay tuned!