Taking Questions Offline

I've recently received a couple of questions by way of people posting comments on recent blog posts as I haven't provided a way for people to get in touch with me directly.

That has now been remedied. If anyone would like to get in touch with me for whatever reason, you can do so by emailing me at creative.abrasion@gmail.com.

Of course as usual, any and all questions are welcome.

Enterprise Canonical Schemas (continued...)

In my last post we discussed the concept of the enterprise canonical schema and its suitability when applied in the context of the various flavours of SOA we've discussed to date. As part of that discussion I stipulated that an enterprise canonical schema should be avoided when engaging in self-contained process-centric service models.

However, canonical schemas can still be very useful when we are engaging in EAI behind the service boundary. That is, when we have one or more services containing a number of legacy systems whose activities need to be orchestrated in support of those services.

Self contained services control and hold their own data locally. Data is represented within each service in such a way to best support the cohesive business processes that execute within that service. If we were to have two different CRMs within an enterprise, both would likely sit in the same Customer Management Service.

The Customer Management Service would have its own service contract expressed in terms of the customer management business domain for the enterprise (rather than in the terms of the CRM applications within the service). We have a layer of abstraction (an anti-corruption layer as termed by Eric Evans) between the CRM applications and the service boundary that coordinates the activities of the CRM applications within the service boundary in support of the service's business processes.

So that we don't have to deal with two entirely different data representations within the orchestrations coordinating the activities of the CRM applications, we develop a service canonical schema which is aligned with the service domain model. If we have multiple services enagaging in EAI internally, then we will have a different canonical schema for each service.

Any messages outbound from one of the CRM applications are first translated to the canonical schema. Likewise, any messages inbound for a CRM application are first translated to that CRM application's native syntax. The orchestrations coordinating the activities of the CRM applications are then expressed in terms of the canonical schema.

Of course, if a service does not include a number of applications that must be integrated in support of that service then there is no need for a canonical schema at all.

Enterprise Canonical Schemas

There are many references in the available guidance on SOA that recommends the use of an enterprise canonical schema. The idea is that you standardise all data transferred between services via messages such that they conform to a single canonical schema.

The process of producing such a schema is known as schema rationalisation. It involves forming an agreement between all relevant stakeholders in the enterprise on a universal representation of all entities (such as customers, orders, etc) relevant to the business.

Any outbound messages from a service are first translated to be consistent with the canonical schema before being sent. Likewise, any messages inbound for a service are first translated in that service's native message format before being delivered.

The purpose of this approach is to decouple interacting services from the native message formats of each service and to minimise the volume of message translation logic necessary to support the architecture.

If we have n services in our enterprise, then we have up to n(n - 1) interactions that require inbound and outbound message translations between native service message formats. Through use of a canonical schema, we only have 2n message transformations that need to occur.

So this all sounds really good right? Well it depends on the flavour of SOA in your enterprise. This approach isn't really going to add much value in a JBOWS architecture. JBOWS architectures don't add much value at all, regardless of the presence of a canonical schema.

But what about Service Oriented Integration (SOI)? Well as it turns out, a canonical schema is very useful here. With this flavour of SOA, applications are exposed directly as services in your SOA. As a result, you don't really have a lot of control over the syntax of messages understood by your services.

Message schemas tend to be quite volatile with SOI as they are based on the applications in your enterprise, rather than the business processes they support. A canonical schema decouples services by shielding them from message syntax variations in other services.

Furthermore, with SOI we may have multiple similar applications participating directly as services in any given architecture. For example, an enterprise may make use of two different CRM packages, which would by extension mean we have two CRM services. Because with SOI business processes are generally executed in middleware sitting between services, when a service sends a message it may not in fact know which CRM instance the message is bound for.

With a canonical schema, this decision is not relevant in constructing the outbound message. The middleware translates the message to the native format of the appropriate recipient as the message is delivered.

A canonical schema is also very useful (in fact one may say essential) when using a layered service model. Entity services abstract native data repositories into a form which is then used by task and process services. By ensuring data exposed by entity services conform to a canonical schema, we simplify our task and process services.

So what about self-contained process-centric service models? Well here a canonical schema actually works against you. With this flavour of SOA, you've already done the hard yards of building service contracts based on cohesive business areas. Your service contracts are aligned with the business (as opposed to technology artefacts) and as such are relatively stable.

Furthermore, with this flavour of SOA each service contract has been tailored to express information optimised for a specific business area. By placing a canonical schema between services, you diminish the semantic fidelity of communication between those services, mitigating the value of all that hard work.

A canonical schema not only standardises the syntax of communications between services. It must also by extension standardise its meaning, or semantics. An enterprise canonical schema tries to build a common language to suit the entire enterprise. All discussions between services are first translated into this language.

This is very likely to be a lossy transformation. Even if you don't lose data, you'll definitely lose information through blurring of semantics.

So as long as you go down the road of self-contained process-centric services, you should stay clear of enterprise canonical schemas. However, a canonical schema can certainly add value if you find yourself in an existing SOI or layered service model environment.

Sharing Concerns between Services

Graeme recently posted a good question regarding sharing of common concerns between services:

"Let's say that a service creates a document but the enterprise as a whole has an investment in a DMS. Would you see the DMS as a centralised data store that should be de-centralised?

Most applications I've seen would just create the document and then call a WS to store it inside the DMS but that goes against the autonomous nature of the service.

Would be interested in your thoughts on this."

There are two alternatives to approaching this problem, and as usual it depends on your specific business requirements.

The first alternative is to consider the DMS application as a distributed component shared by multiple services. In this case, the DMS is not a service in its own right directly accessible by the enterprise-wide service bus. It is a private component of the services leveraging it, sitting inside their respective service boundaries. As such, synchronous request-reply communication with the DMS would be quite acceptable.

There isn't really any problem in sharing a distributed component between services, so long as data is not mixed between services inside that component. Services should share data only by way of explicit message exchange.

You just need to be aware that if the shared component falls over, all dependent services will fail.

A good acid test to determine whether this solution is appropriate is to consider whether the documents managed by each service should be isolated from each other. So if the documents produced by multiple services should be managed together (for example stored together in the same logical folders), then this indicates that this approach is most likely inappropriate.

The alternative is to have a separate autonomous Document Management Service that interacts with your other services by exchanging messages in accordance with their public service contracts. In this case, the DMS application is a private component only of this service. In my experience, this is the better approach.

This particular approach can be tackled in a couple of different ways. Assuming each of the various services in your enterprise generate their own documents, they could either asynchronously send the Document Management Service a command message (such as a StoreDocumentRequest message) containing the document, or publish a business relevant event message (such as a PolicyCancelledNotification message) with the document attached.

With the command message approach, the message would contain any relevant metadata to store against the document. With the event message approach, the Document Management Service would extract the relevant metadata from the event message body.

Although I usually avoid command messages, they are useful and appropriate when the service operation is highly reusable.

Filing a document in a document repository would be a suitable candidate for a command message. That is, its behaviour would be unlikely to differ based on the context of the service sending the command message.

If your users interface with your services via a composite application, then you can have a document management module in that application that allows your users to interface with the Document Management Service in such a way that provides a seamless integrated user experience.

Something else to consider in this area is the generation of documents. If you have software that generates your business documents based on end user managed templates, then this would be a good candidate for a component to be placed in your Document Management Service.

The document management module in your composite application would provide the ability for your users to manually generate documents by sending messages to the document generation component. This component would place the generated document in an output folder and return the URL to the composite application in the reply message. The application would then retrieve and display the document at that URL.

Yes, this interaction between the composite application and document generation component is synchronous request-reply. However this is perfectly acceptable because both the document management module and back end component are behind the boundary of the same service.

If after generating the document the user decides to store the document in the repository, then this would be a separate message from the composite application to the document repository component, which would contain the document URL and associated metadata. The document repository component would pick up the document at the given URL and store it in the repository with the given metadata.

So we have two ways in which documents can be generated and stored. One is where the document is being automatically generated by a service, where the service can send a command message containing the document and associated metadata to the Document Management Service (or publish an event message containing the document onto the service bus to be picked up by the Document Management Service).

The other way is where a user manually generates the document via the document management module within your composite application. The composite application is then responsible for sending the message to the document repository component at the user's request.

SOA Quality Spectrum

We've covered a lot of ground over the last month since I first proposed my definition of SOA, and I wanted to take the opportunity to do a bit of a recap and tie it all together by comparing and contrasting the various flavours of SOA discussed to date.

As I've previously mentioned, there are good SOAs and bad SOAs - with some considerably worse than others. So what are the typical metrics we might use in assessing the quality of an SOA? One might say it depends on the needs of the business the SOA supports. That is, an SOA is of high quality if it clearly aligns with and supports the strategic goals of the business.

Obviously every business is different. Even businesses that compete in the same market and service the same customers have unique characteristics. They have different organisational structures and business processes. Their distinctiveness is what gives them their competitive advantage. It is what differentiates them from their competitors.

Business strategy is heavily influenced by the external forces that operate on a business. Porter's five forces analysis is a good way of identifying these external forces and deriving strategies for gaining competitive advantage.

Regardless of the business and the market in which it operates, an organisation must evolve and adapt as the competitive landscape changes. The ability of an organisation to invoke change quickly, inexpensively and at low risk is a key factor influencing its success. It facilitates innovation and affects how responsive an organisation is to market forces.

Although there are many factors that influence business agility unrelated to IT, a machine is only as fast as its slowest moving part - and in many large organisations, that part is IT. For these organisations, IT becomes the bottleneck, resulting in the perception that IT is a necessary evil, always at odds with the rest of the business.

So what aspect of an IT architecture determines agility? Simply put - coupling. Coupling determines the extent to which change within one software component affects another. With a high quality SOA, a single change to how a business chooses to implement a specific business capability is isolated to a single service. As only the implementation has been affected, the service contract remains unchanged and consequently no other service is affected or even need know about that change.

So although there are other quality metrics that vary from business to business and industry to industry, agility is a key universal quality metric for an IT architecture. Based on this metric alone, I propose the following SOA quality spectrum.


I've included JBOWS, Service Oriented Integration, Layered Service Models and Self-Contained Process-Centric Service Models on the spectrum. I'd be interested to hear from anyone who has seen any other flavours of SOA out there that I've missed. I'm also interested in getting feedback from people who would propose alternative or additional quality metrics for SOA.

Layered Service Models are Bad (continued...)

In my previous post I introduced a couple of example typical business processes for an insurance company and illustrated how they translated to a service model based on process, task and entity services. After listing a number of shortcomings of this model, I promised that I would suggest an alternative service model that didn't suffer from the outlined issues.

As I previously mentioned, services should be centred around cohesive business areas/capabilities. We also wish to decentralise our data so that each service has the data it needs locally to service each request without retrieving data from other services. Furthermore, we want to rely mainly on publish-subscribe messaging in order to avoid the coupling introduced by command messages.

So our first order of business is to identify our cohesive business capabilities upon which we will base our services. Through analysis of the originally described business processes, there appears to be the following five cohesive business areas at play - Premium Quotation, Policy Administration, Commission Management, Billing and Customer Management.

Creating a service around each of these business areas gives us the following service model.


Firstly note the simplicity of this model over the model outlined in my previous post. There are only five services, and the dependencies between the services are limited to communication over three topics.

There is no need for any entity services as each service depicted in the above model houses its own data locally. Consequently we have no need for cross-service transactions.

The business logic previously contained in task and process services relevant to a given business area is now fully contained in the corresponding service. As a result, we have far improved encapsulation. Our data is protected from direct manipulation by other services.

The Policy Creation and Policy Reinstatement processes are now mapped to the above service model as illustrated below.

Policy Creation

Policy Reinstatement

Note also with this service model that we have no synchronous request-reply communication between services whatsoever. This results in a considerably more robust, performant and loosely coupled architecture. The high cohesion found within each service further contributes to the loose coupling.

We have coarser grained, business relevant messages exchanged between services and there are considerably fewer message types to deal with. And to top it all off, this service model will take less effort to implement and considerably less effort to adapt and maintain.

Layered Service Models are Bad

Open any number of books out there on SOA and you'll find guidance on defining the service model for your enterprise in terms of process, task and entity services. This in essence sets up three layers of services, with process services as the top layer, task services underneath the process services, and entity services at the bottom.

The purpose of entity services is to provide an abstracted view of business document repositories. The actual structure of the underlying data and the platform holding the data are abstracted into the form of entity service contracts.

The operations exposed by entity services are your basic CRUD centric operations such as GetCustomer, GetOrdersForCustomer, UpdateCustomer and DeleteCustomer.

Task services each represent a single atomic business action. Task services leverage entity services to manage state.

Process services are then created in support of business processes. Each process service wires up any number of task services in support of a specific business process. Process services can also "invoke" other process services when there are sub-processes to be reused by those processes.

Consider the following two vastly oversimplified business processes:

Policy Creation
This process involves the creation of a new policy. An underwriter uses a composite application to first go and get a premium quotation, and then if the customer accepts the premium create a new policy.


Policy Reinstatement
This process involves reinstating a previously cancelled policy.


Note that the Allocate Commissions and Raise Invoice tasks are "reused" between the Policy Creation and Policy Reinstatement processes.

Translated into a service model based on process, entity and task services, we get the following:


For those who follow this blog, you would probably guess that I'm dead against this kind of service model. There are so many problems with this model it's hard to even know where to start:

Transactionality: The task services must perform updates across numerous entity services. The task services are atomic in nature, which means that either the entire task must succeed or fail. This means we need to have cross-service transactions between entity services.

As I've explained in previous posts, cross-service transactions are bad for performance reasons and because they hurt the autonomy of services. One entity service can have records locked while waiting for another entity service to either commit or rollback its local transaction. One entity service can heavily influence the execution of another.

Performance: As mentioned above, cross-service transactions hurt performance. However, the other thing to notice about this architecture is that it is full of synchronous request-reply communications. This approach simply does not scale.

Robustness: As I've mentioned previously, synchronous request-reply interactions between services seriously hurts the robustness of your architecture. If one of those entity services goes down, you can pretty much say goodbye to a large number (perhaps even the majority) of systems in your enterprise.

Cohesion: With this service model, multiple services deal with very similar business concepts. You have the same entities being managed by multiple services. You have multiple task services dealing with the same entities. This means you are likely to get considerable duplication of logic and information representation between services.

This means your developers need to write considerably more code. Furthermore, the communication between services is likely to be very chatty, involving a large number of finely grained messages.

This in turn means greater coupling between services. This results in even more code to be written in order to handle the vastly increased number of message types. In essence, the proposed services in this model are too finely grained.

Coupling: As mentioned above, the low cohesion between services means greater coupling. You also get temporal coupling as a result of all the synchronous request-reply communications.

Complexity: There are simply too many services defined in this service model. With this approach, you will likely end up with hundreds or even thousands of services supporting an entire enterprise - each with dependencies to be managed and understood.

Encapsulation: By exposing data directly via entity services, any service can update any data in whatever way they see fit. This results in very poor encapsulation. We want our data to be surrounded by business logic that enforces our business rules.

Service Contract Stability: The implementation details of cohesive business processes are exposed in the service contracts of the task and entity services. As the needs of the process services they support change, the service contracts will need to be updated. I'll cover service contract stability in another post.

People are usually allured into using this kind of service model by the promise of reusability. That is in fact the fundamental premise of the architecture - that process, task and entity services can by simply reused by dropping boxes that invoke those services onto a process orchestration design surface.

However as I've previously discussed, the promise of this kind of reuse is false. It is all very good in theory, but as usual the devil is in the detail and you simply don't get this kind of reuse in practice.

In conclusion, services should be self contained; aligning with cohesive business areas/capabilities. The concerns of a single cohesive business capability should be supported by a single service. Not by many process, task and entity services.

In my next post, I'll describe the service model that I would suggest to support these business processes, so stay tuned!

People and Service Boundaries

In previous posts, I've been very clear in pointing out that services shall not ever share data except by way of exchanging explicitly defined messages. However one way in which data can be legitimately shared between services without exchanging messages defined by their service contracts is via people shared between those services (if you'll recall, people sit inside the service boundary).

Consider two autonomous services each containing separate applications shared by the same user. Now consider that as part of a cross-service business process, this user must perform a task within one application immediately followed by a task within the other.

It is possible that the user may have to take information displayed on the screen of the first application and enter it into the second. This is a classic example of swivel chair integration that we would ordinarily resolve with a composite application. But for now let us assume that we are dealing with separate applications.

So, what actually happened here? As it turns out, we have transported data from one service to another without the use of a message! Is this actually a problem? Well no as it turns out. In order to understand why, we need to consider the fact that people actually only participate in the activities of one service at a time.

Although there are many tasks that human beings can do simultaneously (such as jog and listen to music), people cannot actively focus on two problems at the same time. Sure people can switch quickly between problems, but this just gives the illusion of multitasking. The fact is, you cannot interact with two applications at the same time. You need to use one application, and then switch to the other.

This process of turning your attention from one application to the other effectively moves you from one service to the other. As a result of this process, information read from the screen of one application is transported to the other.

Now the fact of the matter is we still haven't exchanged a message conforming to either service's contract. However we have exchanged a message; just not in the traditional sense. Unlike messages sent and received by computer systems, this message has no syntax. It only has semantics.

The message was constructed in the mind of the user when he or she read the information displayed by the first application. When the user then swapped to the second application, the message was "transported" to the second service where its contents were then used to enter data into the second application.

Granted that this message did not conform to any service contract. But this is because the message has no syntax. The message is already understood by the user (and by extension the service in which the user is participating) and as such the service contract is not relevant.

Composite applications can automate this process so that our users do not have to manually copy data from one screen to another. When we do this, we need to be careful that data is not moved between services in a way that is not known to and approved by the user.

Composite applications should only automatically populate screens related to one service with data retrieved from another - they certainly should not automatically start shuffling data from one service to another without the knowledge of the user. To do so certainly would violate our service boundaries.

Consider the example illustrated below.


An underwriter enters some basic information about a new customer into the composite application, which then retrieves an insurance premium quotation. This activity is occurring as part of the Premium Quotation Service.

If the customer decides to proceed with the policy, at the request of the underwriter the composite application takes the previously entered customer information as well as the premium quotation and pre-populates the policy creation screen ready to accept the rest of the customer's details. The underwriter enters the rest of the customer's details and then saves the new policy. This activity occurs as part of the Policy Administration Service.

JBOWS is Bad

JBOWS stands for Just a Bunch of Web Services, and is unfortunately something seen quite frequently in organisations first delving into SOA, or where there is an emphasis on tooling and technology rather than architecture and analysis.

It refers to the scenario where random/unplanned functionality is exposed within an enterprise as Web services. It is a build it and they will come mentality. We usually tend to find this with IT led (rather than business led) SOA initiatives.

This may in fact not be the fault of the IT department(s) involved. It may be that they are given a mandate for an SOA transformation, but there is little or no cooperation from the business in supporting that effort.

Services need to support an explicitly designed enterprise architecture, where each service has a well defined specific responsibility in the context of that architecture. We need a governance framework in place to ensure that services are created, modified, versioned and deployed in such a way that they are properly designed, compatible with each other and consistent with the enterprise architecture.

Where the enterprise architecture, governance and planning processes are lacking or non-existent, then services will be defined with arbitrary responsibilities and boundaries. This leads to overlapping responsibilities between services, which in turn leads to low cohesion and by extension high coupling between services. That is, we get JBOWS.

JBOWS architectures deliver little or no value to the business. If anything, overall business agility will decrease over time as more and more disorganised services spring up with considerably more complex and brittle dependencies that need to be maintained.

Part of the problem is the lack of guidance in the SOA space. Many vendors are pushing SOA as a badge on a technology solution and the message of how to properly design SOA systems is getting lost in the noise.

As I've said before, there are good and bad service oriented architectures. JBOWS is the worst kind of SOA. It costs a lot and very likely will see you worse off than where you started. Avoid it like the plague.

Microsoft Facilitates White Box Services

Microsoft has just recently announced the upcoming release of .NET Framework 3.5 Service Pack 1. This service pack includes a couple of changes to the WCF DataContract serialiser.

The first is attribute-free DataContract serialisation, where it seems we no longer need to place DataContract/DataMember attributes on every serialisable member anymore. They are referring to this as POCO (plain old CLR object) serialisation.

The second is interoperable object references. The DataContract serialiser now supports serialisation of object graphs rather than just trees. This was previously possible in WCF, but the PreserveObjectReferences setting needed to be enabled when creating the DataContract serialiser.

The issue however was that before this service pack, object references were not serialised in a way that was interoperable with other Web service stacks. Now WCF supports an interoperable object reference scheme that allows it to serialise object graphs. This in theory allows us to serialise object graphs that contain circular references in an interoperable way.

According to Microsoft, the big advance here is that now Entity Framework types are serialisable via the DataContract serialiser.

Now all of this new found power may sound really good. But personally, I find this all rather disturbing. Microsoft is effectively empowering developers to directly serialise their domain models.

If we no longer need to decorate classes with DataContract/DataMember attributes, then our domain classes become immediately eligible for serialisation.

But hang on, what about all those object references in my domain model that caused me serialisation grief in the past? And what about those references to the Entity Framework classes in my domain model (which shouldn't be there, but Microsoft made me put there)?

Well, Microsoft has solved that problem with the new found power of the new DataContract serialiser. You're all ready to go and expose your domain model directly in your service contract!

But hang on, that's really bad practice isn't it? Absolutely! If we expose our domain model in our service contract then we have a white box service. We couple our service consumers to our service implementation.

Moreover, domain models have considerably different design constraints to message schemas. Domain models are optimised to represent business logic, whereas message schemas are optimised for representing documents transferred over the wire.

You want to be able to design your domain model to most accurately reflect the problem domain, without having to worry yourself with how efficiently it might be serialised over the wire.

Generally, you will tend to find you want to serialise different aspects of your domain model depending on the message being sent. If you directly serialise your domain objects, you won't have that flexibility to choose different data to be serialised for each message.

So for a number of reasons we don't ever want to directly expose domain objects as part of our service contracts.

Now it seems that Microsoft has added these capabilities to the DataContract serialiser to support the new ADO.NET Data Services framework released as part of this service pack. This technology is intended for supporting RESTful architectures.

In order for this new technology to work, Microsoft needs to directly serialise its entity classes over the wire. Now I'm not bagging REST here (because that will start a flame war), but I disagree with Microsoft's implementation here because again, we have our entity model that we use to represent business logic leaking outside our service boundary. Thus, we have coupling between our RESTful service implementation and the service consumers.

Moreover by making this change to the framework, Microsoft has made it considerably easier for SOA developers to directly expose their domain models as part of their service contracts - and I assure you, the easier the framework makes it to do the wrong thing, the more developers will do it.

Service Granularity Example

A while back, Ayende posted a proposed service model based on an example business context provided by a reader. This provides a good example for discussion around service granularity. For the sake of brevity I won't repeat the complete details of the example or the solution proposed by Ayende here.

In essence, the business in question is a physician job placement organisation, which owns two separate physician job placement companies. There is about a 95% overlap in business process definitions. That is, the business processes between the two subsidiaries are 95% the same. From an enterprise architecture standpoint, this is an interesting example as it illustrates the importance of the business context in determining the IT architecture.

Firstly, we need to understand whether both of these separate subsidiary companies operate independently, or if there are functions shared between the two. If there are many shared functions, then what we are really looking at here is one company with two brands. If functions are not shared, one needs to ask why this is. Certainly one would imagine there would be plenty of synergistic opportunities for sharing functions, resulting in better efficiencies and cost savings.

If for whatever reason the business wishes to operate these subsidiaries independently (this could be because the business has different long term strategic plans for the future of each business, or simply because they wish each business to be able to operate and evolve independently without influence from the other), then we are looking at two separate SOAs - one for each subsidiary.

This doesn't mean going through the exercise of designing and building two architectures. One would presumably put together a single architecture and then tailor it for each subsidiary. One would develop systems supporting that architecture and simply deploy two instances - one for each subsidiary. Now each subsidiary can now evolve independently.

Now, onto the service model itself. In the example, there are three core functions performed by the organisation - marketing and recruiting (which are performed by the Sales department), and credentialing (performed by the Credentialing department).

In this scenario, we need to understand whether the Marketing and Recruitment business processes have sufficient loose coupling and high cohesion to be implemented as separate services. If these processes share a lot of business logic and the information models of these processes have a lot of similarities, then we should implement a single Sales service to support both business processes. We would probably consider implementing these as autonomous components within the Sales service.

Alternatively, we would implement separate Marketing and Recruitment services. Either way, it seems would should have a separate Credentialing service as this is performed by a different department, so I would assume the Credentialing business process is quite independent from the other business processes.

Ayende's proposed solution however outlined eight separate services. At closer inspection, there appears to be somewhat low cohesion between these services. That is, some of them would likely be described using the same domain models.

In fact, the defined services seem to relate more to operations that may be exposed by a service, as opposed to services in their own right. For example, there is an Available Candidates service as well as a Register New Candidate service. Both of these services deal with candidates, supporting different aspects of the overall Recruitment process.

Consequently with this service model we would likely end up with chatty, synchronous request-reply, data-centric operations exposed by each of the services. Pursuing coarser grained services centred around cohesive business processes as suggested above would deliver a more loosely coupled service model.

Swivel Chair Integration is Bad

Recently, we took a look at process mapping as a means to achieve a dynamic rather than static view of the business. We broke down high level process maps into lower level process maps that fell into specific cohesive business areas.

One aspect of business processes that we have not yet explored is the actors executing the activities within each process. Business process is executed both by people and IT systems. Years ago when we didn't have computer systems, all business processes were executed manually. However, these days we automate as much process as possible in order to gain efficiencies and minimise human error.

In my last post, we explored the mechanisms that may be employed in order to orchestrate manual activities with those that are automated by IT systems within a service. But what happens if manual activities within different services must be performed by the same person?

If each service employs different applications through which users interface with the service, then the same user will need to use multiple different applications to do their job. In fact, if two adjacent activities on a process map are performed by the same user across two different applications in two different services, then users may find themselves flipping between applications, sometimes copying and pasting data between windows.

This is what's known as "swivel chair integration" and should be avoided where possible as it is inefficient, error prone, and provides a very poor experience for the end user.

So, we have multiple services with which a single user must interact in support of a given business process, but we wish to do this in a way that provides a single integrated user experience. How do we do this? With a composite application! Different modules within the composite application interact with different services but the functionality for all modules is presented to the user through a single unified interface.

Something to remember is that applications are centred around user experience, while services are centred around cohesive business areas. As it is very likely that we will have user workflows spanning multiple services, composite applications become critical in many SOA implementations.

Human Workflow and SOA

As I have previously stated, many services include both IT systems and people. Business processes within a service are comprised of individual activities that are performed either by an IT system or a person. Activities performed by people are either completely manual (meaning they are performed outside the context of any IT system - for example mailing a document to a customer), or partially manual (meaning they involve a person interacting with a computer system to complete the activity). Activities performed by IT systems are fully automated.

We need a mechanism for coordinating manual activities with automated activities in support of any given business process. One way this can be achieved is to leverage a human workflow system (HWS). In this case, any service involving manual activities would have an HWS behind its service boundary to coordinate the manual and automated activities contained within business processes supported by that service. An HWS allows us to establish worker roles and assign tasks to these roles to be later accepted by workers within each role for completion.

Usually what we see however is the HWS becomes the centrepiece of the service. All service logic executes within the HWS. You become bound to the HWS as your technology platform, which limits your overall flexibility and leaves you somewhat at the mercy of your HWS vendor.

Ideally what we would like to do is rely on the HWS only for assigning tasks to worker roles, managing work queues, notifying workers when new tasks have arrived or been accepted and closing tasks off. This then gives us the freedom to implement the remainder of the service logic however we like and on whatever platform we deem appropriate.

Consider a Policy Administration service which amongst other processes supports the process of writing a new insurance policy. Let us assume that this service includes a Web application for managing policies. A user enters the policy information into the Web interface.

The application logic then assesses the risk and decides that the policy must be referred to a risk assessor. The application then allocates a unique task ID (a GUID is good for this purpose) and sends a message to the HWS (containing the task ID), instructing it to open a new workflow task for a risk assessor, then storing the task ID against the policy in its database.

The HWS places the new task on the shared risk assessor work queue and notifies all risk assessors that a new task has arrived. One of the risk assessors accepts the task via the HWS interface and then actions that task. The HWS client application then spins up a Web browser pointing at the referred policy ready for assessment.

The risk assessor then reviews the risk profile and either approves or declines the policy via the policy administration application. The application then grabs the previously stored task ID for the referral and sends a message to the HWS to close the outstanding task.

One nice thing about this approach is that we remove our dependency on the HWS. If the HWS is unavailable, a risk assessor can still manually look up the policy and approve or decline it and the human workflow task would eventually be closed when it processes the message to do so (sent asynchronously by the policy administration application).

Furthermore, once we have removed all business logic from the responsibilities held by the HWS, we are in a better position to share the HWS between all services.

One key benefit of leveraging a single HWS enterprise wide is where we have the same worker being assigned tasks from multiple services. If there are multiple human workflow systems in operation, then a single worker may need to have multiple HWS clients operating on his or her desktop simultaneously.

In the end, the decision lies with the architect to determine the most appropriate approach for any given service. It may be that hosting an entire service inside an HWS is the most appropriate option based on features offered by the HWS. As usual, there is no one right approach.

White Box Services are Bad

We've recently been discussing domain modelling and process mapping in the context of cohesion and coupling. It is important to note that the domain model of a service and the business processes it supports are implementation details of the service. The only touch points with the outside world should be explicitly defined messages that describe events that occur within business processes and/or commands accepted by the service.

As long as the exposed events and accepted commands do not change, we should be able to update our domain model and business processes of a service without impacting its consumers. When we have service contracts describing messages that are defined in terms of the service domain model, we are leaking internal implementation details outside the service and thus increasing coupling between services.

Simply having a service contract is insufficient to guarantee loose coupling if that contract has strong dependencies on (and such is highly correlated with) the service domain model. This is yet another reason to avoid data centric service contracts, as the service domain model has a tendency to leak into the contract.

Process Mapping

In my last post we discussed domain modelling in the context of SOA, specifically addressing coupling and cohesion. Domain modelling however provides only a static view of our business model. It does not describe how business entities interact over time. For this we look to process mapping.

Now before we get started I'd like to preface this discussion with the disclaimer that I am no process mapping expert. So take what you read here somewhat with a grain of salt.

We can break down the business processes of an organisation in any number of ways. We could for instance start with very high level process maps and then iteratively decompose into lower level process maps until we get down to individual business activities that require no further decomposition.

This process however is not deterministic in nature. Two different business architects independently performing this exercise could produce different sets of completely accurate process maps for the same business. The difference between the two sets of process maps would primarily be where the borders are drawn between process maps. One architect may place logic in one process map that the other may place in another. Overall the same business process is described, but it is broken down in different ways.

As it turns out, there is quite a lot of symmetry between this process mapping exercise and the process of identifying and defining services. That is, we want to define process maps that have high cohesion and loose coupling. Failing to do so results in unstable process maps.

When we have the concerns of a single cohesive business area (described by a cohesive domain model) spread over a number of process maps, then making a change in that single business area will result in a need to update many different process maps. That is, we now have coupling between our process maps. Consequently process maps slowly get out of date due to the amount of effort necessary to keep them in sync with the business.

What we want to do is ensure that each low level process map describes a business process within a single specific cohesive business area. We can use high level process maps to describe how each of these low level process maps interact in support of business processes that span multiple business areas.

For example, consider the following extremely oversimplified high level process map:


We need to assess whether each part of this process pertains to the same cohesive business area. If not, we must subdivide the process further. Let us assume that for this insurance business we decide that the cohesive business areas relevant to this process are Customer Management, Policy Administration and Billing.

We will end up with three cohesive business processes in the following business areas:

Policy Administration


Customer Management


Billing


When it then comes to defining our service model, each cohesive business area becomes a single service. So in the above example, we end up with three services. The domain model and low level process maps that describe that business area then define the responsibilities of that service. The high level process maps tell us how our services must interact, thus helping us define our service contracts.

Note that the above example is quite considerably oversimplified. Real world examples get a lot more complicated very quickly. Process mapping is an extremely challenging task and is not easily done well.

We must be careful with these process mapping and domain modelling exercises not to get caught up in analysis paralysis. Keep the scope of an SOA project small and deliver incrementally. That way you are only ever focussed on the parts of each business area relevant to the specific deliverable at hand.