When Event Messages are not Ideal

In a recent post, I discussed why we should prefer the use of event messages over command messages. There are however situations where command messages are more appropriate.

Say for instance we have a billing service that has the ability to raise invoices. Let us also say that there in a large number other services in the enterprise supporting business processes that at some point require an invoice to be raised. Let us then assume that the action of raising the invoice is to be exactly the same, or at least remarkably similar in all cases.

Using event messages in this scenario would lead us to having the billing service subscribe to different event messages from a number of different services, all of which resulting in the exact same operation in the billing service. This is is somewhat wasteful as the billing service needs to be aware of and process a large number of event message types.

As the required behaviour of the raise invoice operation will not vary between contexts, the context of the originating process is mostly irrelevant. This makes the raise invoice operation of the billing service highly reusable.

As such, we should strongly consider using a single command message sent by all other services to the billing service to invoke the raise invoice operation.

We need to be careful however that the reuse here is mandated by the business, rather than coincidental. If we find that in the future each of these services start having different needs of the billing service in raising invoices, we will greatly overcomplicate this operation and start needing to update the command message structure and/or billing service implementation to compensate. We start suffering the effects of coupling again.

In the field we tend to find that the context of the originating process influences the behaviour of the target service most of the time. Where it doesn't initially, we tend to find that it does at some point in the future. That is why it is best in general to prefer the use of event messages over command messages.

SOA and Reuse

One of the key benefits often touted by SOA practitioners is reuse. It is usually described in the following way:

A business will establish an asset of a number of services over time. When the need arises to implement some new business process, we simply use a process orchestration engine to wire up our existing services in support of the new business process.

The concept is certainly appealing. The ability to be able to support entirely new business processes so quickly, cheaply and easily certainly would drive a lot of business agility. But is this nirvana actually achievable? As usual, the devil is in the detail.

The kind of reuse described here is the kind that we see in traditional programming. One component exposes a function which many other components can invoke, passing parameters to control the execution of the shared component.

This model is not appropriate for SOA because of the coarse grained nature of services and the need for asynchrony. It is also problematic because the calling component is instructing the shared component what to do. This translates to the use of command messages in SOA, which as previously discussed introduces coupling between our services. It is in fact this coupling that prevents us from reusing the shared service effectively.

Component oriented programming has taught us that the larger the granularity of the shared component, the harder it is to reuse in different contexts. In SOA terms, the different contexts we are talking about are the different services sending the shared service the command messages.

When we have Service A sending Service B a command message, Service B is performing a function in the context of the activities occurring in Service A. If we now reuse Service B by having Service C send Service B the same command (albeit perhaps with different content), Service B is now performing a function in the context of the activities occurring in Service C as well.

The problem here is in the intent of the command message. Services A and C are instructing Service B what to do in support of Service A and C's activities respectively. A change in the needs of Service A or C can potentially lead to the need for a change in the implementation of Service B.

As we add more and more services reusing Service B in this way, Service B will become increasingly confused and complex, attempting to cater for the different needs of all the services reusing it.

Consequently, we need to think about reuse in SOA differently than with traditional programming. With SOA the unit of reuse should be the event, not the function. As our SOA matures, we evolve an event catalogue where each event has business relevance (such as a sale event or policy cancelled event).

We can then add a new process that subscribes to relevant business events and potentially publishes new events. If necessary, services are then updated to respond to new events published by the new process.

Now I can hear many of you saying, with the approach described in the opening paragraph we don't have to update any of our existing services - we just wire them up with middleware to support a new process. Well, as previously discussed that is a false promise as the service operations you are attempting to reuse will very likely need to change in the context of different processes leveraging it. So it is likely they will need to be updated anyway.

Furthermore, if a business process supported by an existing service needs to be updated to respond to a new event, then it is completely reasonable that we update the service implementation. That is where the change has taken place! By wiring up existing services with process orchestrations in the middleware you are effectively leaking your service logic outside your service boundary. I'll discuss this in more detail in a future post.

The reason why the event driven approach does not suffer from the issues of the command message approach is that the service publishing the event does not make any assumptions as to which services(s) are subscribed or how they will behave in response to the event.

In conclusion, SOA does indeed deliver reuse in that any given business logic is implemented in one and only one service. Services are written once and then they evolve over time. They are not necessarily static pieces of functionality that are then composed to deliver new value. So although reuse is a key benefit of SOA, the kind of reuse often touted by SOA practitioners is often not achievable.

Mandated vs. Coincidental Reuse

Developers and architects alike often strive to achieve the holy grail of reuse in the systems they design. It makes sense - reuse is a worthy goal. It means we get repeated value from the software we implement. It also means the total volume of software is reduced, reducing the amount of software that needs to be maintained. This means less effort for developers and lower costs to the people footing the bill.

But reuse must be employed with caution. Often in my travels I have seen reuse become a goal unto itself, rather than designing a system that is aligned with the problem domain. Developers often see patterns where the same logic appears multiple times and aim to extract that logic out into a common place to be reused, despite there being no mandate in the context of the business that the logic in fact be the same for all time.

If it is not mandated in the context of the business that the logic be the same, then we very likely will end up shooting ourselves in the foot by reusing it.

Business requirements are fluid. This is why loose coupling is so crucial in software systems. It allows us to better cater for change. Once a system has been deployed, it must then evolve to support the changing needs of the business. We need to ask whether the business may require that logic duplicated in multiple areas be allowed to evolve independently. If so, the logic should not be reused.

Consider an insurance system that manages policies within number of different product families. The logic that is in common between the product families at the time the system is first developed is not necessarily the logic that will be in common in the future - especially if the business expects to be able to evolve the rules of these product families independently. We must identify which rules very likely will be in common for all time and reuse only the logic corresponding to those rules.

Note that this guidance applies not only to the reuse of business logic, but also the reuse of data representation. This is one of the reasons why we do not allow services to share data directly. The representation of a customer in one service must be allowed to evolve independently to the representation in another. If it is mandated by the business that the representations be the same, then it is likely that the two services in fact should be one.

Maintainability is the ultimate dimension of software quality to which we as architects or developers must aspire. Where reuse improves maintainability, then it is appropriate. However when it hurts maintainability, even though it means we must write the same logic multiple times, it should be avoided.

.NET Community of Practice Session Slides

Thank you to all those who attended the .NET Community of Practice session on Thursday evening. For those who missed it, you can download the presentation slides here.

Avoid Command Messages

According to Hohpe and Wolfe in their book Enterprise Integration Patterns, they state that there are three basic types of messages at a semantic level - command messages, document messages, and event messages.

When discriminating between these message types, we are concerned with the semantics of the message name. So for example, a CancelPolicyRequest message is a command message, as it is instructing the service to cancel a policy. A Policy message is a document message, as it contains information about a business document without any context as to what should be done when it is received. And a PolicyCancelledNotification message is an event message as it informs the receiving service that a policy was cancelled, but does not specify what action the receiving service should take in response to the event.

Although command messages do not constitute an RPC interaction, they introduce coupling between services where the other two message types do not. When Service A sends a command message to Service B, Service A is making a decision on behalf of Service B as to what Service B should do in the context of Service A's activities.

The fact that Service A is instructing Service B what to do means that Service A determines what shall be done, whereas Service B decides how it shall be done. This is a subtle but important form of coupling.

Because Service B is performing an operation for Service A, as Service A evolves we may find that Service B is no longer able to meet the needs of Service A. Now we must update Service B due to a change in the needs of Service A. This is the essence of coupling.

The root of the problem here is that Service B's behaviour is being governed by Service A. Service A is making an assumption regarding Service B's behaviour, introducing a dependency.

Moreover, Service B cannot make up its own mind what to do in response to Service A's activities, it must be instructed by Service A.

Consider the case where Service C now must perform some action in the context of Service A's activities. We now need to update Service A to send a command message to Service C as well. Imagine how the complexity here can grow when we have a large number of services!

The solution is to use message types that do not involve instructing a service how to behave. This leaves us with the document and event message types. The document message type tends to appear mostly with the REST style of architecture (which I will discuss in a future post).

With SOA, the preference is to use event messages. In the previous example, Service A would publish an event (such as InvoicePastDueNotification) to which Service B would be subscribed. Upon receipt of the notification, Service B would then make the decision locally as to how to respond to this event; in this case, cancelling the policy. Service B would then very likely publish a PolicyCancelledNotification message, in case other services needed to respond in some way to this event. If the need arose in the future for Service B to respond differently, this would involve only a change in Service B.

If Service C then needed to perform some action (say claw back commissions) in response to the InvoicePastDueNotification message published by Service A, we would simply need to subscribe Service C to the relevant event topic, and then update Service C to behave as needed. Again we have not needed to make a change in Service A.

Here we can see a business workflow occurring that spans multiple services, where the decisions as to how each service contributes to the process are handled locally within each service.

As always however there is a catch. What if in order for Service B to decide how to respond to the event published by Service A, there is insufficient data in the event message? This would mean we would need to update Service A to include more data in the event message - once again we have coupling.

As such, we need to be careful when designing our event messages such that all relevant information regarding the event is included in the message. The needs of the subscribers are not known in advance, so we cannot just include the minimum information in the event message to satisfy the existing intended subscribers.

So in conclusion, prefer the use of event messages over command messages at the service boundary where possible; but take care in the design of your event messages to make sure all relevant information is included.

Reminder: .NET Community of Practice Session

This is just a reminder to join me this Thursday evening as I present to the Perth .NET Community of Practice on SOA design patterns and best practice. The session will cut through much of the hype surrounding SOA and deliver some clear and practical guidance on how to design and build services on the Microsoft platform. Details below:

DATE: April 3, 5:30pm
VENUE: Excom, Level 2, 23 Barrack Street, Perth
COST: Free. All Welcome

What is a Topic?

Publish-subscribe is an asynchronous messaging paradigm, where messages are addressed to topics, rather than specific recipients. All subscribers subscribed to a given topic then receive a copy of any event message published on that topic.

It is common for publish-subscribe infrastructure to involve published messages being sent to an intermediary broker. The broker manages the subscriptions and routing of messages to the appropriate subscribers. Such a broker would normally queue published messages for later delivery where subscribers are unavailable.

In other implementations, the message routing function is pushed out directly to the end points thus eliminating the need for the broker. In these cases, multiple copies of the message are sent directly from the publisher to each of the subscribers. This can be optimised at the network layer by associating topics with IP multicast groups so only a single copy of a published message is sent over the wire.

As explained in my post on defining an endpoint, the endpoints of a service are the endpoints of the channels controlled or owned by that service. In the case where an asynchronous queued transport such as MSMQ is employed, the endpoint will have an associated physical queue associated with it from which the service reads the messages addressed to that endpoint. The endpoint address shares the address of the queue. Moreover, the queue is owned by the endpoint. That is, it should not be shared between endpoints, and certainly not between services.

In the case of a publish-subscribe endpoint, the endpoint has an associated topic onto which the service publishes messages. The topic is owned by the service endpoint, and the endpoint address in this case is the address of the topic. As was the case with a queue, a topic should not be shared between endpoints, and certainly not between services.

In some publish/subscribe implementations, topics can be arranged in a hierarchy. Subscriptions can then be performed against a topic wildcard, rather than being limited to only one topic. This means a subscriber can receive notifications published to a topic as well as any topic underneath it in the hierarchy.

For instance we may have the following topics:



A subscriber that subscribes to org.sales.* will be sent all messages published to org.sales and all of its child topics, whereas a subscription to org.sales will only result in receipt of messages published specifically on that topic.