Sunday, May 25, 2008

Sharing Concerns between Services

Graeme recently posted a good question regarding sharing of common concerns between services:

"Let's say that a service creates a document but the enterprise as a whole has an investment in a DMS. Would you see the DMS as a centralised data store that should be de-centralised?

Most applications I've seen would just create the document and then call a WS to store it inside the DMS but that goes against the autonomous nature of the service.

Would be interested in your thoughts on this."

There are two alternatives to approaching this problem, and as usual it depends on your specific business requirements.

The first alternative is to consider the DMS application as a distributed component shared by multiple services. In this case, the DMS is not a service in its own right directly accessible by the enterprise-wide service bus. It is a private component of the services leveraging it, sitting inside their respective service boundaries. As such, synchronous request-reply communication with the DMS would be quite acceptable.

There isn't really any problem in sharing a distributed component between services, so long as data is not mixed between services inside that component. Services should share data only by way of explicit message exchange.

You just need to be aware that if the shared component falls over, all dependent services will fail.

A good acid test to determine whether this solution is appropriate is to consider whether the documents managed by each service should be isolated from each other. So if the documents produced by multiple services should be managed together (for example stored together in the same logical folders), then this indicates that this approach is most likely inappropriate.

The alternative is to have a separate autonomous Document Management Service that interacts with your other services by exchanging messages in accordance with their public service contracts. In this case, the DMS application is a private component only of this service. In my experience, this is the better approach.

This particular approach can be tackled in a couple of different ways. Assuming each of the various services in your enterprise generate their own documents, they could either asynchronously send the Document Management Service a command message (such as a StoreDocumentRequest message) containing the document, or publish a business relevant event message (such as a PolicyCancelledNotification message) with the document attached.

With the command message approach, the message would contain any relevant metadata to store against the document. With the event message approach, the Document Management Service would extract the relevant metadata from the event message body.

Although I usually avoid command messages, they are useful and appropriate when the service operation is highly reusable.

Filing a document in a document repository would be a suitable candidate for a command message. That is, its behaviour would be unlikely to differ based on the context of the service sending the command message.

If your users interface with your services via a composite application, then you can have a document management module in that application that allows your users to interface with the Document Management Service in such a way that provides a seamless integrated user experience.

Something else to consider in this area is the generation of documents. If you have software that generates your business documents based on end user managed templates, then this would be a good candidate for a component to be placed in your Document Management Service.

The document management module in your composite application would provide the ability for your users to manually generate documents by sending messages to the document generation component. This component would place the generated document in an output folder and return the URL to the composite application in the reply message. The application would then retrieve and display the document at that URL.

Yes, this interaction between the composite application and document generation component is synchronous request-reply. However this is perfectly acceptable because both the document management module and back end component are behind the boundary of the same service.

If after generating the document the user decides to store the document in the repository, then this would be a separate message from the composite application to the document repository component, which would contain the document URL and associated metadata. The document repository component would pick up the document at the given URL and store it in the repository with the given metadata.

So we have two ways in which documents can be generated and stored. One is where the document is being automatically generated by a service, where the service can send a command message containing the document and associated metadata to the Document Management Service (or publish an event message containing the document onto the service bus to be picked up by the Document Management Service).

The other way is where a user manually generates the document via the document management module within your composite application. The composite application is then responsible for sending the message to the document repository component at the user's request.


Andreas Öhlund said...

Nice post! (as usual)

Our customers are mainly in the government sector so I often work with case management and "document producing" projects.

In almost all our projects the customer has a public web site that hosts some of the functionality, eg. enables users to login and submit a request for a house building permit. I wonder how you would view the "extranet" in a SOA context. Should we:

1: Treat the extranet as an independent SOA-service that publish "NewHousePermitRequestEvents" that a HousingDepartmentService subscribes to. (the extranet often house other public services not related to each other)

2: Consider each different part of the extranet to exist inside the service boundary of it's corresponding BusinessService, ie. treat the extranet as "composite application".

I tend to favor option 1, what would be you suggestion?

Bill said...

Hi Andreas,

The extranet is effectively a user interface and as such is the means by which the user participates in one or more services.

If the user participates in multiple services by way of a UI delivered by a single Web application, then that Web application indeed is a composite application.

The extranet should have no bearing on identifying your services. Your services should be centred around cohesive business capabilities at the appropriate level of granularity.

Once your services are defined, you then determine in which services external users must participate.

It may be the case that different users will participate in different services. As such, it would be quite acceptable to have an entirely different Web application for each service.

However if the same user must participate in multiple services, and you wish to give the user an integrated user experience, then you must produce a composite Web application that belongs to multiple services.

As such, I favour option 2. An extranet Web application should not be considered a service in and of itself. It is an application.

Naming can give a bit of insight into the reasoning for this. The extranet doesn't necessarily support any given specific cohesive business capability.

I'm guessing you don't have a business process or capability called "extranet". So you wouldn't have a service named "Extranet Service".

Case management however is a business capability that may be appropriate for a business service.

In this case you would have a Case Mananagement Service that would have an extranet component.


Andreas Öhlund said...

Hi Bill!

I think you're right! I might have dreamt up a "need to collaborate with citizens"-capability to justify making the extranet a "SOA-service".

But haveing a ExtranetService is by experience a nice way to interact with the extranet.(DMZ:s firewalls etc) So what role would you say that my ExtranetService plays?

A technical service wrapper for different CMS products (COTS wrapper)?

Service Oriented Integration?

Or (hopefully not) a bad idea? :)

Bill said...

I suppose it is just a matter of defining service boundaries. So I would call the extranet a composite Web application or a portal.

Web services exposed at your organisational boundary usually are integration points for other organisations rather than service boundaries.

So although you might consider such a mechanism a service wrapper (by virtue of the fact it wraps a COTS application with a Web service interface), I wouldn't consider such a wrapper to sit at the service boundary - unless pursuing a service oriented integration strategy.

Something to consider as well is what business logic the extranet actually contains. If it contains only user interface workflow logic, some simple validation rules and/or a service wrapper, then you definitely should not consider it a service.

If however it contains a substantial amount of business logic, then you'd need to consider what business area the logic belongs to and name the service accordingly.

Things such as firewalls and DMZs should not really impact your service boundaries when using a self-contained process-centric service model.

The service boundary in this case is determined by the business processes supported by the service, which are not affected by deployment issues such as firewalls and DMZs.

There is nothing wrong with having a single service spanning your DMZ, external and internal networks, so long as the service has high cohesion and is loosely coupled with other services.


Robert said...

You've written about services, in effect, encapsulating the data they need and if more then one service needs the data then one is the owner and the others share the data by some means, i.e. messaging.

Let's say two services A and B both need a certain set of data (C) where A is the owner. Let's also say I choose to persist this data ONLY with A and make it available to B (or others) when service A starts.

Is it better/more natural for A to broadcast the data when it starts with a message such as data-C-available or for B to broadcast a message when it starts saying I-need-data-C and wait for the reply.

Bill said...

Hi Robert,

Why is it that you only want to broadcast information from service A when it starts?

What if Service A is running for years without restart? Your other services will never be updated.

You should be attempting to making information available to your services in the most timely way possible - that is, using real time events.

If data in Service A needs to go through some kind of approval first, then you can have Service B subscribe to the approval event and pick up the necessary information from there if needs be.

If I had to pick between the two options you present, then I'd go for the former. However, data-C-available as an event is very data centric. Your events should be business/process centric.

The notification message should convey the semantics that something has happened, not that data is available.


Anonymous said...

The broadcast when service start may not be as on point as I'd like.

Let's say you have a shipping service that can ship a package to any of the three carriers: DHL, FedEx, UPS. This list of carriers is not likely to change often.

There is a UI that allows a user to ship a package. On the UI the user can select the carrier (DHL,FDX,UPS) from a dropdown.

Where does the UI get the list of carriers from? It would seem that the shipping service would be a likely candidate as it has knowledge of shipping related information though a message to GetAvaliableShippers seems not consistent with the spirit of a business event.

If it's not the shipping service, then what's likely to be the source of this knowledge which the shipping service must certainly know about. The set of known shippers must live somewhere. In your experience, where is it likely to live?

Bill said...

Hi Robert,

Where the UI gets the list of carriers from would depend on the function being performed by the user.

If in that particular screen, the user was performing a task that would result in a message being sent to the shipping service (containing the carrier code/ID), then you would want that list drawn from the shipping service to be sure.

However, if the user was engaging with another service via that screen, then you would want to get the list of carriers from that service.

The list of carriers is reference data that will be held by all services that refer to carriers in some way as part of their domain models.

You shouldn't only ever draw carriers from the shipping service because you could end up in a situation where a new carrier is added, but the addition hasn't yet been propagated to other services.

You then go and fire a message up to another service with the new carrier code/ID drawn from the list generated by the shipping service.

That service wouldn't recognise the carrier code/ID and would error out.

So in conclusion, always draw the lists for drop downs from the service in which the user is participating via that screen in the application.


Anonymous said...

What is an example of the business event (message) that you'd propose to retrieve the list of carriers from the appropriate service.

The reason I ask is that I often hear that 'get' is not an event though this scenario seems to warrant a GetAvailableCarriers event.

This seems very CRUD-like and doesn't have the business-centric feel of the other examples you use to illustrate events such as in the insurance policy example.

Bill said...

The communication between the UI and back end component occurs inside the business service boundary. As such, you're not concerning yourself with business relevant events.

Business relevant events are the means by which business services communicate with each other.

You don't want CRUDdy interfaces at the business service boundary. However within the service as a means of communication between a UI tier and application tier is fine.

You can read more about SOA and UIs here.