Tuesday, May 27, 2008

Enterprise Canonical Schemas

There are many references in the available guidance on SOA that recommends the use of an enterprise canonical schema. The idea is that you standardise all data transferred between services via messages such that they conform to a single canonical schema.

The process of producing such a schema is known as schema rationalisation. It involves forming an agreement between all relevant stakeholders in the enterprise on a universal representation of all entities (such as customers, orders, etc) relevant to the business.

Any outbound messages from a service are first translated to be consistent with the canonical schema before being sent. Likewise, any messages inbound for a service are first translated in that service's native message format before being delivered.

The purpose of this approach is to decouple interacting services from the native message formats of each service and to minimise the volume of message translation logic necessary to support the architecture.

If we have n services in our enterprise, then we have up to n(n - 1) interactions that require inbound and outbound message translations between native service message formats. Through use of a canonical schema, we only have 2n message transformations that need to occur.

So this all sounds really good right? Well it depends on the flavour of SOA in your enterprise. This approach isn't really going to add much value in a JBOWS architecture. JBOWS architectures don't add much value at all, regardless of the presence of a canonical schema.

But what about Service Oriented Integration (SOI)? Well as it turns out, a canonical schema is very useful here. With this flavour of SOA, applications are exposed directly as services in your SOA. As a result, you don't really have a lot of control over the syntax of messages understood by your services.

Message schemas tend to be quite volatile with SOI as they are based on the applications in your enterprise, rather than the business processes they support. A canonical schema decouples services by shielding them from message syntax variations in other services.

Furthermore, with SOI we may have multiple similar applications participating directly as services in any given architecture. For example, an enterprise may make use of two different CRM packages, which would by extension mean we have two CRM services. Because with SOI business processes are generally executed in middleware sitting between services, when a service sends a message it may not in fact know which CRM instance the message is bound for.

With a canonical schema, this decision is not relevant in constructing the outbound message. The middleware translates the message to the native format of the appropriate recipient as the message is delivered.

A canonical schema is also very useful (in fact one may say essential) when using a layered service model. Entity services abstract native data repositories into a form which is then used by task and process services. By ensuring data exposed by entity services conform to a canonical schema, we simplify our task and process services.

So what about self-contained process-centric service models? Well here a canonical schema actually works against you. With this flavour of SOA, you've already done the hard yards of building service contracts based on cohesive business areas. Your service contracts are aligned with the business (as opposed to technology artefacts) and as such are relatively stable.

Furthermore, with this flavour of SOA each service contract has been tailored to express information optimised for a specific business area. By placing a canonical schema between services, you diminish the semantic fidelity of communication between those services, mitigating the value of all that hard work.

A canonical schema not only standardises the syntax of communications between services. It must also by extension standardise its meaning, or semantics. An enterprise canonical schema tries to build a common language to suit the entire enterprise. All discussions between services are first translated into this language.

This is very likely to be a lossy transformation. Even if you don't lose data, you'll definitely lose information through blurring of semantics.

So as long as you go down the road of self-contained process-centric services, you should stay clear of enterprise canonical schemas. However, a canonical schema can certainly add value if you find yourself in an existing SOI or layered service model environment.


Captain Ramen said...

Sorry for going off topic but I couldn't figure out how else to contact you. Currently my system architecture uses command messages to get most of the work done. It is entirely driven by external events, e.g. Windows Task Scheduler invoking a Controller/Action in Monorail or some component outside of my control hitting a WCF endpoint.

However I have identified some fragile areas in my system, e.g. one of these external components beyond my control is not guaranteed to deliver a message (and says so in the contract) so I need a backup method to get that data. In this case i would raise an ItemNotFoundEvent.

The service bus would then get a list of IEventHandler of ItemNotFoundEvent from the Kernel and invoke them asynchronously. Before I get started I want to know if you see any problems with this approach. Would it be better to explicitly subscribe to events in my configuration file using string matching?

Bill said...

Just so I make sure I understand you correctly, are you saying that you have a third party component that sends a command message to your WCF endpoint, but does so in such a way that the message is not guaranteed to be delivered?

Can you tell me a bit more about this third party component? Is it an application? Or is it a .NET assembly? Something else perhaps?

Also, when exactly is an ItemNotFoundEvent raised? Who raises this event and under what circumstances? Who is subscribed to this event?


sathish said...

Thanks for the sharing of such information.This is a great stuff of reading. I will pass it on to our audience. Thanking you,CRM Services

Kevin Matin said...

Great Information. That sounds pretty cool. Really helpful thanks for the Article, Great job, Keep posting interesting matters here. Looking forward to it. Thanks and keep it up! All the Best.
Enterprise application development