Wednesday, April 9, 2008

SOA and Referential Integrity

One of the edicts of SOA is that services shall not share data except by way of exchanging explicitly defined messages. One of the questions that tends to pop up quite often then is how we maintain referential integrity between services.

The question stems from the fact that database systems do not allow us to set up foreign key constraints that cross database instances. Thus, with autonomous services potentially all housing their own independent database instances, how do we ensure that a record in one service will reference a valid record in another?

Well the fact is that we can't. But fortunately for us, we actually don't want to. If we were to establish constraints crossing service boundaries, then we would be creating unwanted coupling between services. Moreover, the database instances would need to communicate directly with each other in order to enforce these constraints which would violate our service boundaries.

As it turns out, we don't have to worry about referential integrity between services when we have a decentralised data architecture.

With a centralised data architecture, we have our services pick up their data from other services using CRUD interfaces. We might have a Customer service housing all our customer data and an Order service housing all our order data.

With this approach, we have the ability to create orders in the Order service against a customer ID that doesn't exist in the Customer service. This of course is a problem.

With a decentralised data architecture however, the Order service will contain basic customer data that it would have picked up through receipt of event messages from the Customer service. This way, if we then attempt to create an order for an invalid customer, the foreign key constraint that we would have established between the Customer and Order table in the Order service database would prevent this from occurring.

Something else to consider is that because different services in an enterprise may fall under different ownership domains, they may have different audit requirements. One department may decide that they do not need to keep records, and as such the service is allowed to delete them.

If we had referential integrity enforced between service database instances, then we would no longer be able to offer this flexibility. The audit requirements of one department would then influence another department which would not be well received.

So in conclusion, the fact that we can't have referential integrity enforced between services is a moot point. We don't want it anyway.


Anonymous said...

Let's say that two services each have their own concept of a customer. When these systems exchange messages that involve information about a specific customer, should they not have a unique customer key or something like that in common?


Bill said...

Absolutely. We just don't want to have a referential integrity constraint enforced between services based on that customer key.

Services must be free to enforce their own rules as masters of their own domains.

One service should not be able to prevent another service from deleting any record it sees fit to delete.

If a service publishes an event message containing information about a customer (identified by the customer key) and a subscribing service has no record of that customer, then it is up to the subscribing service how to deal with that condition. It is not a concern of the publishing service.

Assuming the subscribing service is not faulty, then it must permit the deletion of customer records. Otherwise the record would still be there.

As such, it should handle the condition of an event message arriving with a customer key that doesn't exist. One possible course of action would be to ignore the message.

IanIan said...

So should the Order service make its own copy of the Customer?

Naveen Soni said...

Web Design Poole :We are Top Web Design Company in Bournemouth. We offer Website Design in Poole, Bournemouth & Hampshire. Visit us for cheap website design services.

Anonymous said...

You've forgotten about race conditions: just because a registered customer previously existed does not mean they still exist! That last order your service processed for customer Z 3 weeks ago was just 2 days before they cancelled their account with you.

IanIan said...
This comment has been removed by the author.
sefi said...

q9h18n1m29 f8m73k0u53 b6a39q6p15 f2f53s2k16 x0w38s1k43 s9d40y8v93