On the SOA(P) box
On the SOA(P) box
Ever wondered if your crumpets have been toasted in a fully compatible Services Oriented Architecture environment? Stollznow principal consultant Peter Webb trims the fat (as he chews it) off SOA hype in the first of whopping 24 part series on the history and significance of Service Oriented Architectures to Information Management.
Whenever I go along to one of my children’s speech nights at school, and some snotty-faced kid in a school uniform has to give a speech, they invariably start by reading out the dictionary definition of the key word in the speech title. And at that point, I always think “Thank God, this speech may be dull, but at least it will be short”.
Please, please imagine the horror to be faced if the topic was “Service Oriented Architecture”, and one of the most commonly used standard definitions of an SOA is “A paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains. It provides a uniform means to offer, discover, interact with and use capabilities to produce desired effects consistent with measurable preconditions and expectations.”
Here’s a news flash – by that definition, my toaster implements an SOA. It provides a distributed (to everybody in my home) and uniform means to discover (its always next to the bread) the toast making service, interact and use (you shove bread or crumpets in it and hit the “toast” button) and it produces measurable, defined events that meet my expectations viz-a-viz snack food.
It’s obviously almost meaningless techno-babble. This is a pity because the SOA concept is very real and fundamental to how systems are developed now and in the future. It’s actually a great and practical idea and of very particular relevance to information managers, for reasons that will become more clear.
To understand what SOAs are – and why you should care – you first have to understand what problem they were trying to fix, which means how understanding how things used to be done.
Until quite recently, IT Departments were often comparatively relaxed about how imaging and EDMS systems operated internally, and gave business users relative freedom to pick systems almost purely on their merits as business systems. (I know it may not have appeared that way last time you spoke with your CIO, but it could have been far, far worse). The general attitude of IT Departments was that EDMS was a packaged, discrete business system, and as such should obey the same rules as other packaged software such as Finance systems or HR systems. This usually meant compliance with a few basic architectural requirements such as the server hardware platform, the database used, and possibly a few other similar low level technical requirements. Often even these were not rigorously enforced.
In the last few years, many IT Departments have introduced a new standard – new systems should operate in a “Service Oriented Architecture”. They have also determined (in my opinion, correctly) that if anything is to comply with SOA standards, the EDMS/ECM/imaging systems are probably the most important systems to do so. Both business and IT stand to gain a lot from SOAs, particularly in imaging/EDMS/workflow/ECM/CRM and those other systems so close to the heart of many of our readers.
In this, the first of a 24 part series on SOA concepts for Information Managers, we look at the historical context of SOA architecture and the problems they try to solve.
When couples drift apart
Service Oriented Architectures are essentially a tool to allow integration of different application systems – for example to allow Finance or HR systems to exchange information with EDMS systems, or vice-versa.
In the really-good-old-days (the age of the mainframe, say up until the early 1990s) this wasn’t a problem. All business systems ran on a single mainframe, it had standard mechanisms for sharing internal information, and there was very little external information. No systems integration of external systems, so no problem.
By the mid 1990s, this approach was seriously broken. In these not-so-good-old days, business users were busy purchasing packaged systems that weren’t available on the mainframe – mostly those nasty little Unix boxes with databases nobody of ever heard of – and asking IT to integrate these new systems with existing systems.
IT Departments generally responded by trying to limit the number and type of new external systems that Business could install. That they generally failed in this endeavour is a matter of historical record.
To try and allow systems integration, IT Departments introduced a series of “point-to-point” standards for integration. Probably the most important of these were network database integration standards, in particular the SQL/ODBC family of database calls. (Probably the worst was Windows DDE; younger readers will excuse me not passing a tear of nostalgia for its passing).
These integration standards allowed systems to intercommunicate, but had practical problems.
Firstly, these standards meant that systems were “closely coupled” – a change to one system might require modifications to lots of other systems. Consider for example the SQL/ODBC family of database calls mentioned previously. Almost all databases support this standard, but the calling systems have to “know” a lot about the internal operation of the database – the field names, field types, which fields are directly indexed (if you want any kind of performance), and things of that nature. It all worked, but it was hardly a rugged or flexible environment. Changes to a database schema on one system could cause multiple other systems to fail, and it was all hand-tooled to work.
Secondly, they were “point-to-point”, and suffered from what is known as the “n squared” problem. If you had four systems that needed to talk to each other, this was a pain but manageable – six point-to-point links would suffice. To fully interconnect 10 systems requires 45 links, for 20 systems this is 190 links, and generally it goes up as the square of the number of systems that need to be linked. One of my clients has 4,000 staff and 400 servers (don’t ask). They don’t all need to be directly cross-linked, but if they were, you would be looking at 79,800 discrete point-to-point services. A bank or insurance company may have a dozen different business systems for different types of accounts (savings, credit cards, fund management etc), which should talk to common Finance, CRM and fraud control systems and possibly each other. Or consider an internet travel booking site which may have to get rates and occupancy information from 10,000 hotels, each of which also has to talk to 20 other internet travel booking sites, each talking to a million users. You don’t need a calculator to tell you this isn’t going to work using a point-to-point meshed architecture.
By the mid to late 1990s, and partially driven by the explosion in internet it was clear that something new was needed.
The first issue – the low level coupling between different systems – could be resolved by inventing a new type of interface which operated at a higher level. For example, instead of a CRM system calling a customer account system at a low level – for example, issuing a call to “select” all unique customers with “first.name=John” and “last.name=Smith”, issueing a “join” command to link to various account information databases, then retrieving the account information, the CRM should be able to issue a call which says “tell me all about John Smith” and get back a structured list of all information held about John Smith, without needing to know anything about the underlying database that is being accessed. This could even be a full text database (as opposed to a relational database).
This concept became known as a “Service Oriented Architecture”, partly because it provides high level services (rather than low level calls), and partly because three letter acronyms are an industry standard. Or more correctly, the “loose coupling” concept became the SOA, and the elimination of mesh networks became the ESB (the three letter acronym for “Enterprise Services Bus”).
With a definition this vague, almost any software system could be said to adhere to an SOA, or at least be “SOA compliant”, just as showed above that toaster my toaster is as well. (Although, just for the record, my crumpets are not technically SOA compliant; though they certainly “used SOA principles in their design”, and are “fully compatible with SOA environments”, being two of the other phrases so beloved of computer salesmen).
But would software vendors use the vagueness of this definition to claim that their systems were “SOA based”, even if in reality this meant nothing at all about their system? Wouldn’t this contradict the high ethical standards of product disclosure and in-depth technical knowledge displayed by most computer salesmen?
If you believe that’s the case, then you can skip this column in future, and just buy a new toaster for your IT Department’s lunchroom. Way easier.
If you have any doubts, tune into the next instalment. We will leave the prosaic world of toaster-ovens far behind, and start considering what is at the heart of a “real” SOA – the XML standard and SOAP (Simple Object Access Protocol). This is where we get down and dirty; stay tuned.
To be continued ...
Peter Webb is the Principal and in fact only employee of Stollznow Consulting, so the views expressed in this column are those of both him and his employer. Stollznow Consulting provides independent consulting primarily relating to Enterprise Architecture, Information Management and Strategic Planning. Peter can be contacted on 0413 737509 or at pwebb@stollznow.com.au.