How do you unlock the value of the information your organization manages? Business integration is the key technique in unlocking this information. Business integration is the single biggest issue for end-users, and therefore, for the IT industry. Recent Gartner research in this area has highlighted the importance of “service-oriented architectures” in solving the business integration issue. There is a growing realization that integration is best solved using a service-oriented architecture.
This article discusses some historic approaches to building service-oriented architectures and outlines the key failures of previous technologies. In particular, this article shows how Web Services technologies, through the provision of ubiquitous, Internet-based, and document-oriented technologies, provide the compelling foundation for implementing Web Service-oriented architectures, and consequently, that Web Services represent the best solution to business integration.
The Problems in Business Integration
Business integration (that is, the task of tying many information systems together to support a business process) has historically been an extremely complex, costly, and time-consuming task. Dogged by a lack of standards in terms of programming languages, operating systems, application interfaces, and networking protocols, the traditional ways of integrating many diverse applications have necessitated small armies of systems integrators and Byzantine technology.
I would hazard a guess that many reading this article have sat in large rooms with walls covered in paper diagrams that depict possible flows among several applications and mock-ups of screen layouts, only to discover six months and several million dollars later that even getting your CRM system to synchronize data with SAP is proving to be a nightmare!
Service-Oriented Architecture
The technology industry, always quick to spot the commercial opportunity in helping customers solve these business integration problems, came up with the concept of service-oriented architecture. In theory, the idea is compelling in its simplicity. If we can wrap an application with a well-defined interface, it is possible to turn that monolithic application into a “service”. This wrapping process creates an abstraction or insulation layer. The wrapping layer hides all the nasty details of the application – no longer do I care what language, operating system, or database the application uses. The only thing that matters is the interface: the description of the service.
If all applications can appear on the network as a set of services, it is possible to plug all these services into a single information bus. The process of integration is now turned into the software equivalent of building a computer: I merely plug different applications into the bus, and presto, they can all talk to each another and all possible sets of integrations are possible.
Figure 1: Universal Web Services Information Bus
Nice Theory, but What about Reality?
Over the last 10 years three major (and related) attempts have been made at turning this vision into reality. CORBA, J2EE, and DCOM each supported in very explicit ways the concepts of service-oriented architecture. It is useful to look at the things those technologies got right, and perhaps more importantly, where they went wrong. CORBA, J2EE, and DCOM provided many live deployments of integrated applications using service-oriented architectures and so have increased the industry’s collective understanding of how to make a service-oriented architecture work. However, all three have highlighted four major areas where essential changes are needed before a service-oriented architecture can really deliver on its compelling “plug-and-play” vision.
Those four areas are:
- The interface language: In order for a service-oriented architecture to really take off, the language used to wrap the applications must have universal acceptance. In DCOM there is Microsoft-IDL, in CORBA there is CORBA-IDL, in J2EE there is Java. All are popular for sure, but none are universal. Anything short of universal means failure!
- The information bus: When plugging applications into an information bus, the bus must again be ubiquitous. Only one network meets that requirement: the Internet. J2EE, DCOM, and CORBA all provided their own popular, but again not universal information buses.
- The flexibility of the interface: One of the major lessons learned from previous work in service-oriented architecture was the need for flexibility in the interface. In other words, an interface needs to be able to evolve over time and have the capacity to respond to different message types. This is needed because, in the real world, the change of software environments is a feature of life. J2EE and CORBA adopted a position where the interface needed to be a very rigid entity, and not well disposed towards change. DCOM supported some notion of interface changes, but in such a complicated manner that it was extremely hard to use. This inability to evolve interfaces has made service-oriented architecture hard to deploy and manage.
- Distinct, separate interfacing and programming models: All three historical approaches to service-oriented architectures have combined two distinct concepts together: firstly, the concept of building an interface to a piece of software, and secondly, the concept of creating some programming logic to implement that interface. Therefore, if you want to use J2EE, DCOM, or CORBA, you are first-of-all required to learn their programming models and their very substantial and complex API set. To paraphrase Einstein, “every time you add an API, you halve the number of people who can use it.” This means that a lot of often unnecessary complexity has been introduced in creating service-oriented architectures.
Web Services deal comprehensively with each of these four issues.
1. The Interface Language: XML, WSDL, and Microsoft
First and foremost, a Web Services-oriented architecture has the benefit of a universally accepted interface language. Based on the now universally accepted XML, Web Services Definition Language (WSDL) has the backing of every single vendor on the planet. The fact that Microsoft and IBM originally came together to create this standard is of enormous importance. Together, they created the industry momentum that has powered the standard to universal acceptance.
2. The Internet: The Universal Information Bus
As significant as WSDL is, there is almost no comparison with the huge impact that the Internet plays in making a Web Service-oriented architecture a reality. When you consider the universal acceptance of protocols such as HTTP and SMTP, there are some immediate technical and economic observations that have to be made. Firstly, there is nowhere on the planet that cannot be reached by HTTP or SMTP. Because of this ubiquity, the efforts made to make these protocols secure, reliable, manageable, and highly available benefit from enormous and unparalleled economies of scale. There is no economically feasible way to make any other set of network protocols as universally dependable as HTTP and SMTP. Therefore, there is no other set of network protocols that can possibly form the basis for a universal integration bus.
Many will complain that HTTP and SMTP are not the ideal protocols on which to base an information bus. There may indeed be technical merit in those observations, however, they are irrelevant. The fact is that HTTP and SMTP, although not ideal, are perfectly adequate. To mix metaphors, the information bus runs on a dirt road!
3. The Flexibility of the Interface: The Document Metaphor
There are many in the software business who have already made some of the arguments above as to how Web Services technologies have helped move the whole industry along, by providing a universally agreed set of interface and information bus standards.
However, fewer people have really delved into one of the more subtle aspects of Web Services: how Web Services radically change the notion of what an interface is and therefore make it more open to flexible use.
- Complex Endpoints, Simple Data
Many of the architects behind DCOM, CORBA and J2EE were very focused on extending the analogies from hardware into software. When building hardware systems, we have a range of hardware components that can be wired together to create systems. Historically, service-oriented architectures have been driven by a notion of “pin-out diagrams”. There was a theoretical purity to the whole vision.
The J2EE, CORBA and DCOM interfaces are very complex and rigid – they try to capture all of the application behavior. In addition, there is an assumption that much of the data flowing between these rigid interfaces is the kind of data that can be represented in typical programming languages, such as integers, strings, and arrays. In short, the DCOM, CORBA, and J2EE approach to service-oriented architecture can be summarized as “complex endpoint, simple data”.
In practice, much as we would like software components to behave and evolve in a hardware-like manner, they just don’t. Software continues to display all the malleable characteristics that define it!
- Simple Endpoints, Rich Data
Many casual observers will conclude that WSDL is no more than an XML version of the J2EE, CORBA, and DCOM approaches to defining interfaces. This observation is dead wrong and very wide of the mark.
One of the most important innovations in Web Services is the whole notion of the “document paradigm”. Web Services leverage the whole XML document technology to the utmost – in fact, we believe that Web Services are the killer application for XML technology.
A Web Service interface is a document; it is, in the real sense of the word, a “contract”. It has a schema definition and is written using XML – the most flexible and popular format in use today.
As such, it is a much more flexible way of describing interfaces than anything that has previously existed. Also, it maps much more naturally onto how humans think about, operate, and model their own businesses. The language of normal business is the language of purchase orders, receipts, delivery notes, debits, and credits, rather than that of database tables and lines of Java!
Furthermore, in Web Services the data that is exchanged can be any random XML document, using XSLT (the XML transformation technology) to transform it on-the-fly into the desired format so that it can be consumed correctly. In a Web Service-oriented architecture, we are not confined to simple data types. If I want to exchange Microsoft Word documents, pictures, insurance claims, e-mails, or any other type of information, the set of technologies in Web Services make it trivial to do so.
In traditional service-oriented architecture, the interface can accept data in one very rigid format only. Using a Web Service-oriented architecture, I can build interfaces to accept a rich and varied set of input documents.
We can therefore describe Web Service-oriented architecture as “simple endpoints, rich data”.
4. Separating Interfacing from Programming
Finally, a Web Service-oriented architecture succeeds because it has nothing whatsoever to say about the way in which the interface gets implemented. Web Services can as easily provide a wrapper around a mainframe CICS application as they can provide a wrapper around a Java class. Web Services provide only an interface layer and the information bus. That’s all! In most Web Services products supporting integration there are no APIs to be learned.
This radically simplifies the whole task of building and deploying a Web Service-oriented architecture. It means that the pool of talent from which to draw the staff needed to implement a Web Service-based integration is several orders of magnitude larger than the talent pool of those who can program to DCOM, CORBA, or J2EE.
Think E-Mail
E-mail is one of the most widely used and successful business tools for collaboration. From simple point-to-point communications, all the way through to multi-party calendaring and collaboration, e-mail has proven time and again that its flexible architecture has many uses.
Based on a document architecture, a Web Service-oriented architecture is much more akin to e-mail than to J2EE, CORBA, or DCOM. This doesn’t imply that a Web Service-oriented architecture can’t be an enterprise-class solution – in fact, by leveraging standards such as LDAP, reliable messaging (in JMS), and the plethora of security standards and scalability of Web protocols, a Web Service-oriented architecture is every bit as enterprise-ready as anything you would find in DCOM, J2EE, or CORBA. However, philosophically, a Web Service-oriented architecture is geared towards document-based, loosely coupled collaborations and integration, making it more flexible and malleable – the precise characteristics needed to integrate complex information systems.
The Objections
Of course, no article like this would be complete without listing and refuting the standard objections to a Web Service-oriented architecture.
1. Not Mature
The first key objections to a Web Service-oriented architecture is that the underlying set of standards around Web Services (WSDL, SOAP, and UDDI) are not mature and don’t offer sufficient guarantees of interoperability. Critics would argue that open standards such as J2EE and CORBA have taken many years to offer their current level of interoperability.
This, of course, is true. J2EE and CORBA have taken many years to offer a limited amount of interoperability. There are two reasons for this. Firstly, both rely on binary wire protocols, which take years to perfect. If only one bit is out of place in the wire format, the whole conversation will fail. Both CORBA and RMI depend on very sophisticated wire protocols. Ensuring that many different implementations of these protocols are “bit-perfect” when transferring their payloads is an extremely difficult task.
Secondly, because they use their own proprietary protocols, they are not firewall-friendly, so it nearly impossible to make either a J2EE or CORBA endpoint available over the Internet. This means that all the testing for interoperability of these complex wire protocols must be done in one single laboratory. Furthermore, there must be someone willing to test all possible test cases against all possible implementations – in this white room environment. This isn’t an impossible task – it just has never been done for either J2EE or CORBA. In the CORBA case, individual vendors do certain interoperability testing against a subset of possible CORBA implementations. In the J2EE case, each individual vendor certifies to Sun that they have passed an interoperability test suite. Neither are overwhelming examples of a transparent and verifiable process!
In the case of Web Services, these two issues are removed. Web Services are based on an ASCII wire protocol running over HTTP (SMTP and FTP). ASCII protocols have proven time and again how interoperable they are.
In the case of Web Services, testing is entirely transparent. Every vendor makes a Web Service endpoint available on the Internet and anyone is free to test against it – and they do! So when it comes to the issues of Web Service interoperability, there is no hiding. Within 30 minutes of Cape Clear making a Web Service endpoint available on the Internet, we will have people complaining to us if we break interoperability in any way. Like the Internet, Web Services doesn’t need a centralized police force to guarantee interoperability – the very open nature of the network means that anyone who isn’t interoperable can’t sell their product!
2. Unreliable
Next on the objection list comes the infamous issue of reliability. Many would argue that the protocols in J2EE, DCOM, and CORBA have been specifically engineered to offer the very best reliability, scalability, and dependability.
Again, this may be true, but those protocols are so little used in comparison to HTTP that it is impossible to draw any definitive conclusions. What we do know is that HTTP has been proven to work on the biggest network in the world – no other protocol has a similar proof point.
The main objection around HTTP comes from the fact that most normal interactions with it are in the context of a browser. From a browser, we can shut down a connection at any time by hitting the “back” button or the “stop” button. Therefore, the objections around HTTP are based purely on irrational prejudices based on using it from a browser.
The fact is that HTTP opens a connection between two entities (which in the case of Web Services are two machines, not a human and a machine), and that open connection is every bit as reliable and dependable as its CORBA or J2EE equivalent – and is, in fact, many more times scalable than its CORBA or J2EE counterpart.
3. Slow
Having exhausted the previous two arguments, we are finally onto the oldest chestnut of them all – performance. Here, of course, there is a case to answer: Web Services are slower than alternatives – there is a relative (but as we’ll show later, not an absolute) price to be paid for a technology that is vastly easier to use and much more interoperable than its alternatives.
However, firstly let’s bring some comparative data to the argument. In Cape Clear performance benchmarks, we have shown that Web Services are about as fast as J2EE and about 3 or 4 times slower than CORBA. So, if tiptop performance is what you are looking for, to the exclusion of everything else, you’d better call up a CORBA vendor!
It is more compelling, however, to look at the absolute data. Here Web Services is a total winner. Cape Clear data shows a single Intel server capable of handling 100 million Web Service transactions per day.
The Business Implications
So much for theory and technology – what does this translate to in terms of bottom-line impact? In essence, a Web Service-oriented architecture has two profound business impacts.
Firstly, a Web Service-oriented architecture has a dramatic impact on the economics of business integration. It transforms an integration project – rather than taking several years, costing tens of millions of dollars, and with a low probability of success, a Web Service-based integration can be done in months, cost less than $1 million, and have guaranteed success. Already today, there are hundreds of live Web Services deployments, with companies such as Schwab, Sky Broadcasting, AT&T, British Telecom, JP Morgan, and Deutsche Telekom, to name a few, already availing of the compelling benefits of a Web Service-oriented architecture.
Secondly, through the adoption of a Web Service-oriented architecture, an integration project can be driven by business needs and not by technological possibility. Too often, due to the complex and costly nature of “traditional” integration solutions, the end result has been driven by what is possible, not by what is needed. By leveraging the underlying commodities of XML, the Internet, and the more sensible document paradigm, Web Service-oriented architectures open up an integration world where the driver is business needs – not the technology.
First appeared at Cape Clear
Annra O’Toole, CEO, Cape Clear Software
OToole drives the overall vision and corporate strategy for Cape Clear
Software http://www.capeclear.com/. He joined Cape
Clear in November 2000 to participate in the major opportunity being created
by the emergence of the Web Services technologies and the massive benefits
they could bring to the challenges of creating and
integration applications. O’Toole co-founded and served as Chief Technical
Officer of Iona Technologies, prior to joining Cape Clear. He holds an MSc
in Computer Science and an Electronic Engineering degree from Trinity
College, Dublin.
