Enterprise Service Bus - An Overview

Before we start talking about Enterprice Service Bus, lets us se what are the challenges of a distributed software system.

The challanges for a distributed software system

1. Interface or touch-point complexity between different pieces of the software components that are spread out across computers and networks, dependencies are built between those components, computers, and networks. If a software component is moved to reside on a different computer, or if a computer is moved onto a different network, or if the network architecture is modified in any way, there may be a rippling affect across the associated distributed applications.
2. Reduced reliability as more computers involved in processing a user request, the greater the chance a failure will occur. By far the most challenging of the reliability issues will be those that involve software communication. It's vital that when a component makes a request to another component, the response should arrive accurately and in a timely manner. The proper mechanisms must be in place to ensure that messages are re-sent when they are lost, and that they arrive in the same form as when they were sent, even if they are sent over foreign or unreliable networks.
3. There is a security challenge, when distributed components communicate over a network, as there is a potential for that communication to be tapped into, or otherwise spoofed. The potential for trouble increases when this communication spans networks, organizations, and the Internet. Malicious software, or people, may snoop network activity to retrieve sensitive data.

The Enterprise Service Bus Solution
There are patterns and tools that can be used to eliminate the three challanges just discussed. The three high level patterns or principles that provide for robust distributed software are

1. Reliable Communication - most important part of a distributed software system is its communication infrastructure
2. Service Oriented Architecture - Decomposition of software functonality into smaller pieces, that work with platform-neutral data structures and protocols. Examples of technologies in these areas are web services, the Java Message System (JMS), eXtensible Markup Language (XML), and the Simple Object Access Protocol (SOAP), respectively
3. Service Orchestration - Patterns and standards for the integration and proper orchestration of the services created in step 2

A tool is needed to enable these principals -- one that helps to reduce the complexity and costs associated with distributed software systems by implementing the three patterns. This tool is the Enterprise Service Bus (ESB). The ESB is both a tool and a software framework that helps you to create, deploy, and orchestrate (and communicate between) service components in a distributed system ESB solution for Interface or touch point complexity
No longer are the communication lines between disparate systems hard-coded. It helps to envision distributed application development and deployment differently. Adapters connect disparate software, enterprise messaging systems can be leveraged, and services can be integrated without writing a single line of associated code. This lets you focus on the construction of services and their associated business logic and not on the costly chores that go into building a robust distributed system -- the ESB eliminates them. That solves the complexity issue.

ESB solution for Reliability Challenge
ESB has built-in reliable messaging backbone. Because of this, reliability is virtually guaranteed to components built on the ESB. Additionally, the messages themselves, and the act of sending those messages, are normalized. This means that regardless of the initial form of the data you are sending between systems, and the form of communication a particular system is designed to use, the ESB transforms those messages and adapt various protocols to work together

ESB solution for Security Challenge
The ESB provides authentication and authorization services, as well as secure channels for component communication, thus solving most security issues. It has the added benefit of ensuring that security is implemented consistently across all services and components that use the ESB.

The ESB can best be described as having these characteristics

• Provides a reliable messaging infrastructure
• Enables SOA-based system development
• Is XML-based
• Supports web service standards (such as SOAP)
• Is platform independent
• Supports data transformation and routing services
• Enables service orchestration
• Supports transactions and security Integrates with existing standards, frameworks, and legacy systems

Overall, the foundation of the ESB includes enterprise messaging, SOA standards, message transformation, and message routing services, upon which you can efficiently build robust, distributed applications

Enterprise Messaging - Communication Infrastucture of ESB

Enterprise messaging systems provide message delivery guarantees, as well as transactional support, most often with two-phase commit. This provides the ESB, along with the applications built on top of them, a level of built-in reliability required for mission-critical application. For software components developed for an ESB, there is generally nothing special that needs to be done to tap into this service. It's provided transparently by the ESB and the enterprise messaging backbone it itself is built upon.
Enterprise messaging systems, and the ESB, provide the capability for software components to send, listen for, and process messages asynchronously. This helps to decouple components in a distributed software system, further enabling system reliability and scalability

The Java Messaging System (JMS) pecification describes an enterprise messaging system for Java that supports both topic-based and queue-based messaging paradigms. JMS also specifies many types of message objects with the goal of interoperating with other, non-Java/JMS, messaging systems. With enterprise messaging -- and in particular, JMS -- you can build distributed software systems that are truly platform and language-independent. JMS is a specification that describes the properties and behavior of an information pipe for Java software. These are two very important distinctions that must be understood before implementing JMS software. They are

1. JMS Provider - This is an enterprise messaging implementation that adheres to the JMS specification for message definition, delivery, storage, processing, and error handling. It is the implementation of the message pipe itself, not the software that uses it
2. JMS Application- Sometimes called a JMS client, this is the Java software that uses a JMS provider to send and receive messages over an information pipe. This software can be any application, small or large, that generates or processes messages

Standards are required to drive down the cost of integration. These standards need to account for platform and language independence. It also lets the developers focus on building software functionality and not infrastructure, thereby easing the pain of distributed software development. Three forms of messaging have become popular and standardized atleast in definition.

• Request-Response messaging
• Topic -based messaging [Publish and Subsribe ]
• Queue based messaging [ Store and forward ]

The ESB is built directly on top of the JMS information pipe, and leverages everything that JMS has to offer.

SOA Standards - Integration framework of ESB

SOA Standards helps to enable flexibility and reduce complexity

• Well defined interfaces with business level semantics
• Standardized communication protocols
• Flexible recombination of services to enhance software flexibility

SOA drives its technical strategy and vision from the basic concept of a service

• A service is an abstraction that encapsulates a software solution
• Developers build services, use services and develop solutions that aggregate services
• Composition of services into integrated solutions is a key activity

SOA Core Elements

• Service Assembly - technology and language-independent representation of the composition of services into business solutions
• Service Conponent - technology and language-independent representation of a service that can be composed with other services

SOA is a new paradigm for software paradigm. Organizations have discovered the benefits of using both internal services (those developed and deployed within the organization) as well as external ones (those built and deployed by other organizations).

As mentioned above SOA is a new pardigm, how to integrate those core legacy applications that don't support SOA standards into a newer distributed application. For this reason, you may need to use the data transformation functionality provided by the ESB, a built-in adapter, a custom adaptor or a combination. An adapter is a relatively simple piece of software that wraps a component's interface, or transforms data, to interface with a different system.

You can write this software yourself to, in effect, convert the software's interface into a web service or to be JMS-compliant. If the application uses enterprise Java or .NET, the tools used to build the system will more than likely be able to automate this process. For other cases where a data transformation or a built-in solution is not possible, you may need to build a more complex, custom adapter. The custom code you write may need to interface with older applications such as COBOL (see Figure 11) or to transform older, proprietary, data formats, to XML

The adapter pattern is straightforward in most cases, and will solve many of your ESB integration problems. For systems that comply with the ESB interface, yet whose messages do not match the standards for the rest of the system, the ESB message transformation capabilities may be used.

ESB Message Transformation - framework to decouple sender's and receiever's data view

Many services will accept messages in the form of XML, but they may need that XML to conform to a particular XML schema. As more receiver components are plugged into an ESB-based deployment, the more potential data transformations will need to take place. Developing custom code to transform the same message to conform to a different XML schema for each client is inefficient and costly

The ESB provides built-in message transformation services to take care of this for you. ESB allows each listener component plugged into the ESB to specify a format for which it requires the dataThe ESB will perform the transformation for the listening component, without any specialized code required on the part of the sender or the receiver. The net effect is that the receiver's view of the data is decoupled from the sender's view, the overall system's view, or that of any other receiver in the system. Message normalization and transformation is a feature that will also save you a lot of development, testing, and maintenance effort. Because the ESB decouples both the message sender and receiver from the message format (and hence one another), the development of either component will not impact the other.

Normalizing message formats doesn't just benefit message sender and receiver component developers; it enables the ESB to work with the messages; such as to route then effectively across the network and to the systems that require them

ESB Message Routing - framework to seperate service lookups from the components that require the services

In the classic cases of web services or JMS, message senders and receivers would have to find one another, and invoke each others services. The problem with this approach is that every component has some sense of awareness of the other. In most cases, this awareness is hard-coded within the component in a way that makes it difficult and expensive to change and redeploy

The ESB serves as an intermediary for communication and the routing of messages between components. The ESB separates service lookups from the components that require the services, by allowing to define how messages and events can be chained, resulting abstract message flows that components can plug into at runtime dynamically

Lets take a quick look of what a message flow is ?

A service endpoint is defined as a discrete operation devoid of its implementation. Service endpoints are abstract in the sense that you do not need to worry about how they're implemented, where they reside, or how they're plugged into the ESB. You simply know they exist based on their service definition, usually in the form of Service Component Definition Language (SCDL), which is an XML-based standard for defining business logic operations in a SOA. The definition of a message flow begins by chaining together various service endpoints (and hence the components that implement those services), applying any required message transformations, along with any rules for the overall workflow. The rules of the workflow can be defined in terms of the content within the messages, or they can be tied to a generalized workflow, also known as itinerary-based routing

• Content-based Routing - The act of routing messages based on content is generally implemented as script components deployed within the ESB. The script can be code in the form of JavaScript, or it can be in the form of Business Process Execution Language (BPEL) rules. The routing can be based on message validation to ensure that applicable messages contain the data they're supposed to, or that they conform to an expected message format
• Itinerary-based Routing - Itinerary-based message routing is based on pre-defined workflows, or step-by-step processing. This type of routing is similar to a Java Servlet filter chain, where the response from one Servlet becomes the input to the next Servlet in the chain, and so on. Whereas the Servlet chain is defined in a Java EE configuration file, the itinerary-based routing workflow is defined with routing rules in the ESB

BEPL - Framework for Orchestration and Business Process Management. BPEL is an XML-based language specifically designed to describe and orchestrate asynchronous business processes. A business process is really nothing more than a piece of business logic (code) that performs some data-related task, but which is meant to be part of a larger set of processing. In other words, a business process is a relatively small data-crunching task that is meant to be chained with other business processes to perform an overall larger task. These tasks (their methods, inputs, and outputs) can be combined in meaningful ways with BPEL. Plugging component's based on SCA ( Service Component Architecture) into the ESB and controlling them with BPEL allows you to build reliable, distributed, systems with central orchestration and monitoring capabilities.

Conclusion

This article set out to define what an ESB is, what it consists of, and what problems it is designed to solve. As a result you, should have and understanding of when to use an ESB, and what parts best apply to your own problem sets. An important lesson to learn is that, in practice, the ESB itself is not an application server, a messaging server, or a single piece of software that is installed and run on a piece of hardware. Instead ESB should be thought of as a toolkit, or a framework, from which you can choose the pieces to use or ignore as you design and build distributed software systems.

There are various Commercial and Open-source ESB. Next I shall be posting on my experience with Mule- a complete open-source ESB framework. Mule has started discussions for supporting theService Component Architecture specification. (http://mule.mulesource.org/display/SCA/Home)