Welcome!

From the Author of The Agile Architecture Revolution

Jason Bloomberg

Subscribe to Jason Bloomberg: eMailAlertsEmail Alerts
Get Jason Bloomberg via: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Related Topics: Data Services Journal, SOA & WOA Magazine

SOA & WOA: Article

Best Practices for SOA: Building a Data Services Layer

Choosing best-of-breed data access middleware is an important enabler of SOA

The Importance of Best-of-Breed Data Access in Building a Data Services Layer
Ask just about any business executive about the core value IT provides to the business, and they’re likely to respond that data (in the form of useful information) rather than applications form the lifeblood of their business. In essence they’re correct; information technology is by definition all about information. But actually the two – data and applications – can only be separated in theory, not in fact. Data by itself is often inaccessible and/or unintelligible without the applications that process it, and most applications in turn serve no real business purpose without data.

While establishing a data services layer is essential if you want to fully apply SOA best practices to data, it’s equally important to realize that the foundation of that data services layer consists of data access. Data access, in turn, depends on standards such as ODBC, JDBC, ADO.NET, and SDO. Even if you’re using a persistence layer like one built with the open source Hibernate toolkit, for example, high-quality data access middleware is critical. A slow JDBC driver under Hibernate or a slow ADO.NET provider under Microsoft’s Language Integrated Query (LINQ) will inevitably impede the services. The results can be far reaching. Accessing the data in the various data stores across your organization in the most efficient, flexible manner is a core reason for building data services. It’s therefore critical for building all the services in your SOA implementation.

Performance speed is not the least of the issues that can be affected by data access middleware. And, because the data services layer essentially virtualizes data access, those services can hide a plethora of potential data access pitfalls. Other such pitfalls can include:

  • Scalability issues
  • Database platform, application, and version incompatibilities
  • Differences in how the various data sources handle the details of standard data access operations such as create, read, search, update, and delete
  • Data source security priorities, which might vary from database to database, table to table, or even row to row
  • Data mapping challenges arising from semantic differences among heterogeneous data
  • Problems arising from the mixture of structured and unstructured data — such as file formatting issues
  • Differences in versions of SQL

SOA can actually serve to magnify limitations in scalability, performance, and interoperability imposed by data access because organizations are both more likely to reuse the underlying code and leverage existing data services across many services and composite applications.

Your first step then in resolving these data-related issues is to build shared, centralized data services, which will result in an adaptable and easily maintained SOA implementation. This way data access logic only appears in one place, no matter how many applications consume it. The result is that, instead of scattering data-related services invocation code throughout each business service, centralized data access helps you create an environment that enables a best-of-breed data access solution to address all those data access issues.

This last point – the use of best-of-breed access solutions – can’t be stressed strongly enough. Even many people involved in data management rarely give data access middleware more than a passing thought. One reason for this is that commercial databases all include connectivity drivers bundled with the database software, which are often used by default. The fact that these “free” drivers may be less than optimal for a given IT environment typically doesn’t come up unless and until either a technical or a maintenance issue becomes serious enough to be traced back to the data connectivity layer. For reasons we’ve described, such issues can be compounded in an SOA and can also be extremely difficult to diagnose.

Quite simply, it is ill advised to rely on database-bundled data access middleware for your data services layer in an enterprise SOA. The same holds true for freely available open source drivers. As the very foundation of your data services layer, the type of data access you use deserves serious consideration and careful planning. Dynamic environments where multiple services reuse data access code and new services regularly go into production require that such code meet rigorous requirements.

Your best bet is to go with third-party data access middleware, preferably from a supplier whose core business and expertise consists of data connectivity. Look for specific attributes for your data access middleware, including capabilities for boosting query performance. Such attributes can include connection pooling as well as support for tunable data access performance such as adjusting network packet size. For scalability and high availability, data access middleware should be multithreaded and thread-safe, and offer client load balancing and failover to alternate servers.

It’s also important for data access middleware to support different types and versions of databases as well as all the subtle variations in SQL they support. Such support for heterogeneity should also extend to multiple computing platforms, chipsets, and operating systems. In addition, for the best performance and flexibility, data access middleware should offer wire protocol drivers to avoid the overhead and maintenance issues of off-the-shelf database drivers. Wire protocol drivers obviate the need for database client software and libraries, simplifying installation and administration as well as offering a much more efficient and high-performing operation. Such drivers must support the full panoply of relevant standards, including JDBC, ODBC, ADO.NET, and – over time – SDO.

Finally, you should incorporate data access middleware in a comprehensive IT security strategy that covers both network security and database security, with secure communications and secure code. It should integrate with multiple solutions for authentication and authorization, too.

It’s important to choose best-of-breed data access middleware as a critical building block for any SOA initiative you undertake in your business (see sidebar). Data access is a fundamental building block of SOA, and if you fail to make good choices there, the entire services infrastructure will suffer for it. Fundamentally, regardless of the SOA infrastructure that runs above the data services layer, there’s no question that data access remains a key building block technology for SOA.

More Stories By John Goodson

As vice-president of product operations, John Goodson leads the product strategy, direction, and development efforts at DataDirect Technologies. For more than 10 years, he has worked closely with Sun and Microsoft on the development and evolution of database connectivity standards including J2EE, JDBC, .NET, ODBC, and ADO. His active memberships in various standards committees, including the JDBC Expert Group, have helped Goodson's team develop the most
technically advanced data connectivity technologies. He holds a BS in computer science from Virginia Tech.

More Stories By Jason Bloomberg

Jason Bloomberg is the leading expert on architecting agility for the enterprise. As president of Intellyx, Mr. Bloomberg brings his years of thought leadership in the areas of Cloud Computing, Enterprise Architecture, and Service-Oriented Architecture to a global clientele of business executives, architects, software vendors, and Cloud service providers looking to achieve technology-enabled business agility across their organizations and for their customers. His latest book, The Agile Architecture Revolution (John Wiley & Sons, 2013), sets the stage for Mr. Bloomberg’s groundbreaking Agile Architecture vision.

Mr. Bloomberg is perhaps best known for his twelve years at ZapThink, where he created and delivered the Licensed ZapThink Architect (LZA) SOA course and associated credential, certifying over 1,700 professionals worldwide. He is one of the original Managing Partners of ZapThink LLC, the leading SOA advisory and analysis firm, which was acquired by Dovel Technologies in 2011. He now runs the successor to the LZA program, the Bloomberg Agile Architecture Course, around the world.

Mr. Bloomberg is a frequent conference speaker and prolific writer. He has published over 500 articles, spoken at over 300 conferences, Webinars, and other events, and has been quoted in the press over 1,400 times as the leading expert on agile approaches to architecture in the enterprise.

Mr. Bloomberg’s previous book, Service Orient or Be Doomed! How Service Orientation Will Change Your Business (John Wiley & Sons, 2006, coauthored with Ron Schmelzer), is recognized as the leading business book on Service Orientation. He also co-authored the books XML and Web Services Unleashed (SAMS Publishing, 2002), and Web Page Scripting Techniques (Hayden Books, 1996).

Prior to ZapThink, Mr. Bloomberg built a diverse background in eBusiness technology management and industry analysis, including serving as a senior analyst in IDC’s eBusiness Advisory group, as well as holding eBusiness management positions at USWeb/CKS (later marchFIRST) and WaveBend Solutions (now Hitachi Consulting).

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.