SOA purists might scoff at using SOA for integration [1], but for many enterprises, Service Oriented Integration (SOI) remains one of the prime motivations for embarking on the SOA journey. Agreed SOI by itself doesn’t achieve the avowed goals of agility or elimination of redundant IT infrastructure, but it helps the enterprise address real concerns, now. The SOA Manifesto [2] states that Business value is of higher priority over technical strategy; hence easier integration with SOA is a valid goal of a SOA initiative.
Data Services provide the foundation for SOI. A Data Services Layer provides a Unified, Standards based access to different data sources (Databases, Flat files, and spreadsheets). With the proper SOA Governance and SOA Management, a Data Services Layer can provide a controlled and safe access mechanism to data stored in heterogeneous data sources. In the recent 2’nd Annual Department of Defense (DoD) SOA Symposium [3], most speakers professed that the majority of their Service Portfolio was comprised of Data Services, providing access to the data from various sources.
BI has been described as a “broad category of applications and technologies for gathering, providing access to, and analyzing data in order to help users make better business decisions and to improve both the timeliness and the quality of information”.[4] Given the pace at which Data is growing in any modern enterprise, this is easier said than done. In his book, “The Intelligent Company” [5], Bernard Marr states that gold prospectors of the 19’th century had to pan endless tons of worthless silt using a gold pan, in order to find a few golden nuggets. Modern day managers have to almost do the same in order to gain the ‘golden nuggets’ of insight.
The Data Services Layer created to solve the “integration” problem has an important secondary usage; it also lays the foundation for successful BI. The Data Services Layer [6] helps BI by
- Making Data Accessible: A common standards based interface is provided to the Service consumer. This interface hides the actual implementation details and location of the data. This allows BI tools to retrieve and search data, irrespective of the data source type (relational, flat file, mainframe etc) using industry standard protocols.
- Mediating Data: In an enterprise, Data is present in different formats and different syntax. The Data Services Layer can help make the Data more interoperable and usable for BI by mediating the Data into different data types, vocabularies and semantics.
- Abstracting Data: The Data Services Layer decouples the consumer from having to know the intricate relationships between the numerous pieces of data. Hence the BI tools can work at a higher level of abstraction, better suited from them, as they are shielded from the low level details of what, where and how about the data.
- Increasing Organizational Trust in data: As access to data is provided via a reusable, common Service Interfaces, it forces issues to emerge earlier. Hence issues can be easily traced and rectified. Without a common Service Interface, the pedigree of the data can be doubt as there could be multiple “hops” before the end-consumer views or uses the data. This allows more trust in the data used to make “intelligent” decisions with BI.
- Extending the reach of Enterprise Data Warehouses (EDW): Traditionally, many BI strategies use EDW to consolidate data scattered in data silos. However some of the data required for BI might be outside the EDW or the latency at which it is loaded in the EDW might be too high. The Data Services Layer can provide this data at a faster refresh rate and from outside sources, without having to store the data in the EDW.
Thus a well crafted Data Services Layer provides a strong foundation for both SOI and BI. So let the purists discuss, lets add Business value to the enterprise using SOA.
References:
[1] SOA vs SOI [http://apsblog.burtongroup.com/2008/12/soa-vs-soi.html][2] SOA Manifesto [http://www.soa-manifesto.org][3] 2nd Annual DoD SOA Symposium [http://www.soasymposium.com/usa2010/conference_agenda.php][4] What is BI [http://www.musing.eu/musing-paradigm/what-is-bi][5] The Intelligent Company, by Bernard Marr, ISBN: 978-0-470-68595-2[6] Army Data Services Layer, Reference Architecture White Paper, June 2008, Version 1.1
