EDW and BI Apps (Part 1)
Posted by Dylan Wan on October 8, 2010
This may be an old topic, but people seem like to see and look for conflicts and arguments which do not really exist, and keep bringing this topic up.
Some vendors sell the data models, particularly, the enterprise data warehouse model, and tell you that it is their BI solution. For me, data model alone is NOT a BI solution. It is solving a small percentage of your problem if it solves problems at all.
What efforts and costs you really need to spend on is to
– Understanding your reporting requirement
– Determine how to use the analytics to get you the competitive advantage
– Determine how the analytics can be used as part of your decision making process which can be through out the organizations everywhere.
– Analyze where the data were originated and captured
– Analyze and determine how the information can be extracted and delivered
– Design how the data can be organized, consolidated, and transformed so the data can become closer to answer your business questions
What so called enterprise data warehouse model, does not answer most of the above questions.
Actually I cannot believe that people are comparing between a BI solution with the full layers of solutions with the enterprise data warehouse model. There is no comparison. The full layers of solutions are the data model design for answering your business questions, the analytics metadata describes how the data are organized and thus can be easily retrieved, the library of the metrics that you can select from, the prebuilt analytics flow with the guided navigation, the prebuilt adapter to the enterprise business applications. The last point includes the analysis about the source system model and features, as well as defining and comparing the semantic meanings from each system and defining the logical mapping. Also, the tuning of the various layers such as SQL tuning for the BI query and the performance tuning the batch job that extract and deliver the data. Even though you buy a EDW model, I guess that those above tasks still exist.
Please note that I also like the enterprise data warehouse concept, as a tool for consolidating data across multiple systems within an enterprise. I also like to read those books such as the Data Model Reference Guide. I like that they describe the business rule and assumptions using a model approach. However, you will need to see if your organization is really follow those same business assumption. Every organization has different business rules and process. Enterprise apps, such as ERP and CRM vendor, try to make their system adaptable to your process or try to sell their solution as the best practice. The truth is that not all business are the same. Same also hold true for the data warehouse model. A enterprise data warehouse model is a reference model, is a starting point, and is valuable, but please do not compare it with the prepackage business intelligence applications that have done a lot of the tasks I mentioned above.
There are questions about how to integrate between EDW and BI Apps. When the question is being asked, I think that one should prepare that there are manual work involved. First of all, as far as I know, many such enterprise data warehouse does not have prepackage ETL. If you have unique data consolidation requirements, you can go for that EDW model, but you will still need to analyze how to consolidate those data. One thing you can leverage from the pre-packaged BI Apps is that they already have the ETL and know how to extract the data, so you may be able to leverage your investment in that way. However, you will need to keep this in mind. The prepackage apps are defined with the reporting requirement in mind so it may not extract all the information. The EDW modeler, however, does not keep the reporting requirement as the model is designed. Many of them are designed for consolidating data, so you can get study the ETL to get the source data structure but you may not get all the attributes if you are thinking of populating the EDW from the extracted prepackaged BI apps data warehouse.
Also, possible solution is to use the consolidated enterprise data warehouse as the source for BI Apps. I can see the flow makes sense since EDW may have more granular data than the prepackage BI apps. However, it is really hard to have a standard assumption of what EDW should be, typically you won’t get the prebuilt adapter, which I feel is one of the major cost of your efforts.
(to be continued…)