I feel fortunate enough to have been able to attend the TDWI conferences this year. It was a great opportunity to get insight on the BI-related platforms, experience sessions with knowledgeable presenters and to network with other people that share the same passion.
The real take away for me is that traditional data warehousing takes too long and we need to be thinking of extending the BI platform to fit the self-service world. The purpose of this blog is to introduce the basic concepts of the modern BI platform. The goal is that once you grasp the concepts you will be able to apply to any vendor. The extent as to which capabilities to mix and match and which vendors to use varies depending on your customer needs and is out of scope for this blog.
I don't think any of this is new and we do a lot of this today, real-time analytics, real-time streaming, data virtualization, etc., but it always seems to be an afterthought and a lot of times standalone environments.
The BI architecture needs to advance into an extended platform which includes real-time analysis, investigation platform, exploration platform, as well as governance and presentation platform that comes together physically or virtually.
Operational real-time platform would contain the operational systems which are the source of most of your data, real-time analytics, forecasting models, provide streaming analytics, BI services that input to operational screens.
Investigation platform is used to store raw detail of new and unusual sources. Helps determine the value of big data (sensor data or social media) where there is a lot of noise. This is where you determine what is valuable to keep and then distribute to another platform.
Exploration platform is used for data discovery enabling scientist to blend new types of data with existing data to discover new business improvements. Used for data profiling, cause and effect analysis, pattern analysis, etc.
Presentation platform is where the KPI’s, dashboards, and reports are presented to the business users. This is where the EDW will reside and is probably the most important and heaviest used platform. Still the best source for reliable, integrated, and consistent data. This is where you formally productionalize your findings from the other layers and deploy to the business users.
Regardless of which vendor you choose the concept remains the same, traditional BI platform has shifted and needs to be extended to include an analytical portfolio that is agile and user-driven.