Real-time Analytics or History based, which is most suitable?

Traditionally data warehouses do not contain today’s data. They are usually loaded with data via an Extract, Transform and Load (ETL) process from operational systems on a periodic basis, often nightly, but sometimes even weekly. In any case, they are a window on the past.

With the ever-growing pace of business today, are these history-based analytical data sources providing the right insights when they are most needed? In this post, we look at the key differences, or scenarios where one might choose a data warehouse solution versus using real-time data sources to provide operational analytics.

Data warehouses

Data warehouses are designed to answer exactly the types of questions that users would like to pose against real-time data. They are able to analyse particularly large volumes of consolidated data and be sliced by any given dimension built into your warehouse such as time, customer, sales representative, region, product class or category to name just a few.

As data is pre-populated and aggregated, performance is often much faster compared to real-time sources where calculations and data retrieval are performed off transactional sources where the data originated. Generally non-real time, or history-based BI data will be best suited to the following needs:

• • Tactical, strategic or long-term operational decisions outside the flow of production. Business Managers are more likely to require such analysis for supporting business decisions.

• • To build or modify predictive or prescriptive models. Business Analysts tasked with improving business processes and profitability often prefer static data sources to model scenarios and provide recommendations to the business.

• • To produce periodic reports, interactive data discovery and “what if” modelling. All users within a business may lean on data warehouses for this kind of data content, particularly if the operational reporting is from large, varied or separate sources.

• • Traditional online analytical processing (OLAP)-based cubes are often the preferred data source type as they are inexpensive, scalable, and provide good performance over large datasets.

Real-time data sources

In contrast, real-time (or near real-time) operational reporting or analytics will be well suited to the following needs:

• • Where operational decisions are woven into business processes, transactions or other production activities. Best-of-breed ERP solutions will rely on pre-integrated BI tools to interpret and report on data live from its transactional database.

• • To run existing predictive or prescriptive models.

• • Provide the benefits of reporting from both current and historical data concurrently.

• • Allow users to be alerted to important data changes or where thresholds (KPIs) are exceeded in real-time.

• • Where users need to quickly view, analyse and drill down on real time displays, or dashboards in real-time. Fast moving transactional environments such as retail will often rely on this kind of information for hourly store sales performance.

So in reality, both solutions are complementary and can co-exist – they should be given equal consideration when designing the data reporting and analysis framework that’s right for your business.