What is Business Intelligence? Monitoring All Organizational Processes

By: Samuel Scott

Business intelligence (BI) is the use of software, tools, and applications to analyze an organization’s raw data with the goal of optimizing decisions, improving collaboration, and increasing overall performance. The process is the collection and analysis of the personal, team, and organizational information through activities such as data mining, analytical processing, querying, and reporting.

According to a report from Statistics Market Research Consulting, the total BI market is expected to grow to almost $30 billion by 2022. Those who do not understand the concepts of business intelligence will surely be lost long before that time.

It’s crucial to monitor not only IT infrastructures and software applications but also all business processes throughout entire organizations. But let’s start at the beginning.

The History of the Term

The first known appearance of the term “business intelligence” was in “Cyclopædia of Commercial and Business Anecdotes” by Richard Millar Devens in 1865:

The name of Sir Henry Furnese figures largely among the bygone bankers who gave renown to the financiers of that period. Throughout Holland, Flanders, France, and Germany, he maintained a complete and perfect train of business intelligence. The news of the many battles fought was thus received first by him, and the fall of Namur added to his profits, owing to his early receipt of the news.

Of course, the basic idea is to receive information and then adjust one’s actions accordingly.

In a 1958 article in the “IBM Journal” called “A Business Intelligence System” (PDF), Hans Peter Luhn, a researcher for the company, used the term in relation to the definition of “intelligence” in Webster’s dictionary: “the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal.”

From the 1960s to the mid-1980s, decision support systems (DSS) were created and deployed so that executives could use computer-aided models to assist with corporate decision making and planning. DSS led to data warehousing and then analytics processing and then finally the rise of “business intelligence” as a standalone discipline.

In 1989, Howard Dresner, a consultant who would later become a Gartner analyst, proposed the use of the umbrella term “business intelligence” to refer to “concepts and methods to improve business decision making by using fact-based support systems.”

Who Uses BI Platforms?

As we published in a blog post on streaming Big Data for real-time analytics, business intelligence platforms have used in many industries:

Viacom monitors the quality of the video feeds of its numerous television channels
The City of Chicago analyzes seven million data points to predict how to allocate city resources such as emergency vehicles
Buzzfeed uses various statistics to see how the more than 400 million news items that the website publishes every month are performing
The website of Germany’s largest retailer, Otto, sees more than 10,000 interactions every single second. The company stores this information and uses it to give every visitor a customized experience
Marketing teams can monitor digital campaigns as well as conduct A/B and multivariate tests quickly

The list of uses does not stop there. Restaurant chains use business intelligence to make strategic decisions including which branches to close, which items to add or remove from menus, and which processes can be made more efficient. Professional sports teams use statistical analysis both to win championships and to improve the fan experience at stadiums.

In general terms, business intelligence empowers executives to make better and more profitable decisions in terms of how to cut costs, see new opportunities, react quickly to news and other developments, make processes more efficient, gain competitive advantages, obtain leverage in relationships and negotiations, and optimize price points.

Within any industry or company, BI teams often consist of analysts, architects, data management experts, and developers. In modern times, the developer side of business intelligence may now include DevOps engineers. (For more information on this modern R&D function, see our guide on “What is DevOps?”.)

What is Done in BI Analysis?

At its core, business intelligence is the use of data to answer questions such as these:

What has happened?
What is happening?
Why is it happening?
What will happen?
What do we want to happen?

The answers can often be found through historical data analysis (whether descriptive, predictive, or prescriptive) or real-time analytics based on the information in locations and activities including customer databases, supply chains, personnel records, manufacturing data, and sales and marketing campaigns. BI analysis aims to pull relevant data from such sources to help executives to drive business value through actions such as cutting costs or increasing revenues. Historical trends are often examined to create long-term strategic decisions while real-time data can support immediate, tactical decisions.

That data is often stored in internal data warehouses or private clouds but is increasingly saved off-site in public clouds such as Amazon Web Services, Microsoft Azure, or Google Cloud Platform. (See our comparison of the two most popular public clouds: Azure vs. AWS.) Apache Hadoop, an open-source software framework used for distributed storage and processing large sets of data, is another increasingly popular option.

Within the overall framework of business intelligence, data analysis can be used in the following ways:

Analytics — the building of quantitative processes such as data mining and statistical analysis for the purpose of business knowledge discovery
Collaboration — the merging of information from two or more different business functions through practices such as data sharing
Knowledge management — the adoption of best practices by making an organization data-driven
Measurement — the setting of benchmarks that are then used to inform on progress towards the desired business goal
Reporting — the presentation of a specific set of desired information through tools such as data visualization

These activities are performed while using one of the many types of BI applications such as ad hoc analysis, OLAP, mobile, real-time, operational, and open source ones.

How to Implement a BI System

First, make a list of decisions that executives make, what information they use to make them, and in what format they would like to receive the data. Second, be sure that the information provides as much context as possible (not only, for example, that systems are failing more this quarter than last quarter but also why). Third, make sure that the data is clean and not degraded. Finally, deploy the system and train the relevant people to use it — and then collect feedback on a regular basis on how the reports can be improved.

There are many enterprise-level vendors of BI software that can also be used: IBM, Information Builders, Microsoft, MicroStrategy, Oracle, SAP, and SAS. In addition, mid-size providers include Actuate, Alteryx, Logi Analytics, ClikTech, and Tableau. We at Logz.io have demonstrated how to analyze Salesforce data with the ELK Stack:

Another option is for people to use self-service business intelligence (SSBI), which gives executives access to information without needing to involve the IT department directly. IBM Watson Analytics, Microsoft Power BI, Tableau Desktop, Looker, Domo, and Zoho Reports are a few examples. Organizations can also use some of the free and open-source business intelligence tools that can incorporate data mining, predictive analytics, text mining, and statistical analysis.

The Future of Business Intelligence

Business intelligence (BI) has always been associated with the use of tools for viewing and interacting with reports and dashboards as well as visualizing, analyzing, and modeling the results of queries.