Why is Big Data important? What is it?
A good way to think about Big Data is through an example of a “smart device”, the thermostat. With the standard analog thermostat, you turn the dial to increase or decrease temperature and there is a sensor which turns on or off the heating and cooling system. There’s minimal data needed and none collected.
With a digital thermostat, like the Nest Thermostat , the device collects a huge amount of data and uses it to optimize the termperature and optimize the heating/cooling for comfort and energy efficiency. Data collected includes:
- External weather conditions (through a wireless interface)
- When the dial is turned up or down and by how much
- Time of day, day of the week, and season
- Current temperature and any changes in the room
In a sense, this small devices collects, analyzes, and acts upon a much larger amount of data than the analog or even a simple digital thermostat. Multiply this one example by multiple “smart devices” and you have Big Data. The complexity comes from how to manage, analyze, summarize and use the volumes of data that are growing exponetially.
This group of articles from The Economist and McKinsey provide a good background on the business implications of having so much data and being able to analyze and apply it:
- The Economist Special Report: Digital Deluge; defines the promise and also potential negative impact on privacy of being in a data rich environment (from scarcity).
- McKinsey: Big Data the Next Frontier for Innovation, Creativity, and Productivity; Five effects: transparency, experimentation, customized actions based on segmentation, replace decisionmaking with algorithms, innovate new business models, products, and services.