Six Secrets to Big Data Success
Time is the enemy of Big Data victory. Here are six ways to improve your data strategy right now.
Today, it is hard to escape all the chatter about Big Data. As Google chief Eric Schmidt said, "While it took from the dawn of civilization to 2003 to create five exabytes of information, we now create that same volume in just two days!" Although Big Data is all around us, the reality is that only a small fraction of CIOs are tackling this head on. Our experiences working with large organizations on Big Data projects suggest that there is frustration in organizations trying to decide what the best course of action is in this brave new world. There is a lack of vision and a fear of making mistakes. In order to make the transition easier, we believe there are a few fundamental rules that should govern Big Data plans:
1. Invest in the right skills before technology More important than technology is having the right skills, of which three are distinctly required:
- The ability to frame and ask the right business questions, with a clear line of sight as to how the insights will be used. Big Data is noisy and plentiful. The ability to crystallize a business problem and not boil the ocean is critical to being able to generate rapid and relevant insights.
- The ability to use disparate open source software to integrate and analyze structured and unstructured data. There is no single Big Data tool that does it all, meaning one must bring together the best breed of tools to get the job done. In addition, the landscape of tools and technologies is rapidly evolving; locking oneself into a proprietary solution would be very risky. Open source software is the recommended approach and the ability to understand a diverse set of technologies is important.
- The ability to bring the right statistical tools to bear on the data to perform predictive analytics and generate forward-looking insights. The holy grail of Big Data is to be able to predict the future with a high level of certainty. Given data (big or small), the art of reliably predicting the future requires a fundamental knowledge of disciplines like statistics and machine learning. One has to be able to parse the signal from the noise, even more so in the Big Data world where noise is abundant.
These skills can be developed proactively both by training and hiring. For example, find those in your organization who are good at skills 1 and 3, and have a penchant for 2. Give them the opportunity to play Big Data steward. Hire individuals who have strong training in 2 and 3, and who show a penchant for business applications. It is also important to find senior leaders in the organization who not only believe in the power of Big Data, but are also willing to take risks and experiment. These leaders can play a big role in driving rapid adoption and success of data applications.