Building a solid data foundation is prerequisite to becoming data-driven or intelligence-driven. In a previous post, I outlined the progression to grow into an intelligence-driven company. Many companies will have a hard enough time becoming data-driven, let alone intelligence-driven. I will mainly focus on how to become data-driven. Becoming data-driven is a major long-term commitment, and will require enormous time and patience. Most critically, it requires a solid data foundation. This post is an defines a data foundation, and outlines the steps to build build one. In future posts, I will be going into deeper detail about how to achieve various parts of this foundation.
Many companies claim they are data driven, and want to adopt “big data”. Interest and investment in big data continues to grow.
- “IDC says that worldwide revenues for big data and business analytics will grow from $130.1 billion in 2016 to more than $203 billion in 2020, at a compound annual growth rate (CAGR) of 11.7%.” (Forbes)
- But…”Big data investments continue to rise but are showing signs of contracting, according to a recent survey by Gartner, Inc. The survey revealed that 48 percent of companies have invested in big data in 2016, up 3 percent from 2015. However, those who plan to invest in big data within the next two years fell from 31 to 25 percent in 2016.” (Gartner)
Despite the hype and investment in big data and becoming data-driven, the macro numbers show a much different story.
- Only 3% of companies have data that meet basic quality standards. (Harvard Business Review)
- “Through 2017, 60 percent of big data projects will fail to go beyond piloting and experimentation, and will be abandoned.” (Gartner)
- “Only 12% of respondents say those involved in their businesses’ general analytics efforts can stay abreast of emerging industry innovations. More telling, just 2% believe transformations in capabilities at their companies have had a “broad, positive impact.” “ (ZS)
- “While nearly three quarters of respondents said that their organisation has invested or is planning to invest in big data, many remain stuck at the pilot stage. Only 15 percent of businesses reported deploying their big data project to production, effectively unchanged from last year (14 percent).” (Gartner)
- “30% of these executives said that they lack a clear strategy for embedding data and analytics in their companies. And despite the best intentions of the 70% whose companies have strategies, many will lose their way with their data because of one simple reason: people.” (Forbes)
So, investment in data initiatives and technology is expected to grow. Yet thus far, results have been mixed. What gives?
I’ve seen data initiatives fail because of problems in one or more of the following areas, in order of importance:
- Culture & Politics
- Data and Technology
Notice that I put data and technology last. In my experience, the success or failure of data initiatives – and the ability to build a data foundation – have very little to do with data nor technology. Data and technology react to the business, but are not necessarily leading drivers in most (especially non data-driven) organizations.
Most of the time, the non-data/non-technology bullet points make or break data initiatives. Namely, egos and money.
- Egos – Every organization has politics, and these politics can creep into turf wars over new company initiatives. The success of initiatives can often be a zero-sum game. This seems sad to even bring up, but I’ve seen it affect countless initiatives that it is worth bringing up. Be sure to have executive support for your data initiatives.
- Money – Remember, data and technologies are typically viewed as cost centers. Companies appear willing to take a shot at data, but will also pull the plug if expectations are not met. That’s also why expectations are at the top of this list. Setting proper expectations is the key to success.
The Data Foundation
What is a data foundation? A solid data foundation means you can succeed in all of the areas where there may be potential challenges. It means having a game plan and the means to make it happen. Many companies think that simply implementing new technologies will provide sufficient data foundation. This is far from true. A successful data foundation is as much a holistic approach to organizational change as it is investing in the hottest technologies. Without the culture and executive support in place, data initiatives will inevitably fail.
Here are some questions I suggest answering in order to calibrate where you need to focus on your data foundation:
- Do you have clearly defined use cases for using data?
- Do you have KPIs in place?
- Do you have a timeline?
- Do you have use cases that lend themselves to small tests on data?
- Culture & Politics
- Do you have executive support?
- Does anyone have a hidden agenda with the data?
- Are there any holdouts in your organization?
- Do you have a culture that embraces data?
- Do you have the team in place to execute on your data initiatives?
- Do you have a culture that embraces data-driven decision making?
- Are you a company that attracts the best and smartest talent?
- Do you have budget for technology and resources, given your criteria for success and timeline?
- Will your organization be forgiving or accommodating if you slip on your timeline or budget?
- Is data perceived as a cost center or a revenue generator?
- Data and Technology
If you answered yes to all of these questions, then you’ve already got a solid data foundation. You’re likely already data-driven or even intelligence-driven.
If you did not answer all of these questions in the affirmative, that’s OK too. Now you know where you need to focus your energy. Building a solid data foundation will take time and commitment. But the payoff will be worth it.
In addition, please take this excellent data-readiness assessment posted on Harvard Business Review. It’s an oldie but goodie, and will give you a score of where you fall on the data-readiness scale.