Data Silos!? Danger Will Robinson!!
At the recent Strata+Hadoop World conference, I was asked a very thought-provoking question:
What do I see as the biggest cultural threat to organizations trying to realize the full business potential of big data?
I talk frequently about the need for IT and business to collaborate early and throughout the big data journey. I recently wrote a blog (“Creativity Is A Team Activity in Big Data”) about the need to create an environment that allows and encourages business and IT stakeholders to freely contemplate and brainstorm what types of data insights could yield the greatest business value.
However, a significant cultural indicator about whether an organization is able to realize the business value from big data are data silos. The existence of data silos indicates the organization’s unwillingness to share data across the different business units. The existence of data silos is an indicator that parts of the organization are more focused on their own performance then they are the overall good of the organization. And that’s a bad sign for big data.
Here is why data silos are a big data killer:
The existence of data silos is a political issue, not a technology issue.
For some organizations, data fiefdoms have equated to organizational power. Those organizations have invested to create their own, private data warehouses or data silos. While in some cases there may be privacy or compliance reasons for creating these data silos, more times than not the data silos are a remnant of an age gone by where controlling the data meant controlling the power. Man, that’s so old school thinking.
In today’s big data world, sharing data creates more power and clout than monopolizing or hogging the data – essentially capping the possible use cases. Data sharing is not a zero-sum game where others in the organization have to lose (because they don’t have access to your data) so you can win. In fact, data silos can lead to wrong decisions and give more nimble, data-sharing competitors a significant advantage in grabbing more of the industry’s profits. Let’s walk through an example of what I mean.
Financial Services Example
Let’s say that you are a financial services company that markets a full range of financial products including credit and debit cards, checking accounts, savings accounts, retirement accounts, mortgage loans, automobile loans, student loans, brokerage services and high personal wealth services. A huge challenge to that organization (as well as many organizations) is determining the Customer Lifetime Value (CLTV) for each of your customers.
Managing the data infrastructure individually for each part of the organization instead of the whole can lead to sub-optimal and even fatal decisions. For example, the customer who is late on a credit card payment (so you cancel their credit card mid-trip) may have a significant retirement portfolio being managed by the financial services organization. Or rejecting a mortgage application (due to a low income level) to a customer who has retired and is a major customer of the high personal wealth services division could lead to unintended consequences.
If you can’t share the data across the different business units, you can’t create a singular understanding of your customers’ credit card tendencies, checking account propensities, portfolio preferences, financial goals, investment strategies and risk tolerances across all the different lines of businesses. If you can’t share the data across the different business units to provide a more complete “analytic profile” on each of the organization’s customers, then the organization will find it difficult to create a comprehensive customer lifetime value score that allows each part of the organization to prioritize their customer engagement and marketing decisions.
Figure 1: Creating Customer Lifetime Value Score
But wait, the problem gets even worse in the world of big data. Just because a customer is classified as very important via their CLTV score, does not mean that you have a quantifiable understanding of their maximum potential lifetime value (MCLTV). Organizations rife with the politics that inhibits the sharing of data will have even more difficulty in determining the maximum (potential) customer lifetime value that allows the organization to optimize their customer engagement and marketing investments with that individual customer. These types of organizations will struggle to uncover new sources of customer, product and operational data that can benefit the entire organization by creating a more actionable MCLTV score (Figure 2).
Figure 2: Customer Maximum Lifetime Value
As can be seen in Figure 2, without an understanding of a customer’s maximum lifetime value, organizations may be wasting their sales and marketing efforts targeting customers whose value has maxed out. Wouldn’t it be more valuable if you knew the maximum customer lifetime value so that you could market to those untapped customers? If your organization is rife with data silos and won’t share data across the business units, your organization will struggle to create an accurate view of the potential maximum value of each of your customers.
The calculation of a Maximum Customer Lifetime Value (MCLV) is critical if organizations are trying to optimize their sales and marketing investments and to extract as much value (and to deliver as much value) as possible at the individual customer level. However, having siloed insight of customers could lead organizations to waste their sales and marketing efforts targeting customers who have already reached their maximum value.
Data silos indicate problems ahead for organizations that are trying to maximize the business value of big data. Not only is data hording an outdated mentality, but also it can lead to suboptimal and even fatal decisions for the organization. Data silos are a political issue, not a technology issue. That means that data hording is rooted deep into the organization and will significantly limit an organizations ability to get value out of their data and compete against more nimble, data sharing competitors. Can you say disintermediation?
It’s time for organizations to leave that old school thinking behind and move into the age of data abundance and data sharing. If not, organizations really do risk being “Lost In Space.”
By the way, here is some proof that I actually attended the Strata+Hadoop World conference (and wasn’t sneaking out to catch baseball games):
Information Week interview with Jessica Davis:
Bill interview on theCube at Strata: