Do not trust this cat.
When we do a Data Governance assessment, one of the first things we do is interview people from various roles in the organization. One of our initial questions is along the lines of, “What establishes the basis for the trusted data that you use every day?”
Here are some answers to that question from the past decade and my two cents on each one of them.
Consistency
Answer: “I trust the data if it is the same as it has been in the past.”
My two cents: Wow. So, if the data is the same as it has been previously, then it is correct. If it is too different, then it must be wrong. It’s about consistency versus being correct.
Convenience
Answer: “I trust whatever is put in front of me because I don’t know how to verify, don’t have time to verify, don’t care to verify.”
My two cents: This is a convenient approach given how busy folks are and the fact that they’re being asked to do more with fewer people and less time. Still, I want to know how the sausage is made.
Experience
Answer: “I trust my gut and the data supports that feeling… or it doesn’t.”
My two cents: In a day before exception-based analytics being sent to your phone or even green bar reports being dumped on your desk every night, experience was all many people had. Even the smartest and most experienced people should not trust their gut.
Personal
Answer: “I trust this person, so I trust the data they provide.”
My two cents: Before I spent a decade working in Data Governance, I leveraged my personal relationships with people for my trusted data. The problem is that people get promoted, leave the company, or retire. They also make mistakes and get feisty on occasion.
Process
Answer: “I trust how this data is managed.”
My two cents: These folks understand that data gets created and has things done to it, sometimes bizarre in nature, before it lands in a repository where a reporting tool is pointed. I like what these folks are thinking because they’re interested in the process that creates, moves, and stores the data. Unfortunately, sometimes parts or all of these processes are not correctly documented or happen outside of IT managed systems which can introduce variability.
Source
Answer: “I trust the source where the data comes from.”
My two cents: If the new leads report comes from the Sales Force Automation system where lead information is managed, then the data is gold. These folks always want to know the source of the data and want the analyst to get data directly from the horse’s mouth. However, access may be difficult to get, results that are generated may not be repeatable, and revisions are made to incorrect data entries continuously.
Technology
Answer: “I trust the technology that manages and presents the data.”
My two cents: A lot of people align with this basis of trust. Unfortunately, there’s a lot of business logic in Extract, Transform and Load processes. End of day process fails to run (or runs twice), or the technology you trust is a MS Excel spreadsheet on a desktop on the third floor managed by the summer intern. It’s simply not good enough to trust the technology.
I am guilty of making important business decisions based on all of these. I used to combine several of these together because it somehow made it more trustworthy. You’ve done the same thing and maybe you’ve been burned or just lucky to this point.
What got me thinking differently was when I helped lead the conversion of an old data warehouse and reporting tool to updated technologies and capabilities. The goal of the project was to ‘lift and shift’ from old to new. However, as we went along the project, we found opportunities to make the data ‘righter’ than it had been in the past. We saw this as an opportunity to move from subjective analytics to objective analytics. Don’t trust it is right, know it is right.
That phrase still rings true to me today — Don’t trust it is right, know it is right.
How do I know my data is right?
It starts with you. You need to become an educated consumer of the data that you use to manage your piece of the business every day.
Here’s what I’d want to know:
1. Defined – Has the business and technical metadata been defined and agreed upon by the organization?
2. Traceable – Do I know where the data has been and what has happened to it along the way?
3. Quality – Is it of the appropriate quality for what I’m trying to do with it?
If I know those three things, then it doesn’t matter where the data comes from or who gives it to me.
The other phrase I hear now is “fact-based decisions”. A fact is a thing that is indisputably the case. If something is indisputable, it is unable to be challenged or denied. I’d want to know the same three things to make a fact-based decision.
Maybe the right answer to that interview question, “What establishes the basis for the trusted data that you use every day?” is that the basis of trust is built on facts about the data.
As a final thought, I’ll leave you with a quote from one of my favorite fictional characters.
“Archaeology is the search for fact, not truth. If it’s truth you’re interested in, Dr. Tyree’s philosophy class is right down the hall.” ~ Indiana Jones, Jr.
