Creating a Data-Driven Organization: Practical Advice from the Trenches

by Carl Anderson

Many organizations think that simply because they generate a lot of reports or have many dashboards, they are data-driven.

A small amount of clean, trustworthy data can be far more valuable than petabytes of junk.

Reporting tells you what happened in the past. It also provides a baseline from which to observe changes and trends. It can be interesting and can keep some investors and shareholders happy, but it is a fundamentally backward view of the world. To be data-driven, you have to go beyond that. To be forward-looking and engage in analysis, dig in, and find out why the numbers are changing and, where appropriate, make testable predictions or run experiments to gather more data that will shed light on why.

Only by understanding why something happened can you formulate a plan or set of recommendations

You never know what you might need, you often only have one chance to collect the data, and you’ll kick yourself later when you need it and it is no longer accessible.

SQL is a transferable skill; and while there are some small differences in the language among databases (such as MySQL, PostgreSQL, and Access), it is pretty much standardized, so once you know SQL you can switch among different relational databases easily.

Information is captured, processed data, while knowledge is a set of mental models and beliefs about the world built from information over time.

Design a metric to be “as simple as it can be, but not simpler”

if there is a standard, well-defined metric for website bounce rate, then use it unless there is a very good reason to define and implement a home-grown variant.

Being standardized will generate less confusion, especially for colleagues joining your teams from other organizations. It will also make it easier to compare your metrics with others in the same sector, that is, to make use of competitor knowledge and to benchmark your organization’s performance.

Best practice is to have a centralized, automated, documented, versioned single source of truth that different teams draw from.

Bottom line: use standard metrics unless you have very good reasons to deviate from them. If they are unconventional, document how and why they are nonstandard.

Precision (being stable or clustered) and accuracy (being on target)

Metrics should be precise. That is, they should return similar values if repeated with the same conditions — if you compare this to archery, it is the equivalent of a set of arrows being close to each other.

“Increase revenue” is a poorly defined KPI because it has no numerical target. If the organization increases revenue by just $5, staff can claim that the target is met and stop trying.

Reflect what the organization is trying to achieve It is too easy to fall into the trap of tracking what can be easily measured, such as reponse time to answer calls in a customer center, when the true objective might be to increase customer satisfaction.

taking a vague goal, such as “to transform the performance of our customers,” having a conversation to dig into those weasel words, and understanding and replacing them with more sensory-specific language, such as “when our customers work with us, they can achieve their business targets much sooner.” With that, it then becomes easier to define more tangible, concrete metrics to achieve that goal,