Bring Back Opinionated Insights
High-velocity decision-making depends on it.
Let’s say your app just dropped 3.2% in Day 1 user retention. That data appears on your beautiful Friday dashboard.
Now, imagine two companies.
In Org A, a well-intentioned analyst logs the drop, joins a thread, and gets looped into a call. There is a vigorous discussion on possible reasons: seasonality, cohort oddity, or maybe it's the IPL! The analyst says he’ll do a deep dive and circle back next Wednesday with a detailed report. On Wednesday, he confirms what he already suspected on Friday: it was the new app update. They roll back the changes. Everyone’s happy that they approached it objectively and solved a problem.
Except, 30,000 more users uninstalled the app over those five days.
Now, Org B. Same data. Same Friday. However, the analyst with limited initial information takes a call during the Friday meeting (he is good enough to do it and is given the freedom to do it). It’s not 100% conclusive, but the latency spike post-app update feels like a solid bet. The team rolls back. Churn rates go back down.
Org B remembers two key truths when it comes to insights.
#1: Insight is only as useful as the action it spawns
In his now-famous 2016 shareholder letter, where he laid out "day 1" vs. "day 2" companies, Jeff Bezos highlights why high-velocity decision-making is a critical part of maintaining day 1 status.
“Day 2 companies make high-quality decisions, but they make high-quality decisions slowly.”
High-velocity decision-making requires a certain level of opinionated view on using data to synthesize insights.
In the same letter, Bezos called out the following key things to think about:
Most decisions are two-way doors (can be reversed)
Most decisions should be made with around 70% of the information you wish you had. More information isn’t always good for making a decision.
However, to achieve high-velocity decision-making from insights, teams need to be able to frame their insights into a compelling story. There isn’t any such thing as a 100% neutral insight.
#2: Every insight involves reality framing, aka having an opinion
As this great piece by Benn Stancil talks about, every insight, to some extent, is reality framing.
“Data, I’d argue, is consumed in an analogous way. Though it exists is an approximately raw form,4 most analysis comes with some sort of narrative.”
What data gets picked, how it is presented, and what story is being told all influence the eventual impact of the insight.
He goes on to argue that with LLMs, we will begin to see more of this: narratives and commentary around data rather than the raw data, and this could manipulate and shape our thinking, not always in the right direction.
But I’d argue that we have the opposite problem.
We’ve forgotten to be opinionated with data
This may feel counterintuitive in an age of social media where everyone appears to have a visceral opinion, but in most organizations, we’ve moved away from having clear, opinionated insights.
There are three ways in which this manifests:
Extreme data orientation: Organizations take pride in being extremely data-oriented and pepper teams with metrics, which become tables, reports, and charts that are endlessly discussed to arrive at action plans. There exists an almost cultish view of absolute data taking precedence over anyone’s informed opinion.
Tool constraints: The tools and dashboards used for conveying data also don’t help, as they promote “self-service BI,” which suggests that users should examine the raw numbers and form their own opinions, rather than relying on analysts to provide their opinions.
Extreme managerial storytelling: Organizations where the leaders can’t be bothered with data but rely on managers to bring narratives based on data. Here, narratives are often created to support politics and bolster managerial positions, rather than presenting a clear opinion that is based on a solid philosophy of the product or business.
All of this results in decision-making that falls short. It’s either too slow, being mired in endless data pulls, or it’s based on fiction that middle management creates in an attempt to guide budgets or team growth.
AI opens an opportunity to editorialize
Imagine the near future where, instead of getting your insights from a static dashboard, you are getting them from AI. Would you still be happy with the AI just providing you the raw numbers? Do you, the business manager, still want the additional job as a data interpreter because your manager wants you to be data-driven?
What if, instead, the AI were to give you its opinion on what is likely to be happening? Instead of trying to figure it out by yourself, you can make the decisions faster. Moreover, you can make more decisions than you currently can with your manual self-interpretable workflow.
And this is precisely what Bezos was talking about when he spoke about “Day 1 companies” making decisions with imperfect information but making many more decisions.
~Babbage Insight