Think more AI output means more progress? Only if someone can truly understand it.
On October 6, OpenAI released 722 mathematical manuscripts created by an unreleased internal model. The model was posed roughly 4,000 problems, and each result used about three hours of thinking time. Many were breakthroughs in math that hadn’t been solved in decades.
Wharton professor Ethan Mollick responded on LinkedIn with a prediction: “Novel discoveries will start to come faster than we can absorb.”
Mollick was writing about academia. But inside companies, AI is starting to produce results faster than people can absorb them too.
So what happens when answers arrive faster than the people responsible for them can understand them?
Do we just look at the results and confirm they’re good answers without knowing how the AI came to that conclusion? And why do we keep asking for more output when we can barely keep up with what we’re getting?
We Need to Understand What We Stand Behind
When answers come faster than we can follow or understand them, people respond in one of two ways: they either trust the machine and stop validating its results or dismiss the whole thing as “AI slop.”
Both responses miss the point that we likely don’t understand what was just done to achieve the result. And understanding gives us a sense of control. When we can track how answers were derived, we can defend them and build upon them to get the next set of answers (e.g., to learn and make progress).
Researchers from Carnegie Mellon University and Microsoft surveyed 319 workers who use generative AI at work and collected 936 real examples of that use. Workers with more confidence in the AI did less critical thinking about its output, while workers with more confidence in their own abilities did more.
In other words, the less competent we feel in evaluating what AI is doing, the more credibility and authority we give to the tool without questioning it. And that dynamic is likely only going to increase.
Garbage in, garbage out
Faros AI, a company that measures engineering performance, analyzed two years of data from 22,000 developers across 4,000 teams. When AI adoption was high, code changes grew 51 percent larger, median review time increased fivefold, and production incidents per code change tripled.
The code got written faster. The people evaluating it before it shipped couldn’t keep up. More problems reached production.
The same gap is opening up in other business functions. A marketing team can now create dozens of campaign variations in minutes. A market analyst can produce a 30-page competitive assessment in an hour. A paralegal approves AI-drafted contract language because it reads well.
In each case, the dramatic efficiency gains look like progress. But a decision is only as good as someone’s understanding of what’s behind it. Many leaders are asking AI to distill its own overwhelming amount of output into specific actions and strategies. That’s great, but only if you know what went into the meat grinder in the first place. Garbage in, garbage out.
Mathematicians wrote the rule that business needs now
Mathematics has a long-standing rule. The people who put their names on a paper must understand the argument, check that it’s correct, and take responsibility for it.
AI-generated results break that rule. Because the person who prompted the model may not understand the proof in the first place. So, an advisory group at the Institute for Advanced Study gathered more than 600 responses from mathematicians and published recommendations about how to handle this dilemma. A key point: an AI lab that releases results nobody yet understands must make sure human understanding follows. It’s one thing to provide the answer. It’s another to help people – real human beings – understand how it was achieved.
OpenAI has now committed to funding workshops and conferences to help mathematicians understand the results. That response shows human understanding can be built into the process: Machines verify what can be checked automatically, and dedicated time and effort help people grasp the rest.
Grow your capacity to understand
Before you add more AI output, make sure your organization can absorb what it already produces:
-
- Name an owner who understands. For every AI-assisted decision, assign one person who can explain how the answer was reached.
- Let machines check the routine. Give another AI agent the job to validate data and content, so the human owner can focus on higher order thought processes.
- Budget time for understanding. Track how long reviews take, and fund it the way you fund the tools that create the output.
Mathematics is the first field to face Mollick’s prediction directly. It won’t be the last.
I publish Leapfrogging the Headlines to help leaders cut through the noise, gain clarity on what’s happening, and make smarter decisions. Subscribe below to get each issue delivered free.
Join 30,000+ other leaders