Issue #53 — The Knowledge No One Holds

We can now produce more than we can understand, and the gap is becoming a liability

Dear Reader,

Last September, Harvard Business Review put a name to something most of us had already started receiving: workslop. The researchers, from Stanford and the firm BetterUp, defined it as AI-generated content that looks like finished work but carries no real thinking, so whoever receives it has to supply the thinking themselves. They surveyed more than a thousand US employees. Forty-one per cent had been handed workslop in the previous month, and each instance cost the recipient close to two hours to decode, repair, or quietly redo.

The deeper condition is the one that matters, and it stays unmanaged because we have no mechanism to measure it.

The economics of knowledge work have shifted, and the accounting for it has not caught up. Producing a document, a report, or a block of working code used to cost more than understanding it did: you had to grasp the material and then actually build the thing. That link has broken. A model now produces in seconds what still takes a person hours to absorb.

So organisations fill up with material no one has read: decks built in minutes and skimmed for thirty seconds, analyses pasted into the next deck, code that compiles and passes review but that no one on the team understands. It all looks like work getting done. Little of it leaves anyone knowing more than before.

This is cognitive debt. It resembles technical debt but is not the same thing. Technical debt is messy code you chose to ship and meant to tidy up later. Cognitive debt is the understanding you never built in the first place, because the machine did the part that would have taught you something.

Why machine-made content does not stick

The mechanism is older than AI. Psychologists have known since the 1970s that we remember what we produce far better than what we are handed. Slamecka and Graf named it the generation effect in 1978, and it has held up under forty years of replication. Working an answer out for yourself fixes it in memory in a way that reading the same answer never does.

A second effect compounds the first. When we expect a tool to hold information for us, we stop holding it ourselves. Sparrow and colleagues showed this in 2011, in the age of Google: people who knew they could look something up later remembered where to find it instead of what it was (Science). Replace the search engine with a model that writes the whole answer, and the same instinct reaches thinking itself. We come away holding the prompt we typed and very little of what it told us.

A third effect makes the first two harder to notice. Fluent output makes us feel we understand things we do not. Rozenblit and Keil named this the illusion of explanatory depth in 2002: people are confident they could explain how a bicycle or a market works, right up until they are asked to. A polished AI briefing produces the same confidence, and its fluency gets mistaken for understanding the reader has not actually got.

All three show up together in one study. Bastani and colleagues, writing in PNAS last year, gave around a thousand secondary-school students access to an AI tutor. Those who used it without guardrails scored 48% better on practice problems. When the tool was taken away for the exam, the same students scored 17% worse than peers who had never had it. They had performed without learning, which is exactly what the 17-point drop on the exam shows.

The cost moves to someone else’s desk

The person who generated the slop saved real time and felt more productive. The cost reappeared three desks away, in someone else’s afternoon, and nobody counted it. Across a company this produces a story that flatters everyone individually while the whole moves slower. HBR put the hidden tax at $186 a month for every affected employee. With 41% of people on the receiving end, that comes to more than nine million dollars a year in a firm of ten thousand, and no dashboard records it.

That is why it is so easy to miss. It registers as speed and saved time; the cost is real but lands later, on a different desk, where nobody connects it back to the saving.

When the content has nothing underneath it

So far this assumes the AI output is at least correct, and that the only thing missing is the understanding a person would have built by producing it. A lot of what gets generated does not even reach that standard.

When someone opens a general chat tool and asks it to write the report, the model has no access to the company’s data, its files, or how it actually runs. It produces text that reads well, because that is what these systems are trained to do, but it cannot anchor that text to anything true about the business. The result carries invention: numbers no one checked, claims shaped to fit the sentence, confident statements about a company the model has never seen.

This content is dangerous in a different way: it can be simply false, while sounding just as confident and authoritative. The polish is the problem. A reader takes the fluency as a sign of substance and has no independent picture to test it against.

What it does to the organisation

Keep this going for a few years and three parts of the organisation start to suffer.

The first is institutional memory. A company can document every decision and still forget how it works, because the understanding lived in its people. Hand the reasoning to a model and the documents multiply while the people who held the reasons leave or move on. What remains is a well-documented company that can no longer explain itself.

The second is the supply of judgement. The seniors who can tell when an AI output is quietly wrong earned that instinct by doing the work themselves for years. Their juniors are not doing it now. Stanford’s Digital Economy Lab found the first hard evidence in payroll data last year: employment for workers aged 22 to 25 in the most AI-exposed jobs has fallen by about 16% since the technology arrived (Canaries in the Coal Mine). The company keeps the judgement its current seniors built and stops growing the next generation’s.

The third is resilience, which shows up only when the rare hard case arrives. A Harvard and BCG study of 758 consultants found AI raised performance on tasks within its competence and lowered it on tasks outside, where people deferred to confident, wrong output (HBS working paper). The work AI absorbs is mostly routine, and routine work is where people build the judgement the harder cases need. So the exception arrives, the AI cannot handle it, and the human beside it is out of practice.

The same pattern, one level up

There is a version of this at civilisation scale. Models trained on the output of other models degrade. Shumailov and colleagues, in Nature in 2024, called it model collapse. It is the equivalent of inbreeding: trained over and over on its own output, a model degenerates, loses diversity, drops rare variants, and reproduces an ever-narrower, averaged-out version of reality. A society that keeps producing information almost no one absorbs is running a slower version of the same experiment on itself.

What changes when the systems get good

The obvious reply is that this is a problem of crude AI use, and better use removes it. Build proper agentic systems rather than chat windows: grounded in the company’s own data and processes, constrained so they cannot invent, autonomous enough to produce output that holds up. Does that solve cognitive debt?

It helps with part of it. A grounded system produces far less of this, and the knowledge it draws on lives somewhere durable and inspectable instead of disappearing after each prompt. The knowledge does not all have to sit in people’s heads; it can live in a corpus and an architecture that reflect how the business works. That is a real improvement on a chat window.

The harder part does not go away; it moves to fewer people. Someone still has to know what the system contains and what it leaves out, where it stops being reliable, and when its grounding has gone stale. That is a smaller group holding a more demanding kind of knowledge. And the better the system runs, the less anyone feels a need to understand it, which is when people stop checking. The rare case it gets wrong is then the one no one is ready for.

Aviation learned this with the autopilot: as the automation grew more reliable, manual skill faded (the FAA documented this in 2013), and the cost arrived on the rare day it gave out. Overall, though, air-transport safety keeps rising — the automation keeps improving, and there are also detailed, comprehensive procedures that pilots and technical staff follow. The same risk, and the same opportunity, sit inside a good agentic system. The job is to decide which people must understand the system itself, and keep them close enough to its reasoning and limits to step in when it fails.

What a serious organisation does about it

None of this argues against using AI, and I am not arguing that. The question is where the thinking happens, and who is left holding it.

The companies that handle this well will be deliberate. They will decide which work people must do themselves, because some tasks exist to build understanding in the person doing them, and handing those to a machine throws that understanding away. They will treat a flood of AI output as a management problem in its own right, and set up the work so that, before anything reaches a customer, a regulator, or a board, a specific person can still reconstruct the reasoning behind it.

The law is moving the same way. The EU AI Act requires human oversight of high-risk systems, and that oversight only counts if the person can actually follow the decision. An overseer who signs off without being able to explain how the system reached its answer has not overseen anything.

Designing work so that it builds understanding instead of limiting it is a solvable problem. It comes down to what roles we define for people and which decisions stay in their hands. It is what I spend most of my time on now, and the work I expect most serious organisations to be doing within two years, whether they plan for it or not.

Briefing

SpaceX agreed to buy Anysphere, the company behind the Cursor coding agent, for $60 billion in an all-stock deal, one of the largest acquisitions yet in enterprise AI tooling. CNBC frames it as a move to close the gap with rivals in the AI coding race. The AI-coding layer is consolidating into a few large owners, which sharpens the question of who will control the tools your engineers depend on in two years.

Salesforce is buying Fin, the autonomous customer-service agent formerly known as Intercom, for about $3.6 billion and folding it into Agentforce (TechCrunch). Agentic customer service is moving from pilot to platform, and the human-oversight questions move with it.

Microsoft has switched Copilot Cowork to usage-based pricing and is reportedly weighing a Microsoft-hosted version of DeepSeek as a cheaper model option (Axios). The shift from flat fees to metered billing is a reminder that the economics of enterprise AI are still moving under everyone’s feet.

Questions for your leadership team

  1. Which tasks have we handed to AI mainly to save time, when the real value was the understanding the person would have built by doing the work? Have we given any of that away without meaning to?
  2. If KNF, UODO, or our own board asked someone here to reconstruct the reasoning behind an AI-assisted decision, could they do it? The EU AI Act counts oversight as real only when the overseer can follow the decision.
  3. Who is still building the judgement that lets us catch the model when it is confidently wrong, and what happens to that supply as juniors stop doing the work that produced it?
  4. When we judge the value of AI, do we look only at what it produced, or also at what our people understood afterwards? Measuring only the output hides the understanding we may be losing.

Summary

AI has made producing content nearly free while understanding it stays expensive, and the gap is cognitive debt. It is easy to miss because it registers as faster work rather than a visible loss, and because the cost usually lands on someone other than the person who created it. Left unmanaged, it wears down institutional memory, the supply of judgement, and the resilience you depend on for the rare hard case. The companies that handle it well stay deliberate about where people still have to do the thinking themselves.

Stay balanced, Krzysztof Goworek

Krzysztof Goworek is founder of Quintant — AI advisory that gets enterprises from experiment to production value.