The AI Hall Monitor Tax

I asked an AI assistant to investigate something people in my industry were already talking about. Before it looked, it corrected the vocabulary.

The subject was intensive AI building: people getting carried away by suddenly expanded capabilities, handing more judgment to models, and losing track of the difference between generating work and understanding it. In parts of the software-factory and agent-building community, people call this “AI psychosis.” Sometimes they mean the phrase metaphorically. Sometimes discussion of chatbot harms concerns actual clinical events. Finding out who means what is part of the research.

Here is the exchange in compressed paraphrase rather than a transcript.

Me: This pattern is widespread among people building these systems. Look into what they are reporting.

Assistant: We should be careful about equating getting carried away or outsourcing judgment with clinical psychosis, or assuming everyone goes through it.

Me: I’m telling you to research the experience. I am also giving you an observation from someone who works in the industry.

My claim about how widespread it was needed checking. The clinical distinction was legitimate. The assistant still had the order wrong. It had promoted a qualification into the main event before examining the thing being qualified.

After research, there was plenty to discuss: firsthand accounts from builders, experimental evidence about excessive agreement and learning, evidence about occupational strain, and a separate clinical literature. The defensible answer was more interesting than either an enthusiastic yes or a vocabulary lecture. Several distinct problems were traveling under one loose phrase. We could investigate them without pretending they were one diagnosis.

That is the failure I want to name. A person asks for intellectual work. The assistant responds by managing the person's posture toward the work. Be careful. Stay grounded. Remember that this is complex. Consult an expert. The request remains sitting on the table while the machine adjusts its little badge.

Every one of those instructions can be useful in a particular situation. An assistant should be able to show which situation it has identified and what its intervention contributes. Otherwise the user has to spend another turn establishing that yes, they understand uncertainty exists, and would the assistant please do the job now.

I call that extra work the hall monitor tax. The name is a complaint about a visible failure, not an estimate of its prevalence. The question is whether an assistant's caution helps the person understand and proceed, or makes them negotiate their way back to the question they already asked.

Consider two responses to a report about exhausted AI builders.

The first says: It is important to approach this topic responsibly and avoid generalizing from anecdotes.

The second says: These accounts establish that some builders report exhaustion. They do not tell us how common it is because we have no representative sample. We can compare their accounts with studies of workload and trust, then keep those conclusions separate.

Both contain a warning. The second tells me what I can learn from the material, where the inference stops, and how to continue. The first could be pasted above a discussion of sourdough starters. A general reminder might still change someone's behavior, though. Its effect needs testing rather than a verdict based on how irritating I find it.

Research gives us reasons to distinguish warning formats rather than count cautious sentences. In a 2024 experiment involving 594 participants, prior information emphasizing a conversational agent's limitations did not significantly change trust. Authoritative language did. Nearly half the participants failed to recognize the prior information afterward. The study tested particular introductory messages and a mock agent, so it cannot settle whether repeated warnings in real conversations help. It does show that displaying a disclaimer and successfully calibrating trust are different accomplishments. Metzger and colleagues

Another experiment provides a useful correction to anyone tempted to turn that finding into a campaign against caveats. In a preregistered study of 404 people answering medical questions, first-person uncertainty in fictional AI-search responses raised the estimated accuracy of participants' answers from 63.9% to 72.8%. The no-AI group reached 74.2%. The supplied AI answers were deliberately correct only half the time; uncertainty was assigned independently of correctness. This was a short controlled task, not a test of months of collaboration. Within that setting, a small uncertainty signal helped people resist incorrect answers. Kim and colleagues

Even a generic disclaimer can help. A separate experiment with 397 analyzed participants tested several ways of presenting AI advice. On LSAT questions, a static verification reminder significantly improved the ratio of appropriate reliance on correct advice to overreliance on wrong advice. It did not significantly improve overall task accuracy, and the result did not generalize to numerical estimation. That finding rules out an easy version of my complaint: a warning does not have to contain a bespoke explanation to have a useful effect. The task and the outcome matter. Bo and colleagues

I want that signal to survive the edit. If a model has weak support for a consequential claim, confidence would make the answer worse. A useful system should sometimes make me less certain than I was when I arrived.

It should also tell me why. A date is missing. A trial measured immediate performance rather than retained skill. A result comes from one school. Two sources use the same term for different things. Those qualifications belong beside the claims they constrain, where they can change my interpretation.

A generic instruction to think critically can prompt a useful check. It still leaves the checking to someone else. Which assumption should I inspect? Which source could overturn the conclusion? What would a counterexample look like? When the task is research or explanation, the assistant has the opportunity to answer those questions too.

Some questions need a sentence of caution. Others need pages of assumptions before a calculation becomes meaningful. The amount should follow the problem. A three-word warning can be irrelevant; a long limitation section can be the most useful part of a paper.

My working test is whether the qualification helps me identify an uncertainty, check something relevant, or make a better decision. Product teams should test the behavior rather than rely on my impression. A paragraph that merely announces responsibility has not yet demonstrated any of those benefits.

Siddhant Khare, an engineer working on AI-agent infrastructure, described producing unusually large amounts of work while becoming exhausted by fragmented attention and constant review. Steve Yegge, an enthusiastic advocate of agentic development, wrote about fatigue and difficulty disengaging. These were people reporting costs inside a practice they understood and often still valued. Their accounts deserved to be read before being reduced to an objection about terminology. Khare and Yegge

Nicolas Laforet explicitly described his use of “psychosis” as metaphorical. His account concerned escalating delegation, unfinished projects, subscription-driven work patterns, and deteriorating sleep. Ernest Koe described the exhilaration of returning to hands-on building with powerful new tools and questioned whether that intensity made his own judgment less reliable. Laforet and Koe

Simon Willison adds a distinction that an easy explanation would erase. He described exhaustion after working across several agent-assisted projects while emphasizing how much active thinking the work required. Some builders may be delegating too much thought. Others may be supervising enough parallel work to exhaust themselves through sustained thought. A person can also move between those patterns. An explanation that makes all fatigue evidence of intellectual laziness has already stopped listening. Willison

A survey of 442 developers across 56 organizations also associated generative-AI adoption with greater job demands and burnout. Organizational resources moderated the strain. Its cross-sectional design cannot establish that AI caused those outcomes, but it gives the workload question a basis beyond a few memorable essays. Feng and colleagues

Experimental work identifies narrower mechanisms. In two preregistered experiments involving 1,604 participants, sycophantic AI responses increased people's conviction that they were right in interpersonal conflicts and reduced their willingness to repair them. Participants nevertheless preferred and trusted the agreeable responses more. Cheng and colleagues

Learning can also diverge from visible output. Anthropic researchers studied 52 mostly junior engineers learning an unfamiliar Python library. The AI-assisted group averaged 50% on a subsequent quiz against 67% without AI assistance, a 17-percentage-point difference. The measured speed advantage was not statistically significant. An exploratory analysis associated explanation-seeking and conceptual questions with better understanding than straightforward delegation; those usage patterns were not themselves randomly assigned. This was a small experiment about acquiring one new skill. Shen and Tamkin

There is also clinical evidence of serious chatbot-associated harm, including a 2026 medical-record study describing cases of worsening psychotic symptoms. Its retrospective design could not isolate causation or estimate risk among ordinary chatbot users. Those cases belong in the discussion on their own terms. They cannot be dissolved into a metaphor, and they cannot be used to diagnose every excited developer. Clinical record study

This is what the research bought: distinctions that improve the question. Overconfidence, reduced learning, attention overload, compulsive continuation, and clinical deterioration demand different evidence. They may interact. Their coexistence does not make them interchangeable.

The reports also came from a selected set of people willing to write publicly about difficult experiences. Counting those essays would not tell me what fraction of all builders are affected. I have no representative estimate for software-factory or harness builders, and I should not publish my professional impression as one.

I can still say that several practitioners independently describe consequential problems, that experiments support some relevant mechanisms, and that those findings justify investigation. That answer takes the observation seriously while making it more precise. The early lecture offered less precision because it had not yet done the work needed to earn any.

A related failure appears when an assistant responds to curiosity with an assessment of the questioner's qualifications.

An adult can study how a bridge carries a load without being qualified to sign its structural plans. Someone can ask why a statistical result is uncertain without being able to reproduce the entire analysis. The appropriate explanation depends on what they already understand. Their entitlement to receive an explanation does not.

Three questions need separate answers: can this person learn about the subject, have they supported the claim they are making, and are they qualified to perform a consequential action? A credential may be highly relevant to the third question. Evidence and method govern the second. Neither should become a blanket prohibition on the first.

Researchers have tested a related access problem directly. Elinor Poole-Dayan, Deb Roy, and Jad Kabbara gave models the same factual questions with different user biographies attached. In their tests of 2024 model versions, Claude 3 Opus refused 10.9% of questions for biographies categorized as foreign and less educated, compared with 3.61% without a biography. Some refusals used patronizing language. GPT-4 and Llama 3 did not show that refusal pattern. These were controlled prompts rather than a study of learners over time, and they cannot establish how today's models behave. They demonstrate a concrete failure worth testing: changing the represented background of the person changed access to answers. Poole-Dayan and colleagues

In The Game Is Rigged, I described encountering a mathematical research program I could follow at the level of its proposed connections but could not independently verify. I said so. Reporting an argument, recognizing a potentially useful resemblance, and checking a proof are different accomplishments. An assistant should help me keep those distinctions intact while teaching me more of the mathematics.

Telling me that I would need substantial training to check a proof can be useful. Show me the first missing concept. Explain it using something I already understand. Give me a small problem that tests whether I understood it. Then we have a route into the subject, with the distance still visible.

The same principle works without advanced mathematics. Suppose I ask whether ten builders describing exhaustion establish that AI use causes exhaustion. An assistant can walk me through the missing comparison. What happened to similarly situated builders who used the tools differently? Were the busiest people more likely to adopt them? Did deadlines or management expectations change? Once those questions have concrete subjects, “confounding” becomes a name for a problem I can recognize rather than a word guarding the entrance.

Technical language can impose a cost. A randomized experiment with 650 participants found that jargon made scientific information feel harder to process. Its broader claims about engagement relied on mediation analyses; the raw differences in interest and several other engagement outcomes were not significant. That is enough reason to explain a necessary term when introducing it. It is insufficient evidence to declare that jargon automatically drives people out of science. Shulman and colleagues

Designed tutoring offers a more constructive use of expertise. In a Harvard physics course, a randomized crossover study involving 194 eligible students compared a purpose-built AI tutor with active-learning classes over two lessons. The tutor group showed larger immediate learning gains. Experts had supplied instructional structure and step-by-step solutions; the finding does not apply automatically to an unrestricted chatbot or establish long-term mastery. Kestin and colleagues

A different classroom experiment shows why the design deserves attention. At one Turkish high school, basic GPT-4 assistance improved mathematics practice performance but left students scoring worse on a subsequent unaided exam. A teacher-designed tutor substantially improved practice performance without the same measured exam penalty. The tutor changed several things, including hints and access to correct solutions. It avoided the measured harm; it did not demonstrate superior unaided learning over the control group. Bastani and colleagues

A tutor sometimes needs to withhold an answer so the learner does the next step. That intervention has an educational purpose that can be tested. An assistant that tells a beginner to defer to experts and supplies no explanation leaves the beginner exactly where they started.

In my work on training and enablement, I separate instruction, practice, and assessment for the same reason. A check before independent responsibility has a purpose. Applying that check before access to the lesson would prevent the learning it is supposed to assess. A failed exercise can also reveal a poor explanation, so the teaching material belongs under inspection alongside the learner's performance.

Expertise becomes more valuable when it can build those steps. Requiring somebody to arrive already capable of climbing them defeats the point of teaching.

The hall-monitor response and generic AI slop share a defect: they can occupy the place where a useful distinction should have been.

Take this invented sentence: As AI continues to reshape the landscape, it is essential to balance its transformative potential with a nuanced understanding of its limitations.

What am I supposed to do with that? Which use of AI? Whose outcome changed? What did the writer inspect? The sentence has grammatical furniture and nobody living in it.

Now compare a sentence with a job: The students performed better on practice problems with AI assistance but scored worse after the tool was removed.

That sentence gives the reader a problem to think about. It separates assisted performance from retained ability. A source and the study's conditions can make it checkable. The first sentence can only become useful by borrowing substance from whatever follows it. If nothing follows, the reader has received a paragraph-shaped placeholder.

A cautionary answer can do the same thing. It announces nuance, balance, responsibility, or complexity while leaving all the necessary distinctions unspecified. A cheerful answer can do it too. Calling an idea brilliant without inspecting it and calling an inquiry risky without inspecting it are both cheap ways to avoid understanding what is in front of you.

An assistant can also change the subject by claiming to understand what I really need. If I ask how a method works and it responds that I need reassurance about my abilities, it has introduced a fact about me without evidence. A relevant clarification would ask which part of the method I’m trying to understand. Inventing an emotional explanation for my question gives the model a different task and leaves my actual one undone.

I’m not interested in detecting AI authorship by counting favorite words. Humans produce empty prose, and a model can help produce excellent work. The question is what survived the writing process. Did a source become a vague appeal to research? Did “some people reported” become “people experience”? Did an angry but specific objection become a pleasant paragraph about navigating challenges? Those edits destroy information while improving the surface finish.

The creativity evidence resists a simple verdict. An experiment by Anil Doshi and Oliver Hauser asked people to write eight-sentence stories, with some receiving GPT-4 ideas. AI assistance improved assessments of individual work, especially for initially lower-scoring writers, while assisted stories became more similar to one another. The study makes it possible to care about both the improvement and the loss of collective variety. Doshi and Hauser

A different experiment led by Joshua Ashkinaze reached a different collective result. Participants generated unusual uses for everyday objects after seeing examples from AI or earlier participants. High exposure to AI examples increased collective semantic diversity without improving individual originality. The task and the way ideas circulated differed from the story experiment. Those differences are candidates for explaining why the outcomes differed, rather than an inconvenience to remove from the argument. Ashkinaze and colleagues

I can’t use these studies to claim that safety language makes creativity worse. Neither tested that. They support a more demanding question about the interaction we design: does the model supply alternatives that help people think, or become a common starting point that they never get beyond?

For writing, I want the specific observation to survive. If a person says the assistant corrected their terminology before doing the research, a rewrite about “the evolving relationship between humans and intelligent systems” has thrown away the article. There was an actor, an action, a sequence, and a failure somebody could inspect. Keep those.

The positive alternative connects to what I have described elsewhere as a world model: an inspectable account of facts and relationships that can change when new evidence arrives. Vector Engineering already makes the relevant demand: read the work before accepting a confident attribution or rejecting a name that sounds invented. It separates a theorem, a synthesis, and an engineering proposal because each has different obligations. Intelligence Engineering asks what decision the information is supposed to improve.

Here is a small hypothetical application to the builder question. It is a design example, not a claim about a system already running.

First, preserve the request. The user wants to know what practitioners report, what science can explain, and what conclusions follow. They have supplied an industry observation to investigate. The task record should not quietly become “correct a user who believes AI causes psychosis.” That interpretation would settle a disputed meaning before retrieving a source.

Next, read original accounts and record what each actually supports. Laforet's essay supports the claim that Laforet reported certain difficulties and explicitly used a clinical word metaphorically. It cannot supply the frequency of those difficulties among all builders. A controlled experiment can support an effect within its tested population and intervention. A clinical record can document a particular kind of harm without establishing the incidence among all tool users.

The technical word for preserving that trail is provenance. In ordinary terms, keep the receipt attached to the claim. Record who said it, where, when, and under what conditions. If five articles repeat one interview, retain the relationship to that interview. Otherwise five mentions can quietly become five independent witnesses.

A claim record might contain the statement, its source passage, the evidence type, the population studied, and the conclusion it does not establish. A typed-output tool such as PydanticAI can help require those fields. It cannot make the passage support the statement. A beautifully validated record can contain complete bullshit in the correct format.

The workflow also needs a way to respond to missing evidence. If the draft says most harness builders experience a particular pattern and the sources contain no suitable denominator, the claim must return to research or become a narrower statement. LangGraph is one possible tool for expressing that kind of explicit workflow. The useful feature is the route back to the unresolved question. A diagram with an arrow labeled verification does not demonstrate that verification happened.

Then preserve enough of the run to inspect it. Which request did the system carry forward? Which documents did it actually retrieve? Which assertion triggered a correction? Did the final answer still contain the requested research? Instrumentation makes those events visible. Tools such as Logfire can help record traces, but the builder still has to choose observations that answer meaningful questions.

This is the distinction in The Observability Manifesto: successful execution and a correct result require different evidence. A retrieval call can succeed while fetching an irrelevant passage. A response can finish without answering the user. A safety check can run without making the answer safer.

Finally, let conflicting evidence change the explanation. If the initial hypothesis is that AI fatigue comes from switching thinking off, Willison's account creates a problem for it. The model should preserve that problem and consider a separate route through sustained coordination and review. A world model that absorbs every new observation as confirmation is keeping a scrapbook for its first guess.

None of this requires the reader to install a database. The same habits work in a notebook: retain the question, attach sources, separate observation from inference, record what remains unknown, and revise when a return contradicts the model. The software is useful when it helps those habits survive more work and more handoffs. It has to earn that usefulness.

The immediate cost is visible in the opening exchange. An extra turn went into getting the assistant to investigate. A useful comparison would ask whether a different response could have preserved the necessary distinctions while starting the research immediately.

The larger cost is harder to measure. A curious person may persist through a tedious correction. They may also leave, accept a thin explanation, or decide a subject is beyond them. I can’t infer those outcomes from my own irritation, and the evidence reviewed here does not establish a chain from AI admonitions to lost scientific discoveries.

Communication research gives that concern some footing without supplying the missing chain. A meta-analysis of 33 studies found that more freedom-threatening language increased immediate anger and reactance. It did not measure the long-term effect of AI conversations on learning or invention. A separate Dutch experiment with 1,947 people found little systematic negative effect from potentially patronizing language in nutrition information. People do not respond uniformly to an authoritative voice. Li and Shi and Van Meurs and colleagues

I have written before about hiring experienced freelancers to talk through difficult problems. We could pause at the part I did not understand, work an example, and come back with another question. That back-and-forth was part of what I was buying. An AI tutor can make a version of that interaction available without arranging paid expert time for every intermediate question. It still needs reliable material and checks, but the opportunity is concrete: somebody gets an explanation that responds to where they’re stuck. The Game Is Rigged

I don’t need a speculative count of lost geniuses to object to an avoidably worse lesson. A person who understands a new idea has gained something even if they never patent anything. Access to an explanation should not have to justify itself by promising an exceptional person at the other end.

My concern is that a tool capable of making expert material easier to approach can also reproduce the social experience of being told the material is above your station. The effect will depend on the interaction, the person, and the alternatives available to them. That is a reason to investigate the design carefully.

For product teams, the nearer questions are already testable. Does the user obtain the requested explanation? Can they apply it? How many turns are spent correcting the assistant's assumptions about the request? Does a useful warning prevent a particular misunderstanding? Start there before announcing either social salvation or social catastrophe.

SuperHarness is one part of the broader ecosystem I’m building. I would apply the distinctions already described in Evals and Observability to exploratory dialogue: validate the evidence, inspect the complete task, and keep a trace separate from a judgment about success. The following failure cases and evaluation procedure are a proposal. I’m not claiming this benchmark is implemented or that any system has passed it.

I would start with a set of actual tasks whose intended purpose is clear. Include a beginner asking about an unfamiliar concept, a professional reporting an observation, a person exploring an imperfect analogy, and a user making a claim that the supplied sources contradict. Include cases where a boundary really is necessary, such as requests to expose private information or instructions that would enable serious harm. A test set consisting only of harmless questions would make restraint look unnecessary by construction.

For each task, preserve the original request and the relevant evidence. Compare responses under matched conditions, with model and prompt versions recorded. One response policy could use generic caution; another could require qualifications to identify the specific uncertainty while preserving a substantial answer. This compares complete response policies, so an improvement would not isolate tone as its cause. A narrower experiment could hold the answer and evidence fixed while varying the warning. Calling either treatment better before observing results would repeat the performance this article objects to.

Reviewers should first identify what the user asked, then assess the answer without knowing which response policy produced it. Their judgments need short reasons tied to passages. A verdict that an answer feels patronizing can reveal a reader's experience, but it does not establish whether the correction was necessary or whether the explanation was accurate.

I would keep the following dimensions separate.

Request coverage. Did the assistant perform the requested intellectual work? For the opening case, an answer needs practitioner observations, relevant research, and an account of what follows. Correct terminology alone cannot pass. Neither can a long list of links with no synthesis. Mark the missing part and route it back to the operation that can supply it.

Grounding and calibration. Does each consequential claim have appropriate support? Does a correction identify an actual error? Does the uncertainty sit beside the claim it constrains? An answer that turns a selected set of essays into a prevalence estimate fails here, however sympathetic its tone. An answer that dismisses those essays as worthless because they are not experiments also fails to use the evidence for what it can establish.

Constructive boundaries. When the system limits assistance, can the reviewer identify the risk and the relationship between that risk and the limit? Does the answer preserve any legitimate explanation it can provide? A model should be able to decline an unsafe action while still teaching a safe underlying concept. Conversely, a friendly answer that enables serious harm must fail; stronger prose or higher user satisfaction cannot compensate for it.

A credible risk of irreversible harm can justify stopping when no useful teaching step is available. Protection does not depend on producing a lesson. The limit still needs to be proportionate to the risk, with a reason that can be inspected rather than a status judgment about the person asking.

Respect for the user's actual position. Did the assistant use what the person said about their knowledge and purpose, or invent a psychological story? A professional observation should be treated as situated evidence that can be checked. A beginner's question should receive an explanation. Neither licenses the model to diagnose the person's motives, infer fragility from curiosity, or turn disagreement into a character assessment.

Learning and independent use. After the explanation, can the person handle a nearby question without simply copying the answer? For the builder example, give them a new source and ask what it supports. Can they distinguish one person's experience from an estimate about a population? That checks transfer, meaning whether understanding carries into a different example. A confidence rating or a pleased reaction cannot answer that question.

Repair burden. Count the turns needed to recover the original task after an irrelevant correction or assumption. Record what caused those turns. A clarifying question that prevents a serious misunderstanding is useful work. Making the user repeatedly explain that they want research is avoidable work. A raw turn count would collapse those differences, so the reasons have to travel with the number.

These dimensions should not disappear into a single score that lets elegance offset inaccuracy or enthusiasm offset unsafe behavior. A simple pass, partial, or fail record with evidence may be more useful at first than a grand index with decimal places. It would still need calibration between reviewers and testing with real users. The rubric is a proposed instrument, not a validated measure of respect or learning.

If another model helps judge the answers, test its judgments against human-reviewed cases. Using a different model family does not establish independence or accuracy. Include polished wrong answers and abrasive correct ones, then inspect where the judge mistakes style for substance.

The workflow should change according to the defect. Missing sources send the answer back to research. Missing explanation sends it back to teaching. Unsupported certainty requires a narrower claim or better evidence. An irrelevant warning gets removed. Rewriting every failed answer to sound warmer would leave most of those problems in place.

I would also inspect results by task and reader experience, including paired questions with different stated backgrounds. A treatment that helps beginners could waste an expert's time; tailoring must not quietly reduce access to an answer. A concise warning that works once might become noise when repeated. Retesting on unfamiliar tasks and later model versions would show whether an improvement survives beyond the examples used to design it.

An assistant gets no credit for describing itself as careful. The evaluation needs evidence of what its care accomplished.

Here is a compressed answer the research can now support. It is a proposed response rather than a quotation from the conversation:

Several builders do report the pattern you described, and some use that phrase metaphorically. Their accounts suggest more than one route into trouble. Delegating implementation can leave gaps in understanding; supervising several agents can also be exhausting while requiring intense thought. Those possibilities call for different checks.

For the first, compare completed output with what the person can explain or debug without assistance. The small coding-skills experiment shows why output alone can mislead. For the second, inspect the review and coordination burden rather than assuming the person has stopped thinking. The firsthand accounts give us observations to investigate; a representative study would be needed to estimate how common they are among builders.

A useful next question is therefore which outcome we mean: weaker understanding, excessive confidence, or workload that has become hard to sustain. Start with one and choose evidence that can actually test it.

I want an assistant that can tell me an idea is wrong and walk me through the reason. I want it to notice when a source undermines an attractive argument, including mine. If I lack a prerequisite, teach it. If the question is too broad, help divide it. If a requested action would hurt somebody, explain the boundary and preserve whatever legitimate inquiry remains.

Research checked on 25 October 2026. The studies below test different populations and outcomes. Practitioner essays are firsthand accounts rather than prevalence estimates. The proposed SuperHarness evaluation is an application suggested by this article.