I tried to replace myself with AI

AI Replacement Dysfunction is a name researchers have proposed for an intense, persistent fear of losing your job, or becoming obsolete, to AI. It brings paranoia, worthlessness, resentment and hopelessness, even without any other psychiatric condition. That seems realistic, given the changes in coding, in the arts, and now in frontier math. So I wondered whether it applies to my job over the last nine months.

Since late September, every memo review I do has gone through one tool, so I have clean data on what my job actually is. I leave about 140 line comments a week on agent memos, and 47% of them change a decision. That let me set up a natural experiment: can frontier models steer my research better than I do? It turns out the answer is no.

So, am I worried? Not really. At least, not just yet. Read on to see why.

My job is steering

I'm a one-person startup, and I've been building in stealth for about nine months. Claude and Codex do all of the work: the code, the experiments, and the design memos. I read the memos and steer. Everyone just got promoted to exec describes how that works: agents write to Amazon's six-page bar, and I review them the way an exec would.

A comment changes a decision when the agent's next revision changes a recommendation, a plan, a verdict or a spend because of it. One in five of my comments reverses a recommendation outright. Over a year, that's about 7,000 comments and more than 3,000 decisions a model made that I changed.

To test whether a model could make those calls, I gave Claude and Codex my past review comments, then asked them to predict the comments I'd leave on two weeks of memos they had never seen. The result: the best of four setups caught 12%, about one in eight. Nearly every miss needed knowledge that isn't written anywhere a model can read. What's worse, in a sample I checked by hand, about a quarter of the models' other calls were ones I wouldn't have made, and they would have sent the project in the wrong direction.

Why the models miss

I classed every decision-changing comment that no setup caught. 95% of them draw on knowledge that isn't in the memo. They fall into three groups: only partly fixable with instructions, fixable in principle with a bigger picture, and not fixable yet.

• My theory of the problem and the model, 27% of misses. Not fixable yet. "How much of what the model learnt is driven by noise versus examples with real evidence? If the former drowns the latter, then it's no wonder the result is weak."
• The live state of the project, 24%. Fixable with a bigger picture. "Isn't it underway? Can't you ask the agent?"
• Product taste, 21%. Not fixable yet. "This is a rare scenario; your design makes it central."
• First-principles sense of time and cost, 16%. Fixable with a bigger picture. "Why would training just the head need days? I would expect it runs in minutes."
• What a result means for the plan, 8%. Fixable with a bigger picture. "Looks like a null result, but this confound makes it possible that the question wasn't asked correctly."
• Plain language and contradictions inside the memo, 5%. Already in reach. "You say minutes here. Section 5 says a day or more."

The afterword walks through ten examples: what the agent said, what I said, and what it changed.

Only partly fixable with instructions: how to search

Left alone, the agents behave like careful scientists. They change one thing at a time, keep controls, and close a direction when a test fails. They also tend to be conservative: they treat every null as the question failing completely. Sometimes being conservative is the right stance, but when it bleeds into being pessimistic, that becomes self-fulfilling. Being careful and conservative is the right posture once something works. It's the wrong one before. My philosophy is that until you have a positive result, you should combine as many hypotheses as the evidence supports, to raise the odds that the experiment finds something. Once it works, you add arms to remove the confounds and learn what helped. As I put it to one agent: "When you don't have something that's working, carefully planning to change one variable is useless."

The models have the opposite bias, consistently. One in five of my decision changes restates this stance, and in 45% of those, the careful default would have cost a training run or closed a promising direction. It's a stance, and it can be written down. But writing it down as an instruction doesn't fix this, because it leans against the model's priors. When I gave this as an instruction, the agents still fell back on their prior: the setup told to hold off on controls until something works still asked which of three live changes caused a result. Frontier models may emerge that have the same prior as me, perhaps from being in an RL environment where they're rewarded for combining hypotheses to make the most of limited resources. Until that happens, there isn't a clean fix.

Fixable with a bigger picture: what's true today

A plan counted on a cloud quota of 16 GPUs, and the request had come back denied. I wrote: "Got denied. The plan should prioritize the one GPU." Another plan budgeted a run at roughly 40% more than a previous run of a similar kind had used, and I asked why. On a third, an agent proposed doing work another agent was already doing: "Isn't it underway, can't you ask the agent?"

None of this takes judgment. It takes knowing what's in my inbox, what the other agents are running, and what last week's runs cost. Some of it an agent could have checked if it had known to look. The rest lives in my email or my head. Ideally you don't want the agent doing the implementation to also keep these kinds of facts in context. It could be fixed by having other agents whose job is to track them and steer the worker agents. In my experience, this increases token usage non-linearly, because those agents then need to communicate with the worker agents. That said, given more token budget, this is probably the most tractable.

Not fixable yet: hunches and taste

The largest group is my theory of why the research works at all. For example, when an experiment's result came back weak, I asked whether the model was learning mostly from noisy examples, with the noise drowning the few examples that carried strong evidence. If so, a noisy example shouldn't push the weights as hard as a clean one. When a plan leaned on some parts of the data that seemed strong, I pushed back that they may be outliers from the past that no longer represent the distribution. A model trained mostly on noisy examples would potentially have plateaued earlier, and it would have seemed like the model hit its floor, pointing towards model capacity, when it was mostly limited by the quality of its input data.

These are hunches. A hunch is a strong prior from a small number of examples: years of watching the data, a handful of experiments that worked or didn't. When I had Claude grade my own calls across those five weeks, it judged only 5% of the calls like these fully reachable by an agent on its own. In other words, for 95% of these calls, neither Claude nor Codex could have gotten there by themselves. And 73% of calls like these would have been expensive to get wrong. For example, a wrong call may have consumed more held-out data, or used more wall time and cloud compute wandering down unfruitful paths. And once a wrong choice has been made, every agent after it builds on it, searching in a lost wilderness.

Taste is the same shape. On Session Manager, the app I run my agents from, an agent proposed adding a new element to a screen I'm happy with. "Don't ruin the aesthetics of it. It's actually very functional and beautiful now." On another design: "This is a rare scenario, your design makes it central." There's no ground truth for either. There's me, using the thing every day. In theory, taste becomes one of the most tractable problems once a model has all the usage telemetry. At launch, before that data exists, it's still mostly a matter of taste.

The second kind of error

The misses are the first kind of error. The second kind is worse: comments that sound like me and would push the work the wrong way.

About 1 in 20 predictions matched a comment I actually left. Most of the rest were challenges, which is how I review, so they read like me. When I checked a sample, I would have accepted about half as fair points. In other words, these were comments I should have made but missed. I judged most of them to be modest improvements: they would not have dramatically altered the course of my research. About a quarter, on the other hand, were calls I would not have made, and they were actively bad. They applied textbook rules that would produce bad results. Some examples:

• Bringing back the noise. "The pilot keeps the small segments and their results, but the proposed gate removes both. How does a successful training run prove that removing them will improve the result, including the context they provide to larger segments?" It's a sharp, skeptical question, exactly my style. It's also wrong for this project. Earlier work had shown the small segments carry far more noise than signal and make it harder for the model to learn: a lot more data, of poor quality. The model asked to play me asked to bring it back.
• Controls before a win. "If three changes are live at once, how do you know which one caused a measured improvement or regression?" In a mature project, that's a good question. In one still looking for its first positive result, it's the wrong one. It's standard practice to ship a few features together, because most are nulls. You chase confounds when you want to build on a success, not while you're looking for one. This came from the setup that had my view on controls in its instructions.
• Proof before building. "Unused quota does not establish that this pays for itself. What measured benefit would justify the agent time and the engineering to build it?" How would you measure the benefit of features not yet shipped? Applied consistently, this rule stops anything new from being built.

That's the danger. A reviewer trained to sound like me carries my authority with the agent that reads it. When it's wrong, it's wrong in my voice, and it steers the work toward the textbook while looking like steering from me.

Hunches are sample efficiency

Training a model searches parameter space. Every batch gives a gradient, and a run gets millions of cheap, informative signals. My job is to search the space above that: which data, which objective, which experiment to run next. Call it hyperparameter space, in the broad sense. Each step there costs a training run: a day of wall time, hundreds of dollars, and a noisy number at the end. In the five weeks these comments cover, the research settled about 20 hypotheses. That's the whole training set for the judgment that steers it.

Could a model learn to search that space well? In principle, yes. The key is to be incredibly sample efficient: to form a useful belief about what to try next from a couple of dozen noisy outcomes. Humans call that a hunch. I don't see it in models yet.

The usual way around scarce data is to make more of it. AlphaGo did that with self-play. The rules of Go decide who won, instantly and for free, so it could play itself millions of times. Most of the real world has no such referee:

• A research direction. The referee is an expensive run, and a null may only mean you asked the question badly.
• A product design. The referee is people living with it for weeks.
• A founder. The referee takes years, and an investor sees only a few dozen outcomes in a career.

Where examples are few and the rules can't be written down, humans hold their biggest advantage.

Which jobs is this true for?

Not all of them. I'd sort jobs into three kinds.

1. Plenty of data on your decisions. If there's a large record of the kind of decisions you make and how they turned out, the job is already automated, or it's a zombie: it just hasn't been automated yet.
2. Little data, but you can hill-climb. If data is scarce but the task can be scored automatically, some startup or lab is probably working on your job right now. It may not be gone yet, but it's a matter of time. To check, ask yourself: could you build a training environment where decisions like yours are generated synthetically, and good ones can be told from bad ones, even weakly? If yes, you're in this lane. Examples include coding, financial analysis reports and consulting reports. These have the shape where you can start to create judges that tell good ones from bad ones, or at least separate them enough. That gives a gradient to push against.
3. Little data, and it's chaotic. If most of your decisions are judgment calls or hunches, my finding applies to you. To check, ask: given a small set of someone's judgments, could you tell good judgment from bad? By nature, the error bars here are huge. Someone can look like they have great judgment when they were just lucky. But that's the nature of a lot of jobs.

Could I be wrong?

Yes, in several ways.

• There's no control. Someone could argue that my steering was bad all along, and that a much better path through this problem exists. I can't rule that out from one project with one steerer.
• Sample efficiency may improve fast. My guess is that models need two to three orders of magnitude better sample efficiency to do this job. A breakthrough could deliver that, and a lab may already have it without saying so.
• Test-time compute. I run on two or three of the top-tier Claude and Codex plans. With one or two orders of magnitude more compute, I expect quite a few of these problems become tractable. A model could check the live state, rerun the cost math, and argue every memo with itself before it reaches me.
• This model is far from complete. Economic factors beyond the ones here can make a job sticky, or not. A job may be highly repetitive but legally require a human, so it sticks around far longer than this analysis suggests. On the other hand, a high-judgment job can still go if a company has to cut costs, say from the second-order effects of mass disruption from AI. This is one lens, not the whole picture.

One thing I'm fairly confident I'm not wrong about. To learn judgment, you need a way to tell good from not good without using up a small, fixed supply of data. If your work has no cheap referee, and every verdict costs real time, money or held-out data, then more compute and better instructions only take you so far. Only a large jump in sample efficiency will help. Nothing else much will.

Until then, I expect I can keep my job, at least as it's currently shaped 😅. If you're worried about yours, or you're young and about to enter the job market, I'd offer this as one lens on how to prepare for what's coming.

---

Afterword: how I ran the test

• The data. My line comments on agent memos, split by time. Reviewers learned from the older comments and were tested on 43 memos from the two most recent weeks, which they never saw. Those memos drew 83 comments from me that changed a decision.
• The reviewers. Opus 5.5 and Codex, each set up two ways. One was taught the nine kinds of comment I leave, with examples. The other also got my 30 most similar past comments and 50 standing preferences drawn from everything I'd said before. That makes four setups. The Codex review these memos already got served as a reference line.
• The judge. A model, blind to which setup wrote what, matched each prediction against the comments I actually left. A match needed the same concern at the same place in the memo. I spot-checked 20 of its calls and agreed with all 20.
• The bar. Catch at least 40% of my decision-changing comments. I wrote it down before anything ran.

Data

Setup	Caught	95% range
Codex + past comments	10 of 83 (12%)	6–18%
Opus + question types	9 of 83 (11%)	5–17%
Opus + past comments	8 of 83 (10%)	4–17%
Codex + question types	7 of 83 (8%)	3–15%
Today's Codex review	1 of 43 (2%)	0–8%

Teaching more didn't help. Adding my past comments and preferences moved Opus from 9 to 8 and Codex from 7 to 10. That's noise. The preferences taught my style of question. The misses needed something else.

What the reviewers did catch, they caught well. Where a memo stated a number or a rule and the obvious question was answerable from the memo alone, they asked it, often in nearly my words. A memo proposed shutting down idle terminals after three hours with no typing. I asked, "What happens if it's running some background task?" Two setups asked the same thing, unprompted.

Some examples

Each one shows what the agent wrote, what I said, and what changed because of it. The quotes are real, lightly edited so they don't give away the research. Square brackets mark what I replaced.

1 · How I search
Agent: The plan ran [one arm] "and read a 'not learned' on [it] as expected."
Me: "That's wasting an experiment while expecting it to be null. The idea of the experiment is to search the model architecture space, not wander into corners expecting there's nothing there."
What changed: "Agreed: a planned null is a wasted [run]." The arm was redesigned so it could actually find something.

2 · My theory of the problem
Agent: "[The signal] is learnable in principle. What blocks [it] is how two measures reach the model, not the network's capacity."
Me: "How much of what the model learnt is influenced by noise versus [examples] that have more causal evidence? If the former drowns the latter, then it's no wonder the [result] is weak."
What changed: "This is the right question." The agent measured it: 82% of the training examples were luck-level, and training weighted them the same as the clear ones. The next plan added a training arm to test the fix.

3 · The live state of the project
Agent: "1 [GPU] (asked 16). A support case asks for 16; a person is reviewing it."
Me: "Got denied. The plan should prioritize the one GPU. Plan to keep it busy and fed for the next 4 days."
What changed: The plan was rebuilt around one GPU, kept busy until the credits ran out.

4 · The live state of the project
Agent: The [replacement code] is "[another ticket]'s work, not a deletion," so it planned around it.
Me: "Isn't it underway? Can't you ask the agent?"
What changed: It messaged the agent already doing that work, and held its own merge until that agent's run finished. No duplicate work, and no collision.

5 · First-principles sense of time
Agent: "[Run it on] the Mac, unless the timed first [run] says it needs more than a day; then paid A100 hours."
Me: "Why would training just the head need days? I would expect it runs in minutes. What gives?"
What changed: "You're right: retraining the head is minutes." The day-long estimate came from a different, one-time step. The paid GPU came out of the plan.

6 · First-principles sense of cost
Agent: "It costs about $72, with a ceiling of $90."
Me: "Why would it cost 70 bucks? This reduces examples. As costed, [the last comparable run] was about $50."
What changed: "One seed, about $36 (ceiling $45)." The $72 had been two seeds, and the second became optional. The cost halved.

7 · Product taste
Agent: "The app opens that link in the browser until a later app release renders it in-app."
Me: "Don't ruin the aesthetics of it. It's actually very functional and beautiful now. While your idea has merit, you have to think through the design of this carefully. Where would you put it? How would it surface?"
What changed: "You are right." The agent dropped the hand-off to the browser and added no new element. The feature became one more row in a list the app already has, so it shipped without an app release.

8 · What a result means for the plan
Agent: A verdict memo reported a null and stopped there.
Me: "Overall looks like a null result, but not clear on what is next here? Drop this idea? Causal explanations and further experiments are not elaborated."
What changed: The one variant that hadn't been tested went into the next experiment's plan instead of being dropped with the rest.

9 · Sounds like me, but isn't
Reviewer playing me: "The pilot keeps small [segments] and their [results], but the proposed gate removes both. How does a successful training run prove that removing those [segments] will improve [the result], including the context they provide to larger [segments]?"
Me: Small [segments] had already been shown to have far more noise than signal, and they were making it harder for the model to learn. Think of it as a lot more data, of poor quality.
Why it's harmful: An agent reading this in my voice would bring the noise back.

10 · Sounds like me, but isn't
Reviewer playing me: "If three changes are live at once, how do you know which one caused a measured improvement or regression?"
Me: "Wouldn't have said this. It's standard practice to ship few features together. Most are nulls. You do confounders only when you want to build on success, not when looking for hits."
Why it's harmful: It pushes a project still looking for its first win back to changing one thing at a time.