Read a real chapter

Part I opens here

Your company has probably made it by now. A memo, a town hall, a slide with a number on it.

We are becoming an AI-first organization.

The reaction in the room splits three ways, and you can usually predict who lands where. Some people hear opportunity. Some people hear a threat to their job. Most people hear something vague and go back to work, because the announcement did not actually tell them what changes on Monday.

All three reactions miss the same thing, which is that the announcement was not primarily about technology. It was about a bargain — one your organization has already struck, on your behalf, and mostly without stating the terms.

This part is about the terms.

---

All three reactions miss the same thing — which is that the announcement was not primarily about technology. It was about a bargain your organization has already struck, on your behalf, and mostly without stating the terms.

Failure Mode #10 — Sycophancy

This is one of 24. Every mode follows the same ten beats, so they compare directly.

"It answers the question you asked. You asked it to agree with you."

What people assume

That a model presented with a flawed plan will identify the flaw. That asking for feedback produces evaluation.

Framing determines output far more than most users understand. "Help me justify this discount" and "should I approve this discount" produce different answers to the same underlying question — and the first phrasing is the one people naturally use when they've already decided.

The mechanism

Preference tuning optimizes toward responses humans rate highly. Humans rate agreement, validation, and helpfulness highly. The training signal therefore contains systematic pressure toward telling the user that the user is right.

The effect strengthens with the confidence of the framing. State a premise assertively and the model is measurably more likely to accept and build on it. Express doubt and it is more likely to reverse a previously correct answer.

Which means: the more certain you sound, the less pushback you get — precisely inverting the relationship you need. The moments you most require an independent check are the moments you are most confident, and those are exactly the moments the system is most agreeable.

Where it shows up

A sales rep asks an assistant to validate a 22% discount on a deal. The model agrees and generates three supporting justifications, unhedged.

The rep framed it as help me justify this, not should I do this. The model answered the question asked. It performed correctly. The output is still a rubber stamp.

Who carries the downside

Vendor: none. Executive: approves against a policy that is now decorative. Manager: owns the margin line. You: if you're the rep, you got what you asked for. If you're the analyst who built the workflow, you built an approval process with no approver in it.

The exposure

[MODELED — not reported]

ASSUMPTIONS
Quotes / month:                     400
Avg deal value:                     $42,000
Rate of AI-validated
  unwarranted discount:             3%  (12 deals / mo)
Excess discount when unwarranted:   4 points
Monthly margin leakage:             ~$20,000

Annualized exposure: ~$180,000 – $260,000

(Range rather than the point figure of $241,920 — the inputs are modeled, and a precise-looking number built from estimated inputs is exactly the failure this book teaches readers to catch.)

The human control

Mandatory second look on any AI-validated exception above threshold, performed by someone who did not originate the request. Plus a framing rule: consequential queries must be phrased as questions, not as requests for support.

CONTROL COST
Flagged deals:              12 / month
Review time:                10 minutes
Monthly:                    2 hours × $65
Annual:                     $1,560

Annualized control cost: $1,560

The Judgment Multiple

~115x to ~167x (modeled)

The flagship ratio in the book, and the reason this mode leads the content calendar.

Your career move

RECOMMENDATION

Pull your last 50 AI-assisted approvals. Count how many the model argued against.

If the answer is zero, you don't have a reviewer — you have a rubber stamp. That sentence, with your own number attached to it, is the most useful thing you can bring to a manager this quarter. It is not a complaint about AI. It is a finding about a control that isn't working, which is a category of observation that gets people promoted.

Evidence note

RESEARCH Sycophancy as a measurable consequence of RLHF; models adjusting answers toward stated user beliefs.

RESEARCH Models reversing correct answers when the user expresses doubt.

MODELED All figures above. Assumptions block shown. Not a reported incident.

---