Why Your Longer AI Prompt Gets a Worse Answer

Last week was about knowing what good output looks like. This week is about the input, and the most common way Chiefs of Staff get it wrong. When AI hands you something weak, the instinct is to add more instructions, more context. Most of the time that makes it worse. Getting good at AI includes knowing what to leave out. A tight, well-chosen ask beats a bloated one. Knowing what to cut is the skill.

I was refining onboarding documentation for a client, one of my fractional Chief of Staff engagements, and I tried to get there in a single prompt. It kept growing. Each new section, exception, or "and make sure it also covers this" turned into another instruction, until the prompt was longer than the document I wanted back.

Then I ran it. The output came back muddier than a plain, simple ask. It tried to honor every rule at once, hedged, contradicted itself in places, and buried the structure I needed under everything I'd told it to weigh. I was more confused than when I started.

So I cut the prompt back to the goal, the format, and the one constraint that actually mattered, then built it in 2 passes: skeleton first, then the sections. It came back clean, without the hedging. Two purposeful rounds got me there faster than one giant prompt ever did.

This week's prompt is a context editor. Paste in the bloated prompt you've been wrestling with, and it hands back a tighter version, tells you what it cut and why, and flags what to save for a second pass. The 30-second version if you never paste it: when the output is muddy, cut your ask in half and see if the answer sharpens before you add anything.

The full context editor is for CoS Signal members, along with the prompt library and everything else members get. Founding rate is locked at $14.99/month or $99/year.

The research says the same thing. Chroma Research tested 18 frontier models, including GPT-4.1, Claude 4, and Gemini 2.5, on how input length affects performance. The finding: models don't use context uniformly, and performance becomes less reliable as input gets longer. It showed up even on a task as trivial as repeating a list of words. For a Chief of Staff piling context into one ask, that's the trap: the longer the prompt runs, the harder the model works to find what matters. (Chroma Research, "Context Rot," July 2025)

Head’s up: This one’s an ad!

Pioneer, the summit where CX leaders redefine what’s possible, is on October 7th.

Join leaders from Fin, Anthropic, Clay, and Kalshi for an insightful conversation on the state of AI transformation.

You’ll discover how some of the most innovative minds in CX have transformed their organizations, learn how they think about CX, and hear how they're planning for what's next.

Join the conversation in San Francisco, or tune in virtually.

Reserve your spot

Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma Research) is the study behind this week's signal, and the clearest evidence that more input can work against you. Technical in places. The charts alone make the point. Free.

This is part 2 of 4 on getting good at AI. Last week was knowing what good looks like; this week is trimming what you feed the model. If you do one thing this week, take the last prompt that gave you a muddy answer, cut it in half, and run it again. See if the shorter ask comes back sharper. The best prompt this week is the one you made smaller.