What Good AI Output Looks Like for Chiefs of Staff

This starts a new 4-part arc on getting good at AI. It opens here because everything else in the arc rests on one skill: knowing what good output looks like. That sounds obvious until you meet the failure mode. Bad AI output rarely looks bad. It's fluent, confident, well-organized, and wrong in a way that reads exactly like right. The reason a Chief of Staff is the last line before something ships is that you can tell the difference. This week is about how you actually do that, and it starts with a mistake I made myself.

A few weeks back, writing the issue on shadow AI, I had AI pull supporting research. It came back citing a Google product called Workspace Intelligence. I'd never heard of it, and the name had that too-clean, assembled-by-a-model quality. So I flagged it and cut it.

It's real. Google shipped it April 22, and it was the sharpest example in that whole issue of the thing I was writing about.

I didn't check. I ran the claim through a feeling, decided it sounded made up, and deleted it. Looking would have taken 30 seconds. And the feeling cuts both ways: the plausible claim that's wrong sails through for the same reason the real one got cut.

The full output-check prompt is for CoS Signal members, along with the prompt library and everything else members get. Founding rate is locked at $14.99/month or $99/year.

In the Work AI Index 2026, a survey of 6,000 full-time digital workers across the US, UK, and Australia, 41% say they sometimes hand over AI-generated work they couldn't explain if someone asked. The gap widens with use. Heavy users are 64% more likely to ship work they haven't verified or fully understand. The more AI you lean on, the more that check has to run every time, whether the output feels off or not. (Work AI Index 2026, Glean Work AI Institute)

Head’s up, this one’s an ad.

"When I think of revenue, I think of Attio." - Shreman Shrestha, Head of Business at Granola

Here's what that adds up to:

Start Your Trial

15 Times to Use AI, and 5 Not To (Ethan Mollick, One Useful Thing) is from 2024, and the points hold up. His second "not to" is this week's: AI errors are plausible enough that people fall asleep at the wheel and stop checking. Free.

This is part 1 of 4 on getting good at AI, and it starts with output quality because the rest of the arc builds on it. If you do one thing this week, take the last AI output you accepted without really looking, find the one claim you'd have to defend if someone pushed back, and go confirm it. That's the whole practice in miniature. We build from there.