We speak to senior leaders at brands every single day who tell us they’re wary of using AI because of a previous lesson their team, or a team they know, learned the hard way. Important to say it’s not a mistake, it’s a lesson, because something real did get learned, just at a worse moment than anyone would have chosen.
The pattern behind almost all of these lessons is the same: the AI did exactly what it was asked to do, with exactly being the key point. Because nobody had put a safeguard or a guardrail in the prompt, or a human in the loop in the right place to catch errors before they became a problem.
The risk moved, capability didn’t
Most teams already can use AI well. The tools are good. The people using them are, in the vast majority of cases we see, genuinely capable and well-intentioned. That was never the risk.
The risk is that capability arrived faster than governance did. Teams can now do in an afternoon what used to take a week, which means a mistake that used to be caught by the sheer slowness of the process now ships before anyone’s had a chance to notice it.
To be clear, that’s not an argument for wrapping AI in red tape. Used properly, it’s a genuine advantage, and we’d never want a brand to talk itself out of that. The whole point is the “properly”, not less AI, better guardrails around it.
It looks polished. That’s the danger.
Here’s the shape it usually takes. Someone generates documentation, a report, or an analysis for an external audience using AI, and what comes back looks genuinely excellent. It reads well, it’s structured properly, and it does what was asked.
The problem shows up when the inputs weren’t quite right, or the prompt wasn’t specific enough. AI models hallucinate. Everyone knows this in the abstract and still gets caught out by it in practice, because the output looked too good to need checking.
That’s the actual failure mode, and it’s a more dangerous one than an obviously wrong answer: people with real experience, who’d normally spot an error in seconds, wave it through because it’s polished and everyone’s under time pressure. Time pressure is real. It’s still not a reason to skip the check, that’s exactly the moment mistakes get made.
The context gap nobody’s talking about
There’s a second version of this that’s easy to miss. People newer to the industry, who joined after these tools became standard, often never learned to do the underlying task by hand. They learned to prompt, not to do - which means they never built the instinct for what “wrong” actually looks like, because they never had to spot it themselves the slow way.
Checking AI output well requires the same judgement you’d use to check a colleague’s work, and that judgement has to come from somewhere. If it isn’t coming from experience, it needs to come from somewhere else - training, pairing, deliberate exposure to the manual version of the task - or it won’t be there when it’s needed.
Guardrails built from postmortems, not policy
The brands handling this well aren’t the ones with the longest AI policy document. They’re the ones who’ve built their guardrails bottom-up, from the lessons that already happened, rather than top-down from a policy nobody’s read.
What’s allowed to touch live product data without a second pair of eyes. What output has to be checked against source numbers before it leaves the building. Who’s accountable when something goes wrong, and what happens the next time it nearly does. None of this is exciting. All of it is the difference between an afternoon’s inconvenience and a genuinely bad day.
Training, not just capability
The teams furthest ahead on this aren’t asking “can our people use AI?” They’re asking “have we actually shown them what good use looks like, and what it doesn’t?” The hardest part for most teams isn’t the tool. It’s building the judgement to know where the edge of safe use actually is, and that has to be taught, deliberately, rather than assumed.
The tools got good enough to cause real damage a while ago. The question worth asking this week isn’t whether your team can use AI. It’s whether anyone’s actually shown them where the edge is.





