Let’s get the uncomfortable part out of the way: AI coding tools will confidently generate code that is completely, demonstrably wrong. Not occasionally. Regularly. And the confidence is what makes it dangerous.
A junior developer who’s unsure will ask for help. An AI that’s wrong will present its hallucination with the same formatting, the same tone, and the same apparent certainty as its correct output. Learning to tell the difference is the single most important skill in AI-assisted development.
The taxonomy of AI failure
Not all AI mistakes are created equal. Understanding the failure modes helps you spot them faster.
The plausible API that doesn’t exist
This is the classic hallucination. AI generates code using a function, method, or library that sounds like it should exist but doesn’t.
# AI-generated code that looks perfectly reasonable
from sklearn.ensemble import GradientBoostingTransformer
model = GradientBoostingTransformer(n_estimators=100)
model.fit_transform(X_train, y_train)
There’s no GradientBoostingTransformer in scikit-learn. But it sounds right – it follows the naming conventions, uses the expected parameters, and fits the sklearn pattern. AI models are fundamentally pattern-matching systems, and sometimes they match patterns that don’t correspond to reality.
How to catch it: If you’re using an API you don’t recognize, check the docs. This takes ten seconds. The frequency of this failure mode has decreased significantly as models have improved, but it still happens, especially with less popular libraries and newer APIs.
The subtly wrong logic
This one is more insidious. The code runs. The tests pass (if AI wrote those too). But the logic has a flaw that only manifests under specific conditions.
// AI-generated: calculate business days between two dates
function businessDays(start, end) {
let count = 0;
let current = new Date(start);
while (current <= end) {
const day = current.getDay();
if (day !== 0 && day !== 6) count++;
current.setDate(current.getDate() + 1);
}
return count;
}
This works for most inputs. But it doesn’t handle holidays. It mutates the start date if you pass a Date object directly. It doesn’t handle timezone edge cases. And if start is after end, it returns 0 instead of throwing an error or returning a negative number.
None of these are “bugs” in the AI’s generation. They’re missing requirements that the AI didn’t know about. But a developer reviewing this code might glance at it, think “looks right,” and move on.
How to catch it: Think about edge cases before reviewing AI output. What happens with empty input? Boundary conditions? Negative values? Invalid types? If you think about edges before reading the code, you’ll spot gaps the AI didn’t address.
The outdated pattern
AI models are trained on data with a cutoff date, and codebases on the internet contain years of deprecated patterns, old API versions, and superseded best practices.
// AI might generate class components for React
class UserProfile extends React.Component {
constructor(props) {
super(props);
this.state = { user: null };
}
componentDidMount() {
fetchUser(this.props.id).then(user =>
this.setState({ user })
);
}
// ...
}
Technically correct. Functionally fine. But no modern React project should use class components for new code. If your team uses hooks and functional components exclusively, this AI output creates inconsistency that will confuse future developers.
How to catch it: Your project context files (.cursorrules, CLAUDE.md) should specify current patterns and versions. But also trust your gut – if AI suggests something that feels old-fashioned, it probably is.
The confident wrong answer about your system
This is the most dangerous category. AI doesn’t know your production configuration, your deployment pipeline, or the undocumented behavior of your internal services. When it generates code that interacts with these systems, it’s guessing.
“Your Redis cluster handles this automatically” – does it? “This will work with your AWS setup” – will it? AI has no way to verify claims about your specific infrastructure.
How to catch it: Treat any AI claim about your system (as opposed to general programming knowledge) as an educated guess. Verify it against your actual configuration.
Building your verification instinct
Over time, experienced AI-assisted developers build an instinct for when to trust and when to verify. Here’s how to accelerate that learning.
The complexity heuristic
Simple transformations are almost always correct. Formatting a date string, converting between units, mapping a list – AI nails these. As complexity increases, reliability decreases. The relationship is roughly exponential, not linear.
Rule of thumb: If the logic would take you less than a minute to verify mentally, trust it. If it would take longer, verify it.
The novelty heuristic
AI is excellent at common patterns and terrible at novel combinations. A standard CRUD endpoint? Trust it. A custom caching strategy that combines Redis TTL with database-level invalidation triggered by Kafka events? Verify every line.
The security heuristic
Never trust AI with security-critical code without thorough review. Authentication flows, authorization checks, input sanitization, encryption implementations, secret management. These areas have subtle failure modes where “almost right” equals “completely compromised.”
This isn’t a comment on AI capability – it’s basic risk management. The cost of a security bug is orders of magnitude higher than the cost of a logic bug. Adjust your verification accordingly.
What failures teach you
Here’s the counterintuitive part: every AI mistake you catch makes you a better developer. When AI generates a plausible-but-wrong solution, you practice the exact skill that makes senior developers valuable – evaluating code critically and spotting hidden assumptions. When AI uses a deprecated pattern and you catch it, you reinforce your understanding of why the ecosystem moved on. These reps build judgment faster than writing code from scratch.
A practical verification workflow
For any non-trivial AI-generated code:
- Read the approach before the code. Does the overall strategy make sense?
- Check imports and dependencies. Do these packages actually exist?
- Trace the happy path mentally. Walk through with a concrete example.
- Identify the edges. What inputs would break this?
- Run it. Nothing replaces actual execution.
AI will get better. Hallucinations will become rarer. But they’ll never reach zero, and the cost of undetected errors will always be high. The developers who thrive with AI tools aren’t the ones who trust unconditionally or reject reflexively – they’re the ones who’ve developed calibrated judgment about when to lean in and when to look closer.