Prompt Engineering: diagnose the failure before rewriting

TL;DR

What I learned	What to do
The model can know the answer and still discard it	Diagnose whether it’s a confidence failure, not a reasoning failure
More prompt iterations don’t always help	Identify the failure type first
There are 4 distinct failure types with distinct fixes	See the taxonomy table below
The 5 base elements are necessary but not sufficient	Technique + diagnosis = result

When is the problem not the prompt?

After 17 versions of a prompt for a probability problem, I found something that changed how I work with LLMs:

The model found the correct answer in its reasoning… and then discarded it for “not being standard.”

It wasn’t a prompt problem. The model knew the answer. But it wouldn’t commit to it. It was correcting its own correct reasoning to conform with what it considered the expected answer.

That’s not fixed with more context, more examples, or chain of thought. It’s fixed by explicitly giving the model permission to disagree with “the standard.” The right intervention depends on the type of failure — and that’s the key most prompt engineering guides ignore.

The full process is documented in a series:

The model knows how to reason. It just won’t commit — The initial discovery
It got to 0 and called it a contradiction — Why separating contexts isn’t enough
More tokens isn’t a better result — The limits of brute force
The prompt that solves ambiguous problems — The solution: prompt v17b
Taxonomy of LLM failures — When to use each technique

What are the 4 types of LLM failure?

Before touching the prompt, identify which of these four failures is happening. Each has a different fix — mixing them up wastes time.

Failure type	Observable symptom	What to do
Interpretive ambiguity	The model picks one reading of the problem and defends it even if wrong	Give explicit permission to explore alternative readings; include “if this problem allows multiple interpretations, explore them”
Pure calculation	Gets arithmetic wrong, date counting, multi-step operations	Don’t iterate the prompt: use code tools (calculator, code interpreter). Text alone won’t fix arithmetic — extended thinking helps reasoning, not exact computation
Conceptual error	Confuses technical terms or applies a concept to the wrong domain	Give specific hints about the correct concept, or switch to a more capable model
External knowledge	Invents data or claims ignorance about things that need post-training information	Provide web search or the document with the data; no prompt fixes this

Why this matters before touching the prompt: In the experiment, the first 10 iterations assumed the failure was ambiguity or insufficient instruction. It wasn’t. It was a confidence failure: the model reasoned correctly but self-corrected its answer. Until that was diagnosed, no prompt adjustment worked.

What elements make a prompt work?

The fundamentals haven’t changed — but they make more sense now that you know when they’re not enough.

Context

Tell the model who you are and what situation you’re in. It doesn’t need to be long — it needs to be relevant.

Bad:

How do I optimize a SQL query?

Better:

I'm a junior data engineer working with PostgreSQL.
I have a query that takes 30 seconds on a 10M row table.
How can I optimize it?

Clear instruction

The instruction is the core. It needs an action verb, an object, and constraints.

[Verb] + [what] + [how] + [constraints]

Analyze this Python code
identifying potential performance issues
and suggest optimizations
without changing the business logic.

Output format

If you don’t specify it, the model will choose. Sometimes it’ll guess right.

Respond in JSON format with this structure:
{
  "summary": "...",
  "key_points": ["...", "..."],
  "next_step": "..."
}

Examples (Few-shot)

Showing examples is more effective than explaining rules. 2-3 examples are usually enough.

Classify these tweets by sentiment.

Examples:
- "I love this product!" → positive
- "Terrible service, never again" → negative
- "Package arrived today" → neutral

Now classify:
- "It's not bad, but I expected more"
- "Amazing experience!"

Constraints

Telling the model what NOT to do is as important as what it should do.

- Don't make up data. If you don't know something, say "I don't have that information."
- Don't use technical jargon; the audience is non-technical.
- Stick to facts from the attached document only.

What does Chain of Thought actually fix?

Asking the model to reason step by step improves precision on complex problems — but it doesn’t fix pure calculation failures or external knowledge gaps. To use it well, it helps to understand how the model ‘thinks’ (System 1 vs System 2) and where each mode breaks down.

Without CoT:

How many days between March 15, 2024 and June 22, 2024?

With CoT:

How many days between March 15, 2024 and June 22, 2024?
Think step by step, counting days in each month.

The CoT version is slower but more accurate for logic and chained reasoning. For complex arithmetic, use tools — CoT isn’t enough.

Does roleplay make the model more capable?

It adjusts tone and focus, not underlying capabilities. A model that doesn’t know advanced math won’t learn it by pretending to be a mathematician. There’s also a second risk: the model tends to agree with you, especially when you assign it a role that’s “supposed to” validate your idea.

Use roleplay when you need a specific perspective — not to compensate for real model limitations.

How do you iterate without wasting rounds?

Basic flow:

Initial prompt → partially useful response
Identify what failure type it is (use the taxonomy table)
Apply the right intervention for that type
Adjust format if needed

What doesn’t work: keep adding instructions and examples without diagnosing first. Iterations 4 through 12 of the experiment fell into exactly that trap.

Common mistakes to avoid:

Mistake	Why it fails	What to do instead
Too vague	The model fills gaps its own way	Specify object, format, and constraints
15 instructions in one prompt	The model prioritizes some and ignores others	Split into sequential steps
No format specified	The model chooses, sometimes wrong	Ask for exactly what you need
Expecting it to “guess” the ambiguity	The model resolves ambiguity silently	Remove ambiguity before sending
Iterating without diagnosing	More iterations of the wrong type = more wasted time	Diagnose the failure type first

Templates for common cases

For text analysis:

Analyze the following text and extract:
1. Main topic
2. Tone (formal/informal/technical)
3. 3 key points
4. Possible biases or limitations

Text:
[paste text]

For code review:

Review this code looking for:
1. Bugs or logic errors
2. Performance issues
3. Best practice violations
4. Improvement suggestions

Prioritize by impact. Don't comment on minor style issues.

Code:
[paste code]

For summaries:

Summarize this document as:
- 1 context sentence
- 3-5 bullet points with key takeaways
- 1 conclusion sentence

Maximum 200 words total.

Document:
[paste document]

FAQ

Are there “magic words” that improve any prompt? No. “Think step by step” helps with chained reasoning; “respond in JSON” helps when you need structure. But there’s no universal formula — the task type determines what works.

How many examples should I include in few-shot? 2-3 are usually enough. More examples don’t always improve results and add token cost. If the model keeps failing with 3 examples, the problem probably isn’t the number of examples.

Does roleplay make the model more capable? No. It adjusts tone and response style, not underlying capabilities. Don’t assign a role to compensate for real limitations.

When should I stop iterating the prompt? When you identify the failure is type 2 (calculation), type 3 (conceptual with a model that can’t handle it), or type 4 (external knowledge). In those cases, more prompt won’t solve the problem.

Does the system prompt help for long projects? Yes. It’s the most efficient way to maintain persistent context without repeating it every message. If you use Claude, Projects handle this natively.

What does it mean that the model “knows the answer but won’t give it”? That in its intermediate reasoning it reaches the correct conclusion, but in the final answer it discards it to conform with what it considers the “expected” or “standard” answer. The post on LLM sycophancy explains the mechanism in more detail.

Does Chain of Thought solve calculation problems? Not reliably. For complex arithmetic with many steps, CoT reduces errors but doesn’t eliminate them. The real solution is to use code tools or calculators — text isn’t the right medium for precise calculation.

How do I tell if a failure is ambiguity or external knowledge? If the model gives a coherent but wrong answer using internal data that exists, it’s ambiguity or a conceptual error. If it invents specific data (dates, names, figures) that should be recent, it’s external knowledge.

Want practical examples to copy and use? Check out my 50 tested prompts for ChatGPT — they work on any LLM. And if you’re coding with AI, Cursor takes prompting to the next level by integrating your full project context.