AI Prompting
Prompt Engineering — reliable prompts that do what you asked
Techniques for writing prompts that produce correct, well-formatted, repeatable output from LLMs like Claude (Opus 4.x / Sonnet): clear role and instructions, few-shot examples, structured output, chain-of-thought decomposition, delimiters and grounding, constraints, and closing the loop with evals.
Quick reference — reach for these first
System: You are a SUPPORT_TRIAGE_BOT. Task: <ONE_SENTENCE_GOAL>. Audience: <WHO>. Length: <N> words.State role, task, audience and length. Vague prompts are the #1 cause of off-target output; specificity beats cleverness.<examples>\n<example><input>...</input><output>...</output></example>\n... 2-5 total ...\n</examples>2-5 input/output pairs teach format and edge cases faster than any description. Match example style exactly to what you want back.Return ONLY valid JSON matching this schema: {"field":"<type>"}. No prose, no markdown fence.Name the exact format and keys; prefill the assistant turn with `{` to force JSON. Then parse/validate programmatically.Think step by step inside <thinking>...</thinking>, then give the final answer inside <answer>...</answer>.Decomposition improves reasoning accuracy. Separate scratch work from the final answer so you can discard the thinking.Use ONLY the facts in <context>...</context>. If the answer is not there, reply "unknown".XML tags separate instructions from data (and block injection); grounding plus an explicit escape hatch cuts hallucination.The prompt-engineering loop: assemble, run, measure, iterate
Text equivalent: A left-to-right pipeline of six stages. Four input stages feed a model, which feeds a validation stage. Stage 1: Role plus clear instruction. Stage 2: Few-shot examples. Stage 3: Delimited, grounded context. Stage 4: Output schema as JSON. Stage 5 (accent): Claude Opus 4.x or Sonnet at low temperature. Stage 6 (accent): Validate and evaluate against a labeled set. A feedback arrow labeled 'iterate on one change' loops from the validate stage back to the role and instruction stage.
Common tasks
Write the prompt · Ground & constrain · Measure with evals
| Write the prompt | Ground & constrain | Measure with evals |
|---|---|---|
| Write a one-sentence task statement before anything else | Goal: Extract the fields <F1>, <F2>, <F3> from a customer support email. | A teammate can restate the goal from the first line alone. |
| Assign an explicit role in the system prompt | You are a precise information-extraction engine for SUPPORT_TICKETS. | Role names the domain and the behavior, not just "an assistant". |
| Put instructions BEFORE the data, separated by delimiters | <instructions>...</instructions>\n<email>{{RAW_EMAIL}}</email> | Data lives inside its own tag; no instruction text is mixed into the data block. |
| Add 2-5 few-shot examples that match the target format | <examples><example><email>...</email><json>{...}</json></example></examples> | Every example output is byte-for-byte the shape you want returned. |
| Specify the exact output format and keys | Return ONLY JSON: {"category":"...","urgency":"low|med|high","needs_human":true} | Enumerate allowed values; no free-text where an enum will do. |
| Prefill the assistant turn to lock the format | messages=[{role:user,...},{role:assistant,content:"{"}] | First returned character is `{`; model cannot open with prose. |
| Ground the model and add an escape hatch | Use ONLY <email> content. If a field is absent, set it to "unknown". | On an email missing a field, output is "unknown", not an invented value. |
| Constrain length and verbosity | No explanation. Output the JSON object and nothing else. | Response contains only the object; no preamble like "Here is". |
| Lower temperature for deterministic extraction | temperature=0 | Same input yields the same JSON across 5 runs. |
| Build a tiny labeled eval set (10-30 cases) | evalset.jsonl: {"email":"...","expected":{...}} | Set includes edge cases: missing fields, multiple issues, non-English. |
| Score schema-validity + field-exact match | python eval.py --prompt v2 --set evalset.jsonl | Report prints valid-JSON % and per-field exact-match % per prompt version. |
| Iterate one change at a time and compare scores | diff v1 v2; keep the version with higher exact-match | Each prompt change is tied to a measured score delta, not a vibe. |
Extract structured fields from a support email, then evaluate the output
support-triage
System: You are a precise extraction engine for SUPPORT_TICKETS.\n<instructions>\nFrom <email>, return ONLY JSON matching the schema. Use ONLY facts in <email>; if a field is absent use "unknown". No prose, no markdown.\n</instructions>\n<schema>{"category":"billing|technical|account|other","urgency":"low|med|high","account_email":"string|unknown","summary":"string (<=15 words)","needs_human":true|false}</schema>\n<examples>\n<example><email>Hi, I was charged twice for May. Please refund. - a@co.com</email>\n<json>{"category":"billing","urgency":"high","account_email":"a@co.com","summary":"Double-charged for May, wants refund","needs_human":true}</json></example>\n</examples>\n<email>{{RAW_EMAIL}}</email>Subject: App won't load\nHey - since this morning the dashboard shows a blank screen on Chrome. Tried logging out. Not urgent but annoying. Thanks, Priya
python eval.py --prompt v1 --set evalset.jsonl # 20 labeled emails
Troubleshooting — symptom → first check → next check
| Symptom | First check | Establishes | Next check |
|---|---|---|---|
| Output format drifts (extra prose, markdown fences, wrong keys) | Did you (a) show an example of the exact format and (b) validate/parse the result programmatically? | Whether the format was demonstrated and enforced, or only described in words. | Add a matching few-shot example, prefill the assistant turn with `{`, say "ONLY JSON, no fences", and reject+retry on parse failure. |
| Model ignores a specific instruction | Where does the instruction sit — buried mid-paragraph, or at the top in its own line/tag? | Whether the instruction was salient and unambiguous. | Move it up, make it imperative and explicit, isolate it in its own delimited block, and restate the critical rule once near the end. |
| Hallucinated facts or invented field values | Is the model told to use ONLY the provided context, with a defined fallback for missing info? | Whether the answer was grounded or free-floating. | Wrap source data in <context>, instruct "use ONLY this; if absent say 'unknown'", and ask it to quote the supporting span. |
| Inconsistent output across identical runs | What is the temperature, and are there examples pinning the format? | Whether nondeterminism is from sampling or from an underspecified format. | Set temperature=0 (or low), add 2-3 examples, and tighten the schema with enums instead of free text. |
| Responses are too verbose for a field-extraction task | Did you ask for just the field/object and forbid explanation? | Whether brevity was actually requested. | Say "output the value only, no explanation", cap length, and prefill the turn so it cannot open with filler. |
| Reasoning is wrong on multi-step questions | Was the model given room to work the problem before answering? | Whether failure is knowledge or skipped reasoning. | Add "think step by step in <thinking>", then a separate <answer>; for hard tasks decompose into chained prompts. |
| A labeled case regresses after a prompt edit | Does the eval harness re-run the full set and diff per-field scores? | Whether the change is a net improvement or a local fix that broke others. | Re-run evals on every edit, keep the higher-scoring version, and add the regressed case to the permanent set. |
References
- Anthropic — Prompt engineering overviewdocs.anthropic.com
- Anthropic — Use examples (multishot prompting)docs.anthropic.com
- Anthropic — Let Claude think (chain of thought)docs.anthropic.com
- Anthropic — Use XML tags to structure promptsdocs.anthropic.com
- OpenAI — Prompt engineering guideplatform.openai.com