Prompt-Engineering Checklist for Reliable LLM Products

Every generative feature we ship goes through the same checklist before it reaches users. None of it is exotic — that's the point. LLM features fail in boring, predictable ways, and a boring, predictable checklist catches most of them. This is the one we use internally, in the order we apply it.

1. Write the contract before the prompt

Decide what the model owes you: the exact output shape (JSON schema, markdown structure, maximum length), the tone, and what it should do when it can't comply. If you can't validate the output mechanically, you can't tell whether a prompt change made things better or worse.

2. Separate instructions from data

User input belongs in a clearly delimited block, never interpolated into your instructions. This is half of your prompt-injection defense and all of your debuggability: when something goes wrong, you can see exactly what the model was told versus what it was given.

3. Show, don't describe

Two or three worked examples outperform a paragraph of adjectives. If you want terse, show terse. Few-shot examples are also where brand voice actually lives — a style guide in prose is a suggestion; a style guide in examples is a constraint.

4. Build the eval set before you tune

Collect twenty to fifty real inputs — including the ugly ones: empty strings, wrong languages, hostile users, maximum-length pastes. Run every prompt revision against all of them. Prompt engineering without an eval set is vibes engineering with extra steps.

5. Validate output like it came from the internet

Because it did, statistically. Parse the JSON, check enum values, clamp lengths, strip markup where it doesn't belong. Every output your code consumes must survive a malformed response without taking the feature down; retry-with-feedback handles the rest.

6. Decide the refusal path on purpose

Models will refuse, hedge, or hallucinate confidence. Choose what your product does in each case — fall back to a simpler answer, escalate to a human, or say "I don't know" honestly. The unplanned refusal path is where users lose trust fastest.

7. Version prompts like code

Prompts live in the repo, change through pull requests, and reference the model version they were tuned for. When the provider ships a new model, the eval set from step 4 tells you in minutes whether your prompts survived the migration.

8. Log enough to answer "what happened?"

Store the prompt version, model, input hash and output for every call (with PII handled deliberately). The first production incident will be a user report you can't reproduce; logs are the difference between a fix and a shrug.

The meta-rule

Treat the model as a brilliant, unreliable contractor: give it a written spec (1–3), check its work (4–5), plan for the days it doesn't show up (6), and keep records (7–8). Teams that do this ship LLM features that feel dependable. Teams that don't, ship demos.