The clearest example I have is a classifier that filters non-buyers out of paid search leads. It was not shipped because the output looked good. It was shipped because it ran against six months of past leads where the outcome was already known, disqualified about one in five, and blocked zero of the buyers who had gone on to real deals.
That is the pattern I apply to anything a model touches:
- Build a labelled set from history before writing the logic.
- Care more about the false positives than the accuracy figure. Blocking a real buyer costs more than letting a bad lead through.
- Prefer plain rules where a wrong answer is expensive. The lead filter is three readable conditions, not a prompt.
- Log every decision, so the thing can be re-scored later against what actually happened.
- Show the health of the job somewhere a person looks weekly.
A prompt that works in a playground is a demo. The eval, the fallback, and the log are what make it a system.