The single biggest gap between engineers who want the role and engineers who get it is this: the ability to define, for a fuzzy task, what "good" looks like and how to measure it. Everything downstream depends on it.
A client rarely arrives with a metric. They arrive with a wish — "summarize these contracts," "triage these tickets," "answer these citizen queries." Your first job is to turn the wish into an eval set: concrete inputs, expected behavior, and a scoring method that a non-engineer can trust.
The eval set is also your contract. It's how you defend the system when it's questioned, how you catch regressions when the model changes under you, and how you prove the deployment worked. Without it, you're shipping vibes into a regulated environment.
This is why eval-scoping shows up in FDE interviews and why it separates levels. Building with an LLM is now commodity. Knowing what to hold it to — that's the deployed engineer's craft.