The model is the last decision in the architecture

OpenAI’s Decisions API made me want to write about a principle I’ve been building on for a while: The model is the last decision in the architecture, not the first. The API itself: send text, images or JSON, get back a probability, a choice, or a score. No generated text. OpenAI says up to 10x faster than the same model through the Responses API, billed on input only. Same model, GPT-6 Luna, behind a different endpoint. The useful part is the recognition: not every question that needs intelligence needs generation. Classification, routing, gating and scoring are bounded questions, yet we often handle them through general-purpose generation APIs. So the first decision is never “which model.” It’s “which responsibilities does this system have.” Reasoning, generation, classification, scoring, routing are different jobs with different requirements. I separate them from day one, and I keep each one model-agnostic. The second decision isn’t “which model” either. It’s “how do we measure each responsibility.” Accuracy, latency, cost at scale, and what a wrong answer costs. Hard constraints stay deterministic. Within them, routing can pick the model per request based on the task’s requirements. Only then comes the model, and it’s chosen per responsibility, by the eval, not by the vendor’s launch post. Production evals decide when it changes. That also means not swapping in a specialized model for every bounded task just because one exists now. That’s the same mistake from the other direction. The question I’d put to the team: a decision model is 10x cheaper and 2x faster, and 2% less accurate. Do you switch? Not without the eval. Which requests are the 2%? What does a wrong decision cost? How fast does that compound at your scale, against the latency you actually need? The target was never the largest model, the smallest, or the specialized one. It’s the right approach per task, chosen last, on measured outcomes, revisited when the measurements move.