Docs · Internals
Cognitive routing: how a model gets chosen
Cognitive routing lets Project Orchestrator choose the provider and model for a piece of work instead of you choosing each time. It starts by watching, not acting. This page walks through what each mode does, what is never replaced and how to read a decision.
What it does
Three modes. Primary changes nothing: every request goes where it goes today. Mixed lets PO route the executors while the pilot stays on the primary. Full lets PO route everyone.
A stage sets how far it may act. Shadow records what it would have chosen and applies nothing. Advisory applies to executors only. Auto applies according to the mode. The default is primary and shadow, which is the identity.
A provider, model, alias or run that you named is never replaced. Every automatic decision is stored with its reason, its alternatives and why the others were rejected, and can be recomputed to the same result.
The diagram
Drag to move. Pinch, Ctrl + wheel or the + and - keys to zoom; arrow keys to move; 0 to fit; double-click to fit. Select a node to light up its links.
Text description of the diagram
Flow diagram. Settings give a mode (primary, executors_only or full) and a stage (shadow, advisory or auto), with the project overriding the global value and only a person allowed to change them. A task signature describes the request without calling any model. An explicit choice skips the filter and is never substituted. Hard constraints remove the models that cannot do the work; if nobody is left the declared rules apply and the decision says no candidate. The remaining models are scored from learned success minus weighted cost and latency, with capped exploration seeded by the decision id. Whether the pick is applied depends on mode, stage and role: it either fills the project rule slot with routed_by auto, or is only recorded as the shadow pick. Every decision is stored before it is returned. The outcome becomes a reward that updates the chosen arm once, feeds a shadow report and can demote the stage back to shadow. In full mode the model can also change between turns on the same provider. A last node lists what the verified diagram does not establish.
Guided reading
Step 1. Settings
A mode (primary, executors_only or full) and a stage (shadow, advisory or auto), set globally with an override per project. The default is primary and shadow. Only a person can change them: an agent token is refused with 403, and an unknown mode or a weight out of range is a typed 400 that stores nothing.
Step 2. The task signature
Class (simple, complex, creative, retry, utility or chat), size, context needed, tools, images and remaining budget. It is computed with no network call and no model call, and it does not depend on the language of the request. Any attempt after the first is a retry.
Step 3. Hard constraints
A model that fails one is out, and the first failing reason is the one recorded: a remote machine is never a candidate, then consent, health, tool calling, context window, images, budget and trust. An unknown context window is rejected, never assumed. An unknown price passes but is never counted as free. A choice you named never enters this filter.
Step 4. Scoring
Each class, provider and model pair is an arm with a stored success record. A draw from that record, minus weighted cost and latency, gives the utility. The deep and fast aliases only seed a fresh arm and are never a rule. Exploration is capped at 25 percent and seeded by the decision id, so the same inputs give the same decision.
Step 5. Applied or only recorded
Primary and shadow never apply. Mixed applies to executors. Full applies to everyone. Advisory applies to executors only, the pilot is advised but not moved. An applied pick fills the project rule slot with routed_by auto and a readable reason. Otherwise it is stored as the shadow pick and everything else stays identical.
Step 6. The decision log
The decision is written before it is returned, applied or not, with its alternatives from best to worst and the rejected ones with their reason. The reason is one sentence without a URL or a secret. If no model qualifies, the declared rules apply unchanged and the decision says no candidate.
Step 7. Learning, report and demotion
When the work ends, the outcome becomes a reward between 0 and 1 from success, attempts, verification and marginal cost, and updates the chosen arm once. A shadow report shows the agreement rate and the estimated cost gap over the same priced basis. At the auto stage, if the learned picks fall clearly below the baseline of the unapplied ones, the stage goes back to shadow.
Step 8. Per turn in full mode
In full mode at the auto stage the model of the same provider may change between turns, at most once per turn and never two turns in a row. A session still keeps one provider for its whole life. An error or a decision that takes more than 2 seconds leaves the model unchanged.
Why it is built this way
Shadow first
Recording what it would have chosen before acting lets the cost gap be read on real traffic before anything is switched on, and the stage returns to shadow by itself if a learned pick underperforms.
Explicit means explicit
A substituted model changes cost and where your data goes without telling you. A choice you named is never replaced, and the decision is still recorded.
Seeded, so replayable
Exploration is random but its seed is the decision id. Anyone can recompute a decision from its inputs and get the same answer, which is what makes an automatic choice auditable.
Unknown is not free
A price that is not known never counts as zero. The report skips decisions it cannot price on one basis and counts them, and the cost gap stays empty rather than invented.
The report measures cost, not quality
The shadow report compares agreement and estimated cost. Whether the cheaper model was as good is learned slowly through the reward, not read from the report.
What is not established
- The compaction model choice is recorded in the shadow report but not applied: neither engine lets the summary choose its model yet.
- The queries that store arms and decisions in the graph have run in continuous integration only, not on a developer machine.
- Offline, Claude Code is never a candidate: its context window is unknown, and the interface has no label for that case yet.
- The estimated cost gap is empty when the pool knows no price for the models involved.
- Health is read from each provider's own check, cached for 60 seconds, and never against a real remote instance.
Public sources
Where to check each statement, in the public repositories.