Routing and fallbacks

Routing policy controls requests that ask Relay to choose a model. Direct requests for a provider-prefixed model do not invoke automatic selection.

​

Pinned routing

Pinned routing sends eligible requests to one configured model. Use it when consistency matters more than per-request optimization, or while establishing a baseline.

​

Automatic routing

Automatic routing evaluates each request against the project’s routable candidates. Configure:

• Candidates: the models Relay may select
• Tradeoff: cost, quality, or latency
• Fallback model: the deterministic model to use when the preferred selection cannot be completed

Start with a small candidate set whose modalities and tool support match the application. Expand only after comparison data shows a reason.

​

Fallbacks

A fallback protects availability when the preferred route is unavailable or the routing engine cannot complete a selection. It is not a substitute for:

• provider retry and timeout policy
• a provider credential
• application-level handling in route-only mode
• compatible model capabilities

Exercise the fallback before production by using a controlled unavailable candidate or equivalent test environment.

​

Understand a decision

Traffic records distinguish:

• engine: selected by automatic routing
• policy: selected by configured policy
• fallback: selected by fallback behavior
• no routing reason: the request named a model directly

When a custom router participates, Relay also records its identifier and name.