Relay gateway
Point an OpenAI-compatible client at Relay. A named provider model is passed through;oximy/auto asks Relay to select from the configured candidates. This path supports central keys, limits, fallbacks, traffic records, and routing explanations.
Existing gateway
Keep your established gateway in the request path and add Relay as the model-decision layer. At a high level, the application or gateway asks Relay for a decision, then the existing gateway executes the selected model using its own provider credentials and controls.Existing-gateway chaining does not have one universal configuration recipe.
The exact placement depends on your gateway’s routing hooks, authentication
model, and request format.
Observe over OTEL
Keep the current provider path and export OpenTelemetry traces to Relay. This is the lowest-risk way to establish traffic, cost, latency, and workload baselines before changing execution.Route-only
POST request context to Relay’s decision endpoint. Relay returns the selected model and a decision identifier; your application or gateway performs the provider call. Route-only is generally available for teams that need routing intelligence without proxy execution.Capabilities, not a one-time lock
Gateway and OTEL observation can coexist in a project. Shadow comparison is also a separate setting: it can evaluate a candidate alongside observed or gateway traffic, subject to provider access and budget.Use the Relay gateway
Swap an OpenAI-compatible base URL and let Relay execute.
Observe with OTEL
Export traces without changing the provider call path.
Use route-only
Ask Relay for a decision and execute it yourself.
Plan keys and providers
Separate Relay authentication from provider credentials.