OpenAI · Python

The OpenAI Python SDK accepts a base_url argument on the client constructor. Point it at the gateway and every chat completion routes through the classifier without changing the call sites in your application code. This recipe ships a 30-line script that you can drop into a fresh virtualenv, install one package, and run against your own org.

Drop-in switch

Two lines change in your existing client construction.

client = OpenAI(
    base_url="https://gateway.iq-routing.com/v1",
    api_key="gw_live_xxxxxxxx",
)

The base_url swap routes the request through the gateway. Everything below the client (calls to client.chat.completions.create, client.embeddings, client.models.list) hits the gateway, which forwards to the upstream provider the classifier picks.

Verify it routes

import json
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.iq-routing.com/v1",
    api_key=os.environ["IQ_GATEWAY_KEY"],
)

response = client.chat.completions.with_raw_response.create(
    model="auto",
    messages=[{"role": "user", "content": "Summarise: routing-aware gateway."}],
    max_tokens=64,
)

print("X-Request-Id:", response.headers.get("x-request-id"))
telemetry = json.loads(response.headers.get("x-iq-routing", "{}"))
print("Model picked:", telemetry.get("chosen_model"))
print("Provider:", telemetry.get("chosen_provider"))
print("Cache hit:", telemetry.get("cache_hit"))
print(response.parse().choices[0].message.content)

Set IQ_GATEWAY_KEY in your environment, run the script. The gateway emits two response headers per request: X-Request-Id (the correlation handle) and x-iq-routing (a JSON-encoded telemetry blob with the chosen model, provider, classifier verdict, cache-hit flag, latencies, and cost). Paste the X-Request-Id value into /requests?request_id=<uuid> to see the full routing decision (classifier output, fallback hops, the provider's raw response).

The model="auto" argument hands routing to the classifier. Pin a specific model (gpt-4o-mini, claude-haiku-4-5) when your call site wants a deterministic choice.

Using capability aliases

The cap:<name> syntax in the model field tells the gateway to pick a concrete model at routing time based on the capability's stable intent rather than a pinned model id. The operator's per-org overrides plus the focus-mode bias plus the circuit-breaker state shape the resolved tuple. The six default capabilities are reason-heavy, tool-call-strict, long-context-128k, vision, cheap-fast, and json-mode.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.iq-routing.com/v1",
    api_key=os.environ["IQ_GATEWAY_KEY"],
)

completion = client.chat.completions.create(
    model="cap:reason-heavy",
    messages=[{"role": "user", "content": "Plan the migration in three phases."}],
)

The resolved provider plus model surface in the x-iq-routing response header (a JSON-encoded payload with chosen_provider and chosen_model fields, among others), so the SDK consumer can inspect which concrete model handled the request via with_raw_response. Decode the header with json.loads(response.headers["x-iq-routing"]). See the capability aliases docs for the full resolver decision tree plus the override mechanics. The operator can override the default capability mapping via the capabilities dashboard editor at /settings/capabilities.

Common gotchas

The OpenAI Python SDK's default request timeout is 600 seconds. The gateway's p99 budget is 30 seconds. If a request takes longer than 30 seconds the gateway returns 504 with a request_id in the body, but the SDK still waits its full 10 minutes by default. Set an explicit timeout on the client:

client = OpenAI(
    base_url="https://gateway.iq-routing.com/v1",
    api_key=os.environ["IQ_GATEWAY_KEY"],
    timeout=60.0,
)

The SDK's default retry behaviour catches 429 and retries with exponential backoff up to two times. The gateway's 429 already encodes a Retry-After header that the operator's rate-limit cooldown respects; SDK-side retries on top of that double-bill against the cooldown window. Disable SDK retries when you call through the gateway:

client = OpenAI(
    base_url="https://gateway.iq-routing.com/v1",
    api_key=os.environ["IQ_GATEWAY_KEY"],
    timeout=60.0,
    max_retries=0,
)

Streaming works as you would expect against /v1/chat/completions. The gateway forwards the SSE stream chunk-by-chunk; the X-Request-Id header arrives on the first event so the correlation handle is available before the last token.

Try this in the playground

Run this snippet against your gateway key →