LangChain · Python

LangChain's ChatOpenAI and ChatAnthropic wrappers accept the same base_url argument as the underlying SDK. Swap the base URL on the model constructor and every chain that uses the LLM routes through the gateway.

Drop-in switch

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="auto",
    base_url="https://gateway.iq-routing.com/v1",
    api_key="gw_live_xxxxxxxx",
)

langchain-anthropic follows the same pattern with base_url pointing at the gateway origin (no /v1 suffix; LangChain's wrapper appends the path itself).

Verify it routes

import os
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(
    model="auto",
    base_url="https://gateway.iq-routing.com/v1",
    api_key=os.environ["IQ_GATEWAY_KEY"],
    timeout=60,
    max_retries=0,
)

result = llm.invoke([HumanMessage(content="Summarise: routing-aware gateway.")])
print("Content:", result.content)
print("Model picked:", result.response_metadata.get("model_name"))
print("Token usage:", result.response_metadata.get("token_usage"))

Drop into a fresh venv with pip install langchain langchain-openai. Set IQ_GATEWAY_KEY in your environment, run. The response_metadata["model_name"] field carries the chosen upstream model; the token_usage dict matches the standard OpenAI completion tokens shape.

The Anthropic equivalent:

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(
    model="claude-haiku-4-5-20251001",
    base_url="https://gateway.iq-routing.com",
    api_key=os.environ["IQ_GATEWAY_KEY"],
    timeout=60,
    max_retries=0,
)

result = llm.invoke([HumanMessage(content="Say hello.")])

Using capability aliases

The cap:<name> syntax in the model field tells the gateway to pick a concrete model at routing time based on the capability's stable intent rather than a pinned model id. The operator's per-org overrides plus the focus-mode bias plus the circuit-breaker state shape the resolved tuple. The six default capabilities are reason-heavy, tool-call-strict, long-context-128k, vision, cheap-fast, and json-mode.

from langchain_anthropic import ChatAnthropic

chat = ChatAnthropic(
    model="cap:reason-heavy",
    anthropic_api_url="https://gateway.iq-routing.com",
    api_key=os.environ["IQ_GATEWAY_KEY"],
)

result = chat.invoke([HumanMessage(content="Plan the migration in three phases.")])

The resolved provider plus model surface in the x-iq-routing response header (a JSON-encoded payload with chosen_provider and chosen_model fields, among others), so the chain author can inspect which concrete model handled each invocation through the response_metadata callback hook described below. See the capability aliases docs for the full resolver decision tree plus the override mechanics. The operator can override the default capability mapping via the capabilities dashboard editor at /settings/capabilities.

Common gotchas

LangChain wraps the underlying SDK but does not expose the raw response headers by default. The X-Request-Id correlation handle is not in response_metadata. To capture it, register a callback:

from langchain_core.callbacks import BaseCallbackHandler

class RequestIdCapture(BaseCallbackHandler):
    def on_llm_end(self, response, **kwargs):
        gen = response.generations[0][0]
        if gen.generation_info:
            print("Request id:", gen.generation_info.get("system_fingerprint"))

llm.invoke(messages, config={"callbacks": [RequestIdCapture()]})

LangChain's default retry-on-429 is aggressive (up to 6 retries with exponential backoff). The gateway's 429 carries Retry-After; LangChain's retries do not respect the header. Set max_retries=0 on the model constructor and rely on your own queue.

LangSmith tracing (LANGCHAIN_TRACING_V2=true) captures full prompt content. If you route through the gateway, every prompt traces to LangSmith's servers in addition to the gateway's route_decisions log. Pin your tracing scope; the gateway alone has the routing decision, the costs, and the cache hits.

For ChatAnthropic, the gateway pins the Anthropic SDK's minimum beta-header version. If your chain uses the Computer Use beta or a beta not yet supported on the gateway, the call returns 400 with a clear request_id; no silent fall-through.

Try this in the playground

Run this snippet against your gateway key →