LangChain: ChatOpenAI with a custom base URL

Both snippets were run against our live API.

Python

pip install -U langchain-openai

from langchain_openai import ChatOpenAI, OpenAIEmbeddings

llm = ChatOpenAI(
    model="anthropic/claude-sonnet-5.5",
    base_url="https://YOUR-ENDPOINT/v1",
    api_key="YOUR-KEY",
    stream_usage=True,      # with a custom base_url, usage in streams is off unless you turn it on
)
print(llm.invoke("Say hi in 3 words.").content)

for chunk in llm.stream("Count from 1 to 3."):
    print(chunk.content, end="")

emb = OpenAIEmbeddings(
    model="openai/text-embedding-3-small",
    base_url="https://YOUR-ENDPOINT/v1",
    api_key="YOUR-KEY",
    check_embedding_ctx_length=False,   # send plain text, not tiktoken ids
)
print(len(emb.embed_query("hello")))   # 1536

JavaScript / TypeScript

npm install @langchain/openai @langchain/core

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "anthropic/claude-sonnet-5.5",
  apiKey: process.env.YOUR_KEY,
  configuration: { baseURL: "https://YOUR-ENDPOINT/v1" },
});
console.log((await llm.invoke("Say hi in 3 words.")).content);

for await (const chunk of await llm.stream("Count from 1 to 3.")) {
  process.stdout.write(String(chunk.content));
}

FAQ