Swapping the base URL of an OpenAI SDK is easy. Finding out at 2 a.m. that tool calls silently break on the new endpoint is not. This is the short checklist I run against any OpenAI-compatible endpoint (self-hosted vLLM, LiteLLM proxy, or a hosted gateway) before I point coding agents at it.
1. List models
curl -s $OPENAI_BASE_URL/models -H "Authorization: Bearer $OPENAI_API_KEY"
Copy model ids exactly. Do not guess date suffixes or prefixes.
2. Basic completion
curl $OPENAI_BASE_URL/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" -H "Content-Type: application/json" \
-d '{"model":"MODEL_ID","messages":[{"role":"user","content":"Reply with ok"}]}'
3. Streaming
Repeat with "stream": true and confirm your client parses every chunk and the final [DONE].
4. One tool-call round trip
Send a request with a single tools entry and tool_choice: "auto", return a fake tool result, and confirm the model continues. Tool support varies by model and host, and this is where most agent integrations break.
5. Long context
Send a prompt near the size your agent actually uses (for example 30-60k tokens of code). Check latency and that the answer still follows the system prompt.
6. Errors and limits
Fire a short burst of parallel requests and note what you get back: 429s, 5xx, or queuing. Make sure your retry logic is bounded.
7. Timeouts
Set an explicit client timeout (60-120 s for big edits). Default timeouts differ across SDK versions.
8. Logs and usage
Confirm you can see each request in the provider's dashboard or logs. If you cannot see it, you cannot debug a runaway loop.
9. Billing sanity check
Run one real agent task and compare the cost or request count to what you expected. Agent loops re-send context every turn, so per-token bills grow faster than people expect. Flat monthly plans are one way to make that predictable; APIClaw is an example of an OpenAI-compatible gateway priced that way (I build it; independent, not affiliated with any model vendor).
10. Write it down
Keep the working base URL, model ids, timeout and retry settings in your repo config so the next teammate does not repeat the whole process.
If all ten pass, the endpoint is ready for unattended agent runs.