Connecting Dify to a Custom OpenAI-Compatible API: Base URL, Model IDs and the Settings That Break Agents

TITLE: Connecting Dify to a Custom OpenAI-Compatible API: Base URL, Model IDs and the Settings That Break Agents TAGS: dify, openai, llm, api, agents SUBTITLE: How to add any OpenAI-compatible gateway to Dify with the OpenAI-API-compatible provider, and the defaults (Function Call Type, context size, Docker networking, a doubled /v1) that quietly break agents and embeddings.

I'm Aman Kumar. A question I get often: "I added my OpenAI-compatible endpoint to Dify, the chat preview works, but my agent never calls its tools." Sometimes it's "embeddings return 404" or "self-hosted Dify can't reach the API." Usually the base URL and key are fine. The problem is a setting in Dify's model form that defaults to something safe but wrong.

This post walks through connecting Dify to a custom OpenAI-compatible API, field by field, and explains which settings matter for agents. My examples use APIClaw, a gateway I build, so weigh that accordingly. The steps work the same for any endpoint that implements /v1/chat/completions, whether that's a hosted gateway, vLLM or LiteLLM.

Why use the OpenAI-API-compatible provider

Dify has dedicated providers for OpenAI, Anthropic and others, but those are tied to each vendor's own endpoints. For anything else that speaks the OpenAI wire format, Dify has a generic provider plugin called OpenAI-API-compatible. You register each model by hand: you tell Dify the model ID, the endpoint, and what the model can do. Dify doesn't auto-detect any of those capabilities, which is where the trouble starts.

Install the provider

In recent Dify versions, model providers are plugins. Go to your workspace, open Integrations → Model Provider (in older versions it's Settings → Model Providers), find OpenAI-API-compatible, and install it. Once it's installed, it shows up in the provider list with an option to add a model.

You add one entry per model. That's tedious, but it lets you set the right limits and capabilities for each one.

The settings, field by field

Here is what I put in for a chat model:

A few notes on these:

• Model name must be the exact ID. Dify sends it verbatim as the model field, and a near-miss fails. Copy the string from the provider's model hub or from GET /v1/models. On my endpoint that's deepseek-v4-flash for a fast, cheap default; the list also includes IDs like deepseek-v4-pro and claude-sonnet-5-5. Don't guess from vendor naming; gateways format IDs differently.
• The base URL includes /v1. For chat models, Dify appends /chat/completions to the base URL you enter. https://apiclaw.biz/v1 becomes https://apiclaw.biz/v1/chat/completions, which is what you want.
• Completion mode should be Chat. "Completion" mode targets the legacy /v1/completions endpoint, which many gateways and most modern models don't support.

The Function Call Type trap

This is the one that breaks agents. Function Call Type defaults to "Not Support". With that default, Dify treats the model as unable to call tools. Chat still works, the model test passes, and a basic chatbot app behaves normally, so nothing looks wrong.

Then you build an Agent app or add tools to a workflow, and the tools never run. Depending on app type and version, the agent either ignores the tools or falls back to a prompt-based ReAct-style loop that's far less reliable than native tool calls. There's usually no error in the UI.

Set Function Call Type to tool_call. That tells Dify to send OpenAI-style tools in the request and to parse tool_calls from the response. Only do this for models that actually support tool calling.

If your agent "works but never uses tools," check this setting before you touch prompts.

Context size and max tokens

The Model context size and Upper bound for max tokens fields have conservative defaults. If you leave them, two things go wrong:

• Dify trims conversation history and knowledge-base context to fit a window that's smaller than the model's real one, so long chats lose memory earlier than they should and RAG apps get fewer retrieved chunks.
• The max tokens slider in your app is capped at the low default, so long answers get cut off mid-sentence with finish_reason: length.

Set both to the model's documented values from the model hub. Setting them higher than the model supports causes the opposite problem: rejected requests.

Self-hosted Dify in Docker: networking

If you run Dify with Docker Compose, model calls go out from inside the containers, not from your host. That matters in two cases:

• Don't use localhost for something on the host. Inside a container, localhost is the container itself. If your endpoint runs on the host machine, use the host's LAN IP or host.docker.internal where that's supported.
• A public endpoint needs outbound access. For a hosted gateway, the containers must be able to resolve and reach https://apiclaw.biz (or your provider's domain). Firewall egress rules and outbound proxies are the usual cause of timeouts here.

A quick check is to run curl https://apiclaw.biz/v1/models -H "Authorization: Bearer $KEY" from inside the API or plugin container. If that fails, Dify will too, and the fix is networking, not model settings.

Embeddings and speech: the doubled /v1

You can also register embedding, rerank and speech models through the same provider. For some non-LLM model types, the plugin appends /v1 itself before adding the endpoint path. If you enter https://apiclaw.biz/v1, the request goes to /v1/v1/embeddings and returns a 404.

If a chat model works with https://apiclaw.biz/v1 but an embedding or speech model on the same endpoint 404s, try the site root, https://apiclaw.biz, for that model type only. Keep /v1 for LLM entries. Plugin behavior has changed between versions, so test each model type.

How I verify a new model

I don't trust a setup until I've checked three things:

1. The save-time test. When you save the model, Dify sends a small validation request. If it fails, the error text usually tells you whether it's auth (401), a wrong path or model ID (404), or networking (timeout).
2. A preview in a real app. Create a throwaway Agent app with one simple tool, such as the current-time tool, and ask a question that needs it. If the tool call shows up in the run trace, Function Call Type is set correctly.
3. Request logs on both sides. Dify's app logs show what was sent and how long it took. Your gateway's request log should show the same calls with the exact model ID. If the two don't match, you've found the bug.

When I'd use Dify versus calling the API directly

Dify fits when non-developers need to build and adjust apps, when you want knowledge bases, workflows and logs without writing the plumbing, or when you want to swap models from a dropdown. One gateway with several model entries lets you compare models in the same app.

For a single, tightly scoped feature inside an existing codebase, I'd still call the API from the SDK directly. You get full control over retries, timeouts and tool schemas, and there's one less service to run.

Checklist

• Provider: OpenAI-API-compatible, one entry per model
• Model name copied exactly from /v1/models or the model hub
• API Base URL https://apiclaw.biz/v1 for chat models
• Completion mode: Chat
• Context size and max tokens set to the model's real limits
• Function Call Type: tool_call for any model used by agents or tools
• Vision on only for image inputs
• Docker containers can reach the endpoint (no host localhost)
• Embeddings or speech 404 → try the site root for that model type
• Verified with a tool-using test app and the request logs

Ten minutes with a throwaway test app is much cheaper than finding out from users that your agent has been ignoring its tools for a week.