OpenAI Compatibility¶

Verda Inference speaks the OpenAI Chat Completions API. OpenAI SDKs and tools work with Verda once you change two settings:

If you set OPENAI_BASE_URL and OPENAI_API_KEY in your environment, the OpenAI SDKs pick them up without code changes. See Quickstart.

Endpoints¶

Verda Inference serves POST /v1/chat/completions, with and without streaming, POST /v1/responses for clients that use the Responses API, and GET /v1/models, which lists the models your project can call. See Inference Endpoint API. Other OpenAI endpoints, such as embeddings, images or audio, return 404 with the code endpoint_not_supported.

Request parameters¶

You can send the usual Chat Completions parameters, including messages, max_tokens, temperature, top_p, stop, seed, tools, tool_choice and response_format. Tool calling and JSON mode work on models that list them under features on their model page.

A few things work differently:

• n must be 1. Each request returns one choice.
• Send extra parameters at the top level of the JSON body. A body field named extra_body is rejected with 400 unsupported_parameter. The Python SDK's extra_body argument is fine, because the SDK merges it into the body before sending.
• Parameters the API doesn't support are ignored.

Streaming¶

Set "stream": true to receive the response as server-sent events. The stream ends with data: [DONE].

To get token usage for a streamed response, ask for it:

"stream_options": {"include_usage": true}

The last chunk before [DONE] then carries the usage object.

Timeouts¶

A request without streaming times out after 60 seconds. A streamed response can run for up to 20 minutes. For long generations, such as an agent writing a large file, set "stream": true.

Requests that time out aren't charged.

Reasoning¶

Reasoning models think before they answer. They return their thinking in reasoning_content, separate from the answer in content. When streaming, it arrives in delta.reasoning_content. Reasoning counts as output tokens and is billed like them.

Two request fields control it:

• reasoning_effort sets how much the model thinks, for example low or high. Lower effort answers faster and costs less. Each model's page lists the efforts it accepts and its default.
• max_tokens caps the reasoning and the answer together. If the model uses it all while reasoning, content comes back empty and finish_reason is length. Raise max_tokens, or lower reasoning_effort.

Some models always reason, so every request uses some reasoning tokens. The reasoning field of GET /v1/inference/models lists a model's options: supported_efforts (cheapest first), default_effort, and optional, which says whether reasoning can be turned off.

Errors¶

Errors use the OpenAI error format, with error.type, error.code and error.message, so OpenAI SDKs raise them as usual.