Providers

OpenAI-compatible endpoints

OpenAiCompatible (feature http) sends POST {base_url}/chat/completions.

use std::time::Duration;
use typedlm::http::OpenAiCompatible;

let provider = OpenAiCompatible::new("https://api.openai.com/v1", "gpt-5-mini")
    .api_key(key)
    .max_transport_retries(2)                     // default 2
    .initial_backoff(Duration::from_millis(500)); // default, doubled per retry

let local = OpenAiCompatible::ollama("qwen3:32b"); // http://localhost:11434/v1

Connection errors, 429 and 5xx responses are retried; Retry-After takes precedence over the backoff (capped at 60 s). Other 4xx responses fail immediately with ProviderError::Status. These transport retries are separate from repairing invalid answers.

Routers that speak the same protocol and translate to other vendors work the same way: point base_url at them.

Strategies

A provider declares which strategies it supports; the program uses the strongest one unless you set one with Program::strategy.

Strategy	Request	Model sees the schema
NativeSchema	response_format: json_schema	depends on the server, see below
ToolCall	a forced tool respond with the schema as parameters	yes
JsonMode	response_format: json_object, schema in the instructions	yes
PromptOnly	schema in the instructions only	yes

Some servers, Ollama among them, use a native schema only to constrain decoding: the model never sees field descriptions such as “date as YYYY-MM-DD”. Capabilities::native_schema_visible(false) puts the schema into the instructions as well; OpenAiCompatible::ollama sets it.

When a local model writes a tool call as text ({"name": "respond", "arguments": {…}}) instead of a structured tool call, the provider unpacks the arguments.

Schema dialects

The schema generated by schemars is rewritten for the provider; validation always uses the original.

• SchemaDialect::Generic inlines $ref definitions, because servers often render schemas as text and models do not resolve references. Recursive types keep their $ref.
• SchemaDialect::OpenAiStrict follows OpenAI’s strict mode: every property required, optional ones nullable, additionalProperties: false, anyOf instead of oneOf, only supported string formats. The output type must be a struct; strict mode rejects an enum at the root.

Capabilities for other servers

use typedlm::{Capabilities, SchemaDialect, Strategy};

let provider = OpenAiCompatible::new("http://localhost:8080/v1", "my-model").capabilities(
    Capabilities::new(vec![Strategy::JsonMode, Strategy::PromptOnly], SchemaDialect::Generic),
);

Your own provider

use typedlm::{Capabilities, Provider, ProviderError, Request, Response};

struct MyProvider;

impl Provider for MyProvider {
    fn capabilities(&self) -> Capabilities { /* … */ }

    async fn complete(&self, request: Request) -> Result<Response, ProviderError> {
        // request.instructions, request.input (JSON), request.output_schema,
        // request.strategy, request.options, request.repair (earlier answers + feedback)
    }
}

Return the raw answer text in Response::content, the model that actually answered, a FinishReason (Refusal and Length are not repaired) and token usage. Transport retries belong in the provider.

To hold different providers in one collection, use Box<dyn DynProvider>; it implements Provider itself.