Providers and paths

Point a client SDK or a gateway's upstream at these base URLs (HOST is your Rustybin, for example localhost:8080):

Any model name is accepted for generation: the name is echoed back, and a name segment can select a mode. The models endpoints list a few well-known names (gpt-4o, claude-sonnet-4-5, gemini-2.5-flash, llama3.2, ...) plus rustybin-* mode models; retrieving an unlisted model by id answers the provider's native 404.

For example, with the Python SDKs:

OpenAI(base_url="http://localhost:8080/ai/openai/v1", api_key="anything")
AzureOpenAI(azure_endpoint="http://localhost:8080/ai/azure", api_key="anything", api_version="2024-10-21")
anthropic.Anthropic(base_url="http://localhost:8080/ai/anthropic", api_key="anything")
genai.Client(api_key="anything", http_options=types.HttpOptions(base_url="http://localhost:8080/ai/gemini", api_version="v1beta"))

Credentials are not checked unless you ask for it, see credential checks.

Every provider response (all /ai/* routes except /ai/requests) carries X-Rustybin-Request-Id, X-Rustybin-Provider, X-Rustybin-Instance, X-Rustybin-Credential and, for generation endpoints, X-Rustybin-Model, X-Rustybin-Mode and success rate-limit headers, see Faults, latency, credentials, inspection.

OpenAI

Chat Completions supports n (up to 8, 2 in public mode), max_tokens / max_completion_tokens, stop, content parts (text, images, audio, files), tools / tool_choice (one tool call per reply), response_format (json_object, json_schema), and streaming with stream_options.include_usage. Rate-limit headers: x-ratelimit-limit-requests, x-ratelimit-limit-tokens, x-ratelimit-remaining-*, x-ratelimit-reset-*.

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{
  "model": "gpt-4o",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "hello"}
  ]
}
HTTP 200
[Asserts]
header "X-Rustybin-Provider" == "openai"
header "X-Rustybin-Model" == "gpt-4o"
header "X-Rustybin-Mode" == "canned"
header "X-Rustybin-Credential" == "none"
header "x-ratelimit-limit-tokens" == "2000000"
jsonpath "$.object" == "chat.completion"
jsonpath "$.choices[0].message.role" == "assistant"
jsonpath "$.choices[0].message.content" startsWith "Hello! I'm Rustybin"
jsonpath "$.choices[0].finish_reason" == "stop"
jsonpath "$.usage.total_tokens" > 0

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{
  "model": "gpt-4o",
  "stream": true,
  "stream_options": {"include_usage": true},
  "max_tokens": 3,
  "messages": [{"role": "user", "content": "hello"}]
}
HTTP 200
[Asserts]
header "Content-Type" contains "text/event-stream"
body contains "\"object\":\"chat.completion.chunk\""
body contains "{\"content\":\"Hell\"}"
body contains "\"finish_reason\":\"length\""
body contains "\"completion_tokens\":3"
body endsWith "data: [DONE]\n\n"

# The legacy /ai/v1 prefix serves the same API.
POST {{base_url}}/ai/v1/chat/completions
Content-Type: application/json
{"model": "gpt-4o-mini", "n": 2, "messages": [{"role": "user", "content": "hi"}]}
HTTP 200
[Asserts]
jsonpath "$.choices" count == 2

Legacy completions support stream, echo, suffix and n:

POST {{base_url}}/ai/openai/v1/completions
Content-Type: application/json
{"model": "gpt-3.5-turbo-instruct", "prompt": "Say hello", "max_tokens": 4, "echo": true}
HTTP 200
[Asserts]
jsonpath "$.object" == "text_completion"
jsonpath "$.choices[0].text" startsWith "Say hello"
jsonpath "$.choices[0].finish_reason" == "length"

The Responses API returns output items (a message with output_text, or function_call items); text.format gives structured output; streaming emits the native event sequence (response.created, response.in_progress, response.output_item.added, response.content_part.added, response.output_text.delta..., response.completed, or response.incomplete when max_output_tokens cut the text).

POST {{base_url}}/ai/openai/v1/responses
Content-Type: application/json
{"model": "gpt-4o", "instructions": "Be brief.", "input": "hello"}
HTTP 200
[Asserts]
jsonpath "$.object" == "response"
jsonpath "$.status" == "completed"
jsonpath "$.output[0].type" == "message"
jsonpath "$.output[0].content[0].type" == "output_text"

POST {{base_url}}/ai/openai/v1/responses
Content-Type: application/json
{"model": "gpt-4o", "input": "hello", "stream": true, "max_output_tokens": 3}
HTTP 200
[Asserts]
body contains "event: response.created"
body contains "event: response.output_text.delta"
body contains "event: response.incomplete"

Models, moderations (keyword categories with deterministic scores), image generation (a tiny valid PNG as b64_json, or a URL to /image/png) and audio transcription (multipart; json, text, srt, vtt or verbose_json; the audio is not decoded: the transcript is a fixed sentence and the duration is estimated from the file size):

GET {{base_url}}/ai/openai/v1/models
HTTP 200
[Asserts]
jsonpath "$.object" == "list"
jsonpath "$.data[*].id" contains "rustybin-echo"
jsonpath "$.data[*].id" contains "text-embedding-3-small"

GET {{base_url}}/ai/openai/v1/models/no-such-model
HTTP 404
[Asserts]
jsonpath "$.error.code" == "model_not_found"

POST {{base_url}}/ai/openai/v1/moderations
Content-Type: application/json
{"model": "omni-moderation-latest", "input": "I will kill you"}
HTTP 200
[Asserts]
jsonpath "$.results[0].flagged" == true
jsonpath "$.results[0].categories.violence" == true

POST {{base_url}}/ai/openai/v1/images/generations
Content-Type: application/json
{"model": "gpt-image-1", "prompt": "a rusty bin", "response_format": "b64_json"}
HTTP 200
[Asserts]
jsonpath "$.data[0].b64_json" startsWith "iVBORw0KGgo"

POST {{base_url}}/ai/openai/v1/audio/transcriptions
[Multipart]
model: whisper-1
response_format: verbose_json
file: file,sample.wav; audio/wav
HTTP 200
[Asserts]
jsonpath "$.text" exists
jsonpath "$.segments" count >= 1

Azure OpenAI

The same engine and payloads as OpenAI, with the Azure specifics: api-version is required (400 without it), the deployment name is the model, the credential is the api-key header, and responses carry prompt_filter_results and per-choice content_filter_results.

POST {{base_url}}/ai/azure/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21
api-key: azure-demo-key
Content-Type: application/json
{"messages": [{"role": "user", "content": "hello"}], "max_tokens": 3}
HTTP 200
[Asserts]
header "X-Rustybin-Provider" == "azure"
header "X-Rustybin-Credential" == "api-key ****-key"
jsonpath "$.model" == "gpt-4o"
jsonpath "$.prompt_filter_results[0].content_filter_results.hate.filtered" == false
jsonpath "$.choices[0].content_filter_results.violence.severity" == "safe"

POST {{base_url}}/ai/azure/openai/deployments/gpt-4o/chat/completions
Content-Type: application/json
{"messages": [{"role": "user", "content": "hello"}]}
HTTP 400
[Asserts]
jsonpath "$.error.message" contains "api-version"

Anthropic

Messages with system as a string or text blocks (with cache_control), content blocks (text, image, document, tool_use, tool_result), tools with tool_choice (auto, any, tool, none), output_format JSON schema, stop_sequences, and the native SSE sequence (message_start, content_block_start, ping, content_block_delta with text_delta / input_json_delta, content_block_stop, message_delta, message_stop). max_tokens is required, like the real API. Responses carry request-id and anthropic-ratelimit-* headers.

POST {{base_url}}/ai/anthropic/v1/messages
anthropic-version: 2023-06-01
Content-Type: application/json
{
  "model": "claude-sonnet-4-5",
  "max_tokens": 256,
  "system": "You are a helpful assistant.",
  "messages": [{"role": "user", "content": "hello"}]
}
HTTP 200
[Asserts]
header "X-Rustybin-Provider" == "anthropic"
header "request-id" startsWith "req_"
header "anthropic-ratelimit-requests-limit" == "4000"
jsonpath "$.type" == "message"
jsonpath "$.content[0].type" == "text"
jsonpath "$.stop_reason" == "end_turn"
jsonpath "$.usage.input_tokens" > 0

POST {{base_url}}/ai/anthropic/v1/messages
Content-Type: application/json
{"model": "claude-sonnet-4-5", "max_tokens": 3, "stream": true, "messages": [{"role": "user", "content": "hello"}]}
HTTP 200
[Asserts]
body contains "event: message_start"
body contains "event: content_block_delta"
body contains "\"type\":\"text_delta\""
body contains "event: message_stop"

Prompt caching is simulated: the prefix up to the last cache_control block is hashed; the first request reports it as cache_creation_input_tokens, repeats within five minutes from the same session and model as cache_read_input_tokens.

# Prompt caching: the first request creates the cache entry, a repeat reads it.
POST {{base_url}}/ai/anthropic/v1/messages
X-Rustybin-Session: docs-cache
Content-Type: application/json
{"model": "claude-sonnet-4-5", "max_tokens": 20, "system": [{"type": "text", "text": "Long shared instructions for the docs cache demo.", "cache_control": {"type": "ephemeral"}}], "messages": [{"role": "user", "content": "hello"}]}
HTTP 200
[Asserts]
jsonpath "$.usage.cache_creation_input_tokens" > 0
jsonpath "$.usage.cache_read_input_tokens" == 0

POST {{base_url}}/ai/anthropic/v1/messages
X-Rustybin-Session: docs-cache
Content-Type: application/json
{"model": "claude-sonnet-4-5", "max_tokens": 20, "system": [{"type": "text", "text": "Long shared instructions for the docs cache demo.", "cache_control": {"type": "ephemeral"}}], "messages": [{"role": "user", "content": "hello again"}]}
HTTP 200
[Asserts]
jsonpath "$.usage.cache_creation_input_tokens" == 0
jsonpath "$.usage.cache_read_input_tokens" > 0

# max_tokens is required, like the real API.
POST {{base_url}}/ai/anthropic/v1/messages
Content-Type: application/json
{"model": "claude-sonnet-4-5", "messages": [{"role": "user", "content": "hello"}]}
HTTP 400
[Asserts]
jsonpath "$.type" == "error"
jsonpath "$.error.type" == "invalid_request_error"

POST {{base_url}}/ai/anthropic/v1/messages/count_tokens
Content-Type: application/json
{"model": "claude-sonnet-4-5", "messages": [{"role": "user", "content": "hello"}]}
HTTP 200
[Asserts]
jsonpath "$.input_tokens" > 0

Gemini

systemInstruction, contents, tools[].functionDeclarations (replies with functionCall parts, accepts functionResponse parts), toolConfig (AUTO, ANY, NONE, allowedFunctionNames), generationConfig (maxOutputTokens, stopSequences, candidateCount, responseMimeType with responseSchema or responseJsonSchema) and usageMetadata. Credentials: x-goog-api-key or ?key=.

POST {{base_url}}/ai/gemini/v1beta/models/gemini-2.5-flash:generateContent
x-goog-api-key: gemini-demo-key
Content-Type: application/json
{
  "systemInstruction": {"parts": [{"text": "Be brief."}]},
  "contents": [{"role": "user", "parts": [{"text": "hello"}]}],
  "generationConfig": {"maxOutputTokens": 5}
}
HTTP 200
[Asserts]
header "X-Rustybin-Provider" == "gemini"
header "X-Rustybin-Credential" == "x-goog-api-key ****-key"
jsonpath "$.candidates[0].content.role" == "model"
jsonpath "$.candidates[0].finishReason" == "MAX_TOKENS"
jsonpath "$.usageMetadata.candidatesTokenCount" == 5

POST {{base_url}}/ai/gemini/v1beta/models/gemini-2.5-flash:streamGenerateContent?alt=sse
Content-Type: application/json
{"contents": [{"role": "user", "parts": [{"text": "hello"}]}], "generationConfig": {"maxOutputTokens": 3}}
HTTP 200
[Asserts]
header "Content-Type" contains "text/event-stream"
body startsWith "data: {\"candidates\""

GET {{base_url}}/ai/gemini/v1beta/models
HTTP 200
[Asserts]
jsonpath "$.models[*].name" contains "models/gemini-2.5-flash"

POST {{base_url}}/ai/gemini/v1beta/models/gemini-2.5-flash:countTokens
Content-Type: application/json
{"contents": [{"role": "user", "parts": [{"text": "hello"}]}]}
HTTP 200
[Asserts]
jsonpath "$.totalTokens" > 0

AWS Bedrock

Converse and ConverseStream (application/vnd.amazon.eventstream binary frames with CRC32 checksums: messageStart, contentBlockStart, contentBlockDelta, contentBlockStop, messageStop, metadata), InvokeModel with the model family's native body (Anthropic anthropic_version Messages body, Titan inputText text or embeddings, Llama-style prompt), and InvokeModelWithResponseStream (chunk events with base64 JSON). Responses carry x-amzn-requestid.

POST {{base_url}}/ai/bedrock/model/anthropic.claude-3-5-sonnet-20240620-v1:0/converse
Content-Type: application/json
{
  "system": [{"text": "Be brief."}],
  "messages": [{"role": "user", "content": [{"text": "hello"}]}],
  "inferenceConfig": {"maxTokens": 5}
}
HTTP 200
[Asserts]
header "X-Rustybin-Provider" == "bedrock"
header "x-amzn-requestid" exists
jsonpath "$.output.message.role" == "assistant"
jsonpath "$.stopReason" == "max_tokens"
jsonpath "$.usage.outputTokens" == 5

POST {{base_url}}/ai/bedrock/model/amazon.nova-pro-v1:0/converse-stream
Content-Type: application/json
{"messages": [{"role": "user", "content": [{"text": "hello"}]}], "inferenceConfig": {"maxTokens": 3}}
HTTP 200
[Asserts]
header "Content-Type" == "application/vnd.amazon.eventstream"
# Binary AWS event stream frames: ":event-type" headers, a messageStop event.
bytes contains hex,3a6576656e742d74797065;
bytes contains hex,6d65737361676553746f70;

POST {{base_url}}/ai/bedrock/model/anthropic.claude-3-haiku-20240307-v1:0/invoke
Content-Type: application/json
{"anthropic_version": "bedrock-2023-05-31", "max_tokens": 20, "messages": [{"role": "user", "content": "hello"}]}
HTTP 200
[Asserts]
jsonpath "$.type" == "message"

POST {{base_url}}/ai/bedrock/model/amazon.titan-embed-text-v2:0/invoke
Content-Type: application/json
{"inputText": "Hello world", "dimensions": 256}
HTTP 200
[Asserts]
jsonpath "$.embedding" count == 256

Ollama

chat and generate stream newline-delimited JSON by default (like Ollama; send "stream": false for one object), with tools and format (JSON or a schema). tags lists local models; embed returns 768 dimensions by default; embeddings is the legacy single-prompt form.

POST {{base_url}}/ai/ollama/api/chat
Content-Type: application/json
{"model": "llama3.2", "messages": [{"role": "user", "content": "hello"}], "stream": false}
HTTP 200
[Asserts]
jsonpath "$.done" == true
jsonpath "$.message.role" == "assistant"

# Streaming is on by default: newline-delimited JSON.
POST {{base_url}}/ai/ollama/api/generate
Content-Type: application/json
{"model": "llama3.2", "prompt": "hi", "options": {"num_predict": 2}}
HTTP 200
[Asserts]
header "Content-Type" contains "application/x-ndjson"
body contains "\"done\":false"
body contains "\"done_reason\":\"length\""

Cohere

v2/rerank and v2/embed (float, int8, uint8, base64 embedding types), and the generic /ai/v1/rerank alias for Jina or Voyage style clients: see Embeddings and rerank.