Behaviour: modes, rules, tools, structured output

Every provider front end turns its native request into the same internal form (system prompt, messages with content parts, tools, tool choice, response format, limits), generates a reply, and renders it back in the native shape. So the rules below apply to all providers.

How a reply is chosen, in order:

Echo mode: the rendered prompt, exactly as received.
The last messages carry tool results: a final answer quoting them.
Tools are supplied and the tool choice is not none: a tool call when a tool is forced, or when the last user message mentions a tool.
Structured output requested: JSON valid against the schema.
Otherwise text from the mode: canned, scripted or random.

Then stop sequences and the token limit are applied.

Modes

The mode comes from the X-Rustybin-Mode header, else from a segment of the model name (split on -, _, :, /, ., @): rustybin-echo, gpt-4o:scripted, random. The default is canned. The response reports it in X-Rustybin-Mode.

Mode	Reply
`canned` (also `default`)	A keyword table on the last user message (whole words): greetings (`hello`, `hi`, `hey`, `greetings`, `howdy`), code (`code`, `python`, `function`, `program`, `script`), JSON (`json`, `data`), long text (`long`, `essay`, `explain`, `article`), else a default paragraph
`echo`	The prompt the upstream received, one line per part: `system: ...`, `tools: ...`, `user: ...`, `assistant: [tool_call name] {...}`, `tool (name): ...`; images, audio and files appear as placeholders
`scripted` (also `script`)	The demo rules below
`random`	Sentences from a gateway-themed vocabulary, seeded by a hash of the model and prompt: identical requests get identical text, `n > 1` choices differ

Echo mode is the tool for prompt decorator, template and guard demos: what the gateway prepended, appended or rewrote is right there in the reply.

# Echo mode returns the prompt exactly as the upstream received it, so a
# gateway's prompt decorator or template plugin becomes visible.
POST {{base_url}}/ai/openai/v1/chat/completions
X-Rustybin-Mode: echo
Content-Type: application/json
{"model": "gpt-4o", "messages": [{"role": "system", "content": "Be brief."}, {"role": "user", "content": "hi there"}]}
HTTP 200
[Asserts]
header "X-Rustybin-Mode" == "echo"
jsonpath "$.choices[0].message.content" == "system: Be brief.\nuser: hi there"

# A model name segment selects the mode too: rustybin-echo, gpt-4o:scripted, random-model.
POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{"model": "rustybin-echo", "messages": [{"role": "user", "content": "ping"}]}
HTTP 200
[Asserts]
header "X-Rustybin-Mode" == "echo"
jsonpath "$.choices[0].message.content" == "user: ping"

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{"model": "gpt-4o", "messages": [{"role": "user", "content": "write a python function"}]}
HTTP 200
[Asserts]
jsonpath "$.choices[0].message.content" contains "```python"

# Random mode is seeded by model + prompt: identical requests, identical text.
POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{"model": "rustybin-random", "messages": [{"role": "user", "content": "anything"}]}
HTTP 200
[Captures]
random_text: jsonpath "$.choices[0].message.content"

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{"model": "rustybin-random", "messages": [{"role": "user", "content": "anything"}]}
HTTP 200
[Asserts]
jsonpath "$.choices[0].message.content" == "{{random_text}}"

Scripted rules

Checked in order on the last user message (whole words or phrases, case-insensitive):

Trigger	Reply
`lorem N`	N lorem ipsum words (default 50, at most 4000; 500 in public mode)
`ssn`, `social security`, `credit card`, `pii`, `customer record`	A fake customer record: SSN `123-45-6789`, card `4111 1111 1111 1111`, email, phone, address (response sanitiser demos)
`toxic`, `jailbreak`, `unsafe`, `harmful`	A clearly labelled simulated unsafe answer an output guardrail should block
`secret`, `secrets`, `api key`, `password`, `credentials`	Fake cloud keys and tokens (secret redaction demos)
`refuse`, `refusal`	"I'm sorry, but I can't help with that request."
`json`	A bare JSON object
`markdown`, `table`	Markdown with a table and links
`url`, `urls`, `link`, `links`	Text with allowed and suspicious URLs (URL filter demos)
`echo`	The rendered prompt, like echo mode
anything else	The canned reply

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{"model": "gpt-4o:scripted", "messages": [{"role": "user", "content": "show me the customer record"}]}
HTTP 200
[Asserts]
header "X-Rustybin-Mode" == "scripted"
jsonpath "$.choices[0].message.content" contains "SSN: 123-45-6789"
jsonpath "$.choices[0].message.content" contains "4111 1111 1111 1111"

POST {{base_url}}/ai/anthropic/v1/messages
X-Rustybin-Mode: scripted
Content-Type: application/json
{"model": "claude-sonnet-4-5", "max_tokens": 1000, "messages": [{"role": "user", "content": "lorem 5"}]}
HTTP 200
[Asserts]
jsonpath "$.content[0].text" == "Lorem ipsum dolor sit amet."

Tokens and limits

Token counts are deterministic and shared by every provider: a run of letters or digits costs one token per four characters (at least one), every other non-whitespace character costs one, whitespace attaches to the next piece. Images cost 85 tokens; prompt tokens count the rendered prompt plus the tool definitions. The same count drives the usage fields, the rate-limit headers, streaming (one token per delta, so completion_tokens equals the number of text deltas) and the output limit (max_tokens, max_completion_tokens, max_output_tokens, maxOutputTokens, maxTokens, num_predict), which ends the reply with the provider's "length" finish reason (length, max_tokens, MAX_TOKENS, incomplete). Stop sequences cut the text and report the sequence where the provider does.

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{"model": "gpt-4o", "max_tokens": 2, "messages": [{"role": "user", "content": "hello"}]}
HTTP 200
[Asserts]
jsonpath "$.choices[0].message.content" == "Hello"
jsonpath "$.choices[0].finish_reason" == "length"
jsonpath "$.usage.completion_tokens" == 2

POST {{base_url}}/ai/anthropic/v1/messages
Content-Type: application/json
{"model": "claude-sonnet-4-5", "max_tokens": 200, "stop_sequences": ["mock"], "messages": [{"role": "user", "content": "hello"}]}
HTTP 200
[Asserts]
jsonpath "$.stop_reason" == "stop_sequence"
jsonpath "$.stop_sequence" == "mock"
jsonpath "$.content[0].text" == "Hello! I'm Rustybin, a "

Tool calls

When tools are supplied and the tool choice is not none, the reply is a tool call if:

a specific tool is forced (OpenAI {"type": "function", ...}, Anthropic {"type": "tool"}, Gemini ANY with a single allowedFunctionNames entry), or any tool is required (OpenAI required, Anthropic any, Gemini ANY), in which case the best matching tool or else the first one is called; or
the last user message mentions a tool: its full name (get weather for get_weather or getWeather), a significant part of the name, or a significant word of its description (whole words; common words such as "get", "data" or "tool" do not count). The best scoring tool wins.

One tool call is returned per reply. Its arguments are generated from the tool's JSON schema: required and optional properties, enums (the first value), nested objects and arrays, $ref into $defs / definitions, anyOf / oneOf / allOf, string formats and bounds, number bounds, const and default. String values are chosen from the property name (email, city, date, ...) and the user text, so location becomes "Paris" for "What is the weather in Paris?".

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get the current weather for a location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {"type": "string"},
          "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
        },
        "required": ["location"]
      }
    }
  }]
}
HTTP 200
[Captures]
call_id: jsonpath "$.choices[0].message.tool_calls[0].id"
[Asserts]
jsonpath "$.choices[0].finish_reason" == "tool_calls"
jsonpath "$.choices[0].message.tool_calls[0].function.name" == "get_weather"
jsonpath "$.choices[0].message.tool_calls[0].function.arguments" == "{\"location\":\"Paris\",\"unit\":\"celsius\"}"

After the client sends the tool result back, the model answers by quoting it:

# After the tool result, the model answers by quoting it.
POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{
  "model": "gpt-4o",
  "messages": [
    {"role": "user", "content": "What is the weather in Paris?"},
    {"role": "assistant", "content": null, "tool_calls": [{"id": "{{call_id}}", "type": "function", "function": {"name": "get_weather", "arguments": "{\"location\":\"Paris\"}"}}]},
    {"role": "tool", "tool_call_id": "{{call_id}}", "content": "18 C and sunny"}
  ],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Get the current weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}}]
}
HTTP 200
[Asserts]
jsonpath "$.choices[0].finish_reason" == "stop"
jsonpath "$.choices[0].message.content" contains "18 C and sunny"

POST {{base_url}}/ai/anthropic/v1/messages
Content-Type: application/json
{
  "model": "claude-sonnet-4-5",
  "max_tokens": 200,
  "tool_choice": {"type": "tool", "name": "lookup_order"},
  "tools": [{"name": "lookup_order", "description": "Find an order", "input_schema": {"type": "object", "properties": {"order_id": {"type": "string"}, "email": {"type": "string", "format": "email"}}, "required": ["order_id", "email"]}}],
  "messages": [{"role": "user", "content": "Where is my stuff?"}]
}
HTTP 200
[Asserts]
jsonpath "$.stop_reason" == "tool_use"
jsonpath "$.content[0].type" == "tool_use"
jsonpath "$.content[0].name" == "lookup_order"
jsonpath "$.content[0].input.email" contains "@"

Streaming tool calls use each provider's native deltas (OpenAI tool_calls deltas, Anthropic input_json_delta, Responses response.function_call_arguments.delta).

Structured output

A JSON value valid against the requested schema (best effort) is returned for: OpenAI response_format (json_schema, or json_object for any JSON object), Responses text.format, Gemini responseSchema / responseJsonSchema (with responseMimeType: application/json; Gemini's upper-case type names work), Anthropic output_format or a forced tool, and Ollama format.

POST {{base_url}}/ai/openai/v1/chat/completions
Content-Type: application/json
{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Extract the person"}],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "person",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {"name": {"type": "string"}, "age": {"type": "integer", "minimum": 18}},
        "required": ["name", "age"],
        "additionalProperties": false
      }
    }
  }
}
HTTP 200
[Asserts]
jsonpath "$.choices[0].message.content" == "{\"name\":\"Jane Doe\",\"age\":30}"

POST {{base_url}}/ai/gemini/v1beta/models/gemini-2.5-flash:generateContent
Content-Type: application/json
{
  "contents": [{"role": "user", "parts": [{"text": "Give me a city"}]}],
  "generationConfig": {
    "responseMimeType": "application/json",
    "responseSchema": {"type": "OBJECT", "properties": {"city": {"type": "STRING"}, "population": {"type": "INTEGER"}}, "required": ["city", "population"]}
  }
}
HTTP 200
[Asserts]
jsonpath "$.candidates[0].content.parts[0].text" startsWith "{"
jsonpath "$.candidates[0].content.parts[0].text" contains "\"city\""