A matching API is only the first layer.

Published 5 September 2026 · Release-candidate documentation

Changing an SDK's base URL is a small configuration step. Making an agent reliably edit a repository is a much larger interface contract. A service can correctly stream text in an OpenAI-shaped response while still being unsuitable for a coding harness. AblitAI exposes these boundaries explicitly so a familiar request format is not mistaken for support that has not been demonstrated.

1. Transport answers a narrow question

A text request sends messages and receives generated text. A compatible SDK needs the expected endpoint, authentication headers, JSON shape, error body, and streaming framing. Model listing also matters: the configured model must match an available stable ID rather than a client default. AblitAI's OpenAI Python and Node examples cover model listing, text, streaming, and readable errors through the local gateway.

A successful text response answers “can this client speak to this endpoint?” It does not answer “can this client run its normal tools?” The latter introduces data structures and lifecycle transitions that a text completion never exercises. The quickstart deliberately begins with text and an environment-held key.

2. A tool call is structured output with a lifecycle

A coding client usually declares tools such as reading a file or running a shell command. The model must select a permitted tool and supply arguments in the required schema. The inference engine must recognize the model's tool syntax, and the API must return a stable call identifier, name, and argument string. Treating ordinary generated prose as executable instructions is not a substitute for this protocol.

After execution, the client sends a tool result associated with that call ID. The next request includes assistant/tool history, and the model must continue correctly. A single attractive sample call is weak evidence: partial arguments, malformed JSON, empty results, parallel calls, and a following turn can each expose a different bug.

3. Streaming adds another boundary

Text can be streamed as incremental content. Tool arguments are different: a JSON argument string can arrive in several fragments before it becomes valid as a whole. A client needs a consistent index and identifier across those fragments, plus a correct terminal event. A parser must avoid leaking raw control markers into ordinary text or silently dropping a second call.

The vLLM tool-calling documentation describes parser and template requirements. Their existence in an engine does not demonstrate correct behavior for every community artifact. AblitAI's current public route leaves automatic tool choice and the production parser inactive; standard tool-bearing requests fail before inference reservation.

4. A model name does not establish a template contract

The hosted artifact is a pinned community modification of Qwen3.8-27B with AWQ INT4 weights. Its repository title includes MTP, but speculative decoding is not active in this deployment. The same principle applies to tools: family capabilities, template syntax, engine parsers, and a tested serving configuration must agree. Capabilities cannot be imported from a base-model overview by name alone.

The model card records the stable ID, exact artifact revision, engine version, hardware, and inactive features. That makes the unit of compatibility a particular deployment configuration rather than an informal family label. A future parser or context change requires fresh exact-GPU acceptance and stability work.

5. Startup instructions consume context too

A user's short question is only part of a harness request. System instructions, tool descriptions, project rules, history, and template overhead all consume context. The current endpoint has 8,192 total tokens and permits at most 4,096 generated tokens. These limits must accommodate the entire request, not just the sentence typed at the prompt.

In pinned local checks, Claude Code's default startup can exhaust that envelope before an API request is made. A bare invocation reduces startup behavior, but still encounters the closed tool gate. Neither failure can be resolved by advertising a larger context value without changing and testing the actual deployment. The reference pages describe this distinction before installation steps.

6. Each client uses a different interface

OpenCode uses an OpenAI-compatible Chat Completions provider. Codex uses the Responses wire API for the configured provider. Claude Code uses Anthropic Messages and its SSE conventions. Adapting those formats requires distinct event mappings and continuation handling, even when the underlying model is the same.

Anthropic's LLM gateway documentation does not support routing Claude Code to non-Claude models. AblitAI therefore describes that adapter as unofficial and experimental. A wire-level fixture is useful engineering evidence; it is not vendor endorsement or evidence that a normal interactive harness is release-supported.

7. Failures should preserve the account boundary

Unsupported tool, image, audio, and thinking requests should fail clearly before inference and before a credit reservation. Once inference starts, token accounting must survive normal completion, client disconnects, and malformed upstream output. A client retry policy can complicate this: repeated requests after an ambiguous interruption may create additional work even when the first response was not fully displayed.

The billing guide explains reservation and settlement. The error guide separates status codes and warns that some harnesses shorten server details or retry transient failures. Local mock tests check these interfaces without treating generated fixtures as evidence of real model quality.

8. Read support as a matrix

The useful dimensions are protocol implemented, pinned client tested locally, real model behavior checked, and exact deployment release-approved. They must remain separate. OpenAI Python and Node text and streaming have local SDK evidence. Codex and OpenCode are reference-only while tools are closed. Claude Code adds an unofficial-model boundary and a startup-envelope problem. None of the three standard tool-bearing harnesses is release-approved.

The integration matrix names pinned versions and links their setup subsets. Existing user configuration and local tool permissions remain the user's responsibility. A static example should never replace unrelated configuration or encourage bypassing approval controls just to make a smoke test pass.

9. What would change the support label

Promotion needs a reproducible run on the exact GPU candidate: text, streaming, authentication failures, shell/read calls, continuation, parallel calls, malformed arguments, context exhaustion, interruption, accounting, and privacy checks. The installed engine, model revision, parser, context settings, and tested client version belong to the evidence. A change to any relevant part can invalidate that conclusion.

Until that work passes, the productive path is a text integration within the current model envelope. Start with the API quickstart, review the model disclosure, and keep the live service checks separate from capability claims.

Primary references

• OpenAI Responses API reference for the transport contract.
• Codex configuration reference for custom provider settings.
• OpenCode provider documentation for provider configuration.
• Anthropic gateway documentation for supported routing boundaries.
• vLLM tool-calling documentation for parser requirements.