Privacy-first OpenAI-compatible inference, without vague promises.

AblitAI is in launch validation. This article describes the application design and launch configuration being tested; it is not a claim that production validation is complete.

“OpenAI-compatible” describes an interface, not a privacy policy

AblitAI exposes the familiar Chat Completions pattern: a client sends a model identifier and an ordered list of messages to /v1/chat/completions, authenticates with a bearer key, and receives either one JSON response or server-sent streaming events. The open-source vLLM server likewise describes its HTTP service as implementing OpenAI-compatible Completions and Chat APIs.

Compatibility is deliberately narrower than equivalence. AblitAI is not operated by OpenAI, and the label does not promise support for every field, modality, model, or client behavior in another provider's API. Each capability still needs its own documentation and validation. What compatibility does provide is a stable request shape that many developers already understand, without tying the privacy boundary to a proprietary client library.

The request path has four distinct jobs

1. The client sends the API key and request content over HTTPS to the public gateway. The key is a credential and should be handled like a password.
2. The gateway authenticates the key, validates the model and request limits, and reserves enough account credit for the request. It does not need to turn prompt text into an account record to perform those jobs.
3. The gateway relays the validated request to a private inference listener. The model server tokenizes the input and holds working state in memory while generation is active.
4. The gateway returns the response or streaming deltas, then settles the reservation from the inference server's reported token usage and releases any unused reservation.

That path is intentionally short, but “transient” does not mean the content never exists. Prompt and response data must be present in process memory and network buffers long enough to compute and deliver the answer. The accurate boundary is about intentional persistence, not about pretending processing can occur without handling the data.

What the application is designed not to retain

AblitAI's application does not intentionally write prompt or generated-response content to its account database, analytics, operational backups, or application request records. The launch configuration also keeps inference request and output logging disabled and disables HTTP access logging on the model listener. These are specific configuration choices, not a universal property of open-weight models or OpenAI-compatible servers.

Logging controls matter because inference software can be configured differently. vLLM's own request-logger implementation can record prompts, token IDs, generated text, and finish reasons when detailed request and output logging are enabled. A privacy-first deployment therefore has to inspect the actual flags and logging pipeline instead of relying on a product category such as “self-hosted.”

What the account and billing system does retain

A usable paid API still needs durable state. AblitAI keeps a random account number, a cryptographic hash of the API key, credit balances, temporary reservations, usage ledger entries, and timestamps. If checkout is enabled after validation, accounting records can also include opaque payment-provider references, pack snapshots, status, declared country, and reconciliation events. Those records support authentication, metering, replay defense, and accounting; they are not copies of prompts.

The server returns a new plaintext API key once. The browser can store that key and the account number locally, and an offline backup can contain both. Anyone with access to that browser profile or backup may be able to use the key. Conversely, an account number alone cannot reconstruct the key. That recovery tradeoff is part of the privacy model, not an implementation footnote.

Why the inference server stays behind the gateway

The launch design binds the inference service to a loopback interface and reaches it through a restricted private path. The public gateway is the authentication, validation, rate-limit, and billing boundary. This separation is also a security control: vLLM's documentation warns that its built-in API-key option protects selected path prefixes rather than every endpoint, and recommends a reverse proxy or equivalent hardening. Exposing the raw model server would bypass assumptions the gateway is built to enforce.

The metadata and infrastructure boundary remains real

Privacy-first does not mean anonymous. DNS, content-delivery, hosting, networking, and future payment systems can process operational metadata under their own terms. Network systems necessarily observe facts such as endpoints, timing, byte volume, and error status. Security controls can minimize exposure and retention, but they cannot make a network request leave no operational trace anywhere in the stack.

vLLM also documents anonymous usage and configuration statistics as a separate default behavior. Those statistics are not prompt bodies, but they are still an external metadata path. The production gate therefore has to opt out explicitly and verify egress; disabling per-request statistics and content logging is not the same control as disabling upstream usage reporting.

Client devices are another boundary. Local shell history, application telemetry, crash reports, browser storage, proxies, and corporate monitoring can retain content independently of AblitAI's application. Developers handling sensitive data should review the complete path from the originating application to the model response, not only the API provider's database.

Portability still requires feature-by-feature testing

A typical integration changes three values: the base URL, the bearer API key, and the stable model ID qwen3.8-27b-abliterated. Text Chat Completions and server-sent streaming are the public baseline described in the current API documentation. Additional protocol features remain subject to launch validation and should not be inferred from the compatibility label alone.

This distinction improves portability rather than weakening it. A client can test the exact operations it uses, fail clearly on unsupported input, and keep provider-specific behavior out of the rest of the application. Silent acceptance of an unsupported field would be less compatible than a precise error because it could change the meaning of a request.

How the retention boundary becomes testable

The release plan calls for sentinel request text and then requires inspection of application logs, inference logs, the account database, and backup contents for that sentinel. It also requires checks that raw API keys and raw client IP fields are absent from durable account storage, that the inference listener is not publicly reachable, and that interrupted streams release reservations. Those checks turn an architectural intention into evidence about a particular deployed revision.

Evidence can expire. Changing the gateway, model server, logging configuration, backup path, or hosting topology requires the affected checks to run again. Until that production validation is complete, the honest status is launch validation—not a certification and not an absolute promise about every infrastructure participant.

Primary references and current disclosures

• vLLM OpenAI-compatible server documentation explains the API surface and its authentication boundary.
• vLLM's request-logger source shows which request and output fields can be logged when those features are enabled.
• vLLM's usage-statistics disclosure documents its separate anonymous telemetry behavior and opt-out controls.
• AblitAI's Privacy notice is the controlling public disclosure for retained data categories and third-party processing boundaries.
• The model disclosure records the exact checkpoint, serving envelope, and launch-validation status.