# OpenAI-Compatible Provider Capability Probe

Use this for a read-only audit of an unfamiliar OpenAI-compatible LLM provider when the user wants the base URL, `/models`, free models, exact model IDs, and a proven chat completion.

## Read-only workflow

1. **Discover documentation without guessing paths.** Probe the host root, `/docs`, and `/openapi.json`. Prefer the OpenAPI document as route evidence. A vendor UI at `/models` may be HTML; the API route is commonly `/v1/models`.
2. **Establish the OpenAI base URL.** If OpenAPI exposes `/v1/chat/completions`, report the base as `https://HOST/v1`, not the full completion endpoint.
3. **Use the user-authorized key from context without displaying it.** Keep it in memory/environment, send it as `Authorization: Bearer ...`, and redact command output. Never persist the secret in a report, reference, log, or long-term memory.
4. **Query `/v1/models` with the key.** Record HTTP status, content type, total models, exact IDs, and owner metadata. Treat IDs returned by this live endpoint as authoritative for that key at that time.
5. **Classify catalog-free models conservatively.** A `-free` suffix proves catalog designation, not current usability or remaining quota. Say “catalog-designated free” until completion-tested.
6. **Completion-test representative models.** POST `/v1/chat/completions` with a deterministic prompt such as `Reply exactly PROVIDER_OK`, `stream:false`, and a realistic token budget.
7. **Validate semantic success, not only HTTP 200.** Require a non-empty assistant answer matching the task. Inspect `choices[0].message.content`, `finish_reason`, returned model, and usage. Some gateways return quota/top-up notices as successful `chat.completion` objects with HTTP 200 and zero token usage.
8. **Handle reasoning models correctly.** A reasoning model can consume a tiny `max_tokens` budget on hidden reasoning and return `content:null`. Retry once with 128–256 tokens before classifying it as broken.
9. **Report three separate sets:** (a) IDs advertised by `/models`, (b) catalog-designated free IDs, and (c) models proven live on the current key. Do not generalize from one live model to all free models.
10. **Leave the project untouched.** Use one-off in-memory probes or an existing external scratch script. Do not create or modify project files when the user requested research only.

## Minimal probe shape

```python
from openai import OpenAI

client = OpenAI(api_key=KEY, base_url="https://HOST/v1")
models = client.models.list()
reply = client.chat.completions.create(
    model="EXACT_ID_FROM_MODELS",
    messages=[{"role": "user", "content": "Reply exactly PROVIDER_OK"}],
    temperature=0,
    max_tokens=128,
)
assert reply.choices[0].message.content == "PROVIDER_OK"
```

## Status taxonomy

- **Live:** HTTP 200, meaningful non-empty assistant content, plausible usage.
- **Reasoning-budget retry needed:** HTTP 200, `content:null`, completion/reasoning tokens consumed, tiny token cap.
- **Quota-gated:** HTTP 200 or 4xx with top-up/quota message; often zero usage.
- **Auth failure:** 401/403 attributable to key authorization.
- **Wrong route:** HTML content or UI shell where JSON was expected.
- **Advertised only:** present in `/models`, not completion-tested.

## Inferera observation (2026-07-29)

- OpenAI base: `https://api.inferera.com/v1`
- API models route: `GET /v1/models`; `/models` is a web UI.
- OpenAPI route evidence: `/openapi.json` describes `/v1/chat/completions`.
- The live catalog contained hundreds of models and dozens of `-free` IDs; exact counts are time-sensitive and must be refreshed.
- `ling-3.0-flash-free` produced a real completion after increasing `max_tokens` from 32 to 128; the response mapped it to an upstream InclusionAI model ID.
- Several other `-free` IDs returned a top-up/free-trial exhaustion message inside HTTP 200 `chat.completion` responses with zero usage. Therefore, never equate `-free`, presence in `/models`, or HTTP 200 with usable inference.