73 lines
2.6 KiB
Markdown
73 lines
2.6 KiB
Markdown
# Conol.ai Model Downgrade Behavior
|
|
|
|
## Discovery Session: 2026-07-30
|
|
|
|
After a ~4-hour outage (all models returning HTTP 500 `"Failed to create session"`), Conol.ai backend recovered but with GPT-5.5 unavailable. Sessions that requested `gpt-5.5` were silently downgraded.
|
|
|
|
### Raw Observations
|
|
|
|
**Payload sent:**
|
|
```json
|
|
{
|
|
"source": {"type": "home"},
|
|
"messages": [{"type": "text", "content": "Say OK"}],
|
|
"timezone": "Europe/Kyiv",
|
|
"agentModel": "gpt-5.5",
|
|
"agentEffort": "low"
|
|
}
|
|
```
|
|
|
|
**Session created (HTTP 201):**
|
|
```json
|
|
{
|
|
"sessionId": "j0c8y5kosm47bd1nsf2zlmrl",
|
|
"modelDowngraded": true,
|
|
"effectiveModel": "claude-haiku-4-5"
|
|
}
|
|
```
|
|
|
|
**Full chat response:** Model still answered "OK" correctly — the downgrade is transparent to the caller.
|
|
|
|
### Test Matrix (6 payload variants, all HTTP 201)
|
|
|
|
| Variant | Status | modelDowngraded | effectiveModel |
|
|
|---------|--------|-----------------|----------------|
|
|
| full (tz + effort) | 201 | true | claude-haiku-4-5 |
|
|
| no timezone | 201 | true | claude-haiku-4-5 |
|
|
| bare (model only) | 201 | true | claude-haiku-4-5 |
|
|
| no model specified | 201 | true | claude-haiku-4-5 |
|
|
| no effort | 201 | true | claude-haiku-4-5 |
|
|
| source: direct | 201 | true | claude-haiku-4-5 |
|
|
|
|
**Key insight:** Even when NO model is specified (bare `source + messages`), Conol.ai downgrades to haiku. This means the downgrade is based on backend capacity, not the specific model requested.
|
|
|
|
### Gateway Transparency
|
|
|
|
The gateway (`gateway.py`) does NOT expose `modelDowngraded` or `effectiveModel` to the caller. The `/v1/chat/completions` response shows the original model name:
|
|
|
|
```json
|
|
{
|
|
"model": "gpt-5.5",
|
|
"choices": [{"message": {"content": "OK"}}]
|
|
}
|
|
```
|
|
|
|
The caller sees `model: "gpt-5.5"` even though haiku generated the response. This is acceptable — the alternative (returning actual model) would break downstream OMP provider matching.
|
|
|
|
### Distinguishing Downgrade from Outage
|
|
|
|
| Signal | Downgrade | Outage |
|
|
|--------|-----------|--------|
|
|
| HTTP status | 201 | 500 |
|
|
| `sessionId` | present | absent |
|
|
| `modelDowngraded` | `true` | N/A (no response) |
|
|
| `effectiveModel` | fallback model | N/A |
|
|
| Chat works | ✅ Yes | ❌ No |
|
|
| Action needed | None (transparent) | Wait + daemon auto-recovers |
|
|
|
|
### When This Matters
|
|
|
|
- **Don't panic on `modelDowngraded: true`.** It means the system works, just with a cheaper model.
|
|
- **Check `modelDowngraded` before assuming server-down.** If you test one account, get 201, and see `modelDowngraded: true` — the server is alive, only the model is downgraded.
|
|
- **The daemon correctly ignores downgrades.** It only triggers `server_down` on HTTP 500, not on `modelDowngraded`.
|