autoreg-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
72
skills/conol-automation/references/model-downgrades.md
Normal file
72
skills/conol-automation/references/model-downgrades.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Conol.ai Model Downgrade Behavior
|
||||
|
||||
## Discovery Session: 2026-07-30
|
||||
|
||||
After a ~4-hour outage (all models returning HTTP 500 `"Failed to create session"`), Conol.ai backend recovered but with GPT-5.5 unavailable. Sessions that requested `gpt-5.5` were silently downgraded.
|
||||
|
||||
### Raw Observations
|
||||
|
||||
**Payload sent:**
|
||||
```json
|
||||
{
|
||||
"source": {"type": "home"},
|
||||
"messages": [{"type": "text", "content": "Say OK"}],
|
||||
"timezone": "Europe/Kyiv",
|
||||
"agentModel": "gpt-5.5",
|
||||
"agentEffort": "low"
|
||||
}
|
||||
```
|
||||
|
||||
**Session created (HTTP 201):**
|
||||
```json
|
||||
{
|
||||
"sessionId": "j0c8y5kosm47bd1nsf2zlmrl",
|
||||
"modelDowngraded": true,
|
||||
"effectiveModel": "claude-haiku-4-5"
|
||||
}
|
||||
```
|
||||
|
||||
**Full chat response:** Model still answered "OK" correctly — the downgrade is transparent to the caller.
|
||||
|
||||
### Test Matrix (6 payload variants, all HTTP 201)
|
||||
|
||||
| Variant | Status | modelDowngraded | effectiveModel |
|
||||
|---------|--------|-----------------|----------------|
|
||||
| full (tz + effort) | 201 | true | claude-haiku-4-5 |
|
||||
| no timezone | 201 | true | claude-haiku-4-5 |
|
||||
| bare (model only) | 201 | true | claude-haiku-4-5 |
|
||||
| no model specified | 201 | true | claude-haiku-4-5 |
|
||||
| no effort | 201 | true | claude-haiku-4-5 |
|
||||
| source: direct | 201 | true | claude-haiku-4-5 |
|
||||
|
||||
**Key insight:** Even when NO model is specified (bare `source + messages`), Conol.ai downgrades to haiku. This means the downgrade is based on backend capacity, not the specific model requested.
|
||||
|
||||
### Gateway Transparency
|
||||
|
||||
The gateway (`gateway.py`) does NOT expose `modelDowngraded` or `effectiveModel` to the caller. The `/v1/chat/completions` response shows the original model name:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "gpt-5.5",
|
||||
"choices": [{"message": {"content": "OK"}}]
|
||||
}
|
||||
```
|
||||
|
||||
The caller sees `model: "gpt-5.5"` even though haiku generated the response. This is acceptable — the alternative (returning actual model) would break downstream OMP provider matching.
|
||||
|
||||
### Distinguishing Downgrade from Outage
|
||||
|
||||
| Signal | Downgrade | Outage |
|
||||
|--------|-----------|--------|
|
||||
| HTTP status | 201 | 500 |
|
||||
| `sessionId` | present | absent |
|
||||
| `modelDowngraded` | `true` | N/A (no response) |
|
||||
| `effectiveModel` | fallback model | N/A |
|
||||
| Chat works | ✅ Yes | ❌ No |
|
||||
| Action needed | None (transparent) | Wait + daemon auto-recovers |
|
||||
|
||||
### When This Matters
|
||||
|
||||
- **Don't panic on `modelDowngraded: true`.** It means the system works, just with a cheaper model.
|
||||
- **Check `modelDowngraded` before assuming server-down.** If you test one account, get 201, and see `modelDowngraded: true` — the server is alive, only the model is downgraded.
|
||||
- **The daemon correctly ignores downgrades.** It only triggers `server_down` on HTTP 500, not on `modelDowngraded`.
|
||||
Reference in New Issue
Block a user