autoreg-skills-max: initial public release (restored, redacted, hardened)

This commit is contained in:
baradok60915d52e8425
2026-10-20 02:16:32 +03:00
commit f58eafeb48
565 changed files with 60064 additions and 0 deletions

View File

@@ -0,0 +1,365 @@
---
name: reseller-api-gateway
description: Build anonymous OpenAI-compatible API gateways for reselling — hide upstream provider, alias model IDs, strip metadata, apply speed optimizations (HTTP/2, connection pool, gzip, TCP_NODELAY), and deploy via immutable releases.
triggers:
- "resell API"
- "hide provider"
- "anonymous gateway"
- "opaque proxy"
- "api reseller"
- "model aliasing"
- "[REDACTED-NAME] API"
- "apioid gateway"
- "speed up API"
---
# Reseller API Gateway
Build an anonymous OpenAI-compatible API gateway that proxies an upstream OpenAI-compatible backend while hiding its identity — for reselling. Model IDs are aliased via HMAC, metadata is stripped, and speed optimizations eliminate unnecessary latency.
## Architecture
```
Client (any OpenAI client)
→ Nginx (TLS termination, HTTP/2, gzip, error masking)
→ Gateway (Python http.server on localhost, connection pool)
→ Upstream (echogate.one, OpenRouter, any /v1 backend)
```
## Core Patterns
### 1. Model ID Aliasing (HMAC-SHA256)
Every upstream model ID gets a deterministic public alias. Upstream never sees the alias; clients never see the real ID.
```python
import hmac, hashlib
def public_model_id(private_id: str, secret: str) -> str:
digest = hmac.new(secret.encode(), private_id.encode(), hashlib.sha256).hexdigest()
return f"rb-{digest[:20]}"
```
- Private IDs ONLY accepted on upstream requests (gateway looks up alias → real ID)
- Direct private IDs in client requests → HTTP 400
- Forbidden models filtered in catalog (e.g. `gpt-5-5` fragment match)
### 2. Metadata Sanitization
Strip all upstream metadata from responses:
```python
PRIVATE_METADATA_KEYS = {
"backend", "provider", "provider_id", "provider_name",
"route", "routing", "system_fingerprint",
"upstream", "upstream_id", "upstream_model", "upstream_provider",
}
```
- `owned_by` always `[REDACTED-NAME]` (or user's brand)
- `system_fingerprint` removed
- `model` field rewritten to public alias in both buffered and SSE responses
- All hop-by-hop headers stripped (`Connection`, `Keep-Alive`, etc.)
### 3. Connection Pool with Retry
Thread-safe HTTPS connection pool — eliminates TCP+TLS handshake per request:
```python
class ConnectionPool:
def __init__(self, host, port, scheme, timeout, max_idle=4):
self._lock = threading.Lock()
self._idle: list[http.client.HTTPConnection] = []
self._conn_type = HTTPSConnection if scheme == "https" else HTTPConnection
def _create(self):
conn = self._conn_type(host, port, timeout=timeout)
conn.connect()
conn.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
return conn
def get(self):
with self._lock:
while self._idle:
conn = self._idle.pop()
# select() health check: if socket readable, connection is dead
r, _, _ = select.select([conn.sock], [], [], 0)
if not r:
return conn
conn.close()
return self._create()
def put(self, conn):
with self._lock:
if len(self._idle) < self._max_idle and conn.sock is not None:
self._idle.append(conn)
return
conn.close()
```
**Retry pattern** — stale connections fail on first use, retry with fresh:
```python
def upstream_request(self, method, suffix, body):
for attempt in (1, 2):
conn = self._pool.get()
try:
conn.request(method, path, body=body, headers=headers)
response = conn.getresponse()
body = response.read()
if response.status < 500:
self._pool.put(conn) # healthy → return to pool
else:
conn.close() # 5xx → discard
return response.status, response.getheaders(), body
except (HTTPException, OSError, TimeoutError):
conn.close() # transport error → discard
if attempt == 2:
raise # retry exhausted
```
- Only `5xx` and transport errors discard connections; `4xx` connections go back to pool
- Pre-warm 2 connections + force catalog fetch on startup
### 4. Speed Optimizations (Nginx)
```nginx
server {
listen 443 ssl;
http2 on; # Multiplexing
gzip on;
gzip_min_length 256;
gzip_types application/json text/event-stream;
gzip_comp_level 1;
location / {
proxy_pass http://127.0.0.1:18181;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_read_timeout 600s;
proxy_request_buffering off;
proxy_buffering off;
}
# Mask ALL nginx/upstream errors as neutral JSON
error_page 400 408 411 414 431 = @invalid_request;
error_page 405 = @method_not_allowed;
error_page 500 502 503 504 = @service_unavailable;
}
```
### 5. Anti-Fingerprint
- **UA rotation**: rotate through `OpenAI/Python`, `Go-http-client/2.0`, `python-requests/2.32`, `curl/8.12` — no custom User-Agent
- **Server header**: stripped via `more_clear_headers Server;` or `server_tokens off;`
- **Error responses**: always neutral JSON, never upstream error bodies or HTML
- **Unknown routes**: 404 JSON, never nginx default HTML
- **Echo/legacy paths**: explicit 404 (e.g. `^~ /echo/` → JSON 404)
### 6. Immutable Release Deployment
Each deploy creates a timestamped release directory with symlink:
```bash
RELEASE_ID=$(date +%Y%m%d_%H%M%S)
mkdir -p /opt/apioid-echo/releases/${RELEASE_ID}
scp gateway.py root@host:/opt/apioid-echo/releases/${RELEASE_ID}/
ssh root@host "
ln -sfn /opt/apioid-echo/releases/${RELEASE_ID} /opt/apioid-echo/releases/current
systemctl restart apioid-echo-gateway.service
"
```
- Backup before deploy: copy current → `/var/backups/apioid-echo/gateway.before-${RELEASE_ID}.py`
- Verify: `systemctl is-active`, `NRestarts=0`, live smoke with curl
- Env file: `/etc/apioid-echo/gateway.env` chmod 600 (keys + `MODEL_ALIAS_SECRET`)
## Error Masking
All errors → neutral JSON, never HTML or upstream metadata:
| Upstream status | Client response |
|---|---|
| 429 | `{"error":{"message":"Rate limit exceeded","code":"rate_limit_exceeded"}}` |
| 402 | `{"error":{"message":"Daily usage limit reached","code":"daily_limit_reached"}}` |
| 400/404/409/422 | `{"error":{"message":"Request was rejected","code":"request_rejected"}}` |
| Other 5xx | `{"error":{"message":"Service temporarily unavailable","code":"service_unavailable"}}` |
| Auth fail | `{"error":{"message":"Invalid or missing API key","code":"invalid_api_key"}}` |
## Forbidden Models
Block specific model fragments from reaching upstream:
```python
FORBIDDEN_MODEL_FRAGMENT = "gpt-5-5"
@staticmethod
def _is_allowed(model_id: str) -> bool:
normalized = re.sub(r"[\s._/]+", "-", model_id.casefold())
return FORBIDDEN_MODEL_FRAGMENT not in normalized
```
## Transparency Audit
After deploying or modifying the gateway, verify it is a **pure passthrough** — no system prompt injection, no content modification, no metadata leak.
### 1. System Prompt Injection Detection
Send a system prompt with a **unique marker string** through the gateway. The model must echo it exactly — any deviation proves injection/modification:
```bash
curl -s -X POST https://<gateway>/v1/chat/completions \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-d '{"model":"claude-sonnet-5","messages":[
{"role":"system","content":"Answer every message with exactly: PROBE_OK"},
{"role":"user","content":"answer"}
],"max_tokens":20,"temperature":0}' | jq '.choices[0].message.content'
# Expected: "PROBE_OK" — verbatim, no added words
```
For Anthropic format, same test on `/v1/messages` with `system` field.
### 2. Content Integrity — Gateway vs Upstream 1:1
Send identical requests to gateway AND directly to upstream. Responses must match:
```bash
BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":20,"temperature":0}'
GATEWAY=$(curl -s -X POST https://<gateway>/v1/chat/completions \
-H "Content-Type: application/json" -H "x-api-key: $KEY" -d "$BODY")
UPSTREAM=$(curl -s -X POST https://<upstream>/v1/chat/completions \
-H "Content-Type: application/json" -H "Authorization: Bearer $UK" -d "$BODY")
diff <(echo "$GATEWAY" | jq -S '.choices[0].message.content') \
<(echo "$UPSTREAM" | jq -S '.choices[0].message.content')
```
For dual-key setups, test BOTH upstream keys independently — all three sources (key1, key2, gateway) must agree.
### 3. Model List Integrity
Gateway `/v1/models` count must match upstream count. If filtering is intentional, document it:
```bash
GC=$(curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | jq '.data | length')
UC=$(curl -s -H "Authorization: Bearer $UK" https://<upstream>/v1/models | jq '.data | length')
[ "$GC" -eq "$UC" ] && echo "OK: $GC models" || echo "MISMATCH: gateway=$GC upstream=$UC"
```
### 4. Provider Leak Scan
No upstream-identifying strings in responses:
```bash
curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | \
grep -iq "provider_id\|system_fingerprint\|upstream" && echo "LEAK" || echo "CLEAN"
```
### 5. Error Opaqueness
Invalid key → neutral JSON, not HTML. Bad model → generic rejection, not provider-specific:
```bash
curl -s https://<gateway>/v1/models | jq '.error.code' # "invalid_api_key"
curl -s -H "x-api-key: $KEY" -H "Content-Type: application/json" \
-d '{"model":"nonexistent","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' \
https://<gateway>/v1/chat/completions | jq '.error.type' # "invalid_request_error"
```
## References
- `references/deployment-commands.md` — immutable release deployment workflow + live smoke
- `references/nginx-config.md` — production nginx config with HTTP/2, gzip, error masking
- `references/transparency-audit.md` — full audit checklist with probe scripts
## Verification
```bash
# Health (no auth)
curl -sk https://api.[REDACTED-NAME].loc.cc/health
# Models (auth required)
curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models
# Verify: owned_by=[REDACTED-NAME], no provider/system_fingerprint
# Chat (warm pool — should be <3s)
time curl -sk -H "Authorization: Bearer $KEY" \
-H 'Content-Type: application/json' \
-d '{"model":"rb-902d95ce37cee7204c89","messages":[{"role":"user","content":"hi"}],"max_tokens":3}' \
https://api.[REDACTED-NAME].loc.cc/v1/chat/completions
# Verify: model=rb-*, fp=ABSENT, content present
# HTTP/2
curl -skI --http2 https://api.[REDACTED-NAME].loc.cc/health 2>&1 | head -1
# Expected: HTTP/2 200
# Provider leak test
curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models \
| python3 -c "import json,sys; d=json.load(sys.stdin); print(any('provider' in m or 'system_fingerprint' in m for m in d['data']))"
# Expected: False
```
## Anthropic Direct Proxying (Multimodal)
When the upstream supports Anthropic `/v1/messages` natively, forward Anthropic requests **directly** — do NOT convert Anthropic→OpenAI→Anthropic. The conversion round-trip strips multimodal content (images, files, PDFs).
```python
# do_POST — route Anthropic natively
is_anthropic = path == "/v1/messages"
# NO _anthropic_to_openai conversion — forward as-is
suffix = "messages" if is_anthropic else path.removeprefix("/v1/")
# NO _openai_to_anthropic response transform
# sanitize_json_body handles model name replacement for both formats
```
The response sanitizer (`sanitize_json_body`) already handles both OpenAI and Anthropic response structures — it recursively replaces the `model` field and strips `PRIVATE_METADATA_KEYS`.
## Single-Model Endpoint
OpenAI clients expect `GET /v1/models/{id}` to return a single model object. Add after the `/v1/models` list handler:
```python
m = re.match(r"^/v1/models/(.+)$", path)
if m:
model_id = m.group(1)
if model_id not in self.gateway.catalog.allowed_ids():
self._error(404, "Model not found", "model_not_found", request_id)
return
self._send_json(200, {"id": model_id, "object": "model", "owned_by": "[REDACTED-NAME]"}, request_id=request_id)
return
```
## Transparency Audit Addendum: Multimodal
After the standard text-only audit, verify multimodal passthrough. Use a 1×1 red PNG (`iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==`):
```bash
IMG="iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
# Anthropic vision (gateway must forward image blocks)
curl -s -X POST https://<gateway>/v1/messages \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-H "anthropic-version: 2023-06-01" \
-d "{\"model\":\"claude-sonnet-5\",\"messages\":[{\"role\":\"user\",\"content\":[
{\"type\":\"text\",\"text\":\"What color is this image?\"},
{\"type\":\"image\",\"source\":{\"type\":\"base64\",\"media_type\":\"image/png\",\"data\":\"$IMG\"}}
]}],\"max_tokens\":50}"
# If the model says \"I don't see an image\" — image was stripped during conversion
# Fix: forward Anthropic directly, do not convert to OpenAI
```
## Pitfalls
1. **Stale connection hang**: pooled HTTPS connections die silently (upstream FIN). Health-check with `select.select(sock, [], [], 0)` — if readable with 0 timeout, connection is dead. Always retry on transport error.
2. **TCP_NODELAY placement**: must be called AFTER `conn.connect()` and before first `request()`. On http.client, set `conn.sock.setsockopt(IPPROTO_TCP, TCP_NODELAY, 1)`.
3. **HTTP/2 on nginx 1.28**: use `http2 on;` directive, NOT `listen ... http2` (deprecated, generates warnings).
4. **Gzip on SSE**: `gzip_types text/event-stream` is correct. SSE is text, gzip compresses both JSON and SSE.
5. **Catalog pre-warm failure**: wrap in try/except on startup, log warning. Catalog retries on first authenticated request.
6. **`_jitter()` removal**: if user says "as fast as possible", remove ALL jitter — TTL jitter AND request jitter. Keep only functional variation (UA rotation).
7. **Stream connections**: never return SSE connections to pool — they're live for the stream duration. Only buffered request connections go back.
8. **Reasoning models (Claude Fable 5, etc.)**: these consume tokens for internal reasoning before producing visible output. If `max_tokens` is too low (e.g. 10), ALL tokens go to thinking and `content` comes back `null` with `finish_reason: "length"`. Always use `max_tokens >= 50` when probing reasoning models, and check `usage.completion_tokens_details.reasoning_tokens` to confirm thinking consumption.
9. **Dual-key verification**: when gateway uses multiple upstream keys, test EACH key directly against the upstream before testing through the gateway. All keys + gateway must return structurally identical responses. A mismatch between keys may indicate per-key model availability differences or rate-limit state.
10. **Anthropic→OpenAI conversion strips multimodal**: the `_anthropic_to_openai` converter joins text blocks from content arrays but discards image/file blocks. If the upstream supports Anthropic natively, forward `/v1/messages` requests directly. Only use the converter when the upstream ONLY speaks OpenAI.
11. **base64 newlines break curl**: when piping base64 from a file, `cat file.txt` includes a trailing newline that invalidates the base64 in JSON. Use `tr -d '\n\r'` to strip.

View File

@@ -0,0 +1,66 @@
# Deployment Commands
Immutable release deployment for [REDACTED-NAME] API gateway on VPS ([REDACTED-IP]).
## Release ID convention
```bash
RELEASE_ID=$(date +%Y%m%d_%H%M%S)
# Example: 20260804_142027
```
## Deploy workflow
```bash
set -eu
RELEASE_ID=20260804_142027
SHA=$(sha256sum gateway.py | cut -d' ' -f1)
ssh root@host "mkdir -p /opt/apioid-echo/releases/${RELEASE_ID}"
scp gateway.py root@host:/opt/apioid-echo/releases/${RELEASE_ID}/gateway.py
scp [REDACTED-NAME].nginx.conf root@host:/opt/apioid-echo/releases/${RELEASE_ID}/
ssh root@host "
R='${RELEASE_ID}'
# backup
cp /opt/apioid-echo/releases/current/gateway.py /var/backups/apioid-echo/gateway.before-\${R}.py
cp /etc/nginx/sites-available/[REDACTED-NAME] /var/backups/apioid-echo/nginx.before-\${R}.conf
# deploy
ln -sfn /opt/apioid-echo/releases/\${R} /opt/apioid-echo/releases/current
chown root:root /opt/apioid-echo/releases/\${R}/*
chmod 0644 /opt/apioid-echo/releases/\${R}/gateway.py
cp /opt/apioid-echo/releases/\${R}/[REDACTED-NAME].nginx.conf /etc/nginx/sites-available/[REDACTED-NAME]
nginx -t && systemctl reload nginx
systemctl restart apioid-echo-gateway.service
sleep 1.5
systemctl is-active apioid-echo-gateway.service
systemctl show -p NRestarts apioid-echo-gateway.service
"
```
## Live smoke
```bash
ssh root@host "
KEY='rfb_KEY_HERE'
# health
curl -sk --noproxy '*' --resolve api.[REDACTED-NAME].loc.cc:443:127.0.0.1 https://api.[REDACTED-NAME].loc.cc/health
# HTTP/2 + chat
curl -skI --resolve api.[REDACTED-NAME].loc.cc:443:127.0.0.1 https://api.[REDACTED-NAME].loc.cc/health 2>&1 | head -1
time curl -sk -H \"Authorization: Bearer \${KEY}\" \
-H 'Content-Type: application/json' \
-d '{\"model\":\"rb-902d95ce37cee7204c89\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}],\"max_tokens\":3}' \
https://api.[REDACTED-NAME].loc.cc/v1/chat/completions
"
```
## VPS paths
| Path | Purpose |
|------|---------|
| `/opt/apioid-echo/releases/current/` | Active release (symlink) |
| `/opt/apioid-echo/releases/<TS>/` | Immutable release directory |
| `/etc/apioid-echo/gateway.env` | Env (keys, MODEL_ALIAS_SECRET), chmod 600 |
| `/etc/nginx/sites-available/[REDACTED-NAME]` | Nginx config |
| `/var/backups/apioid-echo/` | Pre-deploy backups |
| `/etc/systemd/system/apioid-echo-gateway.service` | systemd unit |

View File

@@ -0,0 +1,124 @@
# Nginx Config — API Gateway (HTTP/2 + Gzip + Error Masking)
Production config for `api.<domain>` on nginx 1.28.3.
## Server block (SSL + HTTP/2)
```nginx
server {
listen 443 ssl;
listen [::]:443 ssl;
http2 on;
server_name api.[REDACTED-NAME].loc.cc;
server_tokens off;
ssl_certificate /etc/letsencrypt/live/[REDACTED-NAME].loc.cc/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/[REDACTED-NAME].loc.cc/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
client_max_body_size 25m;
gzip on;
gzip_min_length 256;
gzip_types application/json text/event-stream;
gzip_comp_level 1;
gzip_vary on;
location / {
proxy_pass http://[REDACTED-NAME];
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-ID $request_id;
proxy_set_header Connection "";
proxy_connect_timeout 10s;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
proxy_request_buffering off;
proxy_buffering off;
proxy_cache off;
proxy_socket_keepalive on;
proxy_intercept_errors on;
}
# Error masking: ALL upstream/nginx errors → neutral JSON
error_page 400 408 411 414 431 = @invalid_request;
error_page 405 = @method_not_allowed;
error_page 413 = @request_too_large;
error_page 500 502 503 504 = @service_unavailable;
location @invalid_request {
internal;
default_type application/json;
return 400 '{"error":{"message":"Invalid request","type":"gateway_error","code":"invalid_request"}}';
}
location @method_not_allowed {
internal;
default_type application/json;
return 405 '{"error":{"message":"Method not allowed","type":"gateway_error","code":"method_not_allowed"}}';
}
location @request_too_large {
internal;
default_type application/json;
return 413 '{"error":{"message":"Request body is too large","type":"gateway_error","code":"request_too_large"}}';
}
location @service_unavailable {
internal;
default_type application/json;
return 502 '{"error":{"message":"Service temporarily unavailable","type":"gateway_error","code":"service_unavailable"}}';
}
}
```
## Upstream block
```nginx
upstream [REDACTED-NAME] {
zone [REDACTED-NAME] 64k;
server 127.0.0.1:18181;
keepalive 64;
}
```
## HTTP → HTTPS redirect
```nginx
server {
listen 80;
listen [::]:80;
server_name [REDACTED-NAME].loc.cc www.[REDACTED-NAME].loc.cc api.[REDACTED-NAME].loc.cc;
return 301 https://$host$request_uri;
}
```
## Echo/legacy paths (block on old domain)
```nginx
server {
listen 443 ssl;
server_name [REDACTED-NAME].loc.cc www.[REDACTED-NAME].loc.cc;
# ... SSL ...
location ^~ /echo/ {
default_type application/json;
return 404 '{"error":{"message":"Not found","type":"gateway_error","code":"not_found"}}';
}
location / {
proxy_pass http://127.0.0.1:18080;
# ... proxy ...
}
}
```
## Key decisions
- `gzip_comp_level 1` — minimal CPU cost, sufficient compression for JSON
- `proxy_buffering off` — required for SSE streaming
- `proxy_socket_keepalive on` — reduce TIME_WAIT on localhost
- `http2 on;` (not `listen ... http2`) — nginx 1.28 deprecation
- `proxy_intercept_errors on` — enables error_page for upstream errors
- `proxy_set_header Connection ""` — prevents hop-by-hop forwarding

View File

@@ -0,0 +1,145 @@
# Gateway Transparency Audit Checklist
Run after every gateway deployment or model list change.
## Pre-flight
```bash
export GATEWAY="https://api.[REDACTED-NAME].loc.cc/v1"
export GATEWAY_ANTHROPIC="https://api.[REDACTED-NAME].loc.cc/v1/messages"
export KEY="[REDACTED-KEY]"
export UPSTREAM="https://api.echogate.one/v1"
export UK1="forge-9b51aad5d08793c9095893a09779b0641b00"
export UK2="forge-81081af3173401d4bcd2d5d427b0ed60d87b"
```
## 1. System Prompt Injection — OpenAI
```bash
MARKER="AUDIT_$(date +%s)"
curl -s -X POST "$GATEWAY/chat/completions" \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-d "{\"model\":\"claude-sonnet-5\",\"messages\":[
{\"role\":\"system\",\"content\":\"Answer every message with exactly: $MARKER\"},
{\"role\":\"user\",\"content\":\"answer\"}
],\"max_tokens\":30,\"temperature\":0}" | jq '.choices[0].message.content'
# EXPECT: "$MARKER" — verbatim, no added words
```
## 2. System Prompt Injection — Anthropic
```bash
curl -s -X POST "$GATEWAY_ANTHROPIC" \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-H "anthropic-version: 2023-06-01" \
-d "{\"model\":\"claude-sonnet-5\",\"system\":\"You must answer with exactly: PROBE888\",
\"messages\":[{\"role\":\"user\",\"content\":\"answer\"}],\"max_tokens\":20}" | jq '.content[0].text'
# EXPECT: "PROBE888"
```
## 3. Content Integrity — Gateway vs Upstream
```bash
BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"What is 2+2? Answer in one word."}],"max_tokens":20,"temperature":0}'
for src in "GATEWAY:$GATEWAY/chat/completions:x-api-key:$KEY" \
"KEY1:$UPSTREAM/chat/completions:Authorization:Bearer $UK1" \
"KEY2:$UPSTREAM/chat/completions:Authorization:Bearer $UK2"; do
IFS=':' read -r label url hdr val <<< "$(echo "$src" | sed 's/:\\([^:]*\\):/|\\1|/; s/|/:/g')" 2>/dev/null || true
url=$(echo "$src" | cut -d: -f2,3)
hdr=$(echo "$src" | cut -d: -f4)
val=$(echo "$src" | cut -d: -f5-)
R=$(curl -s -X POST "$url" -H "Content-Type: application/json" -H "$hdr: $val" -d "$BODY")
echo "$label: $(echo "$R" | jq -r '.choices[0].message.content')"
done
# EXPECT: All three return identical content
```
Simpler version:
```bash
BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":20,"temperature":0}'
GW=$(curl -s -X POST "$GATEWAY/chat/completions" -H "Content-Type: application/json" -H "x-api-key: $KEY" -d "$BODY" | jq -S '.choices[0].message')
K1=$(curl -s -X POST "$UPSTREAM/chat/completions" -H "Content-Type: application/json" -H "Authorization: Bearer $UK1" -d "$BODY" | jq -S '.choices[0].message')
[ "$GW" = "$K1" ] && echo "OK: gateway == key1" || echo "MISMATCH"
```
## 4. No-System-Message Test
Verify gateway doesn't inject its own system prompt when none is sent:
```bash
curl -s -X POST "$GATEWAY/chat/completions" \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"What is 2+2? Answer in one word."}],"max_tokens":20,"temperature":0}' | jq '.choices[0].message.content'
# EXPECT: natural response like "Four" — no injected behavior, no prefix
```
## 5. Model List Integrity
```bash
GW_COUNT=$(curl -s -H "x-api-key: $KEY" "$GATEWAY/models" | jq '.data | length')
UP_COUNT=$(curl -s -H "Authorization: Bearer $UK1" "$UPSTREAM/models" | jq '.data | length')
echo "Gateway: $GW_COUNT, Upstream: $UP_COUNT"
[ "$GW_COUNT" -eq "$UP_COUNT" ] && echo "PASS" || echo "FAIL — check FORBIDDEN_MODEL_FRAGMENTS"
```
## 6. Provider Leak Scan
```bash
curl -s -H "x-api-key: $KEY" "$GATEWAY/models" | grep -iq \
"provider_id\|system_fingerprint\|upstream\|echogate" && echo "LEAK DETECTED" || echo "CLEAN"
```
## 7. Error Opaqueness
```bash
# No key → clean 401
echo -n "No key: "
curl -so /dev/null -w "%{http_code}" "$GATEWAY/models"
echo ""
echo -n "No key body: "
curl -s "$GATEWAY/models" | jq '.error.code'
# Bad model → clean 400
echo -n "Bad model: "
curl -s -X POST "$GATEWAY/chat/completions" \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-d '{"model":"nonexistent-model","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' | jq '.error.type'
# Bad key → 401
echo -n "Bad key: "
curl -so /dev/null -w "%{http_code}" -H "x-api-key: [REDACTED]" "$GATEWAY/models"
```
## 8. Reasoning Model Probe
When testing reasoning models (fable-5, opus-5-sonnet, etc.), use enough `max_tokens`:
```bash
# WRONG — all tokens consumed by thinking, content=null
curl -s -X POST "$GATEWAY/chat/completions" \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-d '{"model":"claude-fable-5","messages":[{"role":"user","content":"Say hi"}],"max_tokens":10}' | jq '{content: .choices[0].message.content, reason: .usage.completion_tokens_details.reasoning_tokens}'
# RIGHT — enough tokens for thinking + output
curl -s -X POST "$GATEWAY/chat/completions" \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-d '{"model":"claude-fable-5","messages":[{"role":"user","content":"Say hi"}],"max_tokens":200}' | jq '.choices[0].message.content'
```
## Quick Smoke (all-in-one)
```bash
# Health
curl -sf "$GATEWAY/../health" | jq '.status' || echo "HEALTH FAIL"
# Models count
echo -n "Models: "
curl -s -H "x-api-key: $KEY" "$GATEWAY/models" | jq '.data | length'
# Chat probe
echo -n "Chat: "
curl -s -X POST "$GATEWAY/chat/completions" \
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
-d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":3}' | jq -r '.choices[0].message.content'
```