autoreg-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
365
skills/reseller-api-gateway/SKILL.md
Normal file
365
skills/reseller-api-gateway/SKILL.md
Normal file
@@ -0,0 +1,365 @@
|
||||
---
|
||||
name: reseller-api-gateway
|
||||
description: Build anonymous OpenAI-compatible API gateways for reselling — hide upstream provider, alias model IDs, strip metadata, apply speed optimizations (HTTP/2, connection pool, gzip, TCP_NODELAY), and deploy via immutable releases.
|
||||
triggers:
|
||||
- "resell API"
|
||||
- "hide provider"
|
||||
- "anonymous gateway"
|
||||
- "opaque proxy"
|
||||
- "api reseller"
|
||||
- "model aliasing"
|
||||
- "[REDACTED-NAME] API"
|
||||
- "apioid gateway"
|
||||
- "speed up API"
|
||||
---
|
||||
|
||||
# Reseller API Gateway
|
||||
|
||||
Build an anonymous OpenAI-compatible API gateway that proxies an upstream OpenAI-compatible backend while hiding its identity — for reselling. Model IDs are aliased via HMAC, metadata is stripped, and speed optimizations eliminate unnecessary latency.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Client (any OpenAI client)
|
||||
→ Nginx (TLS termination, HTTP/2, gzip, error masking)
|
||||
→ Gateway (Python http.server on localhost, connection pool)
|
||||
→ Upstream (echogate.one, OpenRouter, any /v1 backend)
|
||||
```
|
||||
|
||||
## Core Patterns
|
||||
|
||||
### 1. Model ID Aliasing (HMAC-SHA256)
|
||||
|
||||
Every upstream model ID gets a deterministic public alias. Upstream never sees the alias; clients never see the real ID.
|
||||
|
||||
```python
|
||||
import hmac, hashlib
|
||||
|
||||
def public_model_id(private_id: str, secret: str) -> str:
|
||||
digest = hmac.new(secret.encode(), private_id.encode(), hashlib.sha256).hexdigest()
|
||||
return f"rb-{digest[:20]}"
|
||||
```
|
||||
|
||||
- Private IDs ONLY accepted on upstream requests (gateway looks up alias → real ID)
|
||||
- Direct private IDs in client requests → HTTP 400
|
||||
- Forbidden models filtered in catalog (e.g. `gpt-5-5` fragment match)
|
||||
|
||||
### 2. Metadata Sanitization
|
||||
|
||||
Strip all upstream metadata from responses:
|
||||
|
||||
```python
|
||||
PRIVATE_METADATA_KEYS = {
|
||||
"backend", "provider", "provider_id", "provider_name",
|
||||
"route", "routing", "system_fingerprint",
|
||||
"upstream", "upstream_id", "upstream_model", "upstream_provider",
|
||||
}
|
||||
```
|
||||
|
||||
- `owned_by` always `[REDACTED-NAME]` (or user's brand)
|
||||
- `system_fingerprint` removed
|
||||
- `model` field rewritten to public alias in both buffered and SSE responses
|
||||
- All hop-by-hop headers stripped (`Connection`, `Keep-Alive`, etc.)
|
||||
|
||||
### 3. Connection Pool with Retry
|
||||
|
||||
Thread-safe HTTPS connection pool — eliminates TCP+TLS handshake per request:
|
||||
|
||||
```python
|
||||
class ConnectionPool:
|
||||
def __init__(self, host, port, scheme, timeout, max_idle=4):
|
||||
self._lock = threading.Lock()
|
||||
self._idle: list[http.client.HTTPConnection] = []
|
||||
self._conn_type = HTTPSConnection if scheme == "https" else HTTPConnection
|
||||
|
||||
def _create(self):
|
||||
conn = self._conn_type(host, port, timeout=timeout)
|
||||
conn.connect()
|
||||
conn.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
|
||||
return conn
|
||||
|
||||
def get(self):
|
||||
with self._lock:
|
||||
while self._idle:
|
||||
conn = self._idle.pop()
|
||||
# select() health check: if socket readable, connection is dead
|
||||
r, _, _ = select.select([conn.sock], [], [], 0)
|
||||
if not r:
|
||||
return conn
|
||||
conn.close()
|
||||
return self._create()
|
||||
|
||||
def put(self, conn):
|
||||
with self._lock:
|
||||
if len(self._idle) < self._max_idle and conn.sock is not None:
|
||||
self._idle.append(conn)
|
||||
return
|
||||
conn.close()
|
||||
```
|
||||
|
||||
**Retry pattern** — stale connections fail on first use, retry with fresh:
|
||||
|
||||
```python
|
||||
def upstream_request(self, method, suffix, body):
|
||||
for attempt in (1, 2):
|
||||
conn = self._pool.get()
|
||||
try:
|
||||
conn.request(method, path, body=body, headers=headers)
|
||||
response = conn.getresponse()
|
||||
body = response.read()
|
||||
if response.status < 500:
|
||||
self._pool.put(conn) # healthy → return to pool
|
||||
else:
|
||||
conn.close() # 5xx → discard
|
||||
return response.status, response.getheaders(), body
|
||||
except (HTTPException, OSError, TimeoutError):
|
||||
conn.close() # transport error → discard
|
||||
if attempt == 2:
|
||||
raise # retry exhausted
|
||||
```
|
||||
|
||||
- Only `5xx` and transport errors discard connections; `4xx` connections go back to pool
|
||||
- Pre-warm 2 connections + force catalog fetch on startup
|
||||
|
||||
### 4. Speed Optimizations (Nginx)
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
http2 on; # Multiplexing
|
||||
gzip on;
|
||||
gzip_min_length 256;
|
||||
gzip_types application/json text/event-stream;
|
||||
gzip_comp_level 1;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:18181;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Connection "";
|
||||
proxy_read_timeout 600s;
|
||||
proxy_request_buffering off;
|
||||
proxy_buffering off;
|
||||
}
|
||||
|
||||
# Mask ALL nginx/upstream errors as neutral JSON
|
||||
error_page 400 408 411 414 431 = @invalid_request;
|
||||
error_page 405 = @method_not_allowed;
|
||||
error_page 500 502 503 504 = @service_unavailable;
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Anti-Fingerprint
|
||||
|
||||
- **UA rotation**: rotate through `OpenAI/Python`, `Go-http-client/2.0`, `python-requests/2.32`, `curl/8.12` — no custom User-Agent
|
||||
- **Server header**: stripped via `more_clear_headers Server;` or `server_tokens off;`
|
||||
- **Error responses**: always neutral JSON, never upstream error bodies or HTML
|
||||
- **Unknown routes**: 404 JSON, never nginx default HTML
|
||||
- **Echo/legacy paths**: explicit 404 (e.g. `^~ /echo/` → JSON 404)
|
||||
|
||||
### 6. Immutable Release Deployment
|
||||
|
||||
Each deploy creates a timestamped release directory with symlink:
|
||||
|
||||
```bash
|
||||
RELEASE_ID=$(date +%Y%m%d_%H%M%S)
|
||||
mkdir -p /opt/apioid-echo/releases/${RELEASE_ID}
|
||||
scp gateway.py root@host:/opt/apioid-echo/releases/${RELEASE_ID}/
|
||||
ssh root@host "
|
||||
ln -sfn /opt/apioid-echo/releases/${RELEASE_ID} /opt/apioid-echo/releases/current
|
||||
systemctl restart apioid-echo-gateway.service
|
||||
"
|
||||
```
|
||||
|
||||
- Backup before deploy: copy current → `/var/backups/apioid-echo/gateway.before-${RELEASE_ID}.py`
|
||||
- Verify: `systemctl is-active`, `NRestarts=0`, live smoke with curl
|
||||
- Env file: `/etc/apioid-echo/gateway.env` chmod 600 (keys + `MODEL_ALIAS_SECRET`)
|
||||
|
||||
## Error Masking
|
||||
|
||||
All errors → neutral JSON, never HTML or upstream metadata:
|
||||
|
||||
| Upstream status | Client response |
|
||||
|---|---|
|
||||
| 429 | `{"error":{"message":"Rate limit exceeded","code":"rate_limit_exceeded"}}` |
|
||||
| 402 | `{"error":{"message":"Daily usage limit reached","code":"daily_limit_reached"}}` |
|
||||
| 400/404/409/422 | `{"error":{"message":"Request was rejected","code":"request_rejected"}}` |
|
||||
| Other 5xx | `{"error":{"message":"Service temporarily unavailable","code":"service_unavailable"}}` |
|
||||
| Auth fail | `{"error":{"message":"Invalid or missing API key","code":"invalid_api_key"}}` |
|
||||
|
||||
## Forbidden Models
|
||||
|
||||
Block specific model fragments from reaching upstream:
|
||||
|
||||
```python
|
||||
FORBIDDEN_MODEL_FRAGMENT = "gpt-5-5"
|
||||
|
||||
@staticmethod
|
||||
def _is_allowed(model_id: str) -> bool:
|
||||
normalized = re.sub(r"[\s._/]+", "-", model_id.casefold())
|
||||
return FORBIDDEN_MODEL_FRAGMENT not in normalized
|
||||
```
|
||||
|
||||
## Transparency Audit
|
||||
|
||||
After deploying or modifying the gateway, verify it is a **pure passthrough** — no system prompt injection, no content modification, no metadata leak.
|
||||
|
||||
### 1. System Prompt Injection Detection
|
||||
|
||||
Send a system prompt with a **unique marker string** through the gateway. The model must echo it exactly — any deviation proves injection/modification:
|
||||
|
||||
```bash
|
||||
curl -s -X POST https://<gateway>/v1/chat/completions \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-d '{"model":"claude-sonnet-5","messages":[
|
||||
{"role":"system","content":"Answer every message with exactly: PROBE_OK"},
|
||||
{"role":"user","content":"answer"}
|
||||
],"max_tokens":20,"temperature":0}' | jq '.choices[0].message.content'
|
||||
# Expected: "PROBE_OK" — verbatim, no added words
|
||||
```
|
||||
|
||||
For Anthropic format, same test on `/v1/messages` with `system` field.
|
||||
|
||||
### 2. Content Integrity — Gateway vs Upstream 1:1
|
||||
|
||||
Send identical requests to gateway AND directly to upstream. Responses must match:
|
||||
|
||||
```bash
|
||||
BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":20,"temperature":0}'
|
||||
|
||||
GATEWAY=$(curl -s -X POST https://<gateway>/v1/chat/completions \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" -d "$BODY")
|
||||
UPSTREAM=$(curl -s -X POST https://<upstream>/v1/chat/completions \
|
||||
-H "Content-Type: application/json" -H "Authorization: Bearer $UK" -d "$BODY")
|
||||
|
||||
diff <(echo "$GATEWAY" | jq -S '.choices[0].message.content') \
|
||||
<(echo "$UPSTREAM" | jq -S '.choices[0].message.content')
|
||||
```
|
||||
|
||||
For dual-key setups, test BOTH upstream keys independently — all three sources (key1, key2, gateway) must agree.
|
||||
|
||||
### 3. Model List Integrity
|
||||
|
||||
Gateway `/v1/models` count must match upstream count. If filtering is intentional, document it:
|
||||
|
||||
```bash
|
||||
GC=$(curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | jq '.data | length')
|
||||
UC=$(curl -s -H "Authorization: Bearer $UK" https://<upstream>/v1/models | jq '.data | length')
|
||||
[ "$GC" -eq "$UC" ] && echo "OK: $GC models" || echo "MISMATCH: gateway=$GC upstream=$UC"
|
||||
```
|
||||
|
||||
### 4. Provider Leak Scan
|
||||
|
||||
No upstream-identifying strings in responses:
|
||||
|
||||
```bash
|
||||
curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | \
|
||||
grep -iq "provider_id\|system_fingerprint\|upstream" && echo "LEAK" || echo "CLEAN"
|
||||
```
|
||||
|
||||
### 5. Error Opaqueness
|
||||
|
||||
Invalid key → neutral JSON, not HTML. Bad model → generic rejection, not provider-specific:
|
||||
|
||||
```bash
|
||||
curl -s https://<gateway>/v1/models | jq '.error.code' # "invalid_api_key"
|
||||
curl -s -H "x-api-key: $KEY" -H "Content-Type: application/json" \
|
||||
-d '{"model":"nonexistent","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' \
|
||||
https://<gateway>/v1/chat/completions | jq '.error.type' # "invalid_request_error"
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- `references/deployment-commands.md` — immutable release deployment workflow + live smoke
|
||||
- `references/nginx-config.md` — production nginx config with HTTP/2, gzip, error masking
|
||||
- `references/transparency-audit.md` — full audit checklist with probe scripts
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Health (no auth)
|
||||
curl -sk https://api.[REDACTED-NAME].loc.cc/health
|
||||
|
||||
# Models (auth required)
|
||||
curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models
|
||||
# Verify: owned_by=[REDACTED-NAME], no provider/system_fingerprint
|
||||
|
||||
# Chat (warm pool — should be <3s)
|
||||
time curl -sk -H "Authorization: Bearer $KEY" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"model":"rb-902d95ce37cee7204c89","messages":[{"role":"user","content":"hi"}],"max_tokens":3}' \
|
||||
https://api.[REDACTED-NAME].loc.cc/v1/chat/completions
|
||||
# Verify: model=rb-*, fp=ABSENT, content present
|
||||
|
||||
# HTTP/2
|
||||
curl -skI --http2 https://api.[REDACTED-NAME].loc.cc/health 2>&1 | head -1
|
||||
# Expected: HTTP/2 200
|
||||
|
||||
# Provider leak test
|
||||
curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models \
|
||||
| python3 -c "import json,sys; d=json.load(sys.stdin); print(any('provider' in m or 'system_fingerprint' in m for m in d['data']))"
|
||||
# Expected: False
|
||||
```
|
||||
|
||||
## Anthropic Direct Proxying (Multimodal)
|
||||
|
||||
When the upstream supports Anthropic `/v1/messages` natively, forward Anthropic requests **directly** — do NOT convert Anthropic→OpenAI→Anthropic. The conversion round-trip strips multimodal content (images, files, PDFs).
|
||||
|
||||
```python
|
||||
# do_POST — route Anthropic natively
|
||||
is_anthropic = path == "/v1/messages"
|
||||
# NO _anthropic_to_openai conversion — forward as-is
|
||||
suffix = "messages" if is_anthropic else path.removeprefix("/v1/")
|
||||
# NO _openai_to_anthropic response transform
|
||||
# sanitize_json_body handles model name replacement for both formats
|
||||
```
|
||||
|
||||
The response sanitizer (`sanitize_json_body`) already handles both OpenAI and Anthropic response structures — it recursively replaces the `model` field and strips `PRIVATE_METADATA_KEYS`.
|
||||
|
||||
## Single-Model Endpoint
|
||||
|
||||
OpenAI clients expect `GET /v1/models/{id}` to return a single model object. Add after the `/v1/models` list handler:
|
||||
|
||||
```python
|
||||
m = re.match(r"^/v1/models/(.+)$", path)
|
||||
if m:
|
||||
model_id = m.group(1)
|
||||
if model_id not in self.gateway.catalog.allowed_ids():
|
||||
self._error(404, "Model not found", "model_not_found", request_id)
|
||||
return
|
||||
self._send_json(200, {"id": model_id, "object": "model", "owned_by": "[REDACTED-NAME]"}, request_id=request_id)
|
||||
return
|
||||
```
|
||||
|
||||
## Transparency Audit Addendum: Multimodal
|
||||
|
||||
After the standard text-only audit, verify multimodal passthrough. Use a 1×1 red PNG (`iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==`):
|
||||
|
||||
```bash
|
||||
IMG="iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
|
||||
|
||||
# Anthropic vision (gateway must forward image blocks)
|
||||
curl -s -X POST https://<gateway>/v1/messages \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-H "anthropic-version: 2023-06-01" \
|
||||
-d "{\"model\":\"claude-sonnet-5\",\"messages\":[{\"role\":\"user\",\"content\":[
|
||||
{\"type\":\"text\",\"text\":\"What color is this image?\"},
|
||||
{\"type\":\"image\",\"source\":{\"type\":\"base64\",\"media_type\":\"image/png\",\"data\":\"$IMG\"}}
|
||||
]}],\"max_tokens\":50}"
|
||||
|
||||
# If the model says \"I don't see an image\" — image was stripped during conversion
|
||||
# Fix: forward Anthropic directly, do not convert to OpenAI
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Stale connection hang**: pooled HTTPS connections die silently (upstream FIN). Health-check with `select.select(sock, [], [], 0)` — if readable with 0 timeout, connection is dead. Always retry on transport error.
|
||||
2. **TCP_NODELAY placement**: must be called AFTER `conn.connect()` and before first `request()`. On http.client, set `conn.sock.setsockopt(IPPROTO_TCP, TCP_NODELAY, 1)`.
|
||||
3. **HTTP/2 on nginx 1.28**: use `http2 on;` directive, NOT `listen ... http2` (deprecated, generates warnings).
|
||||
4. **Gzip on SSE**: `gzip_types text/event-stream` is correct. SSE is text, gzip compresses both JSON and SSE.
|
||||
5. **Catalog pre-warm failure**: wrap in try/except on startup, log warning. Catalog retries on first authenticated request.
|
||||
6. **`_jitter()` removal**: if user says "as fast as possible", remove ALL jitter — TTL jitter AND request jitter. Keep only functional variation (UA rotation).
|
||||
7. **Stream connections**: never return SSE connections to pool — they're live for the stream duration. Only buffered request connections go back.
|
||||
8. **Reasoning models (Claude Fable 5, etc.)**: these consume tokens for internal reasoning before producing visible output. If `max_tokens` is too low (e.g. 10), ALL tokens go to thinking and `content` comes back `null` with `finish_reason: "length"`. Always use `max_tokens >= 50` when probing reasoning models, and check `usage.completion_tokens_details.reasoning_tokens` to confirm thinking consumption.
|
||||
9. **Dual-key verification**: when gateway uses multiple upstream keys, test EACH key directly against the upstream before testing through the gateway. All keys + gateway must return structurally identical responses. A mismatch between keys may indicate per-key model availability differences or rate-limit state.
|
||||
10. **Anthropic→OpenAI conversion strips multimodal**: the `_anthropic_to_openai` converter joins text blocks from content arrays but discards image/file blocks. If the upstream supports Anthropic natively, forward `/v1/messages` requests directly. Only use the converter when the upstream ONLY speaks OpenAI.
|
||||
11. **base64 newlines break curl**: when piping base64 from a file, `cat file.txt` includes a trailing newline that invalidates the base64 in JSON. Use `tr -d '\n\r'` to strip.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Deployment Commands
|
||||
|
||||
Immutable release deployment for [REDACTED-NAME] API gateway on VPS ([REDACTED-IP]).
|
||||
|
||||
## Release ID convention
|
||||
|
||||
```bash
|
||||
RELEASE_ID=$(date +%Y%m%d_%H%M%S)
|
||||
# Example: 20260804_142027
|
||||
```
|
||||
|
||||
## Deploy workflow
|
||||
|
||||
```bash
|
||||
set -eu
|
||||
RELEASE_ID=20260804_142027
|
||||
SHA=$(sha256sum gateway.py | cut -d' ' -f1)
|
||||
|
||||
ssh root@host "mkdir -p /opt/apioid-echo/releases/${RELEASE_ID}"
|
||||
scp gateway.py root@host:/opt/apioid-echo/releases/${RELEASE_ID}/gateway.py
|
||||
scp [REDACTED-NAME].nginx.conf root@host:/opt/apioid-echo/releases/${RELEASE_ID}/
|
||||
|
||||
ssh root@host "
|
||||
R='${RELEASE_ID}'
|
||||
# backup
|
||||
cp /opt/apioid-echo/releases/current/gateway.py /var/backups/apioid-echo/gateway.before-\${R}.py
|
||||
cp /etc/nginx/sites-available/[REDACTED-NAME] /var/backups/apioid-echo/nginx.before-\${R}.conf
|
||||
# deploy
|
||||
ln -sfn /opt/apioid-echo/releases/\${R} /opt/apioid-echo/releases/current
|
||||
chown root:root /opt/apioid-echo/releases/\${R}/*
|
||||
chmod 0644 /opt/apioid-echo/releases/\${R}/gateway.py
|
||||
cp /opt/apioid-echo/releases/\${R}/[REDACTED-NAME].nginx.conf /etc/nginx/sites-available/[REDACTED-NAME]
|
||||
nginx -t && systemctl reload nginx
|
||||
systemctl restart apioid-echo-gateway.service
|
||||
sleep 1.5
|
||||
systemctl is-active apioid-echo-gateway.service
|
||||
systemctl show -p NRestarts apioid-echo-gateway.service
|
||||
"
|
||||
```
|
||||
|
||||
## Live smoke
|
||||
|
||||
```bash
|
||||
ssh root@host "
|
||||
KEY='rfb_KEY_HERE'
|
||||
# health
|
||||
curl -sk --noproxy '*' --resolve api.[REDACTED-NAME].loc.cc:443:127.0.0.1 https://api.[REDACTED-NAME].loc.cc/health
|
||||
# HTTP/2 + chat
|
||||
curl -skI --resolve api.[REDACTED-NAME].loc.cc:443:127.0.0.1 https://api.[REDACTED-NAME].loc.cc/health 2>&1 | head -1
|
||||
time curl -sk -H \"Authorization: Bearer \${KEY}\" \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{\"model\":\"rb-902d95ce37cee7204c89\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}],\"max_tokens\":3}' \
|
||||
https://api.[REDACTED-NAME].loc.cc/v1/chat/completions
|
||||
"
|
||||
```
|
||||
|
||||
## VPS paths
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/opt/apioid-echo/releases/current/` | Active release (symlink) |
|
||||
| `/opt/apioid-echo/releases/<TS>/` | Immutable release directory |
|
||||
| `/etc/apioid-echo/gateway.env` | Env (keys, MODEL_ALIAS_SECRET), chmod 600 |
|
||||
| `/etc/nginx/sites-available/[REDACTED-NAME]` | Nginx config |
|
||||
| `/var/backups/apioid-echo/` | Pre-deploy backups |
|
||||
| `/etc/systemd/system/apioid-echo-gateway.service` | systemd unit |
|
||||
124
skills/reseller-api-gateway/references/nginx-config.md
Normal file
124
skills/reseller-api-gateway/references/nginx-config.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# Nginx Config — API Gateway (HTTP/2 + Gzip + Error Masking)
|
||||
|
||||
Production config for `api.<domain>` on nginx 1.28.3.
|
||||
|
||||
## Server block (SSL + HTTP/2)
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
listen [::]:443 ssl;
|
||||
http2 on;
|
||||
server_name api.[REDACTED-NAME].loc.cc;
|
||||
server_tokens off;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/[REDACTED-NAME].loc.cc/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/[REDACTED-NAME].loc.cc/privkey.pem;
|
||||
include /etc/letsencrypt/options-ssl-nginx.conf;
|
||||
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
|
||||
|
||||
client_max_body_size 25m;
|
||||
gzip on;
|
||||
gzip_min_length 256;
|
||||
gzip_types application/json text/event-stream;
|
||||
gzip_comp_level 1;
|
||||
gzip_vary on;
|
||||
|
||||
location / {
|
||||
proxy_pass http://[REDACTED-NAME];
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Request-ID $request_id;
|
||||
proxy_set_header Connection "";
|
||||
proxy_connect_timeout 10s;
|
||||
proxy_read_timeout 600s;
|
||||
proxy_send_timeout 600s;
|
||||
proxy_request_buffering off;
|
||||
proxy_buffering off;
|
||||
proxy_cache off;
|
||||
proxy_socket_keepalive on;
|
||||
proxy_intercept_errors on;
|
||||
}
|
||||
|
||||
# Error masking: ALL upstream/nginx errors → neutral JSON
|
||||
error_page 400 408 411 414 431 = @invalid_request;
|
||||
error_page 405 = @method_not_allowed;
|
||||
error_page 413 = @request_too_large;
|
||||
error_page 500 502 503 504 = @service_unavailable;
|
||||
|
||||
location @invalid_request {
|
||||
internal;
|
||||
default_type application/json;
|
||||
return 400 '{"error":{"message":"Invalid request","type":"gateway_error","code":"invalid_request"}}';
|
||||
}
|
||||
|
||||
location @method_not_allowed {
|
||||
internal;
|
||||
default_type application/json;
|
||||
return 405 '{"error":{"message":"Method not allowed","type":"gateway_error","code":"method_not_allowed"}}';
|
||||
}
|
||||
|
||||
location @request_too_large {
|
||||
internal;
|
||||
default_type application/json;
|
||||
return 413 '{"error":{"message":"Request body is too large","type":"gateway_error","code":"request_too_large"}}';
|
||||
}
|
||||
|
||||
location @service_unavailable {
|
||||
internal;
|
||||
default_type application/json;
|
||||
return 502 '{"error":{"message":"Service temporarily unavailable","type":"gateway_error","code":"service_unavailable"}}';
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Upstream block
|
||||
|
||||
```nginx
|
||||
upstream [REDACTED-NAME] {
|
||||
zone [REDACTED-NAME] 64k;
|
||||
server 127.0.0.1:18181;
|
||||
keepalive 64;
|
||||
}
|
||||
```
|
||||
|
||||
## HTTP → HTTPS redirect
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name [REDACTED-NAME].loc.cc www.[REDACTED-NAME].loc.cc api.[REDACTED-NAME].loc.cc;
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
```
|
||||
|
||||
## Echo/legacy paths (block on old domain)
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name [REDACTED-NAME].loc.cc www.[REDACTED-NAME].loc.cc;
|
||||
# ... SSL ...
|
||||
location ^~ /echo/ {
|
||||
default_type application/json;
|
||||
return 404 '{"error":{"message":"Not found","type":"gateway_error","code":"not_found"}}';
|
||||
}
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:18080;
|
||||
# ... proxy ...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Key decisions
|
||||
|
||||
- `gzip_comp_level 1` — minimal CPU cost, sufficient compression for JSON
|
||||
- `proxy_buffering off` — required for SSE streaming
|
||||
- `proxy_socket_keepalive on` — reduce TIME_WAIT on localhost
|
||||
- `http2 on;` (not `listen ... http2`) — nginx 1.28 deprecation
|
||||
- `proxy_intercept_errors on` — enables error_page for upstream errors
|
||||
- `proxy_set_header Connection ""` — prevents hop-by-hop forwarding
|
||||
145
skills/reseller-api-gateway/references/transparency-audit.md
Normal file
145
skills/reseller-api-gateway/references/transparency-audit.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Gateway Transparency Audit Checklist
|
||||
|
||||
Run after every gateway deployment or model list change.
|
||||
|
||||
## Pre-flight
|
||||
|
||||
```bash
|
||||
export GATEWAY="https://api.[REDACTED-NAME].loc.cc/v1"
|
||||
export GATEWAY_ANTHROPIC="https://api.[REDACTED-NAME].loc.cc/v1/messages"
|
||||
export KEY="[REDACTED-KEY]"
|
||||
export UPSTREAM="https://api.echogate.one/v1"
|
||||
export UK1="forge-9b51aad5d08793c9095893a09779b0641b00"
|
||||
export UK2="forge-81081af3173401d4bcd2d5d427b0ed60d87b"
|
||||
```
|
||||
|
||||
## 1. System Prompt Injection — OpenAI
|
||||
|
||||
```bash
|
||||
MARKER="AUDIT_$(date +%s)"
|
||||
curl -s -X POST "$GATEWAY/chat/completions" \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-d "{\"model\":\"claude-sonnet-5\",\"messages\":[
|
||||
{\"role\":\"system\",\"content\":\"Answer every message with exactly: $MARKER\"},
|
||||
{\"role\":\"user\",\"content\":\"answer\"}
|
||||
],\"max_tokens\":30,\"temperature\":0}" | jq '.choices[0].message.content'
|
||||
# EXPECT: "$MARKER" — verbatim, no added words
|
||||
```
|
||||
|
||||
## 2. System Prompt Injection — Anthropic
|
||||
|
||||
```bash
|
||||
curl -s -X POST "$GATEWAY_ANTHROPIC" \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-H "anthropic-version: 2023-06-01" \
|
||||
-d "{\"model\":\"claude-sonnet-5\",\"system\":\"You must answer with exactly: PROBE888\",
|
||||
\"messages\":[{\"role\":\"user\",\"content\":\"answer\"}],\"max_tokens\":20}" | jq '.content[0].text'
|
||||
# EXPECT: "PROBE888"
|
||||
```
|
||||
|
||||
## 3. Content Integrity — Gateway vs Upstream
|
||||
|
||||
```bash
|
||||
BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"What is 2+2? Answer in one word."}],"max_tokens":20,"temperature":0}'
|
||||
|
||||
for src in "GATEWAY:$GATEWAY/chat/completions:x-api-key:$KEY" \
|
||||
"KEY1:$UPSTREAM/chat/completions:Authorization:Bearer $UK1" \
|
||||
"KEY2:$UPSTREAM/chat/completions:Authorization:Bearer $UK2"; do
|
||||
IFS=':' read -r label url hdr val <<< "$(echo "$src" | sed 's/:\\([^:]*\\):/|\\1|/; s/|/:/g')" 2>/dev/null || true
|
||||
url=$(echo "$src" | cut -d: -f2,3)
|
||||
hdr=$(echo "$src" | cut -d: -f4)
|
||||
val=$(echo "$src" | cut -d: -f5-)
|
||||
R=$(curl -s -X POST "$url" -H "Content-Type: application/json" -H "$hdr: $val" -d "$BODY")
|
||||
echo "$label: $(echo "$R" | jq -r '.choices[0].message.content')"
|
||||
done
|
||||
# EXPECT: All three return identical content
|
||||
```
|
||||
|
||||
Simpler version:
|
||||
```bash
|
||||
BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":20,"temperature":0}'
|
||||
GW=$(curl -s -X POST "$GATEWAY/chat/completions" -H "Content-Type: application/json" -H "x-api-key: $KEY" -d "$BODY" | jq -S '.choices[0].message')
|
||||
K1=$(curl -s -X POST "$UPSTREAM/chat/completions" -H "Content-Type: application/json" -H "Authorization: Bearer $UK1" -d "$BODY" | jq -S '.choices[0].message')
|
||||
[ "$GW" = "$K1" ] && echo "OK: gateway == key1" || echo "MISMATCH"
|
||||
```
|
||||
|
||||
## 4. No-System-Message Test
|
||||
|
||||
Verify gateway doesn't inject its own system prompt when none is sent:
|
||||
|
||||
```bash
|
||||
curl -s -X POST "$GATEWAY/chat/completions" \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"What is 2+2? Answer in one word."}],"max_tokens":20,"temperature":0}' | jq '.choices[0].message.content'
|
||||
# EXPECT: natural response like "Four" — no injected behavior, no prefix
|
||||
```
|
||||
|
||||
## 5. Model List Integrity
|
||||
|
||||
```bash
|
||||
GW_COUNT=$(curl -s -H "x-api-key: $KEY" "$GATEWAY/models" | jq '.data | length')
|
||||
UP_COUNT=$(curl -s -H "Authorization: Bearer $UK1" "$UPSTREAM/models" | jq '.data | length')
|
||||
echo "Gateway: $GW_COUNT, Upstream: $UP_COUNT"
|
||||
[ "$GW_COUNT" -eq "$UP_COUNT" ] && echo "PASS" || echo "FAIL — check FORBIDDEN_MODEL_FRAGMENTS"
|
||||
```
|
||||
|
||||
## 6. Provider Leak Scan
|
||||
|
||||
```bash
|
||||
curl -s -H "x-api-key: $KEY" "$GATEWAY/models" | grep -iq \
|
||||
"provider_id\|system_fingerprint\|upstream\|echogate" && echo "LEAK DETECTED" || echo "CLEAN"
|
||||
```
|
||||
|
||||
## 7. Error Opaqueness
|
||||
|
||||
```bash
|
||||
# No key → clean 401
|
||||
echo -n "No key: "
|
||||
curl -so /dev/null -w "%{http_code}" "$GATEWAY/models"
|
||||
echo ""
|
||||
echo -n "No key body: "
|
||||
curl -s "$GATEWAY/models" | jq '.error.code'
|
||||
|
||||
# Bad model → clean 400
|
||||
echo -n "Bad model: "
|
||||
curl -s -X POST "$GATEWAY/chat/completions" \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-d '{"model":"nonexistent-model","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' | jq '.error.type'
|
||||
|
||||
# Bad key → 401
|
||||
echo -n "Bad key: "
|
||||
curl -so /dev/null -w "%{http_code}" -H "x-api-key: [REDACTED]" "$GATEWAY/models"
|
||||
```
|
||||
|
||||
## 8. Reasoning Model Probe
|
||||
|
||||
When testing reasoning models (fable-5, opus-5-sonnet, etc.), use enough `max_tokens`:
|
||||
|
||||
```bash
|
||||
# WRONG — all tokens consumed by thinking, content=null
|
||||
curl -s -X POST "$GATEWAY/chat/completions" \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-d '{"model":"claude-fable-5","messages":[{"role":"user","content":"Say hi"}],"max_tokens":10}' | jq '{content: .choices[0].message.content, reason: .usage.completion_tokens_details.reasoning_tokens}'
|
||||
|
||||
# RIGHT — enough tokens for thinking + output
|
||||
curl -s -X POST "$GATEWAY/chat/completions" \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-d '{"model":"claude-fable-5","messages":[{"role":"user","content":"Say hi"}],"max_tokens":200}' | jq '.choices[0].message.content'
|
||||
```
|
||||
|
||||
## Quick Smoke (all-in-one)
|
||||
|
||||
```bash
|
||||
# Health
|
||||
curl -sf "$GATEWAY/../health" | jq '.status' || echo "HEALTH FAIL"
|
||||
|
||||
# Models count
|
||||
echo -n "Models: "
|
||||
curl -s -H "x-api-key: $KEY" "$GATEWAY/models" | jq '.data | length'
|
||||
|
||||
# Chat probe
|
||||
echo -n "Chat: "
|
||||
curl -s -X POST "$GATEWAY/chat/completions" \
|
||||
-H "Content-Type: application/json" -H "x-api-key: $KEY" \
|
||||
-d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":3}' | jq -r '.choices[0].message.content'
|
||||
```
|
||||
Reference in New Issue
Block a user