autoreg-skills-max: initial public release (restored, redacted, hardened)

2026-10-20 02:16:32 +03:00
commit f58eafeb48
565 changed files with 60064 additions and 0 deletions
--- a/skills/reseller-api-gateway/SKILL.md
+++ b/skills/reseller-api-gateway/SKILL.md
@@ -0,0 +1,365 @@
+---
+name: reseller-api-gateway
+description: Build anonymous OpenAI-compatible API gateways for reselling — hide upstream provider, alias model IDs, strip metadata, apply speed optimizations (HTTP/2, connection pool, gzip, TCP_NODELAY), and deploy via immutable releases.
+triggers:
+  - "resell API"
+  - "hide provider"
+  - "anonymous gateway"
+  - "opaque proxy"
+  - "api reseller"
+  - "model aliasing"
+  - "[REDACTED-NAME] API"
+  - "apioid gateway"
+  - "speed up API"
+---
+
+# Reseller API Gateway
+
+Build an anonymous OpenAI-compatible API gateway that proxies an upstream OpenAI-compatible backend while hiding its identity — for reselling. Model IDs are aliased via HMAC, metadata is stripped, and speed optimizations eliminate unnecessary latency.
+
+## Architecture
+
+```
+Client (any OpenAI client)
+  → Nginx (TLS termination, HTTP/2, gzip, error masking)
+  → Gateway (Python http.server on localhost, connection pool)
+  → Upstream (echogate.one, OpenRouter, any /v1 backend)
+```
+
+## Core Patterns
+
+### 1. Model ID Aliasing (HMAC-SHA256)
+
+Every upstream model ID gets a deterministic public alias. Upstream never sees the alias; clients never see the real ID.
+
+```python
+import hmac, hashlib
+
+def public_model_id(private_id: str, secret: str) -> str:
+    digest = hmac.new(secret.encode(), private_id.encode(), hashlib.sha256).hexdigest()
+    return f"rb-{digest[:20]}"
+```
+
+- Private IDs ONLY accepted on upstream requests (gateway looks up alias → real ID)
+- Direct private IDs in client requests → HTTP 400
+- Forbidden models filtered in catalog (e.g. `gpt-5-5` fragment match)
+
+### 2. Metadata Sanitization
+
+Strip all upstream metadata from responses:
+
+```python
+PRIVATE_METADATA_KEYS = {
+    "backend", "provider", "provider_id", "provider_name",
+    "route", "routing", "system_fingerprint",
+    "upstream", "upstream_id", "upstream_model", "upstream_provider",
+}
+```
+
+- `owned_by` always `[REDACTED-NAME]` (or user's brand)
+- `system_fingerprint` removed
+- `model` field rewritten to public alias in both buffered and SSE responses
+- All hop-by-hop headers stripped (`Connection`, `Keep-Alive`, etc.)
+
+### 3. Connection Pool with Retry
+
+Thread-safe HTTPS connection pool — eliminates TCP+TLS handshake per request:
+
+```python
+class ConnectionPool:
+    def __init__(self, host, port, scheme, timeout, max_idle=4):
+        self._lock = threading.Lock()
+        self._idle: list[http.client.HTTPConnection] = []
+        self._conn_type = HTTPSConnection if scheme == "https" else HTTPConnection
+    
+    def _create(self):
+        conn = self._conn_type(host, port, timeout=timeout)
+        conn.connect()
+        conn.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+        return conn
+    
+    def get(self):
+        with self._lock:
+            while self._idle:
+                conn = self._idle.pop()
+                # select() health check: if socket readable, connection is dead
+                r, _, _ = select.select([conn.sock], [], [], 0)
+                if not r:
+                    return conn
+                conn.close()
+        return self._create()
+    
+    def put(self, conn):
+        with self._lock:
+            if len(self._idle) < self._max_idle and conn.sock is not None:
+                self._idle.append(conn)
+                return
+        conn.close()
+```
+
+**Retry pattern** — stale connections fail on first use, retry with fresh:
+
+```python
+def upstream_request(self, method, suffix, body):
+    for attempt in (1, 2):
+        conn = self._pool.get()
+        try:
+            conn.request(method, path, body=body, headers=headers)
+            response = conn.getresponse()
+            body = response.read()
+            if response.status < 500:
+                self._pool.put(conn)  # healthy → return to pool
+            else:
+                conn.close()          # 5xx → discard
+            return response.status, response.getheaders(), body
+        except (HTTPException, OSError, TimeoutError):
+            conn.close()              # transport error → discard
+            if attempt == 2:
+                raise                 # retry exhausted
+```
+
+- Only `5xx` and transport errors discard connections; `4xx` connections go back to pool
+- Pre-warm 2 connections + force catalog fetch on startup
+
+### 4. Speed Optimizations (Nginx)
+
+```nginx
+server {
+    listen 443 ssl;
+    http2 on;                           # Multiplexing
+    gzip on;
+    gzip_min_length 256;
+    gzip_types application/json text/event-stream;
+    gzip_comp_level 1;
+
+    location / {
+        proxy_pass http://127.0.0.1:18181;
+        proxy_http_version 1.1;
+        proxy_set_header Connection "";
+        proxy_read_timeout 600s;
+        proxy_request_buffering off;
+        proxy_buffering off;
+    }
+
+    # Mask ALL nginx/upstream errors as neutral JSON
+    error_page 400 408 411 414 431 = @invalid_request;
+    error_page 405 = @method_not_allowed;
+    error_page 500 502 503 504 = @service_unavailable;
+}
+```
+
+### 5. Anti-Fingerprint
+
+- **UA rotation**: rotate through `OpenAI/Python`, `Go-http-client/2.0`, `python-requests/2.32`, `curl/8.12` — no custom User-Agent
+- **Server header**: stripped via `more_clear_headers Server;` or `server_tokens off;`
+- **Error responses**: always neutral JSON, never upstream error bodies or HTML
+- **Unknown routes**: 404 JSON, never nginx default HTML
+- **Echo/legacy paths**: explicit 404 (e.g. `^~ /echo/` → JSON 404)
+
+### 6. Immutable Release Deployment
+
+Each deploy creates a timestamped release directory with symlink:
+
+```bash
+RELEASE_ID=$(date +%Y%m%d_%H%M%S)
+mkdir -p /opt/apioid-echo/releases/${RELEASE_ID}
+scp gateway.py root@host:/opt/apioid-echo/releases/${RELEASE_ID}/
+ssh root@host "
+  ln -sfn /opt/apioid-echo/releases/${RELEASE_ID} /opt/apioid-echo/releases/current
+  systemctl restart apioid-echo-gateway.service
+"
+```
+
+- Backup before deploy: copy current → `/var/backups/apioid-echo/gateway.before-${RELEASE_ID}.py`
+- Verify: `systemctl is-active`, `NRestarts=0`, live smoke with curl
+- Env file: `/etc/apioid-echo/gateway.env` chmod 600 (keys + `MODEL_ALIAS_SECRET`)
+
+## Error Masking
+
+All errors → neutral JSON, never HTML or upstream metadata:
+
+| Upstream status | Client response |
+|---|---|
+| 429 | `{"error":{"message":"Rate limit exceeded","code":"rate_limit_exceeded"}}` |
+| 402 | `{"error":{"message":"Daily usage limit reached","code":"daily_limit_reached"}}` |
+| 400/404/409/422 | `{"error":{"message":"Request was rejected","code":"request_rejected"}}` |
+| Other 5xx | `{"error":{"message":"Service temporarily unavailable","code":"service_unavailable"}}` |
+| Auth fail | `{"error":{"message":"Invalid or missing API key","code":"invalid_api_key"}}` |
+
+## Forbidden Models
+
+Block specific model fragments from reaching upstream:
+
+```python
+FORBIDDEN_MODEL_FRAGMENT = "gpt-5-5"
+
+@staticmethod
+def _is_allowed(model_id: str) -> bool:
+    normalized = re.sub(r"[\s._/]+", "-", model_id.casefold())
+    return FORBIDDEN_MODEL_FRAGMENT not in normalized
+```
+
+## Transparency Audit
+
+After deploying or modifying the gateway, verify it is a **pure passthrough** — no system prompt injection, no content modification, no metadata leak.
+
+### 1. System Prompt Injection Detection
+
+Send a system prompt with a **unique marker string** through the gateway. The model must echo it exactly — any deviation proves injection/modification:
+
+```bash
+curl -s -X POST https://<gateway>/v1/chat/completions \
+  -H "Content-Type: application/json" -H "x-api-key: $KEY" \
+  -d '{"model":"claude-sonnet-5","messages":[
+    {"role":"system","content":"Answer every message with exactly: PROBE_OK"},
+    {"role":"user","content":"answer"}
+  ],"max_tokens":20,"temperature":0}' | jq '.choices[0].message.content'
+# Expected: "PROBE_OK" — verbatim, no added words
+```
+
+For Anthropic format, same test on `/v1/messages` with `system` field.
+
+### 2. Content Integrity — Gateway vs Upstream 1:1
+
+Send identical requests to gateway AND directly to upstream. Responses must match:
+
+```bash
+BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":20,"temperature":0}'
+
+GATEWAY=$(curl -s -X POST https://<gateway>/v1/chat/completions \
+  -H "Content-Type: application/json" -H "x-api-key: $KEY" -d "$BODY")
+UPSTREAM=$(curl -s -X POST https://<upstream>/v1/chat/completions \
+  -H "Content-Type: application/json" -H "Authorization: Bearer $UK" -d "$BODY")
+
+diff <(echo "$GATEWAY" | jq -S '.choices[0].message.content') \
+     <(echo "$UPSTREAM" | jq -S '.choices[0].message.content')
+```
+
+For dual-key setups, test BOTH upstream keys independently — all three sources (key1, key2, gateway) must agree.
+
+### 3. Model List Integrity
+
+Gateway `/v1/models` count must match upstream count. If filtering is intentional, document it:
+
+```bash
+GC=$(curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | jq '.data | length')
+UC=$(curl -s -H "Authorization: Bearer $UK" https://<upstream>/v1/models | jq '.data | length')
+[ "$GC" -eq "$UC" ] && echo "OK: $GC models" || echo "MISMATCH: gateway=$GC upstream=$UC"
+```
+
+### 4. Provider Leak Scan
+
+No upstream-identifying strings in responses:
+
+```bash
+curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | \
+  grep -iq "provider_id\|system_fingerprint\|upstream" && echo "LEAK" || echo "CLEAN"
+```
+
+### 5. Error Opaqueness
+
+Invalid key → neutral JSON, not HTML. Bad model → generic rejection, not provider-specific:
+
+```bash
+curl -s https://<gateway>/v1/models | jq '.error.code'          # "invalid_api_key"
+curl -s -H "x-api-key: $KEY" -H "Content-Type: application/json" \
+  -d '{"model":"nonexistent","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' \
+  https://<gateway>/v1/chat/completions | jq '.error.type'       # "invalid_request_error"
+```
+
+## References
+
+- `references/deployment-commands.md` — immutable release deployment workflow + live smoke
+- `references/nginx-config.md` — production nginx config with HTTP/2, gzip, error masking
+- `references/transparency-audit.md` — full audit checklist with probe scripts
+
+## Verification
+
+```bash
+# Health (no auth)
+curl -sk https://api.[REDACTED-NAME].loc.cc/health
+
+# Models (auth required)
+curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models
+# Verify: owned_by=[REDACTED-NAME], no provider/system_fingerprint
+
+# Chat (warm pool — should be <3s)
+time curl -sk -H "Authorization: Bearer $KEY" \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"rb-902d95ce37cee7204c89","messages":[{"role":"user","content":"hi"}],"max_tokens":3}' \
+  https://api.[REDACTED-NAME].loc.cc/v1/chat/completions
+# Verify: model=rb-*, fp=ABSENT, content present
+
+# HTTP/2
+curl -skI --http2 https://api.[REDACTED-NAME].loc.cc/health 2>&1 | head -1
+# Expected: HTTP/2 200
+
+# Provider leak test
+curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models \
+  | python3 -c "import json,sys; d=json.load(sys.stdin); print(any('provider' in m or 'system_fingerprint' in m for m in d['data']))"
+# Expected: False
+```
+
+## Anthropic Direct Proxying (Multimodal)
+
+When the upstream supports Anthropic `/v1/messages` natively, forward Anthropic requests **directly** — do NOT convert Anthropic→OpenAI→Anthropic. The conversion round-trip strips multimodal content (images, files, PDFs).
+
+```python
+# do_POST — route Anthropic natively
+is_anthropic = path == "/v1/messages"
+# NO _anthropic_to_openai conversion — forward as-is
+suffix = "messages" if is_anthropic else path.removeprefix("/v1/")
+# NO _openai_to_anthropic response transform
+# sanitize_json_body handles model name replacement for both formats
+```
+
+The response sanitizer (`sanitize_json_body`) already handles both OpenAI and Anthropic response structures — it recursively replaces the `model` field and strips `PRIVATE_METADATA_KEYS`.
+
+## Single-Model Endpoint
+
+OpenAI clients expect `GET /v1/models/{id}` to return a single model object. Add after the `/v1/models` list handler:
+
+```python
+m = re.match(r"^/v1/models/(.+)$", path)
+if m:
+    model_id = m.group(1)
+    if model_id not in self.gateway.catalog.allowed_ids():
+        self._error(404, "Model not found", "model_not_found", request_id)
+        return
+    self._send_json(200, {"id": model_id, "object": "model", "owned_by": "[REDACTED-NAME]"}, request_id=request_id)
+    return
+```
+
+## Transparency Audit Addendum: Multimodal
+
+After the standard text-only audit, verify multimodal passthrough. Use a 1×1 red PNG (`iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==`):
+
+```bash
+IMG="iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="
+
+# Anthropic vision (gateway must forward image blocks)
+curl -s -X POST https://<gateway>/v1/messages \
+  -H "Content-Type: application/json" -H "x-api-key: $KEY" \
+  -H "anthropic-version: 2023-06-01" \
+  -d "{\"model\":\"claude-sonnet-5\",\"messages\":[{\"role\":\"user\",\"content\":[
+    {\"type\":\"text\",\"text\":\"What color is this image?\"},
+    {\"type\":\"image\",\"source\":{\"type\":\"base64\",\"media_type\":\"image/png\",\"data\":\"$IMG\"}}
+  ]}],\"max_tokens\":50}"
+
+# If the model says \"I don't see an image\" — image was stripped during conversion
+# Fix: forward Anthropic directly, do not convert to OpenAI
+```
+
+## Pitfalls
+
+1. **Stale connection hang**: pooled HTTPS connections die silently (upstream FIN). Health-check with `select.select(sock, [], [], 0)` — if readable with 0 timeout, connection is dead. Always retry on transport error.
+2. **TCP_NODELAY placement**: must be called AFTER `conn.connect()` and before first `request()`. On http.client, set `conn.sock.setsockopt(IPPROTO_TCP, TCP_NODELAY, 1)`.
+3. **HTTP/2 on nginx 1.28**: use `http2 on;` directive, NOT `listen ... http2` (deprecated, generates warnings).
+4. **Gzip on SSE**: `gzip_types text/event-stream` is correct. SSE is text, gzip compresses both JSON and SSE.
+5. **Catalog pre-warm failure**: wrap in try/except on startup, log warning. Catalog retries on first authenticated request.
+6. **`_jitter()` removal**: if user says "as fast as possible", remove ALL jitter — TTL jitter AND request jitter. Keep only functional variation (UA rotation).
+7. **Stream connections**: never return SSE connections to pool — they're live for the stream duration. Only buffered request connections go back.
+8. **Reasoning models (Claude Fable 5, etc.)**: these consume tokens for internal reasoning before producing visible output. If `max_tokens` is too low (e.g. 10), ALL tokens go to thinking and `content` comes back `null` with `finish_reason: "length"`. Always use `max_tokens >= 50` when probing reasoning models, and check `usage.completion_tokens_details.reasoning_tokens` to confirm thinking consumption.
+9. **Dual-key verification**: when gateway uses multiple upstream keys, test EACH key directly against the upstream before testing through the gateway. All keys + gateway must return structurally identical responses. A mismatch between keys may indicate per-key model availability differences or rate-limit state.
+10. **Anthropic→OpenAI conversion strips multimodal**: the `_anthropic_to_openai` converter joins text blocks from content arrays but discards image/file blocks. If the upstream supports Anthropic natively, forward `/v1/messages` requests directly. Only use the converter when the upstream ONLY speaks OpenAI.
+11. **base64 newlines break curl**: when piping base64 from a file, `cat file.txt` includes a trailing newline that invalidates the base64 in JSON. Use `tr -d '\n\r'` to strip.