vlad/autoreg-skills-max
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
autoreg-skills-max/skills/reseller-api-gateway/SKILL.md

15 KiB
Raw Permalink Blame History

name, description, triggers
name description triggers
reseller-api-gateway Build anonymous OpenAI-compatible API gateways for reselling — hide upstream provider, alias model IDs, strip metadata, apply speed optimizations (HTTP/2, connection pool, gzip, TCP_NODELAY), and deploy via immutable releases.
resell API
hide provider
anonymous gateway
opaque proxy
api reseller
model aliasing
[REDACTED-NAME] API
apioid gateway
speed up API

Reseller API Gateway

Build an anonymous OpenAI-compatible API gateway that proxies an upstream OpenAI-compatible backend while hiding its identity — for reselling. Model IDs are aliased via HMAC, metadata is stripped, and speed optimizations eliminate unnecessary latency.

Architecture

Client (any OpenAI client)
  → Nginx (TLS termination, HTTP/2, gzip, error masking)
  → Gateway (Python http.server on localhost, connection pool)
  → Upstream (echogate.one, OpenRouter, any /v1 backend)

Core Patterns

1. Model ID Aliasing (HMAC-SHA256)

Every upstream model ID gets a deterministic public alias. Upstream never sees the alias; clients never see the real ID.

import hmac, hashlib

def public_model_id(private_id: str, secret: str) -> str:
    digest = hmac.new(secret.encode(), private_id.encode(), hashlib.sha256).hexdigest()
    return f"rb-{digest[:20]}"
  • Private IDs ONLY accepted on upstream requests (gateway looks up alias → real ID)
  • Direct private IDs in client requests → HTTP 400
  • Forbidden models filtered in catalog (e.g. gpt-5-5 fragment match)

2. Metadata Sanitization

Strip all upstream metadata from responses:

PRIVATE_METADATA_KEYS = {
    "backend", "provider", "provider_id", "provider_name",
    "route", "routing", "system_fingerprint",
    "upstream", "upstream_id", "upstream_model", "upstream_provider",
}
  • owned_by always [REDACTED-NAME] (or user's brand)
  • system_fingerprint removed
  • model field rewritten to public alias in both buffered and SSE responses
  • All hop-by-hop headers stripped (Connection, Keep-Alive, etc.)

3. Connection Pool with Retry

Thread-safe HTTPS connection pool — eliminates TCP+TLS handshake per request:

class ConnectionPool:
    def __init__(self, host, port, scheme, timeout, max_idle=4):
        self._lock = threading.Lock()
        self._idle: list[http.client.HTTPConnection] = []
        self._conn_type = HTTPSConnection if scheme == "https" else HTTPConnection
    
    def _create(self):
        conn = self._conn_type(host, port, timeout=timeout)
        conn.connect()
        conn.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
        return conn
    
    def get(self):
        with self._lock:
            while self._idle:
                conn = self._idle.pop()
                # select() health check: if socket readable, connection is dead
                r, _, _ = select.select([conn.sock], [], [], 0)
                if not r:
                    return conn
                conn.close()
        return self._create()
    
    def put(self, conn):
        with self._lock:
            if len(self._idle) < self._max_idle and conn.sock is not None:
                self._idle.append(conn)
                return
        conn.close()

Retry pattern — stale connections fail on first use, retry with fresh:

def upstream_request(self, method, suffix, body):
    for attempt in (1, 2):
        conn = self._pool.get()
        try:
            conn.request(method, path, body=body, headers=headers)
            response = conn.getresponse()
            body = response.read()
            if response.status < 500:
                self._pool.put(conn)  # healthy → return to pool
            else:
                conn.close()          # 5xx → discard
            return response.status, response.getheaders(), body
        except (HTTPException, OSError, TimeoutError):
            conn.close()              # transport error → discard
            if attempt == 2:
                raise                 # retry exhausted
  • Only 5xx and transport errors discard connections; 4xx connections go back to pool
  • Pre-warm 2 connections + force catalog fetch on startup

4. Speed Optimizations (Nginx)

server {
    listen 443 ssl;
    http2 on;                           # Multiplexing
    gzip on;
    gzip_min_length 256;
    gzip_types application/json text/event-stream;
    gzip_comp_level 1;

    location / {
        proxy_pass http://127.0.0.1:18181;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_read_timeout 600s;
        proxy_request_buffering off;
        proxy_buffering off;
    }

    # Mask ALL nginx/upstream errors as neutral JSON
    error_page 400 408 411 414 431 = @invalid_request;
    error_page 405 = @method_not_allowed;
    error_page 500 502 503 504 = @service_unavailable;
}

5. Anti-Fingerprint

  • UA rotation: rotate through OpenAI/Python, Go-http-client/2.0, python-requests/2.32, curl/8.12 — no custom User-Agent
  • Server header: stripped via more_clear_headers Server; or server_tokens off;
  • Error responses: always neutral JSON, never upstream error bodies or HTML
  • Unknown routes: 404 JSON, never nginx default HTML
  • Echo/legacy paths: explicit 404 (e.g. ^~ /echo/ → JSON 404)

6. Immutable Release Deployment

Each deploy creates a timestamped release directory with symlink:

RELEASE_ID=$(date +%Y%m%d_%H%M%S)
mkdir -p /opt/apioid-echo/releases/${RELEASE_ID}
scp gateway.py root@host:/opt/apioid-echo/releases/${RELEASE_ID}/
ssh root@host "
  ln -sfn /opt/apioid-echo/releases/${RELEASE_ID} /opt/apioid-echo/releases/current
  systemctl restart apioid-echo-gateway.service
"
  • Backup before deploy: copy current → /var/backups/apioid-echo/gateway.before-${RELEASE_ID}.py
  • Verify: systemctl is-active, NRestarts=0, live smoke with curl
  • Env file: /etc/apioid-echo/gateway.env chmod 600 (keys + MODEL_ALIAS_SECRET)

Error Masking

All errors → neutral JSON, never HTML or upstream metadata:

Upstream status Client response
429 {"error":{"message":"Rate limit exceeded","code":"rate_limit_exceeded"}}
402 {"error":{"message":"Daily usage limit reached","code":"daily_limit_reached"}}
400/404/409/422 {"error":{"message":"Request was rejected","code":"request_rejected"}}
Other 5xx {"error":{"message":"Service temporarily unavailable","code":"service_unavailable"}}
Auth fail {"error":{"message":"Invalid or missing API key","code":"invalid_api_key"}}

Forbidden Models

Block specific model fragments from reaching upstream:

FORBIDDEN_MODEL_FRAGMENT = "gpt-5-5"

@staticmethod
def _is_allowed(model_id: str) -> bool:
    normalized = re.sub(r"[\s._/]+", "-", model_id.casefold())
    return FORBIDDEN_MODEL_FRAGMENT not in normalized

Transparency Audit

After deploying or modifying the gateway, verify it is a pure passthrough — no system prompt injection, no content modification, no metadata leak.

1. System Prompt Injection Detection

Send a system prompt with a unique marker string through the gateway. The model must echo it exactly — any deviation proves injection/modification:

curl -s -X POST https://<gateway>/v1/chat/completions \
  -H "Content-Type: application/json" -H "x-api-key: $KEY" \
  -d '{"model":"claude-sonnet-5","messages":[
    {"role":"system","content":"Answer every message with exactly: PROBE_OK"},
    {"role":"user","content":"answer"}
  ],"max_tokens":20,"temperature":0}' | jq '.choices[0].message.content'
# Expected: "PROBE_OK" — verbatim, no added words

For Anthropic format, same test on /v1/messages with system field.

2. Content Integrity — Gateway vs Upstream 1:1

Send identical requests to gateway AND directly to upstream. Responses must match:

BODY='{"model":"claude-sonnet-5","messages":[{"role":"user","content":"hi"}],"max_tokens":20,"temperature":0}'

GATEWAY=$(curl -s -X POST https://<gateway>/v1/chat/completions \
  -H "Content-Type: application/json" -H "x-api-key: $KEY" -d "$BODY")
UPSTREAM=$(curl -s -X POST https://<upstream>/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer $UK" -d "$BODY")

diff <(echo "$GATEWAY" | jq -S '.choices[0].message.content') \
     <(echo "$UPSTREAM" | jq -S '.choices[0].message.content')

For dual-key setups, test BOTH upstream keys independently — all three sources (key1, key2, gateway) must agree.

3. Model List Integrity

Gateway /v1/models count must match upstream count. If filtering is intentional, document it:

GC=$(curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | jq '.data | length')
UC=$(curl -s -H "Authorization: Bearer $UK" https://<upstream>/v1/models | jq '.data | length')
[ "$GC" -eq "$UC" ] && echo "OK: $GC models" || echo "MISMATCH: gateway=$GC upstream=$UC"

4. Provider Leak Scan

No upstream-identifying strings in responses:

curl -s -H "x-api-key: $KEY" https://<gateway>/v1/models | \
  grep -iq "provider_id\|system_fingerprint\|upstream" && echo "LEAK" || echo "CLEAN"

5. Error Opaqueness

Invalid key → neutral JSON, not HTML. Bad model → generic rejection, not provider-specific:

curl -s https://<gateway>/v1/models | jq '.error.code'          # "invalid_api_key"
curl -s -H "x-api-key: $KEY" -H "Content-Type: application/json" \
  -d '{"model":"nonexistent","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' \
  https://<gateway>/v1/chat/completions | jq '.error.type'       # "invalid_request_error"

References

  • references/deployment-commands.md — immutable release deployment workflow + live smoke
  • references/nginx-config.md — production nginx config with HTTP/2, gzip, error masking
  • references/transparency-audit.md — full audit checklist with probe scripts

Verification

# Health (no auth)
curl -sk https://api.[REDACTED-NAME].loc.cc/health

# Models (auth required)
curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models
# Verify: owned_by=[REDACTED-NAME], no provider/system_fingerprint

# Chat (warm pool — should be <3s)
time curl -sk -H "Authorization: Bearer $KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"rb-902d95ce37cee7204c89","messages":[{"role":"user","content":"hi"}],"max_tokens":3}' \
  https://api.[REDACTED-NAME].loc.cc/v1/chat/completions
# Verify: model=rb-*, fp=ABSENT, content present

# HTTP/2
curl -skI --http2 https://api.[REDACTED-NAME].loc.cc/health 2>&1 | head -1
# Expected: HTTP/2 200

# Provider leak test
curl -sk -H "Authorization: Bearer $KEY" https://api.[REDACTED-NAME].loc.cc/v1/models \
  | python3 -c "import json,sys; d=json.load(sys.stdin); print(any('provider' in m or 'system_fingerprint' in m for m in d['data']))"
# Expected: False

Anthropic Direct Proxying (Multimodal)

When the upstream supports Anthropic /v1/messages natively, forward Anthropic requests directly — do NOT convert Anthropic→OpenAI→Anthropic. The conversion round-trip strips multimodal content (images, files, PDFs).

# do_POST — route Anthropic natively
is_anthropic = path == "/v1/messages"
# NO _anthropic_to_openai conversion — forward as-is
suffix = "messages" if is_anthropic else path.removeprefix("/v1/")
# NO _openai_to_anthropic response transform
# sanitize_json_body handles model name replacement for both formats

The response sanitizer (sanitize_json_body) already handles both OpenAI and Anthropic response structures — it recursively replaces the model field and strips PRIVATE_METADATA_KEYS.

Single-Model Endpoint

OpenAI clients expect GET /v1/models/{id} to return a single model object. Add after the /v1/models list handler:

m = re.match(r"^/v1/models/(.+)$", path)
if m:
    model_id = m.group(1)
    if model_id not in self.gateway.catalog.allowed_ids():
        self._error(404, "Model not found", "model_not_found", request_id)
        return
    self._send_json(200, {"id": model_id, "object": "model", "owned_by": "[REDACTED-NAME]"}, request_id=request_id)
    return

Transparency Audit Addendum: Multimodal

After the standard text-only audit, verify multimodal passthrough. Use a 1×1 red PNG (iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg==):

IMG="iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=="

# Anthropic vision (gateway must forward image blocks)
curl -s -X POST https://<gateway>/v1/messages \
  -H "Content-Type: application/json" -H "x-api-key: $KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d "{\"model\":\"claude-sonnet-5\",\"messages\":[{\"role\":\"user\",\"content\":[
    {\"type\":\"text\",\"text\":\"What color is this image?\"},
    {\"type\":\"image\",\"source\":{\"type\":\"base64\",\"media_type\":\"image/png\",\"data\":\"$IMG\"}}
  ]}],\"max_tokens\":50}"

# If the model says \"I don't see an image\" — image was stripped during conversion
# Fix: forward Anthropic directly, do not convert to OpenAI

Pitfalls

  1. Stale connection hang: pooled HTTPS connections die silently (upstream FIN). Health-check with select.select(sock, [], [], 0) — if readable with 0 timeout, connection is dead. Always retry on transport error.
  2. TCP_NODELAY placement: must be called AFTER conn.connect() and before first request(). On http.client, set conn.sock.setsockopt(IPPROTO_TCP, TCP_NODELAY, 1).
  3. HTTP/2 on nginx 1.28: use http2 on; directive, NOT listen ... http2 (deprecated, generates warnings).
  4. Gzip on SSE: gzip_types text/event-stream is correct. SSE is text, gzip compresses both JSON and SSE.
  5. Catalog pre-warm failure: wrap in try/except on startup, log warning. Catalog retries on first authenticated request.
  6. _jitter() removal: if user says "as fast as possible", remove ALL jitter — TTL jitter AND request jitter. Keep only functional variation (UA rotation).
  7. Stream connections: never return SSE connections to pool — they're live for the stream duration. Only buffered request connections go back.
  8. Reasoning models (Claude Fable 5, etc.): these consume tokens for internal reasoning before producing visible output. If max_tokens is too low (e.g. 10), ALL tokens go to thinking and content comes back null with finish_reason: "length". Always use max_tokens >= 50 when probing reasoning models, and check usage.completion_tokens_details.reasoning_tokens to confirm thinking consumption.
  9. Dual-key verification: when gateway uses multiple upstream keys, test EACH key directly against the upstream before testing through the gateway. All keys + gateway must return structurally identical responses. A mismatch between keys may indicate per-key model availability differences or rate-limit state.
  10. Anthropic→OpenAI conversion strips multimodal: the _anthropic_to_openai converter joins text blocks from content arrays but discards image/file blocks. If the upstream supports Anthropic natively, forward /v1/messages requests directly. Only use the converter when the upstream ONLY speaks OpenAI.
  11. base64 newlines break curl: when piping base64 from a file, cat file.txt includes a trailing newline that invalidates the base64 in JSON. Use tr -d '\n\r' to strip.