43 lines
5.0 KiB
Markdown
43 lines
5.0 KiB
Markdown
# TokenHarbor signup fleet — per-IP rotation, gateway auto-add
|
||
|
||
TokenHarbor.ai (new-api based, `thk_live_*` keys, serves `claude-haiku-5.5:free` etc.).
|
||
Working stack: `C:/Users/User/tmp/` — `th_signup_full.py` (worker), `th_fleet.py` (orchestrator), `th_autoadd_gateway.py` (watcher), `th_proxy_pool.py` (proxy tester). Python311 + `PYTHONPATH=""`.
|
||
|
||
## Core mechanics (proven)
|
||
|
||
1. **Signup is a Next.js server action — plain HTTP POST never completes.** Only a real browser `form.requestSubmit()` works. Drive stock Chrome via CDP (`--remote-debugging-port`, `--proxy-server=http://ip:port`, separate `--user-data-dir` per slot).
|
||
2. **Chrome 154+: `/json/new?url=` requires HTTP PUT** (GET → 405 Method Not Allowed). Use `urllib.request.Request(..., method="PUT")`.
|
||
3. **Clear cookies before every signup** (`Network.clearBrowserCookies` + JS cookie wipe), then navigate to `/login?mode=signup`. A leftover session redirects signup to /dashboard and silently creates keys in the OLD account — labels look like different emails (email prefixes) so it *appears* to work. Symptom: all keys land in one account, balance shared. Fix: fresh cookie state per attempt; verify each new account by its own email verification flow.
|
||
4. **Fill fields with the React-native-setter trick** (Object.getOwnPropertyDescriptor on prototype `.value` + dispatch input/change), then `requestSubmit()`.
|
||
5. **Email verification**: t-online.de IMAP (`secureimap.t-online.de:993`), poll for `tokenharbor.ai/verify-email?token=` link, open it in the same tab → `?verify=success`. Pool: 17k+ mails `email:password` in working_mails.txt.
|
||
6. **Free models toggle** must be clicked on dashboard, then create API key; balance 0.00 is normal — `:free` models work at quota 0. Verify keys with a real `/v1/chat/completions` call (200 + `model: claude-haiku-5.5`), not `/v1/models`.
|
||
|
||
## Rate limiting: per-IP, hourly
|
||
|
||
Error text on the signup page: **"Too many sign-ups from this network. Please try again in an hour."** (~2–4 signups per IP before ban). Detect it in the worker (lowercased body contains `too many sign-ups` or `a bit fast`) and exit(3) so the orchestrator rotates.
|
||
|
||
## Fleet architecture (th_fleet.py)
|
||
|
||
- **IP pool** `th_proxy_pool.json`: `[{proxy, ip, banned_until}]`. Populate/test with ProxyGrab API (`test=tokenharbor.ai`) + proxy_bot `proxies.db` (table `pool`, column `proxy` — format may be `http://ip:port extra`, take `split()[0]`).
|
||
- **N slots**, each: claim free IP (soft-claim = banned_until+30s so siblings skip it) → spawn Chrome on its own port/profile → spawn worker with env `TH_CDP/TH_SUFFIX/TH_SHARD/TH_SHARDS` → tail its log.
|
||
- **Rotation triggers**: worker prints rate_limited → `ban_ip(65min)`, kill worker+chrome, respawn with next IP. Also ban preemptively after ~3 successes (IP is about to hit the limit anyway). Dead proxy (`form_not_loaded:chrome-error://chromewebdata/` twice) → ban 10min, rotate.
|
||
- **Sharding the mail pool**: `random.Random(42).shuffle(pool); pool[SHARD::SHARDS]` — deterministic seed so all workers compute identical slice boundaries; plus global dedup by reading every `th_signup_full*.jsonl` + state files at startup.
|
||
- Pacing inside worker: 150–280s between attempts; dashboard wait loop ≥80s (slow proxies).
|
||
|
||
## Gateway auto-add (th_autoadd_gateway.py)
|
||
|
||
Watcher polls all `th_signup_full*.jsonl` shards every 30s → tests each new key against TH direct → inserts into VPS new-api via `ssh root@<vps> sqlite3 one-api.db`:
|
||
|
||
- `INSERT INTO channels (...) SELECT ... FROM channels WHERE name='<template-channel>'` copying models/model_mapping/group/settings, own key+name, priority 150.
|
||
- `INSERT INTO abilities (...) SELECT` one row per model from the template channel's abilities, channel_id = new id, enabled=1.
|
||
- `systemctl restart new-api` after each add (token/model cache is stale otherwise).
|
||
- Keep seen-keys state JSON so restarts don't duplicate; write state after every add AND after every skip.
|
||
|
||
## Pitfalls
|
||
|
||
- The in-worker push endpoint (`[REDACTED-NAME].loc.cc`) does not resolve from the local machine — the SSH watcher is the real delivery path; ignore its push errors.
|
||
- new-api can return transient **503 "Провайдер временно недоступен"** on the gateway while TH-direct with the same key returns 200 → just `systemctl restart new-api`, don't rebuild channels. Always compare direct-vs-gateway before blaming keys.
|
||
- Duplicate worker processes accumulate across agent restarts — before spawning a fleet, kill by cmdline match (`th_signup_full.py`, `th_fleet.py`, `th_autoadd_gateway.py` via psutil) AND chromes by their `--user-data-dir` marker; leftover workers burn the IP rate limit and cause phantom `server_error`/`rate_limited`.
|
||
- `server_error` / `signup_timeout` from TH under load are often the IP already being throttled, not the site down — rotate before concluding anything.
|
||
- Escaping: SQL with `\"group\"` inside a bash-wrapped ssh sqlite3 command needs the doubled-quote form shown in th_autoadd_gateway.py; python `-c` one-liners with backslashes get mangled by git-bash — write a .py file instead.
|