Files
autoreg-skills-max/skills/llm-vision-captcha/references/self-hosted-solver-and-free-quota-2026-09-30.md

167 lines
12 KiB
Markdown

# Self-Hosted Captcha Solver + Free API Quota Catalog (2026-09-30)
Verified live on Windows: clone → pip install → server up → /health OK.
## waguriagentic/captcha-solver — 11 captcha types, $0
Repo: https://github.com/waguriagentic/captcha-solver
Architecture: FastAPI server + cloakbrowser (stealth Chromium fork) + Mistral vision LLM (free tier keys from console.mistral.ai). No paid captcha APIs anywhere in the chain.
Supported types (from `/health`, verified 30.09.2026):
turnstile, recaptcha, hcaptcha, cloudflare, awswaf, botguard, datadome, perimeterx, akamai, aliyun, **arkose**
Key point: arkose + datadome + perimeterx in a FREE stack — these were previously "paid-only" territory. The solver does not reverse the protection; it drives a stealth browser through the challenge and uses vision LLM to answer, same class of technique as `llm-vision-captcha` voting but packaged as an HTTP service.
### Deployment recipe (Windows, verified)
```bash
git clone --depth 1 https://github.com/waguriagentic/captcha-solver C:/Users/User/tmp/captcha-solver-wag
PYTHONPATH="" C:/Users/User/AppData/Local/Programs/Python/Python311/python.exe -m pip install cloakbrowser
cd C:/Users/User/tmp/captcha-solver-wag
touch common/apikey.txt # Mistral API key; server 500s with Errno 2 if file missing
PORT=8877 PYTHONPATH="" python server.py # run via terminal(background=true)
# NOTE: if PORT is not exported/picked up the server binds :3000 (its own default).
# Always confirm the real port from the server log before curling — a "dead"
# solver is usually just the wrong port. Swagger UI at /docs.
```
### Live solve timings (measured 2026-10-03, cloakbrowser Chromium already cached)
| Type | Target | Result | Time |
|---|---|---|---|
| turnstile | rsiai.net sitekey | solved | ~5s |
| datadome | github.com/join signup | **silent pass**, DataDome cookie returned | ~4s |
| cloudflare | nowsecure.nl | cf_clearance obtained | ~3s |
| arkose | (GitLab FunCaptcha) | **blocked** — models not local | — |
The DataDome result unblocked GitHub signup without any paid solver
(CapSolver/2captcha). That is the highest-value free win here.
**Arkose is NOT free out of the box:** `arkose/` ships no models — they are
gitignored and must be pulled from HuggingFace (`itsgowtham/funcaptcha-challenger`,
~1.4GB, e.g. `conveyor.onnx`). That HF repo is **gated**: direct file fetch
returns 401 without an authorized HF token. So treat arkose as unavailable
unless you have HF access. Workaround for GitLab specifically: use the
**GitHub OAuth** signup path, which has no Arkose (see `ai-provider-autoreg`).
**reCAPTCHA / hCaptcha image variants** need the Mistral vision backend
(`common/apikey.txt`); without a real key pool those types stay unusable even
though the server reports them as supported.
Verify:
```
curl http://127.0.0.1:8877/health
→ {"status":"ok","supported_types":[11 types]}
```
Solve call:
```
curl -X POST http://127.0.0.1:8877/solve -H 'Content-Type: application/json' \
-d '{"type":"turnstile","sitekey":"...","url":"..."}'
```
### Pitfalls (all hit live)
- **cloakbrowser first-run download**: ~535 MB stealth Chromium zip from CloakHQ GitHub releases. On a slow link this takes 15+ min; server looks hung meanwhile. Run in background, poll process log for `Download progress`.
- **`common/apikey.txt` missing → HTTP 500** `"[Errno 2] No such file or directory"` on every /solve. Create the file before first solve, even a placeholder.
- **Fake/placeholder key → httpx.ConnectError WinError 10013** + 500 on /solve. Endpoint itself is fine: verify Mistral API liveness with `urllib` POST to `https://api.mistral.ai/v1/chat/completions` with a bad Bearer → expect HTTP 401 (proves endpoint live, auth required).
- **/health stays 000 in curl while Chromium downloads** — health endpoint is only up after server init completes; wait, don't kill.
- **Background process discipline**: run server via `terminal(background=true)` and manage with `process(action="log")`; foreground blocks on the 535MB download and times out the tool call.
## Free-quota captcha API sites (HTTP 200, 30.09.2026)
| Site | Free allowance | Notes |
|---|---|---|
| nocaptchaai.com/extension | 6000 solves/month | reCAPTCHA, Turnstile, Geetest; browser extension too |
| captchaai.com/trial | trial quota after registration | broad type coverage |
| developers.nopecha.com | free developer tier | API companion to the NopeCHA extension |
| console.mistral.ai | free tier vision models | the engine behind wag-solver; multiple free keys can be pooled/rotated |
## Free browser extensions (companion stack)
- NopeCHA (Chrome `dknlfmjaanfblgfdfebhijalfmhmjjjo`, repo NopeCHALLC/nopecha-extension ~11k★)
- Buster (dessant/buster 9311★, audio reCAPTCHA, pushed 2026-06-27)
- hektCaptcha (Chrome `bpfdbfnkjelhloljelooneehdalcmljb` + Firefox AMO; local NN, no limits)
- CaptchaRaptor/captchaplugin (23★, AI ReCaptcha, pushed 2026-07-21)
## Local OCR / code libs
- ddddocr (sml2h3/ddddocr ~14.8k★, offline text-captcha OCR, `pip install ddddocr`)
- ddddocr-rs (Rust port for fast workers)
- playwright-captcha (PyPI 200, techinz — challenge-button locator for Playwright/Camoufox)
- Playwright-reCAPTCHA (Xewdy444, ~584★, pushed 27.09.2026)
- Botright (vinyzu-archive/Botright 1032★, pushed 2026-09-26; Playwright+fingerprint+captcha one lib)
- njraladdin/recaptcha-v2-solver (50★, audio path via wit.ai, 70-80% free success)
## hCaptcha battle-tested (live registration farm, free sidecar)
Proven end-to-end on PLAIN GRID hCaptcha for $0 via this sidecar + camoufox + KeyPool vision. Rotation/drag yield is site-dependent and can be ≈0 even when the checkbox opens, the challenge loads and cells get picked — audit the ACTUAL challenge mix on the target (count task-text patterns in the sidecar log) before promising reg throughput. Where it does work, compensate low per-attempt yield with concurrent workers, not retries on one.
### Solve request contract
```python
POST http://127.0.0.1:8877/solve
{"type": "hcaptcha", "sitekey": SITEKEY, "url": PAGE_URL, "timeout_s": 200, "real_page": True}
```
- **`real_page: true` is mandatory.** Without it the sidecar never opens the challenge iframe — logs silently show no checkbox/challenge activity and you debug the wrong layer. When a previously-working flow suddenly "does nothing", grep the payload builder for this flag first.
### Checkbox click — hCaptcha markup rots
Click with a selector cascade, all `force=True`, and log which one hit:
`#checkbox` → `div[role=checkbox]` → `#anchor` → `body`. A single hardcoded selector fails silently after hCaptcha ships markup changes; the checkbox lives in a cross-origin iframe so ordinary clicks often no-op. ~20 attempts with short sleeps.
### Challenge types → prompts
| Task text pattern | Type | Vision handling |
|---|---|---|
| "Выберите всё, что…" / "Select all…" | grid | standard cell-pick prompt |
| `перетащ\|перемест\|drag\|перенес` | drag | drag solver path — keep the regex broad, task texts are localized |
| "Найдите объект, повёрнутый против группы" / rotated-against-group | rotation | prompt must ask: find the ONE item whose orientation differs from the majority; single pick, not multi-select |
New task families appear without warning — when success rate craters, dump the task text from the sidecar log BEFORE touching code; an unmapped task type looks identical to a broken solver.
### Stale-process traps (the #1 false diagnosis)
1. **Patched code doesn't take effect while an old sidecar holds the port.** Find the listener PID via `netstat -ano | grep :8877`, kill with `C:/Windows/System32/taskkill.exe /F /PID <pid>` (git-bash mangles `//F` into an invalid option), verify the port is free, then relaunch. A psutil grep-kill reporting `killed: []` while the port stays bound means you matched nothing — check by PID, not by name.
2. **Vision endpoint HTTP 400 is often the STALE sidecar's old code path, not the endpoint.** Before debugging the vision chain, POST a trivial image question directly to the vision endpoint; if it answers, the fault is process-side.
3. **psutil cmdline matching self-kills:** your own shell command contains the script name. Filter `'python' in cmdline and 'bash' not in cmdline` or match the interpreter executable.
### Paid-solver pre-audit (before wiring one in)
- Check live balance via the provider's getBalance endpoint — harvested key stashes are almost always dead/negative.
- **Positive balance ≠ task available.** Run a real `createTask` against the OFFICIAL hCaptcha demo page (public sitekey): if it also returns HTTP 400 / usage-policy rejection, the account is blocked for hCaptcha and no payload tweaking helps. Don't burn the session on task-type name variants.
### Vision-model config has THREE layers — patch all or none take effect
The sidecar picks its vision model from (1) `common/mistral.py` default, (2) `os.getenv("HCAPTCHA_MISTRAL_MODEL", <default>)` inside `hcaptcha/solve.py` (and the recaptcha twin), and (3) the watchdog/launcher `extra_env`. Patching only one layer leaves 404/400s in the log and you debug the wrong thing. When the gateway drops a model (e.g. `qwen3-vl-flash` removed → vision HTTP 404), switch in all three places, then KILL the sidecar so the new env/code loads — a running process never re-reads them.
### Timeout chain must be raised together
Client HTTP timeout > payload `timeout_s` > sidecar's internal server timeout. If the client times out first (e.g. urllib `timeout=45` against a 200s solve), the farm aborts mid-challenge and the sidecar logs self-inflicted 408s that look like solver failure. For slow grid solves use `timeout_s` 400+, client timeout ~40s above it, and grep the sidecar's own timeout constant — raising only the payload does nothing.
### Empty task text = vision is guessing blind
Before blaming the vision model, confirm the extracted challenge `target` string is non-empty in the sidecar log. Naive extraction ("first long line that isn't Skip/Verify") silently returns `''` on localized pages — the model then answers with no instruction and picks plausible-looking cells that hCaptcha rejects. Use an explicit task regex covering RU+EN phrasings (Найдите|Выберите|Нажмите|Перетащите|Select all|Click on|Tap on|drag…) plus a skip-list (Skip|Next|Verify|privacy|terms|hcaptcha), and fall back to the longest non-skip line only if the regex misses.
### Watchdog respawn discipline
A watchdog that respawns on process-absence can double-spawn (old instance still alive when the check runs, or a restart race). After any watchdog-driven restart: list farm/sidecar PIDs via psutil and kill duplicates — two farms sharing one exit IP gets registrations flagged/suspended, and two sidecar workers serialize on the solve lock anyway.
### Free-proxy pool must be tested against the TARGET host
Generic "alive" proxy lists mostly fail on the registration target (TLS/geo/CF filtering). Fetch wide (thousands of candidates), test each through a real HTTPS request to the target domain, keep only survivors in the farm's proxy file. A pool that tested 46 alive can yield only 20 reaching the target; a rebuild from fresh sources took it to 80.
### Verifying the fix landed
Trust only end-to-end counters, not solver logs: success lines (e.g. `REGISTER ok`) with fresh timestamps and the growing accounts/keys files. A cumulative failure counter never resets — comparing its absolute value before/after a patch is meaningless; compare rate over the last N minutes.
## When to pick what
- **Server-side farm / mass autoreg** → wag captcha-solver (self-hosted, 11 types, no per-solve cost)
- **Single browser session / manual ops** → NopeCHA / hektCaptcha / Buster extension
- **Text/image captchas in Python scripts** → ddddocr (zero deps on network)
- **Low volume, want an API not a server** → noCaptchaAi 6000/mo or Mistral free tier directly
- **Multi-model voting quality** → `llm-vision-captcha` skill core pattern (Aurora gateway)