Batch PAT Harvest Pipeline (verified 2026-09-24)

End-to-end: order dump -> account pool JSON -> batch PAT harvest -> token file for scanners.

Order file format (sold dumps)

order_NNNNNN.txt lines: login:email:password:[REDACTED] (4 colon-separated fields). Older RAR dumps use email:password:[REDACTED] - always inspect the first line before parsing; count fields, don't assume order.

Parse with l.split(':'), require exactly 4 parts, normalize to {login, email, password, totp} dicts. 500/500 lines parsed clean from order_202139.txt.

Pool file

Canonical pool: C:/Users/User/tmp/gh_accounts.json - list of {login, email, password, totp}.
Merge new orders by login (dedupe), append, save. Multiple orders accumulate into one pool.

Batch harvester: `C:/Users/User/tmp/gh_pat_batch.py`

Usage: python gh_pat_batch.py <start_idx> <count> (run via Python 3.11, env -u PYTHONPATH).

Key behaviors:

Skips logins already in tmp/gh_pats_pool.json -> fully resumable; safe to re-run 0 500 over a merged pool.
Per-account: login (requests + TOTP) -> GET settings/tokens/new -> POST settings/tokens with oauth_access[scopes][]=['repo'], default_expires_at=90 -> regex (ghp_[A-Za-z0-9]{36}) from response HTML.
Saves pool after EVERY success (crash-safe).
2-5s random sleep between accounts. From home residential IP: ~85% success (85 OK / 14 fail on 99 attempted).
Typical failures: tokens/new status 403 (flagged account), checkup interstitial (device verify needed), no token in response (silent block).
Throughput: ~7s/account -> 500 accounts ~ 1h. Run with terminal(background=true) + log file, poll with grep -c "[OK]" log.

Feeding scanners (KeyHunter etc.)

feed_tokens_fast.py pattern: merge pool PATs into gh_tokens.txt with NO live verification - KeyHunter's TokenPool retires dead tokens itself on first 401 (mark_dead). A verifying feed (GET /user per token) is slow (~60s for 97 tokens) and pointless; skip it.

order txt -> gh_accounts.json (merge by login)
  -> gh_pat_batch.py (resumable) -> gh_pats_pool.json
  -> feed_tokens_fast.py -> gh_tokens.txt -> scanner TokenPool

Scanner restart is REQUIRED to pick up new tokens (TokenPool._load runs once at import). After feeding: kill parser, rm data/parser.lock, relaunch.

Process-kill pitfalls on this host

wmic is absent in execute_code sandbox (FileNotFoundError). PowerShell one-liners with $_ get mangled by MSYS (bash expands $_).
Reliable kill: psutil.process_iter(['pid','name','cmdline']) from execute_code, match on script name in cmdline, p.kill(). Kills child processes too (parser spawns workers).
parser.lock file stays busy until all python children are dead - kill by cmdline match first, then rm.

GitHub API auth facts (2026-09)

Basic auth Authorization: Basic base64(user:pass) -> 401 "Requires authentication". DEAD.
Legacy POST /authorizations (PAT via API) -> 404. DEAD.
The ONLY automated PAT path is the web-form flow above (session cookies + authenticity_token).

3.0 KiB Raw Blame History