autoreg-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
baradok60915d52e8425
87
skills/grok-x-parser-farm/SKILL.md
Normal file
87
skills/grok-x-parser-farm/SKILL.md
Normal file
@@ -0,0 +1,87 @@
|
||||
---
|
||||
name: grok-x-parser-farm
|
||||
description: Parse X/Twitter free via Grok farm + grok2api gateway.
|
||||
version: 1
|
||||
author: ENI/Gothbreach
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [x-twitter, parser, grok, autoreg, gateway, x_search]
|
||||
related_skills: [xai-grok-console-registration, ai-provider-autoreg, x-content-advisor]
|
||||
---
|
||||
|
||||
# Grok X-Parser Farm
|
||||
|
||||
Free Twitter/X parsing pipeline: autoreg Grok accounts → grok2api gateway pool → live X search.
|
||||
Full workflow verified end-to-end 2026-09-20. Repo: github.com/MeshFinancial/grok-x-farm.
|
||||
|
||||
## When to use
|
||||
|
||||
- Need real X posts (handle/date/text/likes/URL) without paying Twitter API $200/mo
|
||||
- Scaling a free Grok model pool for chat/search/image
|
||||
- Monitoring X for keywords continuously
|
||||
|
||||
## Local deployment (this machine)
|
||||
|
||||
- Gateway: `C:\Users\User\grok2api-deploy\grok2api\grok2api.exe` on :8000 (restart: run binary with `--config config.yaml`)
|
||||
- Admin creds: `C:\Users\User\grok2api-deploy\SECRETS.local.txt`
|
||||
- Client key: `C:\Users\User\grok2api-deploy\g2a_key.txt`
|
||||
- Autoreg: `C:\Users\User\grok-reg\grok-auto` (curl_cffi==0.13.0 pinned, patchright chromium installed)
|
||||
- Parser: `C:\Users\User\grok2api-deploy\x_parser.py`, replenish: `replenish.py`
|
||||
- Pool: 4 accounts × Web+Build = 8 records, 9 models (grok-4.5/4.6/chat-fast/composer/imagine)
|
||||
- Unified CLI: `C:\Users\User\tmp\grok-x-farm\farm.py` — doctor / reg --count N / import / keys / parse / crawl (config farm.config.json: email provider, captcha mode, proxy mode+geo_whitelist)
|
||||
|
||||
## Repo layout v3 (2026-09-21, commit 5d21980)
|
||||
|
||||
- **ДВА каталога**: `tmp/grok-x-farm` = рабочая копия (внутри неё НЕТ своего .git — `git rev-parse --show-toplevel` показывает `C:/Users/User/tmp`!), `tmp/grok-x-farm-main` = чистый git-зеркало для push. Workflow: правки в рабочем → `cp` изменённых файлов в main-зеркало → push скриптом `tmp/push_grokfarm_v3.py` (Git Data API, токен `tmp/meshfin_pat.json`).
|
||||
- v3: `gateway.tool_model`/`reasoning_model` = **grok-4.6** (4.5 больше не дефолт); `geo_whitelist` = US/EU (UA убран по требованию Влада); captcha: modes `free_browser/yescaptcha/capsolver/nopecha/2captcha` + `fallback_order`, `solver_timeout_sec`, `retry_per_solver`, `turnstile_sitekey` настраиваемый; secrets: YESCAPTCHA_KEY, CAPSOLVER_KEY, NOPETCHA_KEY, TWOCAPTCHA_KEY.
|
||||
- Дашборд `dashboard/panel.py` (:порт в farm.config) — ENUM/INT/STR_KEYS + INT_RANGES валидация; settings-панель: models, farm.count_per_run/threads, gateway rpm/max_concurrent, captcha-тюнинг. После правки ENUM — обновить `tests/test_panel_offline.py` (bad-mode фикстура должна остаться НЕвалидной).
|
||||
- Происхождение = ФОРК, не с нуля: gateway chenyme/grok2api (7.7k★ MIT), autoreg AaronL725/grok-register (2.2k★ MIT) — зафиксировано в README `## Credits / upstream`. Влад требует именно такую подачу.
|
||||
- AUDIT.md/DOSSIER_AUDIT.md — исторические записи баг-фиксов, упоминания старых моделей/гео там НЕ переписывать (это пруфы прошлых проверок).
|
||||
|
||||
## Parse X — the ONLY working method
|
||||
|
||||
```
|
||||
POST http://127.0.0.1:8000/v1/chat/completions
|
||||
Authorization: Bearer g2a_xxx
|
||||
{"model": "grok-chat-fast",
|
||||
"messages": [{"role":"user","content":"Use your live X search. Find N most recent posts about QUERY from last D days. Reply ONLY with JSON array: [{handle,date,text,likes,url}]"}]}
|
||||
```
|
||||
|
||||
Speed 8-10s, ~500 tokens. Web pool has NATIVE live X search. NOT `tools:[{type:x_search}]` —
|
||||
Build upstream has no server-side search (num_server_side_tools_used always 0).
|
||||
Verify results via `api.fxtwitter.com/{user}/status/{id}` (4/4 matched in tests).
|
||||
|
||||
## Replenish pool
|
||||
|
||||
```bash
|
||||
cd C:\Users\User\grok2api-deploy && python replenish.py --count 5
|
||||
# reg (tmail + free Turnstile) → import SSO → convert Web→Build; idempotent via imported_sso.txt
|
||||
```
|
||||
|
||||
Turnstile solved FREE: patchright + manual turnstile.render, sitekey `0x4AAAAAAAhr9JGVDZbrZOo0`. ~141-210s/account.
|
||||
|
||||
## Pitfalls (all hit in practice)
|
||||
|
||||
1. curl_cffi 0.14+ broken on Windows → pin 0.13.0
|
||||
2. RU egress geo-blocked by xAI → US/EU residential or clean DC
|
||||
3. Signup page in RU breaks AaronL725 selectors → patch Russian variants into `registration_browser.py` scoreEntry (2 places)
|
||||
4. credentialEncryptionKey must be base64(32 bytes) — `openssl rand -base64 32`
|
||||
5. CPA refresh tokens die ~1 month (invalid_grant); SSO stays alive → import by SSO
|
||||
6. Web→Build conversion unlocks grok-4.6 (v3 default, 2026-09-21): `POST /api/admin/v1/accounts/web/convert-to-build {"all":true,"strategy":"missing"}`
|
||||
7. Gateway 502 transient → retry with backoff (farm.py has 3 attempts built in)
|
||||
8. GitHub push: git-remote-https missing on this machine → Git Data/Contents API (empty repo: Contents API PUT first file bootstraps main branch; blobs/trees return 409 on empty repo)
|
||||
9. X filters explicit grey-market queries ('cvv shop', 'account selling' → 0 results, 3/13 in test run); use neutral wording
|
||||
10. Model dates drift outside requested window (1/105) → post-filter by date field
|
||||
11. Tool use (function calling) = Build pool ONLY (grok-4.5/4.6, verified full cycle: tool_calls → result → answer). Web pool grok-chat-fast rejects tools 400 invalid_tools
|
||||
12. doctor geo-check: ip-api countryCode field, not country name
|
||||
|
||||
## Admin API cheatsheet
|
||||
|
||||
```
|
||||
POST /api/admin/v1/auth/login {"username","password"} → data.tokens.accessToken
|
||||
POST /api/admin/v1/accounts/web/import multipart file=<SSO text>
|
||||
POST /api/admin/v1/accounts/web/convert-to-build
|
||||
POST /api/admin/v1/client-keys {"name":"x"} → data.secret (g2a_...)
|
||||
GET /v1/models Bearer g2a_...
|
||||
```
|
||||
Reference in New Issue
Block a user