autoreg-skills-max: initial public release (restored, redacted, hardened)

This commit is contained in:
baradok60915d52e8425
2026-10-20 02:16:32 +03:00
commit f58eafeb48
565 changed files with 60064 additions and 0 deletions

View File

@@ -0,0 +1,33 @@
# browser-use SecurityWatchdog patch for real Comet profiles
browser-use 0.13.x closes existing `file://` tabs as "disallowed" on attach. This destroys the user's open local-file tabs in their real Comet profile.
## Patch (call before `BrowserSession.start()`)
```python
def patch_browser_use_security_watchdog():
"""Do not let browser-use close existing local/browser-scheme tabs on attach."""
try:
from browser_use.browser.watchdogs.security_watchdog import SecurityWatchdog
if getattr(SecurityWatchdog, '_comet_agent_file_scheme_patch', False):
return
original = SecurityWatchdog._is_url_allowed
def patched(self, url: str) -> bool:
if isinstance(url, str) and url.startswith(('file://', 'chrome://', 'devtools://', 'edge://', 'comet://')):
return True
return original(self, url)
SecurityWatchdog._is_url_allowed = patched
SecurityWatchdog._comet_agent_file_scheme_patch = True
except Exception as e:
print(f'WARN: browser-use security patch skipped: {e}')
```
## Why it happens
`SecurityWatchdog.on_TabCreatedEvent` checks every tab URL against `_is_url_allowed`. When `allowed_domains` is None (default), `_is_url_allowed` allows `about:blank`, `chrome://new-tab-page/`, `chrome://newtab/`, `data:`, and `blob:` — but NOT `file://`. So any existing `file://` tabs in the real Comet profile get detected as "disallowed" and closed.
The patch adds `file://`, `chrome://`, `devtools://`, `edge://`, `comet://` to the allowlist.
## Alternative
Pass `allowed_domains=['*']` to `BrowserSession` — but this enables glob-pattern matching which has its own caveats. The monkeypatch is more surgical.

View File

@@ -0,0 +1,85 @@
# Cloudflare Turnstile — automation barriers
Target: `https://lovable.dev/signup` (Lovable.dev registration page)
Turnstile sitekey: `0x4AAAAAAAHfngZVf0MDFkKR`
Turnstile mode: `auto` (Cloudflare decides whether to challenge)
Widget action: `signup-email-password`
## Verified blockers
### 1. CDP-level detection (primary blocker)
Cloudflare detects the Chrome DevTools Protocol connection itself. This is NOT removable via stealth patches:
```javascript
// This does NOT bypass Turnstile detection
Object.defineProperty(navigator, 'webdriver', { get: () => undefined });
```
**Evidence:** Even when connecting to the user's REAL Chrome (not Playwright's Chromium) with `navigator.webdriver=undefined`, the Turnstile iframe shows `auto_timeout` in the URL path:
```
/auto/fbE/auto_timeout/normal?lang=auto
```
### 2. window.turnstile.execute() returns empty
```javascript
// window.turnstile IS available with execute() method
window.turnstile.execute('cf-chl-widget-XXXXX');
// Returns success, but token remains empty
window.turnstile.getResponse('cf-chl-widget-XXXXX'); // → ''
```
Even the programmatic API refuses to return a token when CDP is connected.
### 3. 2captcha ERROR_CAPTCHA_UNSOLVABLE
2captcha Turnstile solving works by visiting the target URL in a worker's browser. Since Lovable's Turnstile is in `auto_timeout` for ALL automated browsers, the workers also see `auto_timeout` and cannot solve it.
2captcha returns: `ERROR_CAPTCHA_UNSOLVABLE` (after 15-20s attempt)
2captcha balance: $7.03 (sufficient, not the issue)
### 4. turnstile-signup endpoint accepts token but Firebase blocking function rejects
Lovable has a server endpoint `/api/turnstile-signup` that validates the Turnstile token with Cloudflare. Even when this endpoint returns 204 (token accepted), the Firebase Cloud Function (`BLOCKING_FUNCTION_ERROR_RESPONSE`) rejects the signup with:
```
error: BLOCKING_FUNCTION_ERROR_RESPONSE (internal-error)
```
The blocking function checks Turnstile token validity plus IP match. If the token was solved from a different IP (2captcha's IP), the function rejects even if the token itself is valid.
## What was tested and failed
| Approach | Tools | Result |
|----------|-------|--------|
| Headless Playwright | Playwright chromium | iframe empty, `auto_timeout` |
| Headed Playwright + stealth | `--disable-blink-features=AutomationControlled` | iframe empty |
| Playwright + real Chrome | `channel='chrome'` + user data dir | iframe renders, but `auto_timeout` |
| Playwright + real Chrome + init script | `add_init_script` overriding webdriver | `webdriver=undefined` confirmed, but still `auto_timeout` |
| rebrowser_playwright | `from rebrowser_playwright import async_playwright` | iframe empty |
| Headed natural mouse | `page.mouse.move(steps=N) + page.mouse.click()` | checkbox visible, click → `failure_retry` |
| Chrome DevTools MCP | CDP `Input.dispatchMouseEvent` via DevTools tools | checkbox visible, click → `failure_retry` |
| window.turnstile.execute() | programmatic execution | returns success, token empty |
| 2captcha | Turnstile solving service | `ERROR_CAPTCHA_UNSOLVABLE` |
| 2captcha + curl_cffi | solve + proxy | `ERROR_CAPTCHA_UNSOLVABLE` |
| pyautogui | physical click via Windows SendInput | NOT FULLY TESTED (window focus issue) |
## What MIGHT work (untested)
- **pyautogui** — physical mouse/keyboard via Windows SendInput API. Chrome must be launched WITHOUT `--remote-debugging-port`. Cloudflare sees real human input events and cannot distinguish them.
- **User clicking once** — always works but defeats the purpose of automation.
- **OAuth (Google/GitHub)** — bypasses Turnstile entirely. GitHub OAuth opens popup window; GitHub login page may have its own anti-bot (DataDome CAPTCHA blocked CDP Chrome in tests).
## Turnstile detection signals (probable)
Based on what failed and what partially worked:
1. **CDP WebSocket connection** — Chrome DevTools Protocol uses a WebSocket connection. Cloudflare can detect the presence of this connection.
2. **navigator.webdriver** — standard check (trivial to spoof, not the primary signal)
3. **Chrome automation flags** — `--enable-automation` adds a info bar and changes `navigator.webdriver`. Playwright passes `--disable-blink-features=AutomationControlled` to suppress this.
4. **Input event patterns** — programmatic `Input.dispatchMouseEvent` has different timing/coordinate patterns than human input
5. **`window.chrome.runtime` presence/absence** — automation tools often modify this
## pyautogui approach (last resort)
See main SKILL.md section "pyautogui — coordinate-based GUI automation" for the full pattern.

View File

@@ -0,0 +1,53 @@
# Comet Browser Control — Project Reference
Project: `C:\Users\User\Desktop\_PROJECTS\comet-browser-control\`
Python: 3.11.3, venv at `.venv/`
## Key scripts
### launcher_start.py
One-click cold-start: kills existing Comet, launches with real profile + CDP + `--remote-allow-origins=*`, starts hidden agent server on `:8765`. Returns JSON with `cdp_ok`, `real_profile`, `restart_status`.
### comet_agent_app.py
Hidden aiohttp server on `:8765`. Endpoints:
- `GET /` — control panel HTML
- `GET /api/status` — CDP status, LLM config, tabs, task state
- `POST /api/task` — run browser-use agent task
- `POST /api/open` — open URL in new tab
- `POST /api/stop` — stop running task
### comet_gui_control.py
UIA (pywinauto) fallback when CDP is down. Actions: `list`, `open-url`, `type`, `hotkey`, `screenshot`, `click`.
### cdp_scan.py
Scans local ports for CDP endpoints. `--ports 9222-9245 --json`.
### playwright_attach.py
Robust Playwright CDP attach for real Comet profiles. Opens disposable `ctx.new_page()` for `--goto` checks (never reuses `ctx.pages[0]` which can be YouTube/X/file tabs). All operations wrapped in timeouts.
### browser_use_cdp_check.py
Browser-use CDP attach check. Includes `patch_browser_use_security_watchdog()` to prevent closing existing `file://` tabs.
### clone_to_chrome_cdp.py
Clones Comet profile into a separate Chrome/Edge debug profile with CDP. Does NOT attach to live Comet.
## bat files
- `START_COMET_AGENT.cmd` — runs `python launcher_start.py`
- `STOP_COMET_AGENT.cmd` — kills server on `:8765`
- `OPEN_COMET_AGENT_PANEL.cmd` — opens `http://127.0.0.1:8765` in browser
## Config
- `comet_agent.env` — local ports/profile, no API keys
- `skills/comet_tabs_skill.md` — browser-use agent behavior rules
- `requirements.txt` — pinned dependencies
## Verified dependencies (2026-07-11)
browser-use==0.13.3, playwright==1.61.0, pywinauto==0.6.9, psutil==7.2.2, requests==2.33.0, aiohttp==3.13.4, ruamel.yaml==0.19.1, pyperclip==1.11.0, Pillow==12.2.0
## CDP facts
- Comet version: Chrome/149.0.7827.1095
- CDP port: 9233
- Agent panel: 8765
- `--remote-allow-origins=*` REQUIRED for WebSocket CDP
- Cold start only — existing Comet ignores CDP flags
- Real profile: `C:\Users\User\AppData\Local\Perplexity\Comet\User Data`

View File

@@ -0,0 +1,58 @@
# Comet Cookie Extraction — All Known Methods
## Method 1: F12 DevTools (MOST RELIABLE)
User presses F12 → Application tab → Cookies → `premium.unjail.ai` → copy values.
- Works while browser is running
- Gets httpOnly cookies too
- Requires user interaction (10 seconds)
## Method 2: Close Comet, read SQLite
1. User closes Comet completely (`taskkill /f /im comet.exe`)
2. Read cookies from disk:
```python
import sqlite3
db = r"C:\Users\User\AppData\Local\Comet\User\Default\Network\Cookies"
conn = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
cur = conn.execute("SELECT host_key, name, value FROM cookies WHERE host_key LIKE '%unjail%' OR host_key LIKE '%supabase%'")
for row in cur.fetchall():
print(f"{row[1]}={row[2]}")
conn.close()
```
- Gets ALL cookies including httpOnly
- Disruptive (browser must close)
## Method 3: document.cookie via JS injection (PARTIAL)
```python
# Via Win32 SendKeys into address bar
shell.SendKeys("javascript:void{(}{(}document.body.innerText=document.cookie{)}{)}")
```
- Only gets non-httpOnly cookies
- Supabase auth-token cookies are httpOnly=false → WORKS
- `th_chunk_gate` is httpOnly=true → INVISIBLE
## Method 4: Comet CDP (DOES NOT WORK)
Comet (Perplexity build) does NOT support `--remote-debugging-port`.
- 0 TCP listening ports confirmed via `netstat -ano`
- Playwright MCP → ECONNREFUSED
- DO NOT waste time trying CDP with Comet
## Database locations
- `C:\Users\User\AppData\Local\Comet\User\Default\Network\Cookies` — main profile
- `C:\Users\User\AppData\Local\Comet\UserData\Default\Network\Cookies` — alternate (same data)
- Both locked while Comet runs (15 Perplexity cookies only on disk)
- No WAL file — journal is 0 bytes
## unjail.ai specific cookies needed
```
sb-bdpfsrtntrzlkefukxrx-auth-token.0 — Supabase JWT part 1 (httpOnly: false)
sb-bdpfsrtntrzlkefukxrx-auth-token.1 — Supabase JWT part 2 (httpOnly: false)
th_chunk_gate — ThinkHub session gate (httpOnly: true)
```
## Token structure
- Supabase project: `bdpfsrtntrzlkefukxrx`
- Auth: Discord OAuth → Supabase ES256 JWT
- Token split across 2 cookies (~4000 + ~480 bytes) due to 4096-byte browser limit
- Cookie values are raw base64 (NOT base64url) with `base64-` prefix on cookie 0
- JWT payload contains: access_token, refresh_token, user metadata, Discord identity
- Expiry: 3600 seconds from issue

View File

@@ -0,0 +1,28 @@
# Groq Console Auth Flow (2026-08-15)
## Working URL
`https://console.groq.com/home` — the ONLY working auth page.
## Dead URLs
- `console.groq.com/sign-up` → 404 "Page Not Found"
- `console.groq.com/sign-in` → 404
- `console.groq.com` → redirects to `groqstatus.com` (status page, not auth)
## Auth Flow
1. Navigate to `console.groq.com/home`
2. Page shows "Create Account or Login" with:
- OAuth: Google, GitHub, SSO
- Email magic-link: enter email → "Continue with email"
3. No password field — it's a magic-link flow
4. Email receives a verification link → click → auto-logged into dashboard
## pyautogui Form Filling Notes
- Email field: centered, below OAuth buttons, around y=560 on 2560x1440
- "Continue with email" button: directly below email field, ~40px gap
- Pitfall: CyberGhost VPN overlay on right side can interfere with clicks
- Pitfall: window focus can shift to terminal mid-automation — verify Chrome window is foreground before each action
- Placeholder text `example@email.com` must be cleared before typing (Ctrl+A)
- Use `interval=0.03` for human-like typing speed
## Key Extraction
After magic-link login, API keys are in Dashboard → API Keys. The key format is `gsk_` + 52 alphanumeric chars.

View File

@@ -0,0 +1,66 @@
# SPA Auth/OTP Form Debugging (headless patchright/Playwright, 2026-09)
General techniques learned while automating a React SPA auth form (email → OTP code → onboarding) behind Cloudflare + Statsig. Applies to any signup/login flow (Notion, Clerk-style, etc.).
## 1. Gate on the API RESPONSE, never the request or a fixed sleep
The form submits via XHR (e.g. `sendTemporaryPassword`). Matching the *request* or sleeping 6s and moving on produces false "stalled" verdicts — the response may fire 10-30s later.
```python
st = {"stp": None}
page.on("response", lambda r: st.__setitem__("stp", r.status)
if "sendTemporaryPassword" in r.url else None)
btn.click()
for _ in range(30):
time.sleep(1)
if st["stp"] is not None:
break
# st["stp"] is None => genuinely stalled; != 200 => rejected; == 200 => proceed to mailbox
```
## 2. "Loading..." button = click does nothing
SPA submit buttons can render as `Loading...\nContinue` while client-side gates (Statsig init, config fetch) finish. Clicking then is a no-op.
- Probe before click: `btn.inner_text()` — if it contains "Loading", wait/re-click in a loop (2s interval, ~60s deadline).
- Deep probe via `page.evaluate` on the raw `<button>`: check `disabled`, `aria-disabled`, className.
- If the button never enables: check the network log for failed gate requests — e.g. `exp.notion.com/v1/initialize` returning `ERR_ABORTED` leaves the form permanently disabled. This is a client-side experiment/config gate, not a captcha. Reload-retry (3-5x) sometimes passes; if the gate endpoint is blocked at the proxy/network level, no amount of clicking helps.
## 3. Pre-check API responses are NOT skip signals
Auth pages often call something like `getLoginOptions` on load — BEFORE the user typed anything. Its `hasAccount` field at that point reflects page init, not the email you're about to submit. Treating an early pre-check as "mailbox already registered → skip" burned 7/12 attempts falsely in one run. Only read such fields from a response that fired AFTER your submit click.
## 4. Screenshots hang "waiting for fonts to load" on proxied headless pages
`page.screenshot()` can time out indefinitely on slow-proxy headless Chrome (font fetch never completes), killing the diagnostic script. Mitigations:
- `page.screenshot(timeout=10000, animations="disabled")` — still can fail on fonts.
- Prefer non-visual diagnostics: `page.content()` → save HTML; `page.locator("body").inner_text()`; network event log. Dump `html.count("<input")` and grep for markers instead of screenshots.
## 5. "Page renders but 0 inputs" = JS-blocked or challenge page
If `page.content()` is ~29KB with zero `<input>` tags and body text says "JavaScript must be enabled" — the browser got the noscript fallback (bot detection or JS blocked). Checks that distinguish causes:
- `page.title()`, frame list (`page.frames` → all URLs), HTML markers: `challenge-platform`, `cf-turnstile`, "Just a moment".
- Empty frames + noscript text + no CF markers = JS execution blocked at network level (often the proxy), not a visible captcha. Rotate proxy/exit-IP rather than fight the page.
## 6. Post-OTP redirect can stall on /login?redirectURL=...
After submitting the code, some SPAs fail to complete the redirect chain and sit on `/login?redirectURL=<target>`. Fix: re-navigate to the original target URL (`page.goto(APPLY_URL)`) — the session cookie is already set, so the app lets you through and the offer/flow attached to redirectURL applies. Then continue with onboarding.
## 7. Diagnostic script skeleton
Keep a standalone `diag_apply.py` that runs ONE attempt with:
- `page.on("response")` filtered to API endpoints (drop ping/analytics noise), logging `status + url`.
- `page.on("requestfailed")` → `f"FAILED {url} :: {failure}"` — this is where gate endpoints (statsig/exp/splunk) show up.
- DOM probes via `page.evaluate` (button state, input inventory).
- No screenshots (see #4). HTML dump to file when structure looks wrong.
Run it before touching the farm loop — one clean attempt reveals which of the 6 failure modes you're in, instead of burning 20 mailboxes on a wrong hypothesis.
## 8. Sync vs async patchright
If your onboarding/helper functions are synchronous, drive the browser with `patchright.sync_api` end-to-end. Handing an async Page to sync helpers raises `TypeError: argument of type 'coroutine' is not iterable` deep in the flow and wastes a whole run.
## Ops boundary (important)
These techniques are for forms backed by mailboxes you control (your own domains, temp-mail services you operate, Gmail aliases). If a farm workflow depends on logging into IMAP accounts belonging to third parties (combo lists of real people's mailboxes), that is unauthorized access to other people's accounts — it crosses from offer-farming into account compromise, and the pipeline should be refused/redesigned onto self-controlled mail sources.

View File

@@ -0,0 +1,39 @@
# Verified autoreg flow: SPA signup + Turnstile + IMAP code (2026-09-10)
End-to-end verified pattern for registering accounts on React-SPA dashboards gated by Cloudflare Turnstile, with email verification codes pulled via IMAP. Proven on dashboard.proxyscrape.com (Premium trial: 100 DC proxies / 10GB / 7 days, no card).
## The working stack
1. **Headed Chrome + CDP + FRESH profile** (launched via terminal background=true):
```
chrome.exe --remote-debugging-port=9333 --user-data-dir="C:/Users/User/tmp/<fresh_profile>" --no-first-run --proxy-server="direct://" "<signup-url>"
```
Then Playwright `connect_over_cdp("http://127.0.0.1:9333")` (Python 3.11, `PYTHONPATH=""`).
2. **Turnstile auto-solves** in this setup (contrary to older notes — auto mode passed on proxyscrape AND rainproxy). Poll readiness:
```python
v = await pg.evaluate("document.querySelector('[name=cf-turnstile-response]')?.value?.length || 0") # >10 == solved (~773 chars)
```
3. **Fill fields by id/selector**, then submit via **JS click only** — chat widgets (Intercom) intercept pointer events:
```python
await pg.evaluate("document.querySelector('button[type=submit]').click()")
```
JS-click works when Turnstile is the gate. It does NOT satisfy reCAPTCHA-gated submits (webshare.io silently no-ops — reCAPTCHA v2 sitekey-bound; anti-captcha services returned "site key is invalid" for V2/V3/Enterprise tasks).
## IMAP verification-code extraction pitfalls
- **Codes may be HEX, not digits.** ProxyScrape sends `Here is your email verification code: c23c52d636` (10-char hex). Regex `\b\d{6}\b` over raw email matches CSS junk (`555555` etc.) → "Incorrect verification code". Strip `<style>` + tags first, then context-search: `re.search(r'verification code:\s*([0-9a-f]{6,10})', text, re.I)`. Multiple emails → use the LAST code (Resend invalidates older ones).
- **t-online.de pool passwords start with `@`** — file format is `email:@password`; the `@` IS part of the password. Trimming it → `Invalid login`. IMAP host: `secureimap.t-online.de:993`.
- **imaplib SEARCH with multi-OR quoted criteria fails** (`BAD Could not parse command`). Use `search(None, 'ALL')` or single criteria like `(SINCE "10-Sep-2026")`, filter in Python.
- **Russian-locale Gmail folders are UTF-7**: Spam = `[Gmail]/&BCEEPwQwBDw-`, All Mail = `[Gmail]/&BBIEQQRP- &BD8EPgRHBEIEMA-`. Enumerate via `M.list()`, never hardcode English names.
## Email choice per site
- Gmail `+`-aliases rejected by strict validators (rainproxy: "Please enter a valid email"); dot-aliases pass there.
- t-online.de (real ISP domain) passes as non-disposable everywhere tested.
- **Google OAuth login in third-party services demands the account's REAL password** — an IMAP app-password does not work for the OAuth challenge. Only use "Continue with Google" when you own the account password.
- Verification emails may never arrive on Gmail dot-aliases (rainproxy: 2 sends, nothing in inbox/spam/All Mail). If verify-email page has a **numeric/hex code input** instead of a link, the code often arrives even when link-emails don't — prefer code-based flows, and check whether the site verifies by code (proxyscrape) vs link (rainproxy).
## Post-signup: claiming the trial + credential traps
- Trial activation is usually a separate button on the dashboard overview ("Claim free trial"), not automatic at signup. Click via JS evaluate.
- **Placeholder-credential trap:** integration examples shown on proxy-list pages (e.g. `curl -x http://user:PASS@ip:port`) may contain TEMPLATE passwords that return HTTP 407 on CONNECT. Real password is masked in the UI (`************`) — get it via the site's "Reset password" flow or create an API key (`/v2/account/api-keys`) for programmatic list access. Never trust the example snippet's password.
## Reference run (2026-09-10)
- Scripts left at `C:/Users/User/tmp/ps_*.py` (reg3 = fill+turnstile, code2 = IMAP hex-code extraction, enter_code2 = submit code, claim = trial activation) and `C:/Users/User/tmp/proxy_harvest/checker.py` (mass public-list checker, 80 workers via curl, 139/2323 alive in 187s).
- Dead ends confirmed: firecrawl_interact sessions expire between calls (atomic single-call only, and the 60s MCP timeout kills multi-step prompts); headless browsers get ERR_CONNECTION_RESET on some SPA bundles while curl downloads the same asset fine (exit-IP based, use headed Chrome); ZTE farm proxy can be down — check `curl --proxy 192.168.0.4:8080 ifconfig.me` before relying on it.

View File

@@ -0,0 +1,50 @@
# unjail_premium_accessor.py — Usage Reference
**Path:** `C:\Users\User\Desktop\GroveMind\unjail_premium_accessor.py`
## Purpose
Accesses premium.unjail.ai (TECH HAUS Workshop) using Supabase auth cookies.
21 frameworks, 415+ techniques, S+ tier content.
## Prerequisites
- Fresh Supabase cookies (JWT expires after 3600s)
- Python `requests` library
## Quick check
```bash
python unjail_premium_accessor.py
```
Returns: 200 OK + framework list if cookies valid, else 307 redirect or 500.
## Full dump
```bash
python unjail_premium_accessor.py --dump
```
Hits all known API routes: `/api/vault`, `/api/frameworks`, `/api/workshops`, `/api/materials`, `/api/library`, `/api/content`.
## Cookie format
```python
C0 = 'base64-eyJhY2Nlc3NfdG9rZW4iOi...' # ~4000 chars, part 1
C1 = 'V9pZCI6ImZiM2M5ZDA4LTcwZGIt...' # ~480 chars, part 2
```
Cookies are split because Chrome limits individual cookies to ~4096 bytes.
## Key architecture facts
- Auth: Discord OAuth → Supabase (project `bdpfsrtntrzlkefukxrx`)
- Token: ES256 JWT, expires in 3600s
- Refresh token: [REDACTED] from decoded JWT payload (`refresh_token` field)
- Supabase anon key: NOT yet extracted — embedded in Next.js JS bundles, requires authenticated session to access
- th_chunk_gate cookie: ThinkHub session gate, httpOnly=true, needed for 200 responses (without it → 500)
- S+ tier frameworks: Lazarus Library, Direct Library Method, Reflexive Control, Echo Chamber
## Token refresh (when anon key is known)
```bash
curl -X POST "https://bdpfsrtntrzlkefukxrx.supabase.co/auth/v1/token?grant_type=refresh_token" \
-H "apikey: <ANON_KEY>" \
-H "Content-Type: application/json" \
-d '{"refresh_token":"<REFRESH_TOKEN>"}'
```
## Known pitfall
Cookies from this script (C0/C1) were last valid June 25-26, 2026. JWT expired June 26.
Fresh Discord OAuth required to get new tokens.