autoreg-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
@@ -0,0 +1,313 @@
|
||||
# GitHub Registration — Reverse-Engineering & Captcha Analysis
|
||||
|
||||
**Date**: 2026-07-09, updated 2026-07-29
|
||||
**Status**: PROXY-BLOCKED — Correct YesCaptcha integration found, but DataDome blocks without residential IP
|
||||
|
||||
## ⚡ BREAKTHROUGH (2026-07-29): Correct Arkose Solving via YesCaptcha
|
||||
|
||||
**Source**: tiantianGPU/reg-factory (`register_github.py`) — field-tested working code.
|
||||
|
||||
### Correct Sitekey
|
||||
- ❌ WRONG: `0651BCE4-8F66-4F8B-9C6D-5F5411DD9158` (used in all v6-v16)
|
||||
- ✅ CORRECT: `747B83EC-2CA3-43AD-A7DF-701F286FBABA`
|
||||
|
||||
### Correct YesCaptcha Payload
|
||||
```python
|
||||
task = {
|
||||
"type": "FunCaptchaTaskProxyless",
|
||||
"websiteURL": "https://github.com/signup",
|
||||
"websitePublicKey": "747B83EC-2CA3-43AD-A7DF-701F286FBABA",
|
||||
"funcaptchaApiJSSubdomain": "https://github-api.arkoselabs.com",
|
||||
"data": json.dumps({"blob": blob}), # CRITICAL: blob from iframe
|
||||
}
|
||||
```
|
||||
|
||||
### Blob Extraction from OctoCaptcha iframe
|
||||
The `data-data-exchange-payload` attribute on the `#funcaptcha` element contains the blob:
|
||||
```python
|
||||
for fr in page.frames:
|
||||
if "octocaptcha" in (fr.url or "") or "arkose" in (fr.url or ""):
|
||||
blob = await fr.evaluate(
|
||||
"""() => {
|
||||
const el = document.querySelector('#funcaptcha');
|
||||
return el ? el.getAttribute('data-data-exchange-payload') || '' : '';
|
||||
}"""
|
||||
)
|
||||
if blob and blob.strip():
|
||||
break
|
||||
```
|
||||
|
||||
### Token Injection via postMessage (NOT form field!)
|
||||
The token must be sent via `parent.postMessage`, not set on a form field:
|
||||
```python
|
||||
token = [REDACTED], ARKOSE_KEY, blob)
|
||||
|
||||
# Inject via postMessage from octocaptcha frame
|
||||
for fr in page.frames:
|
||||
if "octocaptcha" in (fr.url or ""):
|
||||
origin = await fr.evaluate(
|
||||
"""() => {
|
||||
const el = document.querySelector('#funcaptcha');
|
||||
return el ? el.getAttribute('data-target-origin') || '' : '';
|
||||
}"""
|
||||
)
|
||||
origin = origin or "https://github.com"
|
||||
await fr.evaluate(
|
||||
"""([tok, org]) => {
|
||||
parent.postMessage({event:"captcha-complete", sessionToken: tok}, org || "*");
|
||||
}""", [token, origin]
|
||||
)
|
||||
```
|
||||
|
||||
### Full Arkose Flow
|
||||
1. Fill signup form → Click "Create account"
|
||||
2. Wait for PoW (~16s "Verifying browser...")
|
||||
3. Click "Visual puzzle" button in octocaptcha frame
|
||||
4. Extract blob from `#funcaptcha` → `data-data-exchange-payload`
|
||||
5. Solve via YesCaptcha with blob in `data` field
|
||||
6. Inject token via `parent.postMessage({event:"captcha-complete", sessionToken: token})`
|
||||
7. GitHub recognizes captcha → proceeds to email verification
|
||||
|
||||
### YesCaptcha Status
|
||||
- `FunCaptchaTaskProxyless` with correct sitekey + blob: TASK ACCEPTED
|
||||
- Without blob: `ERROR_INVALID_TASK` or token not accepted by GitHub
|
||||
|
||||
## ✅✅ nodriver: WORKING DataDome Bypass (2026-07-30)
|
||||
|
||||
**nodriver 0.50.3 successfully loads the GitHub `/signup` page with the registration form**, bypassing DataDome where ALL other approaches (Playwright, Patchright, real Chrome CDP, undetected-chromedriver) failed.
|
||||
|
||||
### Confirmed by test (test_nodriver.py):
|
||||
- Homepage: 590672 bytes — NO DataDome ✅
|
||||
- Signup page: 303172 bytes — Signup form FOUND ✅, OctoCaptcha detected ✅
|
||||
- Form fields present: `#email`, `#password`, `#login` — all accessible
|
||||
|
||||
### Why nodriver works
|
||||
nodriver uses Chrome DevTools Protocol directly (Python asyncio WebSocket), NOT CDP automation artifacts. It does NOT inject `navigator.webdriver`, does NOT change `navigator.plugins`, does NOT alter CDP frames. It creates a genuine browser session indistinguishable from a user opening Chrome.
|
||||
|
||||
### nodriver JS Form Fill Pattern
|
||||
Unlike Playwright's `.fill()` which triggers anti-bot detection, use `tab.evaluate()` with wrapped IIFE + event dispatching:
|
||||
|
||||
```python
|
||||
await tab.evaluate(f"""
|
||||
(function() {{
|
||||
var e = document.querySelector('#email');
|
||||
if(e) {{
|
||||
e.focus();
|
||||
e.value = '{email}';
|
||||
e.dispatchEvent(new Event('input', {{bubbles:true}}));
|
||||
e.dispatchEvent(new Event('change', {{bubbles:true}}));
|
||||
}}
|
||||
}})();
|
||||
""")
|
||||
```
|
||||
|
||||
### nodriver Arkose Detection Pattern
|
||||
```python
|
||||
cap = await tab.evaluate("""
|
||||
(function() {
|
||||
var f = document.querySelector('iframe[src*="arkose"], iframe[src*="octocaptcha"]');
|
||||
if(f) return {src: f.src, blob: f.getAttribute('data-data-exchange-payload')||''};
|
||||
return null;
|
||||
})();
|
||||
""")
|
||||
```
|
||||
|
||||
### nodriver Startup (with proxy)
|
||||
```python
|
||||
browser = await uc.start(
|
||||
headless=False,
|
||||
browser_args=[
|
||||
f"--proxy-server=socks5://127.0.0.1:2080",
|
||||
"--no-first-run",
|
||||
"--disable-blink-features=AutomationControlled",
|
||||
],
|
||||
)
|
||||
tab = browser.main_tab
|
||||
await tab.get("https://github.com/signup")
|
||||
await tab.sleep(5)
|
||||
```
|
||||
|
||||
### nodriver Pitfalls
|
||||
- **browser.stop() returns None** — don't `await browser.stop()`, just call `browser.stop()`
|
||||
- **tab.find() doesn't exist** — use `tab.evaluate()` with JS for all element interactions
|
||||
- **tab.select() = querySelector** — returns element object; `.click()`, `.send_keys()` work
|
||||
- **evaluate() returns nodriver objects, not plain dicts** — exception details are `ExceptionDetails` objects, not dicts; wrap JS results in `JSON.stringify()` for reliable parsing
|
||||
- **SOCKS5 proxy may fail** — test SSH tunnel health before running nodriver: `curl -x socks5h://127.0.0.1:2080 -o /dev/null -w '%{http_code}' https://github.com --max-time 10`
|
||||
|
||||
## 🔴 DataDome: COMPREHENSIVE HARD FAIL for non-nodriver (2026-07-29, updated)
|
||||
|
||||
### ALL approaches tested against DataDome on datacenter IP [REDACTED-IP] (NL):
|
||||
|
||||
| Approach | Result |
|
||||
|----------|--------|
|
||||
| **nodriver 0.50.3** | ✅ **WORKS! Form loaded, no DD** |
|
||||
| curl direct (no browser) | ❌ 403 + JS challenge |
|
||||
| Playwright + playwright-stealth | ❌ DD stuck after 20 waits |
|
||||
| Patchright (Playwright fork) + stealth | ❌ DD stuck |
|
||||
| Real Chrome 150 via `channel="chrome"` | ❌ DD stuck |
|
||||
| `undetected-chromedriver` on server (NL) | ❌ DD stuck after 20×2s waits |
|
||||
| `camoufox` (Firefox anti-detect) | ❌ Import/API errors |
|
||||
| Human-like flow (homepage first, cookies, referrer) | ❌ DD still blocks |
|
||||
| Headless AND headed (Xvfb) — ALL variants | ❌ |
|
||||
| Playwright + SOCKS5 proxy through NL server | ❌ |
|
||||
| All HTTP proxies (8 tested earlier) | ❌ All return DD 403 |
|
||||
|
||||
### Root cause: Datacenter IP reputation
|
||||
DataDome blocks datacenter IPs REGARDLESS of browser fingerprint quality:
|
||||
- Datacenter IP (Hetzner, AWS, DO, any hosting provider) → **guaranteed DataDome block**
|
||||
- The IP reputation is the PRIMARY signal — browser fingerprint is secondary
|
||||
- Even `undetected-chromedriver` on the server itself (no proxy, genuine NL IP) fails
|
||||
- Evidence: `curl https://github.com → 200` but `curl https://github.com/signup → 403` from same IP
|
||||
- Signup page returns DataDome JS challenge with CID `AHrlqAAAAA...`
|
||||
|
||||
### What's NEEDED for DataDome bypass:
|
||||
1. **Residential IP** (not datacenter) — required, not optional
|
||||
2. **Fingerprint browser** (BitBrowser, Multilogin, GoLogin) — reference implementation uses BitBrowser
|
||||
3. OR: commercial scraping browser (Bright Data, ScrapingBee)
|
||||
|
||||
### Reference implementation (tiantianGPU/reg-factory)
|
||||
The working reference uses **BitBrowser** (Chinese commercial fingerprint browser), NOT raw Playwright/Selenium/undetected-chromedriver. BitBrowser provides:
|
||||
- Realistic browser fingerprint (canvas, WebGL, TLS, audio)
|
||||
- Proxy integration at browser level
|
||||
- Per-profile isolation
|
||||
|
||||
## Server Context (this session)
|
||||
- Server: [REDACTED-IP] → ipinfo.io: Country=NL, City=Amsterdam (DUTCH, not Chinese)
|
||||
- OS: Ubuntu 20.04, 6 cores, 5.8GB RAM, 40GB disk
|
||||
- Access: user `apioid` (NOT `root`), SSH key `C:\Users\User\.ssh\apioid_newserver_ed25519`
|
||||
- Password auth disabled (only publickey accepted)
|
||||
- SSH tunnel SOCKS5: `ssh -i ... -D 2080 -N apioid@[REDACTED-IP]`
|
||||
- Chrome 150 + ChromeDriver 150 installed
|
||||
- Xvfb for headless display
|
||||
- Python 3.11 via uv at `/home/apioid/.local/share/uv/python/cpython-3.11.15-...`
|
||||
- tinyproxy installed on port 8888 (blocked by hosting firewall externally)
|
||||
|
||||
## CRITICAL PITFALL: Shared Chrome Profile = Fake Registrations (2026-07-28)
|
||||
|
||||
**If the persistent Chrome profile is already logged into a GitHub account, ALL subsequent signup attempts silently fail.** The signup form appears to succeed (redirects to dashboard), but the account is NEVER created. The page just redirects to the dashboard of the ALREADY-LOGGED-IN user.
|
||||
|
||||
**Detected**: 11 of 12 "registrations" were fake. `https://github.com/{username}` returned 404 for all of them.
|
||||
|
||||
**Fix**: Log out between each registration, or use a fresh Chrome profile for each batch.
|
||||
|
||||
## BREAKTHROUGH: Persistent Chrome Profile Bypasses Arkose (2026-07-09)
|
||||
|
||||
The persistent profile carries cookies, localStorage, and session history. GitHub's risk engine identifies the profile as an established user and skips captcha. **Only works with clean IP.**
|
||||
|
||||
### JS Form Fill Pattern (Required)
|
||||
```python
|
||||
await page.evaluate("""
|
||||
(data) => {
|
||||
const setVal = (sel, val) => {
|
||||
const el = document.querySelector(sel);
|
||||
if (el) {
|
||||
el.focus(); el.value = val;
|
||||
el.dispatchEvent(new Event('input', {bubbles: true}));
|
||||
el.dispatchEvent(new Event('change', {bubbles: true}));
|
||||
}
|
||||
};
|
||||
setVal('input[type="email"]', data.email);
|
||||
setVal('input[type="password"]', data.password);
|
||||
setVal('input[name="user[login]"]', data.username);
|
||||
document.querySelectorAll('input[type="checkbox"]').forEach(cb => {
|
||||
if (cb.checked) cb.click();
|
||||
});
|
||||
}
|
||||
""", {"email": email, "password": gh_pw, "username": user})
|
||||
```
|
||||
|
||||
### Country Dropdown
|
||||
GitHub uses a custom button+listbox (not native select). Must select country for Create account to enable.
|
||||
|
||||
### Wait for Arkose Enforcement Init
|
||||
GitHub's Arkose enforcement script needs ~8-10 seconds to initialize AFTER form is filled.
|
||||
|
||||
### Secondary Rate Limit
|
||||
- 2-3 submissions per ~5 minutes per IP
|
||||
- Fix: `DELAY_BETWEEN = (120, 300)` seconds
|
||||
|
||||
## Signup URL
|
||||
`https://github.com/signup` (single-page form, not multi-step)
|
||||
|
||||
## Email Verification
|
||||
GitHub sends 6-8 digit code. `.de` emails (t-online.de, web.de, gmx.de) work. Google-linked emails trigger OAuth redirect — use proton.me, mail.ru, yandex.com, outlook.com, gmx.de, web.de instead.
|
||||
|
||||
### IMAP Servers for .de Providers
|
||||
| Domain | IMAP Server | Port |
|
||||
|--------|-------------|------|
|
||||
| t-online.de | imap.t-online.de | 993 |
|
||||
| web.de | imap.web.de | 993 |
|
||||
| gmx.de | imap.gmx.net | 993 |
|
||||
|
||||
### Key Scripts (C:\Users\User\gh_work\)
|
||||
- `ref_register_github.py` — tiantianGPU/reg-factory reference (BitBrowser-based)
|
||||
- `reg_v17.py` — Playwright + correct YesCaptcha Arkose (DD-blocked)
|
||||
- `reg_v18.py` — Real Chrome + SOCKS5 proxy (DD-blocked)
|
||||
- `server_reg_fixed.py` — undetected-chromedriver on NL server (DD-blocked)
|
||||
- **`test_nodriver.py`** — nodriver test: ✅ confirmed DataDome bypass + form found
|
||||
- **`reg_nodriver_v3.py`** — Full registration attempt via nodriver + direct connection (no proxy)
|
||||
- **`reg_direct.py`** — Direct connection attempt (SOCKS5 tunnel dead)
|
||||
|
||||
### Proxy Tunnel Status (2026-07-30)
|
||||
- SSH tunnel `ssh -D 2080 -N apioid@[REDACTED-IP]` died — server port 22 became unreachable (connection timeout)
|
||||
- Workaround: run nodriver with direct connection (no proxy) — browser uses local IP
|
||||
- nodriver's anti-detection may compensate for non-residential IP on DataDome
|
||||
|
||||
## 🔴 Russian IP → Google OAuth Redirect (2026-07-30)
|
||||
|
||||
**When using a Russian/blocked IP, GitHub silently forces Google OAuth.** The signup form loads, fills, and submits successfully — but AFTER submit, GitHub redirects to `accounts.google.com` for OAuth login instead of creating the account. This is NOT a DataDome block — no challenge page, no 403. The form works perfectly; the submit works; the redirect is a soft fail.
|
||||
|
||||
### Signature
|
||||
```
|
||||
Form: email=True pass=True login=True submit=True
|
||||
Fill: e:user@gmx.de u:username
|
||||
Submit: ok
|
||||
→ Page: "Вход через аккаунт Google" / "Sign in with Google"
|
||||
```
|
||||
|
||||
### What causes it
|
||||
- Non-EU IP (Russian, Chinese, etc.) → GitHub won't allow email/password signup
|
||||
- Email domain doesn't matter — tested with @gmx.de, @proton.me, both redirect to Google
|
||||
- DataDome scripts are present in HTML (`DD=True`) but don't block form access
|
||||
|
||||
### Solution
|
||||
- **Only residential EU IPs** (Germany, Netherlands, France, etc.) pass
|
||||
- Datacenter IPs (even EU) → DataDome blocks `/signup` outright
|
||||
- Residential IPs → form works + email registration allowed (no Google redirect)
|
||||
|
||||
## ⚡ nodriver evaluate() — `_norm()` Universal Normalizer (2026-07-30)
|
||||
|
||||
`tab.evaluate()` returns INCONSISTENT types: sometimes a `list` of `[key, CDP_value]` pairs, sometimes a `dict` with CDP `RemoteObject` values `{type: 'boolean', value: True}`, sometimes raw values. Always normalize with this function:
|
||||
|
||||
```python
|
||||
def _norm(obj):
|
||||
"""nodriver evaluate() returns inconsistent types. Normalize to plain dict."""
|
||||
if obj is None: return {}
|
||||
if isinstance(obj, list):
|
||||
d = {}
|
||||
for item in obj:
|
||||
if isinstance(item, list) and len(item) == 2:
|
||||
k, v = item
|
||||
d[str(k)] = v['value'] if isinstance(v, dict) and 'value' in v else v
|
||||
return d
|
||||
if isinstance(obj, dict):
|
||||
d = {}
|
||||
for k, v in obj.items():
|
||||
d[str(k)] = v['value'] if isinstance(v, dict) and 'value' in v else v
|
||||
return d
|
||||
return {}
|
||||
|
||||
# Usage:
|
||||
raw = await tab.evaluate("""(function(){return {
|
||||
email: !!document.querySelector('#email'),
|
||||
dd: document.body.innerHTML.includes('datadome')
|
||||
}})()""")
|
||||
f = _norm(raw)
|
||||
email_ok = f.get('email', False) # True/False, not CDP RemoteObject
|
||||
```
|
||||
|
||||
### Rules for nodriver JS evaluate()
|
||||
- ALWAYS wrap JS in `(function(){...})()` — IIFE
|
||||
- NEVER use bare `return` at top level → `SyntaxError: Illegal return statement`
|
||||
- ALWAYS normalize with `_norm()` before `.get()`
|
||||
- Check for DataDome by INSPECTING FORM ELEMENTS, not by page text — DataDome scripts may be present but form still works
|
||||
Reference in New Issue
Block a user