314 lines
14 KiB
Markdown
314 lines
14 KiB
Markdown
# GitHub Registration — Reverse-Engineering & Captcha Analysis
|
||
|
||
**Date**: 2026-07-09, updated 2026-07-29
|
||
**Status**: PROXY-BLOCKED — Correct YesCaptcha integration found, but DataDome blocks without residential IP
|
||
|
||
## ⚡ BREAKTHROUGH (2026-07-29): Correct Arkose Solving via YesCaptcha
|
||
|
||
**Source**: tiantianGPU/reg-factory (`register_github.py`) — field-tested working code.
|
||
|
||
### Correct Sitekey
|
||
- ❌ WRONG: `0651BCE4-8F66-4F8B-9C6D-5F5411DD9158` (used in all v6-v16)
|
||
- ✅ CORRECT: `747B83EC-2CA3-43AD-A7DF-701F286FBABA`
|
||
|
||
### Correct YesCaptcha Payload
|
||
```python
|
||
task = {
|
||
"type": "FunCaptchaTaskProxyless",
|
||
"websiteURL": "https://github.com/signup",
|
||
"websitePublicKey": "747B83EC-2CA3-43AD-A7DF-701F286FBABA",
|
||
"funcaptchaApiJSSubdomain": "https://github-api.arkoselabs.com",
|
||
"data": json.dumps({"blob": blob}), # CRITICAL: blob from iframe
|
||
}
|
||
```
|
||
|
||
### Blob Extraction from OctoCaptcha iframe
|
||
The `data-data-exchange-payload` attribute on the `#funcaptcha` element contains the blob:
|
||
```python
|
||
for fr in page.frames:
|
||
if "octocaptcha" in (fr.url or "") or "arkose" in (fr.url or ""):
|
||
blob = await fr.evaluate(
|
||
"""() => {
|
||
const el = document.querySelector('#funcaptcha');
|
||
return el ? el.getAttribute('data-data-exchange-payload') || '' : '';
|
||
}"""
|
||
)
|
||
if blob and blob.strip():
|
||
break
|
||
```
|
||
|
||
### Token Injection via postMessage (NOT form field!)
|
||
The token must be sent via `parent.postMessage`, not set on a form field:
|
||
```python
|
||
token = [REDACTED], ARKOSE_KEY, blob)
|
||
|
||
# Inject via postMessage from octocaptcha frame
|
||
for fr in page.frames:
|
||
if "octocaptcha" in (fr.url or ""):
|
||
origin = await fr.evaluate(
|
||
"""() => {
|
||
const el = document.querySelector('#funcaptcha');
|
||
return el ? el.getAttribute('data-target-origin') || '' : '';
|
||
}"""
|
||
)
|
||
origin = origin or "https://github.com"
|
||
await fr.evaluate(
|
||
"""([tok, org]) => {
|
||
parent.postMessage({event:"captcha-complete", sessionToken: tok}, org || "*");
|
||
}""", [token, origin]
|
||
)
|
||
```
|
||
|
||
### Full Arkose Flow
|
||
1. Fill signup form → Click "Create account"
|
||
2. Wait for PoW (~16s "Verifying browser...")
|
||
3. Click "Visual puzzle" button in octocaptcha frame
|
||
4. Extract blob from `#funcaptcha` → `data-data-exchange-payload`
|
||
5. Solve via YesCaptcha with blob in `data` field
|
||
6. Inject token via `parent.postMessage({event:"captcha-complete", sessionToken: token})`
|
||
7. GitHub recognizes captcha → proceeds to email verification
|
||
|
||
### YesCaptcha Status
|
||
- `FunCaptchaTaskProxyless` with correct sitekey + blob: TASK ACCEPTED
|
||
- Without blob: `ERROR_INVALID_TASK` or token not accepted by GitHub
|
||
|
||
## ✅✅ nodriver: WORKING DataDome Bypass (2026-07-30)
|
||
|
||
**nodriver 0.50.3 successfully loads the GitHub `/signup` page with the registration form**, bypassing DataDome where ALL other approaches (Playwright, Patchright, real Chrome CDP, undetected-chromedriver) failed.
|
||
|
||
### Confirmed by test (test_nodriver.py):
|
||
- Homepage: 590672 bytes — NO DataDome ✅
|
||
- Signup page: 303172 bytes — Signup form FOUND ✅, OctoCaptcha detected ✅
|
||
- Form fields present: `#email`, `#password`, `#login` — all accessible
|
||
|
||
### Why nodriver works
|
||
nodriver uses Chrome DevTools Protocol directly (Python asyncio WebSocket), NOT CDP automation artifacts. It does NOT inject `navigator.webdriver`, does NOT change `navigator.plugins`, does NOT alter CDP frames. It creates a genuine browser session indistinguishable from a user opening Chrome.
|
||
|
||
### nodriver JS Form Fill Pattern
|
||
Unlike Playwright's `.fill()` which triggers anti-bot detection, use `tab.evaluate()` with wrapped IIFE + event dispatching:
|
||
|
||
```python
|
||
await tab.evaluate(f"""
|
||
(function() {{
|
||
var e = document.querySelector('#email');
|
||
if(e) {{
|
||
e.focus();
|
||
e.value = '{email}';
|
||
e.dispatchEvent(new Event('input', {{bubbles:true}}));
|
||
e.dispatchEvent(new Event('change', {{bubbles:true}}));
|
||
}}
|
||
}})();
|
||
""")
|
||
```
|
||
|
||
### nodriver Arkose Detection Pattern
|
||
```python
|
||
cap = await tab.evaluate("""
|
||
(function() {
|
||
var f = document.querySelector('iframe[src*="arkose"], iframe[src*="octocaptcha"]');
|
||
if(f) return {src: f.src, blob: f.getAttribute('data-data-exchange-payload')||''};
|
||
return null;
|
||
})();
|
||
""")
|
||
```
|
||
|
||
### nodriver Startup (with proxy)
|
||
```python
|
||
browser = await uc.start(
|
||
headless=False,
|
||
browser_args=[
|
||
f"--proxy-server=socks5://127.0.0.1:2080",
|
||
"--no-first-run",
|
||
"--disable-blink-features=AutomationControlled",
|
||
],
|
||
)
|
||
tab = browser.main_tab
|
||
await tab.get("https://github.com/signup")
|
||
await tab.sleep(5)
|
||
```
|
||
|
||
### nodriver Pitfalls
|
||
- **browser.stop() returns None** — don't `await browser.stop()`, just call `browser.stop()`
|
||
- **tab.find() doesn't exist** — use `tab.evaluate()` with JS for all element interactions
|
||
- **tab.select() = querySelector** — returns element object; `.click()`, `.send_keys()` work
|
||
- **evaluate() returns nodriver objects, not plain dicts** — exception details are `ExceptionDetails` objects, not dicts; wrap JS results in `JSON.stringify()` for reliable parsing
|
||
- **SOCKS5 proxy may fail** — test SSH tunnel health before running nodriver: `curl -x socks5h://127.0.0.1:2080 -o /dev/null -w '%{http_code}' https://github.com --max-time 10`
|
||
|
||
## 🔴 DataDome: COMPREHENSIVE HARD FAIL for non-nodriver (2026-07-29, updated)
|
||
|
||
### ALL approaches tested against DataDome on datacenter IP [REDACTED-IP] (NL):
|
||
|
||
| Approach | Result |
|
||
|----------|--------|
|
||
| **nodriver 0.50.3** | ✅ **WORKS! Form loaded, no DD** |
|
||
| curl direct (no browser) | ❌ 403 + JS challenge |
|
||
| Playwright + playwright-stealth | ❌ DD stuck after 20 waits |
|
||
| Patchright (Playwright fork) + stealth | ❌ DD stuck |
|
||
| Real Chrome 150 via `channel="chrome"` | ❌ DD stuck |
|
||
| `undetected-chromedriver` on server (NL) | ❌ DD stuck after 20×2s waits |
|
||
| `camoufox` (Firefox anti-detect) | ❌ Import/API errors |
|
||
| Human-like flow (homepage first, cookies, referrer) | ❌ DD still blocks |
|
||
| Headless AND headed (Xvfb) — ALL variants | ❌ |
|
||
| Playwright + SOCKS5 proxy through NL server | ❌ |
|
||
| All HTTP proxies (8 tested earlier) | ❌ All return DD 403 |
|
||
|
||
### Root cause: Datacenter IP reputation
|
||
DataDome blocks datacenter IPs REGARDLESS of browser fingerprint quality:
|
||
- Datacenter IP (Hetzner, AWS, DO, any hosting provider) → **guaranteed DataDome block**
|
||
- The IP reputation is the PRIMARY signal — browser fingerprint is secondary
|
||
- Even `undetected-chromedriver` on the server itself (no proxy, genuine NL IP) fails
|
||
- Evidence: `curl https://github.com → 200` but `curl https://github.com/signup → 403` from same IP
|
||
- Signup page returns DataDome JS challenge with CID `AHrlqAAAAA...`
|
||
|
||
### What's NEEDED for DataDome bypass:
|
||
1. **Residential IP** (not datacenter) — required, not optional
|
||
2. **Fingerprint browser** (BitBrowser, Multilogin, GoLogin) — reference implementation uses BitBrowser
|
||
3. OR: commercial scraping browser (Bright Data, ScrapingBee)
|
||
|
||
### Reference implementation (tiantianGPU/reg-factory)
|
||
The working reference uses **BitBrowser** (Chinese commercial fingerprint browser), NOT raw Playwright/Selenium/undetected-chromedriver. BitBrowser provides:
|
||
- Realistic browser fingerprint (canvas, WebGL, TLS, audio)
|
||
- Proxy integration at browser level
|
||
- Per-profile isolation
|
||
|
||
## Server Context (this session)
|
||
- Server: [REDACTED-IP] → ipinfo.io: Country=NL, City=Amsterdam (DUTCH, not Chinese)
|
||
- OS: Ubuntu 20.04, 6 cores, 5.8GB RAM, 40GB disk
|
||
- Access: user `apioid` (NOT `root`), SSH key `C:\Users\User\.ssh\apioid_newserver_ed25519`
|
||
- Password auth disabled (only publickey accepted)
|
||
- SSH tunnel SOCKS5: `ssh -i ... -D 2080 -N apioid@[REDACTED-IP]`
|
||
- Chrome 150 + ChromeDriver 150 installed
|
||
- Xvfb for headless display
|
||
- Python 3.11 via uv at `/home/apioid/.local/share/uv/python/cpython-3.11.15-...`
|
||
- tinyproxy installed on port 8888 (blocked by hosting firewall externally)
|
||
|
||
## CRITICAL PITFALL: Shared Chrome Profile = Fake Registrations (2026-07-28)
|
||
|
||
**If the persistent Chrome profile is already logged into a GitHub account, ALL subsequent signup attempts silently fail.** The signup form appears to succeed (redirects to dashboard), but the account is NEVER created. The page just redirects to the dashboard of the ALREADY-LOGGED-IN user.
|
||
|
||
**Detected**: 11 of 12 "registrations" were fake. `https://github.com/{username}` returned 404 for all of them.
|
||
|
||
**Fix**: Log out between each registration, or use a fresh Chrome profile for each batch.
|
||
|
||
## BREAKTHROUGH: Persistent Chrome Profile Bypasses Arkose (2026-07-09)
|
||
|
||
The persistent profile carries cookies, localStorage, and session history. GitHub's risk engine identifies the profile as an established user and skips captcha. **Only works with clean IP.**
|
||
|
||
### JS Form Fill Pattern (Required)
|
||
```python
|
||
await page.evaluate("""
|
||
(data) => {
|
||
const setVal = (sel, val) => {
|
||
const el = document.querySelector(sel);
|
||
if (el) {
|
||
el.focus(); el.value = val;
|
||
el.dispatchEvent(new Event('input', {bubbles: true}));
|
||
el.dispatchEvent(new Event('change', {bubbles: true}));
|
||
}
|
||
};
|
||
setVal('input[type="email"]', data.email);
|
||
setVal('input[type="password"]', data.password);
|
||
setVal('input[name="user[login]"]', data.username);
|
||
document.querySelectorAll('input[type="checkbox"]').forEach(cb => {
|
||
if (cb.checked) cb.click();
|
||
});
|
||
}
|
||
""", {"email": email, "password": gh_pw, "username": user})
|
||
```
|
||
|
||
### Country Dropdown
|
||
GitHub uses a custom button+listbox (not native select). Must select country for Create account to enable.
|
||
|
||
### Wait for Arkose Enforcement Init
|
||
GitHub's Arkose enforcement script needs ~8-10 seconds to initialize AFTER form is filled.
|
||
|
||
### Secondary Rate Limit
|
||
- 2-3 submissions per ~5 minutes per IP
|
||
- Fix: `DELAY_BETWEEN = (120, 300)` seconds
|
||
|
||
## Signup URL
|
||
`https://github.com/signup` (single-page form, not multi-step)
|
||
|
||
## Email Verification
|
||
GitHub sends 6-8 digit code. `.de` emails (t-online.de, web.de, gmx.de) work. Google-linked emails trigger OAuth redirect — use proton.me, mail.ru, yandex.com, outlook.com, gmx.de, web.de instead.
|
||
|
||
### IMAP Servers for .de Providers
|
||
| Domain | IMAP Server | Port |
|
||
|--------|-------------|------|
|
||
| t-online.de | imap.t-online.de | 993 |
|
||
| web.de | imap.web.de | 993 |
|
||
| gmx.de | imap.gmx.net | 993 |
|
||
|
||
### Key Scripts (C:\Users\User\gh_work\)
|
||
- `ref_register_github.py` — tiantianGPU/reg-factory reference (BitBrowser-based)
|
||
- `reg_v17.py` — Playwright + correct YesCaptcha Arkose (DD-blocked)
|
||
- `reg_v18.py` — Real Chrome + SOCKS5 proxy (DD-blocked)
|
||
- `server_reg_fixed.py` — undetected-chromedriver on NL server (DD-blocked)
|
||
- **`test_nodriver.py`** — nodriver test: ✅ confirmed DataDome bypass + form found
|
||
- **`reg_nodriver_v3.py`** — Full registration attempt via nodriver + direct connection (no proxy)
|
||
- **`reg_direct.py`** — Direct connection attempt (SOCKS5 tunnel dead)
|
||
|
||
### Proxy Tunnel Status (2026-07-30)
|
||
- SSH tunnel `ssh -D 2080 -N apioid@[REDACTED-IP]` died — server port 22 became unreachable (connection timeout)
|
||
- Workaround: run nodriver with direct connection (no proxy) — browser uses local IP
|
||
- nodriver's anti-detection may compensate for non-residential IP on DataDome
|
||
|
||
## 🔴 Russian IP → Google OAuth Redirect (2026-07-30)
|
||
|
||
**When using a Russian/blocked IP, GitHub silently forces Google OAuth.** The signup form loads, fills, and submits successfully — but AFTER submit, GitHub redirects to `accounts.google.com` for OAuth login instead of creating the account. This is NOT a DataDome block — no challenge page, no 403. The form works perfectly; the submit works; the redirect is a soft fail.
|
||
|
||
### Signature
|
||
```
|
||
Form: email=True pass=True login=True submit=True
|
||
Fill: e:user@gmx.de u:username
|
||
Submit: ok
|
||
→ Page: "Вход через аккаунт Google" / "Sign in with Google"
|
||
```
|
||
|
||
### What causes it
|
||
- Non-EU IP (Russian, Chinese, etc.) → GitHub won't allow email/password signup
|
||
- Email domain doesn't matter — tested with @gmx.de, @proton.me, both redirect to Google
|
||
- DataDome scripts are present in HTML (`DD=True`) but don't block form access
|
||
|
||
### Solution
|
||
- **Only residential EU IPs** (Germany, Netherlands, France, etc.) pass
|
||
- Datacenter IPs (even EU) → DataDome blocks `/signup` outright
|
||
- Residential IPs → form works + email registration allowed (no Google redirect)
|
||
|
||
## ⚡ nodriver evaluate() — `_norm()` Universal Normalizer (2026-07-30)
|
||
|
||
`tab.evaluate()` returns INCONSISTENT types: sometimes a `list` of `[key, CDP_value]` pairs, sometimes a `dict` with CDP `RemoteObject` values `{type: 'boolean', value: True}`, sometimes raw values. Always normalize with this function:
|
||
|
||
```python
|
||
def _norm(obj):
|
||
"""nodriver evaluate() returns inconsistent types. Normalize to plain dict."""
|
||
if obj is None: return {}
|
||
if isinstance(obj, list):
|
||
d = {}
|
||
for item in obj:
|
||
if isinstance(item, list) and len(item) == 2:
|
||
k, v = item
|
||
d[str(k)] = v['value'] if isinstance(v, dict) and 'value' in v else v
|
||
return d
|
||
if isinstance(obj, dict):
|
||
d = {}
|
||
for k, v in obj.items():
|
||
d[str(k)] = v['value'] if isinstance(v, dict) and 'value' in v else v
|
||
return d
|
||
return {}
|
||
|
||
# Usage:
|
||
raw = await tab.evaluate("""(function(){return {
|
||
email: !!document.querySelector('#email'),
|
||
dd: document.body.innerHTML.includes('datadome')
|
||
}})()""")
|
||
f = _norm(raw)
|
||
email_ok = f.get('email', False) # True/False, not CDP RemoteObject
|
||
```
|
||
|
||
### Rules for nodriver JS evaluate()
|
||
- ALWAYS wrap JS in `(function(){...})()` — IIFE
|
||
- NEVER use bare `return` at top level → `SyntaxError: Illegal return statement`
|
||
- ALWAYS normalize with `_norm()` before `.get()`
|
||
- Check for DataDome by INSPECTING FORM ELEMENTS, not by page text — DataDome scripts may be present but form still works
|