182 lines
6.0 KiB
Markdown
182 lines
6.0 KiB
Markdown
# CDP Approach — system Chrome via DevTools Protocol
|
|
|
|
When Playwright's bundled Chromium crashes on Windows (exit code 2147483651 = STATUS_BREAKPOINT), use the system Chrome via CDP instead of `launch_persistent_context`.
|
|
|
|
## Why this works
|
|
|
|
Playwright's `chromium-1208` (bundled with playwright 1.58.0) crashes on Windows on `launch_persistent_context`. The system Chrome (151.0.7922.109) works fine when launched manually with `--remote-debugging-port=9222`.
|
|
|
|
## Launch Chrome
|
|
|
|
```bash
|
|
"C:\Program Files\Google\Chrome\Application\chrome.exe" \
|
|
--remote-debugging-port=9222 \
|
|
--user-data-dir="C:/Users/User/gh_work/chrome_profile" \
|
|
--no-sandbox --disable-gpu --disable-software-rasterizer \
|
|
--disable-features=Crashpad --no-first-run --no-default-browser-check
|
|
```
|
|
|
|
Key: `--user-data-dir` must point to the existing warmed-up profile.
|
|
|
|
## CDP Control Script
|
|
|
|
Control via websocket + `Runtime.evaluate`:
|
|
|
|
```python
|
|
import json, urllib.request
|
|
from websocket import create_connection
|
|
|
|
CDP = "http://127.0.0.1:9222"
|
|
|
|
# Create new tab
|
|
req = urllib.request.Request(f"{CDP}/json/new?url=about:blank", method="PUT")
|
|
data = json.loads(urllib.request.urlopen(req).read())
|
|
ws = create_connection(data["webSocketDebuggerUrl"], timeout=15)
|
|
|
|
# Evaluate JS
|
|
ws.send(json.dumps({
|
|
"id": 1, "method": "Runtime.evaluate",
|
|
"params": {"expression": "document.title", "returnByValue": True}
|
|
}))
|
|
|
|
# Navigate
|
|
ws.send(json.dumps({
|
|
"id": 2, "method": "Page.navigate",
|
|
"params": {"url": "https://github.com/signup"}
|
|
}))
|
|
|
|
# Close tab
|
|
urllib.request.urlopen(f"{CDP}/json/close/{tab_id}")
|
|
```
|
|
|
|
## Python 3.14 isolated mode
|
|
|
|
Hermes venv pollutes `sys.path` with playwright paths that cause `PermissionError`. Use `python -I`:
|
|
|
|
```bash
|
|
/c/Users/User/AppData/Local/Python/pythoncore-3.14-64/python.exe -I script.py
|
|
```
|
|
|
|
## UTF-8 encoding fix
|
|
|
|
Windows console uses cp1252. Force UTF-8 at script start:
|
|
|
|
```python
|
|
import sys, io
|
|
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
|
|
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')
|
|
```
|
|
|
|
## Verified
|
|
|
|
- 2026-08-10: Chrome 151 on CDP port 9222, registration flow works through CDP
|
|
- Python 3.14.3 with `-I` flag, playwright import is clean
|
|
|
|
## Critical: Path + Full URL Matching (BOTH required)
|
|
|
|
GitHub's signup flow uses `return_to=` query params extensively. The verification page URL is:
|
|
|
|
`https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Faccount_verifications`
|
|
|
|
**Path alone is `/login` — which does NOT contain "account_verif".** Must check BOTH:
|
|
|
|
```python
|
|
from urllib.parse import urlparse
|
|
|
|
def get_path(url_str):
|
|
return urlparse(url_str).path.rstrip("/")
|
|
|
|
path = get_path(url)
|
|
url_lower = url.lower()
|
|
|
|
# still_on_verify: check BOTH path and full URL
|
|
still_on_verify = ("account_verif" in path and "launch" not in path) or \
|
|
("account_verif" in url_lower and "launch" not in url_lower)
|
|
```
|
|
|
|
**Time-limit still_on_verify:** in the post-OTP loop, only skip for first 10s of the 30s window:
|
|
|
|
```python
|
|
deadline = time.time() + 30
|
|
while time.time() < deadline:
|
|
url = str(cdp_eval(cdp, "window.location.href")[0])
|
|
path = get_path(url)
|
|
url_lower = url.lower()
|
|
|
|
still_on_verify = ("account_verif" in path ...) or ("account_verif" in url_lower ...)
|
|
if still_on_verify and time.time() < deadline - 20: # first 10s only
|
|
time.sleep(1)
|
|
continue
|
|
|
|
# After 10s, evaluate other signals:
|
|
# - has_session cookie?
|
|
# - on /dashboard or /welcome? → SUCCESS
|
|
# - on /login with success flash? → needs login
|
|
# - code error message? → failure
|
|
```
|
|
|
|
This applies to ALL CDP URL checks:
|
|
- After submit: check BOTH `path` and full URL for `account_verif` or `launch`
|
|
- Post-OTP: time-limit the still_on_verify check, then evaluate other signals
|
|
- Rate limit detection: check page body text, not URL
|
|
|
|
## Rate Limit Behavior (2026-08-10)
|
|
|
|
- Secondary rate limit persists 1+ hour on same IP
|
|
- 15-minute wait is insufficient
|
|
- Fields on rate-limited page: all `querySelector` return null (email, password, login all `false`)
|
|
- Page stays on `/signup` but form is effectively dead
|
|
- **Must rotate IP** — waiting on same IP is wasted time
|
|
|
|
## OctoCaptcha Lock — Accounts Born-Dead (2026-08-10)
|
|
|
|
GitHub signup form has `<input name="octocaptcha-token" required>` AND honeypot fields (`required_field_*`). Submitting without the token:
|
|
- Account IS created (GitHub sends verification email)
|
|
- Account is immediately flagged/locked
|
|
- Login fails with "Incorrect username or password" even through real Chrome
|
|
- Verified: manual browser login with just-created credentials also fails
|
|
|
|
Solution: solve octocaptcha before form submission, or use a pre-warmed profile that bypasses the captcha entirely.
|
|
|
|
## CDP Debug Pattern
|
|
|
|
Dump page state via `Runtime.evaluate` to diagnose form issues:
|
|
|
|
```javascript
|
|
JSON.stringify({
|
|
title: document.title,
|
|
url: window.location.href,
|
|
fields: Array.from(document.querySelectorAll('input')).map(el =>
|
|
el.id + '|' + (el.name||'') + '|' + (el.type||'') + '|' + (el.getAttribute('inputmode')||'')
|
|
),
|
|
body: document.body ? document.body.innerText.substring(0, 300) : ''
|
|
})
|
|
```
|
|
|
|
Log before and after each form submission to trace what GitHub actually sees vs what the script filled.
|
|
|
|
## `requestSubmit()` vs `submit()`
|
|
|
|
`form.submit()` bypasses JavaScript event handlers (including octocaptcha validation, CSRF token injection, and anti-bot checks). Use:
|
|
|
|
```javascript
|
|
form.requestSubmit ? form.requestSubmit() : form.submit()
|
|
```
|
|
|
|
`requestSubmit()` triggers proper HTML5 form validation and all event handlers, same as a real user click.
|
|
|
|
## `Runtime.evaluate` Message Format
|
|
|
|
```python
|
|
cdp["ws"].send(json.dumps({
|
|
"id": cdp["id"],
|
|
"method": "Runtime.evaluate",
|
|
"params": {
|
|
"expression": code,
|
|
"returnByValue": True, # return JSON, not RemoteObject
|
|
"awaitPromise": True # wait for async completion
|
|
}
|
|
}))
|
|
```
|
|
|
|
Without `awaitPromise: true`, the call returns immediately without waiting for expression completion. With `returnByValue: true`, the result is a plain JSON value instead of a RemoteObject reference. |