SPA Auth/OTP Form Debugging (headless patchright/Playwright, 2026-09)

General techniques learned while automating a React SPA auth form (email → OTP code → onboarding) behind Cloudflare + Statsig. Applies to any signup/login flow (Notion, Clerk-style, etc.).

1. Gate on the API RESPONSE, never the request or a fixed sleep

The form submits via XHR (e.g. sendTemporaryPassword). Matching the request or sleeping 6s and moving on produces false "stalled" verdicts — the response may fire 10-30s later.

st = {"stp": None}
page.on("response", lambda r: st.__setitem__("stp", r.status)
        if "sendTemporaryPassword" in r.url else None)
btn.click()
for _ in range(30):
    time.sleep(1)
    if st["stp"] is not None:
        break
# st["stp"] is None => genuinely stalled; != 200 => rejected; == 200 => proceed to mailbox

2. "Loading..." button = click does nothing

SPA submit buttons can render as Loading...\nContinue while client-side gates (Statsig init, config fetch) finish. Clicking then is a no-op.

Probe before click: btn.inner_text() — if it contains "Loading", wait/re-click in a loop (2s interval, ~60s deadline).
Deep probe via page.evaluate on the raw <button>: check disabled, aria-disabled, className.
If the button never enables: check the network log for failed gate requests — e.g. exp.notion.com/v1/initialize returning ERR_ABORTED leaves the form permanently disabled. This is a client-side experiment/config gate, not a captcha. Reload-retry (3-5x) sometimes passes; if the gate endpoint is blocked at the proxy/network level, no amount of clicking helps.

3. Pre-check API responses are NOT skip signals

Auth pages often call something like getLoginOptions on load — BEFORE the user typed anything. Its hasAccount field at that point reflects page init, not the email you're about to submit. Treating an early pre-check as "mailbox already registered → skip" burned 7/12 attempts falsely in one run. Only read such fields from a response that fired AFTER your submit click.

4. Screenshots hang "waiting for fonts to load" on proxied headless pages

page.screenshot() can time out indefinitely on slow-proxy headless Chrome (font fetch never completes), killing the diagnostic script. Mitigations:

page.screenshot(timeout=10000, animations="disabled") — still can fail on fonts.
Prefer non-visual diagnostics: page.content() → save HTML; page.locator("body").inner_text(); network event log. Dump html.count("<input") and grep for markers instead of screenshots.

5. "Page renders but 0 inputs" = JS-blocked or challenge page

If page.content() is ~29KB with zero <input> tags and body text says "JavaScript must be enabled" — the browser got the noscript fallback (bot detection or JS blocked). Checks that distinguish causes:

page.title(), frame list (page.frames → all URLs), HTML markers: challenge-platform, cf-turnstile, "Just a moment".
Empty frames + noscript text + no CF markers = JS execution blocked at network level (often the proxy), not a visible captcha. Rotate proxy/exit-IP rather than fight the page.

6. Post-OTP redirect can stall on /login?redirectURL=...

After submitting the code, some SPAs fail to complete the redirect chain and sit on /login?redirectURL=<target>. Fix: re-navigate to the original target URL (page.goto(APPLY_URL)) — the session cookie is already set, so the app lets you through and the offer/flow attached to redirectURL applies. Then continue with onboarding.

7. Diagnostic script skeleton

Keep a standalone diag_apply.py that runs ONE attempt with:

page.on("response") filtered to API endpoints (drop ping/analytics noise), logging status + url.
page.on("requestfailed") → f"FAILED {url} :: {failure}" — this is where gate endpoints (statsig/exp/splunk) show up.
DOM probes via page.evaluate (button state, input inventory).
No screenshots (see #4). HTML dump to file when structure looks wrong. Run it before touching the farm loop — one clean attempt reveals which of the 6 failure modes you're in, instead of burning 20 mailboxes on a wrong hypothesis.

8. Sync vs async patchright

If your onboarding/helper functions are synchronous, drive the browser with patchright.sync_api end-to-end. Handing an async Page to sync helpers raises TypeError: argument of type 'coroutine' is not iterable deep in the flow and wastes a whole run.

Ops boundary (important)

These techniques are for forms backed by mailboxes you control (your own domains, temp-mail services you operate, Gmail aliases). If a farm workflow depends on logging into IMAP accounts belonging to third parties (combo lists of real people's mailboxes), that is unauthorized access to other people's accounts — it crosses from offer-farming into account compromise, and the pipeline should be refused/redesigned onto self-controlled mail sources.

4.8 KiB Raw Permalink Blame History