vlad/autoreg-skills-max
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f58eafeb48979991591e8b4c53ff7b2c015091a7
autoreg-skills-max/skills/bulk-account-registration/references/form-automation-advanced.md

10 KiB
Raw Blame History

Advanced Form Automation Techniques

Techniques discovered during TJC (Dynamic Forms / NGWebSolutions) account registration automation. General patterns applicable to any web form with stubborn client-side validation or invisible reCAPTCHA.

1. Password Field Bypass via setNativeValue + dispatchEvent

The Problem

Some forms use custom JavaScript validation that checks for specific input events before accepting a password field. Standard Playwright methods fail:

Method Result
page.fill(selector, password) Field visually filled, but validation rejects ("The Password field is required")
page.type(selector, password) Same result — validation rejects
page.pressSequentially(selector, password) Same result
page.locator(selector).fill(password) Same result

The root cause: the form's JavaScript validation checks for input events dispatched by the native EventTarget.prototype.dispatchEvent API, but Playwright's fill() and type() dispatch synthetic events that the validation doesn't recognize.

The Fix

Use page.evaluate() to set the value natively and dispatch the event manually:

// In page.evaluate():
const input = document.querySelector('input#Password');
const nativeInputValueSetter = Object.getOwnPropertyDescriptor(
  window.HTMLInputElement.prototype, 'value'
).set;
nativeInputValueSetter.call(input, 'YourPasswordHere123@');
input.dispatchEvent(new Event('input', { bubbles: true }));

Why it works

  • Object.getOwnPropertyDescriptor(..., 'value').set — accesses the native property setter defined by the browser, bypassing any framework-overridden setters (React, Vue, Angular, etc.)
  • .call(input, value) — invokes the setter directly on the element, setting the internal value without triggering the framework's synthetic event system
  • dispatchEvent(new Event('input', { bubbles: true })) — fires a browser-native input event that the form's validation listener recognizes

Pattern (Python + Playwright)

async def set_value_native(page, selector, value):
    """Set form field value bypassing JS framework validation"""
    await page.evaluate(f"""
        (() => {{
            const input = document.querySelector('{selector}');
            const setter = Object.getOwnPropertyDescriptor(
                window.HTMLInputElement.prototype, 'value'
            ).set;
            setter.call(input, '{value}');
            input.dispatchEvent(new Event('input', {{ bubbles: true }}));
        }})()
    """)

⚠️ Pitfall: escaping special characters. If the password contains single quotes, backslashes, or other JS-significant characters, use JSON.stringify(value) or pass the value as a function argument:

await page.evaluate("""
    ([selector, value]) => {
        const input = document.querySelector(selector);
        const setter = Object.getOwnPropertyDescriptor(
            window.HTMLInputElement.prototype, 'value'
        ).set;
        setter.call(input, value);
        input.dispatchEvent(new Event('input', { bubbles: true }));
    }
""", [selector, password])

When to use this

  • Playwright fill() / type() / pressSequentially() all fail
  • The form shows "This field is required" even though the field visually has content
  • The form uses custom JavaScript validation (common in ASP.NET MVC, Angular, React forms)
  • Specifically for password fields — text fields rarely have this issue

Verification

After calling setValueNative, verify the field actually has the value:

document.querySelector('input#Password').value.length  // Should be > 0

2. Invisible reCAPTCHA with g-recaptcha Class

Identifying invisible reCAPTCHA

The form button has these attributes:

<button class="btn btn-default g-recaptcha"
        data-sitekey="6Lc1DN8ZAAAAAJ24AatUEzMn95wjUkjmHt65BfQi"
        data-callback="onSubmit"
        data-action="CreateAccount">
  Create Account
</button>

Key signals:

  • class="g-recaptcha" — Google's CSS class for automatic binding
  • data-callback="onSubmit" — function called when reCAPTCHA challenge succeeds
  • data-sitekey="..." — the reCAPTCHA site key
  • No type="submit" — the button is NOT a submit button; reCAPTCHA handles submission

How it works

  1. Google's reCAPTCHA JS scans the page for elements with class="g-recaptcha"
  2. When the button is clicked, reCAPTCHA intercepts the click
  3. Runs the invisible challenge (no user interaction required)
  4. On success, calls onSubmit(token) — which sets the token in a hidden field and submits the form

The onSubmit callback pattern

Typically looks like:

function onSubmit(token) {
    $("#RecaptchaToken").val(token);
    document.getElementById("createAccountForm").submit();
}

Hidden field for token

The form has a hidden input that stores the reCAPTCHA token:

[REDACTED] 


This is DIFFERENT from the standard `g-recaptcha-response` hidden field. Some custom implementations use their own field name.

### Approach 1: Solve via 2captcha + manual injection

```python
import requests, time

CAPTCHA_KEY = "your_2captcha_key"
SITEKEY = "6Lc1DN8ZAAAAAJ24AatUEzMn95wjUkjmHt65BfQi"
PAGEURL = "https://target.com/register"

# 1. Submit to 2captcha
resp = requests.get("https://2captcha.com/in.php", params={
    "key": CAPTCHA_KEY, "method": "userrecaptcha",
    "googlekey": SITEKEY, "pageurl": PAGEURL, "json": 1
}, timeout=30)
captcha_id = resp.json()["request"]

# 2. Poll for result
for i in range(20):
    time.sleep(5)
    resp = requests.get("https://2captcha.com/res.php", params={
        "key": CAPTCHA_KEY, "action": "get", "id": captcha_id, "json": 1
    }, timeout=15)
    data = resp.json()
    if data.get("status") == 1:
        token = data["request"]
        break

# 3. Inject into browser (via page.evaluate or browser_console)
await page.evaluate("""
    (token) => {
        document.getElementById('RecaptchaToken').value = token;
    }
""", token)

# 4. Trigger onSubmit
await page.evaluate("""
    onSubmit(document.getElementById('RecaptchaToken').value);
""")

Approach 2: Browser-native grecaptcha.execute()

If the reCAPTCHA script is loaded and the widget is rendered:

// In browser_console or page.evaluate:
grecaptcha.execute();

This returns a Promise that resolves to a fresh token. However, calling execute() alone does NOT trigger the onSubmit callback — it just generates a new token. You still need to either:

  • click the g-recaptcha button (which internally calls execute + callback)
  • manually call onSubmit(token) after getting the token

Approach 3: Click the button (preferred)

If the password field is filled correctly (via setNativeValue), simply clicking the g-recaptcha button triggers the full flow:

document.querySelector('.g-recaptcha').click();

The reCAPTCHA library intercepts the click, runs the challenge, and calls onSubmit on success.

Known issues

  • Migration warning: Some reCAPTCHA site keys show "Google reCAPTCHA migration" warnings in the console. This happens when the site key was created under an older Google account/API project. The key may still work for 2captcha solving but the server may reject the token.
  • Token expiry: reCAPTCHA tokens expire after ~2 minutes. If you solve via 2captcha and the form submission takes too long, get a fresh token.
  • Server-side rejection: Even with a valid token, the server may silently reject the form submission (returning the same page with 200, no error message). This usually means the server-side validation failed on something other than the captcha (e.g., duplicate username, email format, anti-forgery token mismatch).

3. ASP.NET Anti-Forgery Token Handling

ASP.NET MVC forms use __RequestVerificationToken as a hidden field AND a cookie (.__AspNetCore.Antiforgery.*). Both must match.

In the browser

The browser automatically handles this — the cookie is set on page load, and the hidden field value is embedded in the form. form.submit() sends both correctly.

Via curl/requests

  1. GET the form page to receive the cookie
  2. Extract the __RequestVerificationToken from the hidden field in the HTML
  3. POST with both the cookie and the token value
# Step 1: GET form (stores cookie)
curl -c cookies.txt -b cookies.txt -L 'https://target.com/Account/CreateAccount'

# Step 2: Extract token from HTML
TOKEN=$(grep -oP '__RequestVerificationToken" value="\K[^"]+' cookies.txt || \
        grep -oP '__RequestVerificationToken" value="\K[^"]+' response.html)

# Step 3: POST with form data
curl -c cookies.txt -b cookies.txt \
  -X POST 'https://target.com/Account/CreateAccount' \
  -d "__RequestVerificationToken=$TOKEN" \
  -d "Username=testuser" \
  -d "Password=[REDACTED]"

Without the correct token: server returns HTTP 400 Bad Request.

4. Chromium SOCKS5 Proxy (unauthenticated)

Playwright/Chromium does NOT support SOCKS5 with authentication. Workaround: set up an SSH tunnel without auth:

# Create tunnel (no auth needed)
ssh -D 1081 -N root@your-server.com

# In Playwright
browser = await playwright.chromium.launch(
    args=["--proxy-server=socks5://localhost:1081"]
)

The tunnel must be running before launching the browser. The socks5h:// prefix (DNS resolution through proxy) is not needed when using --proxy-server — Playwright handles DNS through the proxy automatically.

5. Diagnostic Flow for Form Submission Failures

When a form submits but returns the same page (no error, no redirect):

  1. Check password validation — inspect document.getElementById('Password').value.length → if 0, the password was cleared by the page reload. Use setNativeValue technique.
  2. Check reCAPTCHA token — inspect document.getElementById('RecaptchaToken').value → if empty or stale, re-solve.
  3. Check anti-forgery token — inspect document.querySelector('input[name=__RequestVerificationToken]').value → if empty, the page state reset.
  4. Check for validation errors — document.querySelector('.validation-summary-errors, .text-danger')? → if present, read the error text.
  5. Check browser console — console.clear() then console.log('ready') before action, check for JS errors after.
  6. Check server response directly — intercept via curl with the same cookies and form data to see the raw server response.
  7. Try a fresh page load — if the form state is corrupted, navigate to the URL fresh and re-fill everything.