54 lines
2.0 KiB
Markdown
54 lines
2.0 KiB
Markdown
# Strict Proxy Validation — Pitfall & Fix
|
|
|
|
## The Bug
|
|
|
|
A raw-socket HTTP proxy checker was counting HTTP 400 Bad Request as "live"
|
|
because the validation condition was too loose:
|
|
|
|
```python
|
|
# BROKEN — matches 400, 301, garbage responses
|
|
if data.startswith(b"HTTP/") and b"Example Domain" in data \
|
|
and (" 200 " in first or " 301 " in first or " 302 " in first):
|
|
return True, first
|
|
```
|
|
|
|
The `" 200 " in first` check was sub-string matching on the raw status line
|
|
and could false-match on responses where "200" appeared elsewhere (e.g. in
|
|
a Content-Length header, or in a multi-chunk concatenated response).
|
|
|
|
Result: 905 "live" proxies, but 95% were actually HTTP 400 errors from
|
|
servers that accepted TCP but rejected the plain-HTTP request.
|
|
|
|
## The Fix
|
|
|
|
Parse the HTTP status code explicitly from the status line and ONLY accept `200`:
|
|
|
|
```python
|
|
# STRICT — only HTTP 200 with target body
|
|
first_line = data.split(b"\r\n", 1)[0].decode("latin1", "ignore")
|
|
if not first_line.startswith("HTTP/"):
|
|
return False, f"not HTTP: {first_line[:80]}"
|
|
parts = first_line.split(" ")
|
|
status = parts[1] if len(parts) > 1 else "?"
|
|
if status != "200":
|
|
return False, f"HTTP {status}: {first_line[:120]}"
|
|
if b"Example Domain" not in data:
|
|
return False, f"200 but wrong body"
|
|
return True, first_line
|
|
```
|
|
|
|
## Checklist for Any Proxy Checker
|
|
|
|
1. Parse the status code as an integer/number, not a substring match
|
|
2. Require the target-specific body marker (e.g. `Example Domain` for example.com)
|
|
3. Read ALL available data in a loop, not just 1-2 `recv()` calls
|
|
4. Wrap the entire socket operation in try/except — connection errors are NOT live proxies
|
|
5. Log the actual status line for every "live" result so false positives are visible
|
|
|
|
## Incident
|
|
|
|
Date: 2026-07-09
|
|
Script: `scripts/proxy_checker.py` (hermes root)
|
|
Symptom: 905/2295 "live", but 95% showed `HTTP/1.1 400 Bad Request` in output
|
|
Root cause: substring match on status line + incomplete body verification
|
|
After fix: 507/2295 truly live (all HTTP 200 + Example Domain) |