autoreg-skills-max: initial public release (restored, redacted, hardened)

This commit is contained in:
baradok60915d52e8425
2026-10-20 02:16:32 +03:00
commit f58eafeb48
565 changed files with 60064 additions and 0 deletions

View File

@@ -0,0 +1,38 @@
# Scanner v2 Architecture
## Sources (6 total)
```
scanner.py
├── scrape_linuxdo() → linux.do (资源荟萃, 福利羊毛, AI tags)
├── scrape_nodeloc() → nodeloc.com (AI, VPS, 羊毛党 tags) + cookies
├── scrape_lolz() → lolz.live (forums 86, 54)
├── scrape_github() → github.com/search (repos, 3 queries max)
├── scrape_reddit() → old.reddit.com (r/freeAI, r/ChatGPT, r/LocalLLaMA)
└── scrape_ddg() → html.duckduckgo.com (3 queries)
```
## Flow
1. `asyncio.gather()` → все 6 источников параллельно
2. Каждый scrape возвращает `list[dict]` с ключами: source, title, url, snippet, category, topic
3. `classify_topic(title, snippet)` → эмодзи-категория
4. Дедипликация по URL (в рамках одного прогона) + глобальная (seen_hashes.json)
5. `format_post()` → HTML для Telegram
6. `send_telegram()` → HTTP POST
## Config
```python
CONFIG = {
"bot_token": "7922242368:...",
"chat_id": [REDACTED-TGID],
"state_file": "seen_hashes.json",
"log_file": "scanner.log",
"request_timeout": 30,
}
```
## Nodeloc cookies
Injected as dict into aiohttp session cookies parameter. Must be updated from browser when `_forum_session` expires.