87 lines
2.8 KiB
Markdown
87 lines
2.8 KiB
Markdown
# Scanner v3 Architecture
|
|
|
|
## Files
|
|
|
|
```
|
|
C:\Users\User\ai-abuse-scanner\
|
|
├── scanner.py # Main scanner (508 lines, 19 sources)
|
|
├── tg_monitor.py # Telegram monitor (Telethon, optional)
|
|
├── tg_session_scanner.py # TG session scanner (maintg MCP)
|
|
├── run.bat # Windows manual run
|
|
├── seen_v3.json # State: seen URL hashes
|
|
├── scanner_v3.log # Runtime log
|
|
└── README.md # Documentation
|
|
```
|
|
|
|
## Server deployment
|
|
|
|
```
|
|
/root/ai_scanner.py # Deployed on both DO droplets
|
|
/root/ai_scanner_cron.log # Cron output
|
|
```
|
|
|
|
## Cron jobs
|
|
|
|
| Job ID | Name | Schedule | Where |
|
|
|--------|------|----------|-------|
|
|
| fcb5b37426fa | ai-freebie-scanner | 15min | Hermes (local) |
|
|
| 400504e7dc83 | tg-mainsession-ai-scanner | 10min | Hermes (maintg MCP) |
|
|
| - | crontab | 15min | gothbreach-abuse-1 (SGP) |
|
|
| - | crontab | 15min | eni-nyc1 (NYC) |
|
|
|
|
## Source functions
|
|
|
|
Each source is an async function: `async def src_<name>(session) -> list[dict]`
|
|
|
|
Return format per item:
|
|
```python
|
|
{
|
|
"src": "🐧 Linux.do", # Source emoji + name
|
|
"title": "Topic title", # Clean title
|
|
"url": "https://...", # Full URL
|
|
"cat": "category", # Optional category
|
|
"emoji": "🎮", # Category emoji
|
|
"snippet": "...", # Optional text snippet
|
|
}
|
|
```
|
|
|
|
## Key functions
|
|
|
|
- `match(text)` — keyword filter (returns True if AI-related)
|
|
- `cat_emoji(title)` — classify topic by emoji
|
|
- `safe_html(text)` — aggressive HTML escape for Telegram
|
|
- `build_post(items)` — format @ikhdev1-style post
|
|
- `send_tg(text)` — send to Telegram via HTTP API
|
|
- `is_new(url)` / `mark(url)` — deduplication via MD5
|
|
|
|
## Telegram delivery
|
|
|
|
- Bot: 1234567890:AAEXAMPLE_TOKEN_REPLACED
|
|
- Chat: [REDACTED-TGID] (@[REDACTED-NAME])
|
|
- Format: HTML, parse_mode=HTML, max 4000 chars
|
|
- If >4000: truncated with "...(обрез)"
|
|
|
|
## DO servers
|
|
|
|
API token: [REDACTED]
|
|
|
|
List droplets: `curl -H "Authorization: Bearer <token>" https://api.digitalocean.com/v2/droplets`
|
|
|
|
SSH: `ssh -i ~/.ssh/do_access -o ControlMaster=no -o ControlPath=none -o StrictHostKeyChecking=no root@<IP>`
|
|
|
|
Reset password: `curl -X POST -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{"type":"password_reset"}' https://api.digitalocean.com/v2/droplets/<id>/actions`
|
|
|
|
## Nodeloc cookies
|
|
|
|
Required for auth. Current set in `scanner.py` → `NODELOC_COOKIES` dict.
|
|
Session cookies expire — update from browser when scan fails.
|
|
|
|
## GitHub awesome-lists
|
|
|
|
Scraped from raw README.md:
|
|
- zukixa/cool-ai-stuff
|
|
- cheahjs/free-llm-api-resources
|
|
- zebbern/no-cost-ai
|
|
- ShaikhWarsi/free-ai-tools
|
|
|
|
Each line is checked for keywords and URLs are extracted via regex. |