autoreg-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
baradok60915d52e8425
94
skills/telegram-mass-join/references/bot-ref-checkin-flow.md
Normal file
94
skills/telegram-mass-join/references/bot-ref-checkin-flow.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Mass Bot Ref + Check-in Flow (session pool)
|
||||
|
||||
Recipe for running a points-bot flow (start with owner's ref → language → daily check-in) across the whole CLEAN_ALIVE_SESSIONS pool. Proven end-to-end on @gemini_plusv_bot over 362 sessions.
|
||||
|
||||
## Flow discovery (ALWAYS first)
|
||||
|
||||
Probe 1-3 accounts through the full path and capture reply texts + button types before writing the mass runner:
|
||||
|
||||
```python
|
||||
client.send_message(BOT, '/start ' + REF)
|
||||
msgs = await client.get_messages(BOT, limit=6)
|
||||
for m in reversed(list(msgs)):
|
||||
print(m.id, m.out, m.text)
|
||||
for r in (m.reply_markup.rows if m.reply_markup else []):
|
||||
for b in r.buttons:
|
||||
print(type(b).__name__, b.text, getattr(b,'data',None), getattr(b,'url',None))
|
||||
```
|
||||
|
||||
Key checks during probe:
|
||||
- Does a language screen appear? (fresh vs returning user marker)
|
||||
- Is there a channel-subscribe / forced-join gate anywhere in the flow? Many points bots have NONE at first — but gates can be SWITCHED ON DAYS LATER by the bot owner. Re-probe 1 account before every re-run; classify any unmatched reply text before assuming it's a timeout.
|
||||
- Are menu buttons reply-keyboard (`KeyboardButton`) or inline? Reply buttons are driven by `send_message(bot, '<exact button text>')`, not clicks.
|
||||
|
||||
## Channel gate handling (join → Verify → retry)
|
||||
|
||||
When check-in replies 'Please join our channel first!' with Join Channel + Verify inline buttons:
|
||||
|
||||
1. Extract the channel URL by clicking the join button — the callback answer RETURNS the URL as a string (data/url fields are None):
|
||||
```python
|
||||
res = await msg.click(text='📢 Join Channel')
|
||||
url = res if isinstance(res, str) and res.startswith('http') else None
|
||||
```
|
||||
2. `await client(JoinChannelRequest(await client.get_entity(url)))`
|
||||
3. Re-fetch messages, click the `✅ Verify` inline button, wait 2-3s
|
||||
4. Send the Check-in text AGAIN — gate clears and check-in succeeds
|
||||
|
||||
**ChannelsTooMuchError**: farmed accounts sit at the 500-channel cap. Free a slot by leaving 1-2 unprotected channels, then retry the join in a loop (max ~4 attempts):
|
||||
```python
|
||||
dialogs = await client.get_dialogs()
|
||||
# candidates: broadcast/megagroup entities whose username/title does NOT match KEEP_SUBSTR
|
||||
# KEEP_SUBSTR = target bot channel + owner's own channels ('ghsghs', 'alstack', 'reform', ...)
|
||||
await client(LeaveChannelRequest(entity))
|
||||
```
|
||||
Never leave the target channel or the owner's channels — match a keep-list before leaving.
|
||||
|
||||
**Import pitfall**: runner scripts that call `asyncio.run(main())` at module level execute on import — importing one to reuse `proc()` in a probe launches the whole mass run. Guard with `if __name__ == '__main__':` or probe with a standalone copy.
|
||||
|
||||
## Mass runner parameters (proven)
|
||||
|
||||
- Pool: `tmp/tg_session_ops/inventory.json`, filter `status=='LIVE'` + file exists → ~360 sessions from `Desktop/CLEAN_ALIVE_SESSIONS`
|
||||
- API creds: `API_ID=2040, API_HASH='[REDACTED-KEY]'` (same file as tg_session_ops/clean_accounts.py)
|
||||
- `asyncio.Semaphore(8)`, chunks of 40, 2-4s pause between chunks
|
||||
- `asyncio.wait_for(client.connect(), timeout=22)` — ~10% of pool hits connect TimeoutError (bad DC), non-retryable, skip
|
||||
- Progress dict keyed by session filename, saved after EVERY account → crash-safe resume
|
||||
- Random 1.5-3s sleeps between bot steps; 0 flood, 0 bans observed at this rate
|
||||
|
||||
## Per-account step sequence
|
||||
|
||||
1. `send_message(BOT, '/start ' + REF)` — ref registers on every start, even for already-known users
|
||||
2. If latest bot msg has 'English' button → `msg.click(text='🇬🇧 English')` (inline, data=None — click works)
|
||||
3. `send_message(BOT, '📋 Check-in')` (reply keyboard — send text, not click)
|
||||
4. Read last 3 bot msgs; classify by substring: 'Check-in Successful' → ok (+5 pts, regex `Total Points:\s*`?(\d+)`?`), 'Already Checked In' → done (counts)
|
||||
|
||||
## Result classes and second pass
|
||||
|
||||
- `unauth` (~10%): is_user_authorized() False — dead session
|
||||
- `err: No user has "X" as username` (1-2%): account can't resolve the bot (frozen/blocked) — skip
|
||||
- `checkin=None` (~15% first pass): reply not matched within wait window — delete these progress entries and re-run; most resolve to 'done' on pass 2. If a batch of Nones shares one reply text, it's a NEW GATE, not noise — read the actual reply before retrying.
|
||||
- Corrupt session files raise `ValueError: not enough values to unpack` in TelegramClient init — catch per-account, mark unauth, continue.
|
||||
- Final on 362 sessions: 315 ok / 310 check-ins counted; 5 accounts hit the channel gate, 4 resolved via join→Verify→retry
|
||||
|
||||
## Daily cron automation (check-in resets 00:00 UTC)
|
||||
|
||||
Turn the mass runner into a no_agent cron job:
|
||||
|
||||
1. **Wrapper script** (separate from the runner): load progress JSON → clear `checkin`/`points`/`joined_channel` fields on every record (keep `ref=True` so the runner skips re-start) → `subprocess.run([python, runner], timeout=3600, capture_output)` → print a compact summary (status counter, ok/done/none, points sum, last `DONE.` line). The summary IS the delivery — no_agent cron posts stdout verbatim.
|
||||
2. **Script location rule**: Hermes cron `script=` rejects absolute paths — the file MUST live in `~/AppData/Local/hermes/scripts/` and be passed as a bare filename. Copy the wrapper there; keep the mass runner itself anywhere (wrapper calls it by absolute path).
|
||||
3. **Schedule**: fire shortly after the bot's UTC reset (e.g. `10 3 * * *` MSK = 00:10 UTC). `no_agent=true`, `deliver='telegram:<owner DM id>'`, `enabled_toolsets` irrelevant for no_agent.
|
||||
4. **Smoke-test before trusting**: run the wrapper manually once in background (`> smoke.log 2>&1; echo EXIT=$? >> smoke.log`) and verify `EXIT=0` + summary lines. Same-day re-run is safe — bot answers 'Already Checked In' (counts as done), no double-claim exists.
|
||||
5. Wrapper must NOT import the runner module — runners call `asyncio.run(main())` at module level (see import pitfall); subprocess isolation is the fix.
|
||||
|
||||
Proven artifacts: wrapper `AppData/Local/hermes/scripts/gemini_daily_checkin.py`, job `gemini-bot-daily-checkin` (~7 min full cycle, 362 sessions).
|
||||
|
||||
## Bot economy notes (@gemini_plusv_bot)
|
||||
|
||||
- 25 pts on start, +5/day check-in, +10/ref to the ref owner, 50 pts = 1 credit, 1 credit = Gemini Plus 12mo, 4 = Pro 18mo
|
||||
- Owner's ref link accumulates points; 20 refs = Pro 18mo
|
||||
- Daily cron over the same pool is set up — see section above
|
||||
|
||||
## Files
|
||||
|
||||
- Runner v2 (with channel-gate handling + slot freeing): `C:/Users/User/tmp/gemini_ref_mass2.py` (parameterize BOT/REF for other bots); v1 without gate: `gemini_ref_mass.py`
|
||||
- Progress: `tmp/gemini_refprog.json`, log: `tmp/gemini_ref_log.jsonl`, run logs: `tmp/gemini_ref_run*.log`
|
||||
- Gate channel seen on @gemini_plusv_bot: `@ghsghs_5` (may rotate — always extract from the Join Channel callback, never hardcode)
|
||||
@@ -0,0 +1,61 @@
|
||||
# Chat Filtering Pipeline
|
||||
|
||||
## Source
|
||||
`good_chats_for_mailing.json` — 8,572 pre-filtered chats (from initial 54,063).
|
||||
|
||||
## Filtering Criteria
|
||||
|
||||
### Bad Keywords (exclude)
|
||||
```
|
||||
кликер, clicker, tap, тап, coin, token, airdrop, notcoin, hamster, meme,
|
||||
майнинг, mining, крипт, crypto, nft, nude, порно, xxx, sex, интим,
|
||||
ставк, казино, bet, букмекер, лотере, новост, news, медиа, подпиши,
|
||||
subscribe, розыгрыш, giveaway, конкурс, приз, фейк, накрутк, взлом,
|
||||
хак, cheat, скрипт, ферм, брут, спам, пробив, дур, нарк, трава,
|
||||
шишк, заклад, оруж, пистолет, взрывчат, бот, bot, тест, test, debug,
|
||||
memefi, time farm, tapswap, pixel, not pixel, major, blum, x empire,
|
||||
tomarket, cats, paws, agent 301, cex, okx, wallet, deposit, withdraw,
|
||||
exchange, staking, yield, defi, swap, boost, реферал, referral,
|
||||
партнёр, заработ, игра, game, гейм, play, канал, channel, лент,
|
||||
подборк, официал, official, офф
|
||||
```
|
||||
|
||||
### Good Keywords (include)
|
||||
```
|
||||
барахолк, объявлен, куплю, продам, услуг, работа, ваканс, чат,
|
||||
общение, знакомств, город, отдам, приму, ищу, обмен, аренда,
|
||||
недвижимость, мастер, ремонт, авто, такси, доставк, стройк, дизайн,
|
||||
маркет, продвижен, бизнес, фриланс, жиль, квартир, снять, сдам,
|
||||
красот, маникюр, парикмахер, дет, школ, курс, обучен, тренер,
|
||||
врач, стоматолог, аптек, юрист, бухгалтер, налог, тур, путёвк,
|
||||
гостиниц, еда, ресторан, кафе, продукт, зоо, животн, собак, кошк,
|
||||
мебел, техник, электроник, инструмент, одежд, обув, аксессуар,
|
||||
спорт, вело, йог, фитнес, музык, концерт, театр, кино, книг, чтен,
|
||||
психолог, коуч, переезд, эмиграц, виз, гражданств, цвет, саженц,
|
||||
рассад, огород, волонтёр, благотворител, хостел, отель, санатор,
|
||||
фото, видео, съём, монтаж, ногт, бров, ресниц, перевоз, груз,
|
||||
грузчик, уборк, клининг, химчистк, сантех, электрик, сварщик,
|
||||
программист, разработ, дизайнер, smm, таргет, маркетолог, нян,
|
||||
сиделк, домработниц, праздник, свадьб, юбилей, подарк, сувенир,
|
||||
хендмейд, комиссион, секонд, сток, авито, юла, доска объявлен,
|
||||
бесплатн, даром, район, микрорайон, посёлок, деревн, мкр, жк,
|
||||
жилой комплекс, сосед, подъезд, дом, двор, парковк, гараж
|
||||
```
|
||||
|
||||
### Size Filter
|
||||
- Min: 3,000 participants
|
||||
- Max: 100,000 participants
|
||||
|
||||
### Score Fallback
|
||||
Chats with score ≥ 11 pass even without keyword match.
|
||||
|
||||
## Results
|
||||
- 8,572 → 3,435 unique chats (after dedup by username)
|
||||
- 41.6M total audience reach
|
||||
- All барахолки, чаты объявлений, городские чаты — zero bots/clickers/crypto
|
||||
|
||||
## Top-Performing Chat Types
|
||||
1. Городские барахолки (Дубай, Стамбул, Тбилиси, Кипр, Нячанг, Аланья, Пхукет, Батуми) — score 14
|
||||
2. Чаты объявлений (куплю/продам/услуги) — score 12-14
|
||||
3. Чаты общения/знакомств — score 11-14
|
||||
4. Тематические чаты (работа, недвижимость, авто) — score 8-11
|
||||
101
skills/telegram-mass-join/references/join-executor-patterns.md
Normal file
101
skills/telegram-mass-join/references/join-executor-patterns.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# Join Executor Patterns
|
||||
|
||||
## Script: `join_assigned.py`
|
||||
|
||||
### Key Constants
|
||||
```python
|
||||
BATCH_SIZE = 5 # Accounts processed in parallel
|
||||
BATCH_PAUSE = 15 # Seconds between batches (was 30, lowered for speed)
|
||||
ACCOUNT_TIMEOUT = 120 # Hard timeout per account (was 60, 50×2.5s avg = 125s)
|
||||
```
|
||||
|
||||
### Error Classification
|
||||
```python
|
||||
BAD_REASONS = {
|
||||
'CHANNEL_PRIVATE', 'CHAT_WRITE_FORBIDDEN', 'CHANNEL_INVALID',
|
||||
'USER_BANNED_IN_CHANNEL', 'INVITE_HASH_EXPIRED', 'INVITE_HASH_INVALID',
|
||||
'CHAT_ID_INVALID', 'USERNAME_NOT_OCCUPIED', 'USERNAME_INVALID',
|
||||
'CHAT_FORBIDDEN', 'CHAT_RESTRICTED',
|
||||
}
|
||||
```
|
||||
|
||||
### Join Result Types
|
||||
| Result | Meaning |
|
||||
|--------|---------|
|
||||
| `joined` | Successfully joined |
|
||||
| `already` | Already a member (UserAlreadyParticipantError) |
|
||||
| `flood_N` | Rate limited for N seconds (auto-waits) |
|
||||
| `bad_*` | Chat is private/deleted/invalid |
|
||||
| `err_*` | Unknown error (first 50 chars logged) |
|
||||
| `err_timeout` | Individual chat operation timed out |
|
||||
|
||||
### Account Status Types
|
||||
| Status | Meaning |
|
||||
|--------|---------|
|
||||
| `ok` | All chats processed |
|
||||
| `hard_timeout` | Account took >60s total |
|
||||
| `no_stringsession` | No StringSession found |
|
||||
| `timeout_connect` | connect() timed out |
|
||||
| `crash: ...` | Exception during processing |
|
||||
|
||||
### Async Flow
|
||||
```
|
||||
main() → for each batch of 5 accounts:
|
||||
→ asyncio.gather(process_with_timeout(acc) for acc in batch)
|
||||
→ process_with_timeout → asyncio.wait_for(process_account, timeout=60)
|
||||
→ process_account:
|
||||
→ client.connect() [10s timeout]
|
||||
→ client.get_me() [5s timeout]
|
||||
→ for each chat:
|
||||
→ get_entity(username) [10s timeout]
|
||||
→ JoinChannelRequest(entity) [10s timeout]
|
||||
→ sleep(0.5-1.5s)
|
||||
→ client.disconnect()
|
||||
```
|
||||
|
||||
### Account Lookup
|
||||
Accounts are identified by index in the sorted OK list:
|
||||
```python
|
||||
ok_accounts.sort(key=lambda a: a.get("phone", ""))
|
||||
# ...
|
||||
global_idx = i + idx
|
||||
acc_key = f"account_{global_idx + 1}"
|
||||
acc_chats = assignments.get(acc_key, [])
|
||||
```
|
||||
|
||||
### StringSession Lookup
|
||||
Tries phone first, then name:
|
||||
```python
|
||||
ss = string_sessions.get(phone) or string_sessions.get(name)
|
||||
```
|
||||
|
||||
### Progress Persistence
|
||||
```python
|
||||
# Saved every batch:
|
||||
{
|
||||
"batch_num": 42,
|
||||
"total_joined": 352,
|
||||
"accounts_processed": 210
|
||||
}
|
||||
|
||||
# Joined chats saved every 10 batches:
|
||||
{
|
||||
"baraholllkamariupol": {
|
||||
"username": "baraholllkamariupol",
|
||||
"id": "1856532726",
|
||||
"title": "Барахолка...",
|
||||
"participants": 21513,
|
||||
"joined_by": "12894827641"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Observations
|
||||
- 50 chats/account × 120s timeout: ~28-36 joins per 58-account process
|
||||
- ~25% of accounts succeed (ok), ~40% hard_timeout, ~20% timeout_connect, ~15% no_stringsession
|
||||
- 50 chats × 2.5s avg = 125s → 120s is tight. Some accounts with slow connections hit hard_timeout
|
||||
- timeout_connect accounts have network/DC issues — not recoverable per-run
|
||||
- 3 parallel processes × 58 accounts: ~81 new joins, ~17 min runtime
|
||||
- 0 flood, 0 bans — safe for mass operations
|
||||
- BATCH_PAUSE=15s (was 30s) — sufficient to avoid rate limits, 2x faster
|
||||
- Together with 3-way parallel: ~6x speedup vs original single 30s-pause process
|
||||
@@ -0,0 +1,86 @@
|
||||
# Parallel Multi-Process Join Execution
|
||||
|
||||
## Motivation
|
||||
Single `join_assigned.py` on 174 accounts × 50 chats with 60s timeout = ~50 minutes. Most of that time is waiting for Telegram responses. Running 3 parallel processes on different account ranges cuts it to ~17 minutes.
|
||||
|
||||
## CLI Arguments Added to `join_assigned.py`
|
||||
|
||||
```
|
||||
--start N First account index (0-based, default 0)
|
||||
--end N Last account index (exclusive, default all)
|
||||
--timeout S Account hard timeout in seconds (default 120)
|
||||
--pause S Inter-batch pause in seconds (default 15)
|
||||
--suffix STR Progress file suffix (e.g. "_1", "_2", "_3")
|
||||
```
|
||||
|
||||
## Launch Pattern
|
||||
|
||||
174 accounts → 3 processes of 58 each:
|
||||
|
||||
```bash
|
||||
python -u join_assigned.py --start 0 --end 58 --suffix _1 --timeout 120 --pause 15 &
|
||||
python -u join_assigned.py --start 58 --end 116 --suffix _2 --timeout 120 --pause 15 &
|
||||
python -u join_assigned.py --start 116 --end 174 --suffix _3 --timeout 120 --pause 15 &
|
||||
```
|
||||
|
||||
## Why 120s Timeout for 50 Chats
|
||||
|
||||
Each chat join takes ~2-3 seconds (get_entity + JoinChannelRequest + random delay). 50 chats × 2.5s avg = 125s. 120s timeout covers most accounts while still catching true hangs.
|
||||
|
||||
With 60s timeout and 50 chats → almost all accounts hit hard_timeout. 120s is the sweet spot.
|
||||
|
||||
## Progress File Isolation
|
||||
|
||||
Each process uses its own progress file:
|
||||
- `assigned_join_progress_1.json`
|
||||
- `assigned_join_progress_2.json`
|
||||
- `assigned_join_progress_3.json`
|
||||
|
||||
Each writes to its own `joined_chats{suffix}.json` (NOT shared — race condition otherwise). After all finish:
|
||||
|
||||
```bash
|
||||
python merge_joined.py
|
||||
```
|
||||
|
||||
`merge_joined.py` loads all `joined_chats*.json`, deduplicates by username/id, writes to `joined_chats.json`.
|
||||
|
||||
### ⚠️ Race Condition on Shared Progress File
|
||||
When 3+ processes read/write the same `joined_chats.json`, they overwrite each other's data. Last save wins → ~58% data loss (47 of 81 new joins lost in one run). **Each process MUST use its own suffix throughout:**
|
||||
|
||||
| File | Process _1 | Process _2 | Process _3 |
|
||||
|------|-----------|-----------|-----------|
|
||||
| Joined chats | `joined_chats_1.json` | `joined_chats_2.json` | `joined_chats_3.json` |
|
||||
| Progress | `assigned_join_progress_1.json` | `assigned_join_progress_2.json` | `assigned_join_progress_3.json` |
|
||||
|
||||
Merge step is mandatory after parallel runs.
|
||||
|
||||
## Python Pitfalls Encountered
|
||||
|
||||
### 1. `global` before any reference
|
||||
```python
|
||||
# CRASH: SyntaxError: name 'ACCOUNT_TIMEOUT' is used prior to global declaration
|
||||
async def main():
|
||||
parser.add_argument("--timeout", default=ACCOUNT_TIMEOUT)
|
||||
global ACCOUNT_TIMEOUT # too late
|
||||
|
||||
# FIX: global first
|
||||
async def main():
|
||||
global ACCOUNT_TIMEOUT, BATCH_PAUSE # must be FIRST
|
||||
parser.add_argument("--timeout", default=ACCOUNT_TIMEOUT)
|
||||
```
|
||||
|
||||
### 2. Em-dash in docstrings
|
||||
```python
|
||||
# CRASH: SyntaxError: invalid character '—' (U+2014)
|
||||
"""Join chats — fast and targeted."""
|
||||
|
||||
# FIX: use ASCII double dash
|
||||
"""Join chats -- fast and targeted."""
|
||||
```
|
||||
|
||||
## Expected Results
|
||||
|
||||
- 3× 58 accounts = 174 total
|
||||
- 3× 12 batches = 36 total batches
|
||||
- ~17 minutes total runtime
|
||||
- 500-1500 new joins (from 50 chats/account, most error out)
|
||||
@@ -0,0 +1,66 @@
|
||||
# StringSession Conversion
|
||||
|
||||
## The Problem
|
||||
SQLite-based Telethon sessions deadlock when accessed in parallel. The error:
|
||||
```
|
||||
sqlite3.OperationalError: database is locked
|
||||
```
|
||||
This happens because SQLite uses file-level locking. When multiple accounts try to connect simultaneously, the `.session` files lock each other out.
|
||||
|
||||
## The Fix: StringSession
|
||||
Convert ALL sessions to in-memory StringSession format stored in a single JSON file. No file locking, no database contention.
|
||||
|
||||
## Converter Script
|
||||
Located at: `C:\Users\User\Desktop\_PROJECTS\рассылка\session_converter.py`
|
||||
|
||||
### How It Works
|
||||
```python
|
||||
# For each .session file:
|
||||
client = TelegramClient(session_path, API_ID, API_HASH)
|
||||
await client.connect()
|
||||
ss = client.session.save() # Returns StringSession string
|
||||
string_sessions[phone] = ss
|
||||
await client.disconnect()
|
||||
```
|
||||
|
||||
### Output
|
||||
`generated/sessions_strings.json`:
|
||||
```json
|
||||
{
|
||||
"12894827641": "1AgAOMTQ5...",
|
||||
"14066141705": "1BQANOTE3...",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
## Usage in Join Script
|
||||
```python
|
||||
# Load
|
||||
with open("generated/sessions_strings.json") as f:
|
||||
string_sessions = json.load(f)
|
||||
|
||||
# Use
|
||||
ss = string_sessions.get(phone) or string_sessions.get(name)
|
||||
if not ss:
|
||||
return {"status": "no_stringsession"}
|
||||
|
||||
client = TelegramClient(StringSession(ss), API_ID, API_HASH)
|
||||
await client.connect()
|
||||
```
|
||||
|
||||
## Conversion Results (2026-07-08)
|
||||
- 222 sessions converted
|
||||
- 137 new StringSessions created
|
||||
- 9 failures (corrupted sessions)
|
||||
|
||||
## Why NOT SQLite Retry
|
||||
- WAL mode doesn't fix parallel access
|
||||
- Retry with exponential backoff adds latency
|
||||
- Timeout-based retry still leaves windows for deadlock
|
||||
- StringSession is simpler, faster, and zero-lock
|
||||
|
||||
## Pitfalls
|
||||
- StringSession strings are long (~350 chars) — the JSON file can be large
|
||||
- Invalid/corrupted .session files will fail conversion silently
|
||||
- Some accounts may have both .session and StringSession — prefer StringSession
|
||||
- The `generated/sessions_strings.json` file should be backed up before mass operations
|
||||
Reference in New Issue
Block a user