Parallel Multi-Process Join Execution

Motivation

Single join_assigned.py on 174 accounts × 50 chats with 60s timeout = ~50 minutes. Most of that time is waiting for Telegram responses. Running 3 parallel processes on different account ranges cuts it to ~17 minutes.

CLI Arguments Added to `join_assigned.py`

--start N      First account index (0-based, default 0)
--end N        Last account index (exclusive, default all)
--timeout S    Account hard timeout in seconds (default 120)
--pause S      Inter-batch pause in seconds (default 15)
--suffix STR   Progress file suffix (e.g. "_1", "_2", "_3")

Launch Pattern

174 accounts → 3 processes of 58 each:

python -u join_assigned.py --start 0 --end 58 --suffix _1 --timeout 120 --pause 15 &
python -u join_assigned.py --start 58 --end 116 --suffix _2 --timeout 120 --pause 15 &
python -u join_assigned.py --start 116 --end 174 --suffix _3 --timeout 120 --pause 15 &

Why 120s Timeout for 50 Chats

Each chat join takes ~2-3 seconds (get_entity + JoinChannelRequest + random delay). 50 chats × 2.5s avg = 125s. 120s timeout covers most accounts while still catching true hangs.

With 60s timeout and 50 chats → almost all accounts hit hard_timeout. 120s is the sweet spot.

Progress File Isolation

Each process uses its own progress file:

assigned_join_progress_1.json
assigned_join_progress_2.json
assigned_join_progress_3.json

Each writes to its own joined_chats{suffix}.json (NOT shared — race condition otherwise). After all finish:

python merge_joined.py

merge_joined.py loads all joined_chats*.json, deduplicates by username/id, writes to joined_chats.json.

⚠️ Race Condition on Shared Progress File

When 3+ processes read/write the same joined_chats.json, they overwrite each other's data. Last save wins → ~58% data loss (47 of 81 new joins lost in one run). Each process MUST use its own suffix throughout:

File	Process _1	Process _2	Process _3
Joined chats	`joined_chats_1.json`	`joined_chats_2.json`	`joined_chats_3.json`
Progress	`assigned_join_progress_1.json`	`assigned_join_progress_2.json`	`assigned_join_progress_3.json`

Merge step is mandatory after parallel runs.

Python Pitfalls Encountered

1. `global` before any reference

# CRASH: SyntaxError: name 'ACCOUNT_TIMEOUT' is used prior to global declaration
async def main():
    parser.add_argument("--timeout", default=ACCOUNT_TIMEOUT)
    global ACCOUNT_TIMEOUT  # too late

# FIX: global first
async def main():
    global ACCOUNT_TIMEOUT, BATCH_PAUSE  # must be FIRST
    parser.add_argument("--timeout", default=ACCOUNT_TIMEOUT)

2. Em-dash in docstrings

# CRASH: SyntaxError: invalid character '—' (U+2014)
"""Join chats — fast and targeted."""

# FIX: use ASCII double dash
"""Join chats -- fast and targeted."""

Expected Results

3× 58 accounts = 174 total
3× 12 batches = 36 total batches
~17 minutes total runtime
500-1500 new joins (from 50 chats/account, most error out)

3.1 KiB Raw Permalink Blame History Unescape Escape