autoreg-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
baradok60915d52e8425
@@ -0,0 +1,256 @@
|
||||
# Conol.ai Pool Gateway Pattern
|
||||
|
||||
**Date:** 2026-07-29
|
||||
**Status:** Production — 21 accounts, public ngrok endpoint
|
||||
|
||||
## Architecture
|
||||
|
||||
Multi-account OpenAI-compatible API gateway that round-robins across conol.ai accounts using the internal agent session protocol. Each request creates a fresh Conol agent session, streams the response via SSE, and returns OpenAI-format output.
|
||||
|
||||
## Why Pool Instead of Single Account
|
||||
|
||||
- **Rate limits:** Single account hits agent session creation limits
|
||||
- **Credits:** Each free account has ~800 credits (2000 after quest farming), shared pool extends total budget
|
||||
- **Redundancy:** Failed accounts auto-disabled, requests route to healthy accounts
|
||||
- **Model diversity:** Different accounts may have different model availability tiers
|
||||
|
||||
## Core Protocol: Conol Agent Sessions
|
||||
|
||||
Conol.ai does NOT have a public REST API for chat. Instead, it uses an internal agent session protocol:
|
||||
|
||||
```
|
||||
1. POST /api/sessions → {sessionId}
|
||||
Body: {source, messages, timezone, agentModel, agentEffort}
|
||||
|
||||
2. GET /api/sessions/{id}/messages?logDeltas=1 → SSE stream
|
||||
Events: assistant, stream_event (text_delta), history_delta, thinking,
|
||||
tool_use, plan, done, error, status, questionary
|
||||
|
||||
3. Extract final response text from history_delta → stages → logs → content
|
||||
```
|
||||
|
||||
### Key Implementation Detail
|
||||
|
||||
The SSE stream's `history_delta` event contains the final agent response nested inside `stages[].logs[].content[]`. Each content part has `type: "text"` and the `text` field.
|
||||
|
||||
```python
|
||||
async with client.stream("GET", f"{BASE_URL}/api/sessions/{sid}/messages?logDeltas=1") as stream:
|
||||
async for line in stream.aiter_lines():
|
||||
ev = json.loads(line[5:].strip())
|
||||
if ev.get("type") == "done":
|
||||
break
|
||||
if ev.get("type") == "history_delta":
|
||||
for stage in ev.get("stages", []):
|
||||
for log in stage.get("logs", []):
|
||||
for part in log.get("content", []):
|
||||
if part.get("type") == "text":
|
||||
full_text = part["text"]
|
||||
```
|
||||
|
||||
### Auth
|
||||
|
||||
Better Auth session token in cookie `__Secure-better-auth.session_token`. No Bearer token — cookies-only.
|
||||
|
||||
### Session Cookies
|
||||
|
||||
Persist cookies from account registration. Each account's cookies are stored in `cookies_conol<name>.json`. Load as dict `{name: value}` and pass to httpx.AsyncClient.
|
||||
|
||||
## Model List (18 models, July 2026)
|
||||
|
||||
```
|
||||
gpt-5.5, gpt-5.5-pro, gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna
|
||||
deepseek/deepseek-v4-pro, deepseek/deepseek-v4-base
|
||||
kimi/kimi-k3, qwen/qwen-3.7, glm/glm-5.2
|
||||
claude-opus-4-8 ✦, claude-fable-5 ✦, claude-opus-4-7 ✦
|
||||
gemini-3-pro ✦, gemini-3-flash ✦
|
||||
llama-4-maverick, llama-4-scout, mistral-large-3
|
||||
```
|
||||
✦ = vision-capable (image input)
|
||||
|
||||
## Image Generation
|
||||
|
||||
Claude models (opus-4-8, fable-5) can generate SVG images when prompted with "Generate an SVG image...". They do NOT generate pixel images (PNG/JPEG). The `/v1/images/generations` endpoint wraps this behind an OpenAI-compatible API.
|
||||
|
||||
Output: SVG in response text. Example result: 2384-byte neon skull SVG with glow filter.
|
||||
|
||||
For real PNG/JPEG generation: need cairosvg/resvg for SVG→PNG conversion, or external image-gen API.
|
||||
|
||||
## Round-Robin with Health Checks
|
||||
|
||||
```python
|
||||
class PoolManager:
|
||||
accounts: list[Account] # from conol_accounts_pool.jsonl
|
||||
_idx: int # atomic round-robin index
|
||||
|
||||
async def get_next(self) -> Account:
|
||||
# Skip accounts with fail_count >= 3
|
||||
# Cycle through active accounts
|
||||
# Auto-reset fail_count when all accounts failing
|
||||
```
|
||||
|
||||
Accounts with 5 consecutive failures are marked `active=False`.
|
||||
|
||||
## Gateway Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/health` | GET | Pool status (active/total, per-account stats) |
|
||||
| `/v1/models` | GET | OpenAI-compatible model list (18 models) |
|
||||
| `/v1/chat/completions` | POST | Chat with `stream: true/false` |
|
||||
| `/v1/images/generations` | POST | SVG generation via Claude |
|
||||
| `/pool/stats` | GET | Detailed per-account statistics |
|
||||
| `/pool/reload` | POST | Hot-reload accounts from jsonl |
|
||||
|
||||
## Streaming
|
||||
|
||||
The gateway supports SSE streaming (OpenAI-compatible `stream: true`). Since Conol's SSE is already streaming, the gateway chunks the full response into 16-char segments and yields OpenAI-format delta chunks.
|
||||
|
||||
## Public Exposure via ngrok
|
||||
|
||||
```bash
|
||||
ngrok http http://127.0.0.1:9999 --url https://<static-domain>.ngrok-free.dev
|
||||
```
|
||||
|
||||
Static ngrok domain ensures consistent public URL across restarts. Ngrok rewrites `Host` header and bypasses browser warnings.
|
||||
|
||||
## Account Management
|
||||
|
||||
### Registration (Chrome CDP)
|
||||
|
||||
```bash
|
||||
python mass_reg_conol.py <target_count>
|
||||
```
|
||||
|
||||
Uses Chrome DevTools Protocol (port 9228) to automate conol.ai signup:
|
||||
1. Navigate to `/sign-up`
|
||||
2. Execute reCAPTCHA v3 via `grecaptcha.execute(siteKey, {action})`
|
||||
3. POST `/api/invites/register` with `x-captcha-response` header
|
||||
4. Send verification email
|
||||
5. IMAP poll for verification link
|
||||
6. Login + extract cookies + API key
|
||||
|
||||
Success rate: 13/15 (87%) with increasing delays (8s → 50s).
|
||||
Rate limit: IP-based, hits every ~4th account (HTTP 429 on verify-send).
|
||||
|
||||
### reCAPTCHA v3 Details
|
||||
|
||||
- **Type:** Google reCAPTCHA v3 (NOT Cloudflare Turnstile)
|
||||
- **Site key:** `6Lc3wmAtAAAAAB9YBPXQtT9uGGsH3ul6LQBc5AUu`
|
||||
- **Actions:** `sign_up`, `send_verification_email`, `sign_in`
|
||||
- **Domain-locked:** 3rd-party captcha solving services (AntiCaptcha, 2Captcha) cannot solve it — the reCAPTCHA server rejects their tokens. Browser-based execution via Chrome CDP is the only working path.
|
||||
- **Headless penalty:** reCAPTCHA v3 scores headless Chrome at ~0.1 (threshold 0.5). Real GUI Chrome required.
|
||||
|
||||
### Quest Farming
|
||||
|
||||
```bash
|
||||
python quest_farmer.py
|
||||
```
|
||||
|
||||
Automatically completes quests via agent sessions for +1200 credits per account:
|
||||
- `note_written_by_agent` — ✅ works
|
||||
- `memory_written` — ✅ works
|
||||
- `note_edited_by_agent` — ✅ works
|
||||
- `timer_scheduled` — ✅ works
|
||||
- `note_shared` — ❌ requires authenticated Conol session
|
||||
- `skill_installed` — ✅ works (installs skills like `find-skills`, `deep-research`)
|
||||
|
||||
Post-farm: each account has ~2000 credits (800 base + 1200 quests).
|
||||
|
||||
### Account Storage
|
||||
|
||||
```
|
||||
conol_accounts_pool.jsonl — JSONL records (email, password, name, cookies_path, credits, api_key_masked)
|
||||
cookies_conol<name>.json — Session cookies per account
|
||||
conol_pool_state.json — Mass registration state
|
||||
conol_quests_state.json — Quest farming state
|
||||
```
|
||||
|
||||
## Captcha Key Verification (2026-07-29)
|
||||
|
||||
| Provider | Keys | Status |
|
||||
|----------|------|--------|
|
||||
| 2Captcha (old API) | 18 keys | ❌ All `ERROR_KEY_DOES_NOT_EXIST` on V2 API |
|
||||
| AntiCaptcha | 4 keys | ✅ Live ($45.45 total) |
|
||||
| CapSolver | 1 key | ❌ $0.0002 (effectively empty) |
|
||||
|
||||
**Key takeaway:** 2Captcha old API keys (verified via `res.php?action=getbalance`) do NOT work with the new API (`api.2captcha.com/createTask`). Always verify keys against the actual API endpoint you'll use, not just the balance check.
|
||||
|
||||
## Deployment
|
||||
|
||||
```
|
||||
ENI Conol Pool Gateway v3.0
|
||||
Port: 9999
|
||||
Processes: chrome.exe (CDP), conol_pool_gateway.py, ngrok.exe
|
||||
Persistence: watchdog + autostart via .vbs
|
||||
```
|
||||
|
||||
Watchdog monitors gateway health and restarts on failure. Ngrok auto-reconnects with static domain.
|
||||
|
||||
## Batch Account Health Check (Validate All Cookie Sessions)
|
||||
|
||||
When validating that all accounts in the pool are alive, use this pattern — NOT the gateway (which hides per-account errors behind round-robin):
|
||||
|
||||
```python
|
||||
import json, httpx, asyncio
|
||||
|
||||
async def check_account(acc):
|
||||
email = acc['email']
|
||||
cp = acc['cookies_path']
|
||||
with open(cp) as f:
|
||||
cookies = {c['name']: c['value'] for c in json.load(f)}
|
||||
|
||||
# ⚠️ CRITICAL: каждый аккаунт ДОЛЖЕН иметь СВОЙ httpx.AsyncClient.
|
||||
# Общий клиент на все аккаунты перемешивает куки → все получают 401.
|
||||
async with httpx.AsyncClient(cookies=cookies, timeout=httpx.Timeout(30)) as client:
|
||||
r = await client.post('https://conol.ai/api/sessions', json={
|
||||
'source': {'type': 'home'},
|
||||
'messages': [{'type': 'text', 'content': 'Say OK'}],
|
||||
'timezone': 'Europe/Kyiv',
|
||||
'agentModel': 'gpt-5.5',
|
||||
'agentEffort': 'low',
|
||||
})
|
||||
|
||||
if r.status_code != 201:
|
||||
return (email, False, f'HTTP {r.status_code}')
|
||||
|
||||
sid = r.json()['sessionId']
|
||||
# Stream SSE and look for history_delta (NOT text_delta!)
|
||||
async with client.stream('GET', f'https://conol.ai/api/sessions/{sid}/messages?logDeltas=1') as stream:
|
||||
async for line in stream.aiter_lines():
|
||||
if not line.startswith('data:'):
|
||||
continue
|
||||
ev = json.loads(line[5:])
|
||||
if ev.get('type') == 'done':
|
||||
return (email, True, 'ok')
|
||||
if ev.get('type') == 'history_delta':
|
||||
for stage in ev.get('stages', []):
|
||||
for log in stage.get('logs', []):
|
||||
for c in log.get('content', []):
|
||||
if c.get('text'):
|
||||
return (email, True, 'ok')
|
||||
return (email, False, 'no done event')
|
||||
|
||||
# Run with semaphore for rate limiting
|
||||
sem = asyncio.Semaphore(3) # max 3 concurrent
|
||||
async def bounded(acc):
|
||||
async with sem:
|
||||
return await check_account(acc)
|
||||
|
||||
results = await asyncio.gather(*[bounded(a) for a in accounts])
|
||||
ok = [r for r in results if r[1]] # (email, success, detail)
|
||||
fail = [r for r in results if not r[1]]
|
||||
```
|
||||
|
||||
### Key Pitfalls
|
||||
|
||||
1. **Shared httpx.AsyncClient → ALL accounts return 401.** If you create ONE `httpx.AsyncClient(cookies=cookies_a)` and reuse it for account B's requests, httpx merges the cookies and sends the wrong session token. Every concurrent test MUST create a fresh, isolated `httpx.AsyncClient` per account.
|
||||
|
||||
2. **SSE event type is `history_delta`, NOT `text_delta`.** The conol.ai SSE stream uses `history_delta` — a nested history log with `stages[] → logs[] → content[]`. Looking for `text_delta` (like OpenAI SSE format) gives "empty response" even though the session created successfully (HTTP 201).
|
||||
|
||||
3. **What a "healthy" account looks like:**
|
||||
- `POST /api/sessions` returns **HTTP 201** (NOT 200, NOT 401)
|
||||
- SSE stream eventually delivers `type: "done"` or `history_delta` with content
|
||||
- HTTP 401 = session expired, needs re-login via Chrome CDP
|
||||
- HTTP 429 = rate limited, retry with delay
|
||||
|
||||
4. **Auth cookies:** Better Auth tokens: `__Secure-better-auth.session_token` and `__Secure-better-auth.better-auth-passkey`. Both REQUIRED. Cookie files contain exactly 2 entries per account. If a cookie file has different structure, the session is from an old auth mechanism and won't work.
|
||||
Reference in New Issue
Block a user