single-file CLI (urnet.py) + README quick start + GUIDE with flags, scaling recipe, troubleshooting
This commit is contained in:
ENI
@@ -0,0 +1,325 @@
|
||||
# GUIDE — urnet from zero to a working proxy pool
|
||||
|
||||
Everything here was run and measured on 2026-11-13 on a stock Windows 11 box with Python 3.11 and
|
||||
git-bash. Linux and macOS behave the same; differences are called out where they matter.
|
||||
|
||||
---
|
||||
|
||||
## 1. What you get
|
||||
|
||||
| | per account | 10 accounts |
|
||||
|---|---|---|
|
||||
| traffic | 30 GiB / day (resets 00:00 UTC) | 300 GiB / day |
|
||||
| proxies | 5, each in its own country | 50 |
|
||||
| time to create | ~25 s | ~5 min |
|
||||
| cost | 0 — no card, no email, no captcha | 0 |
|
||||
|
||||
Exits are residential peers (Comcast, Vodafone, Bell, KPN, Bouygues, Telia, Swisscom, SoftBank…).
|
||||
They are not dedicated: the exit IP can change between requests, and not every peer is a home
|
||||
connection — see [§8 Caveats](#8-caveats-worth-knowing).
|
||||
|
||||
## 2. How it works
|
||||
|
||||
URnetwork is a P2P bandwidth network: a client ("network") buys traffic, peers supply the exits.
|
||||
A free network gets 30 GiB/day, and you can publish hosted proxy endpoints on top of it.
|
||||
|
||||
The API is public and needs no browser session:
|
||||
|
||||
```
|
||||
POST /auth/network-create {"terms": true, "product_updates": false}
|
||||
-> {"network": {"network_id": "...", "network_name": "february-exhaust-83", "by_jwt": "..."}}
|
||||
|
||||
POST /auth/regenerate-seedphrase (Bearer <by_jwt>)
|
||||
-> {"seedphrase": "24 words"} # the ONLY recovery path for the account
|
||||
|
||||
POST /account/api-key {"name": "urnet-1"} (Bearer <by_jwt>)
|
||||
-> {"api_key": "urn_..."} # long-lived key for everything below
|
||||
|
||||
POST /network/auth-client {"description": "HTTPS proxy · US",
|
||||
"device_spec": "resident proxy",
|
||||
"proxy_config": {"initial_device_state": {"country_code": "us"}}}
|
||||
(Bearer <urn_...>)
|
||||
-> {"client_id": "...", "proxy_config_result": {
|
||||
"https_proxy_url": "https://<token>.crisp.bringyour.com:7155",
|
||||
"socks_proxy_url": "socks5h://crisp.bringyour.com:7153",
|
||||
"http_proxy_url": "http://crisp.bringyour.com:7154" }}
|
||||
|
||||
GET /network/proxies (Bearer <urn_...>) -> all proxies of the account
|
||||
GET /subscription/balance (Bearer <urn_...>) -> bytes left today
|
||||
```
|
||||
|
||||
Base URL: `https://api.bringyour.com`. Every authenticated call takes
|
||||
`Authorization: Bearer <jwt or urn_ key>`.
|
||||
|
||||
`urnet.py` wraps exactly these calls. Nothing else is required.
|
||||
|
||||
## 3. Install
|
||||
|
||||
### Windows
|
||||
|
||||
1. Install Python 3.8+ from python.org (tick "Add python.exe to PATH").
|
||||
2. Verify:
|
||||
```bash
|
||||
python --version
|
||||
curl --version
|
||||
```
|
||||
`curl` ships with Windows 10 1803+ (`C:\Windows\System32\curl.exe`). If it is missing, install it,
|
||||
or run without proxy flags (direct mode).
|
||||
3. Get the code:
|
||||
```bash
|
||||
git clone https://git.reformboss.com/reform/ur-network-farm.git
|
||||
cd ur-network-farm
|
||||
```
|
||||
|
||||
### Linux / macOS
|
||||
|
||||
```bash
|
||||
sudo apt install -y python3 curl git # Debian/Ubuntu; brew install python curl git on macOS
|
||||
git clone https://git.reformboss.com/reform/ur-network-farm.git
|
||||
cd ur-network-farm
|
||||
```
|
||||
|
||||
There is nothing to install for the script itself — standard library only.
|
||||
|
||||
### Sanity check
|
||||
|
||||
```bash
|
||||
python urnet.py --help
|
||||
```
|
||||
|
||||
Expected: usage block listing `farm | collect | verify | export | balance`.
|
||||
|
||||
## 4. First account in one minute
|
||||
|
||||
```bash
|
||||
python urnet.py farm --count 1
|
||||
```
|
||||
|
||||
Expected output (names are random):
|
||||
|
||||
```
|
||||
farm: 1 account(s) x 5 proxies | egress: direct | bank: bank.jsonl
|
||||
#1 OK february-exhaust-83 proxies=5 [us,de,nl,fr,gb]
|
||||
{"accounts_ok": 1, "proxies_ok": 5, "failed": 0, "bank": "bank.jsonl"}
|
||||
|
||||
next: python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
python urnet.py verify --pool pool.jsonl --limit 10
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
```bash
|
||||
python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
python urnet.py verify --pool pool.jsonl --limit 3
|
||||
python urnet.py export --pool pool.jsonl --format txt --out proxies.txt
|
||||
python urnet.py balance --bank bank.jsonl
|
||||
```
|
||||
|
||||
Expected `verify` output:
|
||||
|
||||
```
|
||||
1 us 72.89.45.91 US Freeport AS701 Verizon Business
|
||||
1 de 77.187.40.197 DE Frankfurt am M AS6805 Telefonica Germany GmbH
|
||||
1 nl 81.204.241.210 NL Amsterdam AS1136 KPN B.V.
|
||||
|
||||
TOTAL 3 | alive 3 | unique_ip 3 | countries 3 | hosting_exits 0
|
||||
```
|
||||
|
||||
`balance` shows what is left today:
|
||||
|
||||
```
|
||||
1 february-exhaust-83 left 29.71 GiB of 30.00 GiB open 292.05 MiB | proxies 5
|
||||
|
||||
accounts 1 | live keys 1 | traffic left today 29.71 GiB
|
||||
```
|
||||
|
||||
## 5. Commands
|
||||
|
||||
### `farm` — create accounts, keys and proxies
|
||||
|
||||
| flag | default | meaning |
|
||||
|---|---|---|
|
||||
| `--count N` | 1 | how many accounts to create |
|
||||
| `--proxies N` | 5 | proxies per account, hard max **5** |
|
||||
| `--countries us,de,nl` | built-in list of 15 | ISO codes, cycled across proxies |
|
||||
| `--bank FILE` | `bank.jsonl` | where accounts are appended |
|
||||
| `--start N` | 1 | label of the first account |
|
||||
| `--proxy URL` | — | single egress for every request |
|
||||
| `--proxy-file FILE` | — | one proxy URL per line, rotates per account |
|
||||
| `--shuffle` | off | randomise the proxy file order |
|
||||
| `--delay SEC` | 1.3 | pause between proxy calls (keep >= 1.3) |
|
||||
| `--curl` | off | force curl as the transport |
|
||||
| `--show-seed` | off | print a seed prefix in the log |
|
||||
|
||||
Notes:
|
||||
|
||||
- Each account is written to `bank.jsonl` **after every step**, so a crash or Ctrl-C never loses an
|
||||
account that already exists. Re-running with a new `--start` continues where you stopped.
|
||||
- Names are server-generated (`february-exhaust-83`); you cannot choose them.
|
||||
- Every account stores `jwt`, `seed` and `urn`. The seed is the only recovery path — back the file up.
|
||||
|
||||
### `collect` — bank to pool
|
||||
|
||||
```bash
|
||||
python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
# collected 49 unique proxies from bank.jsonl -> pool.jsonl (15 countries: AT,BR,CA,CH,DE,ES,FR,GB,...)
|
||||
```
|
||||
|
||||
Duplicates are removed by URL. Proxies created in the web UI are picked up as long as their account
|
||||
line is in `bank.jsonl`.
|
||||
|
||||
### `verify` — is the proxy really alive
|
||||
|
||||
```bash
|
||||
python urnet.py verify --pool pool.jsonl --limit 20 --shuffle --out verify.json
|
||||
```
|
||||
|
||||
One `curl https://ipinfo.io/json` per proxy. Summary line:
|
||||
|
||||
```
|
||||
TOTAL 20 | alive 19 | unique_ip 18 | countries 9 | hosting_exits 2
|
||||
```
|
||||
|
||||
- `alive` — the proxy answered.
|
||||
- `unique_ip` — how many **distinct** exit IPs you really have; this is the number that matters.
|
||||
- `hosting_exits` — rows whose ASN looks like a datacenter/VPN (`3xK`, `OVH`, `Hetzner`, `M247`…).
|
||||
They still proxy traffic but they are not residential; drop them for anything geo-sensitive.
|
||||
|
||||
### `export` — for other tools
|
||||
|
||||
```bash
|
||||
python urnet.py export --pool pool.jsonl --format txt --out proxies.txt # one URL per line
|
||||
python urnet.py export --pool pool.jsonl --format csv --out proxies.csv --cc us # filter by country
|
||||
python urnet.py export --pool pool.jsonl --format env --out proxy.env # HTTP_PROXY/HTTPS_PROXY
|
||||
```
|
||||
|
||||
### `balance` — traffic left
|
||||
|
||||
```bash
|
||||
python urnet.py balance --bank bank.jsonl
|
||||
```
|
||||
|
||||
Walks every account and prints `left / start / open_transfer` plus the total left today. `start` is
|
||||
the daily allowance (30 GiB), so `left` is what you can still push until 00:00 UTC.
|
||||
|
||||
## 6. Scaling past the IP cap
|
||||
|
||||
Free accounts are capped **per IP address** (~4, then HTTP 429). The cap follows the address you
|
||||
connect from — `px.txt` exists to change that address.
|
||||
|
||||
```bash
|
||||
# step 1 - a few accounts directly
|
||||
python urnet.py farm --count 3
|
||||
|
||||
# step 2 - export what they gave you as egress
|
||||
python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
python urnet.py export --pool pool.jsonl --format txt --out px.txt
|
||||
|
||||
# step 3 - keep farming, rotating egress through your own proxies
|
||||
python urnet.py farm --count 20 --proxies 5 --proxy-file px.txt --start 100
|
||||
|
||||
# step 4 - grow the pool again
|
||||
python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
```
|
||||
|
||||
Measured: step 3 ran 8 accounts back-to-back with zero 429s. After each wave the pool grows
|
||||
(5 new proxies per account), so the next wave has more egress to rotate through.
|
||||
|
||||
Guard rails the script applies itself:
|
||||
|
||||
- retries only while failures stay under sanity thresholds, then stops with a clear message instead
|
||||
of hammering the API;
|
||||
- keeps `--delay 1.3` between proxy calls (the per-account limit is ~5 proxies/minute);
|
||||
- writes partial progress, so a stop costs you nothing.
|
||||
|
||||
Practical free ceiling: ~4 accounts per fresh IP, and each account yields 5 more IPs, so the pool
|
||||
compounds. 12 accounts = 360 GiB/day and 49 usable proxies is what one evening produced.
|
||||
|
||||
## 7. Using the proxies
|
||||
|
||||
```bash
|
||||
# curl
|
||||
curl --proxy "$(head -1 proxies.txt)" https://ipinfo.io/json
|
||||
|
||||
# environment (whole shell)
|
||||
python urnet.py export --pool pool.jsonl --format env --out proxy.env
|
||||
set -a; . ./proxy.env; set +a
|
||||
```
|
||||
|
||||
```python
|
||||
# python requests, random pick
|
||||
import random, requests
|
||||
urls = [l.strip() for l in open("proxies.txt") if l.strip()]
|
||||
r = requests.get("https://ipinfo.io/json", proxies={"https": random.choice(urls)}, timeout=30)
|
||||
```
|
||||
|
||||
All URLs are HTTPS CONNECT with the credential inside the URL — paste and go, no user/password
|
||||
fields to fill.
|
||||
|
||||
## 8. Caveats worth knowing
|
||||
|
||||
- **Exit IP is not pinned.** The same URL returned `188.103.48.237 AS3209 Vodafone Augsburg` and
|
||||
later `80.171.40.200 AS6805 Telefonica Essen`. For anything needing a stable address, enable
|
||||
*Fixed IP* on the proxy in the web UI, or treat the address as best-effort.
|
||||
- **The exit country can be wrong.** A proxy created with `country_code: br` came back as
|
||||
`209.50.178.206 AS200373 3xK Tech GmbH, Toronto`. The network is P2P; a peer can sit anywhere.
|
||||
Always `verify` and filter by `country`.
|
||||
- **`country_code` is mandatory** — omit `initial_device_state.country_code` and the API answers
|
||||
`Invalid location`.
|
||||
- **5 proxies is the free cap.** The 6th call returns `client_limit_exceeded` with
|
||||
`upgrade_required: true`.
|
||||
- **`--proxies` above 5 is pointless** — the extra calls only return the cap error.
|
||||
- **Do not do this in a browser.** The UI "Copy URL" copies the card you clicked, not the one you
|
||||
meant, and clipboard reads fail when the window is not focused. The API returns every URL.
|
||||
|
||||
## 9. Troubleshooting
|
||||
|
||||
| symptom | cause | fix |
|
||||
|---|---|---|
|
||||
| `429 Too many accounts have been created recently from your network address` | per-IP account cap (~4) | `--proxy-file px.txt` to rotate egress, or wait for the reset |
|
||||
| `Invalid location` | missing/invalid `country_code` | pass `--countries us,de` (2-letter ISO) |
|
||||
| `client_limit_exceeded` / `upgrade_required` | already 5 proxies | delete one in the UI or lower `--proxies` |
|
||||
| `curl returned nothing through this proxy` | dead proxy or rotated token | re-run `collect`; if it persists the account was reset |
|
||||
| `Remote end closed connection without response` | urllib cannot do TLS-in-TLS to an `https://` proxy | the script auto-uses curl; make sure `curl` is installed |
|
||||
| `create failed (0): URLError ...` with no proxy | no internet / DNS / firewall | `curl -s https://api.bringyour.com` and check connectivity |
|
||||
| account created but no proxies | hit the 5/min rate | raise `--delay` to 2.0 and retry with a new `--start` |
|
||||
| `balance` prints nothing for some rows | `bank.jsonl` keeps failed creates for the record | expected — those have no `urn` |
|
||||
| proxies answer `FAIL` in `verify` | peer went offline | normal churn; re-run `collect` and `verify`, drop dead rows |
|
||||
|
||||
Exit code is `0` on success, `1` when nothing was produced — safe to use in scripts.
|
||||
|
||||
## 10. Hygiene
|
||||
|
||||
- `bank.jsonl` = full control of every account. `chmod 600 bank.jsonl`, keep it out of git
|
||||
(`.gitignore` already lists it), back it up.
|
||||
- `pool.jsonl` and `proxies.txt` contain proxy tokens — anyone with the file can spend your traffic.
|
||||
- Nothing here needs a card, an email or a phone number, so there is no PII to leak.
|
||||
- The tool writes no logs besides the bank/pool files.
|
||||
|
||||
## 11. FAQ
|
||||
|
||||
**Can I choose the account name?** No, the server generates it.
|
||||
|
||||
**Can I use these proxies in a browser?** Yes, as an HTTPS proxy. Chrome:
|
||||
`--proxy-server="https://<token>.crisp.bringyour.com:7155"`. Playwright/Camoufox: the same string as
|
||||
`proxy.server`.
|
||||
|
||||
**Do they work for Gmail / Google sign-ups?** No. Google treats datacenter and rotating residential
|
||||
ranges as suspicious; those flows need a mobile or home line.
|
||||
|
||||
**How much traffic is enough?** One account covers ~30 GiB of scraping per day. Ten accounts are a
|
||||
comfortable pool for a small crawler; heavy media work needs Pro (10 TiB/month).
|
||||
|
||||
**Does the free tier expire?** No trial clock — the daily allowance simply resets every day.
|
||||
|
||||
**What happens after 30 GiB?** Requests stop passing until the 00:00 UTC reset; `balance` shows the
|
||||
remaining bytes.
|
||||
|
||||
**Is there a way to get more per account?** Pro: 10 TiB/month and 1000 proxies, $3 for the first
|
||||
year (USDC / Zcash / Monero or a card).
|
||||
|
||||
## 12. Scope
|
||||
|
||||
This tool creates accounts you own and prints the proxy endpoints they generate. It handles no card
|
||||
data, no third-party accounts and no payment flows. URnetwork's terms of service apply — read them
|
||||
before running large waves, and do not use the pool for anything you would not do from your own line.
|
||||
@@ -1,89 +1,142 @@
|
||||
# ur-network-farm
|
||||
# urnet — URnetwork free-tier farm
|
||||
|
||||
Free-tier **residential** egress from URnetwork (ur.io) without a card, an email, or a browser.
|
||||
Everything runs over the public API — an account, an API key and up to five hosted proxies in
|
||||
about 20 seconds.
|
||||
Turn URnetwork ([ur.io](https://ur.io)) into a rotating **residential** proxy pool. Every account
|
||||
you create gives **30 GiB of traffic per day** and **5 hosted proxies** — no card, no email, no
|
||||
captcha, no browser. Everything runs over the public API.
|
||||
|
||||
```bash
|
||||
python urnet.py farm --count 1 --proxies 3 # account + proxies, ~25 s
|
||||
python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
python urnet.py verify --pool pool.jsonl --limit 3
|
||||
```
|
||||
|
||||
That is the whole quick start. One account ~ 25 seconds ~ 30 GiB/day.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python **3.8+** — standard library only, nothing to `pip install`
|
||||
- `curl` — optional, but required when you route a run through a proxy (`--proxy` / `--proxy-file`).
|
||||
Windows 10/11, macOS and most Linux distros ship it. Without curl, direct (non-proxied) calls work.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
git clone https://git.reformboss.com/reform/ur-network-farm.git
|
||||
cd ur-network-farm
|
||||
python urnet.py --help
|
||||
```
|
||||
|
||||
No virtualenv, no requirements.txt, no build step, no config file.
|
||||
|
||||
## First run
|
||||
|
||||
```bash
|
||||
python urnet.py farm --count 1 # 1 account, 5 proxies in 5 countries
|
||||
|
||||
python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
python urnet.py export --pool pool.jsonl --format txt --out proxies.txt
|
||||
python urnet.py verify --pool pool.jsonl --limit 5
|
||||
python urnet.py balance --bank bank.jsonl
|
||||
```
|
||||
|
||||
`proxies.txt` is a plain list of ready-to-use URLs:
|
||||
|
||||
```
|
||||
https://<token>.crisp.bringyour.com:7155
|
||||
```
|
||||
|
||||
Use one directly:
|
||||
|
||||
```bash
|
||||
curl --proxy "$(head -1 proxies.txt)" https://ipinfo.io/json
|
||||
```
|
||||
|
||||
## How it works
|
||||
|
||||
Four API calls per account, no UI involved:
|
||||
|
||||
```
|
||||
POST /auth/network-create {"terms":true,"product_updates":false} -> network.by_jwt
|
||||
POST /auth/regenerate-seedphrase -> seedphrase (24 words)
|
||||
POST /account/api-key {"name":"farm-1"} -> urn_... key
|
||||
POST /account/api-key {"name":"urnet-1"} -> urn_... key
|
||||
POST /network/auth-client {"device_spec":"resident proxy",
|
||||
"proxy_config":{"initial_device_state":{"country_code":"us"}}}
|
||||
-> https_proxy_url (creds inside)
|
||||
GET /network/proxies -> every proxy of the account
|
||||
GET /subscription/balance -> bytes left today
|
||||
-> https_proxy_url
|
||||
```
|
||||
|
||||
Base: `https://api.bringyour.com`. Auth: `Authorization: Bearer <by_jwt | urn_key>`.
|
||||
Base: `https://api.bringyour.com`. Auth header: `Authorization: Bearer <jwt | urn_ key>`.
|
||||
`GET /network/proxies` returns every proxy of an account, `GET /subscription/balance` returns
|
||||
traffic left today.
|
||||
|
||||
## Limits (measured 2026-11-13)
|
||||
## Limits (measured, not guessed)
|
||||
|
||||
| limit | value | proof |
|
||||
|---|---|---|
|
||||
| traffic per account | **30 GiB / day**, resets 00:00 UTC | `start_balance_byte_count: 32212254720`, `start_time 2026-11-13T00:00:00Z → end_time 2026-11-14T01:00:00Z` |
|
||||
| active proxies | **5 per account** | 6th call → `{"error":{"client_limit_exceeded":true,"upgrade_required":true,"message":"Your plan's limit of 5 active proxies is reached..."}}` |
|
||||
| proxy creation rate | ~5 / minute / account | 429 on bursts; script sleeps 1.3 s |
|
||||
| instant accounts per IP | ~4, then 429 | `Too many accounts have been created recently from your network address. This limit is scoped to the address you are connecting from and is shared with everyone` |
|
||||
| countries | 20+ residential providers | 15 verified by `ipinfo` in one evening (US DE NL FR GB CA BR SE PL CH JP SG IT ES AT) |
|
||||
| limit | value |
|
||||
|---|---|
|
||||
| traffic per account | **30 GiB / day**, resets 00:00 UTC |
|
||||
| proxies per account | **5 active** — the 6th answers `client_limit_exceeded` |
|
||||
| proxy creation rate | ~5 / minute / account |
|
||||
| accounts per IP | **~4**, then `429 Too many accounts have been created recently from your network address` |
|
||||
| countries seen | 15 verified by `ipinfo`: US DE NL FR GB CA BR SE PL CH JP SG IT ES AT |
|
||||
|
||||
Bypass for the IP cap: create the first ~4 accounts directly, then feed their own proxy URLs back
|
||||
into the farm as rotating egress (`--proxy-file px.txt`). 8 more accounts went through with zero 429.
|
||||
Proof of the caps, straight from the API:
|
||||
|
||||
Pro tier ($3 first year, USDC/Zcash/Monero or card) raises it to 10 TiB/month and 1000 proxies.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# 5 accounts, 5 proxies each, direct connection
|
||||
python ur_farm.py --count 5 --proxies 5 --bank bank.jsonl
|
||||
|
||||
# more accounts, rotating egress through proxies we already own
|
||||
python ur_farm.py --count 20 --proxies 5 --proxy-file px.txt --bank bank.jsonl --start 500
|
||||
|
||||
# export every proxy URL into a flat pool
|
||||
python collect.py bank.jsonl > pool.jsonl
|
||||
|
||||
# liveness pass over the pool (ipinfo through each proxy)
|
||||
python verify.py pool.jsonl --limit 20
|
||||
```
|
||||
start_balance_byte_count: 32212254720 (30 GiB, window 2026-11-13T00:00:00Z -> 2026-11-14T01:00:00Z)
|
||||
6th proxy: {"error":{"client_limit_exceeded":true,"upgrade_required":true,
|
||||
"message":"Your plan's limit of 5 active proxies is reached..."}}
|
||||
```
|
||||
|
||||
`bank.jsonl` line shape:
|
||||
Workaround for the IP cap: create the first few accounts directly, then feed their own proxy URLs
|
||||
back into the farm as rotating egress — `--proxy-file px.txt`. Recipe:
|
||||
[GUIDE.md](GUIDE.md#scaling-past-the-ip-cap).
|
||||
|
||||
```json
|
||||
{"idx":300,"network_id":"...","name":"february-exhaust-83","jwt":"...","seed":"24 words",
|
||||
"urn":"urn_...","proxies":[{"cc":"us","url":"https://<token>@host:7155","socks":"socks5h://..."}]}
|
||||
```
|
||||
## Commands
|
||||
|
||||
| command | what it does |
|
||||
|---|---|
|
||||
| `farm` | create accounts: JWT, seed phrase, `urn_` API key, hosted proxies |
|
||||
| `collect` | flatten `bank.jsonl` into a deduplicated proxy pool |
|
||||
| `verify` | ipinfo through every proxy -> `alive / unique_ip / countries / hosting_exits` |
|
||||
| `export` | pool -> `txt` / `csv` / `env` |
|
||||
| `balance` | traffic left per account |
|
||||
|
||||
Every flag, recipe and failure mode: **[GUIDE.md](GUIDE.md)**.
|
||||
|
||||
## Proxy URL shape
|
||||
|
||||
```
|
||||
https://<40-char-token>.crisp.bringyour.com:7155 # HTTPS CONNECT, token is the credential
|
||||
socks5h://crisp.bringyour.com:7153 # SOCKS5 with username/password auth
|
||||
socks5h://crisp.bringyour.com:7153 # SOCKS5, username/password auth
|
||||
http://crisp.bringyour.com:7154 # plain HTTP
|
||||
```
|
||||
|
||||
`https_proxy_url` carries the token inline, so `curl --proxy "$url"` works as-is.
|
||||
|
||||
## Pitfalls
|
||||
## Files it writes
|
||||
|
||||
- **`country_code` is mandatory** in `initial_device_state` — omit it and the call answers `Invalid location`.
|
||||
- **Exit IP is not pinned.** The same proxy URL returned `188.103.48.237 AS3209 Vodafone Augsburg`
|
||||
and later `80.171.40.200 AS6805 Telefonica Essen`. Enable *Fixed IP* in the proxy options when a
|
||||
stable address matters, or treat stability as a known risk.
|
||||
- **Save the seed before anything else.** The account has no email and no password recovery; losing
|
||||
the JWT/seed loses the account.
|
||||
- **Do not drive the browser for this.** UI proxy creation needs clipboard reads that fail when the
|
||||
document is not focused, and copies the URL of the wrong card. `GET /network/proxies` returns every
|
||||
URL directly.
|
||||
- Proxies are residential but shared: several accounts can land on the same provider
|
||||
(two `Vodafone Hildesheim` exits were seen), so `unique_ip` is the metric, not `alive`.
|
||||
- **Not every exit is residential.** A proxy created with `country_code: br` came back as
|
||||
`209.50.178.206 AS200373 3xK Tech GmbH, Toronto` — a hosting ASN in the wrong country. The network
|
||||
is P2P, so peers supply the exit; check `ipinfo` (`org`/`country`) per proxy before trusting it for
|
||||
anything geo-sensitive, and drop rows where the ASN is a hosting provider.
|
||||
| file | content | secret? |
|
||||
|---|---|---|
|
||||
| `bank.jsonl` | one line per account: name, jwt, seed, `urn_` key, proxies | **yes** |
|
||||
| `pool.jsonl` | deduplicated proxy URLs | yes (token inside URL) |
|
||||
| `proxies.txt` / `.csv` | exports for other tools | yes |
|
||||
| `verify.json` | pool enriched with ip / country / org | no |
|
||||
|
||||
`.gitignore` excludes all of them. Keep `bank.jsonl` at `chmod 600`: the account has no email and no
|
||||
password recovery, so losing the file loses the account, and leaking it hands the account over.
|
||||
|
||||
## Troubleshooting (short version)
|
||||
|
||||
| symptom | fix |
|
||||
|---|---|
|
||||
| `429 Too many accounts ... from your network address` | add `--proxy-file px.txt` (rotate egress), or wait for the reset |
|
||||
| `Invalid location` | `country_code` missing — pass `--countries us,de` |
|
||||
| `Your plan's limit of 5 active proxies is reached` | 5 proxies per account is the free cap; `--proxies 5` max |
|
||||
| `curl returned nothing through this proxy` | proxy dead or token rotated — re-run `collect` |
|
||||
| nothing created, no error | `curl` not on PATH — install it or drop the proxy flags |
|
||||
|
||||
Full table with causes: [GUIDE.md](GUIDE.md#troubleshooting).
|
||||
|
||||
## Scope
|
||||
|
||||
Free-tier testing of a public API on accounts we create ourselves. No card data, no third-party
|
||||
accounts, no payment flows. Respect the provider's terms — this is a field note, not a licence.
|
||||
Free-tier testing of a public API on accounts you create yourself. No card data, no third-party
|
||||
accounts, no payment flows. Respect the provider's terms of service.
|
||||
|
||||
-31
@@ -1,31 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Flatten a bank.jsonl into a proxy pool: one JSON line per proxy URL.
|
||||
|
||||
Usage: python collect.py [bank.jsonl] > pool.jsonl
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
|
||||
src = sys.argv[1] if len(sys.argv) > 1 else "bank.jsonl"
|
||||
seen = set()
|
||||
out = sys.stdout
|
||||
n = 0
|
||||
for line in open(src, encoding="utf-8"):
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except Exception:
|
||||
continue
|
||||
for p in rec.get("proxies") or []:
|
||||
url = p.get("url")
|
||||
if not url or url in seen:
|
||||
continue
|
||||
seen.add(url)
|
||||
row = {"acct": rec.get("idx"), "name": rec.get("name"), "cc": p.get("cc"),
|
||||
"url": url, "socks": p.get("socks"), "client_id": p.get("client_id")}
|
||||
out.write(json.dumps(row, ensure_ascii=False) + "\n")
|
||||
n += 1
|
||||
sys.stderr.write("proxies: %d (unique) from %d bank lines\n" % (n, len(seen)))
|
||||
-143
@@ -1,143 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
"""URnetwork farm — instant networks, API keys, hosted proxies. No browser.
|
||||
|
||||
POST /auth/network-create {"terms": true} -> by_jwt (an account with no email)
|
||||
then: seedphrase, urn_ key, N hosted HTTPS proxies by country.
|
||||
|
||||
Usage: python ur_farm.py --count 5 [--proxies 5] [--bank bank.jsonl]
|
||||
"""
|
||||
import argparse, json, os, ssl, subprocess, sys, time, urllib.request, urllib.error
|
||||
|
||||
sys.stdout.reconfigure(encoding="utf-8")
|
||||
BASE = "https://api.bringyour.com"
|
||||
CTX = ssl.create_default_context()
|
||||
CTX.check_hostname = False
|
||||
CTX.verify_mode = ssl.CERT_NONE
|
||||
CODES = ["us", "de", "nl", "fr", "gb", "ca", "br", "se", "pl", "ch", "jp", "sg", "it", "es", "at"]
|
||||
|
||||
|
||||
PROXY = None
|
||||
|
||||
|
||||
def call(path, method="GET", payload=None, tok=None, timeout=60):
|
||||
if PROXY:
|
||||
return call_curl(path, method, payload, tok, timeout)
|
||||
data = json.dumps(payload).encode() if payload is not None else None
|
||||
h = {"Accept": "application/json", "User-Agent": "Mozilla/5.0"}
|
||||
if data:
|
||||
h["Content-Type"] = "application/json"
|
||||
if tok:
|
||||
h["Authorization"] = "Bearer " + tok
|
||||
req = urllib.request.Request(BASE + path, data=data, headers=h, method=method)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout, context=CTX) as r:
|
||||
body = r.read(60000).decode("utf-8", "replace")
|
||||
try:
|
||||
return r.status, json.loads(body)
|
||||
except Exception:
|
||||
return r.status, body
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, e.read(2000).decode("utf-8", "replace")[:400]
|
||||
except Exception as e:
|
||||
return 0, "%s: %s" % (type(e).__name__, str(e)[:120])
|
||||
|
||||
|
||||
def call_curl(path, method, payload, tok, timeout):
|
||||
cmd = ["curl", "-s", "-m", str(timeout), "--proxy", PROXY, "-X", method,
|
||||
BASE + path, "-H", "Accept: application/json",
|
||||
"-H", "User-Agent: Mozilla/5.0"]
|
||||
if tok:
|
||||
cmd += ["-H", "Authorization: Bearer " + tok]
|
||||
if payload is not None:
|
||||
cmd += ["-H", "Content-Type: application/json", "-d", json.dumps(payload)]
|
||||
try:
|
||||
out = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout + 20).stdout
|
||||
try:
|
||||
return 200, json.loads(out)
|
||||
except Exception:
|
||||
return 0, out[:400]
|
||||
except Exception as e:
|
||||
return 0, "%s: %s" % (type(e).__name__, str(e)[:100])
|
||||
|
||||
|
||||
def one(idx, nproxy, country_offset):
|
||||
st, d = call("/auth/network-create", "POST",
|
||||
{"terms": True, "product_updates": False})
|
||||
if not isinstance(d, dict) or "network" not in d:
|
||||
return {"idx": idx, "error": "create %s %s" % (st, str(d)[:160])}
|
||||
net = d["network"]
|
||||
jwt = net.get("by_jwt")
|
||||
if not jwt:
|
||||
return {"idx": idx, "error": "no jwt"}
|
||||
rec = {"idx": idx, "network_id": net.get("network_id"),
|
||||
"name": net.get("network_name"), "jwt": jwt, "proxies": []}
|
||||
|
||||
st, sd = call("/auth/regenerate-seedphrase", "POST", {}, jwt)
|
||||
seed = sd.get("seedphrase") if isinstance(sd, dict) else None
|
||||
if isinstance(seed, list):
|
||||
seed = " ".join(seed)
|
||||
rec["seed"] = seed
|
||||
|
||||
st, kd = call("/account/api-key", "POST", {"name": "farm-%d" % idx}, jwt)
|
||||
key = kd.get("api_key") if isinstance(kd, dict) else None
|
||||
if not key:
|
||||
rec["error"] = "api-key %s %s" % (st, str(kd)[:120])
|
||||
return rec
|
||||
rec["urn"] = key
|
||||
|
||||
for i in range(nproxy):
|
||||
cc = CODES[(country_offset + i) % len(CODES)]
|
||||
st, pd = call("/network/auth-client", "POST", {
|
||||
"description": "HTTPS proxy · %s" % cc.upper(),
|
||||
"device_spec": "resident proxy",
|
||||
"proxy_config": {"initial_device_state": {"country_code": cc}}}, key)
|
||||
r = (pd or {}).get("proxy_config_result") if isinstance(pd, dict) else None
|
||||
if r and r.get("https_proxy_url"):
|
||||
rec["proxies"].append({"cc": cc, "url": r["https_proxy_url"],
|
||||
"socks": r.get("socks_proxy_url"),
|
||||
"client_id": pd.get("client_id")})
|
||||
else:
|
||||
err = (pd or {}).get("error") if isinstance(pd, dict) else pd
|
||||
rec.setdefault("proxy_errors", []).append("%s: %s" % (cc, str(err)[:80]))
|
||||
time.sleep(1.3)
|
||||
return rec
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--count", type=int, default=5)
|
||||
ap.add_argument("--proxies", type=int, default=5)
|
||||
ap.add_argument("--bank", default="bank.jsonl")
|
||||
ap.add_argument("--start", type=int, default=100)
|
||||
ap.add_argument("--proxy", default="", help="http(s) proxy URL for the whole run")
|
||||
ap.add_argument("--proxy-file", default="", help="file with one proxy URL per line (rotates)")
|
||||
a = ap.parse_args()
|
||||
|
||||
global PROXY
|
||||
pool = []
|
||||
if a.proxy_file and os.path.exists(a.proxy_file):
|
||||
pool = [l.strip() for l in open(a.proxy_file, encoding="utf-8") if l.strip()]
|
||||
made, okp = 0, 0
|
||||
for n in range(a.count):
|
||||
if a.proxy:
|
||||
PROXY = a.proxy
|
||||
elif pool:
|
||||
PROXY = pool[n % len(pool)]
|
||||
idx = a.start + n
|
||||
rec = one(idx, a.proxies, n * 3)
|
||||
with open(a.bank, "a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
|
||||
if rec.get("urn"):
|
||||
made += 1
|
||||
okp += len(rec.get("proxies", []))
|
||||
print("acct%d %-16s proxies=%d %s" % (idx, rec.get("name"),
|
||||
len(rec.get("proxies", [])), ",".join(p["cc"] for p in rec.get("proxies", []))))
|
||||
else:
|
||||
print("acct%d FAIL %s" % (idx, rec.get("error")))
|
||||
time.sleep(1.5)
|
||||
print(json.dumps({"accounts_ok": made, "proxies_ok": okp, "bank": a.bank}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,499 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""urnet - URnetwork free-tier farm: accounts, API keys, hosted proxies.
|
||||
|
||||
One file, standard library only (curl is optional, used as a fallback transport).
|
||||
Works on Windows / Linux / macOS with Python 3.8+.
|
||||
|
||||
python urnet.py farm --count 3 --proxies 5
|
||||
python urnet.py collect --bank bank.jsonl --out pool.jsonl
|
||||
python urnet.py verify --pool pool.jsonl
|
||||
python urnet.py export --pool pool.jsonl --format txt --out proxies.txt
|
||||
python urnet.py balance --bank bank.jsonl
|
||||
|
||||
Docs: README.md (quick start) and GUIDE.md (everything else).
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import shutil
|
||||
import ssl
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
API = "https://api.bringyour.com"
|
||||
DEFAULT_COUNTRIES = ["us", "de", "nl", "fr", "gb", "ca", "br", "se", "pl", "ch",
|
||||
"jp", "sg", "it", "es", "at"]
|
||||
UA = "urnet/1.1 (+https://git.reformboss.com/reform/ur-network-farm)"
|
||||
HAVE_CURL = shutil.which("curl") is not None
|
||||
|
||||
SSLCTX = ssl.create_default_context()
|
||||
SSLCTX.check_hostname = False
|
||||
SSLCTX.verify_mode = ssl.CERT_NONE
|
||||
|
||||
|
||||
def setup_stdout():
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def human(n):
|
||||
if n is None:
|
||||
return "?"
|
||||
n = float(n)
|
||||
for unit in ("B", "KiB", "MiB", "GiB", "TiB"):
|
||||
if abs(n) < 1024 or unit == "TiB":
|
||||
return ("%.2f %s" % (n, unit)) if unit != "B" else ("%d B" % n)
|
||||
n /= 1024.0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- transport
|
||||
def http_json(path, method="GET", payload=None, token=None, proxy=None,
|
||||
timeout=60, use_curl=False):
|
||||
"""Return (status, parsed_body_or_error_string). Never raises."""
|
||||
url = API + path
|
||||
body = json.dumps(payload).encode() if payload is not None else None
|
||||
|
||||
if proxy and (use_curl or HAVE_CURL):
|
||||
cmd = ["curl", "-s", "-m", str(timeout), "--proxy", proxy, "-X", method, url,
|
||||
"-H", "Accept: application/json", "-H", "User-Agent: " + UA]
|
||||
if token:
|
||||
cmd += ["-H", "Authorization: Bearer " + token]
|
||||
if body:
|
||||
cmd += ["-H", "Content-Type: application/json", "-d", body.decode()]
|
||||
try:
|
||||
out = subprocess.run(cmd, capture_output=True, text=True,
|
||||
timeout=timeout + 25).stdout
|
||||
except Exception as exc:
|
||||
return 0, "%s: %s" % (type(exc).__name__, str(exc)[:120])
|
||||
st, body = _parse(out)
|
||||
if not out.strip() and not use_curl:
|
||||
return 0, "curl returned nothing through this proxy"
|
||||
return st, body
|
||||
|
||||
handlers = []
|
||||
if proxy:
|
||||
handlers.append(urllib.request.ProxyHandler({"http": proxy, "https": proxy}))
|
||||
else:
|
||||
handlers.append(urllib.request.ProxyHandler({})) # ignore env proxies
|
||||
handlers.append(urllib.request.HTTPSHandler(context=SSLCTX))
|
||||
opener = urllib.request.build_opener(*handlers)
|
||||
|
||||
headers = {"Accept": "application/json", "User-Agent": UA}
|
||||
if body is not None:
|
||||
headers["Content-Type"] = "application/json"
|
||||
if token:
|
||||
headers["Authorization"] = "Bearer " + token
|
||||
req = urllib.request.Request(url, data=body, headers=headers, method=method)
|
||||
try:
|
||||
with opener.open(req, timeout=timeout) as resp:
|
||||
return _parse(resp.read(120000).decode("utf-8", "replace"), resp.status)
|
||||
except urllib.error.HTTPError as exc:
|
||||
return exc.code, exc.read(4000).decode("utf-8", "replace")
|
||||
except Exception as exc:
|
||||
return 0, "%s: %s" % (type(exc).__name__, str(exc)[:140])
|
||||
|
||||
|
||||
def _parse(raw, status=200):
|
||||
try:
|
||||
return status, json.loads(raw)
|
||||
except Exception:
|
||||
return status, raw[:400]
|
||||
|
||||
|
||||
def err_text(body):
|
||||
"""Pull a readable message out of an error body."""
|
||||
if isinstance(body, dict):
|
||||
e = body.get("error")
|
||||
if isinstance(e, dict):
|
||||
return e.get("message") or json.dumps(e)[:200]
|
||||
if isinstance(e, str):
|
||||
return e
|
||||
return json.dumps(body)[:200]
|
||||
return str(body)[:200]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- core steps
|
||||
def create_account(proxy=None, use_curl=False, timeout=60):
|
||||
st, d = http_json("/auth/network-create", "POST",
|
||||
{"terms": True, "product_updates": False},
|
||||
proxy=proxy, use_curl=use_curl, timeout=timeout)
|
||||
if not isinstance(d, dict) or "network" not in d:
|
||||
return None, "create failed (%s): %s" % (st, err_text(d))
|
||||
net = d["network"]
|
||||
if not net.get("by_jwt"):
|
||||
return None, "create failed: no by_jwt in response"
|
||||
return {"network_id": net.get("network_id"),
|
||||
"name": net.get("network_name"),
|
||||
"jwt": net["by_jwt"]}, None
|
||||
|
||||
|
||||
def get_seed(jwt, proxy=None, use_curl=False):
|
||||
st, d = http_json("/auth/regenerate-seedphrase", "POST", {}, jwt,
|
||||
proxy=proxy, use_curl=use_curl)
|
||||
if isinstance(d, dict):
|
||||
seed = d.get("seedphrase")
|
||||
if isinstance(seed, list):
|
||||
seed = " ".join(seed)
|
||||
if seed:
|
||||
return seed, None
|
||||
return None, "seed failed (%s): %s" % (st, err_text(d))
|
||||
|
||||
|
||||
def get_key(jwt, name="urnet", proxy=None, use_curl=False):
|
||||
st, d = http_json("/account/api-key", "POST", {"name": name}, jwt,
|
||||
proxy=proxy, use_curl=use_curl)
|
||||
if isinstance(d, dict) and d.get("api_key"):
|
||||
return d["api_key"], None
|
||||
return None, "api-key failed (%s): %s" % (st, err_text(d))
|
||||
|
||||
|
||||
def make_proxy(key, cc, proxy=None, use_curl=False):
|
||||
st, d = http_json("/network/auth-client", "POST", {
|
||||
"description": "HTTPS proxy · %s" % cc.upper(),
|
||||
"device_spec": "resident proxy",
|
||||
"proxy_config": {"initial_device_state": {"country_code": cc}},
|
||||
}, key, proxy=proxy, use_curl=use_curl, timeout=90)
|
||||
if isinstance(d, dict):
|
||||
r = d.get("proxy_config_result") or {}
|
||||
if r.get("https_proxy_url"):
|
||||
return {"cc": cc, "url": r["https_proxy_url"],
|
||||
"socks": r.get("socks_proxy_url"),
|
||||
"client_id": d.get("client_id")}, None
|
||||
return None, err_text(d)
|
||||
return None, "proxy failed (%s): %s" % (st, str(d)[:160])
|
||||
|
||||
|
||||
def list_proxies(key, proxy=None, use_curl=False):
|
||||
st, d = http_json("/network/proxies", "GET", None, key,
|
||||
proxy=proxy, use_curl=use_curl)
|
||||
if not isinstance(d, dict):
|
||||
return [], "proxies failed (%s): %s" % (st, str(d)[:160])
|
||||
out = []
|
||||
for c in d.get("clients") or []:
|
||||
pc = c.get("proxy_client") or {}
|
||||
if pc.get("https_proxy_url"):
|
||||
out.append({"cc": (c.get("description") or "").split("·")[-1].strip().lower(),
|
||||
"url": pc["https_proxy_url"],
|
||||
"socks": pc.get("socks_proxy_url"),
|
||||
"client_id": c.get("client_id")})
|
||||
return out, None
|
||||
|
||||
|
||||
def get_balance(key, proxy=None, use_curl=False):
|
||||
st, d = http_json("/subscription/balance", "GET", None, key,
|
||||
proxy=proxy, use_curl=use_curl)
|
||||
if not isinstance(d, dict):
|
||||
return None, "balance failed (%s): %s" % (st, str(d)[:160])
|
||||
return {"left": d.get("balance_byte_count"),
|
||||
"start": d.get("start_balance_byte_count"),
|
||||
"open_transfer": d.get("open_transfer_byte_count")}, None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- helpers
|
||||
def load_proxies(path):
|
||||
if not path or not os.path.exists(path):
|
||||
return []
|
||||
out = []
|
||||
for line in open(path, encoding="utf-8"):
|
||||
line = line.strip()
|
||||
if line and not line.startswith("#"):
|
||||
out.append(line)
|
||||
return out
|
||||
|
||||
|
||||
def append_jsonl(path, rec):
|
||||
with open(path, "a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
|
||||
fh.flush()
|
||||
|
||||
|
||||
def is_ip_limit(msg):
|
||||
low = str(msg).lower()
|
||||
return "too many accounts" in low or "429" in low
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- commands
|
||||
def cmd_farm(a):
|
||||
pool = load_proxies(a.proxy_file)
|
||||
if a.proxy:
|
||||
pool = [a.proxy]
|
||||
if a.shuffle and len(pool) > 1:
|
||||
random.shuffle(pool)
|
||||
|
||||
countries = [c.strip().lower() for c in a.countries.split(",") if c.strip()] \
|
||||
if a.countries else DEFAULT_COUNTRIES
|
||||
|
||||
made = proxies = 0
|
||||
failures = 0
|
||||
print("farm: %d account(s) x %d proxies | egress: %s | bank: %s"
|
||||
% (a.count, a.proxies, ("%d rotating" % len(pool)) if pool else "direct", a.bank))
|
||||
|
||||
for n in range(a.count):
|
||||
idx = a.start + n
|
||||
px = None
|
||||
if pool:
|
||||
px = pool[(n + random.randrange(len(pool))) % len(pool)]
|
||||
|
||||
rec = {"idx": idx, "created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"egress": "rotating" if px else "direct", "proxies": []}
|
||||
acc, err = create_account(px, a.curl)
|
||||
if err:
|
||||
failures += 1
|
||||
print(" #%d FAIL %s" % (idx, err))
|
||||
if is_ip_limit(err):
|
||||
if not pool:
|
||||
print(" -> this IP has hit the account cap. Add --proxy-file px.txt "
|
||||
"with proxy URLs you already own, or wait for the reset.")
|
||||
elif failures >= max(3, len(pool) // 4 + 2):
|
||||
print(" -> every egress is capped right now; stopping.")
|
||||
break
|
||||
append_jsonl(a.bank, dict(rec, error=err))
|
||||
continue
|
||||
|
||||
rec.update(acc)
|
||||
seed, e1 = get_seed(acc["jwt"], px, a.curl)
|
||||
if e1:
|
||||
rec["error"] = e1
|
||||
else:
|
||||
rec["seed"] = seed
|
||||
key, e2 = get_key(acc["jwt"], "urnet-%d" % idx, px, a.curl)
|
||||
if e2:
|
||||
rec["error"] = e2
|
||||
append_jsonl(a.bank, rec)
|
||||
failures += 1
|
||||
print(" #%d FAIL %s" % (idx, e2))
|
||||
continue
|
||||
rec["urn"] = key
|
||||
|
||||
for i in range(a.proxies):
|
||||
cc = countries[(n * a.proxies + i) % len(countries)]
|
||||
pr, e3 = make_proxy(key, cc, px, a.curl)
|
||||
if pr:
|
||||
rec["proxies"].append(pr)
|
||||
else:
|
||||
rec.setdefault("proxy_errors", []).append("%s: %s" % (cc, e3))
|
||||
time.sleep(a.delay)
|
||||
|
||||
append_jsonl(a.bank, rec)
|
||||
made += 1
|
||||
proxies += len(rec["proxies"])
|
||||
print(" #%d OK %-18s proxies=%d [%s]%s"
|
||||
% (idx, rec.get("name"), len(rec["proxies"]),
|
||||
",".join(p["cc"] for p in rec["proxies"]),
|
||||
(" seed=%s…" % (rec.get("seed") or "")[:12]) if a.show_seed else ""))
|
||||
time.sleep(a.delay)
|
||||
|
||||
print(json.dumps({"accounts_ok": made, "proxies_ok": proxies,
|
||||
"failed": failures, "bank": a.bank}))
|
||||
if made:
|
||||
print("\nnext: python urnet.py collect --bank %s --out pool.jsonl" % a.bank)
|
||||
print(" python urnet.py verify --pool pool.jsonl --limit 10")
|
||||
return 0 if made else 1
|
||||
|
||||
|
||||
def cmd_collect(a):
|
||||
rows, seen = [], set()
|
||||
if a.bank and os.path.exists(a.bank):
|
||||
for line in open(a.bank, encoding="utf-8"):
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
rec = json.loads(line)
|
||||
except Exception:
|
||||
continue
|
||||
for p in rec.get("proxies") or []:
|
||||
url = p.get("url")
|
||||
if not url or url in seen:
|
||||
continue
|
||||
seen.add(url)
|
||||
rows.append({"acct": rec.get("idx"), "name": rec.get("name"),
|
||||
"cc": p.get("cc"), "url": url, "socks": p.get("socks"),
|
||||
"client_id": p.get("client_id")})
|
||||
with open(a.out, "w", encoding="utf-8") as fh:
|
||||
for r in rows:
|
||||
fh.write(json.dumps(r, ensure_ascii=False) + "\n")
|
||||
ccs = sorted({(r["cc"] or "?")[:2].upper() for r in rows})
|
||||
print("collected %d unique proxies from %s -> %s (%d countries: %s)"
|
||||
% (len(rows), a.bank, a.out, len(ccs), ",".join(ccs)))
|
||||
return 0 if rows else 1
|
||||
|
||||
|
||||
def cmd_verify(a):
|
||||
if not os.path.exists(a.pool):
|
||||
print("pool not found: %s (run: python urnet.py collect --bank bank.jsonl)" % a.pool)
|
||||
return 1
|
||||
rows = [json.loads(l) for l in open(a.pool, encoding="utf-8") if l.strip()]
|
||||
if a.limit:
|
||||
if a.shuffle:
|
||||
random.shuffle(rows)
|
||||
rows = rows[:a.limit]
|
||||
|
||||
alive, ips, ccs, hosting = 0, set(), set(), []
|
||||
# hosting ASNs that showed up in real pools; plain legal suffixes (GmbH/LLC/Ltd)
|
||||
# are NOT hosting - Vodafone GmbH is a residential ISP.
|
||||
hosting_words = ("hosting", "vpn", "datacenter", "data center", "colocation", "colo",
|
||||
"cloud", "amazon", "digitalocean", "ovh", "hetzner", "linode",
|
||||
"vultr", "contabo", "m247", "choopa", "leaseweb", "3xk",
|
||||
"serverius", "psychz", "hostkey", "aeza")
|
||||
for r in rows:
|
||||
d = _ipinfo(r["url"], a.timeout)
|
||||
d = d or {}
|
||||
r.update(d)
|
||||
ip = d.get("ip")
|
||||
if ip:
|
||||
alive += 1
|
||||
ips.add(ip)
|
||||
ccs.add(d.get("country"))
|
||||
org = (d.get("org") or "")
|
||||
flag = ""
|
||||
if org and any(w in org.lower() for w in hosting_words):
|
||||
flag = " <- hosting ASN"
|
||||
hosting.append(ip)
|
||||
print("%-4s %-3s %-16s %-3s %-14s %-28s%s"
|
||||
% (r.get("acct"), r.get("cc"), ip or "FAIL", d.get("country") or "--",
|
||||
(d.get("city") or "")[:14], org[:28], flag))
|
||||
|
||||
if a.out:
|
||||
json.dump(rows, open(a.out, "w", encoding="utf-8"), ensure_ascii=False, indent=1)
|
||||
print("\nTOTAL %d | alive %d | unique_ip %d | countries %d | hosting_exits %d"
|
||||
% (len(rows), alive, len(ips), len(ccs), len(hosting)))
|
||||
return 0 if alive else 1
|
||||
|
||||
|
||||
def _ipinfo(proxy_url, timeout=35):
|
||||
if HAVE_CURL:
|
||||
try:
|
||||
out = subprocess.run(["curl", "-s", "-m", str(timeout), "--proxy", proxy_url,
|
||||
"https://ipinfo.io/json"],
|
||||
capture_output=True, text=True, timeout=timeout + 25).stdout
|
||||
d = json.loads(out)
|
||||
return {"ip": d.get("ip"), "country": d.get("country"),
|
||||
"city": d.get("city"), "org": d.get("org")}
|
||||
except Exception:
|
||||
return {}
|
||||
handlers = [urllib.request.ProxyHandler({"http": proxy_url, "https": proxy_url}),
|
||||
urllib.request.HTTPSHandler(context=SSLCTX)]
|
||||
opener = urllib.request.build_opener(*handlers)
|
||||
req = urllib.request.Request("https://ipinfo.io/json", headers={"User-Agent": UA})
|
||||
try:
|
||||
with opener.open(req, timeout=timeout) as resp:
|
||||
d = json.loads(resp.read(8000).decode("utf-8", "replace"))
|
||||
return {"ip": d.get("ip"), "country": d.get("country"),
|
||||
"city": d.get("city"), "org": d.get("org")}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def cmd_export(a):
|
||||
rows = [json.loads(l) for l in open(a.pool, encoding="utf-8") if l.strip()]
|
||||
if a.cc:
|
||||
want = {c.strip().lower() for c in a.cc.split(",")}
|
||||
rows = [r for r in rows if (r.get("cc") or "").lower() in want]
|
||||
if a.limit:
|
||||
rows = rows[:a.limit]
|
||||
if a.format == "txt":
|
||||
body = "\n".join(r["url"] for r in rows) + "\n"
|
||||
elif a.format == "csv":
|
||||
body = "cc,url\n" + "\n".join("%s,%s" % (r.get("cc"), r["url"]) for r in rows) + "\n"
|
||||
else: # env
|
||||
body = "HTTP_PROXY=%s\nHTTPS_PROXY=%s\n" % (rows[0]["url"], rows[0]["url"]) if rows else ""
|
||||
with open(a.out, "w", encoding="utf-8") as fh:
|
||||
fh.write(body)
|
||||
print("exported %d proxies -> %s (%s)" % (len(rows), a.out, a.format))
|
||||
return 0 if rows else 1
|
||||
|
||||
|
||||
def cmd_balance(a):
|
||||
rows = [json.loads(l) for l in open(a.bank, encoding="utf-8") if l.strip()]
|
||||
total = 0
|
||||
live = 0
|
||||
for rec in rows:
|
||||
key = rec.get("urn")
|
||||
if not key:
|
||||
continue
|
||||
b, err = get_balance(key)
|
||||
if err:
|
||||
print("%-4s %-18s %s" % (rec.get("idx"), rec.get("name"), err))
|
||||
continue
|
||||
live += 1
|
||||
left = b.get("left") or 0
|
||||
total += left
|
||||
print("%-4s %-18s left %-10s of %-10s open %s | proxies %d"
|
||||
% (rec.get("idx"), rec.get("name"), human(left), human(b.get("start")),
|
||||
human(b.get("open_transfer")), len(rec.get("proxies") or [])))
|
||||
print("\naccounts %d | live keys %d | traffic left today %s"
|
||||
% (len(rows), live, human(total)))
|
||||
return 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- cli
|
||||
def build_parser():
|
||||
p = argparse.ArgumentParser(
|
||||
prog="urnet",
|
||||
description="URnetwork free-tier farm: instant accounts, API keys, hosted proxies.",
|
||||
epilog="quick start: python urnet.py farm --count 1 && python urnet.py collect "
|
||||
"--bank bank.jsonl --out pool.jsonl && python urnet.py verify --pool pool.jsonl")
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
f = sub.add_parser("farm", help="create accounts, keys and proxies")
|
||||
f.add_argument("--count", type=int, default=1, help="accounts to create (default 1)")
|
||||
f.add_argument("--proxies", type=int, default=5, help="proxies per account, max 5 (default 5)")
|
||||
f.add_argument("--countries", default="", help="comma list of ISO codes, e.g. us,de,nl")
|
||||
f.add_argument("--bank", default="bank.jsonl", help="output jsonl (default bank.jsonl)")
|
||||
f.add_argument("--start", type=int, default=1, help="first account index label")
|
||||
f.add_argument("--proxy", default="", help="single egress proxy URL for every request")
|
||||
f.add_argument("--proxy-file", default="", help="file with one proxy URL per line (rotates)")
|
||||
f.add_argument("--shuffle", action="store_true", help="shuffle the proxy file")
|
||||
f.add_argument("--delay", type=float, default=1.3, help="seconds between proxy calls")
|
||||
f.add_argument("--curl", action="store_true", help="use curl instead of urllib for proxied calls")
|
||||
f.add_argument("--show-seed", action="store_true", help="print seed prefix in the log")
|
||||
f.set_defaults(func=cmd_farm)
|
||||
|
||||
c = sub.add_parser("collect", help="flatten bank.jsonl into a proxy pool")
|
||||
c.add_argument("--bank", default="bank.jsonl")
|
||||
c.add_argument("--out", default="pool.jsonl")
|
||||
c.set_defaults(func=cmd_collect)
|
||||
|
||||
v = sub.add_parser("verify", help="liveness pass: ipinfo through each proxy")
|
||||
v.add_argument("--pool", default="pool.jsonl")
|
||||
v.add_argument("--limit", type=int, default=0, help="check only N proxies (0 = all)")
|
||||
v.add_argument("--shuffle", action="store_true")
|
||||
v.add_argument("--timeout", type=int, default=35)
|
||||
v.add_argument("--out", default="", help="write enriched pool to this file")
|
||||
v.set_defaults(func=cmd_verify)
|
||||
|
||||
e = sub.add_parser("export", help="write pool as txt / csv / env")
|
||||
e.add_argument("--pool", default="pool.jsonl")
|
||||
e.add_argument("--format", choices=["txt", "csv", "env"], default="txt")
|
||||
e.add_argument("--cc", default="", help="only these countries, e.g. us,de")
|
||||
e.add_argument("--limit", type=int, default=0)
|
||||
e.add_argument("--out", default="proxies.txt")
|
||||
e.set_defaults(func=cmd_export)
|
||||
|
||||
b = sub.add_parser("balance", help="traffic left per account")
|
||||
b.add_argument("--bank", default="bank.jsonl")
|
||||
b.set_defaults(func=cmd_balance)
|
||||
return p
|
||||
|
||||
|
||||
def main():
|
||||
setup_stdout()
|
||||
args = build_parser().parse_args()
|
||||
try:
|
||||
return args.func(args)
|
||||
except KeyboardInterrupt:
|
||||
print("\ninterrupted - partial progress is already in the bank file")
|
||||
return 130
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,49 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Liveness pass over a proxy pool: curl ipinfo through each URL.
|
||||
|
||||
Usage: python verify.py pool.jsonl [--limit N] [--timeout SEC]
|
||||
Prints one line per proxy and a summary: alive / unique_ip / countries.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
try:
|
||||
sys.stdout.reconfigure(encoding="utf-8")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("pool")
|
||||
ap.add_argument("--limit", type=int, default=0, help="0 = all")
|
||||
ap.add_argument("--timeout", type=int, default=35)
|
||||
ap.add_argument("--out", default="verify.json")
|
||||
a = ap.parse_args()
|
||||
|
||||
rows = [json.loads(l) for l in open(a.pool, encoding="utf-8") if l.strip()]
|
||||
if a.limit:
|
||||
rows = rows[:a.limit]
|
||||
|
||||
alive, ips, ccs = 0, set(), set()
|
||||
for r in rows:
|
||||
out = subprocess.run(["curl", "-s", "-m", str(a.timeout), "--proxy", r["url"],
|
||||
"https://ipinfo.io/json"],
|
||||
capture_output=True, text=True, timeout=a.timeout + 25).stdout
|
||||
try:
|
||||
d = json.loads(out)
|
||||
except Exception:
|
||||
d = {}
|
||||
r.update({"ip": d.get("ip"), "country": d.get("country"),
|
||||
"city": d.get("city"), "org": d.get("org")})
|
||||
if d.get("ip"):
|
||||
alive += 1
|
||||
ips.add(d["ip"])
|
||||
ccs.add(d.get("country"))
|
||||
print("%-4s %-3s %-16s %-3s %-14s %s" % (r.get("acct"), r.get("cc"), d.get("ip", "FAIL"),
|
||||
d.get("country", "--"), (d.get("city") or "")[:14], (d.get("org") or "")[:26]))
|
||||
|
||||
json.dump(rows, open(a.out, "w", encoding="utf-8"), ensure_ascii=False, indent=1)
|
||||
print("\nTOTAL %d | alive %d | unique_ip %d | countries %d %s"
|
||||
% (len(rows), alive, len(ips), len(ccs), sorted(c for c in ccs if c)))
|
||||
Reference in New Issue
Block a user