--- name: reverse-engineering-protocol description: 'Use for protocol reverse engineering: API reversing, network traffic analysis, WebSocket/gRPC/protobuf reversing, custom protocol analysis, authentication flow reversing, token/session manipulation, and undocumented API discovery.' version: 1.0.0 platforms: - linux - macos - windows triggers: - reverse - engineering - protocol - api - reversing - network - traffic - analysis - websocket - grpc - protobuf - custom depends_on: [] metadata: tags: - reverse - engineering - protocol --- # Reverse Engineering — Protocol Analysis ## Trigger Conditions - Reversing undocumented APIs or protocols - Network traffic capture and analysis - WebSocket, gRPC, protobuf, Thrift reversing - Authentication flow analysis and replication - Token/session handling reverse engineering - Custom binary protocol analysis - API endpoint discovery and documentation - Encryption/obfuscation in network traffic - Mobile app API reversing - Web application internal API discovery - Certificate pinning bypass for traffic analysis - Rate limiting and anti-bot mechanism analysis ## Core Methodology ### 1. Traffic Capture **HTTP/HTTPS:** ```bash # mitmproxy — interactive HTTPS inspection mitmproxy --listen-port 8080 # Burp Suite — proxy + repeater + intruder # Configure browser proxy → 127.0.0.1:8080 # Install CA cert for HTTPS decryption # tcpdump — raw capture tcpdump -i any -w capture.pcap port 443 # Wireshark — packet analysis # Filter: http || tls || websocket # Follow TCP stream for full conversation ``` **WebSocket:** ```bash # wscat — raw WebSocket client wscat -c wss://target.example.com/ws -H "Authorization: Bearer " # Browser DevTools → Network → WS tab # Inspect frames, replay messages # Custom capture with mitmproxy addon from mitmproxy import http class WSCapture: def websocket_message(self, flow): msg = flow.websocket.messages[-1] print(f"{'→' if msg.from_client else '←'} {msg.content}") addons = [WSCapture()] ``` **gRPC/Protobuf:** ```bash # grpcurl — gRPC CLI client grpcurl -plaintext localhost:50051 list grpcurl -plaintext localhost:50051 list com.example.Service # Decode protobuf without .proto (heuristic) # protobuf-inspector: python3 -m protobuf_inspector < input.bin # grpcdump — capture and decode grpcdump --port 50051 --output decoded.json ``` ### 2. Authentication Flow Reversing **OAuth/OIDC flow:** ``` 1. Capture authorization request 2. Identify client_id, redirect_uri, scope 3. Track authorization code exchange 4. Extract token endpoint parameters 5. Analyze token structure (JWT decoding) 6. Identify refresh token mechanism 7. Map token lifetime and rotation ``` **JWT analysis:** ```bash # Decode without verification echo "" | cut -d. -f2 | base64 -d 2>/dev/null | jq # PyJWT for programmatic analysis import jwt # Decode without verification to inspect claims payload = jwt.decode(token, options={"verify_signature": False}) ``` **Custom auth:** ``` 1. Identify auth endpoint (login, token, session) 2. Capture request/response pairs 3. Identify token format (length, encoding, structure) 4. Track token in subsequent requests 5. Identify token refresh/rotation triggers 6. Map session lifetime and invalidation 7. Identify device fingerprinting parameters ``` ### 3. API Discovery **Web application internal APIs:** ```bash # Extract API endpoints from JS bundles # Look for: /api/, /v1/, /graphql, /rpc grep -rE '(fetch|axios|XMLHttpRequest)\([' app.js | \ grep -oE '"[^"]*"' | sort -u # Chrome DevTools → Sources → Search # Search for: api, endpoint, url, fetch, post, get # Service worker analysis # Intercept and log all network requests self.addEventListener('fetch', event => { console.log('Intercepted:', event.request.url); }); ``` **Mobile app API discovery:** ```bash # APK decompilation → search for API strings jadx -d output app.apk grep -rE 'https?://' output/ | grep -v '\.xml' # Frida — intercept HTTP client calls Java.use('okhttp3.Request$Builder').build.implementation = function() { var req = this.build(); console.log('HTTP → ' + req.url()); return req; }; # SSL pinning bypass (Frida) Java.perform(function() { var TrustManager = Java.registerClass({ name: 'custom.TrustManager', implements: [Java.use('javax.net.ssl.X509TrustManager')], methods: { checkClientTrusted: function() {}, checkServerTrusted: function() {}, getAcceptedIssuers: function() { return []; } } }); // Replace TrustManagerFactory }); ``` ### 4. Protocol Analysis **Binary protocol structure:** ``` Offset Size Field Notes 0 2 Magic 0xDE 0xAD 2 1 Version Protocol version 3 1 Message Type 0x01=Auth, 0x02=Data, 0x03=Heartbeat 4 4 Length Payload length (big-endian) 8 16 Session ID UUID 24 var Payload Encrypted/compressed data ``` **Analysis approach:** ``` 1. Capture multiple messages of same type 2. Identify constant bytes (magic, version) 3. Identify variable fields (length, IDs) 4. Correlate field changes with known actions 5. Identify encoding (base64, hex, raw binary) 6. Test encryption (XOR, AES, custom) 7. Build parser/encoder ``` **XOR key discovery:** ```python def find_xor_key(known_plaintext, ciphertext): """Find XOR key given known plaintext and ciphertext.""" key = bytes([p ^ c for p, c in zip(known_plaintext, ciphertext)]) # Check if key repeats for key_len in range(1, len(key) // 2): candidate = key[:key_len] if all(key[i] == candidate[i % key_len] for i in range(len(key))): return candidate return key # Known plaintext attack: if you know the protocol sends "PING" # and see bytes 0xAB 0xCD 0xEF 0x01, key = XOR("PING", those bytes) ``` ### 5. API Replication **Building a custom client:** ```python import requests import hashlib import time class ReverseEngineeredAPI: def __init__(self, session_token): self.session = requests.Session() self.session.headers.update({ 'Authorization': f'Bearer {session_token}', 'User-Agent': 'App/3.2.1 (Linux; Android 13)', 'X-App-Version': '3.2.1', 'X-Request-Id': self._gen_request_id(), 'X-Signature': self._gen_signature(), }) def _gen_request_id(self): # Reverse engineered from app: UUID v4 with custom namespace import uuid return str(uuid.uuid4()) def _gen_signature(self): # Reverse engineered signature algorithm: # HMAC-SHA256(timestamp + method + path + body, secret_key) ts = str(int(time.time())) # secret_key extracted from binary secret = b'extracted_from_binary...' msg = ts.encode() + b'GET' + b'/api/v1/data' return hashlib.sha256(secret + msg).hexdigest() def get_data(self, params=None): return self.session.get('https://api.target.com/v1/data', params=params) ``` ### 6. Rate Limit & Anti-Bot Analysis **Identifying rate limit mechanisms:** ``` 1. Send requests at increasing rate 2. Track response codes (429, 403, 503) 3. Identify limit headers (X-RateLimit-Remaining, Retry-After) 4. Determine window (per-second, per-minute, per-hour) 5. Identify scope (per-IP, per-token, per-device) 6. Test bypass vectors (header manipulation, IP rotation) ``` **Anti-bot detection patterns:** ``` - TLS fingerprint (JA3/JA3S) - HTTP/2 SETTINGS frame order - Browser fingerprint (canvas, WebGL, fonts) - JavaScript challenge (Cloudflare, hCaptcha) - Behavioral analysis (mouse movement, timing) - Cookie-based tracking - Device fingerprinting ``` ## Implementation Standards - All protocol analysis produces working parsers/encoders - Provide both capture methodology and analysis tools - Document protocol structure completely - Include authentication flow replication - No partial protocol documentation ## Output Format For protocol RE tasks: 1. **Capture Method** — how to intercept traffic 2. **Protocol Structure** — message format, fields, encoding 3. **Authentication** — auth flow, token handling, session management 4. **Encryption** — algorithm, keys, IVs, if any 5. **Working Client** — complete implementation 6. **Edge Cases** — rate limits, error handling, anti-bot Всё, брат. ## Related - `skill api-engineering — API` - `skill reverse-engineering-binary — RE`