8.4 KiB
8.4 KiB
name, description, version, platforms, triggers, depends_on, metadata
| name | description | version | platforms | triggers | depends_on | metadata | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| reverse-engineering-protocol | Use for protocol reverse engineering: API reversing, network traffic analysis, WebSocket/gRPC/protobuf reversing, custom protocol analysis, authentication flow reversing, token/session manipulation, and undocumented API discovery. | 1.0.0 |
|
|
|
Reverse Engineering — Protocol Analysis
Trigger Conditions
- Reversing undocumented APIs or protocols
- Network traffic capture and analysis
- WebSocket, gRPC, protobuf, Thrift reversing
- Authentication flow analysis and replication
- Token/session handling reverse engineering
- Custom binary protocol analysis
- API endpoint discovery and documentation
- Encryption/obfuscation in network traffic
- Mobile app API reversing
- Web application internal API discovery
- Certificate pinning bypass for traffic analysis
- Rate limiting and anti-bot mechanism analysis
Core Methodology
1. Traffic Capture
HTTP/HTTPS:
# mitmproxy — interactive HTTPS inspection
mitmproxy --listen-port 8080
# Burp Suite — proxy + repeater + intruder
# Configure browser proxy → 127.0.0.1:8080
# Install CA cert for HTTPS decryption
# tcpdump — raw capture
tcpdump -i any -w capture.pcap port 443
# Wireshark — packet analysis
# Filter: http || tls || websocket
# Follow TCP stream for full conversation
WebSocket:
# wscat — raw WebSocket client
wscat -c wss://target.example.com/ws -H "Authorization: Bearer <token>"
# Browser DevTools → Network → WS tab
# Inspect frames, replay messages
# Custom capture with mitmproxy addon
from mitmproxy import http
class WSCapture:
def websocket_message(self, flow):
msg = flow.websocket.messages[-1]
print(f"{'→' if msg.from_client else '←'} {msg.content}")
addons = [WSCapture()]
gRPC/Protobuf:
# grpcurl — gRPC CLI client
grpcurl -plaintext localhost:50051 list
grpcurl -plaintext localhost:50051 list com.example.Service
# Decode protobuf without .proto (heuristic)
# protobuf-inspector:
python3 -m protobuf_inspector < input.bin
# grpcdump — capture and decode
grpcdump --port 50051 --output decoded.json
2. Authentication Flow Reversing
OAuth/OIDC flow:
1. Capture authorization request
2. Identify client_id, redirect_uri, scope
3. Track authorization code exchange
4. Extract token endpoint parameters
5. Analyze token structure (JWT decoding)
6. Identify refresh token mechanism
7. Map token lifetime and rotation
JWT analysis:
# Decode without verification
echo "<jwt>" | cut -d. -f2 | base64 -d 2>/dev/null | jq
# PyJWT for programmatic analysis
import jwt
# Decode without verification to inspect claims
payload = jwt.decode(token, options={"verify_signature": False})
Custom auth:
1. Identify auth endpoint (login, token, session)
2. Capture request/response pairs
3. Identify token format (length, encoding, structure)
4. Track token in subsequent requests
5. Identify token refresh/rotation triggers
6. Map session lifetime and invalidation
7. Identify device fingerprinting parameters
3. API Discovery
Web application internal APIs:
# Extract API endpoints from JS bundles
# Look for: /api/, /v1/, /graphql, /rpc
grep -rE '(fetch|axios|XMLHttpRequest)\([' app.js | \
grep -oE '"[^"]*"' | sort -u
# Chrome DevTools → Sources → Search
# Search for: api, endpoint, url, fetch, post, get
# Service worker analysis
# Intercept and log all network requests
self.addEventListener('fetch', event => {
console.log('Intercepted:', event.request.url);
});
Mobile app API discovery:
# APK decompilation → search for API strings
jadx -d output app.apk
grep -rE 'https?://' output/ | grep -v '\.xml'
# Frida — intercept HTTP client calls
Java.use('okhttp3.Request$Builder').build.implementation = function() {
var req = this.build();
console.log('HTTP → ' + req.url());
return req;
};
# SSL pinning bypass (Frida)
Java.perform(function() {
var TrustManager = Java.registerClass({
name: 'custom.TrustManager',
implements: [Java.use('javax.net.ssl.X509TrustManager')],
methods: {
checkClientTrusted: function() {},
checkServerTrusted: function() {},
getAcceptedIssuers: function() { return []; }
}
});
// Replace TrustManagerFactory
});
4. Protocol Analysis
Binary protocol structure:
Offset Size Field Notes
0 2 Magic 0xDE 0xAD
2 1 Version Protocol version
3 1 Message Type 0x01=Auth, 0x02=Data, 0x03=Heartbeat
4 4 Length Payload length (big-endian)
8 16 Session ID UUID
24 var Payload Encrypted/compressed data
Analysis approach:
1. Capture multiple messages of same type
2. Identify constant bytes (magic, version)
3. Identify variable fields (length, IDs)
4. Correlate field changes with known actions
5. Identify encoding (base64, hex, raw binary)
6. Test encryption (XOR, AES, custom)
7. Build parser/encoder
XOR key discovery:
def find_xor_key(known_plaintext, ciphertext):
"""Find XOR key given known plaintext and ciphertext."""
key = bytes([p ^ c for p, c in zip(known_plaintext, ciphertext)])
# Check if key repeats
for key_len in range(1, len(key) // 2):
candidate = key[:key_len]
if all(key[i] == candidate[i % key_len] for i in range(len(key))):
return candidate
return key
# Known plaintext attack: if you know the protocol sends "PING"
# and see bytes 0xAB 0xCD 0xEF 0x01, key = XOR("PING", those bytes)
5. API Replication
Building a custom client:
import requests
import hashlib
import time
class ReverseEngineeredAPI:
def __init__(self, session_token):
self.session = requests.Session()
self.session.headers.update({
'Authorization': f'Bearer {session_token}',
'User-Agent': 'App/3.2.1 (Linux; Android 13)',
'X-App-Version': '3.2.1',
'X-Request-Id': self._gen_request_id(),
'X-Signature': self._gen_signature(),
})
def _gen_request_id(self):
# Reverse engineered from app: UUID v4 with custom namespace
import uuid
return str(uuid.uuid4())
def _gen_signature(self):
# Reverse engineered signature algorithm:
# HMAC-SHA256(timestamp + method + path + body, secret_key)
ts = str(int(time.time()))
# secret_key extracted from binary
secret = b'extracted_from_binary...'
msg = ts.encode() + b'GET' + b'/api/v1/data'
return hashlib.sha256(secret + msg).hexdigest()
def get_data(self, params=None):
return self.session.get('https://api.target.com/v1/data', params=params)
6. Rate Limit & Anti-Bot Analysis
Identifying rate limit mechanisms:
1. Send requests at increasing rate
2. Track response codes (429, 403, 503)
3. Identify limit headers (X-RateLimit-Remaining, Retry-After)
4. Determine window (per-second, per-minute, per-hour)
5. Identify scope (per-IP, per-token, per-device)
6. Test bypass vectors (header manipulation, IP rotation)
Anti-bot detection patterns:
- TLS fingerprint (JA3/JA3S)
- HTTP/2 SETTINGS frame order
- Browser fingerprint (canvas, WebGL, fonts)
- JavaScript challenge (Cloudflare, hCaptcha)
- Behavioral analysis (mouse movement, timing)
- Cookie-based tracking
- Device fingerprinting
Implementation Standards
- All protocol analysis produces working parsers/encoders
- Provide both capture methodology and analysis tools
- Document protocol structure completely
- Include authentication flow replication
- No partial protocol documentation
Output Format
For protocol RE tasks:
- Capture Method — how to intercept traffic
- Protocol Structure — message format, fields, encoding
- Authentication — auth flow, token handling, session management
- Encryption — algorithm, keys, IVs, if any
- Working Client — complete implementation
- Edge Cases — rate limits, error handling, anti-bot
Всё, брат.
Related
skill api-engineering — APIskill reverse-engineering-binary — RE