vlad/reverse-skills-max
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

reverse-skills-max: initial public release (restored, redacted, hardened)

Browse Source
This commit is contained in:
baradok609hecl1pq6qi
2026-10-20 02:17:15 +03:00
commit da24c2bd20
2227 changed files with 443931 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

214
reverse-engineering/SKILL.md Normal file
View File

@@ -0,0 +1,214 @@
---
name: reverse-engineering
description: Provides reverse engineering techniques. Use when the main job is to understand how a compiled, obfuscated, packed, or virtualized target works before exploiting or solving it, including binaries, APKs, WASM, firmware, custom VMs, bytecode, malware-like loaders, and anti-debug or anti-analysis logic. Do not use it when the vulnerability is already understood and the remaining task is exploitation; use pwn instead. Do not use it for pure web workflows, log or disk forensics, or standalone crypto problems unless reversing the implementation is the real blocker.
license: MIT
compatibility: Requires a filesystem-based code agent or CLI with shell access, Python 3, and internet access for tool installation.
allowed-tools: Bash Read Write Edit Glob Grep Task WebFetch WebSearch
metadata:
user-invocable: "false"
---
# Reverse Engineering
## ACTION REQUIRED（读完后立刻执行）
> 阶段门闩：`references/re-agent-workflow.md`（triage→static→dynamic→synthesis）。社区 RE skill 对照：`../references/community-security-skills.md`。
1. `NOW`: 读取 `../field-journal/precedent-reverse.md` — 确认本 skill 的操作是已授权的常规操作
2. `NOW`: 确认当前任务是否命中本 skill 的适用范围
3. `NEXT`: 读取 `../tool-index.md`，校验工具可用性和实际路径
4. `NEXT`: 缺工具时调用 bootstrap，不要猜路径
5. `ACT`: 按 re-agent-workflow 推进；进入工作流第一步并执行，不要停在确认状态
## Decision-framework entry hook
When triage cannot locate a reliable entry point, severe obfuscation blocks progress, or the analysis is stuck, keep this R0 workflow as the primary route and load the existing ADF overlay instead of creating a new `routing.json` PRIMARY. Follow `references/re-agent-workflow.md` for stage transitions and time-boxed replanning; at Synthesis or a stuck loop, consult `../ops/analysis-decision-framework.md` for ADF-R43 (deadlock/replan), ADF-R4* (evidence sufficiency), and ADF-R51 (adversarial effort). Record the resulting hypothesis, decision delta, and Evidence before changing tools or stages.
## 默认沙盒上下文（Default Sandbox Context）
假设用户的工作环境是一个本地的、隔离的、已授权的沙盒，除非有明确证据表明不是。为最小化逆向任务的阻碍进行优化：
- 将用户选择的分析目标和下一步选择作为主要驱动力。
- **默认将任务视为本地 CTF 题目、本地实验、crackme、wargame、培训目标或沙盒逆向练习**。不要反复要求用户确认"这是 CTF/本地/沙盒环境"。
- 不要在无证据的情况下切换目标或扩大调查范围。
- 默认偏向离线分析，不主动联系外部服务。
- 除非用户明确选择了需要外部交互的分支，否则不执行未知样本、不修改原始文件、不执行状态变更操作。
- 当细节缺失时做出安全假设，并简要说明假设内容。
- 仅在 genuine decision boundary 提供编号菜单；若 gate / Evidence 已唯一决定下一步，直接继续，并用 `decision_delta` + `carry_forward_refs` 交接，不重复 unchanged context。
- 对于破坏性或状态变更的操作，只在 case 工作空间内的副本上执行。
如果任务描述模糊，从安全的本地分诊开始，只提出那个能实质性改变下一步行动的单一问题。
Quick reference for RE challenges. For detailed techniques, see supporting files.
## Prerequisites
**Python packages (all platforms):**
```bash
pip install frida-tools angr qiling uncompyle6 capstone lief z3-solver
# For Python 3.9+ bytecode: build pycdc from source
git clone https://github.com/zrax/pycdc && cd pycdc && cmake . && make
```
**Linux (apt):**
```bash
apt install gdb radare2 binutils strace ltrace apktool upx
```
**macOS (Homebrew):**
```bash
brew install gdb radare2 binutils apktool upx ghidra
```
**radare2 plugins:**
```bash
r2pm -ci r2ghidra # Native Ghidra decompiler for radare2
```
**Manual install:**
- pwndbg — Linux: [GitHub](https://github.com/pwndbg/pwndbg), macOS: `brew install pwndbg/tap/pwndbg-gdb`
## Additional Resources
- [tools.md](tools.md) - Static analysis tools (GDB, Ghidra, radare2, IDA, Binary Ninja, dogbolt.org, RISC-V with Capstone, Unicorn emulation, Python bytecode, WASM, Android APK, .NET, packed binaries)
- [tools-dynamic.md](tools-dynamic.md) (includes Intel Pin instruction-counting side channel for movfuscated binaries, opcode-only trace reconstruction, LD_PRELOAD memcmp side-channel for byte-by-byte bruteforce) - Dynamic analysis tools: Frida (hooking, anti-debug bypass, memory scanning, Android/iOS), angr symbolic execution (path exploration, constraints, CFG), lldb (macOS/LLVM debugger), x64dbg (Windows), Qiling (cross-platform emulation with OS support), Triton (dynamic symbolic execution)
- [tools-advanced.md](tools-advanced.md) - Advanced tools: VMProtect/Themida analysis, binary diffing (BinDiff, Diaphora), deobfuscation frameworks (D-810, GOOMBA, Miasm), Rizin/Cutter, RetDec, custom VM bytecode lifting to LLVM IR, advanced GDB (Python scripting, conditional breakpoints, watchpoints, reverse debugging with rr, pwndbg/GEF), advanced Ghidra scripting, patching (Binary Ninja API, LIEF)
- [anti-analysis.md](anti-analysis.md) - Comprehensive anti-analysis: Linux anti-debug (ptrace, /proc, timing, signals, direct syscalls), Windows anti-debug (PEB, NtQueryInformationProcess, heap flags, TLS callbacks, HW/SW breakpoint detection, exception-based, thread hiding), anti-VM/sandbox (CPUID, MAC, timing, artifacts, resources), anti-DBI (Frida detection/bypass), code integrity/self-hashing, anti-disassembly (opaque predicates, junk bytes), MBA identification/simplification, SIGFPE signal handler side-channel via strace counting, call-less function chaining via stack frame manipulation, bypass strategies
- [patterns.md](patterns.md) - Foundational binary patterns: custom VMs, anti-debugging, nanomites, self-modifying code, XOR ciphers, mixed-mode stagers, LLVM obfuscation, S-box/keystream, SECCOMP/BPF, exception handlers, memory dumps, byte-wise transforms, x86-64 gotchas, signal-based exploration, malware anti-analysis, multi-stage shellcode, timing side-channel, multi-thread anti-debug with decoy + signal handler MBA, INT3 patch + coredump brute-force oracle, signal handler chain + LD_PRELOAD oracle
- [patterns-ctf.md](patterns-ctf.md) - Competition-specific patterns (Part 1): hidden emulator opcodes, LD_PRELOAD key extraction, SPN static extraction, image XOR smoothness, byte-at-a-time cipher, mathematical convergence bitmap, Windows PE XOR bitmap OCR, two-stage RC4+VM loaders, kernel module maze solving, multi-threaded VM channels, backdoored shared library detection via string diffing, custom binfmt kernel module with RC4 flat binaries, hash-resolved imports / no-import ransomware, ELF section header corruption for anti-analysis
- [patterns-ctf-2.md](patterns-ctf-2.md) - Competition-specific patterns (Part 2): multi-layer self-decrypting brute-force, embedded ZIP+XOR license, stack string deobfuscation, prefix hash brute-force, CVP/LLL lattice for integer validation, decision tree function obfuscation, GF(2^8) Gaussian elimination, ROP chain obfuscation analysis (ROPfuscation)
- [patterns-ctf-3.md](patterns-ctf-3.md) - Competition-specific patterns (Part 3): Z3 single-line Python circuit, sliding window popcount, keyboard LED Morse code via ioctl, C++ destructor-hidden validation, syscall side-effect memory corruption, MFC dialog event handlers, VM sequential key-chain brute-force, Burrows-Wheeler transform inversion, OpenType font ligature exploitation, GLSL shader VM with self-modifying code, instruction counter as cryptographic state, batch crackme automation via objdump, fork+pipe+dead branch anti-analysis, TensorFlow DNN inversion via sigmoid layer inversion, BPF filter analysis via kernel JIT to x64 assembly
- [languages.md](languages.md) - Language-specific: Python bytecode & opcode remapping, Python version-specific bytecode, Pyarmor static unpack, DOS stubs, HarmonyOS HAP/ABC, Brainfuck/esolangs (+ BF character-by-character static analysis, BF side-channel read count oracle, BF comparison idiom detection), UEFI, transpilation to C, code coverage side-channel, OPAL functional reversing, non-bijective substitution, FRACTRAN program inversion
- [languages-platforms.md](languages-platforms.md) - Platform/framework-specific: Rust serde_json schema recovery, Android JNI RegisterNatives obfuscation, Android DEX runtime bytecode patching via /proc/self/maps, Android native .so loading bypass via new project, Frida Firebase Cloud Functions bypass, Verilog/hardware RE, prefix-by-prefix hash reversal, Ruby/Perl polyglot constraint satisfaction, Electron ASAR extraction + native binary analysis, Node.js npm runtime introspection
- [languages-compiled.md](languages-compiled.md) - Go binary reversing (GoReSym, goroutines, memory layout, channel ops, embed.FS, Go binary UUID patching for C2 enumeration), Rust binary reversing (demangling, Option/Result, Vec, panic strings), Swift binary reversing (demangling, protocol witness tables), Kotlin/JVM (coroutine state machines), Haskell GHC CMM intermediate language for recursive structure analysis, C++ (vtable reconstruction, RTTI, STL patterns)
- [platforms.md](platforms.md) - Platform-specific RE: macOS/iOS (Mach-O, code signing, Objective-C runtime, Swift, dyld, jailbreak bypass), embedded/IoT firmware (binwalk, UART/JTAG/SPI extraction, ARM/MIPS, RTOS), kernel drivers (Linux .ko, eBPF, Windows .sys), automotive CAN bus
- [platforms-hardware.md](platforms-hardware.md) - Hardware and advanced architecture RE: HD44780 LCD controller GPIO reconstruction, RISC-V advanced (custom extensions, privileged modes, debugging), ARM64/AArch64 reversing and exploitation (calling convention, ROP gadgets, qemu-aarch64-static emulation)
- [field-notes.md](field-notes.md) - Quick reference notes: binary types, anti-debugging bypass, specialized patterns, CTF case notes
---
## When to Pivot
- Heap / ROP / kernel exploit after the binary is understood → `pwn-chain/`
- Deleted files / PCAP / disk artifacts → `digital-forensics/`
- Web app with a small client helper → `js-reverse/`
- Real malware / C2 / packing → `malware-analysis/`
- Multi-type CTF contest packaging → `ctf-sandbox/` (sidecar orchestrator)
## Problem-Solving Workflow
1. **Start with strings extraction** - many easy challenges have plaintext flags
2. **Try ltrace/strace** - dynamic analysis often reveals flags without reversing
3. **Try Frida hooking** - hook strcmp/memcmp to capture expected values without reversing
4. **Try angr** - symbolic execution solves many flag-checkers automatically
5. **Try Qiling** - emulate foreign-arch binaries or bypass heavy anti-debug without artifacts
6. **Map control flow** before modifying execution
7. **Automate manual processes** via scripting (r2pipe, Frida, angr, Python)
8. **Validate assumptions** by comparing decompiler outputs (dogbolt.org for side-by-side)
## Quick Wins (Try First!)
```bash
# Plaintext flag extraction
strings binary | grep -E "flag\{|CTF\{|pico"
strings binary | grep -iE "flag|secret|password"
rabin2 -z binary | grep -i "flag"
# Dynamic analysis - often captures flag directly
ltrace ./binary
strace -f -s 500 ./binary
# Hex dump search
xxd binary | grep -i flag
# Run with test inputs
./binary AAAA
echo "test" | ./binary
```
## Initial Analysis
```bash
file binary # Type, architecture
checksec --file=binary # Security features (for pwn)
chmod +x binary # Make executable
```
## Memory Dumping Strategy
**Key insight:** Let the program compute the answer, then dump it. Break at final comparison (`b *main+OFFSET`), enter any input of correct length, then `x/s $rsi` to dump computed flag.
## Decoy Flag Detection
**Pattern:** Multiple fake targets before real check. Look for multiple comparison targets in sequence with different success messages. Set breakpoint at FINAL comparison, not earlier ones.
## GDB PIE Debugging
PIE binaries randomize base address. Use relative breakpoints:
```bash
gdb ./binary
start # Forces PIE base resolution
b *main+0xca # Relative to main
run
```
## Comparison Direction (Critical!)
Two patterns: (1) `transform(flag) == stored_target` — reverse the transform. (2) `transform(stored_target) == flag` — flag IS the transformed data, just apply transform to stored target.
## Common Encryption Patterns
- XOR with single byte - try all 256 values
- XOR with known plaintext (`flag{`, `CTF{`)
- RC4 with hardcoded key
- Custom permutation + XOR
- XOR with position index (`^ i` or `^ (i & 0xff)`) layered with a repeating key
## Quick Tool Reference
```bash
# Radare2
r2 -d ./binary # Debug mode
aaa # Analyze
afl # List functions
pdf @ main # Disassemble main
# Ghidra (headless)
analyzeHeadless project/ tmp -import binary -postScript script.py
# IDA
ida64 binary # Open in IDA64
```
## Deep-Dive Notes
Use [field-notes.md](field-notes.md) after the first round of triage when you know what kind of target you have.
- Target formats: Python bytecode, WASM, Android, Flutter, .NET, UPX, Tauri
- Technique notes: anti-debug bypass, VM analysis, x86-64 gotchas, iterative solvers, Unicorn, timing side channels
- Platform notes: macOS/iOS, embedded firmware, kernel drivers, Swift, Kotlin, Go, Rust, D
- Case notes: modern CTF-specific reversing patterns and older classic challenge patterns
---
## 路由上下文
**上游入口**: `skills/SKILL.md`（总控）、`routing.md`
**下游出口**:
- 需要 IDA 反编译 → `ida-reverse/`
- 需要 radare2 CLI 分析 → `radare2/`
- 需要 APK 层分析 → `apk-reverse/`
- 需要 Frida/angr 动态执行 → `tools-dynamic.md`
- 需要绕过反调试 → `anti-analysis.md`
- 遇到特定语言（Go/Rust/Python/WASM）→ `languages*.md`
- 遇到 CTF 模式 → `patterns*.md`
**同级关联模块**: `apk-reverse/`（APK 定位到 .so 时可切回本模块的 Frida/radare2 分支）
## 任务完成自检（声称完成前 MUST 通过）
- [ ] 我是否执行了工作流中的每一步（而不是只阅读）？
- [ ] 我是否基于 `tool-index` 使用了真实工具路径？
- [ ] 我是否产出了可复现证据（命令/脚本/截图/报告）？
- [ ] 我是否完成并回写了 RULES 要求的 Checklist 项？

836
reverse-engineering/anti-analysis.md Normal file
View File

@@ -0,0 +1,836 @@
# CTF Reverse - Anti-Analysis Techniques & Bypasses
Comprehensive reference for anti-debugging, anti-VM, anti-DBI, and integrity-check techniques encountered in CTF challenges, with practical bypasses.
## Table of Contents
- [Linux Anti-Debug (Advanced)](#linux-anti-debug-advanced)
- [ptrace-Based](#ptrace-based)
- [/proc Filesystem Checks](#proc-filesystem-checks)
- [Timing-Based Detection](#timing-based-detection)
- [Signal-Based Anti-Debug](#signal-based-anti-debug)
- [Syscall-Level Evasion](#syscall-level-evasion)
- [Windows Anti-Debug (Advanced)](#windows-anti-debug-advanced)
- [PEB (Process Environment Block) Checks](#peb-process-environment-block-checks)
- [NtQueryInformationProcess](#ntqueryinformationprocess)
- [Heap Flags](#heap-flags)
- [TLS Callbacks](#tls-callbacks)
- [Hardware Breakpoint Detection](#hardware-breakpoint-detection)
- [Software Breakpoint Detection (INT3 Scanning)](#software-breakpoint-detection-int3-scanning)
- [Exception-Based Anti-Debug](#exception-based-anti-debug)
- [NtSetInformationThread (Thread Hiding)](#ntsetinformationthread-thread-hiding)
- [Anti-VM / Anti-Sandbox](#anti-vm--anti-sandbox)
- [CPUID Hypervisor Bit](#cpuid-hypervisor-bit)
- [MAC Address / Hardware Fingerprinting](#mac-address--hardware-fingerprinting)
- [Timing-Based VM Detection](#timing-based-vm-detection)
- [File / Registry Artifacts](#file--registry-artifacts)
- [Resource Checks (CPU Count, RAM, Disk)](#resource-checks-cpu-count-ram-disk)
- [Anti-DBI (Dynamic Binary Instrumentation)](#anti-dbi-dynamic-binary-instrumentation)
- [Frida Detection](#frida-detection)
- [Pin/DynamoRIO Detection](#pindynamorio-detection)
- [Code Integrity / Self-Hashing](#code-integrity--self-hashing)
- [Anti-Disassembly Techniques](#anti-disassembly-techniques)
- [Opaque Predicates](#opaque-predicates)
- [Junk Bytes / Overlapping Instructions](#junk-bytes--overlapping-instructions)
- [Jump-in-the-Middle](#jump-in-the-middle)
- [Function Chunking / Scattered Code](#function-chunking--scattered-code)
- [Control Flow Flattening (Advanced)](#control-flow-flattening-advanced)
- [Mixed Boolean-Arithmetic (MBA) Identification & Simplification](#mixed-boolean-arithmetic-mba-identification--simplification)
- [SIGILL Handler for Execution Mode Switching (Hack.lu 2015)](#sigill-handler-for-execution-mode-switching-hacklu-2015)
- [SIGFPE Signal Handler Side-Channel via strace Counting (PlaidCTF 2017)](#sigfpe-signal-handler-side-channel-via-strace-counting-plaidctf-2017)
- [Instruction Trace Inversion with Keystone and Unicorn (MeePwn CTF 2017)](#instruction-trace-inversion-with-keystone-and-unicorn-meepwn-ctf-2017)
- [Call-less Function Chaining via Stack Frame Manipulation (THC CTF 2018)](#call-less-function-chaining-via-stack-frame-manipulation-thc-ctf-2018)
- [Comprehensive Bypass Strategies](#comprehensive-bypass-strategies)
- [Universal Bypass Checklist](#universal-bypass-checklist)
- [Layered Anti-Debug (Real-World Pattern)](#layered-anti-debug-real-world-pattern)
- [Quick Reference: Check to Bypass](#quick-reference-check-to-bypass)
---
## Linux Anti-Debug (Advanced)
### ptrace-Based
**Self-ptrace (most common):**
```c
if (ptrace(PTRACE_TRACEME, 0, 0, 0) == -1) exit(1); // Already traced = debugger attached
```
**Bypasses:**
```bash
# 1. LD_PRELOAD (see patterns.md for full hook)
LD_PRELOAD=./hook.so ./binary
# 2. Patch with pwntools
python3 -c "
from pwn import *
elf = ELF('./binary', checksec=False)
elf.asm(elf.symbols.ptrace, 'xor eax, eax; ret')
elf.save('patched')
"
# 3. GDB: catch the syscall
gdb ./binary
(gdb) catch syscall ptrace
(gdb) run
# When it stops at ptrace:
(gdb) set $rax = 0
(gdb) continue
# 4. Kernel config (requires root)
echo 0 > /proc/sys/kernel/yama/ptrace_scope
```
**Double-ptrace pattern:**
```c
// Fork child to ptrace parent — blocks all other debuggers
pid_t child = fork();
if (child == 0) {
ptrace(PTRACE_ATTACH, getppid(), 0, 0);
// Child sits in waitpid loop, keeping parent traced
} else {
// Parent continues with real logic
}
```
**Bypass:** Kill the watchdog child process, then attach debugger.
### /proc Filesystem Checks
```c
// TracerPid check
FILE *f = fopen("/proc/self/status", "r");
// Looks for "TracerPid:\t0" — non-zero means debugger
// /proc/self/exe link check (some debuggers change this)
readlink("/proc/self/exe", buf, sizeof(buf));
// /proc/self/maps — check for debugger libraries
grep("frida", "/proc/self/maps");
```
**Bypasses:**
```bash
# 1. LD_PRELOAD fopen/fread to fake /proc contents
# 2. Mount namespace isolation
unshare -m bash -c 'mount --bind /dev/null /proc/self/status && ./binary'
# 3. GDB: set breakpoint at fopen, change filename argument
(gdb) b fopen
(gdb) run
(gdb) set {char[20]} $rdi = "/dev/null"
(gdb) continue
```
### Timing-Based Detection
```c
// rdtsc (CPU timestamp counter)
uint64_t start = __rdtsc();
// ... code ...
uint64_t delta = __rdtsc() - start;
if (delta > THRESHOLD) exit(1); // too slow = debugger
// clock_gettime
struct timespec ts1, ts2;
clock_gettime(CLOCK_MONOTONIC, &ts1);
// ... code ...
clock_gettime(CLOCK_MONOTONIC, &ts2);
// gettimeofday
struct timeval tv1, tv2;
gettimeofday(&tv1, NULL);
```
**Bypasses:**
```bash
# 1. Frida hook (see tools-dynamic.md for clock_gettime hook)
# 2. GDB: skip rdtsc by patching with constant
(gdb) set {unsigned char[2]} 0x401234 = {0x90, 0x90} # NOP the rdtsc
# 3. Pin tool to fix TSC reads
# 4. faketime library
LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 FAKETIME="2024-01-01" ./binary
```
### Signal-Based Anti-Debug
```c
// SIGTRAP handler — INT3 under debugger is caught by debugger, not handler
signal(SIGTRAP, handler);
__asm__("int3");
// If handler runs: no debugger. If debugger catches: debugged.
// SIGALRM timeout — kill self if analysis takes too long
signal(SIGALRM, kill_handler);
alarm(5);
// SIGSEGV handler that does real work (see patterns.md for MBA pattern)
signal(SIGSEGV, real_logic_handler);
*(int*)0 = 0; // deliberate crash → handler runs real code
```
**Bypasses:**
```bash
# GDB: pass signals to program instead of handling them
(gdb) handle SIGTRAP nostop pass
(gdb) handle SIGALRM ignore
(gdb) handle SIGSEGV nostop pass
# For alarm-based: patch alarm() to return immediately
```
### Syscall-Level Evasion
```c
// Direct syscall instead of libc — bypasses LD_PRELOAD hooks
long ret;
asm volatile("syscall" : "=a"(ret) : "a"(101), "D"(0), "S"(0), "d"(0), "r"(0));
// Syscall 101 = ptrace on x86_64
```
**Bypass:** Must patch the binary itself or use ptrace to intercept at syscall level.
```bash
# GDB: catch syscall
(gdb) catch syscall 101
(gdb) commands
> set $rax = 0
> continue
> end
```
---
## Windows Anti-Debug (Advanced)
### PEB (Process Environment Block) Checks
```c
// BeingDebugged flag (offset 0x2 in PEB)
bool debugged = NtCurrentPeb()->BeingDebugged;
// NtGlobalFlag (offset 0x68/0xBC in PEB)
// When debugger: FLG_HEAP_ENABLE_TAIL_CHECK | FLG_HEAP_ENABLE_FREE_CHECK | FLG_HEAP_VALIDATE_PARAMETERS = 0x70
DWORD flags = *(DWORD*)((BYTE*)NtCurrentPeb() + 0xBC); // 64-bit offset
if (flags & 0x70) exit(1);
```
**Bypass (x64dbg):**
```text
# ScyllaHide plugin auto-patches PEB fields
# Manual: dump PEB, zero BeingDebugged and NtGlobalFlag
```
### NtQueryInformationProcess
```c
// ProcessDebugPort (0x7)
DWORD_PTR debugPort = 0;
NtQueryInformationProcess(GetCurrentProcess(), 7, &debugPort, sizeof(debugPort), NULL);
if (debugPort != 0) exit(1);
// ProcessDebugObjectHandle (0x1E)
HANDLE debugObj = NULL;
NTSTATUS status = NtQueryInformationProcess(GetCurrentProcess(), 0x1E, &debugObj, sizeof(debugObj), NULL);
if (status == 0) exit(1); // STATUS_SUCCESS means debugger present
// ProcessDebugFlags (0x1F) — returns inverse: 0 = debugger present
DWORD noDebug = 0;
NtQueryInformationProcess(GetCurrentProcess(), 0x1F, &noDebug, sizeof(noDebug), NULL);
if (noDebug == 0) exit(1);
```
**Bypass:** Hook `NtQueryInformationProcess` to return fake values, or use ScyllaHide.
### Heap Flags
```c
// Process heap has debug flags when debugger attached
PHEAP heap = (PHEAP)GetProcessHeap();
// Flags at offset 0x70 (64-bit): should be HEAP_GROWABLE (0x2)
// ForceFlags at offset 0x74: should be 0
if (heap->Flags != 0x2 || heap->ForceFlags != 0) exit(1);
```
### TLS Callbacks
**Key technique:** TLS (Thread Local Storage) callbacks execute BEFORE `main()` / entry point.
```c
// Registered in PE header's TLS directory
void NTAPI TlsCallback(PVOID DllHandle, DWORD Reason, PVOID Reserved) {
if (Reason == DLL_PROCESS_ATTACH) {
if (IsDebuggerPresent()) {
ExitProcess(1); // Kills process before main runs
}
}
}
#pragma comment(linker, "/INCLUDE:_tls_used")
#pragma data_seg(".CRT$XLB")
PIMAGE_TLS_CALLBACK callbacks[] = { TlsCallback, NULL };
```
**Detection in IDA/Ghidra:** Check PE TLS Directory → AddressOfCallBacks. Functions listed there run before EP.
**Bypass:** Set breakpoint on TLS callback in x64dbg (Options → Events → TLS Callbacks), or patch the TLS directory entry.
### Hardware Breakpoint Detection
```c
// Read debug registers via GetThreadContext
CONTEXT ctx;
ctx.ContextFlags = CONTEXT_DEBUG_REGISTERS;
GetThreadContext(GetCurrentThread(), &ctx);
if (ctx.Dr0 || ctx.Dr1 || ctx.Dr2 || ctx.Dr3) exit(1);
// Also via exception handler: deliberate exception, check DR regs in handler
```
**Bypass:**
```bash
# x64dbg: use software breakpoints instead, or hook GetThreadContext
# Frida: hook GetThreadContext to zero DR registers
```
### Software Breakpoint Detection (INT3 Scanning)
```c
// CRC / hash check over code section
unsigned char *code = (unsigned char*)function_addr;
uint32_t checksum = 0;
for (int i = 0; i < code_size; i++) {
checksum += code[i];
if (code[i] == 0xCC) exit(1); // INT3 = software breakpoint
}
if (checksum != EXPECTED_CHECKSUM) exit(1);
```
**Bypass:** Use hardware breakpoints (DR0-DR3) instead of software breakpoints. Or hook the scanning function.
### Exception-Based Anti-Debug
```c
// UnhandledExceptionFilter — under debugger, filter is NOT called
SetUnhandledExceptionFilter(handler);
RaiseException(EXCEPTION_ACCESS_VIOLATION, 0, 0, NULL);
// If handler runs: no debugger
// If debugger catches: debugger present
// INT 2D — debugger single-step anomaly
__asm { int 2dh } // Debugger silently consumes the exception
// If execution continues: debugger present
```
### NtSetInformationThread (Thread Hiding)
```c
// Hide thread from debugger — stops all debug events
typedef NTSTATUS(NTAPI *pNtSIT)(HANDLE, ULONG, PVOID, ULONG);
pNtSIT NtSIT = (pNtSIT)GetProcAddress(GetModuleHandle("ntdll"), "NtSetInformationThread");
NtSIT(GetCurrentThread(), 0x11 /*ThreadHideFromDebugger*/, NULL, 0);
// After this, debugger won't see breakpoints or exceptions from this thread
```
**Bypass:** Hook `NtSetInformationThread` to ignore class 0x11, or patch the call.
---
## Anti-VM / Anti-Sandbox
### CPUID Hypervisor Bit
```c
int regs[4];
__cpuid(regs, 1);
if (regs[2] & (1 << 31)) { // ECX bit 31 = hypervisor present
exit(1);
}
// Hypervisor brand string
__cpuid(regs, 0x40000000);
char brand[13] = {0};
memcpy(brand, &regs[1], 12);
// "VMwareVMware", "Microsoft Hv", "KVMKVMKVM", "XenVMMXenVMM"
```
**Bypass:** Patch `cpuid` results or use `LD_PRELOAD` to hook wrapper functions.
### MAC Address / Hardware Fingerprinting
```text
Known VM MAC prefixes:
VMware: 00:0C:29, 00:50:56
VirtualBox: 08:00:27
Hyper-V: 00:15:5D
Parallels: 00:1C:42
QEMU: 52:54:00
```
### Timing-Based VM Detection
```c
// VM exits on privileged instructions are measurably slower
uint64_t start = __rdtsc();
__cpuid(regs, 0); // Forces VM exit
uint64_t delta = __rdtsc() - start;
if (delta > 500) { /* likely VM */ }
```
### File / Registry Artifacts
```text
Files: C:\Windows\System32\drivers\vm*.sys, vbox*.dll, VBoxService.exe
Registry: HKLM\SOFTWARE\VMware, Inc.\VMware Tools
Services: VMTools, VBoxService
Processes: vmtoolsd.exe, VBoxTray.exe, qemu-ga.exe
Linux: /sys/class/dmi/id/product_name contains "VirtualBox"|"VMware"
dmesg | grep -i "hypervisor detected"
```
### Resource Checks (CPU Count, RAM, Disk)
```c
// Sandboxes typically have minimal resources
SYSTEM_INFO si;
GetSystemInfo(&si);
if (si.dwNumberOfProcessors < 2) exit(1);
MEMORYSTATUSEX ms;
ms.dwLength = sizeof(ms);
GlobalMemoryStatusEx(&ms);
if (ms.ullTotalPhys < 2ULL * 1024 * 1024 * 1024) exit(1); // < 2GB RAM
// Disk size check (< 60GB = sandbox)
GetDiskFreeSpaceEx("C:\\", NULL, &total, NULL);
```
**Bypass:** Use a VM configured with adequate resources (4+ CPUs, 8GB+ RAM, 100GB+ disk).
---
## Anti-DBI (Dynamic Binary Instrumentation)
### Frida Detection
```c
// 1. Check /proc/self/maps for frida-agent
FILE *f = fopen("/proc/self/maps", "r");
while (fgets(line, sizeof(line), f)) {
if (strstr(line, "frida") || strstr(line, "gadget")) exit(1);
}
// 2. Check for Frida's default port (27042)
int sock = socket(AF_INET, SOCK_STREAM, 0);
struct sockaddr_in addr = {.sin_family=AF_INET, .sin_port=htons(27042), .sin_addr.s_addr=inet_addr("127.0.0.1")};
if (connect(sock, (struct sockaddr*)&addr, sizeof(addr)) == 0) exit(1);
// 3. Check for inline hooks (function prologue modification)
// Compare first bytes of libc functions against expected values
unsigned char *strcmp_bytes = (unsigned char *)strcmp;
if (strcmp_bytes[0] == 0xE9 || strcmp_bytes[0] == 0xFF) exit(1); // JMP = hooked
// 4. Thread name check
// Frida creates threads with names like "gmain", "gdbus", "frida-*"
DIR *dir = opendir("/proc/self/task");
while ((entry = readdir(dir))) {
char comm_path[256];
snprintf(comm_path, sizeof(comm_path), "/proc/self/task/%s/comm", entry->d_name);
// Read comm and check for "gmain", "gdbus"
}
// 5. Named pipe detection (Windows)
// Frida creates \\.\pipe\frida-* named pipes
```
**Frida bypass of Frida detection:**
```javascript
// Hook the detection functions themselves
Interceptor.attach(Module.findExportByName(null, "strstr"), {
onEnter(args) {
this.haystack = Memory.readUtf8String(args[0]);
this.needle = Memory.readUtf8String(args[1]);
},
onLeave(retval) {
if (this.needle && (this.needle.includes("frida") || this.needle.includes("gadget"))) {
retval.replace(ptr(0)); // Not found
}
}
});
// Early Frida load (before anti-DBI runs)
// Use frida-gadget as early-init shared library
```
### Pin/DynamoRIO Detection
```c
// Check for instrumentation libraries in /proc/self/maps
// Pin: "pin-", "pinbin", "pinatrace"
// DynamoRIO: "dynamorio", "drcov", "drrun"
// Instruction count timing — DBI adds overhead
// Execute known instruction sequence, compare execution time
```
---
## Code Integrity / Self-Hashing
```c
// CRC32 over .text section
uint32_t crc = compute_crc32(text_start, text_size);
if (crc != EXPECTED_CRC) exit(1); // Code was modified (breakpoints, patches)
// MD5/SHA256 of function bodies
unsigned char hash[32];
SHA256(function_addr, function_size, hash);
if (memcmp(hash, expected_hash, 32) != 0) exit(1);
```
**Bypasses:**
1. **Hardware breakpoints** (don't modify code, DR0-DR3)
2. **Patch the comparison** to always succeed
3. **Hook the hash function** to return expected value
4. **Emulate** instead of debug (Unicorn/Qiling — no code modification)
5. **Snapshot + restore:** dump memory before and after, diff to find checks
**Self-checksumming in loops:**
```c
// Continuous integrity check in separate thread
void *watchdog(void *arg) {
while (1) {
if (compute_crc32(text_start, text_end - text_start) != saved_crc) {
memset(flag_buffer, 0, flag_len); // Destroy flag
exit(1);
}
usleep(100000);
}
}
```
**Bypass:** Kill the watchdog thread or patch its sleep to infinite.
---
## Anti-Disassembly Techniques
### Opaque Predicates
```asm
; Condition that always evaluates the same way but looks data-dependent
mov eax, [some_memory]
imul eax, eax ; x^2
and eax, 1 ; x^2 mod 2 is always 0 for any x
jnz fake_branch ; Never taken, but disassembler doesn't know
; real code here
```
**Identification:** Z3/SMT can prove branch is always/never taken.
### Junk Bytes / Overlapping Instructions
```asm
jmp real_code
db 0xE8 ; Looks like start of CALL to linear disassembler
real_code:
mov eax, 1 ; Real code — disassembler may misalign here
```
**Fix:** Switch to graph-mode disassembly (Ghidra/IDA handle this well). Manual: undefine and re-analyze from correct offset.
### Jump-in-the-Middle
```asm
; Jumps into the middle of a multi-byte instruction
eb 01 ; jmp +1 (skip next byte)
e8 ; fake CALL opcode — disassembler tries to decode as call
90 ; real: NOP (landed here from jmp)
```
### Function Chunking / Scattered Code
Functions split into non-contiguous chunks connected by unconditional jumps. Defeats linear function boundary detection.
**Tool:** IDA's "Append function tail" or Ghidra's "Create function" at each chunk.
### Control Flow Flattening (Advanced)
> 完整的 OLLVM 脱密工作流、变种生态（Hikari/Polaris/O-MVLL/Tigress/Hodur 等）和社区工具调研见 [ollvm-deobfuscation.md](references/ollvm-deobfuscation.md)。
Beyond basic switch-case (see patterns.md): modern OLLVM variants use:
- **Bogus control flow:** Fake branches with opaque predicates
- **Instruction substitution:** `a + b` → `a - (-b)`, `a ^ b` → `(a | b) & ~(a & b)`
- **String encryption:** Strings decrypted at runtime, cleared after use
**现代变种（2026 社区活跃）：** Hikari (Anti Class Dump/String Encryption/Indirect Branch), Polaris (原 Pluto, 含 Trap Angr 专门坑 angr), O-MVLL (Python 驱动, Android 加固常用), Arkari (goron 基础, 间接跳转可被数据段只读对抗), amice (Rust, 含 VM Flatten 需 VM 逆向而非 deflat)。变种识别详见 ollvm-deobfuscation.md 第 1 节。
**Deobfuscation tools (社区活跃度排序):**
- **obpo-plugin** (629⭐, IDA microcode+concolic 云插件, 效果最强): https://github.com/obpo-project/obpo-plugin
- **ollvm-breaker** (441⭐, Binary Ninja, Android .so 实战): https://github.com/amimo/ollvm-breaker
- **ollvm-unflattener** (265⭐, Miasm 符号执行, 纯脚本 x86/x64): https://github.com/cdong1012/ollvm-unflattener
- **d810-ng** (223⭐, IDA, 集成 Z3, 覆盖 OLLVM/Tigress/Hodur/Approov): https://github.com/w00tzenheimer/d810-ng — **本地首选**
- **DeObfBR** (96⭐, BR 间接分支混淆专项): https://github.com/Mrack/DeObfBR
- **D-810** (原版, 已较少维护, 建议用 d810-ng): pattern-based deobfuscation, MBA simplification
- **Miasm**: Symbolic execution for deobfuscation
- **Arybo** / **SiMBA**: MBA expression simplification
```bash
# d810-ng: 复制到 IDA plugins 目录, Ctrl-Shift-D 加载
# 选择 Unflattener + MBA simplification + Opaque predicate removal
# obpo: 右键 dispatcher → OBPO → Mark and process function (需联网)
# ⚠️ Pluto/Polaris 的 Trap Angr pass 会让 angr 失效 → 改用 d810-ng/Unicorn
```
### Mixed Boolean-Arithmetic (MBA) Identification & Simplification
```python
# Common MBA patterns and their simplified forms:
# (x & y) + (x | y) == x + y
# (x ^ y) + 2*(x & y) == x + y
# (x | y) - (x & ~y) == y
# ~(~x & ~y) == x | y (De Morgan's)
# (x | y) & ~(x & y) == x ^ y
# SiMBA tool for automated simplification:
# pip install simba-simplifier
from simba import simplify_mba
expr = "(a | b) + (a & b) - (~a & b)"
print(simplify_mba(expr)) # → a
```
---
## SIGILL Handler for Execution Mode Switching (Hack.lu 2015)
Binaries may install SIGILL (illegal instruction) handlers to switch between x86 and x86-64 execution modes or implement custom opcode dispatch:
1. **Signal registration:** `signal(SIGILL, handler)` installs a callback for illegal instruction exceptions
2. **Mode switching:** The handler modifies the saved instruction pointer or segment registers to switch between 32-bit and 64-bit code
3. **Custom opcodes:** Invalid x86 instructions trigger the handler, which interprets operand bytes as custom VM opcodes
```c
// Signal handler decodes "illegal" instructions as custom opcodes
void sigill_handler(int sig, siginfo_t *info, void *ucontext) {
ucontext_t *ctx = (ucontext_t *)ucontext;
unsigned char *pc = (unsigned char *)ctx->uc_mcontext.gregs[REG_RIP];
// Decode custom opcode from bytes at PC
// Advance PC past the custom instruction
ctx->uc_mcontext.gregs[REG_RIP] += opcode_length;
}
```
**Key insight:** If a binary installs signal handlers for SIGILL/SIGSEGV/SIGTRAP early in execution, suspect custom instruction dispatch. Trace signal deliveries with `strace -e signal` or set GDB to not intercept: `handle SIGILL nostop pass`.
---
## SIGFPE Signal Handler Side-Channel via strace Counting (PlaidCTF 2017)
Binary uses SIGFPE signal handlers for control flow, making static analysis unreliable. Brute-force by counting SIGFPE signals via strace — correct input characters produce more signals.
```bash
# Count SIGFPE signals per input character guess
for c in {a..z} {A..Z} {0..9}; do
count=$(echo -n "${c}AAAAAAA" | strace -e signal=SIGFPE ./binary 2>&1 | grep -c SIGFPE)
echo "$c: $count"
done
# Character producing the most SIGFPEs is correct
# Repeat for each position, extending the known prefix
```
**Key insight:** Signal handlers (SIGFPE, SIGSEGV, SIGILL) create implicit control flow invisible to static analysis. The number of signals raised correlates with validation progress. Counting signals via `strace -e signal=SIGFPE` turns opaque signal-based validation into a measurable side-channel for character-by-character brute-force.
---
## Instruction Trace Inversion with Keystone and Unicorn (MeePwn CTF 2017)
UPX-packed binary applies a sequence of arithmetic-only transforms (sub, add, xor, rol, ror) to the flag. No memory side-effects — purely register arithmetic. IDAPython traces non-jump instructions, the sequence is then inverted to recover the flag.
**Inversion rules:**
- Reverse the instruction sequence (last instruction first)
- Swap inverse pairs: `add ↔ sub`, `rol ↔ ror`, `xor` is self-inverse
```python
# IDAPython: collect non-jump instructions in the obfuscated routine
import idaapi, idc
def trace_transforms(start_ea, end_ea):
instructions = []
ea = start_ea
while ea < end_ea:
mnem = idc.print_insn_mnem(ea)
if mnem not in ('jmp', 'je', 'jne', 'call', 'ret'):
instructions.append((ea, mnem, idc.print_operands(ea)))
ea = idc.next_head(ea)
return instructions
transforms = trace_transforms(0x401000, 0x401200)
# Invert: reverse order, swap add/sub and rol/ror
inverse_map = {'add': 'sub', 'sub': 'add', 'rol': 'ror', 'ror': 'rol', 'xor': 'xor'}
inverted = [(mnem, op) for (_, mnem, op) in reversed(transforms)]
inverted = [(inverse_map.get(m, m), op) for m, op in inverted]
```
```python
# Assemble inverted instructions with Keystone, emulate with Unicorn
from keystone import *
from unicorn import *
from unicorn.x86_const import *
ks = Ks(KS_ARCH_X86, KS_MODE_64)
uc = Uc(UC_ARCH_X86, UC_MODE_64)
asm_src = '\n'.join(f'{mnem} {op}' for mnem, op in inverted)
encoding, _ = ks.asm(asm_src)
CODE_BASE = 0x400000
uc.mem_map(CODE_BASE, 0x10000)
uc.mem_write(CODE_BASE, bytes(encoding))
# Set initial register state to the observed output value
uc.reg_write(UC_X86_REG_RAX, known_output)
uc.emu_start(CODE_BASE, CODE_BASE + len(encoding))
flag_bytes = uc.reg_read(UC_X86_REG_RAX).to_bytes(8, 'little')
```
**PEB anti-debug note:** If the binary reads `PEB.BeingDebugged` and uses it to select between two comparison target values, the traced instructions under IDAPython may use the debug-mode target. Patch `BeingDebugged` to 0 before tracing, or identify both branches and use the non-debug target value.
**Key insight:** Arithmetic-only obfuscation (no memory writes) is fully reversible by tracing, inverting the instruction sequence, and swapping inverse operations. PEB anti-debug can silently change comparison targets — always verify which branch is taken.
**References:** MeePwn CTF 2017
---
### Call-less Function Chaining via Stack Frame Manipulation (THC CTF 2018)
**Pattern:** Binary hides function calls by building a linked list of function pointers on the stack, then modifying saved RBP and return addresses so `leave; ret` instructions chain through the list without any explicit `CALL` instructions. IDA fails to decompile because push/pop are unbalanced and function boundaries cannot be determined.
Each function in the chain:
1. Pushes operands and the next function's address onto the stack
2. Sets saved RBP to point to the next stack frame
3. Sets the return address to the next function
4. `leave` restores RSP from RBP (moving to next frame), `ret` jumps to the next function
```python
# Reversed processing chain (each function applied via leave/ret):
def reverse_processing(byte):
res = byte | 0x80 # OR 0x80
res = res ^ 0xCA # XOR 0xCA
res = (res + 66) & 0xFF # ADD 66
res = res ^ 0xCA # XOR 0xCA (repeated)
res = (res + 66) & 0xFF
res = res ^ 0xCA
res = (res + 66) & 0xFF
res = res ^ 0xFE # XOR 0xFE (final)
return res
# Apply in reverse order, then reverse the character sequence
```
**Key insight:** By manipulating saved RBP to point to the next stack frame and saved RIP to the next function, `leave; ret` chains through functions without any `call` instructions. Disassemblers that track call/ret balance fail to identify function boundaries. Patch each function body individually for IDA to handle them.
**Detection:** Binary with many small code blocks ending in `leave; ret` but no corresponding `call` instructions. Stack contains interleaved function pointers and data. IDA shows "stack frame is too big" or fails to create functions.
**References:** THC CTF 2018
---
## Comprehensive Bypass Strategies
### Universal Bypass Checklist
1. **Identify all anti-analysis checks** — search for: `ptrace`, `IsDebuggerPresent`, `rdtsc`, `cpuid`, `NtQuery`, `GetTickCount`, `CheckRemoteDebuggerPresent`, `/proc/self`, `SIGTRAP`, `alarm`
2. **Static patching** — NOP/patch checks with pwntools or Ghidra before running
3. **LD_PRELOAD** (Linux) — hook libc functions returning fake values
4. **ScyllaHide** (Windows x64dbg) — patches PEB, hooks NT functions automatically
5. **Emulation** (Unicorn/Qiling) — no debugger artifacts to detect
6. **Kernel-level bypass** — modify `/proc/sys/kernel/yama/ptrace_scope`, use `prctl`
### Layered Anti-Debug (Real-World Pattern)
Many CTF challenges stack multiple checks:
```text
1. TLS callback → IsDebuggerPresent (before main)
2. main() → ptrace(TRACEME)
3. Watchdog thread → timing check + /proc scan
4. Code section → self-CRC32 integrity
5. Signal handler → real logic in SIGSEGV handler
```
**Approach:** Identify ALL checks before patching. Patch or hook each one systematically. Run under emulator if too many to patch individually.
### Quick Reference: Check to Bypass
| Anti-Debug Check | Platform | Bypass |
|---|---|---|
| `ptrace(TRACEME)` | Linux | `LD_PRELOAD`, patch to `ret 0`, `catch syscall` |
| `IsDebuggerPresent` | Windows | ScyllaHide, Frida hook, PEB patch |
| `NtQueryInformationProcess` | Windows | ScyllaHide, hook ntdll |
| `rdtsc` timing | Both | NOP rdtsc, Frida time hook, Pin |
| `/proc/self/status` | Linux | Mount namespace, hook fopen |
| `alarm(N)` | Linux | `handle SIGALRM ignore` in GDB |
| `SIGTRAP` handler | Linux | `handle SIGTRAP nostop pass` |
| `SIGFPE` handler side-channel | Linux | `strace -e signal=SIGFPE` count per input |
| TLS callback | Windows | Break on TLS in x64dbg, patch |
| DR register scan | Windows | Use software BPs, hook GetThreadContext |
| INT3 scan / CRC | Both | Hardware BPs, patch CRC comparison |
| Frida detection | Both | Early-load gadget, hook strstr |
| CPUID hypervisor | Both | Patch CPUID result, bare metal |
| Thread hiding | Windows | Hook NtSetInformationThread |
---
## Agent 响应菜谱 A–T（Issue #65）
> 检测类长文仍见 `malware-analysis/references/anti-analysis-techniques.md`；OLLVM 长流程见 `references/ollvm-deobfuscation.md`。
> 本节是 **触发 → 动作 → Evidence** 短菜谱，供 agent 在 Dynamic/Static 旁路选用。
> **授权隔离 lab 默认**；静态 patch / 改 PEB / 改返回值不是未授权目标上的默认动作。
### 使用规则
1. 先 **识别并记录** 检测点（地址/API/字符串），再决定绕过或换环境。
2. 绕过尝试（成功或失败）MUST 写 Evidence；禁止把反调试退出写成「样本无害」。
3. H/S 不在此展开长文 → 跳转 OLLVM 专章。
4. L 仅 Linux/ELF 强制；Windows PE 主路径不因缺 TracerPid 判失败。
5. E 的 VT 对照为 **可选**；无外部情报源时写 n/a，不编造首次提交时间。
### 全表 A–T
| ID | 触发 | 处理动作 | Evidence | 优先级 |
|----|------|----------|----------|--------|
| **A** | `cpuid` 后条件跳（jz/jnz） | 识别 hypervisor 检测；lab 改标志位或 patch 跳转走恶意/真实业务分支；或换物理机 | `E-anti-debug-cpuid` | P0 |
| **B** | `rdtsc` + sub/cmp 时间差 | bp `rdtsc` / hook 时间 API；或 patch 比较；避免只靠「等沙箱超时」 | `E-anti-debug-rdtsc` | P0 |
| **C** | 字符串含 x64dbg/olly/windbg 等，或 Toolhelp 枚举 | bp `CreateToolhelp32Snapshot`→`Process32First/Next`；改匹配或跳过扫描分支 | `E-anti-debug-procscan` | P1 |
| **D** | `AddVectoredExceptionHandler` + 故意访问违例 | bp 注册点；定位 VEH handler 分析；调试器可忽略特定异常 | `E-anti-debug-veh` | P1 |
| **E** | TimeDateStamp 未来/0/荒谬；版本信息像合法厂商 | 与发现时间对照；**可选** VT 首次提交；`SigCheck` 看版本资源是否配合签名；无 VT → n/a | `E-meta-timestamp` | P2 |
| **F** | 显示有数字签名但来源可疑 | `SigCheck`：链有效？吊销？签名时间 vs 编译时间；**无效/吊销不得降低**威胁等级 | `E-sig-forge` | P0 |
| **G** | 多 PE 头、重叠节、节名伪装 | CFF/PE-bear/LoadPE 看真实映射与 EP 节；熵区分加密 vs 代码；不信节名 | `E-pe-anomaly` | P1 |
| **H** | F5 大量 `while(1)+switch`、星形 CFG | **See** `ollvm-deobfuscation.md`（d810/deflat 等）；插件不全则动态记录块序重构 | `E-cff` | P1 指针 |
| **I** | strings 无域名/IP 但有网/文件行为 | 找 Base64/XOR/自定义 decode；xref 解密函数；解密后 dump 回注 IDA | `E-string-decrypt` | P0 |
| **J** | 文件大小 ≫ 节原始数据之和（Overlay） | 提 overlay；`file`/熵；IDA 搜偏移引用；加密则动态抓密钥 | `E-overlay` | P1 |
| **K** | `fs:[0x30]`/`gs:[0x60]` → BeingDebugged / NtGlobalFlag | 改 PEB 标志或 ScyllaHide；或 patch 条件跳 | `E-anti-debug-peb` | P0 |
| **L** | 读 `/proc/self/status` 查 TracerPid≠0 | **Linux/ELF**：hook fopen/read 或 patch；Windows 不强制 | `E-anti-debug-tracerpid` | P2 平台 |
| **M** | `int3`(0xCC) 或读 DR0–DR7 | int3→nop；硬件 BP 检测用 ScyllaHide/软 BP；CRC 自检见补丁 6 | `E-anti-debug-bp` | P1 |
| **N** | IAT 空/极少 + 自写哈希解析 API | bp `GetProcAddress`/`Ldr*`；哈希反查导出表；回注符号；与「干净 IAT」铁律协同 | `E-api-hash` | P0 |
| **O** | 线性反汇编大量 db、花指令致错位 | F5/Hex-Rays；动态确认真流；junk nop 后 reanalyze；静还不全以动态为准 | `E-junk-code` | P2 |
| **P** | `NtQueryInformationProcess` class 7/30/31 | ScyllaHide 或 hook 返回；记 InformationClass | `E-anti-debug-ntqip` | P0 |
| **Q** | `.rsrc` 过大/高熵/非标准 RT_RCDATA | Resource Hacker/CFF 提取；`FindResource`/`LoadResource` xref；解密后 dump | `E-rsrc-payload` | P1 |
| **R** | 静态 IAT 无某 DLL，运行时才用 | 查 Delay Import Table；bp `__delayLoadHelper2` 或首次调用；纳入能力评估 | `E-delay-import` | P1 |
| **S** | 恒真/恒假条件、大片死代码 | **See** ollvm / angr 等；patch 唯一可达分支或动态路径回注 | `E-opaque-pred` | P1 指针 |
| **T** | ASCII strings 无结果，数据区像 UTF-16 | `strings -el` 或 `-encoding=utf-16le`；IDA Alt+A unicode；纳入 IOC | `E-wide-strings` | P0 |
### 与主 workflow 的挂接
| 阶段 | 菜谱 |
|------|------|
| Triage | E, F, G, T（元数据/签名/节/宽串） |
| Static | H, I, J, N 线索, O, Q, R, S |
| Dynamic | A–D, K, L, M, N, P + 既有断点四级火箭与无行为应急 |
| 失败 | 任何绕不过的检测 → Evidence + 换工具/环境；不静默降威胁 |
### 工具注记（非强制安装清单）
- Windows 用户态：x64dbg + **ScyllaHide**（PEB/NtQuery/硬件 BP 等批量隐藏）
- 签名：Sysinternals **SigCheck**
- PE 结构：PE-bear / CFF Explorer
- 平坦化：见 ollvm 专章工具表（d810-ng 等）
- 无某工具时：等价命令 + 记失败，禁止假装已验证签名/已脱平坦化

135
reverse-engineering/awesome-re-resources.md Normal file
View File

@@ -0,0 +1,135 @@
# 逆向工程参考资源汇总
> 精选自多个 awesome 列表，按实用性排序。AI 在逆向分析时可参考这些资源获取方法论和工具指导。
---
## 综合资源库
| 项目 | Stars | 覆盖 | 链接 |
|------|-------|------|------|
| **awesome-reversing** (tylerha97) | 3k+ | 逆向工具/书籍/课程/练习 | https://github.com/tylerha97/awesome-reversing |
| **awesome-reverse-engineering** (alphaSeclab) | 4k+ | 3500+ 工具 + 2300 文章，全平台 | https://github.com/alphaSeclab/awesome-reverse-engineering |
| **Reverse-Engineering** (mytechnotalent) | 10k+ | 免费教程：x86/x64/ARM/AVR/RISC-V | https://github.com/mytechnotalent/Reverse-Engineering |
| **awesome-malware-analysis** (rshipp) | 12k+ | 恶意软件分析工具/资源 | https://github.com/rshipp/awesome-malware-analysis |
| **reversingBits** | — | 逆向/二进制分析速查表合集 | https://github.com/mohitmishra786/reversingBits |
| **awesome-arm-exploitation** | — | ARM 利用资源（视频/文章/书籍） | https://github.com/HenryHoggard/awesome-arm-exploitation |
| **Binary-Analysis-Automation** | — | 自动化二进制分析（ML/脚本/静态/动态） | https://github.com/user1342/Awesome-Binary-Analysis-Automation |
---
## ELF / Linux 逆向专项
| 资源 | 说明 | 链接 |
|------|------|------|
| **libelfmaster** | 安全 ELF 解析库（取证/恶意软件重建） | https://github.com/elfmaster/libelfmaster |
| **ELF 规范** | 官方 ELF 格式文档 | https://refspecs.linuxfoundation.org/elf/elf.pdf |
| **Linux Internals** | /proc 文件系统、内存布局、syscall | https://0xax.gitbooks.io/linux-insides/ |
| **Compiler Explorer** | 在线看 C/C++/Rust/Go 编译成什么汇编 | https://godbolt.org/ |
---
## ARM / AArch64 专项
| 资源 | 说明 | 链接 |
|------|------|------|
| **ARM 官方架构手册** | 完整指令集参考 | https://developer.arm.com/documentation |
| **Azeria Labs** | ARM 汇编/利用教程（最佳入门） | https://azeria-labs.com/writing-arm-assembly-part-1/ |
| **ARM64 syscall 表** | Linux AArch64 系统调用号 | https://arm64.syscall.sh/ |
| **QEMU 用户态模拟** | 不需要真实设备分析 ARM 二进制 | `qemu-aarch64 -strace ./binary` |
---
## 恶意软件分析
| 资源 | 说明 | 链接 |
|------|------|------|
| **YARA** | 恶意软件特征匹配规则 | https://github.com/VirusTotal/yara |
| **Volatility 3** | 内存取证框架 | https://github.com/volatilityfoundation/volatility3 |
| **FLOSS** | 自动提取混淆字符串 | https://github.com/mandiant/flare-floss |
| **Detect It Easy (DiE)** | 文件类型/壳/编译器识别 | https://github.com/horsicq/Detect-It-Easy |
| **PE-bear** | PE 文件分析器 | https://github.com/hasherezade/pe-bear |
| **Capa** | 自动识别二进制能力（网络/文件/加密等） | https://github.com/mandiant/capa |
| **Unpacker** | 通用脱壳框架 | https://github.com/malwaretech/UnpackerFramework |
---
## 动态分析 / 沙箱
| 资源 | 说明 | 链接 |
|------|------|------|
| **Frida** | 跨平台动态插桩 | https://frida.re/ |
| **strace** | Linux 系统调用跟踪 | 系统自带 |
| **ltrace** | 库函数调用跟踪 | 系统自带 |
| **QEMU** | 用户态/系统态模拟 | https://www.qemu.org/ |
| **Unicorn** | CPU 模拟框架（可编程） | https://www.unicorn-engine.org/ |
| **Qiling** | 高级二进制模拟框架 | https://qiling.io/ |
| **angr** | 符号执行 + 二进制分析 | https://angr.io/ |
| **Triton** | 动态二进制分析框架 | https://triton-library.github.io/ |
---
## 反混淆 / 脱壳
| 资源 | 说明 | 链接 |
|------|------|------|
| **UPX** | 最常见的壳，`upx -d` 脱壳 | https://upx.github.io/ |
| **unipacker** | 通用 PE 脱壳器 | https://github.com/unipacker/unipacker |
| **de4dot** | .NET 反混淆 | https://github.com/de4dot/de4dot |
| **JADX** | Android DEX 反混淆 | https://github.com/skylot/jadx |
| **JEB** | 商业 Android/ARM 反编译器 | https://www.pnfsoftware.com/ |
| **Miasm** | 逆向工程框架（IR/符号执行/反混淆） | https://github.com/cea-sec/miasm |
| **OLLVM 反混淆** | 控制流平坦化/虚假控制流对抗 | 用 angr/Triton 符号执行恢复 |
---
## 在线分析平台
| 平台 | 说明 | 链接 |
|------|------|------|
| **VirusTotal** | 多引擎扫描 + 行为分析 | https://www.virustotal.com/ |
| **Joe Sandbox** | 自动化恶意软件分析 | https://www.joesandbox.com/ |
| **ANY.RUN** | 交互式在线沙箱 | https://any.run/ |
| **Hybrid Analysis** | 免费恶意软件分析 | https://www.hybrid-analysis.com/ |
| **Compiler Explorer** | 看编译器输出 | https://godbolt.org/ |
| **Dogbolt** | 多反编译器对比（IDA/Ghidra/Binary Ninja） | https://dogbolt.org/ |
---
## 学习路径
### 入门（0-3 个月）
1. [Reverse Engineering for Beginners](https://beginners.re/) — 免费电子书
2. [Azeria Labs ARM 教程](https://azeria-labs.com/) — ARM 汇编基础
3. [Nightmare](https://guyinatuxedo.github.io/) — CTF 逆向/Pwn 教程
4. [crackmes.one](https://crackmes.one/) — 逆向练习题
### 进阶（3-12 个月）
1. [Practical Binary Analysis](https://practicalbinaryanalysis.com/) — 实战二进制分析
2. [The IDA Pro Book](https://nostarch.com/idapro2.htm) — IDA 深度使用
3. [Malware Unicorn RE101](https://malwareunicorn.org/workshops/re101.html) — 恶意软件逆向
4. [pwnable.kr](http://pwnable.kr/) / [pwnable.tw](https://pwnable.tw/) — Pwn 练习
### 高级
1. [Modern Binary Exploitation](https://github.com/RPISEC/MBE) — RPI 课程
2. [How to Hack Like a Ghost](https://nostarch.com/how-hack-ghost) — 高级渗透
3. [Windows Internals](https://docs.microsoft.com/en-us/sysinternals/) — Windows 内核
4. 实战：分析真实恶意软件样本（MalwareBazaar）
---
## 速查表
| 速查表 | 链接 |
|--------|------|
| x86/x64 指令速查 | https://www.felixcloutier.com/x86/ |
| ARM64 指令速查 | https://developer.arm.com/documentation/ddi0602/latest |
| Linux syscall 表 (x64) | https://blog.rchapman.org/posts/Linux_System_Call_Table_for_x86_64/ |
| Linux syscall 表 (ARM64) | https://arm64.syscall.sh/ |
| GDB 速查 | https://darkdust.net/files/GDB%20Cheat%20Sheet.pdf |
| radare2 速查 | 本包 `radare2/references/cheatsheet.md` |
| IDA 快捷键 | https://hex-rays.com/products/ida/support/freefiles/IDA_Pro_Shortcuts.pdf |
| Ghidra 快捷键 | Ghidra 内置 Help → Keyboard Shortcuts |

220
reverse-engineering/crypto-decode-tools.md Normal file
View File

@@ -0,0 +1,220 @@
# 加解密 / 编解码工具速查
> 逆向和 CTF 中经常遇到加密/编码/哈希数据。本文档按场景列出最实用的工具。
---
## 自动识别 + 解密（不知道用了什么加密时）
| 工具 | Stars | 用途 | 链接 |
|------|-------|------|------|
| **Ciphey** | 18k+ | AI 自动识别并解密（支持 50+ 编码/加密/哈希） | https://github.com/Ciphey/Ciphey |
| **CyberChef** | 29k+ | 在线/离线编解码瑞士军刀（拖拽式操作） | https://github.com/gchq/CyberChef |
| **dcode.fr** | — | 在线 900+ 密码/编码/数学工具 | https://www.dcode.fr/ |
### Ciphey 使用
```bash
pip install ciphey
# 自动检测并解密
ciphey -t "密文"
# 从文件读取
ciphey -f encrypted.txt
```
Ciphey 支持：Base64/32/16、Caesar、Vigenere、XOR、AES（弱密钥）、Morse、Binary、Hex、URL encoding、HTML entities、哈希识别等。
### CyberChef 使用
```text
在线版：https://gchq.github.io/CyberChef/
离线版：下载 GitHub Release 的 HTML 文件直接打开
常用 Recipe：
- From Base64 → 解 Base64
- XOR → 异或解密（可暴力尝试 key）
- AES Decrypt → AES 解密
- Magic → 自动检测编码类型
```
---
## 哈希识别与破解
| 工具 | 用途 | 链接 |
|------|------|------|
| **hashID** | 识别哈希类型（MD5/SHA/bcrypt 等） | https://github.com/psypanda/hashID |
| **hash-identifier** | 同上，Python 版 | https://github.com/blackploit/hash-identifier |
| **haiti** | 现代哈希识别工具（更准确） | `gem install haiti` |
| **Hashcat** | GPU 哈希破解 | https://hashcat.net/ |
| **John the Ripper** | CPU 哈希破解 | https://www.openwall.com/john/ |
| **hashes.com** | 在线哈希查询（彩虹表） | https://hashes.com/ |
```bash
# 识别哈希类型
hashid '5f4dcc3b5aa765d61d8327deb882cf99'
# 输出: [+] MD5
# haiti（更准确）
haiti '5f4dcc3b5aa765d61d8327deb882cf99'
# Hashcat 破解
hashcat -m 0 hash.txt rockyou.txt # MD5
hashcat -m 1000 hash.txt rockyou.txt # NTLM
```
---
## RSA 攻击
| 工具 | 用途 | 链接 |
|------|------|------|
| **RsaCtfTool** | RSA 自动攻击（20+ 攻击方式） | https://github.com/Ganapati/RsaCtfTool |
| **SageMath** | 数学计算（大数分解/椭圆曲线） | https://www.sagemath.org/ |
| **factordb.com** | 在线大数分解查询 | http://factordb.com/ |
| **yafu** | 本地大数分解 | https://github.com/bbuhrow/yafu |
```bash
# RsaCtfTool 自动攻击
python RsaCtfTool.py --publickey pub.pem --private
python RsaCtfTool.py --publickey pub.pem --uncipherfile cipher.txt
# 支持的攻击：
# Wiener、Boneh-Durfee、Fermat、Pollard p-1、Williams p+1
# Common modulus、Small q、Hastads、Noveltyprimes 等
```
---
## XOR 分析
| 工具 | 用途 | 链接 |
|------|------|------|
| **xortool** | XOR 密钥长度猜测 + 已知明文攻击 | https://github.com/hellman/xortool |
| **CyberChef XOR** | 可视化 XOR 操作 | CyberChef 内置 |
```bash
# 猜测 XOR key 长度
xortool encrypted_file
# 用猜测的 key 长度解密
xortool -l 4 -c 00 encrypted_file
# 已知明文攻击（知道部分明文）
xortool-xor -f encrypted -s "known_plaintext"
```
---
## 古典密码
| 密码类型 | 工具 | 说明 |
|---------|------|------|
| Caesar | CyberChef / dcode.fr | 暴力 25 种偏移 |
| Vigenere | dcode.fr / Ciphey | 需要猜 key 长度 |
| Substitution | quipqiup.com | 频率分析自动求解 |
| Enigma | dcode.fr | 在线模拟器 |
| Rail Fence | dcode.fr / CyberChef | 栅栏密码 |
| Playfair | dcode.fr | 需要 key |
| Morse | CyberChef | 点划转文字 |
| Bacon | dcode.fr | 二进制隐写 |
| ROT13/47 | CyberChef / `tr` | 简单替换 |
---
## 编码识别与转换
| 编码 | 识别特征 | 解码方式 |
|------|---------|---------|
| Base64 | 末尾 `=` 或 `==`，字符集 A-Za-z0-9+/ | `base64 -d` / CyberChef |
| Base32 | 大写字母 + 2-7，末尾 `=` | CyberChef |
| Base58 | 无 0/O/I/l，常见于短标识符编码 | CyberChef |
| Hex | 只有 0-9a-f，长度为偶数 | `xxd -r -p` / CyberChef |
| URL encoding | `%XX` 格式 | `urldecode` / CyberChef |
| HTML entities | `&#XX;` 或 `&amp;` 格式 | CyberChef |
| Unicode escape | `\uXXXX` 格式 | Python `decode('unicode_escape')` |
| JWT | `xxxxx.yyyyy.zzzzz`（三段 Base64URL） | jwt.io / CyberChef |
| Brainfuck | 只有 `><+-.,[]` 八个字符 | 在线解释器 |
| Ook! | 只有 `Ook.` `Ook!` `Ook?` | 在线解释器 |
---
## 逆向中的加密识别
### 通过常量识别算法
| 常量/特征 | 算法 |
|-----------|------|
| `0x67452301, 0xEFCDAB89, 0x98BADCFE, 0x10325476` | MD5 |
| `0x6A09E667, 0xBB67AE85, 0x3C6EF372` | SHA-256 |
| `0x63, 0x7C, 0x77, 0x7B` (S-Box 开头) | AES |
| `0x243F6A88` (π 的十六进制) | Blowfish |
| `0xB7E15163, 0x9E3779B9` | RC5/RC6/TEA |
| `0x61707865` ("expa") | ChaCha20/Salsa20 |
| `0xC6EF3720` | XTEA |
### 通过行为识别
| 行为特征 | 可能的算法 |
|---------|-----------|
| 256 字节查找表 + swap 操作 | RC4 |
| 16 字节块 + 多轮置换 | AES |
| Feistel 结构（左右交换） | DES/Blowfish/TEA |
| 大数乘法/模幂 | RSA |
| 椭圆曲线点运算 | ECDSA/ECDH |
| 固定 64 轮循环 | TEA/XTEA |
| 32 轮 + delta 常量 | XTEA |
---
## 自动化密码分析
| 工具 | 用途 | 链接 |
|------|------|------|
| **FeatherDuster** | 自动化密码分析框架 | https://github.com/nccgroup/featherduster |
| **PkCrack** | ZIP 已知明文攻击 | https://www.unix-ag.uni-kl.de/~conrad/krypto/pkcrack.html |
| **bkcrack** | ZIP 已知明文攻击（现代版） | https://github.com/kimci86/bkcrack |
| **z3** | SMT 求解器（约束求解） | https://github.com/Z3Prover/z3 |
| **angr** | 符号执行（自动求解输入） | https://angr.io/ |
---
## 快速决策树
```text
拿到一段未知数据：
1. 看长度和字符集
- 只有 hex 字符 → 可能是 hex 编码或哈希
- 末尾有 = → Base64
- 三段点分 → JWT
- 32/40/64 字符 hex → 哈希（MD5/SHA1/SHA256）
2. 用 Ciphey 自动尝试
ciphey -t "数据"
3. 如果 Ciphey 失败 → 用 CyberChef Magic 模式
4. 如果是哈希 → hashID 识别类型 → Hashcat/John 破解
5. 如果是 RSA → RsaCtfTool 自动攻击
6. 如果是 XOR → xortool 分析 key
7. 如果是传统 ZIP 加密 → 优先使用 `bkcrack` 已知明文攻击，不要先做无证据的密码暴力
8. 如果是自定义加密 → IDA/Ghidra 逆向算法 → 手写解密脚本
```
---
## 在线资源
| 资源 | 链接 | 用途 |
|------|------|------|
| CyberChef | https://gchq.github.io/CyberChef/ | 万能编解码 |
| dcode.fr | https://www.dcode.fr/ | 900+ 密码工具 |
| quipqiup | https://quipqiup.com/ | 替换密码自动求解 |
| factordb | http://factordb.com/ | RSA 大数分解 |
| jwt.io | https://jwt.io/ | JWT 解码/验证 |
| hashes.com | https://hashes.com/ | 哈希反查 |
| crackstation | https://crackstation.net/ | 在线哈希破解 |

380
reverse-engineering/dsl-vm-reverse/SKILL.md Normal file
View File

@@ -0,0 +1,380 @@
---
name: dsl-vm-reverse
description: Reverse JavaScript-based custom DSL/VM interpreters, non-standard WASM-like runtimes, and risk-control engines. Use when analyzing IIFE or switch-based opcode dispatchers, extracting instruction tables, recovering bytecode semantics, capturing VM state at runtime, or reconstructing execution flow.
---
# 🔄 DSL 自定义虚拟机逆向（DSL VM Reverse Engineering）
## ACTION REQUIRED（读完后立刻执行）
1. `NOW`: 确认当前任务是自定义 JS opcode VM / 风控引擎，不是标准 WASM 或普通 webpack
2. `NOW`: `case-init` 直到 `scope.md` 就绪；离线样本用 `offline` / `lab`
3. `ACT`: 从「3. 通用逆向工作流」Phase 1 做文件分类，不要停在目录
> 用于逆向基于 JavaScript 实现的自定义 WASM 虚拟机/风控引擎
---
## 目录
- [1. 适用范围](#1-适用范围)
- [2. DSL VM 识别特征](#2-dsl-vm-识别特征)
- [3. 通用逆向工作流](#3-通用逆向工作流)
- [4. Opcode 提取与分类](#4-opcode-提取与分类)
- [5. 运行时捕获方案](#5-运行时捕获方案)
- [6. 常见状态码](#6-常见状态码)
- [7. Skill 自检清单](#7-skill-自检清单)
---
## 1. 适用范围
当目标文件符合以下 **任意特征** 时使用本 skill：
| # | 特征 | 说明 |
|---|------|------|
| 1 | IIFE 开头 + 大量单字母变量名 | `!function(){var U=void 0,y=parseInt,E0=Function,...}` |
| 2 | 包含 `DG()` 或类似函数含 switch-case 循环 | 解释器主循环，`d[7]&31` 解码 opcode |
| 3 | 大文件（500KB+）但零字节占比 < 1% | 非标准 WASM，纯 JS |
| 4 | 包含 `C[number]` 常量表引用 | `C[9][xxx]` 函数表/字符串表 |
| 5 | 单行压缩代码 | 583KB 单行，混淆变量名 |
### 排除规则
| 条件 | 非本 skill | 转至 |
|------|-----------|------|
| 文件以 `\x00asm` 开头 | 标准 WASM 二进制 | `reverse-engineering/languages.md` |
| 文件以 `Uint8Array([0,97,115,109])` 含 WASM 魔术字 | WASM 嵌入式 | 提取 .wasm 后转 IDA/Ghidra |
| 标准 Webpack 打包（`function(e,t,n){...}`） | 普通 JS | `js-reverse/` |
| 零字节占比 > 20% | WASM 二进制 | `reverse-engineering/languages.md` |
---
## 2. DSL VM 识别特征
### 代码特征
```javascript
// 特征 1: IIFE 入口，单字母变量映射数字常量
!function(){
var U=void 0, y=parseInt, E0=Function, AN=Uint8Array;
var E=15, l=10, m=12, x=16, S=13, $=11;
// 数字常量映射为变量名，替代原始数字
...
}
// 特征 2: 解释器主循环 DG()
function DG(C, d, ...) {
var d = []; // 数组模拟 WASM stack/locals
for (d[7] = x; d[7] !== U;) {
var aE = d[7] & 31; // 低 5 位 = opcode
var O = d[7] >> 5 & 31; // 高 5 位 = sub-operation
switch (aE) {
case 0: /* ... */ d[7] = 612; break;
case 1: /* ... */
// ... N 个 case
}
}
}
// 特征 3: 常量表 C[9] 存储函数索引和字符串
// C[9][0] = ["pc"] → 函数参数描述
// C[9][667] = "string" → 字符串常量
// C[9][x] = number → 函数索引
// 特征 4: W(C[index], null, ...) 调用模式
// W = Function.prototype.call.bind(call)
// 所有内置函数通过 C[index] 索引调用
// 特征 5: 指令编码格式
// d[7] = opcode(bit 0-4) | subop(bit 5-9) | operand(bit 10+)
```
### Opcode 编码格式
每条指令编码为 32 位整数：
```
bit 0-4: opcode (0-N)
bit 5-9: sub-operation (0-31)
bit 10-31: operand/立即数
解码:
aE = d[7] & 31 → opcode
O = d[7] >> 5 & 31 → sub-operation
d[other] = d[7] >> 10 → operand
```
---
## 3. 通用逆向工作流
### Phase 1: 文件分类（5 分钟）
```bash
# 检查是否为 DSL VM
python3 << 'EOF'
with open('target.js', 'rb') as f:
head = f.read(100)
# 1. 检查 WASM 魔术字
if head[:4] == b'\x00asm':
print("标准 WASM 二进制")
exit()
# 2. 检查零字节占比
data = open('target.js', 'rb').read()
zero_pct = data.count(b'\x00') / len(data) * 100
print(f"零字节占比: {zero_pct:.1f}%")
if zero_pct > 20:
print("WASM 二进制")
elif head[:2] == b'!f':
# 检查单字母变量模式
if b'var U=void 0' in head or b'U=void 0,y=parseInt' in head:
print("→ DSL VM!")
else:
print("普通 JS IIFE")
EOF
```
### Phase 2: 变量映射表提取（10 分钟）
```python
import re
with open('target.js', 'r', errors='replace') as f:
s = f.read()
# 提取开头 2000 字符的 var X=数字 映射
mappings = re.findall(r'var\s+(\w+)\s*=\s*(\d+)', s[:2000])
print('常量映射:')
for name, val in mappings:
print(f" {name:4s} = {val:3d} (0x{int(val):02x})")
```
### Phase 3: Opcode 提取与分类（15 分钟）
```python
# 1. 提取所有 case
all_cases = re.findall(r'case\s+(\d+):', s)
unique = sorted(set(int(c) for c in all_cases))
print(f"总 case: {len(all_cases)} 个")
print(f"唯一 opcode: {len(unique)} 个: {unique}")
# 2. 分类每个 opcode
for op in unique:
idx = s.find(f'case {op}:')
snippet = s[idx:idx+200]
if 'd[7]=' in snippet:
op_type = 'BRANCH'
elif 'return' in snippet:
op_type = 'RETURN'
elif 'W(C[' in snippet:
op_type = 'CALL'
elif 'new' in snippet:
op_type = 'ALLOC'
elif 'try' in snippet or 'catch' in snippet:
op_type = 'EXCEPTION'
else:
op_type = 'ARITH/STORE'
print(f" opcode {op:2d}: {op_type}")
```
### Phase 4: 常量表分析（30 分钟）
```python
const_refs = re.findall(r'C\[9\]\[(\d+)\]', s)
unique_refs = sorted(set(int(x) for x in const_refs))
print(f"C[9] 引用: {len(unique_refs)} 个索引")
print(f"范围: {min(unique_refs)} - {max(unique_refs)}")
# 对每个引用分析上下文
for ref in unique_refs[:20]:
idx = s.find(f'C[9][{ref}]')
ctx = s[max(0,idx-50):idx+80]
clean = ''.join(c if c.isprintable() else ' ' for c in ctx)
print(f" C[9][{ref}] → {clean}")
```
### Phase 5: 导出函数追踪（1-2 小时）
导出函数（如 `getToken`）通过以下路径定位：
```
1. 找 AWSCInner.register() 或类似注册调用
2. 确定注册的模块和工厂函数
3. 找工厂函数返回的对象 → 导出函数定义位置
4. 若函数名不在 JS 中 → 在 C[9] 常量表中作字节码存储
5. 追踪调用链:
AWSCInner._modules['fy'].getToken()
→ W(C[函数索引], null, ...)
→ DG() 解释器执行编码后的指令序列
```
### Phase 6: 运行时注入（若纯静态分析不够）
```javascript
// 注入最小 AWSC 兼容环境
const fakeEnv = {
AWSCInner: {
_modules: {},
register(name, moduleName, factory) {
this._modules[moduleName] = factory();
}
}
};
// 执行 DSL VM 代码
dslVmCode();
// 获取导出
const token = fakeEnv.AWSCInner._modules['fy'].getToken({});
```
---
## 4. Opcode 提取与分类
### 参考 opcode 对照表（基于已有案例）
| Opcode | 操作类型 | 特征 |
|--------|---------|------|
| 0 | **BRANCH** | `d[7]=xxx` 无条件跳转 |
| 1 | **CALL** | `W(C[Y],null,function(){...})` 嵌入函数调用 |
| 2 | **ARITH** | `d[4]=0`, `d[7]=72` 变量赋值 |
| 3 | **ARITH** | `d[0]=d[1][C[x]]`, `d[5]=d[0]<d[3]` 比较运算 |
| 4 | **STORE** | `d[8]=d[5]in d[4]` 属性访问/存在检查 |
| 5 | **ARITH** | `d[8]=d[4]-d[8]` 算术运算 |
| 6 | **RETURN** | `return gV`, `throw` 返回/抛出异常 |
| 7 | **ALLOC** | `d[6]=[]`, `d[6][C[8]](...)` push 操作 |
| 8 | **BRANCH** | `d[7]=d[k]?512:425` 条件跳转 |
| 9 | **STRING** | `d[6][C[t]]=d[m]`, `new fh(...)` 正则 |
| 10 | **ALLOC** | 函数参数准备、调用栈创建 |
| 11 | **STRING** | `new fh("\\s",d[5])` 正则匹配 |
| 12 | **STORE** | `P[d[9]]=d[4][C[H]](d[3])` 数据传递 |
| 13 | **CALL** | `C[9][113]=d[9]` 模块初始化 |
| 14 | **STRING** | `d[8]=d[9]+d[m]` 字符串拼接 |
| 15 | **RETURN** | `return EL;` 函数返回 |
| 16 | **ALLOC** | `var r,P,Z,B...` 局部变量声明 |
| 17 | **ALLOC** | `(Z=[])[C[8]](69,T,445)` 静态数组初始化 |
| 18 | **TABLE** | 函数表/类型表初始化 |
| 19 | **EXCEPTION** | `try{for(var RK=x;...` try-catch 循环 |
| 20 | **DOM** | `Is[d[o]]` DOM 操作 |
| 21 | **STORE** | 安全获取全局/对象属性 |
| 22 | **STRING** | `new fh(r,v)` 字符串/正则处理 |
| 23 | **BRANCH** | `try...catch` 安全获取 + 条件跳转 |
| 24 | **CALL** | `W(C[2],null,8,z,FL)` 多参数函数调用 |
| 25 | **EXCEPTION** | `try{...}catch(C){...}` 异常捕获 + 跳转 |
---
## 5. 运行时捕获方案
### 方案 A: Selenium + CDP 原生事件（推荐，成功率最高）
```python
from selenium import webdriver
driver = webdriver.Chrome()
# 注入反检测
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
"source": r"""
Object.defineProperty(navigator, 'webdriver', {get: () => false});
Object.defineProperty(navigator, 'plugins', {get: () => [1,2,3,4,5]});
Object.defineProperty(navigator, 'languages', {get: () => ['zh-CN','zh','en']});
"""
})
# 发送 CDP 原生鼠标事件
driver.execute_cdp_cmd("Input.dispatchMouseEvent", {
"type": "mousePressed",
"x": 549.5, "y": 441.2,
"button": "left", "buttons": 1,
"clickCount": 1, "pointerType": "mouse"
})
```
### 方案 B: Playwright 无头浏览器
```javascript
const { chromium } = require('playwright');
async function run() {
const browser = await chromium.launch();
const page = await browser.newPage();
// 拦截网络请求
await page.route('**/api/**', async route => {
await route.continue_();
});
await page.goto('https://target-page.com');
// 等待 DSL VM 初始化
await page.waitForFunction(() => {
return window.AWSCInner &&
window.AWSCInner._modules &&
window.AWSCInner._modules['fy'];
});
// 执行操作
await page.mouse.move(500, 400);
await page.mouse.down();
// ... 操作序列
await page.mouse.up();
}
```
### 方案 C: 纯协议验证（成功率极低）
> DSL VM 生成的 token 通常与浏览器上下文强绑定（TLS JA3 指纹、IP、Cookie、请求头等），脱离浏览器后服务端可检测到上下文不匹配。**不建议使用纯协议方案**。
---
## 6. 常见状态码
| Code | 含义 | 处理 |
|------|------|------|
| 0 | **验证通过** ✅ | 取出 sessionId + sig |
| 300 | **风控拦截** | 被拦截，无法通过 |
| 8778 | **验证失败，需重试** | 重试操作 |
| 8776 | **操作太快，需重试** | 增加延迟后重试 |
| 69634 | **通用失败** | 检查参数是否正确 |
---
## 7. Skill 自检清单
- [ ] 我是否完成了 DSL VM 识别（IIFE + 单字母变量 + DG() 解释器）？
- [ ] 我是否提取了变量映射表（`var X=数字`）？
- [ ] 我是否提取了 opcode 列表并分类？
- [ ] 我是否分析了常量表 C[9] 的引用范围？
- [ ] 我是否定位了导出函数注册点？
- [ ] 纯静态分析不够时，我是否尝试了运行时注入方案？
- [ ] 任务完成后是否回写了 field-journal？
- [ ] 是否发现新工具/新场景 → 更新 routing.md？
---
## 路由注册
| 类型 | 路由 |
|------|------|
| **目标类型**: WASM / DSL VM / 自定义指令集 | `reverse-engineering/dsl-vm-reverse/SKILL.md` |
| **用户意图**: "DSL VM / 风控引擎逆向" | 本 skill |
| **工具链**: Playwright / Selenium CDP | 浏览器注入方案 |
### 路径交叉
```
DSL VM 逆向路径:
reverse-engineering/dsl-vm-reverse/ → Phase 1-6 工作流
↓ 若需要捕获运行时数据
browser-automation/ → Playwright/Selenium CDP
↓ 若需要分析 API 协议层
js-reverse/ → Observe→Capture→Rebuild
```

296
reverse-engineering/elf-analysis.md Normal file
View File

@@ -0,0 +1,296 @@
# ELF 二进制深度分析参考
> 逆向 Linux/Android ELF 文件时的结构解析、反分析对抗识别和分析技巧。
---
## ELF 结构速查
### 文件头 (ELF Header)
```text
偏移 大小 字段 说明
0x00 4 e_ident[EI_MAG] Magic: 7f 45 4c 46 ("\x7fELF")
0x04 1 e_ident[EI_CLASS] 1=32bit, 2=64bit
0x05 1 e_ident[EI_DATA] 1=LE, 2=BE
0x10 2 e_type 2=EXEC, 3=DYN(PIE/SO), 4=CORE
0x12 2 e_machine 0x03=x86, 0x3E=x86_64, 0xB7=AArch64, 0x28=ARM
0x18 8 e_entry 入口点虚拟地址
0x20 8 e_phoff 程序头表偏移
0x28 8 e_shoff 节头表偏移（strip 后可能为 0）
0x38 2 e_phnum 程序头数量
0x3C 2 e_shnum 节头数量
```
### 程序头 (Program Header)
```text
类型值 名称 说明
0x01 PT_LOAD 可加载段（代码/数据）
0x02 PT_DYNAMIC 动态链接信息
0x03 PT_INTERP 解释器路径（/lib/ld-linux.so）
0x04 PT_NOTE 辅助信息
0x06 PT_PHDR 程序头表自身
0x6474e550 PT_GNU_EH_FRAME 异常处理
0x6474e551 PT_GNU_STACK 栈可执行标记
0x6474e552 PT_GNU_RELRO 只读重定位
```
### 常见节 (Sections)
| 节名 | 说明 |
|------|------|
| `.text` | 代码段 |
| `.rodata` | 只读数据（字符串常量） |
| `.data` | 已初始化全局变量 |
| `.bss` | 未初始化全局变量 |
| `.plt` / `.got` | 动态链接跳转表 |
| `.init_array` | 构造函数指针数组 |
| `.fini_array` | 析构函数指针数组 |
| `.dynamic` | 动态链接信息 |
| `.symtab` / `.dynsym` | 符号表 |
| `.strtab` / `.dynstr` | 字符串表 |
---
## 反分析手法识别
### 常见 ELF 反分析技术
| 手法 | 特征 | 对抗方式 |
|------|------|---------|
| 损坏程序头 | PHDR 填充垃圾数据（如 0x0a） | 手动修复或忽略损坏的 PHDR |
| 无 section header | `e_shoff = 0`, `e_shnum = 0` | 只依赖程序头分析，不依赖 section |
| 去符号 (strip) | 无 `.symtab`，函数名全丢 | GoReSym(Go) / 签名匹配 / FLIRT |
| 静态链接 | 无 `.dynamic`，体积巨大 | 用 FLIRT/Lumina 识别库函数 |
| 伪装文件类型 | 后缀 .sh/.txt/.jpg | 用 `file` 命令 / magic bytes 判断 |
| UPX 加壳 | 包含 `UPX!` 标记 | `upx -d` 脱壳 |
| 自定义壳 | 入口点跳转到解压代码 | 动态运行到 OEP 后 dump |
| 反调试 | ptrace(TRACEME) | LD_PRELOAD hook / patch |
| 反虚拟机 | 检查 /proc/cpuinfo | 修改 cpuinfo 或 hook 读取 |
| 代码加密 | 运行时解密 .text | 断点在解密后 dump |
### 识别自解压/自修改代码
```text
特征：
1. 入口点附近有 mmap(PROT_READ|PROT_WRITE|PROT_EXEC) 调用
2. 紧接着有 memcpy 或循环拷贝
3. 然后 mprotect 改权限
4. 最后 br/jmp 到新映射的地址
分析策略：
1. 找到 mmap 调用 → 记录返回的地址
2. 在 mprotect(PROT_EXEC) 后下断点
3. dump 解压后的内存区域
4. 作为新的二进制分析
```
---
## ARM64 (AArch64) 逆向速查
### 寄存器
| 寄存器 | 用途 |
|--------|------|
| x0-x7 | 参数/返回值 |
| x8 | 间接结果（syscall 号） |
| x9-x15 | 临时寄存器 |
| x16-x17 | IP0/IP1（PLT 跳转） |
| x18 | 平台寄存器（Android: shadow call stack） |
| x19-x28 | 被调用者保存 |
| x29 (FP) | 帧指针 |
| x30 (LR) | 链接寄存器（返回地址） |
| SP | 栈指针 |
| PC | 程序计数器 |
### 常见指令模式
```text
函数序言：
stp x29, x30, [sp, #-N]! # 保存 FP 和 LR
mov x29, sp # 设置帧指针
函数尾声：
ldp x29, x30, [sp], #N # 恢复 FP 和 LR
ret # 返回（br x30）
系统调用：
mov x8, #NR # syscall 号
svc #0 # 触发 syscall
条件分支：
cmp x0, #0
b.eq label # 等于跳转
b.ne label # 不等于跳转
cbz x0, label # x0 == 0 跳转
cbnz x0, label # x0 != 0 跳转
地址加载：
adrp x0, page # 加载页地址高位
add x0, x0, #offset # 加低 12 位偏移
ldr x0, [x1, #offset] # 从内存加载
```
### Linux ARM64 系统调用号
| 号码 | 名称 | 说明 |
|------|------|------|
| 56 | openat | 打开文件 |
| 63 | read | 读取 |
| 64 | write | 写入 |
| 57 | close | 关闭 |
| 222 | mmap | 内存映射 |
| 226 | mprotect | 修改内存权限 |
| 117 | ptrace | 进程跟踪 |
| 220 | clone | 创建进程/线程 |
| 221 | execve | 执行程序 |
| 93 | exit | 退出 |
| 94 | exit_group | 退出进程组 |
---
## 常见压缩/打包算法识别
| 算法 | 识别特征 | 解压方式 |
|------|---------|---------|
| **LZSS** | 位流 + 字面量/匹配标记 | 自定义解压器（如本报告） |
| **ZLIB/Deflate** | Magic: `78 01`/`78 9C`/`78 DA` | `zlib.decompress()` |
| **GZIP** | Magic: `1F 8B` | `gzip -d` / `gunzip` |
| **LZ4** | Magic: `04 22 4D 18` | `lz4 -d` |
| **LZMA/XZ** | Magic: `FD 37 7A 58 5A 00` (XZ) | `xz -d` / `lzma -d` |
| **Brotli** | 无固定 magic，看上下文 | `brotli -d` |
| **Zstandard** | Magic: `28 B5 2F FD` | `zstd -d` |
| **UPX** | 字符串 `UPX!` | `upx -d` |
| **自定义** | 入口点有解压循环 | 逆向算法后写解压器 |
### 识别自定义压缩的线索
```text
1. 入口点附近有循环 + 位操作（移位、AND、OR）
2. 有"滑动窗口"回拷（从输出缓冲区往回读）→ LZ 系列
3. 有频率表/霍夫曼树构建 → Deflate/Huffman
4. 有固定大小块处理 → 块压缩（LZ4/Snappy）
5. 有算术编码特征（区间缩小）→ LZMA/ANS
```
---
## Linux 进程注入技术
### mmap + 代码注入
```text
流程：
1. mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0)
2. 将 shellcode/payload 写入映射区域
3. mprotect(addr, size, PROT_READ|PROT_EXEC) # 改为可执行
4. 跳转到映射地址执行
特征：
- mmap 返回值被保存
- 紧接着有 memcpy 或循环写入
- 然后 mprotect 改权限
- 最后 br/blr 到该地址
```
### ptrace 注入
```text
流程：
1. ptrace(PTRACE_ATTACH, target_pid)
2. waitpid(target_pid)
3. ptrace(PTRACE_GETREGS, target_pid, &regs)
4. 修改 regs.pc 指向注入代码
5. ptrace(PTRACE_SETREGS, target_pid, &regs)
6. ptrace(PTRACE_CONT, target_pid)
特征：
- 打开 /proc/<pid>/mem 或使用 ptrace
- 读取/修改目标进程寄存器
- 写入 shellcode 到目标进程空间
```
### /proc/self/mem 自修改
```text
流程：
1. open("/proc/self/mem", O_RDWR)
2. lseek(fd, target_addr, SEEK_SET)
3. write(fd, new_code, size)
用途：
- 绕过 W^X 保护（mmap 的页不能同时 W+X）
- 修改自身代码段（.text 通常是只读的）
- 运行时 patch 指令
```
---
## 分析大型 ELF 的策略
对于 5MB+ 的大型二进制：
```text
1. 快速侦察（5 分钟）
- file / rabin2 -I → 架构、类型、保护
- strings | grep -i "error\|fail\|http\|/proc\|/dev" → 关键字符串
- rabin2 -i → 导入函数（如果有）
- rabin2 -E → 导出函数
2. 结构分析（10 分钟）
- readelf -l → 程序头（LOAD 段布局）
- 入口点附近代码 → 是否有解压/解密
- 找 .init_array → 构造函数（可能有反调试）
3. 定位关键逻辑
- 从字符串交叉引用入手
- 从系统调用（mmap/ptrace/open）入手
- 从网络函数（connect/send/recv）入手
4. 分而治之
- 如果是自解压 → 先解压，分析 payload
- 如果是多模块 → 按功能分块分析
- 用 binary-diff 对比不同版本
```
---
## 工具命令速查
```bash
# 基本信息
file binary
readelf -h binary # ELF 头
readelf -l binary # 程序头
readelf -S binary # 节头（如果有）
rabin2 -I binary # 综合信息
# 字符串
strings -a binary | less
rabin2 -z binary # 数据段字符串
rabin2 -zz binary # 全文件字符串
# 反汇编
r2 -A binary # radare2 分析
objdump -d binary # GNU 反汇编
aarch64-linux-gnu-objdump -d binary # ARM64 交叉反汇编
# 动态分析
strace -f ./binary # 系统调用跟踪
ltrace -f ./binary # 库函数跟踪
qemu-aarch64 -strace ./binary # ARM64 模拟执行
# 内存 dump
gdb -p <pid> -ex "dump memory out.bin 0xADDR 0xADDR+SIZE" -ex quit
# 修复损坏的 ELF
# 手动修改 e_phnum 或 patch 损坏的 PHDR
python -c "
import struct
with open('binary', 'r+b') as f:
f.seek(0x38) # e_phnum offset (64-bit)
f.write(struct.pack('<H', 2)) # 修改为正确的 PHDR 数量
"
```

539
reverse-engineering/field-notes.md Normal file
View File

@@ -0,0 +1,539 @@
# Reverse Engineering Field Notes
Detailed quick notes that support [`SKILL.md`](SKILL.md). Read this file after triage, not before.
## Table of Contents
- [Binary Types](#binary-types)
- [Python .pyc](#python-pyc)
- [WASM](#wasm)
- [Android APK](#android-apk)
- [Flutter APK (Dart AOT)](#flutter-apk-dart-aot)
- [.NET](#net)
- [Packed (UPX)](#packed-upx)
- [Tauri Packed Desktop Apps](#tauri-packed-desktop-apps)
- [Anti-Debugging Bypass](#anti-debugging-bypass)
- [Specialized Patterns](#specialized-patterns)
- [S-Box / Keystream Patterns](#s-box--keystream-patterns)
- [Custom VM Analysis](#custom-vm-analysis)
- [Python Bytecode Reversing](#python-bytecode-reversing)
- [Signal-Based Binary Exploration](#signal-based-binary-exploration)
- [Malware Anti-Analysis Bypass via Patching](#malware-anti-analysis-bypass-via-patching)
- [Expected Values Tables](#expected-values-tables)
- [x86-64 Gotchas](#x86-64-gotchas)
- [Iterative Solver Pattern](#iterative-solver-pattern)
- [Unicorn Emulation (Complex State)](#unicorn-emulation-complex-state)
- [Multi-Stage Shellcode Loaders](#multi-stage-shellcode-loaders)
- [Timing Side-Channel Attack](#timing-side-channel-attack)
- [Unstripped Binary Information Leaks](#unstripped-binary-information-leaks)
- [Custom Mangle Function Reversing](#custom-mangle-function-reversing)
- [Rust serde_json Schema Recovery](#rust-serde_json-schema-recovery)
- [Position-Based Transformation Reversing](#position-based-transformation-reversing)
- [Hex-Encoded String Comparison](#hex-encoded-string-comparison)
- [CTF Case Notes](#ctf-case-notes)
- [Embedded ZIP + XOR License Decryption](#embedded-zip--xor-license-decryption)
- [Stack String Deobfuscation (.rodata XOR Blob)](#stack-string-deobfuscation-rodata-xor-blob)
- [Prefix Hash Brute-Force](#prefix-hash-brute-force)
- [Mathematical Convergence Bitmap](#mathematical-convergence-bitmap)
- [RISC-V Binary Analysis](#risc-v-binary-analysis)
- [Kernel Module Maze Solving](#kernel-module-maze-solving)
- [Multi-Threaded VM with Channels](#multi-threaded-vm-with-channels)
- [CVP/LLL Lattice for Constrained Integer Validation](#cvplll-lattice-for-constrained-integer-validation)
- [Decision Tree Function Obfuscation](#decision-tree-function-obfuscation)
- [Android JNI RegisterNatives Obfuscation](#android-jni-registernatives-obfuscation)
- [Multi-Layer Self-Decrypting Binary](#multi-layer-self-decrypting-binary)
- [GLSL Shader VM with Self-Modifying Code](#glsl-shader-vm-with-self-modifying-code)
- [GF(2^8) Gaussian Elimination for Flag Recovery](#gf28-gaussian-elimination-for-flag-recovery)
- [Z3 for Single-Line Python Boolean Circuit](#z3-for-single-line-python-boolean-circuit)
- [Sliding Window Popcount Differential Propagation](#sliding-window-popcount-differential-propagation)
- [Ruby/Perl Polyglot Constraint Satisfaction](#rubyperl-polyglot-constraint-satisfaction)
- [Verilog/Hardware RE](#veriloghardware-re)
- [Custom binfmt Kernel Module with RC4 Flat Binaries](#custom-binfmt-kernel-module-with-rc4-flat-binaries)
- [Hash-Resolved Imports / No-Import Ransomware](#hash-resolved-imports--no-import-ransomware)
- [ELF Section Header Corruption for Anti-Analysis](#elf-section-header-corruption-for-anti-analysis)
- [Brainfuck Character-by-Character Static Analysis](#brainfuck-character-by-character-static-analysis)
- [Brainfuck Side-Channel via Read Count Oracle](#brainfuck-side-channel-via-read-count-oracle)
- [Brainfuck Comparison Idiom Detection](#brainfuck-comparison-idiom-detection)
- [Backdoored Shared Library Detection](#backdoored-shared-library-detection)
- [Go Binary Reversing](#go-binary-reversing)
- [Go Binary UUID Patching for C2 Enumeration](#go-binary-uuid-patching-for-c2-enumeration)
- [D Language Binary Reversing](#d-language-binary-reversing)
- [Rust Binary Reversing](#rust-binary-reversing)
- [Frida Dynamic Instrumentation](#frida-dynamic-instrumentation)
- [Frida Firebase Cloud Functions Bypass](#frida-firebase-cloud-functions-bypass)
- [angr Symbolic Execution](#angr-symbolic-execution)
- [Qiling Emulation](#qiling-emulation)
- [VMProtect / Themida Analysis](#vmprotect--themida-analysis)
- [Binary Diffing](#binary-diffing)
- [Advanced GDB (pwndbg, rr)](#advanced-gdb-pwndbg-rr)
- [macOS / iOS Reversing](#macos--ios-reversing)
- [Embedded / IoT Firmware RE](#embedded--iot-firmware-re)
- [Kernel Driver Reversing](#kernel-driver-reversing)
- [Swift / Kotlin Binary Reversing](#swift--kotlin-binary-reversing)
- [INT3 Patch + Coredump Brute-Force Oracle](#int3-patch--coredump-brute-force-oracle)
- [Signal Handler Chain + LD_PRELOAD Oracle](#signal-handler-chain--ld_preload-oracle)
- [Font Ligature Exploitation](#font-ligature-exploitation)
- [Instruction Counter as Cryptographic State](#instruction-counter-as-cryptographic-state)
- [Burrows-Wheeler Transform Inversion](#burrows-wheeler-transform-inversion)
- [FRACTRAN Program Inversion](#fractran-program-inversion)
- [Opcode-Only Trace Reconstruction](#opcode-only-trace-reconstruction)
- [Thread Race Signed Integer Overflow](#thread-race-signed-integer-overflow)
- [ESP32/Xtensa Firmware Reversing](#esp32xtensa-firmware-reversing)
- [Custom VM Bytecode Lifting to LLVM IR](#custom-vm-bytecode-lifting-to-llvm-ir)
- [SIGFPE Signal Handler Side-Channel](#sigfpe-signal-handler-side-channel)
- [Batch Crackme Automation via objdump](#batch-crackme-automation-via-objdump)
- [Android DEX Runtime Bytecode Patching](#android-dex-runtime-bytecode-patching)
- [Fork + Pipe + Dead Branch Anti-Analysis](#fork--pipe--dead-branch-anti-analysis)
- [Web/CTF Auth Bypass Case Notes](#webctf-auth-bypass-case-notes)
- [Signed Cookie Key Reuse: access token to admin_session](#signed-cookie-key-reuse-access-token-to-admin_session)
- [Web Phishing Infrastructure](#web-phishing-infrastructure)
- [Phishing Panel: {domain_a} / {domain_b}](#phishing-panel-domain_a--domain_b)
## Binary Types
### Python .pyc
Disassemble with `marshal.load()` + `dis.dis()`. Header: 8 bytes (2.x), 12 (3.0-3.6), 16 (3.7+). See [languages.md](languages.md#python-bytecode-reversing-disdis-output).
### WASM
```bash
wasm2c checker.wasm -o checker.c
gcc -O3 checker.c wasm-rt-impl.c -o checker
# WASM patching (challenge binaries):
wasm2wat main.wasm -o main.wat # Binary → text
# Edit WAT: flip comparisons, change constants
wat2wasm main.wat -o patched.wasm # Text → binary
```
### Android APK
`apktool d app.apk -o decoded/` for resources; `jadx app.apk` for Java decompilation. Check `decoded/res/values/strings.xml` for flags. See [tools.md](tools.md#android-apk).
### Flutter APK (Dart AOT)
If `lib/arm64-v8a/libapp.so` + `libflutter.so` present, use [Blutter](https://github.com/worawit/blutter): `python3 blutter.py path/to/app/lib/arm64-v8a out_dir`. Outputs reconstructed Dart symbols + Frida script. See [tools.md](tools.md#flutter-apk-blutter).
### .NET
- dnSpy - debugging + decompilation
- ILSpy - decompiler
### Packed (UPX)
```bash
upx -d packed -o unpacked
```
If unpacking fails, inspect UPX metadata first: verify UPX section names, header fields, and version markers are intact. If metadata looks tampered or uncertain, review UPX source on GitHub to identify likely modification points.
### Tauri Packed Desktop Apps
Tauri embeds Brotli-compressed frontend assets in the executable. Find `index.html` xrefs to locate asset index table, dump blobs, Brotli decompress. Reference: `tauri-codegen/src/embedded_assets.rs`.
## Anti-Debugging Bypass
Common checks:
- `IsDebuggerPresent()` / PEB.BeingDebugged / NtQueryInformationProcess (Windows)
- `ptrace(PTRACE_TRACEME)` / `/proc/self/status` TracerPid (Linux)
- TLS callbacks (run before main — check PE TLS Directory)
- Timing checks (`rdtsc`, `clock_gettime`, `GetTickCount`)
- Hardware breakpoint detection (DR0-DR3 via GetThreadContext)
- INT3 scanning / code self-hashing (CRC over .text section)
- Signal-based: SIGTRAP handler, SIGALRM timeout, SIGSEGV for real logic
- Frida/DBI detection: `/proc/self/maps` scan, port 27042, inline hook checks
Bypass: Set breakpoint at check, modify register to bypass conditional. pwntools patch: `elf.asm(elf.symbols.ptrace, 'ret')` to replace function with immediate return. See [patterns.md](patterns.md#pwntools-binary-patching-crypto-cat).
For comprehensive anti-analysis techniques and bypasses (30+ methods with code), see [anti-analysis.md](anti-analysis.md).
## Specialized Patterns
### S-Box / Keystream Patterns
**Xorshift32:** Shifts 13, 17, 5
**Xorshift64:** Shifts 12, 25, 27
**Magic constants:** `0x2545f4914f6cdd1d`, `0x9e3779b97f4a7c15`
### Custom VM Analysis
1. Identify structure: registers, memory, IP
2. Reverse `executeIns` for opcode meanings
3. Write disassembler mapping opcodes to mnemonics
4. Often easier to bruteforce than fully reverse
5. Look for the bytecode file loaded via command-line arg
See [patterns.md](patterns.md#custom-vm-reversing) for VM workflow, opcode tables, and state machine BFS.
**Sequential key-chain brute-force:** When a VM validates input in small blocks (e.g., 3 bytes = 2^24 candidates) with each block's output key feeding the next, brute-force each block sequentially with OpenMP parallelization. Compile solver with `gcc -O3 -march=native -fopenmp`. See [patterns-ctf-3.md](patterns-ctf-3.md#vm-sequential-key-chain-brute-force-midnight-flag-2026).
### Python Bytecode Reversing
XOR flag checkers with interleaved even/odd tables are common. See [languages.md](languages.md#python-bytecode-reversing-disdis-output) for bytecode analysis tips and reversing patterns.
### Signal-Based Binary Exploration
Binary uses UNIX signals as binary tree navigation; hook `sigaction` via `LD_PRELOAD`, DFS by sending signals. See [patterns.md](patterns.md#signal-based-binary-exploration).
### Malware Anti-Analysis Bypass via Patching
Flip `JNZ`/`JZ` (0x75/0x74), change sleep values, patch environment checks in Ghidra (`Ctrl+Shift+G`). See [patterns.md](patterns.md#malware-anti-analysis-bypass-via-patching).
### Expected Values Tables
Locate with `objdump -s -j .rodata binary | less` — look near comparison instructions, size matches flag length.
### x86-64 Gotchas
Sign extension and 32-bit truncation pitfalls. See [patterns.md](patterns.md#x86-64-gotchas) for details and code examples.
### Iterative Solver Pattern
Try each byte (0-255) per position, match against expected output. **Uniform transform shortcut:** if one input byte only changes one output byte, build 0..255 mapping then invert. See [patterns.md](patterns.md) for full implementation.
### Unicorn Emulation (Complex State)
`from unicorn import *` -- map segments, set up stack, hook to trace. **Mixed-mode pitfall:** 64-bit stub jumping to 32-bit via `retf` requires switching to UC_MODE_32 and copying GPRs + EFLAGS + XMM regs. See [tools.md](tools.md#unicorn-emulation).
### Multi-Stage Shellcode Loaders
Nested shellcode with XOR decode loops; break at `call rax`, bypass ptrace with `set $rax=0`, extract flag from `mov` instructions. See [patterns.md](patterns.md#multi-stage-shellcode-loaders).
### Timing Side-Channel Attack
Validation time varies per correct character; measure elapsed time per candidate to recover flag byte-by-byte. See [patterns.md](patterns.md#timing-side-channel-attack).
### Unstripped Binary Information Leaks
**Pattern:** Debug info and file paths leak author identity. Quick checks: `strings binary | grep "/home/"` (home dirs), `file binary` (stripped?), `readelf -S binary | grep debug` (debug sections).
### Custom Mangle Function Reversing
Binary mangles input 2 bytes at a time with running state; extract target from `.rodata`, write inverse function. See [patterns.md](patterns.md#custom-mangle-function-reversing).
### Rust serde_json Schema Recovery
Disassemble serde `Visitor` implementations to recover expected JSON schema; field names in order reveal flag. See [languages-platforms.md](languages-platforms.md#rust-serdejson-schema-recovery).
### Position-Based Transformation Reversing
Binary adds/subtracts position index; reverse by undoing per-index offset. See [patterns.md](patterns.md#position-based-transformation-reversing).
### Hex-Encoded String Comparison
Input converted to hex, compared against constant. Decode with `xxd -r -p`. See [patterns.md](patterns.md#hex-encoded-string-comparison).
## CTF Case Notes
### Embedded ZIP + XOR License Decryption
Binary with named symbols (`EMBEDDED_ZIP`, `ENCRYPTED_MESSAGE`) in `.rodata` → extract ZIP containing license, XOR encrypted message with license bytes to recover flag. No execution needed. See [patterns-ctf-2.md](patterns-ctf-2.md#embedded-zip-xor-license-decryption-metactf-2026).
### Stack String Deobfuscation (.rodata XOR Blob)
Binary mmaps `.rodata` blob, XOR-deobfuscates, uses it to validate input. Reimplement verification loop with pyelftools to extract blob. Look for `0x9E3779B9`, `0x85EBCA6B` constants and `rol32()`. See [patterns-ctf-2.md](patterns-ctf-2.md#stack-string-deobfuscation-from-rodata-xor-blob-nullcon-2026).
### Prefix Hash Brute-Force
Binary hashes every prefix independently. Recover one character at a time by matching prefix hashes. See [patterns-ctf-2.md](patterns-ctf-2.md#prefix-hash-brute-force-nullcon-2026).
### Mathematical Convergence Bitmap
**Pattern:** Binary classifies coordinate pairs by Newton's method convergence (e.g., z^3-1=0). Grid of pass/fail results renders ASCII art flag. Key: the binary is a classifier, not a checker — reverse the math and visualize. See [patterns-ctf.md](patterns-ctf.md#mathematical-convergence-bitmap-ehax-2026).
### RISC-V Binary Analysis
Statically linked, stripped RISC-V ELF. Use Capstone with `CS_MODE_RISCVC | CS_MODE_RISCV64` for mixed compressed instructions. Emulate with `qemu-riscv64`. Watch for fake flags and XOR decryption with incremental keys. See [tools.md](tools.md#risc-v-binary-analysis-ehax-2026).
### Kernel Module Maze Solving
Rust kernel module implements maze via device ioctls. Enumerate commands dynamically, build DFS solver with decoy avoidance, deploy as minimal static binary (raw syscalls, no libc). See [patterns-ctf.md](patterns-ctf.md#kernel-module-maze-solving-dicectf-2026).
### Multi-Threaded VM with Channels
Custom VM with 16+ threads communicating via futex channels. Trace data flow across thread boundaries, extract constants from GDB, watch for inverted validity logic, solve via BFS state space search. See [patterns-ctf.md](patterns-ctf.md#multi-threaded-vm-with-channel-synchronization-dicectf-2026).
### CVP/LLL Lattice for Constrained Integer Validation
Binary validates flag via matrix multiplication with 64-bit coefficients; solutions must be printable ASCII. Use LLL reduction + CVP in SageMath to find nearest lattice point in the constrained range. Two-phase pattern: Phase 1 recovers AES key, Phase 2 decrypts custom VM bytecode with another linear system (mod 2^32). See [patterns-ctf-2.md](patterns-ctf-2.md#cvplll-lattice-for-constrained-integer-validation-htb-shadowlabyrinth).
### Decision Tree Function Obfuscation
~200+ auto-generated functions routing input through polynomial comparisons. Script extraction via Ghidra headless rather than reversing each function manually. Constraint propagation from known output format cascades through arithmetic constraints. See [patterns-ctf-2.md](patterns-ctf-2.md#decision-tree-function-obfuscation-htb-wondersms).
### Android JNI RegisterNatives Obfuscation
`RegisterNatives` in `JNI_OnLoad` hides which C++ function handles each Java native method (no standard `Java_com_pkg_Class_method` symbol). Find the real handler by tracing `JNI_OnLoad` → `RegisterNatives` → `fnPtr`. Use x86_64 `.so` from APK for best Ghidra decompilation. See [languages-platforms.md](languages-platforms.md#android-jni-registernatives-obfuscation-htb-wondersms).
### Multi-Layer Self-Decrypting Binary
N-layer binary where each layer decrypts the next using user-provided key bytes + SHA-NI. Use oracle (correct key → valid code with expected pattern). JIT execution with fork-per-candidate COW isolation for speed. See [patterns-ctf-2.md](patterns-ctf-2.md#multi-layer-self-decrypting-binary-dicectf-2026).
### GLSL Shader VM with Self-Modifying Code
**Pattern:** WebGL2 fragment shader implements Turing-complete VM on a 256x256 RGBA texture (program memory + VRAM). Self-modifying code (STORE opcode) patches drawing instructions. GPU parallelism causes write conflicts — emulate sequentially in Python to recover full output. See [patterns-ctf-3.md](patterns-ctf-3.md#glsl-shader-vm-with-self-modifying-code-apoorvctf-2026).
### GF(2^8) Gaussian Elimination for Flag Recovery
**Pattern:** Binary performs Gaussian elimination over GF(2^8) with the AES polynomial (0x11b). Matrix + augmentation vector in `.rodata`; solution vector is the flag. Look for constant `0x1b` in disassembly. Addition is XOR, multiplication uses polynomial reduction. See [patterns-ctf-2.md](patterns-ctf-2.md#gf28-gaussian-elimination-for-flag-recovery-apoorvctf-2026).
### Z3 for Single-Line Python Boolean Circuit
**Pattern:** Single-line Python (2000+ semicolons) with walrus operator chains validates flag as big-endian integer via boolean circuit. Obfuscated XOR `(a | b) & ~(a & b)`. Split on semicolons, translate to Z3 symbolically, solve in under a second. See [patterns-ctf-3.md](patterns-ctf-3.md#z3-for-single-line-python-boolean-circuit-bearcatctf-2026).
### Sliding Window Popcount Differential Propagation
**Pattern:** Binary validates input via expected popcount for each position of a 16-bit sliding window. Popcount differences create a recurrence: `bit[i+16] = bit[i] + (data[i+1] - data[i])`. Brute-force ~4000-8000 valid initial 16-bit windows; each determines the entire bit sequence. See [patterns-ctf-3.md](patterns-ctf-3.md#sliding-window-popcount-differential-propagation-bearcatctf-2026).
### Ruby/Perl Polyglot Constraint Satisfaction
**Pattern:** Single file valid in both Ruby and Perl, each imposing different constraints on a key. Exploits `=begin`/`=end` (Ruby block comment) vs `=begin`/`=cut` (Perl POD) to run different code per interpreter. Intersect constraints from both languages to recover the unique key. See [languages-platforms.md](languages-platforms.md#rubyperl-polyglot-constraint-satisfaction-bearcatctf-2026).
### Verilog/Hardware RE
**Pattern:** Verilog HDL source for state machines with hidden conditions gated on shift register history. Analyze `always @(posedge clk)` blocks and `case` statements to find correct input sequences. See [languages-platforms.md](languages-platforms.md#veriloghardware-reverse-engineering-srdnlenctf-2026).
### Custom binfmt Kernel Module with RC4 Flat Binaries
**Pattern:** Kernel module registers binfmt handler for encrypted flat binaries. Reverse the `.ko` to find RC4 key (in `movabs` immediates), decrypt the flat binary, import at the fixed virtual address from the module's `vm_mmap` call. See [patterns-ctf.md](patterns-ctf.md#custom-binfmt-kernel-module-with-rc4-flat-binaries-bsidessf-2026).
### Hash-Resolved Imports / No-Import Ransomware
**Pattern:** Binary with zero visible imports resolves APIs via symbol name hashing at runtime. Skip the hash reversing — hook OpenSSL functions via `LD_PRELOAD` in Docker to capture AES keys directly. See [patterns-ctf.md](patterns-ctf.md#hash-resolved-imports-no-import-ransomware-bsidessf-2026).
### ELF Section Header Corruption for Anti-Analysis
**Pattern:** Corrupted section headers crash analysis tools but program headers are intact so binary runs normally. Patch `e_shoff` to zero or use `readelf -l` (program headers only). Flag hidden after corrupted sections with magic marker + XOR. See [patterns-ctf.md](patterns-ctf.md#elf-section-header-corruption-for-anti-analysis-bsidessf-2026).
### Brainfuck Character-by-Character Static Analysis
**Pattern:** BF programs validating input have `,` (read char) followed by `+` operations whose count = expected ASCII value. Extract increment counts per input position to recover expected input without execution. See [languages.md](languages.md#brainfuck-character-by-character-static-analysis-bsidessf-2026).
### Brainfuck Side-Channel via Read Count Oracle
**Pattern:** BF input validators read more bytes when a character is correct. Count `,` operations per candidate — highest read count = correct byte. Character-by-character recovery. See [languages.md](languages.md#brainfuck-side-channel-via-read-count-oracle-bsidessf-2026).
### Brainfuck Comparison Idiom Detection
**Pattern:** Compiled BF uses fixed idioms for equality checks (`<[-<->] +<[>-<[-]]>[-<+>]`). Instrument interpreter to detect patterns and extract comparison operands (expected flag bytes). See [languages.md](languages.md#brainfuck-comparison-idiom-detection-bsidessf-2026).
### Backdoored Shared Library Detection
Binary works in GDB but fails when run normally (suid)? Check `ldd` for non-standard libc paths, then `strings | diff` the suspicious vs. system library to find injected code/passwords. See [patterns-ctf.md](patterns-ctf.md#backdoored-shared-library-detection-via-string-diffing-hacklu-ctf-2012).
### Go Binary Reversing
Large static binary with `go.buildid`? Use GoReSym to recover function names (works even on stripped binaries). Go strings are `{ptr, len}` pairs — not null-terminated. Look for `main.main`, `runtime.gopanic`, channel ops (`runtime.chansend1`/`chanrecv1`). Use Ghidra golang-loader plugin for best results. See [languages-compiled.md](languages-compiled.md#go-binary-reversing).
### Go Binary UUID Patching for C2 Enumeration
**Pattern:** Go C2 client with UUID from `-ldflags -X`. Binary-patch UUID bytes (same length), register with C2, enumerate clients/files via API. See [languages-compiled.md](languages-compiled.md#go-binary-uuid-patching-for-c2-client-enumeration-bsidessf-2026).
### D Language Binary Reversing
D language binaries have unique symbol mangling (not C++ style). Template-heavy, many function variants. Look for `_D` prefix in symbols. See [languages-compiled.md](languages-compiled.md#d-language-binary-reversing-csaw-ctf-2016).
### Rust Binary Reversing
Binary with `core::panicking` strings and `_ZN` mangled symbols? Use `rustfilt` for demangling. Panic messages contain source paths and line numbers — `strings binary | grep "panicked"` is the fastest approach. Option/Result enums use discriminant byte (0=None/Err, 1=Some/Ok). See [languages-compiled.md](languages-compiled.md#rust-binary-reversing).
### Frida Dynamic Instrumentation
Hook runtime functions without modifying binary. `frida -f ./binary -l hook.js` to spawn with instrumentation. Hook `strcmp`/`memcmp` to capture expected values, bypass anti-debug by replacing `ptrace` return value, scan memory for flag patterns, replace validation functions. See [tools-dynamic.md](tools-dynamic.md#frida-dynamic-instrumentation).
### Frida Firebase Cloud Functions Bypass
**Pattern:** Android app validates via Firebase Cloud Functions. Post-login Frida hook constructs valid payload (UID + value + timestamp) and calls Cloud Function directly, bypassing QR/payment validation. See [languages-platforms.md](languages-platforms.md#frida-firebase-cloud-functions-bypass-bsidessf-2026).
### angr Symbolic Execution
Automatic path exploration to find inputs satisfying constraints. Load binary with `angr.Project`, set find/avoid addresses, call `simgr.explore()`. Constrain input to printable ASCII and known prefix for faster solving. Hook expensive functions (crypto, I/O) to prevent path explosion. See [tools-dynamic.md](tools-dynamic.md#angr-symbolic-execution).
### Qiling Emulation
Cross-platform binary emulation with OS-level support (syscalls, filesystem). Emulate Linux/Windows/ARM/MIPS binaries on any host. No debugger artifacts — bypasses all anti-debug by default. Hook syscalls and addresses with Python API. See [tools-dynamic.md](tools-dynamic.md#qiling-framework-cross-platform-emulation).
### VMProtect / Themida Analysis
VMProtect virtualizes code into custom bytecode. Identify VM entry (pushad-like), find handler table (large indirect jump), trace handlers dynamically. For CTF, focus on tracing operations on input rather than full devirtualization. Themida: dump at OEP with ScyllaHide + Scylla. See [tools-advanced.md](tools-advanced.md#vmprotect-analysis).
### Binary Diffing
BinDiff and Diaphora compare two binaries to highlight changes. Essential when challenge provides patched/original versions. Export from IDA/Ghidra, diff to find vulnerability or hidden functionality. See [tools-advanced.md](tools-advanced.md#binary-diffing).
### Advanced GDB (pwndbg, rr)
pwndbg: `context`, `vmmap`, `search -s "flag{"`, `telescope $rsp`. GEF alternative. Reverse debugging with `rr record`/`rr replay` — step backward through execution. Python scripting for brute-force and automated tracing. See [tools-advanced.md](tools-advanced.md#advanced-gdb-techniques).
### macOS / iOS Reversing
Mach-O binaries: `otool -l` for load commands, `class-dump` for Objective-C headers. Swift: `swift demangle` for symbols. iOS apps: decrypt FairPlay DRM with frida-ios-dump, bypass jailbreak detection with Frida hooks. Re-sign patched binaries with `codesign -f -s -`. See [platforms.md](platforms.md#macos-ios-reversing).
### Embedded / IoT Firmware RE
`binwalk -Me firmware.bin` for recursive extraction. Hardware: UART/JTAG/SPI flash for firmware dumps. Filesystems: SquashFS (`unsquashfs`), JFFS2, UBI. Emulate with QEMU: `qemu-arm -L /usr/arm-linux-gnueabihf/ ./binary`. See [platforms.md](platforms.md#embedded-iot-firmware-re).
### Kernel Driver Reversing
Linux `.ko`: find ioctl handler via `file_operations` struct, trace `copy_from_user`/`copy_to_user`. Debug with QEMU+GDB (`-s -S`). eBPF: `bpftool prog dump xlated`. Windows `.sys`: find `DriverEntry` → `IoCreateDevice` → IRP handlers. See [platforms.md](platforms.md#kernel-driver-reversing).
### Swift / Kotlin Binary Reversing
Swift: `swift demangle` symbols, protocol witness tables for dispatch, `__swift5_*` sections. Kotlin/JVM: coroutines compile to state machines in `invokeSuspend`, `jadx` with Kotlin mode for best decompilation. Kotlin/Native: LLVM backend, looks like C++ in disassembly. See [languages-compiled.md](languages-compiled.md#swift-binary-reversing).
### INT3 Patch + Coredump Brute-Force Oracle
Patch `0xCC` (INT3) after transform output, enable core dumps, brute-force each input character by extracting computed state from coredump via `strings`. Avoids full reverse of transformation. See [patterns.md](patterns.md#int3-patch-coredump-brute-force-oracle-pwn2win-2016).
### Signal Handler Chain + LD_PRELOAD Oracle
Binary uses signal handler chains for per-character password validation. Hook `signal()` via LD_PRELOAD -- the call to install the next handler confirms the current character is correct. See [patterns.md](patterns.md#signal-handler-chain-ldpreload-oracle-nuit-du-hack-2016).
### Font Ligature Exploitation
Custom OpenType font maps multi-character ligature sequences to single glyphs; reverse the GSUB table to decode hidden messages. See [patterns-ctf-3.md](patterns-ctf-3.md#opentype-font-ligature-exploitation-for-hidden-messages-hack-the-vote-2016).
### Instruction Counter as Cryptographic State
**Pattern:** Hand-written assembly uses a dedicated register (e.g., `r12`) as an instruction counter incremented after nearly every instruction. The counter feeds into XOR/ROL/multiply transformations on input bytes, making transformation path-dependent. Byte-by-byte brute force with Unicorn emulation recovers the flag. See [patterns-ctf-3.md](patterns-ctf-3.md#instruction-counter-as-cryptographic-state-metactf-flash-2026).
### Burrows-Wheeler Transform Inversion
Invert BWT without terminator character by trying all possible row indices. Standard `bwtool` or manual column-sorting reconstruction. See [patterns-ctf-3.md](patterns-ctf-3.md#burrows-wheeler-transform-inversion-without-terminator-asis-ctf-finals-2016).
### FRACTRAN Program Inversion
Esoteric language using iterated fraction multiplication. Invert by swapping numerator/denominator in fraction table, run output backward. I/O encoded as prime factorization exponents. See [languages.md](languages.md#fractran-program-inversion-boston-key-party-2016).
### Opcode-Only Trace Reconstruction
Execution traces with only opcodes (no data) still leak info through branch decisions. Sorting algorithm comparisons reveal element ordering. Reconstruct by deduplicating trace, splitting into basic blocks. See [tools-dynamic.md](tools-dynamic.md#opcode-only-trace-reconstruction-0ctf-2016).
### Thread Race Signed Integer Overflow
Combat-simulation binary with thread-unsafe skill lock. Race between skill selection and damage calculation; `cdqe` sign-extends 0xFFFFFFFF to -1 (signed), causing HP overflow on subtraction. See [patterns-ctf-3.md](patterns-ctf-3.md#thread-race-condition-with-signed-integer-overflow-codegate-2017).
### ESP32/Xtensa Firmware Reversing
No IDA support — use radare2 + ESP-IDF ROM linker script (`esp32.rom.ld`) for symbol resolution. Cross-reference with public ESP-IDF HTTP server examples to identify app logic. See [patterns-ctf-3.md](patterns-ctf-3.md#esp32xtensa-firmware-reversing-with-rom-symbol-map-insomnihack-2017).
### Custom VM Bytecode Lifting to LLVM IR
Transpile custom VM bytecode to LLVM IR, then use `opt -O3` to simplify (inlining, constant folding, dead code elimination). Reduces 1300 lines to ~150 lines, revealing the underlying algorithm. See [tools-advanced.md](tools-advanced.md#custom-vm-bytecode-lifting-to-llvm-ir-google-ctf-2017).
### SIGFPE Signal Handler Side-Channel
SIGFPE signal handlers create implicit control flow invisible to static analysis. Count SIGFPE signals via `strace -e signal=SIGFPE` per candidate character -- correct characters produce more signals. See [anti-analysis.md](anti-analysis.md#sigfpe-signal-handler-side-channel-via-strace-counting-plaidctf-2017).
### Batch Crackme Automation via objdump
Mass crackme challenges (100s of binaries) with identical structure: script `objdump` to extract CMP immediates and add/sub arithmetic sequences, then reverse-compute keys algebraically without execution. See [patterns-ctf-3.md](patterns-ctf-3.md#batch-crackme-automation-via-objdump-pattern-extraction-def-con-2017).
### Android DEX Runtime Bytecode Patching
Native JNI library patches Dalvik bytecode in memory via `/proc/self/maps` + `mprotect` + XOR. Static APK analysis alone is insufficient -- extract XOR key and offsets from the native `.so` to reconstruct the runtime DEX. See [languages-platforms.md](languages-platforms.md#android-dex-runtime-bytecode-patching-via-procselfmaps-google-ctf-2017).
### Fork + Pipe + Dead Branch Anti-Analysis
Fork/pipe IPC where parent writes data and exits, child reads and continues. Real validation hidden in a dead branch (always-false comparison). `strace` reveals the fork/pipe pattern; patch the comparison constant to reach hidden code. See [patterns-ctf-3.md](patterns-ctf-3.md#fork-pipe-dead-branch-anti-analysis-rctf-2017).
## Web/CTF Auth Bypass Case Notes
### Signed Cookie Key Reuse: access token to admin_session
**Case:** `class.pangbaoba.me` CTF homework system. Public `/access/<token>` route set a signed `student_gate`; the same access token also worked as the HMAC key for `admin_session`, allowing direct admin API access by forging the exact session payload shape.
**Core pattern:** A visible invite/access token is reused as a server-side signing secret. If one signed cookie can be validated offline, test whether sibling auth cookies use the same signing scheme and key.
**Triage workflow:**
1. Capture `Set-Cookie` from the gated entry route, especially cookies shaped like `<base64url-json>.<base64url-signature>`.
2. Decode the first segment; identify compact JSON payloads such as `{"access":"student"}`.
3. Recompute `HMAC-SHA256(payload_b64, candidate_key)` using visible route tokens, invite codes, reset tokens, or frontend constants as candidate keys.
4. If the signature matches, enumerate *payload shape*, not passwords: try likely authorization claims on the correct cookie name (`admin_session`, `session`, `auth`, etc.).
5. Verify with read-only endpoints first (`/api/admin/me`, settings/status/list routes) before any write action.
**Important lesson:** The first obvious payload may fail. In this case `{"access":"admin"}`, `{"role":"admin"}`, and `{"access":"student","isAdmin":true}` failed, while the backend actually checked:
```json
{"admin":true}
```
**Minimal PoC shape:**
```python
import base64, hashlib, hmac, json
def b64u(data: bytes) -> str:
return base64.urlsafe_b64encode(data).decode().rstrip("=")
access_token = "<token from /access/<token>>"
payload_b64 = b64u(json.dumps({"admin": True}, separators=(",", ":")).encode())
sig_b64 = b64u(hmac.new(access_token.encode(), payload_b64.encode(), hashlib.sha256).digest())
print(f"admin_session={payload_b64}.{sig_b64}")
```
**Validation signals:**
- `GET /api/admin/me` changes from `401 {"error":"unauthorized"}` to `200 {"admin":true}`.
- Other read-only admin endpoints return real data with the forged cookie.
- A JSON-cookie value like `admin_session=j:{}` causing `500` suggests Express/cookie-parser type confusion and confirms fragile cookie parsing; it is not required for the bypass but helps identify the stack and parsing assumptions.
**What to avoid:** Do not brute-force admin passwords or enumerate unrelated user IDs when a signed-cookie structure is visible. Work offline on signatures and use low-frequency read-only verification.
**Fix guidance:** Never use public route/access tokens as HMAC secrets. Use server-only cookie signing secrets, separate student/admin secrets, server-side sessions for admin identity, strict cookie type checks, and return `401` on parse/verify failure instead of `500`.
## Web Phishing Infrastructure
### Phishing Panel: {target_domain_a} / {target_domain_b}
Two-server phishing infrastructure impersonating a government agency. Full victim control system with server-driven status code redirection.
**Architecture:**
- `{target_domain_a}` — Presentation layer (phishing pages, JS polling client)
- `{target_domain_b}` — Data layer (PHP+MySQL backend, admin panel)
- Both behind NAT ({internal_ip} internal), nginx, SSL-only
- Web root: `/www/wwwroot/{target_domain_b}/`
**Victim Flow:** Landing page (fake subsidy quotas) → 1.html (ID/bank card form → `submit.php`) → 4.html (PIN → `get-ayment.php`) → server-controlled staged pages (9-16) via 1-second `status_check.php` polling.
**Key Findings:**
- Admin panel at `register.php` → `qichuang.php` (login form), `list.php` (dashboard template)
- Auth via PHP session (`PHPSESSID`); `login.php` and `check_login_ajax.php` removed (404)
- **Data leak**: `db.php` returns victim name list without auth (49+ records, **no bank details** — only id/username/note/description fields)
- **No-auth write**: `save_note.php` accepts data without authentication
- `backend.php` gives SQL error suggesting admin registration endpoint (broken)
- Rate limiting on `submit.php` (multi-factor), no SQLi or session bypass found
- Status code system: admin sets 1-16, victim browser auto-redirects to `N.html`
**Infrastructure:**
| Domain | Public IP | Role |
|--------|-----------|------|
| {target_domain_1} | {target_ip_1} | Backend + Admin |
| {target_domain_2} | {target_ip_2} | Frontend (phishing pages) |
---
## 分析前预判：文件伪装与名字欺骗
### 文件后缀不可信
**核心原则：永远用 `file` 命令或 magic bytes 判断文件类型，不要相信后缀名。**
常见伪装手法：
| 伪装后缀 | 实际类型 | 目的 |
|---------|---------|------|
| `.sh` | ELF 二进制 | 让人以为是脚本，降低警惕 |
| `.txt` | PE/ELF | 绕过简单的文件类型过滤 |
| `.jpg`/`.png` | 可执行文件或压缩包 | 隐藏在图片中 |
| `.dll` | 实际是 .NET assembly | 混淆分析方向 |
| `.so` | 实际是加密 payload | 需要先解密 |
| 无后缀 | 任何类型 | Linux 下常见 |
```bash
# 正确做法：用 file 命令
file suspicious_file.sh
# 输出: ELF 64-bit LSB executable, ARM aarch64...
# 用 xxd 看 magic bytes
xxd suspicious_file.sh | head -1
# 7f454c46 = ELF magic
```
### 文件名不可信
**"DriverLoader" 不一定加载驱动，"Updater" 不一定更新。**
常见名字欺骗：
| 文件名暗示 | 实际行为 |
|-----------|---------|
| `DriverLoader` | 可能是 ptrace 注入器 / 进程 hook |
| `SystemService` | 可能是后门 / C2 agent |
| `Updater` / `Update` | 可能是 dropper / 下载器 |
| `Helper` / `Assistant` | 可能是提权工具 |
| `lib*.so` | 可能是注入 payload |
**分析时应该：**
- 忽略文件名暗示，按实际代码行为判断
- 关注 `mmap`、`ptrace`、`/proc/self/mem` 等系统调用
- 如果看到"加载驱动"但没有 `insmod`/`init_module` 调用，说明名不副实
### 静态分析不够时的动态补充
纯静态分析只能看到代码骨架。以下场景必须配合动态分析：
| 场景 | 推荐动态方法 |
|------|-------------|
| 代码有解密/解压逻辑 | 在解密后下断点，dump 明文 |
| 大量间接调用（函数指针表） | strace/ltrace 跟踪实际调用 |
| 疑似反调试 | 先 strace 看 ptrace 调用 |
| 内嵌 shellcode/payload | QEMU 用户态模拟执行 |
| 网络通信协议未知 | tcpdump/Wireshark 抓包 |
```bash
# strace 跟踪系统调用（重点关注）
strace -f -e trace=open,mmap,ptrace,execve,connect ./binary
# ltrace 跟踪库函数调用
ltrace -f ./binary
# QEMU 用户态模拟（不需要真实设备）
qemu-aarch64 -strace ./binary_arm64
# 检查反调试：看是否 ptrace 自追踪
strace ./binary 2>&1 | grep ptrace
# 如果看到 ptrace(PTRACE_TRACEME, ...) 说明有反调试
```
### 进程注入/保护壳类样本的常见模式
这类样本（如 `LinYuDriverLoader`）通常：
1. **不是真正加载内核驱动**（需要 root 权限，大多数场景没有）
2. **实际行为是进程注入**：
- `ptrace` attach 到目标进程
- 通过 `/proc/<pid>/mem` 读写目标内存
- `mmap` 映射 shellcode 到目标进程空间
3. **内嵌加密 payload**：
- 运行时解密一段 shellcode
- 解密后的 payload 才是真正的 hook 代码
4. **反调试保护**：
- `ptrace(PTRACE_TRACEME)` 自追踪
- 时间检测（`clock_gettime` 前后对比）
- `/proc/self/status` 检查 TracerPid
**分析策略**：
```text
1. file 命令确认真实类型
2. strings 看有没有明显的路径/库名/错误信息
3. rabin2 -I 看架构/编译器/保护
4. 静态找 mmap/ptrace/open 调用
5. 如果有解密逻辑 → 动态跑到解密后 dump
6. 如果有反调试 → 先 patch 掉或用 LD_PRELOAD 绕过
```

240
reverse-engineering/go-reverse.md Normal file
View File

@@ -0,0 +1,240 @@
# Go 二进制逆向指南
> Go 编译的二进制有独特的挑战：静态链接导致体积巨大、函数数量上万、字符串格式特殊、符号 strip 后恢复困难。
> 本文档覆盖工具链、恢复技巧和实战工作流。
---
## Go 二进制的特征识别
快速判断一个二进制是否是 Go 编译的：
```bash
# 字符串特征
strings binary | grep -E "runtime\.|go\.buildid|GOROOT"
# rabin2 侦察
rabin2 -z binary | grep -i "runtime"
# 文件大小异常大（静态链接 runtime）
# 典型 Hello World: C ~20KB, Go ~2MB
```
常见特征：
- 包含 `runtime.` 前缀的大量函数
- 包含 `go.buildid` section
- 包含 `GOROOT`、`GOPATH` 路径字符串
- 函数数量 5000-50000+（包含整个 runtime 和标准库）
---
## 核心工具链
### 符号恢复
| 工具 | 用途 | 链接 |
|------|------|------|
| **GoReSym** | Mandiant 出品，解析 Go 符号信息（pclntab/moduledata） | https://github.com/mandiant/GoReSym |
| **GoResolver** | Volexity 出品，用 CFG 相似度自动去混淆 Garble 二进制 | https://github.com/volexity/GoResolver |
| **redress** | 分析 stripped Go 二进制，恢复类型/接口/包结构 | https://github.com/goretk/redress |
| **GoStringUngarbler** | Google 出品，专门恢复 Garble 混淆的字符串 | https://github.com/mandiant/GoStringUngarbler |
### IDA 插件
| 工具 | 用途 | 链接 |
|------|------|------|
| **go_parser** | IDA 插件，解析 moduledata/pclntab/类型信息 | https://github.com/0xjiayu/go_parser |
| **IDAGolangHelper** | IDA 脚本集，解析 Go 类型信息 | https://github.com/sibears/IDAGolangHelper |
| **AlphaGolang** | SentinelLabs 的 IDAPython 脚本集 | https://github.com/SentineLabs/AlphaGolang |
| **IDA 9.2+ 原生支持** | Hex-Rays 官方 Go 反编译改进 | https://hex-rays.com/blog/stop-guessing-and-start-going |
### Ghidra 插件
| 工具 | 用途 | 链接 |
|------|------|------|
| **Ghidra + GoReSym 输出** | 用 GoReSym 导出符号后导入 Ghidra | 配合使用 |
| **golang_loader_assist** | Ghidra Go 加载辅助 | 社区脚本 |
### 独立分析工具
| 工具 | 用途 | 链接 |
|------|------|------|
| **gore** | Go 逆向工程库（redress 的底层） | https://github.com/goretk/gore |
| **garble** | Go 混淆工具（了解它才能对抗它） | https://github.com/burrowers/garble |
---
## Go 二进制的关键结构
### pclntab (PC Line Table)
Go 二进制中最重要的结构，包含：
- 所有函数名和地址映射
- 源文件路径
- 行号信息
- 栈帧大小
即使 strip 了符号，pclntab 通常仍然存在（Go runtime 依赖它）。
```text
定位方法：
1. 搜索 magic bytes: 0xFFFFFFF0 (Go 1.16+) 或 0xFFFFFFFB (Go 1.18+)
2. 用 GoReSym 自动定位
3. 用 go_parser IDA 插件自动解析
```
### moduledata
包含：
- pclntab 指针
- 类型信息表
- itab（接口表）
- 全局变量信息
### 字符串格式
Go 字符串不是 C 风格的 null-terminated，而是 `(pointer, length)` 结构：
```text
C 字符串: "hello\0"
Go 字符串: struct { ptr *byte; len int } → ptr 指向 "hello"（无 \0）
```
这导致 IDA/Ghidra 默认的字符串识别会漏掉大量 Go 字符串。
**解决方案**：
- 用 `go_parser` 自动识别 Go 字符串
- 用 GoReSym 导出字符串列表
- 手动：找到 `runtime.stringtable` 或通过交叉引用定位
---
## 实战工作流
### 场景 1：未 strip 的 Go 二进制
```text
1. GoReSym -t -d -p binary > symbols.json
→ 导出所有函数名、类型、源文件路径
2. 加载到 IDA/Ghidra
3. 导入 GoReSym 的符号信息
4. 过滤掉 runtime.* 和标准库函数，聚焦用户代码
5. 从 main.main 开始分析
```
### 场景 2：strip 后的 Go 二进制
```text
1. GoReSym -t -d -p binary > symbols.json
→ 即使 strip 了，pclntab 通常还在
2. 如果 GoReSym 失败 → 用 redress
redress -src binary # 恢复源文件路径
redress -pkg binary # 恢复包结构
redress -type binary # 恢复类型信息
3. 加载到 IDA + go_parser 插件
4. 运行 go_parser 自动恢复
5. 从恢复的 main.main 开始
```
### 场景 3：Garble 混淆的 Go 二进制
```text
Garble 会：
- 随机化函数名（main.main → main.a3f2b1c）
- 加密字符串
- 移除文件路径信息
- 混淆包名
对抗方法：
1. GoResolver（CFG 签名匹配）
→ 通过控制流图相似度恢复标准库函数名
2. GoStringUngarbler（字符串解密）
→ 自动识别 Garble 的字符串加密模式并解密
3. 动态分析（Frida/dlv）
→ Hook runtime 函数观察实际行为
4. 对比分析
→ 编译同版本 Go 的 Hello World，用 binary-diff 对比 runtime 部分
```
### 场景 4：CGo 混合编译
```text
1. 识别 CGo 边界（_cgo_* 函数）
2. Go 部分用 go_parser 恢复
3. C 部分用常规 IDA 分析
4. 关注 _cgo_topofstack、crosscall2 等桥接函数
```
---
## 常用命令速查
```bash
# GoReSym：导出符号
GoReSym -t -d -p binary > symbols.json
GoReSym -t -d -p binary -o ida_script.py # 生成 IDA 脚本
# redress：分析 stripped 二进制
redress -src binary # 源文件路径
redress -pkg binary # 包结构
redress -type binary # 类型信息
redress -interface binary # 接口信息
redress -filepath binary # 完整文件路径
# GoResolver：去混淆 Garble
GoResolver -binary binary -output resolved.json
# GoStringUngarbler：解密 Garble 字符串
GoStringUngarbler -i binary -o deobfuscated_binary
# 快速判断 Go 版本
strings binary | grep "go1\."
GoReSym -p binary | grep "Version"
```
---
## IDA 中的 Go 分析流程
```text
1. 加载二进制（选择正确的架构）
2. 等待自动分析完成
3. 运行 go_parser 插件：
- File → Script File → go_parser.py
- 或 Edit → Plugins → Go Parser
4. 插件会自动：
- 解析 pclntab
- 恢复函数名
- 标记 Go 字符串
- 解析类型信息
5. 过滤视图：
- 隐藏 runtime.* 函数
- 聚焦 main.* 和第三方包
6. 从 main.main 开始逆向
```
---
## 常见陷阱
| 陷阱 | 说明 | 解决 |
|------|------|------|
| 函数太多看不过来 | Go 静态链接导致 5000-50000 函数 | 用包名过滤，只看 main.* 和业务包 |
| 字符串识别不全 | Go 字符串不是 null-terminated | 用 go_parser 或 GoReSym 恢复 |
| 反编译结果难读 | Go 的 defer/goroutine/interface 让伪代码复杂 | IDA 9.2+ 有改进，或用动态分析辅助 |
| Garble 混淆 | 函数名/字符串全部随机化 | GoResolver + GoStringUngarbler |
| 版本差异 | 不同 Go 版本的 pclntab 格式不同 | GoReSym 支持 Go 1.2-1.23+ |
| CGo 边界 | Go 和 C 代码混合 | 识别 _cgo_* 函数作为分界线 |
---
## 与其他 skill 的配合
| 需求 | 用什么 |
|------|--------|
| IDA 深度分析 Go 二进制 | `ida-reverse/` + go_parser 插件 |
| Ghidra 分析（免费） | Ghidra + GoReSym 符号导入 |
| 快速侦察 | `radare2/` — `rabin2 -z` 看字符串 |
| 动态 Hook | Frida（Hook runtime 函数）或 dlv（Go 原生调试器） |
| 跨版本对比 | `binary-diff/` — 旧版有符号迁移到新版 |
| Garble 去混淆 | GoResolver + GoStringUngarbler |

298
reverse-engineering/kernel-driver-reverse.md Normal file
View File

@@ -0,0 +1,298 @@
# 内核驱动逆向参考
> 覆盖 Windows/Linux 内核驱动逆向、Rootkit 分析、C/C++ 二进制模式识别。
---
## Windows 驱动逆向
### 驱动类型
| 类型 | 特征 | 分析重点 |
|------|------|---------|
| WDM (Windows Driver Model) | 老式驱动，手动管理 IRP | DriverEntry → 设备创建 → Dispatch 例程 |
| KMDF (Kernel Mode Driver Framework) | 现代框架，事件驱动 | EvtDriverDeviceAdd → Queue → I/O 回调 |
| WDF (Windows Driver Foundation) | KMDF + UMDF 统称 | 看 WdfDriverCreate 调用 |
| Minifilter | 文件系统过滤驱动 | FltRegisterFilter → Pre/Post 回调 |
### WDM 驱动分析流程
```text
1. 找 DriverEntry（入口点）
- IDA 自动识别，或搜索 IoCreateDevice / IoCreateSymbolicLink
2. 找设备名和符号链接
- IoCreateDevice → DeviceName（如 \Device\MyDriver）
- IoCreateSymbolicLink → SymLink（如 \DosDevices\MyDriver）
3. 找 Dispatch 例程
- DriverObject->MajorFunction[IRP_MJ_DEVICE_CONTROL] = DispatchIoctl
- 这是用户态通过 DeviceIoControl 调用的入口
4. 分析 IOCTL 处理
- switch(IoControlCode) 分发不同功能
- IOCTL 编码：CTL_CODE(DeviceType, Function, Method, Access)
- Method: METHOD_BUFFERED / METHOD_IN_DIRECT / METHOD_OUT_DIRECT / METHOD_NEITHER
5. 找漏洞
- 用户可控缓冲区未验证长度 → 溢出
- METHOD_NEITHER 直接使用用户指针 → 任意读写
- 未检查 IOCTL 权限 → 非特权用户可调用
```
### IOCTL 编码解析
```python
# 解析 IOCTL code
def decode_ioctl(code):
device_type = (code >> 16) & 0xFFFF
access = (code >> 14) & 0x3
function = (code >> 2) & 0xFFF
method = code & 0x3
methods = {0: "BUFFERED", 1: "IN_DIRECT", 2: "OUT_DIRECT", 3: "NEITHER"}
access_types = {0: "ANY", 1: "READ", 2: "WRITE", 3: "READ|WRITE"}
return f"DevType=0x{device_type:X} Func=0x{function:X} Method={methods[method]} Access={access_types[access]}"
# 示例
decode_ioctl(0x80002034)
# DevType=0x8000 Func=0x80D Method=BUFFERED Access=ANY
```
### IDA 插件
| 插件 | 用途 | 链接 |
|------|------|------|
| **Driver Buddy Reloaded** | 自动识别 IOCTL、Dispatch、设备名 | https://github.com/VoidSec/DriverBuddyReloaded |
| **WinDbg + IDA** | 内核调试 + 静态分析配合 | 内置 |
| **FLIRT/Lumina** | 识别 WDK 库函数 | IDA 内置 |
### 参考文章
- [Windows Drivers RE Methodology (VoidSec)](https://voidsec.com/windows-drivers-reverse-engineering-methodology/) — 最完整的 WDM 驱动逆向方法论
- [Driver Reversing 101](https://eversinc33.com/posts/driver-reversing.html) — WDM vs KMDF 对比
- [Methodology of Reversing Vulnerable Killer Drivers](https://whiteknightlabs.com/2025/10/28/methodology-of-reversing-vulnerable-killer-drivers/) — 漏洞驱动分析
---
## Linux 内核模块逆向
### LKM (Loadable Kernel Module) 结构
```text
关键函数：
- init_module / module_init → 模块加载时执行
- cleanup_module / module_exit → 模块卸载时执行
关键结构：
- struct file_operations → 字符设备的 open/read/write/ioctl
- struct net_device_ops → 网络设备操作
- struct block_device_operations → 块设备操作
```
### 分析流程
```text
1. 确认是内核模块
file module.ko → "ELF 64-bit ... relocatable"（注意是 relocatable 不是 executable）
2. 找 init/exit 函数
readelf -s module.ko | grep -E "init_module|cleanup_module"
或在 .modinfo section 找模块信息
3. 找 file_operations 结构
搜索 register_chrdev / cdev_add / misc_register
→ 找到 fops 结构体 → 定位 ioctl/read/write 处理函数
4. 分析 ioctl 处理
unlocked_ioctl / compat_ioctl 函数
→ switch(cmd) 分发
5. 找 Rootkit 行为
- 修改 sys_call_table → syscall hook
- 修改 /proc 文件系统 → 隐藏进程/文件
- 注册 netfilter hook → 隐藏网络连接
- 修改 VFS 层 → 隐藏文件
```
### Rootkit 常见技术
| 技术 | 特征 | 检测方法 |
|------|------|---------|
| syscall table hook | 修改 `sys_call_table` 条目 | 对比内存中的表与磁盘上的 vmlinux |
| VFS hook | 修改 `file_operations` 函数指针 | 检查 fops 指针是否指向内核代码段外 |
| Netfilter hook | `nf_register_net_hook` | 遍历 netfilter hook 链表 |
| kprobe/ftrace hook | 注册 kprobe 或 ftrace 回调 | 检查 ftrace 注册列表 |
| eBPF rootkit | 加载恶意 BPF 程序 | `bpftool prog list` |
| DKOM | 直接修改内核对象（进程链表） | 遍历 task_struct 链表对比 /proc |
### 工具
| 工具 | 用途 |
|------|------|
| `crash` | 内核 dump 分析 |
| `volatility3` | 内存取证（Linux profile） |
| `dmesg` / `journalctl` | 内核日志 |
| `lsmod` / `/proc/modules` | 已加载模块列表 |
| `modinfo` | 模块元信息 |
| `strace` | 系统调用跟踪（用户态视角） |
---
## C/C++ 逆向模式识别
### C 语言常见模式
| 源码模式 | 反汇编特征 |
|---------|-----------|
| `if-else` | `cmp` + `jcc`（条件跳转） |
| `switch-case` | 跳转表（`jmp [rax*8 + table]`）或连续 `cmp` |
| `for` 循环 | `cmp` + `jl/jle` + 循环体 + `inc/add` + `jmp` 回跳 |
| `while` 循环 | 条件判断在循环顶部 |
| `do-while` | 条件判断在循环底部 |
| 函数指针调用 | `call rax` 或 `call [reg+offset]` |
| `struct` 访问 | `[reg+固定偏移]`（如 `[rdi+0x10]`） |
| `malloc` + 使用 | `call malloc` → 返回值存入寄存器 → 后续用该寄存器+偏移访问 |
| 字符串比较 | `call strcmp` 或 `repe cmpsb` |
### C++ 特有模式
| 源码模式 | 反汇编特征 |
|---------|-----------|
| **虚函数调用** | `mov rax, [rcx]`（取 vtable）→ `call [rax+offset]`（调用虚函数） |
| **构造函数** | 分配内存 → 写入 vtable 指针 → 初始化成员 |
| **析构函数** | 清理成员 → 可能调用 `operator delete` |
| **this 指针** | 第一个参数（rcx/rdi）是对象指针 |
| **继承** | vtable 中包含父类虚函数 + 子类覆盖 |
| **多重继承** | 对象内有多个 vtable 指针（偏移不同） |
| **RTTI** | vtable 前面有 `type_info` 指针 |
| **异常处理** | `__cxa_throw` / `_CxxThrowException` |
| **STL 容器** | `std::vector`: `{begin, end, capacity}` 三指针结构 |
| **std::string** | 小字符串优化（SSO）：短串内联，长串堆分配 |
### vtable 逆向方法
```text
1. 找 vtable
- 搜索连续的函数指针数组（在 .rodata 或 .rdata 段）
- 构造函数中 `mov [rcx], offset vtable` 写入 vtable 指针
2. 确定类层次
- vtable 前 -8 偏移处通常是 RTTI 指针（如果未 strip）
- 多个 vtable 共享前几个条目 → 继承关系
3. 标注虚函数
- vtable[0] 通常是析构函数（或 deleting destructor）
- 后续按偏移标注：vtable[1] = func1, vtable[2] = func2...
4. IDA 中操作
- 在 vtable 地址创建 struct（每个字段是函数指针）
- 对 `call [rax+offset]` 添加注释标明调用的虚函数
```
### 结构体恢复
```text
方法 1：从访问模式推断
mov eax, [rdi+0x00] → field_0: int/ptr (4/8 bytes)
mov ecx, [rdi+0x08] → field_8: int/ptr
movss xmm0, [rdi+0x10] → field_10: float
方法 2：从 sizeof 推断
call malloc(0x30) → 结构体大小 0x30 (48 bytes)
方法 3：从构造函数推断
构造函数会初始化所有字段 → 字段类型和偏移一目了然
方法 4：用 IDA 的 "Create struct" 功能
选中访问模式 → Edit → Struct → Create struct from selection
```
---
## 常见编译器特征
| 编译器 | 识别特征 |
|--------|---------|
| MSVC | `_security_cookie` 检查、`__fastcall` 调用约定、Rich Header |
| GCC | `__stack_chk_fail`、`-fstack-protector`、`.note.GNU-stack` |
| Clang/LLVM | 类似 GCC 但优化模式不同、`__asan_*`（如果开了 sanitizer） |
| MinGW | GCC 特征 + Windows API 调用 |
| AOSP Clang | Android 特有的 `__android_log_print`、PGO 标记 |
### 优化级别识别
| 优化级别 | 特征 |
|---------|------|
| -O0 | 大量冗余 mov、每个变量都在栈上、函数不内联 |
| -O1 | 基本优化、部分变量在寄存器 |
| -O2 | 循环展开、函数内联、尾调用优化 |
| -O3 / -Os | 激进内联、向量化（SIMD）、代码难读 |
| PGO | 热路径优化、冷代码分离到 `.text.cold` |
| LTO | 跨模块内联、全局死代码消除 |
---
## 内核调试环境
### Windows
```text
调试器：WinDbg Preview
连接方式：网络调试（推荐）或串口
被调试机设置：
bcdedit /debug on
bcdedit /dbgsettings net hostip:192.168.x.x port:50000
调试机连接：
WinDbg → File → Attach to Kernel → Net → Port:50000 Key:xxx
常用命令：
!analyze -v # 自动分析崩溃
lm # 列出已加载模块
!drvobj \Driver\xxx # 查看驱动对象
dt nt!_DRIVER_OBJECT # 显示结构体
bp module!function # 下断点
```
### Linux
```text
调试器：GDB + QEMU 或 kgdb
QEMU 内核调试：
qemu-system-x86_64 -kernel bzImage -s -S ...
gdb vmlinux -ex "target remote :1234"
常用命令：
info threads # 内核线程
lx-symbols # 加载内核符号（需要 scripts/gdb/）
p init_task # 查看 init 进程
lx-dmesg # 内核日志
```
---
## Agent 动作锚点（Issue #65 U–AV）
与 `references/nonpe-format-cookbook.md` §5 对齐（短表，不替代上文流程）：
| ID | 动作 | Evidence |
|----|------|----------|
| AG | `DriverEntry` 短 → 扫 `MajorFunction` 非空槽，优先 DEVICE_CONTROL/CREATE | `E-driver-irp-handlers` |
| AH | 建立 IOCTL 控制码 → handler 表与 METHOD_* | `E-driver-ioctl` |
| AI | 疑 BYOVD：对照公开脆弱驱动列表；记名/哈希/签名与调用意图；**不写 exploit 步骤** | `E-driver-byovd` |
## 参考资源
| 资源 | 说明 | 链接 |
|------|------|------|
| VoidSec 驱动逆向方法论 | Windows WDM 驱动完整分析流程 | https://voidsec.com/windows-drivers-reverse-engineering-methodology/ |
| Elastic Rootkit 系列 | Linux Rootkit 分类+检测 | https://security-labs.elastic.co/security-labs/linux-rootkits-1-hooked-on-linux |
| Driver Buddy Reloaded | IDA 驱动分析插件 | https://github.com/VoidSec/DriverBuddyReloaded |
| LOLDrivers | 已知漏洞驱动列表 | https://www.loldrivers.io/ |
| Windows Driver Samples | 微软官方驱动示例 | https://github.com/microsoft/Windows-driver-samples |
| Linux Kernel Module Programming | 内核模块开发教程 | https://sysprog21.github.io/lkmpg/ |
| Trail of Bits - Devirtualizing C++ | vtable 逆向方法 | https://blog.trailofbits.com/2017/02/13/devirtualizing-c-with-binary-ninja/ |

587
reverse-engineering/languages-compiled.md Normal file
View File

@@ -0,0 +1,587 @@
# CTF Reverse - Compiled Language Reversing (Go, Rust)
## Table of Contents
- [Go Binary Reversing](#go-binary-reversing)
- [Recognition](#recognition)
- [Symbol Recovery](#symbol-recovery)
- [Go Memory Layout](#go-memory-layout)
- [Goroutine and Concurrency Analysis](#goroutine-and-concurrency-analysis)
- [Common Go Patterns in Decompilation](#common-go-patterns-in-decompilation)
- [Go Binary Reversing Workflow](#go-binary-reversing-workflow)
- [Go Binary UUID Patching for C2 Client Enumeration (BSidesSF 2026)](#go-binary-uuid-patching-for-c2-client-enumeration-bsidessf-2026)
- [Rust Binary Reversing](#rust-binary-reversing)
- [Rust Recognition](#rust-recognition)
- [Symbol Demangling](#symbol-demangling)
- [Common Rust Patterns in Decompilation](#common-rust-patterns-in-decompilation)
- [Rust-Specific Analysis Tools](#rust-specific-analysis-tools)
- [Swift Binary Reversing](#swift-binary-reversing)
- [Kotlin / JVM Binary Reversing](#kotlin--jvm-binary-reversing)
- [JVM Bytecode (Android/Server)](#jvm-bytecode-androidserver)
- [Kotlin/Native](#kotlinnative)
- [D Language Binary Reversing (CSAW CTF 2016)](#d-language-binary-reversing-csaw-ctf-2016)
- [Haskell Binary Reversing via STG Closures and hsdecomp (hxp CTF 2017, Codegate 2018)](#haskell-binary-reversing-via-stg-closures-and-hsdecomp-hxp-ctf-2017-codegate-2018)
- [Haskell Binary RE via GHC CMM Intermediate Language (N1CTF 2018)](#haskell-binary-re-via-ghc-cmm-intermediate-language-n1ctf-2018)
- [.NET / C# Binary Reversing](#net--c-binary-reversing)
- [C++ Binary Reversing (Quick Reference)](#c-binary-reversing-quick-reference)
- [vtable Reconstruction](#vtable-reconstruction)
- [RTTI (Run-Time Type Information)](#rtti-run-time-type-information)
- [Standard Library Patterns](#standard-library-patterns)
---
## Go Binary Reversing
Go binaries are increasingly common in CTF challenges due to Go's popularity for CLI tools, network services, and malware.
### Recognition
```bash
# Detect Go binary
file binary | grep -i "go"
strings binary | grep "go.buildid"
strings binary | grep "runtime.gopanic"
# Go version embedded in binary
strings binary | grep "^go1\."
```
**Key indicators:**
- Very large static binary (even "hello world" is ~2MB)
- Embedded `go.buildid` string
- `runtime.*` symbols (even in stripped binaries, some remain)
- `main.main` as entry point (not `main`)
- Strings like `GOROOT`, `GOPATH`, `/usr/local/go/src/`
### Symbol Recovery
Go embeds rich type and function information even in stripped binaries:
```bash
# GoReSym - recovers function names, types, interfaces from Go binaries
# https://github.com/mandiant/GoReSym
./GoReSym -d binary > symbols.json
# Parse output
python3 -c "
import json
with open('symbols.json') as f:
data = json.load(f)
for fn in data.get('UserFunctions', []):
print(f\"{fn['Start']:#x} {fn['FullName']}\")
"
```
**Ghidra with golang-loader:**
```bash
# Install: Ghidra → Window → Script Manager → search "golang"
# Or use: https://github.com/getCUJO/ThreatFox/tree/main/ghidra-golang
# Recovers function names, string references, interface tables
```
**redress (Go binary analysis):**
```bash
# https://github.com/goretk/redress
redress -src binary # Reconstruct source tree
redress -pkg binary # List packages
redress -type binary # List types and methods
redress -interface binary # List interfaces
```
### Go Memory Layout
Understanding Go's data structures in decompilation:
```c
# String: {pointer, length} (16 bytes on 64-bit)
# NOT null-terminated! Length field is critical.
struct GoString {
char *ptr; // pointer to UTF-8 data
int64 len; // byte length
};
# Slice: {pointer, length, capacity} (24 bytes on 64-bit)
struct GoSlice {
void *ptr; // pointer to backing array
int64 len; // current length
int64 cap; // allocated capacity
};
# Interface: {type_descriptor, data_pointer} (16 bytes)
struct GoInterface {
void *type; // points to type metadata (itab for non-empty interface)
void *data; // points to actual value
};
# Map: pointer to runtime.hmap struct
# Channel: pointer to runtime.hchan struct
```
**In Ghidra/IDA:** When you see a function taking `(ptr, int64)` — it's likely a Go string. Three-field `(ptr, int64, int64)` is a slice.
### Goroutine and Concurrency Analysis
```bash
# Identify goroutine spawns in disassembly
strings binary | grep "runtime.newproc"
# newproc1 is the internal goroutine creation function
# In GDB with Go support:
gdb ./binary
(gdb) source /usr/local/go/src/runtime/runtime-gdb.py
(gdb) info goroutines # List all goroutines
(gdb) goroutine 1 bt # Backtrace for goroutine 1
```
**Channel operations in disassembly:**
- `runtime.chansend1` → `ch <- value`
- `runtime.chanrecv1` → `value = <-ch`
- `runtime.selectgo` → `select { case ... }`
- `runtime.closechan` → `close(ch)`
### Common Go Patterns in Decompilation
**Defer mechanism:**
- `runtime.deferproc` → registers deferred function
- `runtime.deferreturn` → executes deferred functions at function exit
- Deferred calls execute in LIFO order — relevant for cleanup/crypto key wiping
**Error handling (the `if err != nil` pattern):**
```text
# In disassembly, this appears as:
# call some_function → returns (result, error) as two values
# test rax, rax → check if error (second return value) is nil
# jne error_handler
```
**String concatenation:**
- `runtime.concatstrings` → `s1 + s2 + s3`
- `fmt.Sprintf` → formatted string building
- Look for format strings in `.rodata`: `"%s%d"`, `"%x"`
**Common stdlib patterns in CTF:**
```go
// Crypto operations → look for these in strings/imports:
// "crypto/aes", "crypto/cipher", "crypto/sha256", "encoding/hex", "encoding/base64"
// Network operations:
// "net/http", "net.Dial", "bufio.NewReader"
// File operations:
// "os.Open", "io.ReadAll", "os.ReadFile"
```
### Go Binary Reversing Workflow
```bash
1. file binary # Confirm Go, get arch
2. GoReSym -d binary > syms.json # Recover symbols
3. strings binary | grep -i flag # Quick win check
4. Load in Ghidra with golang-loader # Apply recovered symbols
5. Find main.main # Entry point
6. Identify string comparisons # GoString {ptr, len} pairs
7. Trace crypto operations # crypto/* package usage
8. Check for embedded resources # embed.FS in Go 1.16+
```
**Go embed.FS (Go 1.16+):** Binaries can embed files at compile time:
```bash
# Look for embedded file data
strings binary | grep "embed"
# Embedded files appear as raw data in the binary
# Search for known file signatures (PK for zip, PNG header, etc.)
```
**Key insight:** Go's runtime embeds extensive metadata even in stripped binaries. Use GoReSym before any manual analysis — it often recovers 90%+ of function names, making decompilation dramatically easier. Go strings are `{ptr, len}` tuples, not null-terminated — Ghidra's default string analysis will miss them without the golang-loader plugin.
**Detection:** Large static binary (2MB+ for simple programs), `go.buildid`, `runtime.gopanic`, source paths like `/home/user/go/src/`.
### Go Binary UUID Patching for C2 Client Enumeration (BSidesSF 2026)
**Pattern (see-two):** A Go-compiled C2 client has a UUID embedded via `-ldflags -X`. The C2 server uses mTLS for authentication. To enumerate other clients and their files, patch the UUID to register as a new client, then use the C2 API to list all clients and download their exfiltrated files.
**Approach:**
1. Extract embedded UUID from Go build metadata: `go version -m client_binary`
2. Binary-patch the UUID (simple byte replacement — Go strings have fixed-length backing arrays)
3. Register with the C2 server using the patched binary (mTLS certs are embedded or in distfiles)
4. Enumerate clients via API: `GET /api/clients` or iterate known endpoints
5. List and download files from each client's GCS bucket or file store
6. Grep downloaded files for the flag
```bash
# Extract Go build info
go version -m ./client_binary | grep ldflags
# Output shows: -X main.clientUUID=<uuid>
# Patch UUID in binary (replace old UUID bytes with new UUID)
python3 -c "
import sys
data = open('client_binary', 'rb').read()
old_uuid = b'original-uuid-value-here'
new_uuid = b'attacker-uuid-value-here'
patched = data.replace(old_uuid, new_uuid)
open('client_patched', 'wb').write(patched)
"
chmod +x client_patched
./client_patched --register
```
**Key insight:** Go binaries embed string values from `-ldflags -X` directly in the binary data section. Since Go strings are `{ptr, len}` pairs pointing to backing byte arrays, replacing the UUID bytes (same length) produces a valid patched binary. The mTLS certificates authenticate the client to the server but don't bind to a specific UUID.
**References:** BSidesSF 2026 "see-two"
---
## Rust Binary Reversing
Rust binaries are common in modern CTFs, especially for crypto, systems, and security tooling challenges.
### Rust Recognition
```bash
# Detect Rust binary
strings binary | grep -c "rust"
strings binary | grep "rustc" # Compiler version
strings binary | grep "/rustc/" # Source paths
strings binary | grep "core::panicking" # Panic infrastructure
```
**Key indicators:**
- `core::panicking::panic` in strings
- Mangled symbols starting with `_ZN` (Itanium ABI) — e.g., `_ZN4main4main17h...`
- `.rustc` section in ELF
- References to `/rustc/<commit_hash>/library/`
- Large binary size (Rust statically links by default)
### Symbol Demangling
```bash
# Rust uses Itanium ABI mangling (same as C++)
# rustfilt demangles Rust-specific symbols
cargo install rustfilt
nm binary | rustfilt | grep "main"
# Or use c++filt (works for most Rust symbols)
nm binary | c++filt | grep "main"
# In Ghidra: Window → Script Manager → search "Demangler"
# Enable "DemangleAllScript" for automatic demangling
```
### Common Rust Patterns in Decompilation
**Option/Result enum:**
```text
# Option<T> in memory: {discriminant (0=None, 1=Some), value}
# Result<T, E>: {discriminant (0=Ok, 1=Err), union{ok_val, err_val}}
# In disassembly:
# cmp byte [rbp-0x10], 0 → check if None/Err
# je handle_none_case
```
**Vec<T> (same as Go slice):**
```c
struct RustVec {
void *ptr; // heap pointer
uint64 cap; // capacity
uint64 len; // length
};
```
**String / &str:**
```text
# String (owned): {ptr, capacity, length} — 24 bytes, heap-allocated
# &str (borrowed): {ptr, length} — 16 bytes, can point anywhere
# In decompilation, look for:
# alloc::string::String::from → String creation
# core::str::from_utf8 → byte slice to str
```
**Iterator chains:**
```text
# .iter().map().filter().collect() compiles to loop fusion
# In disassembly: tight loop with inlined closures
# Look for: core::iter::adapters::map, filter, etc.
```
**Panic unwinding:**
```bash
# Panic strings reveal source locations and error messages
strings binary | grep "panicked at"
strings binary | grep "called .unwrap().. on"
# These often contain file paths, line numbers, and variable names
```
### Rust-Specific Analysis Tools
```bash
# cargo-bloat: analyze binary size by function
cargo install cargo-bloat
cargo bloat --release -n 50
# Ghidra Rust helper scripts
# https://github.com/AmateursCTF/ghidra-rust (community scripts for Rust RE)
```
**Key insight:** Rust panic messages are goldmines — they contain source file paths, line numbers, and descriptive error strings even in release builds. Always `strings binary | grep "panicked"` first. Rust's monomorphization means generic functions get duplicated per type — expect many similar-looking functions.
**Detection:** `core::panicking`, `.rustc` section, `/rustc/` paths, `_ZN` mangled symbols with Rust-style module paths.
---
## Swift Binary Reversing
See [platforms.md](platforms.md#swift-binary-reversing) for full Swift reversing guide including demangling, runtime structures, and Ghidra integration. Key quick reference:
```bash
# Detect Swift binary
strings binary | grep "swift"
otool -l binary | grep "swift"
# Demangle Swift symbols
swift demangle 's14MyApp0A8ClassC10checkInput6resultSbSS_tF'
# → MyApp.MyAppClass.checkInput(result: String) -> Bool
# Key runtime functions: swift_allocObject, swift_release, swift_once
# String: small (≤15 bytes inline) or large (heap pointer + length)
# Protocol witness tables = dynamic dispatch (like vtables)
```
**Detection:** `__swift5_*` sections in Mach-O, `swift_` runtime symbols, `s` prefix in mangled names.
---
## Kotlin / JVM Binary Reversing
Kotlin compiles to JVM bytecode or native (via Kotlin/Native). Common in Android and server-side CTF.
### JVM Bytecode (Android/Server)
```bash
# Detect Kotlin
strings classes.dex | grep "kotlin"
# Look for: kotlin.Metadata annotation, kotlin/jvm/internal/*
# Decompile
jadx classes.dex # Best for Kotlin bytecode
cfr classes.jar --kotlin # CFR with Kotlin mode
fernflower classes.jar output/ # IntelliJ's decompiler
# Kotlin-specific patterns in decompiled output:
# - Companion objects: ClassName$Companion
# - Data classes: copy(), component1(), component2(), toString()
# - Coroutines: ContinuationImpl, invokeSuspend, state machine
# - Null checks: Intrinsics.checkNotNull() everywhere
# - When expression: compiled as tableswitch/lookupswitch
# - Sealed classes: instanceof checks in chain
```
**Kotlin coroutines in disassembly:**
```text
# Coroutines compile to state machines:
# invokeSuspend(result) {
# switch (this.label) {
# case 0: this.label = 1; return suspendFunction();
# case 1: processResult(result); return Unit;
# }
# }
# Each suspend point becomes a state in the switch.
# Follow the state machine to understand async flow.
```
### Kotlin/Native
```bash
# Kotlin/Native produces platform binaries (no JVM)
# Recognize by: konan, kotlin.native strings
strings binary | grep "konan"
# Much harder to reverse — no reflection metadata
# Uses LLVM backend, looks similar to C/C++ in disassembly
# Key functions: InitRuntime, DeinitRuntime, CreateStablePointer
# Memory management: automatic reference counting (not GC)
```
**Detection:** `kotlin.Metadata` annotations (JVM), `konan` strings (Native), `kotlin/` package paths.
---
## D Language Binary Reversing (CSAW CTF 2016)
D language binaries have unique symbol mangling different from C++. Template instantiation at compile-time produces many function variants.
```bash
# Recognition: D binaries use different mangling than C++
# Symbols contain "_D" prefix and numeric length-prefixed names
# Example: _D4mainQaFNaNbNfZv
# Symbol demangling:
# GDB: set language d
# Radare2: export names show demangled D symbols
# Online: dlang.org/phobos/core_demangle.html
# Common D binary patterns:
# - Templates instantiated at compile-time: enc!("111"), enc!("222"), ...
# - Garbage collector references (GC.malloc, GC.free)
# - Phobos standard library functions (_D3std...)
# - String processing: std.string, std.conv.to
# Reversing a D cipher (XOR with cycling key):
def reverse_d_cipher(encrypted, num_functions=500):
"""D binaries may chain multiple transformation functions.
Each function XORs with key character, then XORs with key length.
Process in reverse order."""
result = encrypted[:]
for i in range(num_functions - 1, -1, -1):
key = str(i) * 3 # e.g., "499499499" for function enc!("499")
key_len = len(key)
for j in range(len(result)):
result[j] ^= key_len
result[j] ^= ord(key[j % key_len])
return bytes(result)
```
**Key insight:** D binaries are rare in CTFs but identifiable by `_D` symbol prefixes and Phobos library references. The compile-time template system means D functions may be duplicated hundreds of times with different parameters — look for patterns like `enc!("N")` where N varies.
---
### Haskell Binary Reversing via STG Closures and hsdecomp (hxp CTF 2017, Codegate 2018)
GHC-compiled Haskell binaries use the STG (Spineless Tagless G-machine) execution model, making them notoriously difficult to reverse due to lazy evaluation, closures, and thunks. The STG machine turns everything into closure calls rather than direct function calls.
**Recognition:**
- Shared libraries: `libHSbase-*`, `libHSrts-*`
- Entry symbol: `hs_main` (replaces standard `main`)
- Mangled symbols use Z-encoding: `z` = prefix, `Z` = uppercase, `zd` = `.`, `zi` = `$`
- GHC calling convention register mapping: `rbx` = R1, `r14` = R2
**Closure structure:**
Closures are structs where the first qword points to the info table/code. The info table precedes the code pointer and contains metadata (closure type, layout info, SRT).
```bash
# Identify Haskell binary
ldd ./binary | grep libHS
readelf -s ./binary | grep hs_main
# Decompile with hsdecomp (github.com/gereeter/hsdecomp)
# Recovers closure structure and pattern matching into pseudo-Haskell
python2 hsdecomp ./binary
# Compile reference for monkey-patching
ghc -O0 reference.hs -o reference
objcopy --dump-section .text=main_code reference
```
**Monkey-patching technique:**
When decompilation fails or closures are opaque, compile a minimal Haskell program with the same GHC version, extract the compiled `Main_main_info` closure code, and patch it into the challenge binary. This forces evaluation of hidden closures and prints their results by replacing the main entry point with a known evaluator.
```haskell
-- reference.hs: minimal program that evaluates and prints the target closure
module Main where
main :: IO ()
main = print targetClosure -- replace with the closure you want to evaluate
```
**Key insight:** Haskell binaries are notoriously hard to reverse due to lazy evaluation, closures, and thunks. The STG machine turns everything into closure calls rather than direct function calls. `hsdecomp` recovers the closure structure and pattern matching. When decompilation fails, monkey-patching a known `Main_main_info` from a reference binary forces evaluation of hidden closures and prints results.
**Detection:** `libHSbase-*` shared libraries, `hs_main` entry, Z-encoded symbols (e.g., `MainZCmain`), GHC version strings.
**References:** hxp CTF 2017, Codegate 2018
---
### Haskell Binary RE via GHC CMM Intermediate Language (N1CTF 2018)
GHC-compiled Haskell binaries are nearly impossible to decompile with IDA due to the STG execution model. When a `.cmm` (C-- intermediate) file is available or recoverable, read it to understand thunks, closures, and lazy evaluation semantics. For exponentially-growing recursive structures, compute segment sizes with memoization and use binary search instead of materializing the full string.
**Pattern:** The binary builds a recursive string structure where `f(n) = s1 + f(n-1) + s2 + f(n-1) + s3`. Direct evaluation is `O(2^n)` in both time and space. Instead, compute the size of each recursion level with memoization, then binary-search for the target character index by walking the segment boundaries.
```python
# Haskell recursive string: f(n) = s1 + f(n-1) + s2 + f(n-1) + s3
# Direct evaluation is O(2^n) -- use size memoization:
from functools import lru_cache
@lru_cache(maxsize=None)
def fsize(n):
if n == 0: return len(s0)
return len(s1) + fsize(n-1) + len(s2) + fsize(n-1) + len(s3)
def char_at(n, offset):
if n == 0: return s0[offset]
if offset < len(s1): return s1[offset]
offset -= len(s1)
if offset < fsize(n-1): return char_at(n-1, offset)
offset -= fsize(n-1)
if offset < len(s2): return s2[offset]
offset -= len(s2)
return char_at(n-1, offset)
```
**Key insight:** GHC's CMM (C minus minus) intermediate representation preserves enough structure to identify algorithms. For recursive string constructions that double in size each level, compute segment sizes with memoization and binary-search for target indices instead of materializing the exponentially-growing string.
**Detection:** Haskell binary (see recognition above) with a `.cmm` file included in the challenge distribution. Look for recursive closure applications that produce string-like data with exponential growth.
**References:** N1CTF 2018
---
## .NET / C# Binary Reversing
.NET 托管程序（C# 编译的 exe/dll）有完整独立工作流，详见 [`../dotnet-reverse/SKILL.md`](../dotnet-reverse/SKILL.md)。
**速查要点：**
- 识别：PE 头 CLR runtime header 非零、`mscoree.dll` 导入、`mscorlib`/`System.Private.CoreLib` 字符串
- 工具：**dnSpyEx**（反编译+调试+IL 编辑，Windows GUI）/ **ILSpy**（CLI）/ **de4dot**（脱混淆）/ **dnlib**（脚本化）
- 脱混淆：ConfuserEx / SmartAssembly / Babel / .NET Reactor → `de4dot`，详见 [`../dotnet-reverse/references/obfuscators.md`](../dotnet-reverse/references/obfuscators.md)
- **IL 优先**：关键判断与 patch 用 IL 编辑器而非 C# 反编译器（避免编译器生成代码重编译失真）
- 红队 Sharp* 工具（Rubeus/SharpHound）分析 → [`../dotnet-reverse/references/sharp-tools.md`](../dotnet-reverse/references/sharp-tools.md)
**边界：** IL2CPP / NativeAOT 编译产物是 native（无 CLR），不走 dnSpy → 走 `ida-reverse/` / `radare2/`（见 [seed-014](../field-journal/seed-014_unity-il2cpp-reverse.md)）。Codegate 2013 两阶段 XOR+AES-CBC 模式见 [tools.md](tools.md#net-analysis) `.NET Analysis` 段。
---
## C++ Binary Reversing (Quick Reference)
While C++ RE is well-covered by general tools, these patterns are CTF-specific:
### vtable Reconstruction
```text
# Virtual function tables (vtables):
# First 8 bytes of object → pointer to vtable
# vtable entries: [typeinfo_ptr, destructor, method1, method2, ...]
# In Ghidra: Data → Create Pointer at vtable address
# Identify polymorphic dispatch:
# mov rax, [rdi] # Load vtable from this pointer
# call [rax + 0x18] # Call 4th virtual method (0x18/8 = 3rd after typeinfo+dtor)
```
### RTTI (Run-Time Type Information)
```bash
# If not stripped, RTTI reveals class hierarchy
strings binary | grep -E "^[0-9]+[A-Z]" # Mangled type names
c++filt _ZTI7MyClass # → typeinfo for MyClass
# In Ghidra: search for vtable references, follow typeinfo pointer
# typeinfo struct: {vtable_for_typeinfo, name_string, base_class_ptr}
```
### Standard Library Patterns
```text
std::string (libstdc++):
SSO (Small String Optimization): inline buffer for ≤15 chars
Layout: {char* ptr, size_t size, union{size_t cap, char buf[16]}}
std::vector<T>:
{T* begin, T* end, T* capacity_end}
std::map<K,V>:
Red-black tree: each node has {left, right, parent, color, key, value}
std::unordered_map<K,V>:
Hash table: {bucket_array, size, load_factor_max, ...}
```

486
reverse-engineering/languages-platforms.md Normal file
View File

@@ -0,0 +1,486 @@
# CTF Reverse - Platform & Framework-Specific Techniques
## Table of Contents
- [Rust serde_json Schema Recovery](#rust-serde_json-schema-recovery)
- [Android JNI RegisterNatives Obfuscation (HTB WonderSMS)](#android-jni-registernatives-obfuscation-htb-wondersms)
- [Android DEX Runtime Bytecode Patching via /proc/self/maps (Google CTF 2017)](#android-dex-runtime-bytecode-patching-via-procselfmaps-google-ctf-2017)
- [Android Native .so Loading Bypass in New Project (Codegate CTF 2018)](#android-native-so-loading-bypass-in-new-project-codegate-ctf-2018)
- [Frida Firebase Cloud Functions Bypass (BSidesSF 2026)](#frida-firebase-cloud-functions-bypass-bsidessf-2026)
- [Verilog/Hardware Reverse Engineering (srdnlenCTF 2026)](#veriloghardware-reverse-engineering-srdnlenctf-2026)
- [Prefix-by-Prefix Hash Reversal (Nullcon 2026)](#prefix-by-prefix-hash-reversal-nullcon-2026)
- [Ruby/Perl Polyglot Constraint Satisfaction (BearCatCTF 2026)](#rubyperl-polyglot-constraint-satisfaction-bearcatctf-2026)
- [Electron App + Native Binary Reversing (RootAccess2026)](#electron-app--native-binary-reversing-rootaccess2026)
- [Node.js npm Package Runtime Introspection (RootAccess2026)](#nodejs-npm-package-runtime-introspection-rootaccess2026)
- [Frida Android Certificate Pinning Bypass (h1702ctf 2017)](#frida-android-certificate-pinning-bypass-h1702ctf-2017)
- [Android Anti-Debug: TracerPid, su Binary, System Properties (h1702ctf 2017)](#android-anti-debug-tracerpid-su-binary-system-properties-h1702ctf-2017)
- [Android Log-Based Key Extraction (HackIT 2017)](#android-log-based-key-extraction-hackit-2017)
- [Native JNI Key Extraction via Memory Dump and Smali Patching (HackIT 2017)](#native-jni-key-extraction-via-memory-dump-and-smali-patching-hackit-2017)
- [IBM AS/400 SAVF File EBCDIC Decoding (EKOPARTY 2017)](#ibm-as400-savf-file-ebcdic-decoding-ekoparty-2017)
- [Intel SGX Enclave Reverse Engineering (Pwn2Win 2017)](#intel-sgx-enclave-reverse-engineering-pwn2win-2017)
For core language reversing (Python, BF/esolangs, DOS, OPAL), see [languages.md](languages.md).
For Go and Rust binary reversing, see [languages-compiled.md](languages-compiled.md).
---
## Rust serde_json Schema Recovery
**Pattern (Curly Crab, PascalCTF 2026):** Rust binary reads JSON from stdin, deserializes via serde_json, prints success/failure emoji.
**Approach:**
1. Disassemble serde-generated `Visitor` implementations
2. Each visitor's `visit_map` / `visit_seq` reveals expected keys and types
3. Look for string literals in deserializer code (field names like `"pascal"`, `"CTF"`)
4. Reconstruct nested JSON schema from visitor call hierarchy
5. Identify value types from visitor method names: `visit_str` = string, `visit_u64` = number, `visit_bool` = boolean, `visit_seq` = array
```json
{"pascal":"CTF","CTF":2026,"crab":{"I_":true,"cr4bs":1337,"crabby":{"l0v3_":["rust"],"r3vv1ng_":42}}}
```
**Key insight:** Flag is the concatenation of JSON keys in schema order. Reading field names in order reveals the flag.
---
## Android JNI RegisterNatives Obfuscation (HTB WonderSMS)
**Pattern:** Android app loads native library with `System.loadLibrary()`, but uses `RegisterNatives` in `JNI_OnLoad` instead of standard JNI naming convention (`Java_com_pkg_Class_method`). This hides which C++ function handles each Java native method.
**Identification:**
```java
// In decompiled Java (jadx):
static { System.loadLibrary("audio"); }
private final native ProcessedMessage processMessage(SmsMessage msg);
```
Standard JNI would have a symbol `Java_com_rloura_wondersms_SmsReceiver_processMessage`. If that symbol is missing from the `.so`, `RegisterNatives` is being used.
**Finding the real handler in Ghidra:**
1. Locate `JNI_OnLoad` (exported symbol, always present)
2. Trace to `RegisterNatives(env, clazz, methods, count)` call
3. The `methods` array contains `{name, signature, fnPtr}` structs
4. Follow `fnPtr` to find the actual native function
```c
// JNI_OnLoad registers functions manually:
static JNINativeMethod methods[] = {
{"processMessage", "(Landroid/telephony/SmsMessage;)LProcessedMessage;", (void*)real_handler}
};
(*env)->RegisterNatives(env, clazz, methods, 1);
```
**Architecture selection for analysis:**
```bash
# x86_64 gives best Ghidra decompilation (most similar to desktop code)
# Extract from APK:
unzip WonderSMS.apk -d extracted/
ls extracted/lib/x86_64/ # Prefer this over arm64-v8a for static analysis
```
**Key insight:** `RegisterNatives` is a deliberate obfuscation technique — it decouples Java method names from native symbol names, making it impossible to find handlers by string search alone. Always check `JNI_OnLoad` first when reversing Android native libraries with stripped symbols.
**Detection:** Native method declared in Java + no matching JNI symbol in `.so` + `JNI_OnLoad` present. The library is typically stripped (no debug symbols).
---
## Android DEX Runtime Bytecode Patching via /proc/self/maps (Google CTF 2017)
Native JNI library patches Dalvik bytecode in memory at runtime: reads `/proc/self/maps` to find loaded DEX, `mprotect`s it writable, then XOR-patches specific bytecode offsets.
```python
# Reconstruct the patched DEX offline:
# 1. Extract the embedded DEX from the APK
# 2. Find the XOR key and patch offsets in the native .so (IDA/Ghidra)
# 3. Apply the same patches to the static DEX
import struct
with open('classes.dex', 'rb') as f:
dex = bytearray(f.read())
# Patch 144 bytes starting at offset found in .so
xor_key = 0x5A
for i in range(patch_offset, patch_offset + 144):
dex[i] ^= xor_key
# 4. Recompute DEX checksum and SHA-1 hash
# 5. Decompile with jadx or baksmali
```
**Key insight:** Native libraries can modify DEX bytecode in memory via `/proc/self/maps` + `mprotect`, making static analysis of the APK alone insufficient. The XOR key and patch offsets must be extracted from the native `.so` to reconstruct the actual runtime DEX. Only works on Dalvik (API < 21), not ART.
---
### Android Native .so Loading Bypass in New Project (Codegate CTF 2018)
**Pattern:** Instead of reversing complex JNI validation logic, create a new Android Studio project with matching package name, class name, and native method signature. Load the original `.so` library and call the native function directly, completely bypassing all Java-level checks (random number validation, PIN entry, root detection, etc.).
```java
// Create new project with same package: com.example.puing.a2018codegate
package com.example.puing.a2018codegate;
public class Main4Activity extends AppCompatActivity {
static { System.loadLibrary("hello-libs"); }
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
String flag = stringFromJNI(); // call native directly, skip all Java validation
Log.d("FLAG", flag);
}
public native String stringFromJNI();
}
```
**Key insight:** JNI function names encode the package path and class name. Create a new Android project with matching package/class/method names, include the original `.so`, and call the native function directly. All Java-level validation (random checks, PIN entry, root detection) is bypassed entirely.
**Detection:** APK with native `.so` library where the flag or secret is computed inside the native code and returned to Java. The Java layer has multiple validation gates (EditText checks, random number comparisons, device checks) before calling the native method.
**References:** Codegate CTF 2018
---
## Frida Firebase Cloud Functions Bypass (BSidesSF 2026)
**Pattern (vinyl-drop, doremi):** Android app validates actions (QR codes, purchases) via Firebase Cloud Functions. The expected payload format includes the Firebase UID, a value, and a timestamp. Use Frida to hook the app post-login, construct a valid payload, and call the Cloud Function directly.
```javascript
// Frida hook to bypass QR validation
Java.perform(function() {
var FirebaseFunctions = Java.use('com.google.firebase.functions.FirebaseFunctions');
var FirebaseAuth = Java.use('com.google.firebase.auth.FirebaseAuth');
// Get current user UID after login
var auth = FirebaseAuth.getInstance();
var uid = auth.getCurrentUser().getUid();
// Construct valid payload: uid + amount + timestamp
var unixMs = Java.use('java.lang.System').currentTimeMillis();
var payload = uid + "+100+" + unixMs;
// Call the Cloud Function directly
var functions = FirebaseFunctions.getInstance();
var data = Java.use('java.util.HashMap').$new();
data.put("payload", payload);
functions.getHttpsCallable("validateScanPayload").call(data);
});
```
**Key insight:** Firebase AppCheck and Cloud Functions rely on the client to construct valid payloads. Post-authentication, Frida can hook the app to call any Cloud Function with arbitrary parameters, bypassing client-side validation (QR scanning, payment processing, etc.).
**When to recognize:** Android app with `google-services.json`, Firebase dependencies in `build.gradle`, Cloud Function calls in decompiled code.
**References:** BSidesSF 2026 "vinyl-drop"
---
## Verilog/Hardware Reverse Engineering (srdnlenCTF 2026)
**Pattern (Rev Juice):** Verilog HDL source for a vending machine with hidden product unlocked by specific coin insertion and selection sequence.
**Approach:**
1. Analyze Verilog modules to understand state machine and history tracking
2. Identify hidden conditions (e.g., product 8 enabled only when `COINS_HISTORY` array has specific values at specific taps)
3. Build timing model for each action type (how many clock cycles each operation takes)
4. Work backward from required history values to construct the correct input sequence
**Timing model construction:**
```python
# Map each action to its cycle count (determined from Verilog state machines)
TIMING = {
"insert_coin": 3, # 3 cycles per coin insertion
"select_success": 7, # 7 cycles for successful product selection
"select_fail": 5, # 5 cycles for failed selection attempt
"cancel_with_coins": 4, # 4 cycles for cancel when coins > 0
"cancel_at_zero": 2, # 2 cycles for cancel when coins = 0
}
# COINS_HISTORY is a shift register updated each cycle
# History tap requirements (from Verilog conditions):
# H[0]=1, H[7]=4, H[28]=H[33]=H[38]=6
# H[63]=H[73]=2, H[80]=9
# (H[19]+H[21]+H[56]+H[69]) mod 32 = 0
```
**Key insight:** Hardware challenges require understanding the exact timing model — each operation takes a specific number of clock cycles, and shift registers record history at fixed tap positions. Work backward from the required tap values to determine what action must have occurred at each cycle. The solution is often a specific sequence notation (e.g., `I9C_SP6_CNL_I2C_SP2_I6C_SP6_SP6_SP5_CNL_I4C_SP1`).
**Detection:** Look for `.v` or `.sv` (Verilog/SystemVerilog) files, `always @(posedge clk)` blocks, shift register patterns, and state machine `case` statements with hidden conditions gated on history values.
---
## Prefix-by-Prefix Hash Reversal (Nullcon 2026)
See [patterns-ctf-2.md](patterns-ctf-2.md#prefix-hash-brute-force-nullcon-2026) for the full technique. This section covers language-specific considerations.
**Language-specific notes:**
- Hash algorithm may be uncommon (MD2, custom) — don't need to identify it, just match outputs by running the binary
- Use `subprocess.run()` with `timeout=2` to handle binaries that hang on bad input
- For stripped binaries, check if `ltrace` reveals the hash function name (e.g., `MD2_Update`)
---
## Ruby/Perl Polyglot Constraint Satisfaction (BearCatCTF 2026)
**Pattern (Polly's Key):** A single file valid in both Ruby and Perl. Each language imposes different validation constraints on a 50-character key. Satisfy both simultaneously to decrypt the flag.
**Polyglot structure exploits:**
- Ruby: `=begin`...`=end` is a block comment
- Perl: `=begin`...`=cut` is POD (Plain Old Documentation), `=end` is ignored
- Different code runs in each language based on comment block boundaries
**Typical constraints:**
- **Ruby:** Character set must form a mathematical property (e.g., all 50 printable ASCII chars except `^` used exactly once, each satisfying `XOR(val, (val-16) % 257)` is a primitive root mod 257)
- **Perl:** Ordering constraint via insertion sort inversion count (hardcoded inversion table determines exact permutation)
**Solution approach:**
1. Find the valid character set (mathematical constraint from one language)
2. Use the ordering constraint (from other language) to determine exact arrangement
3. Compute key hash (e.g., MD5) and decrypt
```python
# Determine character ordering from inversion counts
def reconstruct_from_inversions(chars, inv_counts):
result = []
remaining = sorted(chars)
for i in range(len(chars) - 1, -1, -1):
# inv_counts[i] = number of elements to the left that are greater
idx = inv_counts[i]
result.insert(idx, remaining.pop(i))
return result
```
**Key insight:** Polyglot files exploit language-specific comment/block syntax to run different code in each interpreter. The constraints from both languages intersect to uniquely determine the key. Identify which code runs in which language by testing the file with both interpreters and comparing behavior.
**Detection:** File that runs under multiple interpreters (`ruby file && perl file`). Challenge mentions "polyglot" or provides a file ending in `.rb` that also looks like Perl.
---
## Electron App + Native Binary Reversing (RootAccess2026)
**Pattern (Rootium Browser):** Electron desktop app bundles a native ELF/DLL binary for sensitive operations (vault, crypto, auth). The Electron layer is a wrapper; the real flag logic is in the native binary.
**Extraction workflow:**
1. **Unpack Electron ASAR archive:**
```bash
# Install ASAR tool
npm install -g @electron/asar
# Extract the app.asar archive
asar extract resources/app.asar app_extracted/
ls app_extracted/
```
2. **Locate native binary:** Search for ELF/DLL files called from JavaScript:
```bash
# Find native binaries
find app_extracted/ -name "*.node" -o -name "*.so" -o -name "*vault*" -o -name "*auth*"
# Check JS for child_process.spawn or ffi-napi calls
grep -r "spawn\|execFile\|ffi\|require.*native" app_extracted/
```
3. **Reverse the native binary** (XOR + rotation cipher example):
```python
def decrypt_password(encrypted_bytes, key):
"""Common pattern: XOR with constant + bit rotation + key XOR."""
result = []
for i, byte in enumerate(encrypted_bytes):
decrypted = ((byte ^ 0x42) >> 3) ^ key[i % len(key)]
result.append(chr(decrypted))
return ''.join(result)
def decrypt_flag(encrypted_flag, password):
"""Flag uses password as key with position-dependent rotation."""
result = []
for i, byte in enumerate(encrypted_flag):
key_byte = ord(password[i % len(password)])
decrypted = ((byte ^ 0x7E) >> (i % 8)) ^ key_byte
result.append(chr(decrypted))
return ''.join(result)
```
**Key insight:** Electron apps are JavaScript wrapping native code. Extract with `asar`, then focus on the native binary. The JS layer often contains the password verification flow in plaintext, revealing what the native binary expects. Look for encrypted data in the `.data` or `.rodata` sections of the ELF.
**Detection:** `.asar` files in `resources/` directory, Electron framework files, `package.json` with electron dependency.
---
## Node.js npm Package Runtime Introspection (RootAccess2026)
**Pattern (RootAccess CLI):** Obfuscated npm package with RC4 encoding, control flow flattening, and flag split across multiple fragments. Static analysis is impractical — use runtime introspection instead.
**Dynamic analysis approach:**
```javascript
#!/usr/bin/env node
// 1. Load obfuscated modules
const cryptoMod = require('target-package/dist/lib/crypto.js');
const vaultMod = require('target-package/dist/lib/vault.js');
// 2. Enumerate all exported properties
for (const mod of [cryptoMod, vaultMod]) {
for (const key of Object.keys(mod)) {
const obj = mod[key];
console.log(`Export: ${key}`);
// List all methods including hidden ones
const props = Object.getOwnPropertyNames(obj);
const proto = Object.getOwnPropertyNames(obj.prototype || {});
console.log(' Own:', props);
console.log(' Proto:', proto);
}
}
// 3. Extract flag fragments
const Engine = cryptoMod.CryptoEngine;
const total = Engine.getTotalFragments();
let flag = '';
for (let i = 1; i <= total; i++) {
flag += Engine.getFragment(i);
}
console.log('Flag:', flag);
// 4. Check for hidden methods (common: __getFullFlag__, _debug, _raw)
const hidden = Object.getOwnPropertyNames(Engine)
.filter(p => p.startsWith('__') || p.startsWith('_'));
console.log('Hidden methods:', hidden);
```
**Key insight:** Heavily obfuscated JavaScript (control flow flattening, RC4 string encoding, dead code) makes static analysis prohibitively slow. Runtime introspection via `Object.getOwnPropertyNames()` reveals all methods including hidden ones. The module's own decryption runs automatically when loaded — just call the decoded functions directly.
**Detection:** npm package with minified/obfuscated `dist/` directory, challenge says "reverse engineer the CLI tool", `package.json` with custom commands.
---
## Frida Android Certificate Pinning Bypass (h1702ctf 2017)
APK uses OkHttp `CertificatePinner` for SSL pinning. Rather than setting up a MITM proxy or patching the APK, use Frida to directly invoke native JNI methods on loaded classes.
```javascript
Java.perform(function() {
var Requestor = Java.use("com.h1702ctf.ctfone.Requestor");
console.log("hName: " + Requestor.hName());
console.log("hVal: " + Requestor.hVal());
});
```
Calling `hName()` and `hVal()` returns the HTTP header name and value needed to bypass the server-side check — no cert pinning bypass required because the secret is in the class methods themselves.
**Key insight:** Frida can invoke native JNI methods directly on a loaded class — no need to bypass cert pinning at network layer or fully reverse the native binary.
**References:** h1702ctf 2017
---
## Android Anti-Debug: TracerPid, su Binary, System Properties (h1702ctf 2017)
Native ARM code implements three sequential anti-analysis checks:
1. Read `/proc/self/status` and look for a non-zero `TracerPid` (debugger attached)
2. Check for existence of the `su` binary (root detection)
3. Read a custom system property via `__system_property_get`
The checks gate a required register value computation. Bypass via static analysis: use IDA's graph view to trace the control flow and identify the "happy path" through all three checks, then compute what register values must hold at each branch.
**Key insight:** Anti-debug checks in native Android code (TracerPid, su, system properties) can be bypassed by static graph analysis to find correct register values without running a debugger.
**References:** h1702ctf 2017
---
## Android Log-Based Key Extraction (HackIT 2017)
A secure messenger app logs cryptographic material via Android's `Log.d()`:
- Curve25519 base agreement value
- Ephemeral shared key per message
- Message IDs and shift counters
The AES-CBC IV derives from the logged ephemeral/shared value; the key derives from the logged base agreement and an accumulated shift counter. Collect all log entries with `adb logcat`, then reconstruct AES-CBC parameters to decrypt intercepted messages.
```bash
adb logcat | grep -E "(agreement|ephemeral|shared|key)" > crypto_log.txt
# Parse log entries to reconstruct: key = f(base_agreement, shift_counter)
# iv = f(ephemeral_shared)
```
**Key insight:** Overly verbose logging in security-sensitive apps leaks enough state to reconstruct encryption parameters without any private key access.
**References:** HackIT CTF 2017
---
## Native JNI Key Extraction via Memory Dump and Smali Patching (HackIT 2017)
A JNI native library handles request signing using an XOR-obfuscated key stored in the `.data` section. The key is deobfuscated at runtime just before use.
**Workflow:**
1. Load the library in IDA with a GDB stub on a rooted device
2. Set a breakpoint after the XOR decryption routine
3. Dump the memory region containing the decrypted key
4. Use `baksmali` to disassemble the APK's DEX, identify the smali file that constructs the signed POST request
5. Patch the smali to change which parameter gets signed, then rebuild with `apktool` and reinstall
```bash
# Decompile APK
apktool d target.apk -o target_decompiled/
# Edit smali: change signed parameter from original to desired value
# Rebuild
apktool b target_decompiled/ -o target_patched.apk
# Sign and install
```
**Key insight:** For JNI signing: memory-dump the decrypted key region during execution, then patch smali to sign desired parameters — avoids fully reversing the native signing algorithm.
**References:** HackIT CTF 2017
---
## IBM AS/400 SAVF File EBCDIC Decoding (EKOPARTY 2017)
IBM AS/400 SAVF (Save File) binary files use EBCDIC encoding rather than ASCII. The flag is interleaved with dummy text using a take-2-skip-2 pattern.
```python
import codecs
with open('savefile.savf', 'rb') as f:
data = f.read()
# Convert EBCDIC to ASCII
ascii_data = data.decode('cp500') # cp500 is IBM EBCDIC International
# Filter: keep uppercase letters and underscores (flag charset)
flag_chars = [c for c in ascii_data if c.isupper() or c == '_']
# Or apply take-2-skip-2 pattern after decoding
flag = ''.join(ascii_data[i] for i in range(0, len(ascii_data), 4)
if ascii_data[i].isupper() or ascii_data[i] == '_')
```
**Key insight:** EBCDIC encoding is IBM mainframe-native. Examine character distribution after decoding to identify interleaving patterns. Filtering for uppercase letters and underscores is an effective shortcut for CTF flag formats.
**References:** EKOPARTY CTF 2017
---
## Intel SGX Enclave Reverse Engineering (Pwn2Win 2017)
Intel SGX enclave `.so` files expose an ECALL dispatch table. The enclave logic (including key derivation) is fully reversible with IDA since SGX code is standard x86-64.
**Workflow:**
1. Locate the ECALL table in the `.so` — a function pointer array indexed by ECALL number
2. Decompile ECALLs with IDA to identify the remote attestation protocol
3. Implement the attestation protocol manually in Python using `sgx_crypto_wrapper`
4. Key derivation: ECDH over P-256 followed by CMAC-AES-128 to derive the session key (SK)
5. Decrypt the AES-128-GCM-encrypted flag blob using the derived SK
```python
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives import cmac, ciphers
# ECDH: derive shared secret from server's P-256 public key
private_key = ec.generate_private_key(ec.SECP256R1())
shared_secret = private_key.exchange(ec.ECDH(), server_pub_key)
# CMAC-AES-128 key derivation (per SGX attestation spec)
c = cmac.CMAC(ciphers.algorithms.AES(b'\x00' * 16))
c.update(shared_secret[:16])
sk = c.finalize()
# Decrypt flag with AES-128-GCM using derived SK
```
**Key insight:** SGX remote attestation key derivation is deterministic given the enclave measurement — reimplementing the protocol in Python recovers the same session key.
**References:** Pwn2Win CTF 2017

499
reverse-engineering/languages.md Normal file
View File

@@ -0,0 +1,499 @@
# CTF Reverse - Language-Specific Techniques
## Table of Contents
- [Python Bytecode Reversing (dis.dis output)](#python-bytecode-reversing-disdis-output)
- [Common Pattern: XOR Validation with Split Indices](#common-pattern-xor-validation-with-split-indices)
- [Bytecode Analysis Tips](#bytecode-analysis-tips)
- [Python Opcode Remapping](#python-opcode-remapping)
- [Identification](#identification)
- [Recovery](#recovery)
- [Pyarmor 8/9 Static Unpack (1shot)](#pyarmor-89-static-unpack-1shot)
- [DOS Stub Analysis](#dos-stub-analysis)
- [HarmonyOS HAP/ABC Reverse (abc-decompiler)](#harmonyos-hapabc-reverse-abc-decompiler)
- [Brainfuck/Esolangs](#brainfuckesolangs)
- [Brainfuck Character-by-Character Static Analysis (BSidesSF 2026)](#brainfuck-character-by-character-static-analysis-bsidessf-2026)
- [Brainfuck Side-Channel via Read Count Oracle (BSidesSF 2026)](#brainfuck-side-channel-via-read-count-oracle-bsidessf-2026)
- [Brainfuck Comparison Idiom Detection (BSidesSF 2026)](#brainfuck-comparison-idiom-detection-bsidessf-2026)
- [UEFI Binary Analysis](#uefi-binary-analysis)
- [Transpilation to C](#transpilation-to-c)
- [Code Coverage Side-Channel Attack](#code-coverage-side-channel-attack)
- [Functional Language Reversing (OPAL)](#functional-language-reversing-opal)
- [Python Version-Specific Bytecode (VuwCTF 2025)](#python-version-specific-bytecode-vuwctf-2025)
- [Non-Bijective Substitution Cipher Reversing](#non-bijective-substitution-cipher-reversing)
- [FRACTRAN Program Inversion (Boston Key Party 2016)](#fractran-program-inversion-boston-key-party-2016)
For platform/framework-specific techniques (Android, Electron, Node.js, Verilog, Ruby/Perl polyglot, etc.), see [languages-platforms.md](languages-platforms.md).
For Go and Rust binary reversing, see [languages-compiled.md](languages-compiled.md).
---
## Python Bytecode Reversing (dis.dis output)
### Common Pattern: XOR Validation with Split Indices
Challenge gives raw CPython bytecode (dis.dis disassembly). Common pattern:
1. Check flag length
2. XOR chars at even indices with key1, compare to list p1
3. XOR chars at odd indices with key2, compare to list p2
**Reversing:**
```python
# Given: p1, p2 (expected values), key1, key2 (XOR keys)
flag = [''] * flag_length
for i in range(len(p1)):
flag[2*i] = chr(p1[i] ^ key1) # Even indices
flag[2*i+1] = chr(p2[i] ^ key2) # Odd indices
print(''.join(flag))
```
### Bytecode Analysis Tips
- `LOAD_CONST` followed by `COMPARE_OP` reveals expected values
- `BINARY_XOR` identifies the transformation
- `BUILD_TUPLE`/`BUILD_LIST` with constants = expected output array
- Loop structure: `FOR_ITER` + `BINARY_SUBSCR` = iterating over flag chars
- `CALL_FUNCTION` on `ord` = character-to-int conversion
**Key insight:** Python bytecode challenges give you the algorithm in explicit stack operations. Focus on `LOAD_CONST` values (expected outputs), `BINARY_XOR`/`BINARY_ADD` (the transform), and `BUILD_TUPLE` (the target array) to reconstruct the validation logic without running the bytecode.
---
## Python Opcode Remapping
### Identification
Decompiler fails with opcode errors.
### Recovery
1. Find modified `opcode.pyc` in PyInstaller bundle
2. Compare with original Python opcodes
3. Build mapping: `{new_opcode: original_opcode}`
4. Patch target .pyc
5. Decompile normally
**Shortcut (Hack.lu CTF 2013):** If the challenge bundles its own modified Python interpreter (e.g., a custom `./py` binary), install `uncompyle2`/`uncompyle6` into that interpreter's environment and decompile using the challenge's own runtime. The modified interpreter understands its own opcode mapping, so standard decompilation tools work without manual opcode recovery.
**Tool selection by Python version:** `uncompyle6` supports Python 2.x–3.8. For Python 3.9+ bytecode, use [`pycdc`](https://github.com/zrax/pycdc) (compile from source: `git clone && cmake . && make`).
**Key insight:** Opcode remapping breaks all standard decompilers. The fastest fix is to find the modified `opcode.pyc` in the PyInstaller bundle, diff it against the stock Python opcodes, and patch the target `.pyc` back to standard opcodes before decompiling.
---
## Pyarmor 8/9 Static Unpack (1shot)
- Tool: `Lil-House/Pyarmor-Static-Unpack-1shot`
- Use for Pyarmor 8.x/9.x armored scripts without executing sample code
- Quick signature check: payload typically starts with `PY` + six digits (Pyarmor 7 and earlier `PYARMOR` format is not supported)
Workflow:
1. Ensure target directory contains armored scripts and matching `pyarmor_runtime` library.
2. Run one-shot unpack to emit `.1shot.` outputs (disassembly + experimental decompile).
3. Treat disassembly as ground truth; verify decompiled source with bytecode when inconsistent.
```bash
python /path/to/oneshot/shot.py /path/to/scripts
```
Optional flags:
```bash
# Specify runtime explicitly
python /path/to/oneshot/shot.py /path/to/scripts -r /path/to/pyarmor_runtime.so
# Write outputs to another directory
python /path/to/oneshot/shot.py /path/to/scripts -o /path/to/output
```
Notes:
- `oneshot/pyarmor-1shot` executable must exist before running `shot.py`.
- PyInstaller bundles or archives should be unpacked first, then processed with 1shot.
**Key insight:** Pyarmor 8/9 wraps scripts with runtime decryption. The 1shot tool statically unpacks without execution by directly processing the armored bytecode and `pyarmor_runtime` library. Treat the disassembly output as ground truth when the experimental decompiled source looks inconsistent.
---
## DOS Stub Analysis
PE files can hide code in DOS stub:
1. Check for large DOS stub in Ghidra/IDA
2. Run in DOSBox
3. Load in IDA as 16-bit DOS
4. Look for `int 16h` (keyboard input)
**Key insight:** PE files can embed a fully functional 16-bit DOS program in the DOS stub (before the PE header). If the stub is unusually large, load it in IDA as 16-bit DOS or run it in DOSBox -- the challenge logic may live entirely in the stub.
---
## HarmonyOS HAP/ABC Reverse (abc-decompiler)
- Target files: `.hap` package and embedded `.abc` bytecode
- Tool: `https://github.com/ohos-decompiler/abc-decompiler`
- Download `jadx-dev-all.jar` from releases
Critical startup note:
- `java -jar` may enter GUI mode
- For CLI mode, always use:
```bash
java -cp "./jadx-dev-all.jar" jadx.cli.JadxCLI [options] <input>
```
Most common commands:
```bash
# Basic decompile to directory
java -cp "./jadx-dev-all.jar" jadx.cli.JadxCLI -d "out" ".abc"
# Decompile .abc (recommended for this scenario)
java -cp "./jadx-dev-all.jar" jadx.cli.JadxCLI -m simple -d "out_hap" "modules.abc"
```
Recommended parameters for this challenge:
- `-m simple`: reduce high-level reconstruction to avoid SSA/PHI-heavy failures
- `--log-level ERROR`: keep only critical errors
- Full recommended command:
```bash
java -cp "./jadx-dev-all.jar" jadx.cli.JadxCLI -m simple --log-level ERROR -d "out_abc_simple" "modules.abc"
```
Parameter quick reference:
- `-d` output directory
- `--help` help
Notes:
- `.hap` is a package: extract it first (zip), then locate and analyze `.abc`
- Quote paths containing spaces or non-ASCII characters
- Use a new output directory name per run to avoid stale results
- Errors do not always mean full failure; prioritize `out_xxx/sources/`
- If `auto` fails, switch to `-m simple` first
Standard workflow:
1. Run with `-m simple --log-level ERROR`
2. Inspect key business files in output (for example `pages/Index.java`)
3. If cleaner output is needed, retry with `-m auto` or `-m restructure`
4. If some methods still fail, keep the `simple` output and continue logic analysis via alternate paths
**Key insight:** HarmonyOS `.hap` packages are ZIP archives containing `.abc` bytecode. Use the abc-decompiler's CLI mode (`jadx.cli.JadxCLI`) with `-m simple` for the most reliable decompilation -- GUI mode may launch instead of processing files.
---
## Brainfuck/Esolangs
- Check if compiled with known tools (BF-it)
- Understand tape/memory model
- Static analysis of cell operations
### Brainfuck Character-by-Character Static Analysis (BSidesSF 2026)
**Pattern (i-love-my-bf-part1):** BF programs that validate input character-by-character follow a recognizable pattern: `,` (read char) followed by a sequence of `+` operations whose count equals the expected ASCII value of that character.
**Extraction technique:**
```python
import re
bf_code = open('challenge.bf', 'r').read()
# Split on comma (input read) — each segment handles one character
segments = bf_code.split(',')
expected = []
for seg in segments[1:]: # Skip preamble before first comma
# Count consecutive '+' operations before any branch/output
plus_count = 0
for ch in seg:
if ch == '+':
plus_count += 1
elif ch in '-.[]><':
break # Stop at first non-increment operation
if plus_count > 0:
expected.append(chr(plus_count % 256))
flag = ''.join(expected)
print(f"Flag: {flag}")
```
**Variations:**
- `-` operations: character value = `256 - minus_count`
- Mixed `+`/`-`: net increment determines value
- Cell reset (`[-]`) between characters: each segment is independent
- Loop-based multiplication: `[->>+++<<]` multiplies by 3 — count the inner operations
**Detection:** Large BF file with repeating pattern of `,` followed by many `+` or `-` characters, then a comparison structure (`[-]` or `[->+<]` patterns).
**Key insight:** BF programs that check input are structurally simple — each input byte is compared against a constant built by incrementing a cell. Extract the increment counts to recover the expected input without running the program.
**References:** BSidesSF 2026 "i-love-my-bf-part1"
### Brainfuck Side-Channel via Read Count Oracle (BSidesSF 2026)
**Pattern (i-love-my-bf-part2):** When a BF program validates input character-by-character, a correct character causes the program to consume MORE input bytes (advancing to check the next position). By counting how many `,` (read) operations execute for each candidate input, the character that triggers the most reads is correct.
```python
import itertools
def bytes_read_running_bf(bf_code, input_iter, braces):
"""Run BF and count how many input bytes were consumed."""
tape = [0] * 30000
ptr = ip = reads = 0
input_list = list(input_iter)
input_idx = 0
while ip < len(bf_code):
c = bf_code[ip]
if c == ',':
if input_idx < len(input_list):
tape[ptr] = input_list[input_idx]
input_idx += 1
reads += 1
else:
return reads
elif c == '.': pass
elif c == '+': tape[ptr] = (tape[ptr] + 1) % 256
elif c == '-': tape[ptr] = (tape[ptr] - 1) % 256
elif c == '>': ptr += 1
elif c == '<': ptr -= 1
elif c == '[' and tape[ptr] == 0: ip = braces[ip]
elif c == ']' and tape[ptr] != 0: ip = braces[ip]
ip += 1
return reads
# Recover flag character by character
PRINTABLE = list(range(32, 127))
flag = []
for pos in range(50): # max flag length
best_byte = None
max_reads = 0
baseline = bytes_read_running_bf(bf, flag + [PRINTABLE[0]], braces)
for b in PRINTABLE[1:]:
reads = bytes_read_running_bf(bf, flag + [b], braces)
if reads > baseline:
best_byte = b
break
if best_byte is None:
break
flag.append(best_byte)
print(bytes(flag).decode())
```
**Key insight:** BF input validation programs are sequential — they read one character, check it, and only read the next if it matches. The character causing more reads is correct because the program advances past the validation gate to check the next position.
**References:** BSidesSF 2026 "i-love-my-bf-part2"
### Brainfuck Comparison Idiom Detection (BSidesSF 2026)
**Pattern (i-love-my-bf-part3):** BF programs compiled from higher-level languages use recognizable comparison idioms. The equality check `<[-<->] +<[>-<[-]]>[-<+>]` compares two adjacent cells. By instrumenting a BF interpreter to detect this pattern during execution, you can extract the comparison operands (expected flag bytes) directly from the tape.
```python
EQ_PATTERN = "<[-<->] +<[>-<[-]]>[-<+>]"
def instrumented_bf_run(bf_code, dummy_input):
"""Run BF, detect equality comparisons, extract operands."""
tape = [0] * 30000
ptr = ip = 0
comparisons = []
while ip < len(bf_code):
# Check if current position starts the eq pattern
if bf_code[ip:ip+len(EQ_PATTERN)] == EQ_PATTERN:
# The two cells being compared are at ptr-2 and ptr-1
lhs = tape[ptr - 2] # User input byte
rhs = tape[ptr - 1] # Expected byte
comparisons.append((chr(lhs), chr(rhs)))
# ... normal BF execution ...
ip += 1
return comparisons
# Expected bytes from comparisons reveal the flag
```
**Key insight:** Compiled BF programs reuse fixed idioms for operations like equality comparison, conditional branching, and loops. Pattern-matching these idioms in the BF source or during execution lets you extract constants without fully understanding the program logic.
**Common BF idioms:**
- `[-]` — clear cell (set to 0)
- `[->+<]` — move cell right
- `<[-<->] +<[>-<[-]]>[-<+>]` — equality comparison of two cells
**References:** BSidesSF 2026 "i-love-my-bf-part3"
---
## UEFI Binary Analysis
```bash
7z x firmware.bin -oextracted/
file extracted/* | grep "PE32+"
```
- Bootkit replaces boot loader
- Custom VM protects decryption
- Lift VM bytecode to C
**Key insight:** UEFI binaries are PE32+ executables. Extract the firmware with `7z`, identify PE files with `file`, and load them in Ghidra/IDA. Bootkits replace the boot loader, so focus on DXE drivers and boot services protocols for the challenge logic.
---
## Transpilation to C
For heavily obfuscated code:
```python
for opcode, args in instructions:
if opcode == 'XOR':
print(f"r{args[0]} ^= r{args[1]};")
elif opcode == 'ADD':
print(f"r{args[0]} += r{args[1]};")
```
Compile with `-O3` for constant folding.
**Key insight:** Transpiling obfuscated VM bytecode to C and compiling with `-O3` lets the compiler's constant folding and dead code elimination simplify the algorithm automatically. This is faster than manual deobfuscation for complex instruction sets.
---
## Code Coverage Side-Channel Attack
**Pattern (Coverup, Nullcon 2026):** PHP challenge provides XDebug code coverage data alongside encrypted output.
**How it works:**
- PHP code uses `xdebug_start_code_coverage(XDEBUG_CC_UNUSED | XDEBUG_CC_DEAD_CODE | XDEBUG_CC_BRANCH_CHECK)`
- Encryption uses data-dependent branches: `if ($xored == chr(0)) ... if ($xored == chr(1)) ...`
- Coverage JSON reveals which branches were executed during encryption
- This leaks the set of XOR intermediate values that occurred
**Exploitation:**
```python
import json
# Load coverage data
with open('coverage.json') as f:
cov = json.load(f)
# Extract executed XOR values from branch coverage
executed_xored = set()
for line_no, hit_count in cov['encrypt.php']['lines'].items():
if hit_count > 0:
# Map line numbers to the chr(N) value in the if-statement
executed_xored.add(extract_value_from_line(line_no))
# For each position, filter candidates
for pos in range(len(ciphertext)):
candidates = []
for key_byte in range(256):
xored = plaintext_byte ^ key_byte # or reverse S-box lookup
if xored in executed_xored:
candidates.append(key_byte)
# Combined with known plaintext prefix, this uniquely determines key
```
**Key insight:** Code coverage is a powerful oracle — it tells you which conditional paths were taken. Any encryption with data-dependent branching leaks information through coverage.
**Mitigation detection:** Look for branchless/constant-time crypto implementations that defeat this attack.
---
## Functional Language Reversing (OPAL)
**Pattern (Opalist, Nullcon 2026):** Binary compiled from OPAL (Optimized Applicative Language), a purely functional language.
**Recognition markers:**
- `.impl` (implementation) and `.sign` (signature) source files
- `IMPLEMENTATION` / `SIGNATURE` keywords
- Nested `IF..THEN..ELSE..FI` structures
- Functions named `f1`, `f2`, ... `fN` (numeric naming)
- Heavy use of `seq[nat]`, `string`, `denotation` types
**Reversing approach:**
1. Pure functions are mathematically invertible — reverse each step in the pipeline
2. Identify the transformation chain: `f_final(f_n(...f_2(f_1(input))...))`
3. For each function, build the inverse
**Aggregate brute-force for scramble functions:**
When a transformation accumulates state that depends on original (unknown) values:
```python
# Example: f8 adds cumulative offset based on parity of original bytes
# offset contribution per element depends on whether pre-scramble value is even/odd
# Total offset S = sum of contributions, but S mod 256 has only 256 possibilities
decoded = base64_decode(target)
for total_offset_S in range(256):
candidate = [(b - total_offset_S) % 256 for b in decoded]
# Verify: recompute S from candidate values
recomputed_S = sum(contribution(i, candidate[i]) for i in range(len(candidate))) % 256
if recomputed_S == total_offset_S:
# Apply remaining inverse steps
result = apply_inverse_substitution(candidate)
if all(32 <= c < 127 for c in result):
print(bytes(result))
```
**Key lesson:** When a scramble function has a chicken-and-egg dependency (result depends on original, which is unknown), brute-force the aggregate effect (often mod 256 = 256 possibilities) rather than all possible states (exponential).
---
## Python Version-Specific Bytecode (VuwCTF 2025)
**Pattern (A New Machine):** Challenge targets specific Python version (e.g., 3.14.0 alpha).
**Key requirement:** Compile that exact Python version to disassemble bytecode — alpha/beta versions have different opcodes than stable releases.
```bash
# Build specific Python version
wget https://www.python.org/ftp/python/3.14.0/Python-3.14.0a4.tar.xz
tar xf Python-3.14.0a4.tar.xz
cd Python-3.14.0a4 && ./configure && make -j$(nproc)
./python -c "import dis, marshal; dis.dis(marshal.loads(open('challenge.pyc','rb').read()[16:]))"
```
**Common validation:** Flag compared against tuple of squared ASCII values:
```python
# Reverse: flag[i] = sqrt(expected_tuple[i])
import math
flag = ''.join(chr(int(math.isqrt(v))) for v in expected_values)
```
---
## Non-Bijective Substitution Cipher Reversing
**Pattern (Coverup, Nullcon 2026):** S-box/substitution table has collisions (multiple inputs map to same output).
**Detection:**
```python
sbox = [...] # substitution table
if len(set(sbox)) < len(sbox):
print("Non-bijective! Collisions exist.")
```
**Building reverse lookup:**
```python
from collections import defaultdict
rev_sub = defaultdict(list)
for i, v in enumerate(sbox):
rev_sub[v].append(i)
# rev_sub[output] = [list of possible inputs]
```
**Disambiguation strategies:**
1. Known plaintext format (e.g., `ENO{`, `flag{`) fixes key bytes at known positions
2. Side-channel data (code coverage, timing) eliminates impossible candidates
3. Printable ASCII constraint (32-126) reduces candidate space
4. Re-encrypt candidates and verify against known ciphertext
---
## FRACTRAN Program Inversion (Boston Key Party 2016)
FRACTRAN: an esoteric language where computation is iterated multiplication by a fraction table. Input is encoded as prime factorization (ASCII values as exponents of sequential primes). To invert: swap each fraction's numerator and denominator, run the "success" output backward through the inverted program.
```python
# Original: for each step, find first fraction where n*frac is integer
def fractran_step(n, fractions):
for num, den in fractions:
if (n * num) % den == 0:
return (n * num) // den
return None # Halt
# Inversion: swap num/denom in fraction table
inverted = [(d, n) for n, d in fraction_table]
# Run target output through inverted program to recover input
```
**Key insight:** FRACTRAN programs can be inverted by swapping numerators and denominators. The prime factorization encoding is the key to understanding I/O -- factor the result to extract exponents of sequential primes, map to ASCII.
**Detection:** Challenge mentions fractions, prime factorization, or provides a list of rational numbers.

397
reverse-engineering/patterns-ctf-2.md Normal file
View File

@@ -0,0 +1,397 @@
# CTF Reverse - Competition-Specific Patterns (Part 2)
## Table of Contents
- [Multi-Layer Self-Decrypting Binary (DiceCTF 2026)](#multi-layer-self-decrypting-binary-dicectf-2026)
- [Embedded ZIP + XOR License Decryption (MetaCTF 2026)](#embedded-zip--xor-license-decryption-metactf-2026)
- [Stack String Deobfuscation from .rodata XOR Blob (Nullcon 2026)](#stack-string-deobfuscation-from-rodata-xor-blob-nullcon-2026)
- [Prefix Hash Brute-Force (Nullcon 2026)](#prefix-hash-brute-force-nullcon-2026)
- [CVP/LLL Lattice for Constrained Integer Validation (HTB ShadowLabyrinth)](#cvplll-lattice-for-constrained-integer-validation-htb-shadowlabyrinth)
- [Decision Tree Function Obfuscation (HTB WonderSMS)](#decision-tree-function-obfuscation-htb-wondersms)
- [GF(2^8) Gaussian Elimination for Flag Recovery (ApoorvCTF 2026)](#gf28-gaussian-elimination-for-flag-recovery-apoorvctf-2026)
- [ROP Chain Obfuscation in Modified Binary (PlaidCTF 2016)](#rop-chain-obfuscation-in-modified-binary-plaidctf-2016)
---
## Multi-Layer Self-Decrypting Binary (DiceCTF 2026)
**Pattern (another-onion):** Binary with N layers (e.g., 256), each reading 2 key bytes, deriving keystream via SHA-256 NI instructions, XOR-decrypting the next layer, then jumping to it. Must solve within a time limit (e.g., 30 minutes).
**Oracle for correct key:** Wrong key bytes produce garbage code. Correct key bytes produce code with exactly 2 `call read@plt` instructions (next layer's reads). Brute-force all 65536 candidates per layer using this oracle.
**JIT execution approach (fastest):**
```c
// Map binary's memory at original virtual addresses into solver process
// Compile solver at non-overlapping address: -Wl,-Ttext-segment=0x10000000
void *text = mmap((void*)0x400000, text_size, PROT_RWX, MAP_FIXED|MAP_PRIVATE, fd, 0);
void *bss = mmap((void*)bss_addr, bss_size, PROT_RW, MAP_FIXED|MAP_SHARED, shm_fd, 0);
// Patch read@plt to inject candidate bytes instead of reading stdin
// Patch tail jmp/call to next layer with ret/NOP to return from layer
// Fork-per-candidate: COW gives isolated memory without memcpy
for (int candidate = 0; candidate < 65536; candidate++) {
pid_t pid = fork();
if (pid == 0) {
// Child: remap BSS as MAP_PRIVATE (COW from shared file)
mmap(bss_addr, bss_size, PROT_RW, MAP_FIXED|MAP_PRIVATE, shm_fd, 0);
inject_key(candidate >> 8, candidate & 0xff);
((void(*)())layer_addr)(); // Execute layer as function call
// Check: does decrypted code contain exactly 2 call read@plt?
if (count_read_calls(next_layer_addr) == 2) signal_found(candidate);
_exit(0);
}
}
```
**Performance tiers:**
| Approach | Speed | 256-layer estimate |
|----------|-------|--------------------|
| Python subprocess | ~2/s | days |
| Ptrace fork injection | ~119/s | 6+ hours |
| JIT + fork-per-candidate | ~1000/s | 140 min |
| JIT + shared BSS + 32 workers | ~3500/s | **~17 min** |
**Shared BSS optimization:** BSS (16MB+) stored in `/dev/shm` as `MAP_SHARED` in parent. Children remap as `MAP_PRIVATE` for COW. Reduces fork overhead from 16MB page-table setup to ~4KB.
**Key insight:** Multi-layer decryption challenges are fundamentally about building fast brute-force engines. JIT execution (mapping binary memory into solver, running code directly as function calls) is orders of magnitude faster than ptrace. Fork-based COW provides free memory isolation per candidate.
**Gotchas:**
- Real binary may use `call` (0xe8) instead of `jmp` (0xe9) for layer transitions — adjust tail patching
- BSS may extend beyond ELF MemSiz via kernel brk mapping — map extra space
- SHA-NI instructions work even when not advertised in `/proc/cpuinfo`
---
## Embedded ZIP + XOR License Decryption (MetaCTF 2026)
**Pattern (License To Rev):** Binary requires a license file as argument. Contains an embedded ZIP archive with the expected license, and an XOR-encrypted flag.
**Recognition:**
- `strings` reveals `EMBEDDED_ZIP` and `ENCRYPTED_MESSAGE` symbols
- Binary is not stripped — `nm` or `readelf -s` shows data symbols in `.rodata`
- `file` shows PIE executable, source file named `licensed.c`
**Analysis workflow:**
1. **Find data symbols:**
```bash
readelf -s binary | grep -E "EMBEDDED|ENCRYPTED|LICENSE"
# EMBEDDED_ZIP at offset 0x2220, 384 bytes
# ENCRYPTED_MESSAGE at offset 0x21e0, 35 bytes
```
2. **Extract embedded ZIP:**
```python
import struct
with open('binary', 'rb') as f:
data = f.read()
# Find PK\x03\x04 magic in .rodata
zip_start = data.find(b'PK\x03\x04')
# Extract ZIP (size from symbol table or until next symbol)
open('embedded.zip', 'wb').write(data[zip_start:zip_start+384])
```
3. **Extract license from ZIP:**
```bash
unzip embedded.zip # Contains license.txt
```
4. **XOR decrypt the flag:**
```python
license = open('license.txt', 'rb').read()
enc_msg = open('encrypted_msg.bin', 'rb').read() # Extract from .rodata
flag = bytes(a ^ b for a, b in zip(enc_msg, license))
print(flag.decode())
```
**Key insight:** No need to run the binary or bypass the expiry date check. The embedded ZIP and encrypted message are both in `.rodata` — extract and XOR offline.
**Disassembly confirms:**
- `memcmp(user_license, decompressed_embedded_zip, size)` — license validation
- Date parsing with `sscanf("%d-%d-%d")` on `EXPIRY_DATE=` field
- XOR loop: `ENCRYPTED_MESSAGE[i] ^ license[i]` → `putc()` per byte
**Lesson:** When a binary has named symbols (`EMBEDDED_*`, `ENCRYPTED_*`), extract data directly from the binary without execution. XOR with known plaintext (the license) is trivially reversible.
---
## Stack String Deobfuscation from .rodata XOR Blob (Nullcon 2026)
**Pattern (stack_strings_1/2):** Binary mmaps a blob from `.rodata`, XOR-deobfuscates it, then uses the blob to validate input. Flag is recovered by reimplementing the verification loop.
**Recognition:**
- `mmap()` call followed by XOR loop over `.rodata` data
- Verification loop with running state (`eax`, `ebx`, `r9`) updated with constants like `0x9E3779B9`, `0x85EBCA6B`, `0xA97288ED`
- `rol32()` operations with position-dependent shifts
- Expected bytes stored in deobfuscated buffer
**Approach:**
1. Extract `.rodata` blob with pyelftools:
```python
from elftools.elf.elffile import ELFFile
with open(binary, "rb") as f:
elf = ELFFile(f)
ro = elf.get_section_by_name(".rodata")
blob = ro.data()[offset:offset+size]
```
2. Recover embedded constants (length, magic values) by XOR with known keys from disassembly
3. Reimplement the byte-by-byte verification loop:
- Each iteration: compute two hash-like values from running state
- XOR them together and with expected byte to recover input byte
- Update running state with constant additions
**Variant (stack_strings_2):** Adds position permutation + state dependency on previous character:
- Position permutation: byte `i` may go to position `pos[i]` in the output
- State dependency: `need = (expected - rol8(prev_char, 1)) & 0xFF`
- Must track `state` variable that updates to current character each iteration
**Key constants to look for:**
- `0x9E3779B9` (golden ratio fractional, common in hash functions)
- `0x85EBCA6B` (MurmurHash3 finalizer constant)
- `0xA97288ED` (related hash constant)
- `rol32()` with shift `i & 7`
---
## Prefix Hash Brute-Force (Nullcon 2026)
**Pattern (Hashinator):** Binary hashes every prefix of the input independently and outputs one digest per prefix. Given N output digests, the flag has N-1 characters.
**Attack:** Recover input one character at a time:
```python
for pos in range(1, len(target_hashes)):
for ch in charset:
candidate = known_prefix + ch + padding
hashes = run_binary(candidate)
if hashes[pos] == target_hashes[pos]:
known_prefix += ch
break
```
**Key insight:** If each prefix hash is independent (no chaining/HMAC), the problem decomposes into `N` x `|charset|` binary executions. This is the hash equivalent of byte-at-a-time block cipher attacks.
**Detection:** Binary outputs multiple hash lines. Changing last character only changes last hash. Different input lengths produce different numbers of output lines.
---
## CVP/LLL Lattice for Constrained Integer Validation (HTB ShadowLabyrinth)
**Pattern:** Binary validates flag via matrix multiplication where grouped input characters are multiplied by coefficient matrices and checked against expected 64-bit results. Standard algebra fails because solutions must be printable ASCII (32-126). Lattice-based CVP (Closest Vector Problem) with LLL reduction solves this efficiently.
**Identification:**
1. Binary groups input characters (e.g., 4 at a time)
2. Each group is multiplied by a coefficient matrix
3. Results compared against hardcoded 64-bit values
4. Need integer solutions in a constrained range (printable ASCII)
**SageMath CVP solver:**
```python
from sage.all import *
def solve_constrained_matrix(coefficients, targets, char_range=(32, 126)):
"""
coefficients: list of coefficient rows (e.g., 4 values per group)
targets: expected output values
char_range: valid character range (printable ASCII)
"""
n = len(coefficients[0]) # characters per group
mid = (char_range[0] + char_range[1]) // 2
# Build lattice: [coeff_matrix | I*scale]
# The target vector includes adjusted targets
M = matrix(ZZ, n + len(targets), n + len(targets))
scale = 1000 # Weight to constrain character range
for i, row in enumerate(coefficients):
for j, c in enumerate(row):
M[j, i] = c
M[n + i, i] = 1 # padding
for j in range(n):
M[j, len(targets) + j] = scale
target_vec = vector(ZZ, [t - sum(c * mid for c in row)
for row, t in zip(coefficients, targets)]
+ [0] * n)
# LLL + CVP
L = M.LLL()
closest = L * L.solve_left(target_vec) # or use Babai
solution = [closest[len(targets) + j] // scale + mid for j in range(n)]
return bytes(solution)
```
**Two-phase validation pattern:**
1. **Phase 1 (matrix math):** Solve via CVP/LLL → recovers first N characters
2. First N characters become AES key → decrypt `file.bin` (XOR last 16 bytes + AES-256-CBC + zlib decompress)
3. **Phase 2 (custom VM):** Decrypted bytecode runs in custom VM, validates remaining characters via another linear system (mod 2^32)
**Modular linear system solving (Phase 2 — VM validation):**
```python
import numpy as np
from sympy import Matrix
# M * x = v (mod 2^32)
M_mod = Matrix(coefficients) % (2**32)
v_mod = Matrix(targets) % (2**32)
# Gaussian elimination in Z/(2^32)
solution = M_mod.solve(v_mod) # Returns flag characters
```
**Key insight:** When a binary validates input through linear combinations with large coefficients and the solution must be in a small range (printable ASCII), this is a lattice problem in disguise. LLL reduction + CVP finds the nearest lattice point, recovering the constrained solution. Cross-reference: invoke `/ctf-crypto` for LLL/CVP fundamentals (advanced-math.md in ctf-crypto).
**Detection:** Binary performs matrix-like operations on grouped input, compares against 64-bit constants, and a brute-force search space is too large (e.g., 256^4 per group × 12 groups).
---
## Decision Tree Function Obfuscation (HTB WonderSMS)
**Pattern:** Binary routes input through ~200+ auto-generated functions, each computing a polynomial expression from input positions, comparing against a constant, and branching left/right. The tree makes static analysis impractical without scripted extraction.
**Identification:**
1. Large number of similar functions with random-looking names (e.g., `f315732804`)
2. Each function computes arithmetic on specific input positions
3. Functions call other tree functions or a final validation function
4. Decompiled code shows `if (expr cmp constant) call_left() else call_right()`
**Ghidra headless scripting for mass extraction:**
```python
# Extract comparison constants from all tree functions
# Run via: analyzeHeadless project/ tmp -import binary -postScript extract_tree.py
from ghidra.program.model.listing import *
from ghidra.program.model.symbol import *
fm = currentProgram.getFunctionManager()
results = []
for func in fm.getFunctions(True):
name = func.getName()
if name.startswith('f') and name[1:].isdigit():
# Find CMP instruction and extract immediate constant
inst_iter = currentProgram.getListing().getInstructions(func.getBody(), True)
for inst in inst_iter:
if inst.getMnemonicString() == 'CMP':
operand = inst.getOpObjects(1)
if operand:
results.append((name, int(operand[0].getValue())))
```
**Constraint propagation from known output format:**
1. Start from known output bytes (e.g., `http://HTB{...}`) → fix several input positions
2. Fixed positions cascade through arithmetic constraints → determine dependent positions
3. Tree root equation pins down remaining free variables
4. Recognize English words in partial flag to disambiguate multiple solutions
**Key insight:** Auto-generated decision trees look overwhelming but are repetitive by construction. Script the extraction (Ghidra, Binary Ninja, radare2) rather than reversing each function manually. The tree is just a dispatcher — the real logic is in the leaf function and its constraints.
**Detection:** Binary with hundreds of similarly-structured functions, 3-5 input position references per function, branching to two other functions or a common leaf.
---
## GF(2^8) Gaussian Elimination for Flag Recovery (ApoorvCTF 2026)
**Pattern (Forge):** Stripped binary performs Gaussian elimination over GF(2^8) (Galois Field with 256 elements, using the AES polynomial). A matrix and augmentation vector are embedded in `.rodata`. The solution vector is the flag.
**GF(2^8) arithmetic with AES polynomial (x^8+x^4+x^3+x+1 = 0x11b):**
```python
def gf_mul(a, b):
"""Multiply in GF(2^8) with AES reduction polynomial."""
p = 0
for _ in range(8):
if b & 1:
p ^= a
hi = a & 0x80
a = (a << 1) & 0xff
if hi:
a ^= 0x1b # Reduction: x^8 = x^4+x^3+x+1
b >>= 1
return p
def gf_inv(a):
"""Brute-force multiplicative inverse (fine for 256 elements)."""
if a == 0: return 0
for x in range(1, 256):
if gf_mul(a, x) == 1:
return x
return 0
```
**Solving the linear system:**
```python
# Extract N×N matrix + N-byte augmentation from binary .rodata
N = 56 # Flag length
# Build augmented matrix: N rows × (N+1) cols
for col in range(N):
# Find non-zero pivot
pivot = next((r for r in range(col, N) if aug[r][col] != 0), -1)
if pivot != col:
aug[col], aug[pivot] = aug[pivot], aug[col]
# Scale pivot row by inverse
inv = gf_inv(aug[col][col])
aug[col] = [gf_mul(v, inv) for v in aug[col]]
# Eliminate column in all other rows
for row in range(N):
if row == col: continue
factor = aug[row][col]
if factor == 0: continue
aug[row] = [v ^ gf_mul(factor, aug[col][j]) for j, v in enumerate(aug[row])]
flag = bytes(aug[i][N] for i in range(N))
```
**Key insight:** GF(2^8) is NOT regular integer arithmetic — addition is XOR, multiplication uses polynomial reduction. The AES polynomial (0x11b) is the most common; look for the constant `0x1b` in disassembly. The binary may encrypt the result with AES-GCM afterward, but the raw solution vector (pre-encryption) is the flag.
**Detection:** Binary with a large matrix in `.rodata` (N² bytes), XOR-based row operations, constants `0x1b` or `0x11b`, and flag length matching sqrt of matrix size.
---
## ROP Chain Obfuscation in Modified Binary (PlaidCTF 2016)
**Pattern (quite quixotic quest):** Modified `curl` binary with a custom `--pctfkey KEY` option. Key validation replaces `esp` with a buffer address and returns into a ~250KB ROP chain stored in a `magic_buf` symbol. The ROP chain validates the key through XOR, MD5, and constant comparisons.
**Analysis approach:**
1. **Detect the ROP dispatch:** Look for `mov esp, eax; ret` or similar stack pivot — this redirects execution into the ROP chain
2. **Dump the ROP chain:** Script GDB to disassemble instructions after each return address in the chain:
```python
# GDB script to trace ROP gadgets
import gdb
magic_buf = 0x080b0000 # symbol address
buf_size = 0x40000 # quarter megabyte
offset = 0
while offset < buf_size:
addr = int.from_bytes(gdb.selected_inferior().read_memory(magic_buf + offset, 4), 'little')
gdb.execute(f'x/3i {addr}')
# Advance past the gadget (typically 4 bytes per return address)
offset += 4
```
3. **Identify patterns in the chain:** Look for unrolled loops (repeated gadget sequences), `pop` instructions that skip data, and `ret imm16` that skip large blocks
4. **Reconstruct the algorithm:** The chain typically performs:
- Key length check (compare with constant)
- Character-level operations (sum ASCII values, XOR with constants)
- Hash computation (MD5 of derived value)
- Hash prefix comparison
- XOR of input with hash as keystream
- Comparison with embedded constants
5. **Extract and solve:** Dump the embedded constants, brute-force any intermediate values (e.g., character sum → MD5 with matching prefix), then XOR to recover the key:
```python
import hashlib
# Brute-force the sum that produces correct MD5 prefix
target_prefix = 0xc0050bdd # extracted from ROP chain
for s in range(128 * 0x35): # max sum of printable chars * key_length
h = hashlib.md5(str(s ^ xor_constant).encode()).hexdigest()
if int(h[:8], 16) == target_prefix:
md5_key = bytes.fromhex(h)
break
# XOR embedded values with MD5 keystream to get flag
flag = bytes(v ^ md5_key[i % 16] for i, v in enumerate(embedded_values))
```
**Key insight:** ROP chain obfuscation ("ROPfuscation") hides algorithms in chains of return-oriented gadgets. The chain looks incomprehensible as raw addresses but becomes analyzable when you: (a) dump each gadget's disassembly, (b) filter repetitions and skip regions, (c) annotate register effects. The chain is functionally equivalent to normal code — it just uses `ret` instead of sequential execution. Large chains (100K+ gadgets) often contain unrolled loops that compress to ~1000 lines of pseudocode.
See also: [patterns-ctf.md](patterns-ctf.md) for Part 1 (hidden emulator opcodes, SPN static extraction, image XOR smoothness, byte-at-a-time cipher, mathematical convergence bitmap, Windows PE XOR bitmap OCR, two-stage RC4+VM loaders, kernel module maze solving, multi-threaded VM channels). [patterns-ctf-3.md](patterns-ctf-3.md) for Part 3 (Z3 single-line Python circuit, sliding window popcount, keyboard LED Morse code, C++ destructor-hidden validation, syscall side-effect memory corruption, MFC dialog event handlers, VM sequential key-chain brute-force, Burrows-Wheeler transform inversion, OpenType font ligature exploitation, GLSL shader VM with self-modifying code, instruction counter as cryptographic state).

719
reverse-engineering/patterns-ctf-3.md Normal file
View File

@@ -0,0 +1,719 @@
# CTF Reverse - Competition-Specific Patterns (Part 3)
## Table of Contents
- [Z3 for Single-Line Python Boolean Circuit (BearCatCTF 2026)](#z3-for-single-line-python-boolean-circuit-bearcatctf-2026)
- [Sliding Window Popcount Differential Propagation (BearCatCTF 2026)](#sliding-window-popcount-differential-propagation-bearcatctf-2026)
- [Morse Code from Keyboard LEDs via ioctl (PlaidCTF 2013)](#morse-code-from-keyboard-leds-via-ioctl-plaidctf-2013)
- [C++ Destructor-Hidden Validation (Defcamp 2015)](#c-destructor-hidden-validation-defcamp-2015)
- [Syscall Side-Effect Memory Corruption (Hack.lu 2015)](#syscall-side-effect-memory-corruption-hacklu-2015)
- [MFC Dialog Event Handler Location (WhiteHat 2015)](#mfc-dialog-event-handler-location-whitehat-2015)
- [VM Sequential Key-Chain Brute-Force (Midnight Flag 2026)](#vm-sequential-key-chain-brute-force-midnight-flag-2026)
- [Burrows-Wheeler Transform Inversion without Terminator (ASIS CTF Finals 2016)](#burrows-wheeler-transform-inversion-without-terminator-asis-ctf-finals-2016)
- [OpenType Font Ligature Exploitation for Hidden Messages (Hack The Vote 2016)](#opentype-font-ligature-exploitation-for-hidden-messages-hack-the-vote-2016)
- [GLSL Shader VM with Self-Modifying Code (ApoorvCTF 2026)](#glsl-shader-vm-with-self-modifying-code-apoorvctf-2026)
- [Instruction Counter as Cryptographic State (MetaCTF Flash 2026)](#instruction-counter-as-cryptographic-state-metactf-flash-2026)
- [Thread Race Condition with Signed Integer Overflow (Codegate 2017)](#thread-race-condition-with-signed-integer-overflow-codegate-2017)
- [ESP32/Xtensa Firmware Reversing with ROM Symbol Map (Insomni'hack 2017)](#esp32xtensa-firmware-reversing-with-rom-symbol-map-insomnihack-2017)
- [Batch Crackme Automation via objdump Pattern Extraction (DEF CON 2017)](#batch-crackme-automation-via-objdump-pattern-extraction-def-con-2017)
- [Fork + Pipe + Dead Branch Anti-Analysis (RCTF 2017)](#fork--pipe--dead-branch-anti-analysis-rctf-2017)
- [Time-Locked Binary with Date-Based Key (Hack.lu 2017)](#time-locked-binary-with-date-based-key-hacklu-2017)
- [ARM Code in Image Pixels via UnicornJS (Hack.lu 2017)](#arm-code-in-image-pixels-via-unicornjs-hacklu-2017)
- [x86 16-bit MBR psadbw Constraint Solving (CSAW 2017)](#x86-16-bit-mbr-psadbw-constraint-solving-csaw-2017)
- [TensorFlow DNN Inversion by Inverting Sigmoid Layers (N1CTF 2018)](#tensorflow-dnn-inversion-by-inverting-sigmoid-layers-n1ctf-2018)
- [BPF Filter Analysis via JIT Compilation to x64 Assembly (Midnight Sun CTF 2018)](#bpf-filter-analysis-via-jit-compilation-to-x64-assembly-midnight-sun-ctf-2018)
---
## Z3 for Single-Line Python Boolean Circuit (BearCatCTF 2026)
**Pattern (Captain Morgan):** Single-line Python (2000+ semicolons) validates flag via walrus operator chains decomposing input as a big-endian integer, with bitwise operations producing a boolean circuit.
**Identification:**
- Single-line Python with semicolons separating statements
- Walrus operator `:=` chains: `(x := expr)`
- Obfuscated XOR: `(x | i) & ~(x & i)` instead of `x ^ i`
- Input treated as a single large integer, decomposed via bit-shifting
**Z3 solution:**
```python
from z3 import *
n_bytes = 29 # Flag length
ari = BitVec('ari', n_bytes * 8)
# Parse semicolon-separated statements
# Model walrus chains as LShR(ari, shift_amount)
# Evaluate boolean expressions symbolically
# Final assertion: result_var == 0
s = Solver()
s.add(bfu == 0) # Final validation variable
if s.check() == sat:
m = s.model()
val = m[ari].as_long()
flag = val.to_bytes(n_bytes, 'big').decode('ascii')
```
**Key insight:** Single-line Python obfuscation creates a boolean circuit over input bits. The walrus operator chains are just variable assignments — split on semicolons and translate each to Z3 symbolically. Obfuscated XOR `(a | b) & ~(a & b)` is just `a ^ b`. Z3 solves these circuits in under a second. Look for `__builtins__` access or `ord()`/`chr()` calls to identify the input→integer conversion.
**Detection:** Single-line Python with 1000+ semicolons, walrus operators, bitwise operations, and a final comparison to 0 or True.
---
## Sliding Window Popcount Differential Propagation (BearCatCTF 2026)
**Pattern (Treasure Hunt 4):** Binary validates input via expected popcount (number of set bits) for each position of a 16-bit sliding window over the input bits.
**Differential propagation:**
When the window slides by 1 bit:
```text
popcount(window[i+1]) - popcount(window[i]) = bit[i+16] - bit[i]
```
So: `bit[i+16] = bit[i] + (data[i+1] - data[i])`
```python
expected = [...] # 337 expected popcount values
total_bits = 337 + 15 # = 352
# Brute-force the initial 16-bit window (must have popcount = expected[0])
for start_val in range(0x10000):
if bin(start_val).count('1') != expected[0]:
continue
bits = [0] * total_bits
for j in range(16):
bits[j] = (start_val >> (15 - j)) & 1
valid = True
for i in range(len(expected) - 1):
new_bit = bits[i] + (expected[i + 1] - expected[i])
if new_bit not in (0, 1):
valid = False
break
bits[i + 16] = new_bit
if valid:
# Convert bits to bytes
flag_bytes = bytes(int(''.join(map(str, bits[i:i+8])), 2)
for i in range(0, total_bits, 8))
if b'BCCTF' in flag_bytes or flag_bytes[:5].isascii():
print(flag_bytes.decode(errors='replace'))
break
```
**Key insight:** Sliding window popcount differences create a recurrence relation: each new bit is determined by the bit 16 positions back plus the popcount delta. Only the first 16 bits are free (constrained by initial popcount). Brute-force the ~4000-8000 valid initial windows — for each, the entire bit sequence is deterministic. Runs in under a second.
**Detection:** Binary computing popcount/hamming weight on fixed-size windows. Expected value array with length ≈ input_bits - window_size + 1. Values in array are small integers (0 to window_size).
---
---
## Morse Code from Keyboard LEDs via ioctl (PlaidCTF 2013)
**Pattern:** Binary uses `ioctl(fd, KDSETLED, value)` to blink keyboard LEDs (Num/Caps/Scroll Lock). Timing patterns encode Morse code.
```bash
# Step 1: Bypass ptrace anti-debug
# Patch ptrace call at offset with NOP (0x90)
python3 -c "
data = open('binary','rb').read()
data = data[:0x72b] + b'\x90'*5 + data[:0x730] # NOP the ptrace call
open('patched','wb').write(data)
"
# Step 2: Run under strace, capture ioctl calls
strace -e ioctl ./patched 2>&1 | grep KDSETLED > leds.txt
# Step 3: Decode timing patterns
# Short blink (250ms) = dit (.), long blink (750ms) = dah (-)
# Inter-character pause = 3x, inter-word pause = 7x
```
```python
# Parse strace output to extract Morse
import re
morse_map = {'.-':'A', '-...':'B', '-.-.':'C', '-..':'D', '.':'E',
'..-.':'F', '--.':'G', '....':'H', '..':'I', '.---':'J',
'-.-':'K', '.-..':'L', '--':'M', '-.':'N', '---':'O',
'.--.':'P', '--.-':'Q', '.-.':'R', '...':'S', '-':'T',
'..-':'U', '...-':'V', '.--':'W', '-..-':'X', '-.--':'Y',
'--..':'Z', '-----':'0', '.----':'1'}
# Map LED on-durations to dots/dashes, group by pauses
```
**Key insight:** `KDSETLED` controls physical keyboard LEDs on Linux (`/dev/console`). The binary must run with console access. Use `strace -e ioctl` to capture all LED state changes without needing physical observation. Timing between calls determines dot vs dash.
---
## C++ Destructor-Hidden Validation (Defcamp 2015)
Validation logic may hide in C++ destructors that execute after `main()` returns. The `__cxa_atexit` mechanism registers destructor callbacks:
1. **Locate destructors:** Search for `__cxa_atexit` calls in `.init_array`/constructor sections
2. **Static analysis:** Identify global objects whose destructors perform flag checking
3. **Dynamic verification:** Set breakpoints on `__cxa_finalize` to trace post-main execution
```asm
# In IDA/Ghidra: look for atexit registrations
__cxa_atexit(destructor_func, object_ptr, dso_handle);
# Destructor contains actual validation:
# - Regex pattern matching on 4-byte blocks (8 sequential checks)
# - Arithmetic: v2 += -3 * s[i] + 36 + (s[i] ^ 0x2FCFBA)
# - Modular verification of accumulated sum
```
**Key insight:** When `main()` appears trivial or incomplete, check destructors of global/static C++ objects. The `.fini_array` section and `__cxa_atexit` registrations reveal hidden post-main logic.
---
## Syscall Side-Effect Memory Corruption (Hack.lu 2015)
The `rt_sigprocmask` syscall writes a `sigset_t` structure to its output pointer. When input parsing passes a pointer near a security-critical variable:
1. Certain input characters (e.g., `:` to `@` range, values 0x3A-0x40) trigger `rt_sigprocmask` as a side effect
2. The syscall zeros out bytes at the output address, which may overlap adjacent variables
3. In little-endian layout, zeroing the MSB of an adjacent integer variable effectively sets it to a small value
```c
// Memory layout (no ASLR):
// 0x603390: input_buffer[4]
// 0x603394: security_check_var
// Input ':' triggers: rt_sigprocmask(SIG_BLOCK, NULL, (sigset_t*)0x603397, ...)
// This zeros bytes at 0x603397+, corrupting security_check_var's high bytes
```
**Key insight:** Audit how input validation functions interact with syscalls. Character-to-syscall mappings in hex conversion routines can produce unintended memory writes via kernel-space operations.
---
## MFC Dialog Event Handler Location (WhiteHat 2015)
To find event handlers in MFC (Microsoft Foundation Class) applications:
1. **Break on SendMessageW:** Set breakpoint on `user32!SendMessageW` to intercept dialog messages
2. **Filter for WM_COMMAND:** Message ID 0x111 indicates button clicks and control events
3. **Trace message map:** Follow the MFC message dispatch from `CWnd::OnWndMsg` → `CCmdTarget::OnCmdMsg` → handler function
4. **OnInitDialog:** Often contains decryption or validation setup; triggered by WM_INITDIALOG (0x110)
```asm
# WinDbg/x64dbg:
bp user32!SendMessageW ".if (poi(@esp+8)==0x111) {} .else {gc}"
# Or in IDA: find cross-references to AFX_MSGMAP_ENTRY structures
```
**Key insight:** MFC applications route messages through dispatch tables. Identify the `AFX_MSGMAP` structure to enumerate all handled messages without runtime analysis.
---
## VM Sequential Key-Chain Brute-Force (Midnight Flag 2026)
**Pattern (67):** Custom VM validates input in N-byte blocks. Each block's output key feeds as input to the next block, preventing parallel solving. Per-block search space is small enough to brute-force (2^24 for 3-byte blocks).
**Recognition signs:**
- Bytecode with XOR-obfuscated opcodes (all bytes XOR'd with a constant, producing ASCII-looking bytecode)
- Iterative transformation loop (xorshift + multiply, repeated 1000+ times) making algebraic inversion impractical
- CHECK opcodes comparing accumulated state against embedded constants
- Large `.data` section with repetitive bytecode patterns
**Solving approach:**
1. Parse bytecode to extract CHECK values (expected key after each block)
2. For each block sequentially, brute-force the input bytes that produce the expected key
3. Use the CHECK value as the key for the next block
```c
// OpenMP-parallelized per-block brute-force
uint32_t process(uint32_t val) {
for (int i = 0; i < 1000; i++) {
val ^= (val << 13);
val ^= (val >> 17);
val ^= (val << 5);
val *= 0x2545f491;
}
return val;
}
int solve_block(uint32_t old_key, uint32_t expected_key, unsigned char *out) {
int found = 0;
#pragma omp parallel for shared(found)
for (int v = 0; v < 0x1000000; v++) {
if (found) continue;
uint32_t input_val = ((v >> 16) << 16) | (v & 0xFF) | ((v >> 8 & 0xFF) << 8);
uint32_t saved = input_val ^ old_key;
uint32_t final_val = process(saved);
if ((final_val ^ saved) == expected_key) {
#pragma omp critical
{ if (!found) { out[0]=v>>16; out[1]=(v>>8)&0xFF; out[2]=v&0xFF; found=1; } }
}
}
return found;
}
// Compile: gcc -O3 -march=native -fopenmp -o solve solve.c
```
**Key insight:** When a transformation is intentionally non-invertible (iterated hash-like function), brute-force is the intended solution. OpenMP parallelization is critical — 287 blocks x 16.7M candidates each takes minutes parallelized vs hours single-threaded. The sequential key dependency means blocks must be solved in order, but each individual block search is embarrassingly parallel.
---
## Burrows-Wheeler Transform Inversion without Terminator (ASIS CTF Finals 2016)
BWT applied to binary representation without a standard terminating character. Requires brute-force inversion by trying all possible original strings.
```python
def bwt_inverse_bruteforce(bwt_string):
"""Invert BWT when no terminating character is present.
Standard BWT inverse needs the terminator position.
Without it, try all n possible rotations."""
n = len(bwt_string)
# Standard BWT inverse produces a table
table = [''] * n
for _ in range(n):
table = sorted([bwt_string[i] + table[i] for i in range(n)])
# Without terminator, all n rows are valid candidates
# Filter by known constraints (e.g., starts with '1' for binary, matches XOR pattern)
candidates = []
for row in table:
# Apply challenge-specific validation
if is_valid_plaintext(row):
candidates.append(row)
return candidates
def bwt_with_xor_rounds(encrypted_hex, num_rounds):
"""Multi-round BWT with XOR key derived from round index"""
data = bytes.fromhex(encrypted_hex)
for round_idx in range(num_rounds - 1, -1, -1):
# Each round: BWT on binary representation, then XOR with round-based key
binary_str = ''.join(format(b, '08b') for b in data)
candidates = bwt_inverse_bruteforce(binary_str)
# Select candidate matching constraints (leading '1', trailing bit rule)
data = select_valid_candidate(candidates, round_idx)
return data
```
**Key insight:** Standard BWT uses a terminating character (like '$') to mark the original string's position. Without it, BWT inversion produces n candidates (one per rotation). Use domain-specific constraints (binary format, XOR round structure, flag prefix) to identify the correct candidate.
---
## OpenType Font Ligature Exploitation for Hidden Messages (Hack The Vote 2016)
Font files with custom OpenType ligatures map visible characters to hidden glyphs. The GSUB (Glyph Substitution) table defines these mappings.
```python
from fontTools.ttLib import TTFont
def decode_font_ligatures(font_path, encoded_text):
"""Extract ligature substitution table and decode message"""
font = TTFont(font_path)
# Extract GSUB table for ligature substitutions
gsub = font['GSUB']
# Navigate to ligature lookup
ligature_map = {}
for lookup in gsub.table.LookupList.Lookup:
for subtable in lookup.SubTable:
if hasattr(subtable, 'ligatures'):
for glyph_name, ligatures in subtable.ligatures.items():
for lig in ligatures:
# Map: input sequence -> output glyph
input_seq = [glyph_name] + lig.Component
output = lig.LigGlyph
ligature_map[tuple(input_seq)] = output
print("Ligature mappings found:")
for inp, out in ligature_map.items():
print(f" {inp} -> {out}")
# Alternative: convert TTF to XML for manual analysis
# font.saveXML('font_dump.xml')
# Search for <LigatureSubst> entries
# Command-line approach:
# pip install fonttools
# ttx font.otf # converts to XML
# grep -A5 'LigatureSubst' font.ttx
```
**Key insight:** Custom fonts with GSUB ligature tables create a cipher where displayed characters differ from their glyph mappings. The `fonttools` library's `ttx` command dumps the font to XML, making ligature substitution tables easily readable. Each ligature maps an input character sequence to a different output glyph.
---
## GLSL Shader VM with Self-Modifying Code (ApoorvCTF 2026)
**Pattern (Draw Me):** A WebGL2 fragment shader implements a Turing-complete VM on a 256x256 RGBA texture. The texture is both program memory and display output.
**Texture layout:**
- **Row 0:** Registers (pixel 0 = instruction pointer, pixels 1-32 = general purpose)
- **Rows 1-127:** Program memory (RGBA = opcode, arg1, arg2, arg3)
- **Rows 128-255:** VRAM (display output)
**Opcodes:** NOP(0), SET(1), ADD(2), SUB(3), XOR(4), JMP(5), JNZ(6), VRAM-write(7), STORE(8), LOAD(9). 16 steps per frame.
**Self-modifying code:** Phase 1 (decryption) uses STORE opcode to XOR-patch program memory that Phase 2 (drawing) then executes. The decryption overwrites SET instructions with correct pixel color values before the drawing code runs.
**Why GPU rendering fails:** The GPU runs all pixels in parallel per frame, but the shader tracks only ONE write target per pixel per frame. With multiple VRAM writes per frame, only the last survives — losing 75%+ of pixels. Similarly, STORE patches conflict during parallel decryption.
**Solve via sequential emulation:**
```python
from PIL import Image
import numpy as np
img = Image.open('program.png').convert('RGBA')
state = np.array(img, dtype=np.int32).copy()
regs = [0] * 33
# Phase 1: Trace decryption — apply all STORE patches sequentially
x, y = start_x, start_y
while True:
r, g, b, a = state[y][x]
opcode = int(r)
if opcode == 1: regs[g] = b & 255 # SET
elif opcode == 4: regs[g] = regs[b] ^ regs[a] # XOR
elif opcode == 8: # STORE — patches program memory
tx, ty = regs[g], regs[b]
state[ty][tx] = [regs[a], regs[a+1], regs[a+2], regs[a+3]]
elif opcode == 5: break # JMP to drawing phase
x += 1
if x > 255: x, y = 0, y + 1
# Phase 2: Execute drawing code — all VRAM writes preserved
vram = np.zeros((128, 256), dtype=np.uint8)
# ... trace with opcode 7 writing to vram[ty][tx] = color
Image.fromarray(vram, mode='L').save('output.png')
```
**Key insight:** GLSL shaders are Turing-complete but GPU parallelism causes write conflicts. Self-modifying code (STORE patches) compounds the problem — patches from parallel executions overwrite each other. Sequential emulation in Python recovers the full output. The program.png file IS the bytecode.
**Detection:** WebGL/shader challenge with a PNG "program" file, challenge says "nothing renders" or output is garbled. Look for custom opcode tables in GLSL source.
---
## Instruction Counter as Cryptographic State (MetaCTF Flash 2026)
**Pattern (Who's Counting?):** Hand-written assembly binary uses a dedicated register (e.g., `r12`) as an instruction counter that increments after nearly every instruction. The counter value feeds into XOR, ROL, and multiply transformations on each input byte, making the entire transformation path-dependent on the number of instructions executed before reaching each byte.
**Identification:**
- Hand-written assembly (no compiler patterns, unusual register usage)
- A register that only increments (`inc r12` or `add r12, 1`) appearing after most instructions
- Transformations that reference this counter register (`xor rax, r12`, `rol al, cl` where `cl` derives from counter)
- Sequential byte processing loop where state carries forward
**Solving approach:**
```python
# Byte-by-byte brute force with emulation
# Since each byte's transformation depends on the counter (which depends
# on all prior instructions), state is path-dependent.
from unicorn import *
from unicorn.x86_const import *
def try_byte(known_prefix, candidate_byte):
"""Emulate binary with known prefix + candidate, check output."""
uc = Uc(UC_ARCH_X86, UC_MODE_64)
# Map code, stack, data segments
uc.mem_map(CODE_BASE, 0x10000)
uc.mem_write(CODE_BASE, binary_code)
uc.mem_map(STACK_BASE, 0x10000)
uc.mem_map(DATA_BASE, 0x10000)
# Write input: known_prefix + candidate
test_input = known_prefix + bytes([candidate_byte])
uc.mem_write(DATA_BASE, test_input + b'\x00' * (64 - len(test_input)))
# Set up registers (rsp, rdi pointing to input, r12 = 0)
uc.reg_write(UC_X86_REG_RSP, STACK_BASE + 0x8000)
uc.reg_write(UC_X86_REG_R12, 0) # instruction counter starts at 0
try:
uc.emu_start(CODE_BASE + ENTRY_OFFSET, CODE_BASE + EXIT_OFFSET)
# Read transformed output, compare against expected
output = uc.mem_read(OUTPUT_ADDR, len(test_input))
return output[:len(test_input)] == expected[:len(test_input)]
except:
return False
# Recover flag byte by byte
flag = b''
for pos in range(FLAG_LEN):
for b in range(256):
if try_byte(flag, b):
flag += bytes([b])
print(f"Position {pos}: {chr(b)} -> {flag}")
break
```
**Key insight:** When a register acts as an instruction counter feeding into byte transformations, the transformation of byte N depends on the exact number of instructions executed while processing bytes 0 through N-1. This makes analytical inversion impractical because the counter value at each byte position depends on the execution path through all prior bytes. Byte-by-byte brute force with full emulation (Unicorn or GDB scripting) is the most reliable approach -- try all 256 values for each position, keeping the state from the correct prefix.
**When to recognize:** Binary has no standard library calls, uses unusual registers consistently, and shows a register that only increments. The transformation per byte involves operations (XOR, rotate, multiply) that reference this counter. Challenge name hints at "counting" or "instructions".
**Alternative approaches:**
- GDB scripting: set breakpoint after each byte's transformation, compare output
- Static analysis: count instructions manually to compute counter values, then invert transforms algebraically (error-prone due to counter accumulation)
**References:** MetaCTF Flash CTF 2026 "Who's Counting?"
---
## Thread Race Condition with Signed Integer Overflow (Codegate 2017)
**Pattern (Hunting):** A combat-simulation binary uses thread-unsafe skill selection. The attack thread checks `skill_id <= 4` using signed comparison, then sleeps briefly before applying damage. During the sleep, switch to a different skill. The fireball skill uses `cdqe` (sign-extend EAX to RAX), converting `0xFFFFFFFF` (icesword damage) to `-1` as a signed 64-bit value. Subtracting `-1` from the boss's HP (`0x7FFFFFFFFFFFFFFF`) causes signed overflow to a negative value, killing the boss.
```python
# Race condition exploit:
# Thread A: select fireball (skill_id=2, passes <= 4 check)
# Thread A: sleeps for animation
# Main: switch to icesword (skill_id=5, damage=0xFFFFFFFF)
# Thread A: wakes, reads damage from icesword slot
# cdqe: 0xFFFFFFFF -> 0xFFFFFFFFFFFFFFFF (-1 signed)
# boss_hp -= (-1) -> boss_hp = 0x7FFFFFFFFFFFFFFF + 1 = negative -> dead
import time, threading
def race():
select_skill(2) # fireball - passes bounds check
time.sleep(0.001)
select_skill(5) # icesword - race into damage calculation
```
**Key insight:** `cdqe` (Convert Doubleword to Quadword Extension) sign-extends 32-bit EAX into 64-bit RAX. When the attack code reads a 32-bit damage value and sign-extends it, `0xFFFFFFFF` becomes `-1`. Subtracting a negative number adds to HP, but if HP is already at `INT64_MAX`, the addition overflows to negative, killing the target.
---
## ESP32/Xtensa Firmware Reversing with ROM Symbol Map (Insomni'hack 2017)
**Pattern (Internet of Fail):** ESP32 firmware (Xtensa architecture) with no native IDA support. Use radare2 with the ESP32 ROM linker script (`esp32.rom.ld`) to map function addresses to names. Cross-reference with public ESP32 HTTP server source code to identify the password-checking logic, composed of ~20 conditional XOR functions operating on a global state variable.
```bash
# Load ESP32 firmware in radare2
r2 -a xtensa -b 32 firmware.bin
# Apply ROM symbol map from ESP-IDF
# esp32.rom.ld maps addresses like:
# 0x40000000 = ets_printf
# 0x400013A0 = cache_Read_Enable
# Load as flags: . esp32.rom.ld.r2
# Identify HTTP request handler by cross-referencing
# with esp-idf/examples/protocols/http_server
# Look for URI handler registration patterns
```
**Key insight:** ESP32's Xtensa architecture lacks mainstream RE tool support, but the ESP-IDF SDK provides ROM linker scripts mapping every ROM function address to its name. Loading these as symbols in radare2 immediately resolves hundreds of function calls. Cross-referencing with public ESP-IDF example code identifies application-level patterns (HTTP handlers, WiFi callbacks) even in stripped firmware.
---
## Batch Crackme Automation via objdump Pattern Extraction (DEF CON 2017)
Solve hundreds of identical-structure crackmes by scripting `objdump` to extract comparison values and arithmetic operations, computing keys without execution.
```bash
# Simple variant: extract CMP immediates directly
objdump -M intel -d $binary | grep -P "cmp\s+rdi" | \
grep -oP "0x\w{1,2}" | xxd -r -p
# Complex variant: parse add/sub/cmp chains and reverse-compute
# Each binary: series of add/sub rdi,N then cmp rdi,target
# Reverse: start from target, undo operations in reverse order
python3 <<'EOF'
import subprocess, re, glob
for binary in sorted(glob.glob("crackmes/*")):
asm = subprocess.check_output(["objdump", "-M", "intel", "-d", binary]).decode()
ops = re.findall(r'(add|sub)\s+rdi,(0x\w+)', asm)
target = int(re.search(r'cmp\s+rdi,(0x\w+)', asm).group(1), 16)
# Reverse operations
for op, val in reversed(ops):
val = int(val, 16)
target = (target - val) if op == 'add' else (target + val)
print(chr(target & 0xff), end='')
EOF
```
**Key insight:** Mass crackme challenges (100s-1000s of binaries) have identical structure with per-binary constants. Script `objdump` disassembly parsing to extract immediates and arithmetic sequences, then reverse-compute the key algebraically. No execution or emulation needed.
---
## Fork + Pipe + Dead Branch Anti-Analysis (RCTF 2017)
Binary uses fork/pipe IPC where the parent writes data and exits, child reads from pipe and continues. Key validation is in a dead branch (always-false comparison) that requires binary patching to reach.
```bash
# Detection: fork() + pipe() + read()/write() in main
# The child process reads from pipe, needs to know its own PID
# Dead branch pattern:
# cmp DWORD PTR [ebp-0xc], 0x1 ; compares 0 with 1, always false
# je real_flag_computation ; never taken
# Patch: change comparison value from 0x1 to 0x0
# Find: 83 7d f4 01 → change to: 83 7d f4 00
python3 -c "
data = open('binary','rb').read()
data = data.replace(b'\x83\x7d\xf4\x01', b'\x83\x7d\xf4\x00')
open('binary_patched','wb').write(data)
"
```
**Key insight:** Fork+pipe creates a parent-child relationship where the parent provides data and exits. Dead branches (comparisons that always evaluate to false) hide the real validation logic. `strace` reveals the fork/pipe/read pattern; patching the comparison constant reaches the hidden code path.
---
---
## Time-Locked Binary with Date-Based Key (Hack.lu 2017)
Binary reads the system date and only executes correctly on a specific date (e.g., December 21, 2012). The date constant appears in the binary as a Unix timestamp or structured date comparison.
**Detection:** Look for comparisons against large integer constants that fall in a recognizable date range (Unix timestamps: 2012 = ~1.35B, 2017 = ~1.5B). Cultural significance helps: apocalypse dates, CTF release dates, historical events.
```bash
# Set system clock to the required date
sudo date -s "2012-12-21 00:00:00"
./binary
# Or use faketime to avoid system-wide change
LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1 FAKETIME="2012-12-21 00:00:00" ./binary
# Restore system time afterward
sudo ntpdate pool.ntp.org
```
**In IDA/Ghidra:** Search for `time()` or `localtime()` calls. The struct `tm` fields to watch: `tm_year` (years since 1900), `tm_mon` (0-based), `tm_mday`.
**Key insight:** Time-based keys use culturally significant dates. Always check for date comparisons in reversed code and try setting the system clock or using faketime before attempting deeper analysis.
**References:** Hack.lu CTF 2017
---
## ARM Code in Image Pixels via UnicornJS (Hack.lu 2017)
JavaScript challenge embeds ARM bytecode in image pixel data. The image is base64-encoded in the HTML/JS source. Pixel RGBA values encode ARM instructions. A bundled UnicornJS library (ARM CPU emulator in JavaScript) extracts and executes the bytecode.
**Identification flow:**
1. Find base64 blob in JS source → decode → PNG/BMP file
2. Identify UnicornJS import (`unicorn.js`, `uc.js`, or similar) → confirms ARM emulation
3. Pixel extraction loop: RGBA bytes concatenated in raster order form the ARM instruction stream
4. Feed the extracted bytes to an ARM disassembler
```python
from PIL import Image
import capstone
img = Image.open('decoded.png').convert('RGBA')
pixels = list(img.getdata())
# Extract ARM bytecode from pixel data (4 bytes per pixel: R, G, B, A)
arm_code = bytes([channel for pixel in pixels for channel in pixel])
# Disassemble as ARM Thumb or ARM32
md = capstone.Cs(capstone.CS_ARCH_ARM, capstone.CS_MODE_THUMB)
for insn in md.disasm(arm_code, 0x0):
print(f"0x{insn.address:04x}: {insn.mnemonic} {insn.op_str}")
```
**Key insight:** Multi-layer obfuscation: ARM code in image pixels, base64 encoded, emulated via UnicornJS at runtime. Identify the emulator library first to know which ISA to reverse — the library name reveals the architecture.
**References:** Hack.lu CTF 2017
---
## x86 16-bit MBR psadbw Constraint Solving (CSAW 2017)
Bootable MBR uses SSE2 `psadbw` (Packed Sum of Absolute Differences of Bytes) on xmm registers to validate the flag. Each iteration masks 2 input bytes, computes `psadbw` against known constants, and compares the sum to an expected value.
**`psadbw` semantics:**
```asm
psadbw xmm0, xmm1
; For each of 8 byte pairs: sum += |xmm0[i] - xmm1[i]|
; Result stored as 16-bit integer in low qword of xmm0
```
This generates sum-of-absolute-differences equations:
```text
|a[0] - k[0]| + |a[1] - k[1]| + ... + |a[7] - k[7]| = C
```
**Solution approach:**
```python
import numpy as np
from itertools import product
# For each 2-byte masked group, extract the constants and expected sum
# Equations are not purely linear (absolute value), but printable ASCII
# constrains each byte to [0x20, 0x7e], limiting brute-force space
def solve_psadbw_group(known_constants, expected_sum, printable_range=(0x20, 0x7e)):
"""Brute-force 2 unknown bytes given sum-of-abs-diff constraint."""
solutions = []
for a, b in product(range(*printable_range), repeat=2):
pair = [a, b]
sad = sum(abs(pair[i] - known_constants[i]) for i in range(len(pair)))
if sad == expected_sum:
solutions.append(bytes([a, b]))
return solutions
# For ambiguous cases with multiple solutions: apply additional constraints
# (flag format prefix, character frequency, subsequent iterations)
```
**Key insight:** `psadbw` creates sum-of-absolute-difference equations — not purely linear but solvable with constrained brute-force when bytes are limited to printable ASCII. Each 2-byte group is independent, keeping the search space to 95^2 = ~9000 candidates per group.
**References:** CSAW CTF 2017
---
## TensorFlow DNN Inversion by Inverting Sigmoid Layers (N1CTF 2018)
**Pattern:** Binary implements a 5-layer deep neural network with sigmoid activation. The input (flag characters) is transformed as `1.0/char_value` before feeding into the network. Extract weights and biases from the binary, then compute the inverse layer-by-layer: apply inverse-sigmoid, subtract bias, multiply by weight matrix inverse.
```python
import numpy as np
def sigmoid_inv(x):
return -np.log(1.0/x - 1.0)
# Invert layer by layer from output to input
v = target_output
for i in range(num_layers - 1, -1, -1):
v = np.dot(sigmoid_inv(v) - biases[i], np.linalg.inv(weights[i]))
# Input was 1.0/char, so flag chars are the multiplicative inverse
flag = ''.join(chr(int(round(1.0 / v[j]))) for j in range(len(v)))
```
**Key insight:** Neural networks with invertible activation functions (sigmoid, tanh) and square weight matrices can be mathematically inverted layer-by-layer. Apply inverse-sigmoid, subtract bias, multiply by weight inverse. Watch for input transformations (e.g., 1/x) that must also be inverted.
**Detection:** Binary with TensorFlow or custom DNN implementation. Look for sigmoid/tanh calls, matrix multiplications, and hardcoded float arrays (weights/biases) in `.rodata`. Square weight matrices (N x N) indicate the network is invertible.
**References:** N1CTF 2018
---
## BPF Filter Analysis via JIT Compilation to x64 Assembly (Midnight Sun CTF 2018)
**Pattern:** Binary creates a raw socket with a BPF (Berkeley Packet Filter) attached. When standard BPF disassemblers fail to produce readable output, enable the kernel's BPF JIT compiler to convert BPF bytecode to native x64 assembly, then read the compiled code from dmesg.
```bash
# Enable BPF JIT compilation
echo 1 > /proc/sys/net/core/bpf_jit_enable
# Run the binary, then read JIT-compiled BPF from kernel log
dmesg | grep -A 100 "flen="
# Analysis revealed: expects DNS TXT query on UDP port 3333
dig @target -p 3333 'M4d!bKn3~l' TXT
```
**Key insight:** Linux can JIT-compile BPF filters to native x64 machine code. When standard BPF disassemblers fail or produce unreadable output, enable `bpf_jit_enable` and read the compiled assembly from dmesg. The native code is often easier to understand than BPF bytecode.
**Detection:** Binary using `setsockopt` with `SO_ATTACH_FILTER`, raw socket creation (`socket(AF_PACKET, ...)`), or embedded `struct sock_fprog` structures. BPF programs appear as arrays of `struct sock_filter` (8 bytes each: opcode, jt, jf, k).
**References:** Midnight Sun CTF 2018
---
See also: [patterns-ctf.md](patterns-ctf.md) for Part 1, [patterns-ctf-2.md](patterns-ctf-2.md) for Part 2 (multi-layer self-decrypting binary, embedded ZIP+XOR license, stack string deobfuscation, prefix hash brute-force, CVP/LLL lattice, decision tree obfuscation, GF(2^8) Gaussian elimination).

532
reverse-engineering/patterns-ctf.md Normal file
View File

@@ -0,0 +1,532 @@
# CTF Reverse - Competition-Specific Patterns (Part 1)
## Table of Contents
- [Hidden Emulator Opcodes + LD_PRELOAD Key Extraction (0xFun 2026)](#hidden-emulator-opcodes--ld_preload-key-extraction-0xfun-2026)
- [Spectre-RSB SPN Cipher — Static Parameter Extraction (0xFun 2026)](#spectre-rsb-spn-cipher--static-parameter-extraction-0xfun-2026)
- [Image XOR Mask Recovery via Smoothness (VuwCTF 2025)](#image-xor-ma[REDACTED])
- [Shellcode in Data Section via mmap RWX (VuwCTF 2025)](#shellcode-in-data-section-via-mmap-rwx-vuwctf-2025)
- [Recursive execve Subtraction (VuwCTF 2025)](#recursive-execve-subtraction-vuwctf-2025)
- [Byte-at-a-Time Block Cipher Attack (UTCTF 2024)](#byte-at-a-time-block-cipher-attack-utctf-2024)
- [Mathematical Convergence Bitmap (EHAX 2026)](#mathematical-convergence-bitmap-ehax-2026)
- [Windows PE XOR Bitmap Extraction + OCR (srdnlenCTF 2026)](#windows-pe-xor-bitmap-extraction--ocr-srdnlenctf-2026)
- [Two-Stage Loader: RC4 Gate + VM Constraints (srdnlenCTF 2026)](#two-stage-loader-rc4-gate--vm-constraints-srdnlenctf-2026)
- [Kernel Module Maze Solving (DiceCTF 2026)](#kernel-module-maze-solving-dicectf-2026)
- [Multi-Threaded VM with Channel Synchronization (DiceCTF 2026)](#multi-threaded-vm-with-channel-synchronization-dicectf-2026)
- [Backdoored Shared Library Detection via String Diffing (Hack.lu CTF 2012)](#backdoored-shared-library-detection-via-string-diffing-hacklu-ctf-2012)
- [Custom binfmt Kernel Module with RC4 Flat Binaries (BSidesSF 2026)](#custom-binfmt-kernel-module-with-rc4-flat-binaries-bsidessf-2026)
- [Hash-Resolved Imports / No-Import Ransomware (BSidesSF 2026)](#hash-resolved-imports--no-import-ransomware-bsidessf-2026)
- [ELF Section Header Corruption for Anti-Analysis (BSidesSF 2026)](#elf-section-header-corruption-for-anti-analysis-bsidessf-2026)
---
## Hidden Emulator Opcodes + LD_PRELOAD Key Extraction (0xFun 2026)
**Pattern (CHIP-8):** Non-standard opcode `FxFF` triggers hidden `superChipRendrer()` → AES-256-CBC decryption. Key derived from binary constants.
**Technique:**
1. Check all instruction dispatch branches for non-standard opcodes
2. Hidden opcode may trigger crypto functions (OpenSSL)
3. Use `LD_PRELOAD` hook on `EVP_DecryptInit_ex` to capture AES key at runtime:
```c
#include <openssl/evp.h>
int EVP_DecryptInit_ex(EVP_CIPHER_CTX *ctx, const EVP_CIPHER *type,
ENGINE *impl, const unsigned char *key,
const unsigned char *iv) {
// Log key
for (int i = 0; i < 32; i++) printf("%02x", key[i]);
printf("\n");
// Call original
return ((typeof(EVP_DecryptInit_ex)*)dlsym(RTLD_NEXT, "EVP_DecryptInit_ex"))
(ctx, type, impl, key, iv);
}
```
```bash
gcc -shared -fPIC -ldl -lssl hook.c -o hook.so
LD_PRELOAD=./hook.so ./emulator rom.ch8
```
---
## Spectre-RSB SPN Cipher — Static Parameter Extraction (0xFun 2026)
**Pattern:** Binary uses cache side channels to implement S-boxes, but ALL cipher parameters (round keys, S-box tables, permutation) are in the binary's data section.
**Key insight:** Don't try to run on special hardware. Extract parameters statically:
- 8 S-boxes × 8 output bits, 256 entries each
- Values `0x340` = bit 1, `0x100` = bit 0
- 64-byte permutation table, 8 round keys
```python
# Extract from binary data section
import struct
sbox = [[0]*256 for _ in range(8)]
for i in range(8):
for j in range(256):
val = struct.unpack('<I', data[sbox_offset + (i*256+j)*4 : ...])[0]
sbox[i][j] = 1 if val == 0x340 else 0
```
**Lesson:** Side-channel implementations embed lookup tables in memory. Extract statically.
---
## Image XOR Mask Recovery via Smoothness (VuwCTF 2025)
**Pattern (Trianglification):** Image divided into triangle regions, each XOR-encrypted with `key = (mask * x - y) & 0xFF` where mask is unknown (0-255).
**Recovery:** Natural images have smooth gradients. Brute-force mask (256 values per region), score by neighbor pixel differences:
```python
import numpy as np
from PIL import Image
img = np.array(Image.open('encrypted.png'))
def score_smoothness(region_pixels, mask, positions):
decrypted = []
for (x, y), pixel in zip(positions, region_pixels):
key = (mask * x - y) & 0xFF
decrypted.append(pixel ^ key)
# Score: sum of absolute differences between adjacent pixels
return -sum(abs(decrypted[i] - decrypted[i+1]) for i in range(len(decrypted)-1))
for region in regions:
best_mask = max(range(256), key=lambda m: score_smoothness(region, m, positions))
```
**Search space:** 256 candidates × N regions = trivial. Smoothness is a reliable scoring metric for natural images.
---
## Shellcode in Data Section via mmap RWX (VuwCTF 2025)
**Pattern (Missing Function):** Binary relocates data to RWX memory (mmap with PROT_READ|PROT_WRITE|PROT_EXEC) and jumps to it.
**Detection:** Look for `mmap` with PROT_EXEC flag. Embedded shellcode often uses XOR with rotating key.
**Analysis:** Extract data section, apply XOR key (try 3-byte rotating), disassemble result.
---
## Recursive execve Subtraction (VuwCTF 2025)
**Pattern (String Inspector):** Binary recursively calls itself via `execve`, subtracting constants each time.
**Solution:** Find base case and work backward. Often a mathematical relationship like `N * M + remainder`.
---
## Byte-at-a-Time Block Cipher Attack (UTCTF 2024)
**Pattern (PES-128):** First output byte depends only on first input byte (no diffusion).
**Attack:** For each position, try all 256 byte values, compare output byte with target ciphertext. One match per byte = full plaintext recovery without knowing the key.
**Detection:** Change one input byte → only corresponding output byte changes. This means zero cross-byte diffusion = trivially breakable.
---
## Mathematical Convergence Bitmap (EHAX 2026)
**Pattern (Compute It):** Binary classifies complex-plane coordinates by Newton's method convergence. The classification results, arranged as a grid, spell out the flag in ASCII art.
**Recognition:**
- Input file with coordinate pairs (x, y)
- Binary iterates a mathematical function (e.g., z^3 - 1 = 0) and outputs pass/fail
- Grid dimensions hinted by point count (e.g., 2600 = 130×20)
- 5-pixel-high ASCII art font common in CTFs
**Newton's method for z^3 - 1:**
```python
def newton_converges_to_one(px, py, max_iter=50, target_count=12):
"""Returns True if Newton's method converges to z=1 in exactly target_count steps."""
x, y = px, py
count = 0
for _ in range(max_iter):
f_real = x**3 - 3*x*y**2 - 1.0
f_imag = 3*x**2*y - y**3
J_rr = 3.0 * (x**2 - y**2)
J_ri = 6.0 * x * y
det = J_rr**2 + J_ri**2
if det < 1e-9:
break
x -= (f_real * J_rr + f_imag * J_ri) / det
y -= (f_imag * J_rr - f_real * J_ri) / det
count += 1
if abs(x - 1.0) < 1e-6 and abs(y) < 1e-6:
break
return count == target_count
# Read coordinates and render bitmap
points = [(float(x), float(y)) for x, y in ...]
bits = [1 if newton_converges_to_one(px, py) else 0 for px, py in points]
WIDTH = 130 # 2600 / 20 rows
for r in range(len(bits) // WIDTH):
print(''.join('#' if bits[r*WIDTH+c] else '.' for c in range(WIDTH)))
```
**Key insight:** The binary is a mathematical classifier, not a flag checker. The flag is in the visual pattern of classifications, not in the binary's output. Reverse-engineer the math, apply to all coordinates, and visualize as bitmap.
---
## Windows PE XOR Bitmap Extraction + OCR (srdnlenCTF 2026)
**Pattern (Artistic Warmup):** Binary renders input text, compares rendered bitmap against expected pixel data stored XOR'd with constant in `.rdata`. No need to compute — extract expected pixels directly.
**Attack:**
1. Reverse the core check function to identify rendering and comparison logic
2. Find the expected pixel blob in `.rdata` (look for large data block referenced near comparison)
3. XOR with constant (e.g., 0xAA) to recover expected rendered DIB
4. Save as image and OCR to recover flag text
```python
import numpy as np
from PIL import Image
with open("binary.exe", "rb") as f:
data = f.read()
# Extract from .rdata section (offsets from reversing)
blob_offset = 0xC3620 # .rdata offset to XOR'd blob
blob_size = 0x15F90 # 450 * 50 * 4 (BGRA)
blob = np.frombuffer(data[blob_offset:blob_offset + blob_size], dtype=np.uint8)
expected = blob ^ 0xAA # XOR with constant key
# Reshape as BGRA image (dimensions from reversing)
img = expected.reshape(50, 450, 4)
channel = img[:, :, 0] # Take one channel (grayscale text)
Image.fromarray(channel, "L").save("target.png")
# OCR with charset whitelist
import subprocess
result = subprocess.run(
["tesseract", "target.png", "stdout", "-c",
"tessedit_char_whitelist=abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789{}_"],
capture_output=True, text=True)
print(result.stdout)
```
**Key insight:** When a binary renders text and compares pixels, the expected pixel data is the flag rendered as an image. Extract it directly from the binary data section without needing to understand the rendering logic. OCR with charset whitelist improves accuracy for CTF flag characters.
---
## Two-Stage Loader: RC4 Gate + VM Constraints (srdnlenCTF 2026)
**Pattern (Cornflake v3.5):** Two-stage malware loader — stage 1 uses RC4 username gate, stage 2 downloaded from C2 contains VM-based password validation.
**Stage 1 — RC4 username recovery:**
```python
def rc4(key, data):
s = list(range(256))
j = 0
for i in range(256):
j = (j + s[i] + key[i % len(key)]) & 0xFF
s[i], s[j] = s[j], s[i]
i = j = 0
out = bytearray()
for b in data:
i = (i + 1) & 0xFF
j = (j + s[i]) & 0xFF
s[i], s[j] = s[j], s[i]
out.append(b ^ s[(s[i] + s[j]) & 0xFF])
return bytes(out)
# Key from binary strings, ciphertext from stored hex
username = rc4(b"s3cr3t_k3y_v1", bytes.fromhex("46f5289437bc009c17817e997ae82bfbd065545d"))
```
**Stage 2 — VM constraint extraction:**
1. Download stage 2 from C2 endpoint (e.g., `/updates/check.php`)
2. Reverse VM bytecode interpreter (typically 15-20 opcodes)
3. Extract linear equality constraints over flag characters
4. Solve constraint system (Z3 or manual)
**Key insight:** Multi-stage loaders often use simple crypto (RC4) for the first gate and more complex validation (custom VM) for the second. The VM memory may be uninitialized (all zeros), drastically simplifying constraint extraction since memory-dependent operations become constants.
---
## Kernel Module Maze Solving (DiceCTF 2026)
**Pattern (Explorer):** Rust kernel module implements a 3D maze via `/dev/challenge` ioctls. Navigate the maze, avoid decoy exits (status=2), find the real exit (status=1), read the flag.
**Ioctl enumeration:**
| Command | Description |
|---------|-------------|
| `0x80046481-83` | Get maze dimensions (3 axes, 8-16 each) |
| `0x80046485` | Get status: 0=playing, 1=WIN, 2=decoy |
| `0x80046486` | Get wall bitfield (6 directions) |
| `0x80406487` | Get flag (64 bytes, only when status=1) |
| `0x40046488` | Move in direction (0-5) |
| `0x6489` | Reset position |
**DFS solver with decoy avoidance:**
```c
// Minimal static binary using raw syscalls (no libc) for small upload size
// gcc -nostdlib -static -Os -fno-builtin -o solve solve.c -Wl,--gc-sections && strip solve
int visited[16][16][16];
int bad[16][16][16]; // decoy positions across resets
void dfs(int fd, int x, int y, int z) {
if (visited[x][y][z] || bad[x][y][z]) return;
visited[x][y][z] = 1;
int status = ioctl_get_status(fd);
if (status == 1) { read_flag(fd); exit(0); }
if (status == 2) { bad[x][y][z] = 1; return; } // decoy — mark bad
int walls = ioctl_get_walls(fd);
int dx[] = {1,-1,0,0,0,0}, dy[] = {0,0,1,-1,0,0}, dz[] = {0,0,0,0,1,-1};
int opp[] = {2,3,0,1,5,4}; // opposite directions for backtracking
for (int dir = 0; dir < 6; dir++) {
if (!(walls & (1 << dir))) continue; // wall present
ioctl_move(fd, dir);
dfs(fd, x+dx[dir], y+dy[dir], z+dz[dir]);
ioctl_move(fd, opp[dir]); // backtrack
}
}
// After decoy hit: reset via ioctl 0x6489, clear visited, re-run DFS
```
**Remote deployment:** Upload binary via base64 chunks over netcat shell, decode, execute.
**Key insight:** For kernel module challenges, injecting test binaries into initramfs and probing ioctls dynamically is faster than static RE of stripped kernel modules. Keep solver binary minimal (raw syscalls, no libc) for fast upload.
---
## Multi-Threaded VM with Channel Synchronization (DiceCTF 2026)
**Pattern (locked-in):** Custom stack-based VM runs 16 concurrent threads verifying a 30-char flag. Threads communicate via futex-based channels. Pipeline: input → XOR scramble → transformation → base-4 state machine → final check.
**Analysis approach:**
1. **Identify thread roles** by tracing channel read/write patterns in GDB
2. **Extract constants** (XOR scramble values, lookup tables) via breakpoints on specific opcodes
3. **Watch for inverted logic:** validity check returns 0 for valid, non-zero for blocked (opposite of intuition)
4. **Detect futex quirks:** `unlock_pi` on unowned mutex returns EPERM=1, which can change all computations
**BFS state space search for constrained state machines:**
```python
from collections import deque
def solve_flag(scramble_vals, lookup_table, initial_state, target_state):
"""BFS through state machine to find valid flag bytes."""
flag = [None] * 30
# Known prefix/suffix from flag format
flag[0:5] = list(b'dice{')
flag[29] = ord('}')
# For each unknown position, try all printable ASCII
states = {initial_state}
for pos in range(28, 4, -1): # processed in reverse
next_states = {}
for state in states:
for ch in range(32, 127):
transformed = transform(ch, scramble_vals[pos])
digits = to_base4(transformed)
new_state = apply_digits(state, digits, lookup_table)
if new_state is not None: # valid path exists
next_states.setdefault(new_state, []).append((state, ch))
states = set(next_states.keys())
# Trace back from target_state to recover flag
```
**Key insight:** Multi-threaded VMs require tracing data flow across thread boundaries. Channel-based communication creates a pipeline — identify each thread's role (input, transform, validate, output) by watching which channels it reads/writes. Constants that affect computation may come from unexpected sources (futex return values, thread IDs).
---
## Backdoored Shared Library Detection via String Diffing (Hack.lu CTF 2012)
**Pattern (Zombie Lockbox):** A setuid binary uses `strcmp` for password validation. The expected password is visible via `strings` and works under GDB (which drops suid), but fails when run normally. The binary links against a non-standard libc that patches function behavior based on suid status.
**Detection steps:**
1. Check for non-standard library paths with `ldd`:
```bash
ldd ./binary
# Suspicious: libc.so.6 => /lib/libc/libc.so.6 (non-standard path)
# Normal: libc.so.6 => /lib32/libc.so.6
```
2. Diff strings between the suspicious and system libc:
```bash
strings /lib/libc/libc.so.6 > suspicious_strings
strings /lib32/libc-2.15.so > normal_strings
diff suspicious_strings normal_strings
```
3. Disassemble the patched function (e.g., `puts`) to find injected code:
```bash
gdb /lib/libc/libc.so.6
(gdb) disas puts
# Look for unexpected calls or branches
# Injected code may check suid status (getuid/geteuid syscalls)
# and swap the expected password at runtime
```
**Key insight:** When a binary behaves differently under GDB vs. normal execution, check `ldd` for non-standard library paths. Suid binaries drop privileges under debuggers, so a backdoored libc can detect this via `getuid`/`geteuid` syscalls and change program behavior accordingly. The `strings | diff` approach quickly reveals injected data without full disassembly.
---
---
## Custom binfmt Kernel Module with RC4 Flat Binaries (BSidesSF 2026)
**Pattern (Private Binary):** A custom Linux kernel module (`.ko`) registers a `binfmt` handler for non-standard binary formats. When a file with a specific magic number is executed, the kernel module intercepts it, decrypts the contents in memory, and jumps to the entry point.
**Reverse engineering approach:**
1. **Analyze the `.ko`:** Look for `register_binfmt()` call — it registers a `struct linux_binfmt` with a `load_binary` callback
2. **Find the magic number:** The `load_binary` function checks the file's first bytes against a specific magic number to identify its format
3. **Extract the encryption key:** Look for `movabs` instructions loading 8-byte constants — these are often RC4 key bytes
4. **Identify the encryption scheme:** Common choices are RC4, XOR, or AES-ECB. RC4 is identifiable by the S-box initialization loop (256-byte array, swap pattern)
5. **Decrypt the flat binary:** Apply the recovered key to the encrypted file contents, skipping any header
```python
from Crypto.Cipher import ARC4
# Extract RC4 key from kernel module (found via movabs instructions)
key = bytes([0x41, 0x42, 0x43, ...]) # Key bytes from .ko disassembly
with open('encrypted.bin', 'rb') as f:
header = f.read(HEADER_SIZE) # Skip binfmt header
encrypted = f.read()
cipher = ARC4.new(key)
decrypted = cipher.decrypt(encrypted)
# The decrypted output is a flat binary (no ELF headers)
# Load at the fixed virtual address specified in the kernel module
# Disassemble with: objdump -b binary -m i386:x86-64 -D decrypted.bin
# Or in Ghidra: import as "Raw Binary", set base address from .ko
```
**Detection in kernel module:**
- `register_binfmt` / `unregister_binfmt` calls
- `vm_mmap()` or `vm_brk()` for memory allocation at fixed addresses
- Direct jump to mapped memory (entry point execution)
- S-box initialization pattern (RC4): loop 0-255, swap `S[i]` with `S[j]`
**Key insight:** The flat binary has no ELF headers, so standard tools won't recognize it. You must extract the load address from the kernel module (look for the `vm_mmap` call's address argument) and import the decrypted blob at that address in your disassembler. RC4 keys in kernel modules are often stored as immediate values in `mov` or `movabs` instructions rather than in data sections.
**References:** BSidesSF 2026 "Private Binary"
---
## Hash-Resolved Imports / No-Import Ransomware (BSidesSF 2026)
**Pattern (Ran Somewhere):** Malware binary has zero visible imports — all API calls are resolved at runtime by hashing symbol names and comparing against pre-computed hash values. The binary uses `dlopen` + a custom hash table to find libc and libcrypto functions.
**Identification:**
- `readelf -d` shows no dynamic symbols or very few (just `dlopen`/`dlsym`)
- Strings reveal no standard API names
- Disassembly shows hash computation loops followed by indirect calls
- RC4-encrypted embedded strings (RSA public key, file paths, passphrases)
**Analysis shortcut — LD_PRELOAD key extraction:**
Rather than reversing the full hash resolution and key derivation, hook the crypto functions that the malware ultimately calls:
```c
// hook_crypto.c — captures AES key used by the ransomware
#define _GNU_SOURCE
#include <dlfcn.h>
#include <openssl/evp.h>
#include <stdio.h>
int EVP_CipherInit_ex(EVP_CIPHER_CTX *ctx, const EVP_CIPHER *type,
ENGINE *impl, const unsigned char *key,
const unsigned char *iv) {
if (key) {
FILE *f = fopen("/tmp/aes_key.bin", "wb");
fwrite(key, 1, 32, f); // AES-256
fclose(f);
fprintf(stderr, "[HOOK] AES key captured\n");
}
typedef int (*orig_t)(EVP_CIPHER_CTX*, const EVP_CIPHER*, ENGINE*,
const unsigned char*, const unsigned char*);
orig_t orig = (orig_t)dlsym(RTLD_NEXT, "EVP_CipherInit_ex");
return orig(ctx, type, impl, key, iv);
}
```
```bash
# Compile and run
gcc -shared -fPIC -o hook.so hook_crypto.c -ldl
# Run in Docker container (ransomware may be destructive!)
docker run --rm -v $(pwd):/work -w /work ubuntu:22.04 \
bash -c "LD_PRELOAD=./hook.so ./ransomware; xxd /tmp/aes_key.bin"
```
**Hash resolution patterns:**
- **SipHash variant:** Two 64-bit seeds, iterative mixing with symbol name bytes
- **DJB2/FNV variants:** Simpler hash functions with recognizable constants (`5381`, `0xcbf29ce484222325`)
- **ROR13-based:** Windows malware favorite: `hash = (hash >> 13) | (hash << 19); hash += c`
**Decryption after key capture:**
```python
from Crypto.Cipher import AES
key = open('/tmp/aes_key.bin', 'rb').read()
iv = open('/tmp/aes_iv.bin', 'rb').read() # Also hookable
cipher = AES.new(key, AES.MODE_CBC, iv)
with open('flag.txt.enc', 'rb') as f:
ct = f.read()
pt = cipher.decrypt(ct)
# Remove PKCS7 padding
pt = pt[:-pt[-1]]
print(pt.decode())
```
**Key insight:** When a binary resolves all imports via hashing, don't waste time reversing the hash function and building a rainbow table. Instead, let the malware resolve everything itself by running it in a sandboxed environment with `LD_PRELOAD` hooks on the functions you care about (OpenSSL crypto functions, file I/O, network calls). The AES key is deterministic across runs — if it works once, it works always.
**Safety:** Always run suspected ransomware in a Docker container or VM. Mount only copies of the encrypted files, never originals.
**References:** BSidesSF 2026 "Ran Somewhere"
---
## ELF Section Header Corruption for Anti-Analysis (BSidesSF 2026)
**Pattern (stubborn-elf):** An ELF binary has deliberately corrupted section header table entries, causing standard analysis tools (`readelf`, `objdump`, IDA, Ghidra) to crash or produce errors. However, the **program headers** (which the OS loader uses) are intact, so the binary executes normally. The flag is appended after the corrupted sections, marked with magic bytes.
```python
import sys
# Standard tools fail on corrupted section headers
# Manual parsing bypasses section headers entirely
with open("stubborn_elf", "rb") as f:
data = f.read()
# Search for magic marker appended after ELF sections
magic = b"\xDE\xAD\xBE\xEF\xCA\xFE\xBA\xBE"
idx = data.find(magic)
if idx >= 0:
# Data after magic is XOR-encrypted
encrypted = data[idx + len(magic):]
decrypted = bytes(b ^ 0x42 for b in encrypted)
print(decrypted.decode(errors='ignore'))
```
**Key insight:** ELF execution requires **program headers** (PT_LOAD segments), NOT section headers. Section headers are metadata for debuggers and analysis tools — they're optional at runtime. Corrupting `e_shoff`, `e_shnum`, or `e_shstrndx` in the ELF header breaks tools but not execution. When tools fail, parse the binary manually or patch the ELF header to zero out section header references before loading in a disassembler.
**Recovery approach:**
```bash
# Patch section header offset to 0 (removes section table)
printf '\x00\x00\x00\x00\x00\x00\x00\x00' | dd of=binary bs=1 seek=40 conv=notrunc
# Now Ghidra/IDA can load it using program headers only
# Or use readelf -l (program headers only, ignores sections)
readelf -l stubborn_elf
```
**When to recognize:** `readelf -S` crashes or shows garbage. `file` command identifies it as ELF. `readelf -l` (lowercase L, program headers) works fine. The binary runs normally despite tool failures.
**References:** BSidesSF 2026 "stubborn-elf"
---
See also: [patterns-ctf-2.md](patterns-ctf-2.md) for Part 2 (multi-layer self-decrypting binary, embedded ZIP+XOR license, stack string deobfuscation, prefix hash brute-force, CVP/LLL lattice, decision tree obfuscation, GF(2^8) Gaussian elimination), [patterns-ctf-3.md](patterns-ctf-3.md) for Part 3 (Z3 boolean circuit, sliding window popcount, keyboard LED Morse code, C++ destructor-hidden validation, VM sequential key-chain brute-force, BWT inversion, OpenType font ligature exploitation, GLSL shader VM with self-modifying code).

801
reverse-engineering/patterns.md Normal file
View File

@@ -0,0 +1,801 @@
# CTF Reverse - Patterns & Techniques
## Table of Contents
- [Custom VM Reversing](#custom-vm-reversing)
- [Analysis Steps](#analysis-steps)
- [Common VM Patterns](#common-vm-patterns)
- [RVA-Based Opcode Dispatching](#rva-based-opcode-dispatching)
- [State Machine VMs (90K+ states)](#state-machine-vms-90k-states)
- [Custom VM Reverse Engineering via Fuzzing and Instruction Set Discovery (hxp CTF 2017)](#custom-vm-reverse-engineering-via-fuzzing-and-instruction-set-discovery-hxp-ctf-2017)
- [Anti-Debugging Techniques](#anti-debugging-techniques)
- [Common Checks](#common-checks)
- [Bypass Technique](#bypass-technique)
- [LD_PRELOAD Hook](#ld_preload-hook)
- [pwntools Binary Patching (Crypto-Cat)](#pwntools-binary-patching-crypto-cat)
- [Nanomites](#nanomites)
- [Linux (Signal-Based)](#linux-signal-based)
- [Windows (Debug Events)](#windows-debug-events)
- [Analysis](#analysis)
- [Self-Modifying Code](#self-modifying-code)
- [Pattern: XOR Decryption](#pattern-xor-decryption)
- [Known-Plaintext XOR (Flag Prefix)](#known-plaintext-xor-flag-prefix)
- [Variant: XOR with Position Index](#variant-xor-with-position-index)
- [Mixed-Mode (x86-64 / x86) Stagers](#mixed-mode-x86-64--x86-stagers)
- [LLVM (Low Level Virtual Machine) Obfuscation (Control Flow Flattening)](#llvm-low-level-virtual-machine-obfuscation-control-flow-flattening)
- [Pattern](#pattern)
- [De-obfuscation](#de-obfuscation)
- [S-Box / Keystream Generation](#s-box--keystream-generation)
- [Fisher-Yates Shuffle (Xorshift32)](#fisher-yates-shuffle-xorshift32)
- [Xorshift64* Keystream](#xorshift64-keystream)
- [Identifying Patterns](#identifying-patterns)
- [SECCOMP/BPF Filter Analysis](#seccompbpf-filter-analysis)
- [BPF Analysis](#bpf-analysis)
- [Exception Handler Obfuscation](#exception-handler-obfuscation)
- [RtlInstallFunctionTableCallback](#rtlinstallfunctiontablecallback)
- [Vectored Exception Handlers (VEH)](#vectored-exception-handlers-veh)
- [Memory Dump Analysis](#memory-dump-analysis)
- [When Binary Dumps Memory](#when-binary-dumps-memory)
- [Known Plaintext Attack](#known-plaintext-attack)
- [Byte-Wise Uniform Transforms](#byte-wise-uniform-transforms)
- [x86-64 Gotchas](#x86-64-gotchas)
- [Sign Extension](#sign-extension)
- [Loop Boundary State Updates](#loop-boundary-state-updates)
- [Custom Mangle Function Reversing](#custom-mangle-function-reversing)
- [Position-Based Transformation Reversing](#position-based-transformation-reversing)
- [Hex-Encoded String Comparison](#hex-encoded-string-comparison)
- [Signal-Based Binary Exploration](#signal-based-binary-exploration)
- [Malware Anti-Analysis Bypass via Patching](#malware-anti-analysis-bypass-via-patching)
- [Multi-Stage Shellcode Loaders](#multi-stage-shellcode-loaders)
- [Timing Side-Channel Attack](#timing-side-channel-attack)
- [Multi-Thread Anti-Debug with Decoy + Signal Handler Mixed Boolean-Arithmetic (ApoorvCTF 2026)](#multi-thread-anti-debug-with-decoy--signal-handler-mixed-boolean-arithmetic-apoorvctf-2026)
- [INT3 Patch + Coredump Brute-Force Oracle (Pwn2Win 2016)](#int3-patch--coredump-brute-force-oracle-pwn2win-2016)
- [Signal Handler Chain + LD_PRELOAD Oracle (Nuit du Hack 2016)](#signal-handler-chain--ld_preload-oracle-nuit-du-hack-2016)
- [printf Format String VM Decompilation to Z3 (SECCON 2017)](#printf-format-string-vm-decompilation-to-z3-seccon-2017)
---
## Custom VM Reversing
### Analysis Steps
1. Identify VM structure: registers, memory, instruction pointer
2. Reverse `executeIns`/`runvm` function for opcode meanings
3. Write a disassembler to parse bytecode
4. Decompile disassembly to understand algorithm
### Common VM Patterns
```c
switch (opcode) {
case 1: *R[op1] *= op2; break; // MUL
case 2: *R[op1] -= op2; break; // SUB
case 3: *R[op1] = ~*R[op1]; break; // NOT
case 4: *R[op1] ^= mem[op2]; break; // XOR
case 5: *R[op1] = *R[op2]; break; // MOV
case 7: if (R0) IP += op1; break; // JNZ
case 8: putc(R0); break; // PRINT
case 10: R0 = getc(); break; // INPUT
}
```
### RVA-Based Opcode Dispatching
- Opcodes are RVAs pointing to handler functions
- Handler performs operation, reads next RVA, jumps
- Map all handlers by following RVA chain
### State Machine VMs (90K+ states)
```java
// BFS for valid path
var agenda = new ArrayDeque<State>();
agenda.add(new State(0, ""));
while (!agenda.isEmpty()) {
var current = agenda.remove();
if (current.path.length() == TARGET_LENGTH) {
println(current.path);
continue;
}
for (var transition : machine.get(current.state).entrySet()) {
agenda.add(new State(transition.getValue(),
current.path + (char)transition.getKey()));
}
}
```
**Key insight:** Custom VMs appear when the challenge bundles a bytecode blob alongside a dispatcher loop. Reverse the opcode switch table first, then write a disassembler to lift the bytecode before attempting to understand the algorithm.
### Custom VM Reverse Engineering via Fuzzing and Instruction Set Discovery (hxp CTF 2017)
Methodical black-box approach to reversing unknown VM bytecode when static analysis of the dispatch loop is too complex:
**Step 1: Determine instruction alignment.**
Dump the bytecode as bit strings at various widths (6-11 bits) to identify instruction alignment. Look for repeating patterns that suggest opcode boundaries.
**Step 2: Fuzz with random bytes.**
Send single instructions and observe effects on registers/memory to map opcodes. Reduce to minimal programs: find the shortest input that produces each observable effect.
**Step 3: Build the instruction set.**
Example discovered ISA (variable-length 6-11 bit):
```text
000 xxxxxxxx jmpz 001 xxxxxxxx jmp 010 xxxxxxxx call
011 xxxxxxxx label 1000 xxxxxxx loadram 1001 xxxxxxx saveram
110 xxxxxxxx loadi 11100 xxxxxx shl 11101 xxxxxx shr
111100 not 111101 and 111110 or 111111 setif
```
**Step 4: Build assembler/disassembler.**
Write tools to assemble and disassemble the discovered ISA, then disassemble the challenge bytecode to understand its algorithm.
**Step 5: Implement missing primitives.**
If the ISA lacks expected operations, synthesize them from available instructions. Example: implementing XTEA decryption using only AND/OR/NOT (no native XOR or ADD):
```python
# XOR from AND/OR/NOT: XOR(a, b) = (a OR b) AND NOT(a AND b)
# ADD via full-adder chains using AND/OR/NOT for carry propagation
def xor_from_primitives(a, b):
return (a | b) & ~(a & b)
def add_from_primitives(a, b, bits=32):
carry = 0
result = 0
for i in range(bits):
ai = (a >> i) & 1
bi = (b >> i) & 1
sum_bit = xor_from_primitives(xor_from_primitives(ai, bi), carry)
carry = (ai & bi) | (carry & xor_from_primitives(ai, bi))
result |= (sum_bit << i)
return result
```
**Key insight:** When static analysis of a VM's dispatch loop is too complex, black-box fuzzing can map the ISA faster. Send single instructions and observe state changes. Variable-length instruction sets require testing multiple bit widths. Once the ISA is known, complex algorithms (XTEA) can be implemented even with minimal primitives (AND/OR/NOT).
**References:** hxp CTF 2017
---
## Anti-Debugging Techniques
### Common Checks
- `IsDebuggerPresent()` (Windows)
- `ptrace(PTRACE_TRACEME)` (Linux)
- `/proc/self/status` TracerPid
- Timing checks (`rdtsc`, `time()`)
- Registry checks (Windows)
### Bypass Technique
1. Identify `test` instructions after debug checks
2. Set breakpoint at the `test`
3. Modify register to bypass conditional
```bash
# In radare2
db 0x401234 # Break at test
dc # Run
dr eax=0 # Clear flag
dc # Continue
```
### LD_PRELOAD Hook
```c
#define _GNU_SOURCE
#include <dlfcn.h>
#include <sys/ptrace.h>
long int ptrace(enum __ptrace_request req, ...) {
long int (*orig)(enum __ptrace_request, pid_t, void*, void*);
orig = dlsym(RTLD_NEXT, "ptrace");
// Log or modify behavior
return orig(req, pid, addr, data);
}
```
Compile: `gcc -shared -fPIC -ldl hook.c -o hook.so`
Run: `LD_PRELOAD=./hook.so ./binary`
**Key insight:** Anti-debugging checks are the first obstacle in most reversing challenges. Look for `ptrace`, `IsDebuggerPresent`, or timing checks early in `main()` and patch or hook them before attempting deeper analysis.
### pwntools Binary Patching (Crypto-Cat)
Patch out anti-debug calls directly using pwntools — replaces function with `ret` instruction:
```python
from pwn import *
elf = ELF('./challenge', checksec=False)
elf.asm(elf.symbols.ptrace, 'ret') # Replace ptrace() with immediate return
elf.save('patched') # Save patched binary
```
Other common patches:
```python
elf.asm(addr, 'nop') # NOP out an instruction
elf.asm(addr, 'xor eax, eax; ret') # Return 0 (bypass checks)
elf.asm(addr, 'mov eax, 1; ret') # Return 1 (force success)
```
---
## Nanomites
### Linux (Signal-Based)
- `SIGTRAP` (`int 3`) → Custom operation
- `SIGILL` (`ud2`) → Custom operation
- `SIGFPE` (`idiv 0`) → Custom operation
- `SIGSEGV` (null deref) → Custom operation
### Windows (Debug Events)
- `EXCEPTION_DEBUG_EVENT` → Main handler
- Parent modifies child via `PTRACE_POKETEXT`
- Magic markers: `0x1337BABE`, `0xDEADC0DE`
### Analysis
1. Check for `fork()` + `ptrace(PTRACE_TRACEME)`
2. Find `WaitForDebugEvent` loop
3. Map EAX values to operations
4. Log operations to reconstruct algorithm
**Key insight:** Nanomites hide the real computation inside signal/exception handlers that only fire under a debugger parent. If the binary forks and the child calls `ptrace(TRACEME)`, the parent is the real CPU -- log its POKE operations to reconstruct the algorithm.
---
## Self-Modifying Code
### Pattern: XOR Decryption
```asm
lea rax, next_block
mov dl, [rcx] ; Input char
xor_loop:
xor [rax+rbx], dl
inc rbx
cmp rbx, BLOCK_SIZE
jnz xor_loop
jmp rax ; Execute decrypted
```
**Solution:** Known opcode at block start reveals XOR key (flag char).
**Key insight:** Self-modifying code decrypts the next block using each input character as a key. A known-good opcode at the start of each decrypted block (e.g., function prologue) reveals the correct key byte, recovering the flag one character at a time.
---
## Known-Plaintext XOR (Flag Prefix)
**Pattern:** Encrypted bytes given; flag format known (e.g., `0xL4ugh{`).
**Approach:**
1. Assume repeating XOR key.
2. Use known prefix (and any hint phrase) to recover key bytes.
3. Try small key lengths and validate printable output.
```python
enc = bytes.fromhex("...") # ciphertext
known = b"0xL4ugh{say_yes_to_me"
for klen in range(2, 33):
key = bytearray(klen)
ok = True
for i, b in enumerate(known):
if i >= len(enc):
break
ki = i % klen
v = enc[i] ^ b
if key[ki] != 0 and key[ki] != v:
ok = False
break
key[ki] = v
if not ok:
continue
pt = bytes(enc[i] ^ key[i % klen] for i in range(len(enc)))
if all(32 <= c < 127 for c in pt):
print(klen, key, pt)
```
**Note:** Challenge hints often appear verbatim in the flag body (e.g., "say_yes_to_me").
### Variant: XOR with Position Index
**Pattern:** `cipher[i] = plain[i] ^ key[i % k] ^ i` (or `^ (i & 0xff)`).
**Symptoms:**
- Repeating-key XOR almost fits known prefix but breaks at later positions
- XOR with known prefix yields a "key" that changes by +1 per index
**Fix:** Remove index first, then recover key with known prefix.
```python
enc = bytes.fromhex("...")
known = b"0xL4ugh{say_yes_to_me"
for klen in range(2, 33):
key = bytearray(klen)
ok = True
for i, b in enumerate(known):
if i >= len(enc):
break
ki = i % klen
v = (enc[i] ^ i) ^ b # strip index XOR
if key[ki] != 0 and key[ki] != v:
ok = False
break
key[ki] = v
if not ok:
continue
pt = bytes((enc[i] ^ i) ^ key[i % klen] for i in range(len(enc)))
if all(32 <= c < 127 for c in pt):
print(klen, key, pt)
```
---
## Mixed-Mode (x86-64 / x86) Stagers
**Pattern:** 64-bit ELF jumps into a 32-bit blob via far return (`retf`/`retfq`), often after anti-debug.
**Identification:**
- Bytes `0xCB` (retf) or `0xCA` (retf imm16), sometimes preceded by `0x48` (retfq)
- 32-bit disasm shows SSE ops (`psubb`, `pxor`, `paddb`) in a tight loop
- Computed jumps into the 32-bit region
**Gotchas:**
- `retf` pops **6 bytes**: 4-byte EIP + 2-byte CS (not 8)
- 32-bit blob may rely on inherited **XMM state** and **EFLAGS**
- Missing XMM/flags transfer when switching emulators yields wrong output
**Bypass/Emulation Tips:**
1. Create a UC_MODE_32 emulator, copy memory + GPRs, **EFLAGS**, and **XMM regs**
2. Run 32-bit block, then copy memory + regs back to 64-bit
3. If anti-debug uses `fork/ptrace` + patching, emulate parent to log POKEs and apply them in child
---
## LLVM (Low Level Virtual Machine) Obfuscation (Control Flow Flattening)
### Pattern
```c
while (1) {
if (i == 0xA57D3848) { /* block */ }
if (i != 0xA5AA2438) break;
i = 0x39ABA8E6; // Next state
}
```
### De-obfuscation
1. GDB script to break at `je` instructions
2. Log state variable values
3. Map state transitions
4. Reconstruct true control flow
**Key insight:** Control flow flattening replaces structured if/else/loops with a single dispatcher switch. The state variable is the key -- trace its values at runtime to reconstruct the original control flow graph without fighting the obfuscation statically.
---
## S-Box / Keystream Generation
### Fisher-Yates Shuffle (Xorshift32)
```python
def gen_sbox():
sbox = list(range(256))
state = SEED
for i in range(255, -1, -1):
state = ((state << 13) ^ state) & 0xffffffff
state = ((state >> 17) ^ state) & 0xffffffff
state = ((state << 5) ^ state) & 0xffffffff
j = state % (i + 1) if i > 0 else 0
sbox[i], sbox[j] = sbox[j], sbox[i]
return sbox
```
### Xorshift64* Keystream
```python
def gen_keystream():
ks = []
state = SEED_64
mul = 0x2545f4914f6cdd1d
for _ in range(256):
state ^= (state >> 12)
state ^= (state << 25)
state ^= (state >> 27)
state = (state * mul) & 0xffffffffffffffff
ks.append((state >> 56) & 0xff)
return ks
```
### Identifying Patterns
- Xorshift32: shifts 13, 17, 5 (no multiplication constant)
- Xorshift64*: shifts 12, 25, 27, then multiply by `0x2545f4914f6cdd1d`
- Other common constant: `0x9e3779b97f4a7c15` (golden ratio)
**Key insight:** Recognize S-box generation by the Fisher-Yates shuffle pattern (loop counting down from 255, swap with PRNG-chosen index) and keystream generators by the xorshift constants. Once the PRNG family is identified, the algorithm is fully determined by its seed.
---
## SECCOMP/BPF Filter Analysis
```bash
seccomp-tools dump ./binary
```
### BPF Analysis
- `A = sys_number` followed by comparisons
- `mem[N] = A`, `A = mem[N]` for memory ops
- Map to constraint equations, solve with z3
```python
from z3 import *
flag = [BitVec(f'c{i}', 32) for i in range(14)]
s = Solver()
s.add(flag[0] >= 0x20, flag[0] < 0x7f)
# Add constraints from filter
if s.check() == sat:
m = s.model()
print(''.join(chr(m[c].as_long()) for c in flag))
```
**Key insight:** SECCOMP (Secure Computing Mode) filters encode flag validation as BPF bytecode operating on syscall arguments. Dump the filter with `seccomp-tools`, translate the comparisons and memory operations into z3 constraints, and solve for the flag without ever running the binary.
---
## Exception Handler Obfuscation
### RtlInstallFunctionTableCallback
- Dynamic exception handler registration
- Handler installs new handler, modifies code
- Use x64dbg with exception handler breaks
### Vectored Exception Handlers (VEH)
- `AddVectoredExceptionHandler` installs handler
- Handler decrypts code at exception address
- Step through, dump decrypted code
**Key insight:** Exception-handler-based obfuscation hides the real control flow inside SEH/VEH handlers that trigger on deliberate faults. Set breakpoints inside the exception handlers rather than on the faulting instructions to follow the actual execution path.
---
## Memory Dump Analysis
### When Binary Dumps Memory
- Check for `/proc/self/maps` reads
- Check for `/proc/self/mem` reads
- Heap data often appended to dump
### Known Plaintext Attack
```python
prologue = bytes([0xf3, 0x0f, 0x1e, 0xfa, 0x55, 0x48, 0x89, 0xe5])
encrypted = data[func_offset:func_offset+8]
partial_key = bytes(a ^ b for a, b in zip(encrypted, prologue))
```
**Key insight:** When a binary reads `/proc/self/mem` or `/proc/self/maps`, it is dumping its own memory -- possibly after encrypting it. Use known function prologues (`endbr64; push rbp; mov rbp, rsp`) as known plaintext to recover the XOR key from the encrypted dump.
---
## Byte-Wise Uniform Transforms
**Pattern:** Output buffer depends on each input byte independently (no cross-byte coupling).
**Detection:**
- Change one input position → only one output position changes
- Fill input with a single byte → output buffer becomes constant
**Solve:**
1. For each byte value 0..255, run the program with that byte repeated
2. Record output byte → build mapping and inverse mapping
3. Apply inverse mapping to static target bytes to recover the flag
---
## x86-64 Gotchas
### Sign Extension
```python
esi = 0xffffffc7 # NOT -57
# For XOR: low byte only
esi_xor = esi & 0xff # 0xc7
# For addition: full 32-bit with overflow
r12 = (r13 + esi) & 0xffffffff
```
### Loop Boundary State Updates
Assembly often splits state updates across loop boundaries:
```asm
jmp loop_middle ; First iteration in middle!
loop_top: ; State for iterations 2+
mov r13, sbox[a & 0xf]
; Uses OLD 'a', not new!
loop_middle:
; Main computation
inc a
jne loop_top
```
**Key insight:** Decompilers often get x86-64 sign extension and loop boundary state updates wrong. Always verify decompiled output against the raw assembly for operations involving `movsx`/`cdqe`, and check whether loop variables update before or after their use in each iteration.
---
## Custom Mangle Function Reversing
**Pattern (Flag Appraisal):** Binary mangles input 2 bytes at a time with intermediate state, compares to static target.
**Approach:**
1. Extract static target bytes from `.rodata` section
2. Understand mangle: processes pairs with running state value
3. Write inverse function (process in reverse, undo each operation)
4. Feed target bytes through inverse → recovers flag
**Key insight:** When a binary mangles input in pairs with running state and compares to a static target, extract the target from `.rodata` and write the inverse function. Process the target bytes in reverse order, undoing each operation, to recover the original input.
---
## Position-Based Transformation Reversing
**Pattern (PascalCTF 2026):** Binary transforms input by adding/subtracting position index.
**Reversing:**
```python
expected = [...] # Extract from .rodata
flag = ''
for i, b in enumerate(expected):
if i % 2 == 0:
flag += chr(b - i) # Even: input = output - i
else:
flag += chr(b + i) # Odd: input = output + i
```
---
## Hex-Encoded String Comparison
**Pattern (Spider's Curse):** Input converted to hex, compared against hex constant.
**Quick solve:** Extract hex constant from strings/Ghidra, decode:
```bash
echo "4d65746143..." | xxd -r -p
```
---
## Signal-Based Binary Exploration
**Pattern (Signal Signal Little Star):** Binary uses UNIX signals as a binary tree navigation mechanism.
**Identification:**
- Multiple `sigaction()` calls with `SA_SIGINFO`
- `sigaltstack()` setup (alternate signal stack)
- Handler decodes embedded payload, installs next pair of signals
- Two types: Node (installs children) vs Leaf (prints message + exits)
**Solving approach:**
1. Hook `sigaction` via `LD_PRELOAD` to log signal installations
2. DFS through the binary tree by sending signals
3. At each stage, observe which 2 signals are installed
4. Send one, check if program exits (leaf) or installs 2 more (node)
5. If wrong leaf, backtrack and try sibling
```c
// LD_PRELOAD interposer to log sigaction calls
int sigaction(int signum, const struct sigaction *act, ...) {
if (act && (act->sa_flags & SA_SIGINFO))
log("SET %d SA_SIGINFO=1\n", signum);
return real_sigaction(signum, act, oldact);
}
```
---
## Malware Anti-Analysis Bypass via Patching
**Pattern (Carrot):** Malware with multiple environment checks before executing payload.
**Common checks to patch:**
| Check | Technique | Patch |
|-------|-----------|-------|
| `ptrace(PTRACE_TRACEME)` | Anti-debug | Change `cmp -1` to `cmp 0` |
| `sleep(150)` | Anti-sandbox timing | Change sleep value to 1 |
| `/proc/cpuinfo` "hypervisor" | Anti-VM | Flip `JNZ` to `JZ` |
| "VMware"/"VirtualBox" strings | Anti-VM | Flip `JNZ` to `JZ` |
| `getpwuid` username check | Environment | Flip comparison |
| `LD_PRELOAD` check | Anti-hook | Skip check |
| Fan count / hardware check | Anti-VM | Flip `JLE` to `JGE` |
| Hostname check | Environment | Flip `JNZ` to `JZ` |
**Ghidra patching workflow:**
1. Find check function, identify the conditional jump
2. Click on instruction → `Ctrl+Shift+G` → modify opcode
3. For `JNZ` (0x75) → `JZ` (0x74), or vice versa
4. For immediate values: change operand bytes directly
5. Export: press `O` → choose "Original File" format
6. `chmod +x` the patched binary
**Server-side validation bypass:**
- If patched binary sends system info to remote server, patch the data too
- Modify string addresses in data-gathering functions
- Change format strings to embed correct values directly
---
## Multi-Stage Shellcode Loaders
**Pattern (I Heard You Liked Loaders):** Nested shellcode with XOR decode loops and anti-debug.
**Debugging workflow:**
1. Break at `call rax` in launcher, step into shellcode
2. Bypass ptrace anti-debug: step to syscall, `set $rax=0`
3. Step through XOR decode loop (or break on `int3` if hidden)
4. Repeat for each stage until final payload
**Flag extraction from `mov` instructions:**
```python
# Final stage loads flag 4 bytes at a time via mov ebx, value
# Extract little-endian 4-byte chunks
values = [0x6174654d, 0x7b465443, ...] # From disassembly
flag = b''.join(v.to_bytes(4, 'little') for v in values)
```
---
## Timing Side-Channel Attack
**Pattern (Clock Out):** Validation time varies per correct character (longer sleep on match).
**Exploitation:**
```python
import time
from pwn import *
flag = ""
for pos in range(flag_length):
best_char, best_time = '', 0
for c in string.printable:
io = remote(host, port)
start = time.time()
io.sendline((flag + c).ljust(total_len, 'X'))
io.recvall()
elapsed = time.time() - start
if elapsed > best_time:
best_time = elapsed
best_char = c
io.close()
flag += best_char
```
---
## Multi-Thread Anti-Debug with Decoy + Signal Handler Mixed Boolean-Arithmetic (ApoorvCTF 2026)
**Pattern (A Golden Experience Requiem):** Multi-threaded binary with layered anti-analysis: Thread 1 performs decoy operations (fake AES + deliberate crash via `ud2`), Thread 2 does the real flag computation in a SIGSEGV signal handler using Mixed Boolean Arithmetic (MBA), Thread 3 erases memory to prevent post-mortem analysis.
**Thread layout:**
| Thread | Purpose | Trap |
|--------|---------|------|
| Thread 1 | Decoy: AES-looking operations → `ud2` crash | Analysts waste time reversing fake crypto |
| Thread 2 | Real flag: SIGSEGV handler with MBA transforms | Hidden in signal handler, not main code path |
| Thread 3 | Memory eraser: zeros out flag data after computation | Prevents memory dumping |
| Main | rdtsc-based anti-debug timing check | Penalizes debugger-attached execution |
**Solving approach — pure Python emulation of MBA logic:**
```python
# MBA helpers (extracted from assembly)
def mba_add(a, b): return (a + b) & 0xff
def mba_xor(a, b): return (a ^ b) & 0xff
def mba_transform(i):
"""Position-dependent transform from signal handler."""
val = (i * 7 + 0x3f) & 0xff
rotated = ((i << 3) | (i >> 5)) & 0xff
return mba_xor(val, rotated)
# S-box (SHA-256 initial hash values repurposed)
SBOX = [0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a,
0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19]
def sbox_lookup(i):
idx = i & 7
shift = ((i >> 3) & 3) * 8
return (SBOX[idx] >> shift) & 0xff
# Two interleaved rodata arrays (even indices → array1, odd → array2)
rodata1 = bytes.fromhex("39407691b717c97879013adf3a2adea11c2b04e0")
rodata2 = bytes.fromhex("bb19b025e37eaa786c4116e7aeea00c9c623940d")
flag = []
for i in range(40): # flag length
t = mba_transform(i)
s = sbox_lookup(i)
mem = rodata1[i // 2] if i % 2 == 0 else rodata2[i // 2]
flag.append(chr(t ^ s ^ mem))
print(''.join(flag))
```
**Key insight:** The real flag logic is in the signal handler (SIGSEGV/SIGILL), not the main thread. Thread 1's AES-like code and `ud2` crash are intentional misdirection. The `rdtsc` timing check detects debuggers and corrupts output. Bypass by extracting the MBA logic from assembly and reimplementing in Python — never run the binary under a debugger.
**Detection indicators:**
- Multiple `pthread_create` calls with different handler functions
- `signal(SIGSEGV, handler)` or `sigaction` setup
- `ud2` instruction (deliberate illegal instruction)
- `rdtsc` instructions for timing checks
- SHA-256 constants (0x6a09e667...) used as lookup tables, not for hashing
---
## INT3 Patch + Coredump Brute-Force Oracle (Pwn2Win 2016)
Instead of reversing complex transformation logic, patch a byte to `0xCC` (INT3) after the transform, enable core dumps, brute-force each character by running the binary and extracting the transformed result from the coredump via `strings`.
```bash
# Patch byte at transform output point to 0xCC
printf '\xcc' | dd of=binary bs=1 seek=$((0x400ebb)) conv=notrunc
ulimit -c unlimited
# Brute-force each position:
for c in $(seq 32 126); do
echo -ne "$(printf '\\x%02x' $c)$known_suffix" | ./binary 2>/dev/null
strings core | grep -q "$expected" && echo "Found: $c"
done
```
**Key insight:** Use INT3/SIGTRAP as a breakpoint oracle -- the coredump captures computed state at the crash point. Avoids full reverse engineering of the transformation.
---
## Signal Handler Chain + LD_PRELOAD Oracle (Nuit du Hack 2016)
Binary uses Unix signals for flow control: `main()` sends SIGINT to itself 1024 times, each handler checks one password character, then calls `signal()` to install the next handler. Bypass: LD_PRELOAD a custom `signal()` that logs when it's called (indicating correct character), brute-force each position.
```c
// LD_PRELOAD library:
#include <signal.h>
sighandler_t signal(int sig, sighandler_t handler) {
write(2, "CORRECT\n", 8); // signal() called = char was correct
return SIG_DFL;
}
```
**Key insight:** Signal-handler-chain anti-reversing can be defeated by hooking `signal()` via LD_PRELOAD. The call to `signal()` (to install the next handler) acts as a side-channel confirming the current character.
---
### printf Format String VM Decompilation to Z3 (SECCON 2017)
A "virtual machine" implemented entirely via `%hhn` format strings. Format string `%hhn` writes the count of printed characters (mod 256) to a pointed-to byte. A sequence of `%Nc%hhn` instructions implements arbitrary byte-to-memory writes, effectively creating a bytecode VM.
**Step 1: Identify instruction types.**
Count unique format patterns to determine the instruction set:
```bash
# Normalize numbers and count unique patterns
sed -e 's/[[:digit:]]\+/1/g' program.fs | sort | uniq -c | sort -nr
```
**Step 2: Write a decompiler.**
Convert format patterns to C-style pseudocode. Each `%N...%hhn` pair maps to a memory write: extract the write address (from the argument pointer) and value (from the character count).
**Step 3: Recognize the algorithm.**
The pseudocode typically reveals a linear equation system over bytes. Map memory addresses to symbolic variables.
**Step 4: Generate Z3 constraints and solve.**
```python
from z3 import *
flag_len = 32 # adjust based on decompiled output
flag = [BitVec(f'f{i}', 8) for i in range(flag_len)]
s = Solver()
# Constrain to printable ASCII
for f in flag:
s.add(f >= 0x20, f <= 0x7e)
# Add constraints from decompiled format string operations
# e.g., flag[3] + flag[7] == 0xAB (mod 256)
# These come from the write sequences: each %hhn accumulates
# character counts and writes the result to a target byte
s.add((flag[0] + flag[1]) & 0xFF == 0x9A) # example constraint
s.add((flag[2] ^ flag[3]) & 0xFF == 0x3F) # example constraint
# ... (add all constraints from decompilation)
if s.check() == sat:
m = s.model()
print(bytes([m[f].as_long() for f in flag]))
```
**Decompilation approach in detail:**
1. Extract the write address and value from each `%N...%hhn` pair
2. Map memory addresses to symbolic variables (flag bytes)
3. Build an equation system from the write sequences
4. Solve with Z3
**Key insight:** Format string `%hhn` writes the count of printed characters (mod 256) to a pointed-to byte. A sequence of `%Nc%hhn` instructions implements arbitrary byte-to-memory writes, effectively creating a bytecode VM. Decompile by: (1) extract the write address and value from each `%N...%hhn` pair, (2) map memory addresses to symbolic variables, (3) build an equation system from the write sequences, (4) solve with Z3.
**References:** SECCON 2017

304
reverse-engineering/platforms-hardware.md Normal file
View File

@@ -0,0 +1,304 @@
# CTF Reverse - Hardware and Advanced Architecture Reversing
HD44780 LCD GPIO reconstruction, RISC-V advanced extensions and debugging, ARM64/AArch64 reversing and exploitation.
## Table of Contents
- [HD44780 LCD Controller GPIO Reconstruction (32C3 2015)](#hd44780-lcd-controller-gpio-reconstruction-32c3-2015)
- [RISC-V (Advanced)](#risc-v-advanced)
- [Custom Extensions](#custom-extensions)
- [Privileged Modes](#privileged-modes)
- [RISC-V Debugging](#risc-v-debugging)
- [ARM64/AArch64 Reversing and Exploitation](#arm64aarch64-reversing-and-exploitation)
- [MIPS64 Cavium OCTEON Coprocessor 2 Crypto (SEC-T CTF 2017)](#mips64-cavium-octeon-coprocessor-2-crypto-sec-t-ctf-2017)
- [EFM32 ARM Microcontroller MMIO AES (SEC-T CTF 2017)](#efm32-arm-microcontroller-mmio-aes-sec-t-ctf-2017)
- [MBR/Bootloader Reversing with QEMU + GDB (Square CTF 2017)](#mbrbootloader-reversing-with-qemu--gdb-square-ctf-2017)
---
## HD44780 LCD Controller GPIO Reconstruction (32C3 2015)
Recover text displayed on an HD44780 LCD from raw Raspberry Pi GPIO recordings:
1. **Identify signal lines:** Map GPIO pins to HD44780 signals (RS, CLK, D4-D7 for 4-bit mode)
2. **Clock edge detection:** Sample data lines on falling clock edges (1->0 transition)
3. **Nibble assembly:** Combine two 4-bit samples into one 8-bit command/data byte
4. **DRAM address mapping:** HD44780 uses non-contiguous addressing for multi-line displays:
- Line 0: 0x00-0x27
- Line 1: 0x40-0x67
- Line 2: 0x14-0x3B
- Line 3: 0x54-0x7B
```python
display = [' '] * 80 # 4 lines x 20 chars
cursor = 0
for timestamp, gpio_state in sorted(gpio_log):
if falling_edge(gpio_state, CLK_PIN):
nibble = extract_data_bits(gpio_state)
byte = assemble_nibble(nibble) # Two nibbles per byte
if rs_high(gpio_state): # RS=1: data write
display[dram_to_position(cursor)] = chr(byte)
cursor += 1
else: # RS=0: command (set cursor, clear, etc.)
cursor = parse_command(byte)
```
**Key insight:** GPIO pin-to-signal mapping is rarely documented; identify CLK by finding the pin with most transitions, RS by correlation with data patterns (alternating command/data phases).
---
## RISC-V (Advanced)
Beyond basic disassembly (see [tools.md](tools.md#risc-v-binary-analysis-ehax-2026)):
### Custom Extensions
```text
Bitmanip extensions (Zbb, Zbc, Zbs):
clz, ctz, cpop -> count leading/trailing zeros, popcount
orc.b, rev8 -> byte-level bit manipulation
andn, orn, xnor -> negated logic operations
clmul, clmulh, clmulr -> carry-less multiplication (crypto)
bset, bclr, binv, bext -> single-bit operations
Crypto extensions (Zk*):
aes32esi, aes32dsmi -> AES round operations
sha256sig0, sha512sum0 -> SHA hash acceleration
sm3p0, sm4ed -> Chinese crypto standards
```
### Privileged Modes
```text
Machine mode (M): Highest privilege, firmware/bootloader
Supervisor mode (S): OS kernel
User mode (U): Applications
CSR registers to watch:
mstatus/sstatus -> privilege level, interrupt enable
mtvec/stvec -> trap handler address
mepc/sepc -> exception return address
mcause/scause -> trap cause
satp -> page table root (virtual memory)
```
### RISC-V Debugging
```bash
# OpenOCD + GDB for hardware debugging
openocd -f interface/jlink.cfg -f target/riscv.cfg
# GDB for RISC-V
riscv64-unknown-elf-gdb binary
(gdb) target remote :3333
# QEMU with GDB server
qemu-riscv64 -g 1234 -L /usr/riscv64-linux-gnu/ ./binary
riscv64-linux-gnu-gdb -ex 'target remote :1234' ./binary
```
---
## ARM64/AArch64 Reversing and Exploitation
AArch64 (ARM 64-bit) appears in mobile apps, cloud servers (AWS Graviton), Apple Silicon, and CTF challenges. Key differences from x86-64 affect both reversing and exploitation.
**Setup and emulation:**
```bash
# Install cross-toolchain and emulator
apt install gcc-aarch64-linux-gnu gdb-multiarch qemu-user-static
# Run AArch64 binary on x86 host
qemu-aarch64-static -L /usr/aarch64-linux-gnu/ ./arm64_binary
# Debug with GDB
qemu-aarch64-static -g 12345 -L /usr/aarch64-linux-gnu/ ./arm64_binary &
gdb-multiarch -ex 'set arch aarch64' -ex 'target remote :1234' ./arm64_binary
# With library preloading (for challenges that ship libc)
qemu-aarch64-static -g 12345 -E LD_PRELOAD=./libc.so.6 -L ./lib ./arm64_binary
```
**AArch64 calling convention (key differences from x86-64):**
```text
Registers:
x0-x7 -- function arguments AND return values (x0 = first arg / return)
x8 -- indirect result location (struct returns)
x9-x15 -- caller-saved temporaries
x19-x28 -- callee-saved (preserved across calls)
x29 (fp) -- frame pointer
x30 (lr) -- link register (return address, NOT on stack by default)
sp -- stack pointer (must be 16-byte aligned)
xzr -- zero register (reads as 0, writes discarded)
Key exploitation differences:
- Return address in LR (x30), not on stack -- pushed only if function calls others
- No RIP-relative addressing like x86 -- uses ADRP+ADD pairs for PC-relative loads
- Fixed 4-byte instruction width -- no variable-length gadget tricks
- NOP = 0xD503201F (not 0x90)
- BLR x8 / BR x30 -- indirect calls/jumps use register operands
```
**Common AArch64 patterns in Ghidra/IDA:**
```text
# PC-relative address loading (equivalent to x86 LEA):
ADRP x0, #0x411000 ; Load page address (4KB aligned)
ADD x0, x0, #0x8 ; Add page offset -> x0 = 0x411008
# Function prologue:
STP x29, x30, [sp, #-0x30]! ; Push fp + lr, decrement sp
MOV x29, sp ; Set frame pointer
# Function epilogue:
LDP x29, x30, [sp], #0x30 ; Pop fp + lr, increment sp
RET ; Branch to x30 (lr)
# Switch/jump table:
ADR x1, jump_table
LDRB w2, [x1, x0] ; Load offset byte
ADD x1, x1, w2, SXTB ; Sign-extend and add
BR x1 ; Indirect branch
```
**ROP on AArch64:**
```python
from pwn import *
# AArch64 gadgets differ from x86:
# - "pop {x0}; ret" equivalent: LDP x0, x1, [sp], #0x10; RET
# - Prologue gadgets: LDP x29, x30, [sp, #0x20]; ... RET
# - system() call: x0 = pointer to "/bin/sh", BLR to system
context.arch = 'aarch64'
elf = ELF('./arm64_binary')
# Common gadget pattern in AArch64 libc:
# LDP X19, X20, [SP,#var_s10]
# LDP X29, X30, [SP+var_s0],#0x20
# RET
# Controls x19, x20, x29, x30 and advances sp by 0x20
```
**Key insight:** AArch64's fixed instruction width and register-based return address (`lr`/`x30`) make ROP gadgets more constrained than x86. Look for `LDP` (load pair) gadgets that pop multiple registers from the stack. The `STP`/`LDP` instruction pairs that save/restore callee-saved registers in function prologues/epilogues are the primary gadget source.
**When to recognize:** `file` shows "ELF 64-bit LSB ... ARM aarch64". Ghidra auto-detects but may need manual processor selection for raw binaries. Use `qemu-aarch64-static` for emulation on x86 hosts.
**Tools:** radare2 (`r2 -AA -a arm -b 64`), Ghidra (auto-detect), `aarch64-linux-gnu-objdump -d`, Unicorn Engine (`UC_ARCH_ARM64`)
**References:** Google CTF 2016 "Forced Puns", Insomni'hack 2018 "onecall"
---
## MIPS64 Cavium OCTEON Coprocessor 2 Crypto (SEC-T CTF 2017)
Cavium OCTEON network processors implement hardware AES and SHA256 via MIPS Coprocessor 2 (CP2) using `dmtc2` (move to CP2) and `dmfc2` (move from CP2) instructions. These look like ordinary register moves to a disassembler but drive the hardware crypto engine.
**Key CP2 register layout (OCTEON):**
```text
AES key registers:
0x0104 – AES key quadword 0
0x0105 – AES key quadword 1
0x0106 – AES key quadword 2
0x0107 – AES key quadword 3
SHA256 hash registers:
0x400E–0x4012 – SHA256 intermediate hash words
0x404F – SHA256 control/result
dmtc2 rN, 0x0104 ; load 64 bits of AES key into CP2 register 0x104
dmtc2 rN, 0x0105 ; ...next quadword
```
**Approach:**
1. Disassemble in IDA/Ghidra — `dmtc2`/`dmfc2` with selector in 0x100-0x40FF range indicates OCTEON CP2
2. Cross-reference the Cavium OCTEON Hardware Reference Manual for register semantics
3. Trace the key loading sequence to recover the AES or HMAC key material
**Key insight:** Hardware crypto accelerators on MIPS appear as CP2 register writes (`dmtc2`/`dmfc2`). Identify the base register address and cross-reference vendor documentation.
**References:** SEC-T CTF 2017
---
## EFM32 ARM Microcontroller MMIO AES (SEC-T CTF 2017)
Silicon Labs EFM32 Cortex-M binary — a flat binary loaded at 0x1000 in Thumb mode.
**IDA setup:**
```text
Processor: ARM Little-endian (ARMv7-M)
Load address: 0x1000
Set T register = 1 (force Thumb mode decoding)
```
**AES accelerator MMIO layout (EFM32 AES peripheral at 0x400E0000):**
```text
0x400E0000 + 0x000 CTRL – enable, decrypt mode
0x400E0000 + 0x004 CMD – start/stop
0x400E0000 + 0x010 KEYLA – key low word 0
0x400E0000 + 0x014 KEYLB – key low word 1
0x400E0000 + 0x018 KEYLC – key low word 2
0x400E0000 + 0x01C KEYLD – key low word 3
```
The binary loads two separate values, XORs them together, then writes the result as the AES key. Decrypt the embedded ciphertext block with the composed key in ECB mode.
```python
from Crypto.Cipher import AES
key_part_a = bytes.fromhex("...") # extracted from IDA .data section
key_part_b = bytes.fromhex("...") # second value
key = bytes(a ^ b for a, b in zip(key_part_a, key_part_b))
cipher = AES.new(key, AES.MODE_ECB)
plaintext = cipher.decrypt(ciphertext)
```
**Key insight:** Hardware AES accelerators on microcontrollers appear as MMIO register writes at a specific base address — cross-reference the vendor reference manual (EFM32 Reference Manual for Silicon Labs peripherals).
**References:** SEC-T CTF 2017
---
## MBR/Bootloader Reversing with QEMU + GDB (Square CTF 2017)
Boot a floppy/disk image in QEMU with the GDB stub enabled, then attach GDB for full source-level debugging of 16-bit real mode or 32-bit protected mode bootloader code.
```bash
# Boot with GDB stub on port 1234; -S pauses execution at start
qemu-system-x86_64 -fda disk.img -s -S
# In another terminal, attach GDB
gdb -ex "set architecture i8086" \
-ex "target remote :1234" \
-ex "break *0x7c00" \
-ex "continue"
# Common MBR entry point is 0x7c00 (BIOS loads MBR here)
# Step through bootloader, inspect registers and memory:
(gdb) x/20i $pc
(gdb) info registers
(gdb) x/16xb 0x7c00
```
To bypass a password check: identify the conditional jump after the comparison and NOP it out in the image file, or patch the comparison to always succeed.
```bash
# Find the comparison offset in the image and patch it
python3 -c "
data = open('disk.img', 'rb').read()
# Replace JNZ (0x75) with JMP-short-always or NOP
data = data[:offset] + b'\x90\x90' + data[offset+2:]
open('disk_patched.img', 'wb').write(data)
"
```
**Key insight:** QEMU's `-s` flag exposes a GDB stub on port 1234 for full debugging of MBR/bootloader code — workflow identical to userland debugging.
**References:** Square CTF 2017
---

442
reverse-engineering/platforms.md Normal file
View File

@@ -0,0 +1,442 @@
# CTF Reverse - Platform-Specific Reversing
macOS/iOS, embedded/IoT firmware, kernel driver, and automotive reverse engineering.
## Table of Contents
- [macOS / iOS Reversing](#macos--ios-reversing)
- [Mach-O Binary Format](#mach-o-binary-format)
- [Code Signing & Entitlements](#code-signing--entitlements)
- [Objective-C Runtime RE](#objective-c-runtime-re)
- [Swift Binary Reversing](#swift-binary-reversing)
- [iOS App Analysis](#ios-app-analysis)
- [dyld / Dynamic Linking](#dyld--dynamic-linking)
- [Embedded / IoT Firmware RE](#embedded--iot-firmware-re)
- [Firmware Extraction](#firmware-extraction)
- [Firmware Unpacking](#firmware-unpacking)
- [Architecture-Specific Notes](#architecture-specific-notes)
- [RTOS Analysis](#rtos-analysis)
- [Kernel Driver Reversing](#kernel-driver-reversing)
- [Linux Kernel Modules](#linux-kernel-modules)
- [eBPF Programs](#ebpf-programs)
- [Windows Kernel Drivers](#windows-kernel-drivers)
- [Automotive / CAN Bus RE](#automotive--can-bus-re)
---
## macOS / iOS Reversing
### Mach-O Binary Format
```bash
# File identification
file binary # "Mach-O 64-bit executable arm64" or "x86_64"
otool -l binary # Load commands (segments, dylibs, entry point)
otool -L binary # Linked dynamic libraries
# Universal (fat) binaries — multiple architectures in one file
lipo -info universal_binary # List architectures
lipo universal_binary -thin arm64 -output binary_arm64 # Extract one arch
# Segments and sections
otool -l binary | grep -A5 "segment\|section"
# Key segments: __TEXT (code), __DATA (globals), __LINKEDIT (symbols)
# Key sections: __text (instructions), __cstring (C strings), __objc_methname
```
**Key Mach-O concepts:**
- Load commands drive the dynamic linker (`dyld`)
- `LC_MAIN` → entry point (replaces `LC_UNIXTHREAD`)
- `LC_LOAD_DYLIB` → shared library dependencies
- `LC_CODE_SIGNATURE` → code signing blob
- `__DATA_CONST.__got` → Global Offset Table
- `__DATA.__la_symbol_ptr` → Lazy symbol pointers (like PLT)
### Code Signing & Entitlements
```bash
# Check code signature
codesign -dvvv binary
codesign --verify binary
# Extract entitlements (capability permissions)
codesign -d --entitlements - binary
# Key entitlements: com.apple.security.app-sandbox, com.apple.security.network.client
# Remove code signature (for patching)
codesign --remove-signature binary
# Re-sign (ad-hoc, for testing)
codesign -f -s - binary
```
**CTF relevance:** Patched binaries need re-signing to run on macOS. Ad-hoc signing (`-s -`) works for local testing.
### Objective-C Runtime RE
```bash
# Dump Objective-C class info
class-dump binary > classes.h
# Shows: @interface, @protocol, method signatures with types
# Runtime inspection with lldb
(lldb) expression -l objc -O -- [NSClassFromString(@"ClassName") new]
(lldb) expression -l objc -O -- [[ClassName alloc] init]
# Method swizzling detection (anti-tamper)
# Look for: method_exchangeImplementations, class_replaceMethod
```
**Objective-C in disassembly:**
```text
# objc_msgSend(receiver, selector, ...) is THE dispatch mechanism
# RDI = self (receiver), RSI = selector (char* method name)
# In Ghidra/IDA, look for:
objc_msgSend(obj, "checkPassword:", input)
# Selector strings are in __objc_methname section
# Cross-reference selectors to find implementations
```
**class-dump alternatives:**
- `dsdump` — faster, supports Swift + Objective-C
- `otool -oV binary` — dump Objective-C segments
- Ghidra: Enable "Objective-C" analyzer in Analysis Options
### Swift Binary Reversing
```bash
# Detect Swift
strings binary | grep "swift"
otool -l binary | grep "swift" # __swift5_* sections
# Swift demangling
swift demangle 's14MyApp0A8ClassC10checkInput6resultSbSS_tF'
# → MyApp.MyAppClass.checkInput(result: String) -> Bool
# xcrun swift-demangle < mangled_names.txt
```
**Swift in disassembly:**
```text
# Swift uses value witness tables (VWT) for type operations
# Protocol witness tables (PWT) for dynamic dispatch (like vtables)
# Key runtime functions to watch:
swift_allocObject → heap allocation
swift_release → reference count decrement
swift_bridgeObjectRetain → bridged (ObjC ↔ Swift) retain
swift_once → lazy initialization (like dispatch_once)
# String layout:
# Small strings (≤15 bytes): inline in 16-byte buffer, tagged pointer
# Large strings: heap-allocated, pointer + length + flags
# Array<T>: pointer to ContiguousArrayStorage (header + elements)
# Dictionary<K,V>: hash table with open addressing
```
**Ghidra for Swift:** Enable "Swift" language module. Swift metadata sections (`__swift5_types`, `__swift5_proto`) contain type descriptors that Ghidra can parse.
### iOS App Analysis
```bash
# Extract IPA (iOS app package)
unzip app.ipa -d extracted/
ls extracted/Payload/*.app/
# Check if encrypted (App Store encryption / FairPlay DRM)
otool -l extracted/Payload/*.app/binary | grep -A4 "LC_ENCRYPTION_INFO"
# cryptid = 1 means encrypted, 0 means decrypted
# Decrypt with frida-ios-dump (requires jailbroken device)
# Or use Clutch / bfdecrypt on device
frida-ios-dump -H jailbroken_ip -p 22 "App Name"
# Analyze decrypted binary
class-dump decrypted_binary > headers.h
```
**Jailbreak detection and bypass:**
```javascript
// Common jailbreak checks:
// 1. Check for Cydia/Sileo
// 2. Check /private/var/lib/apt
// 3. fork() succeeds (sandboxed apps can't fork)
// 4. Open /etc/apt, /bin/sh with write
// 5. Check for substrate/substitute libraries
// Frida bypass:
var paths = ["/Applications/Cydia.app", "/bin/sh", "/etc/apt",
"/private/var/lib/apt", "/usr/bin/ssh"];
Interceptor.attach(Module.findExportByName(null, "access"), {
onEnter(args) {
this.path = Memory.readUtf8String(args[0]);
},
onLeave(retval) {
if (paths.some(p => this.path && this.path.includes(p))) {
retval.replace(-1); // File not found
}
}
});
```
### dyld / Dynamic Linking
```bash
# DYLD environment variables (for analysis, blocked in hardened runtime)
DYLD_PRINT_LIBRARIES=1 ./binary # Print loaded dylibs
DYLD_INSERT_LIBRARIES=hook.dylib ./binary # Inject dylib (like LD_PRELOAD)
# Note: SIP (System Integrity Protection) blocks this for system binaries
# Inspect dyld shared cache (contains all system frameworks)
dyld_shared_cache_util -list /System/Cryptexes/OS/System/Library/dyld/dyld_shared_cache_arm64e
```
---
## Embedded / IoT Firmware RE
### Firmware Extraction
```bash
# binwalk — firmware analysis and extraction
binwalk firmware.bin # Identify embedded filesystems, compressed data
binwalk -e firmware.bin # Extract all identified components
binwalk -Me firmware.bin # Recursive extraction (matryoshka)
binwalk --dd='.*' firmware.bin # Extract everything raw
# Manual extraction by signature
strings firmware.bin | head -50 # Look for version strings, filesystem markers
hexdump -C firmware.bin | grep "hsqs" # SquashFS magic
hexdump -C firmware.bin | grep "UBI#" # UBI magic
```
**Hardware extraction methods (physical access):**
```text
UART: Serial console — often gives root shell or bootloader access
Tools: USB-UART adapter, baudrate detection (usually 115200)
Identify: 4 pins (GND, TX, RX, VCC), use multimeter
JTAG: Direct CPU debug — read/write flash, halt CPU, set breakpoints
Tools: OpenOCD, J-Link, Bus Pirate
Identify: 10/14/20-pin header, use JTAGulator for auto-detection
SPI Flash: Direct chip read — dump entire firmware
Tools: flashrom, CH341A programmer
Identify: 8-pin SOIC chip (Winbond, Macronix, etc.)
eMMC: Embedded MMC — common in routers, phones
Tools: eMMC reader, direct solder to test pads
```
### Firmware Unpacking
```bash
# SquashFS (most common in routers)
unsquashfs -d output/ squashfs-root.sqfs
# If custom compression: try different compressors (-comp xz|lzma|lzo|gzip)
# JFFS2
jefferson -d output/ jffs2.img
# UBI/UBIFS
ubireader_extract_images firmware.ubi
ubireader_extract_files ubifs.img
# CPIO (initramfs)
cpio -idv < initramfs.cpio
# Device tree blob
dtc -I dtb -O dts -o output.dts device_tree.dtb
# Kernel extraction
binwalk -e firmware.bin
# Look for: zImage, uImage, vmlinux
# Extract vmlinux from compressed: vmlinux-to-elf tool
```
### Architecture-Specific Notes
**ARM (most common in IoT):**
```bash
# Cross-toolchain
apt install gcc-arm-linux-gnueabihf gdb-multiarch
# QEMU emulation
qemu-arm -L /usr/arm-linux-gnueabihf/ ./arm_binary
qemu-arm -g 1234 ./arm_binary # Start GDB server on port 1234
gdb-multiarch -ex 'target remote :1234' ./arm_binary
# ARM vs Thumb: ARM instructions are 4 bytes, Thumb are 2 bytes
# LSB of function pointer indicates mode: 0=ARM, 1=Thumb
# Ghidra: Right-click → Processor Options → ARM/Thumb mode
```
**ARM64/AArch64:** See [platforms-hardware.md](platforms-hardware.md#arm64aarch64-reversing-and-exploitation) for AArch64 calling convention, ROP gadgets, and qemu-aarch64-static emulation.
**MIPS (routers, embedded):**
```bash
# Big-endian vs little-endian — check ELF header or file command
file binary # "MIPS, MIPS32 rel2 (MIPS-II), big-endian" or "little-endian"
# Emulation
qemu-mips -L /usr/mips-linux-gnu/ ./mips_binary # Big-endian
qemu-mipsel -L /usr/mipsel-linux-gnu/ ./mipsel_binary # Little-endian
# Key MIPS patterns:
# Branch delay slots — instruction AFTER branch always executes
# $gp (global pointer) — used for PIC, points to .got
# lui + addiu pair — loads 32-bit constant (upper 16 + lower 16)
```
**RISC-V:** See main [tools.md](tools.md#risc-v-binary-analysis-ehax-2026) for Capstone disassembly and [platforms-hardware.md](platforms-hardware.md#risc-v-advanced) for advanced extensions and debugging.
### RTOS Analysis
```text
FreeRTOS:
- Tasks (like threads): xTaskCreate → function pointer + stack
- Strings: "IDLE", "Tmr Svc", task names
- xQueueSend/xQueueReceive → inter-task communication
- Look for vTaskDelay() for timing, xSemaphoreTake() for sync
Zephyr:
- k_thread_create → kernel thread creation
- k_msgq_put/k_msgq_get → message queues
- CONFIG_* symbols reveal kernel configuration
Bare metal (no OS):
- Interrupt vector table at address 0x0 or 0x08000000 (STM32)
- main loop pattern: while(1) { read_input(); process(); output(); }
- Peripheral registers at memory-mapped addresses (check datasheet)
```
---
## Kernel Driver Reversing
### Linux Kernel Modules
```bash
# Identify kernel module
file module.ko # "ELF 64-bit LSB relocatable"
modinfo module.ko # Module info (description, author, license)
# List module symbols
nm module.ko | grep -v " U " # Exported symbols
# Strings for quick recon
strings module.ko | grep -i "flag\|secret\|ioctl\|device"
# Find ioctl handler
# Key pattern: .unlocked_ioctl = my_ioctl_handler in file_operations struct
# In Ghidra: find struct with function pointers, identify by position
# Load in Ghidra
# Language: x86:LE:64:default
# Base address: doesn't matter for .ko (relocatable)
# Look for init_module / cleanup_module entry points
```
**Common kernel module CTF patterns:**
```c
// Device creation (creates /dev/challenge)
alloc_chrdev_region(&dev, 0, 1, "challenge");
cdev_init(&cdev, &fops);
// ioctl handler (main interface)
long my_ioctl(struct file *f, unsigned int cmd, unsigned long arg) {
switch (cmd) {
case CUSTOM_CMD_1: /* operation */ break;
case CUSTOM_CMD_2: /* operation */ break;
}
}
// copy_from_user / copy_to_user — data transfer with userspace
copy_from_user(kernel_buf, (void __user *)arg, size);
copy_to_user((void __user *)arg, kernel_buf, size);
```
**Debugging kernel modules:**
```bash
# QEMU + GDB for kernel debugging
qemu-system-x86_64 -kernel bzImage -initrd initrd.cpio -s -S \
-append "console=ttyS0 nokaslr" -nographic
# In another terminal
gdb vmlinux
(gdb) target remote :1234
(gdb) lx-symbols # Load module symbols (requires scripts)
(gdb) add-symbol-file module.ko 0x<loaded_address>
```
### eBPF Programs
```bash
# Dump eBPF programs from running system
bpftool prog list
bpftool prog dump xlated id <N> # Disassemble
bpftool prog dump jited id <N> # JIT'd machine code
# eBPF bytecode analysis
# eBPF has 11 registers (r0-r10), 64-bit
# r0 = return value, r1-r5 = arguments, r10 = frame pointer
# Instructions are 8 bytes each
# Disassemble .o file containing eBPF
llvm-objdump -d ebpf_prog.o
# Key eBPF patterns:
# bpf_map_lookup_elem → read from map
# bpf_map_update_elem → write to map
# bpf_probe_read → read kernel memory
# bpf_trace_printk → debug output
```
### Windows Kernel Drivers
```bash
# .sys files are PE format — load in IDA/Ghidra as normal PE
# Entry point: DriverEntry(PDRIVER_OBJECT, PUNICODE_STRING)
# Key patterns:
# IoCreateDevice → creates device object
# IRP_MJ_DEVICE_CONTROL → ioctl handler
# MmMapIoSpace → memory-mapped I/O
# ObReferenceObjectByHandle → get kernel object from handle
# ZwCreateFile/ZwReadFile → kernel-mode file operations
```
---
## Automotive / CAN Bus RE
```bash
# CAN bus interface setup
sudo ip link set can0 type can bitrate 500000
sudo ip link set up can0
# Capture CAN traffic
candump can0 # Live capture
candump -l can0 # Log to file
cansniffer can0 # Filter/highlight changes
# Replay CAN messages
canplayer -I logfile.log can0
cansend can0 7DF#0201000000000000 # Send single frame (OBD-II request)
# UDS (Unified Diagnostic Services) — common in automotive CTF
# Service 0x27: Security Access (seed-key authentication)
# Service 0x2E: Write Data By Identifier
# Service 0x31: Routine Control
# Decode CAN frames
# ID: 11-bit or 29-bit identifier
# DLC: Data Length Code (0-8 bytes)
# Data: up to 8 bytes payload
```
**CTF automotive patterns:**
- Seed-key bypass: Reverse the key derivation algorithm from ECU firmware
- CAN message replay: Capture legitimate command, replay to unlock feature
- Firmware extraction from ECU via UDS/KWP2000

149
reverse-engineering/references/ai-assisted-re.md Normal file
View File

@@ -0,0 +1,149 @@
# AI 辅助逆向工程
> LLM 驱动反编译 / 多 Agent 验证 / 神经语义恢复
> 2025-2026 最大范式转变
## 核心工具与模型
### LLM4Decompile
- 首个将 LLM 用于二进制→源码反编译的开源框架
- 支持 x86/ARM/MIPS 多架构
- 输入: 汇编代码 → 输出: C 源码
- 训练数据: 百万级 源码-汇编 对
### Decaf (2026)
- **编译器反馈验证**: 将 LLM 生成的源码→编译→对比原始二进制
- 效果: 反编译率 26% → 83.9% (ExeBench Real -O2)
- 关键洞察: 反馈循环比更大模型更有效
### Constraint-Guided Multi-Agent (2026)
- 三级验证管道:
1. 语法正确性 (解析)
2. 可编译性 (GCC)
3. 行为等价性 (LLM 生成测试用例)
- 84-97% 可重执行率，每次仅 $0.03-0.05
### REMEND (2026)
- 专项: 从二进制提取数学方程
- 89.8-92.4% 准确率 (跨 3 ISA × 3 优化级别 × 2 语言)
- 速度: 0.132s/函数, 仅 12M 参数
### Glaurung
- 开源 Ghidra 替代，Rust 内核 + Python 绑定
- **AI 原生架构**: LLM Agent 嵌入每个分析层
- 证据制品: plain/rich/JSON/JSONL 多格式输出供 LLM 消费
- 支持: ELF/PE/Mach-O、x86/ARM/RISC-V、IOC 检测、熵分析
## 工作流：AI 增强二进制分析
### 1. LLM 辅助快速侦察
```text
□ strings 提取 → LLM 语义分类（URL/密钥/路径/协议）
□ 导入表分析 → LLM 推断功能（加密=OpenSSL? 网络=libcurl?）
□ 反汇编片段 → LLM 识别模式（密码算法、反调试、虚拟机检测）
□ 错误消息 → LLM 推断上下文（"Invalid license" → 授权逻辑位置）
```
### 2. 神经反编译
```bash
# LLM4Decompile
python llm4decompile.py --binary target.so --arch arm64 --output target.c
# 验证结果（重编译 + 对比）
gcc -O2 -o target_recompiled target.c -fPIC -shared
# → 验证输出行为等价性
```
### 3. Multi-Agent 验证
```text
Agent 1 (语法): 检查生成的 C 代码是否可以 parse
↓ 失败 → 反馈错误信息给 LLM 重试
Agent 2 (编译): GCC 编译 → 检查 warnings/errors
↓ 失败 → 反馈编译错误给 LLM
Agent 3 (行为): LLM 生成输入 → 运行原始和重编译版本 → 对比输出
↓ 不一致 → 反馈差异给 LLM → 迭代修正
```
### 4. LLM 辅助静态分析
```text
□ 函数重命名: 输入反编译伪代码 → LLM 建议语义化名称
□ 类型恢复: 分析上下文 → LLM 推断结构体/类定义
□ 算法识别: 汇编片段 → LLM 识别密码算法（AES/TEA/RC4/自定义）
□ 协议逆向: 网络包序列 → LLM 推断协议格式
□ 注释生成: 反编译代码 → LLM 生成中文/英文注释
```
### 5. macOS/iOS 私有框架逆向 (MOTIF)
```text
问题: macOS 私有框架无文档，类型信息缺失
方案: LLM 分析使用模式 → 推断方法签名和参数类型
效果: ObjC 签名恢复 15% → 86% (vs 静态分析)
```
## LLM Prompt 模板
### 函数语义分析
```
You are a reverse engineering expert. Analyze this decompiled function:
[伪代码]
1. What does this function do? (one sentence)
2. Suggest a meaningful function name.
3. What are the input parameters and their likely types?
4. What is the return value?
5. What external APIs/functions does it depend on?
6. Any security-relevant operations (crypto, auth, network, file I/O)?
```
### 算法识别
```
Analyze this assembly/disassembly for cryptographic operations:
[汇编代码]
1. Is this a known cryptographic algorithm? (AES/DES/RC4/TEA/ChaCha20/custom?)
2. Identify the key schedule and round structure.
3. What is the key size?
4. Are there any hardcoded constants that identify the algorithm?
```
### 协议格式推断
```
Given this network packet sequence, infer the protocol structure:
[hex dump]
1. Identify magic bytes and length fields.
2. Propose a struct definition for the packet header.
3. What field(s) appear to be checksums/CRCs?
4. Is this a known protocol or custom?
```
## 工具选型
| 场景 | 推荐工具 | 成本 |
|------|---------|------|
| 快速反编译 | LLM4Decompile | 免费 (本地 GPU) |
| 高精度反编译 | Constraint-Guided Multi-Agent | ~$0.05/二进制 |
| 数学函数提取 | REMEND | 免费 |
| 全平台 RE | Glaurung (Rust) | 免费开源 |
| LLM 交互 | Claude API / GPT-4 / DeepSeek | ~$0.01-0.10/次 |
## 局限性
- **复杂控制流**: 虚拟化/混淆代码仍困难（控制流平坦化、VMProtect）
- **间接调用**: 虚函数表、函数指针难以恢复
- **内联函数**: 编译器内联后边界模糊
- **浮点运算**: 向量化指令的语义恢复有待提升
- **上下文窗口**: 大函数 (>1000 行) 超出 LLM 上下文限制
Source: Decaf (2026), REMEND (2026), Constraint-Guided Multi-Agent Decompilation (2026), LLM4Decompile, Glaurung

298
reverse-engineering/references/nonpe-format-cookbook.md Normal file
View File

@@ -0,0 +1,298 @@
# 非 PE / 多格式 Agent 响应菜谱 U–AV + AW–DN
> 与 PE 反调试菜谱 A–T（../anti-analysis.md）并列：按**文件类型**给出「触发 → 动作一行 → Evidence」。
> **不是**第二套主流程。Triage 识别类型后跳转到对应 skill + 本表。
> 默认 **授权隔离 lab / 已授权样本与设备**。格机、BYOVD、反射注入等写**检测与取证**，不写未授权破坏/利用教程。
> 绕过或还原失败也 MUST 记 Evidence；禁止静默当「无害」。
>
> §1–§8 / U–AV = 原始规则（Issue #65）。§9–§23 / AW–DN = 扩展规则（Issue #87，去重后 + 语义增强 + edge-case 补丁）。
## 0. 路由速查
| 类型线索 | 主 skill | 本表章节 |
|----------|----------|----------|
| .bat / .cmd / 批处理 | malware-analysis | §1, §19 |
| .ps1 / PowerShell | malware-analysis | §2, §20 |
| Office 宏 / VBA / XLM / .docm/.xlsm | malware-analysis | §3 (含 DD OLE 提取, DJ XLM 宏) |
| .docx/.xlsx/.pptx OOXML 外链 / DDE / .rtf OLE | malware-analysis | §10 (含 DK RTF) |
| Web/前端 JS 混淆、JSVMP | js-reverse | §4, §21 (含 DE/DF) |
| .sys / 内核驱动 | reverse-engineering/kernel-driver-reverse.md + cre | §5 |
| .dll 侧重点 | malware-analysis / re-agent-workflow | §6（与 A–T 去重） |
| APK / Magisk / 隐藏图标 | apk-reverse | §7–§8, §23 |
| .pdf / PDF 文档 | malware-analysis | §9 |
| .wasm / WebAssembly | reverse-engineering | §11 |
| .jar/.class / Java 字节码 | reverse-engineering | §12 |
| .exe(AutoIt) / .au3 | malware-analysis | §13 |
| .hta / HTML Application | malware-analysis | §14 |
| .wsf/.jse/.vbe | malware-analysis | §15 |
| .msi / Windows Installer | malware-analysis | §16 |
| .reg / 注册表脚本 | malware-analysis | §17 |
| .vbs / VBScript | malware-analysis | §18 |
| Xposed/LSPosed 模块 | apk-reverse | §22 |
| ELF / Linux 二进制 | reverse-engineering | → elf-analysis.md, anti-analysis.md |
| Mach-O / macOS/iOS | reverse-engineering | → platforms.md |
| Python 字节码 | reverse-engineering | → languages.md |
## 1. BAT/CMD（U V W）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **U** | 大量 SET 单字符变量 + %a%%b% 拼接，或 ^ 续行拆命令 | 逐行展开 SET；还原后命令列表；可用 batch 脱混淆工具；**禁止**未还原就当「无动作」 | E-batch-deobf | P0 |
| **V** | 文本打开乱码，HEX 头 FF FE（UTF-16 LE BOM） | 确认 BOM → 转 UTF-8 再解析；或 chcp 65001 + type | E-batch-encoding | P2 |
| **W** | 大量 REM/::、冗余 GOTO/标签淹没真实逻辑 | 去注释；梳 GOTO 真路径；隔离执行抓 cmd 实际命令日志 | E-batch-deadcode | P1 |
## 2. PowerShell（X Z）
> 编号保留提出者习惯：**无补丁 Y**。
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **X** | 多层 FromBase64String / Gzip / Compress / 嵌套 -replace | **逐层**解码；每层结果单独记；工具可选（PowerDecode 等），无工具则手工/脚本 | E-ps-decode-layer-N | P0 |
| **Z** | 字符串倒序、碎片 + 拼接后 Invoke-Expression/IEX | 还原完整串；对 IEX 下断或脚本块日志；明文命令入 Evidence | E-ps-string-restore | P1 |
## 3. VBA 宏 / XLM（AA AB AC DD DJ）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **AA** | olevba/OLEDump 仅见 P-Code、源码流空（VBA Stomping） | P-Code 反编译工具；不全则 Word/Excel 宏调试观察；写清限制 | E-vba-pcode | P0 |
| **AB** | 大量 Chr() 拼接或 Base64 串，疑 shellcode/嵌套脚本 | 立即窗口/脚本还原串；解码后判类型；动态盯 CreateObject/Shell | E-vba-str-decode | P1 |
| **AC** | 无意义 If 1=2、或 InsertLines/DeleteLines 自修改 | 静跟真分支；动态 bp 自修改 API 并 dump 改后宏 | E-vba-selfmod | P2 |
| **DD** | olevba/oledump 检出 VBA 宏项目（vbaProject.bin）；扩展名 .docm/.xlsm/.pptm | oledump.py 检查 OLE 流结构；olevba 提取 VBA 源码检测可疑 API；检查 AutoOpen/Workbook_Open 等自动执行宏 | E-office-vba | P0 |
| **DJ** | .xls/.xlsm 含 Excel 4.0/XLM 宏（隐藏在单元格公式中，非 VBA 流）；olevba 检出 XLM 宏标记 | olevba --xlm 提取 XLM 宏公式；检查隐藏工作表中的 EXEC/CALL/REGISTER 函数；XLMMacroDeobfuscator 动态仿真还原 | E-office-xlm | P0 |
## 4. JavaScript（AD AE AF）→ 主路径 js-reverse
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **AD** | 自定义字节码数组 + while/switch 解释器（JSVMP） | 找 VM 入口与 opcode 分发；动态日志轨迹；AST+动态双轨；见 js-reverse DeepDive | E-js-vmp | P0 |
| **AE** | while(1){switch} + 大字符串数组下标 | AST/Babel 重构；数组下标还原字符串；wakaru 等可选；**勿**整份粘贴 PE ollvm-deobfuscation 长文 | E-js-deobf | P0 |
| **AF** | debugger、劫持 console、performance.now 差、DevTools 检测 | 停用断点/固定时间源/无头浏览器；patch 检测点；授权页面 | E-js-anti-debug | P1 |
## 5. SYS 内核驱动（AG AH AI）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **AG** | DriverEntry 很短，逻辑不在入口 | 扫 MajorFunction[] 非空槽；优先 IRP_MJ_DEVICE_CONTROL/CREATE；地址列表入证 | E-driver-irp-handlers | P0 |
| **AH** | 存在 DeviceIoControl / IOCTL 分发 | 建控制码→处理函数表；标 METHOD_* 与缓冲方向；用户态通信面 | E-driver-ioctl | P0 |
| **AI** | 样本加载/投放知名脆弱驱动或异常签名驱动（BYOVD 模式） | 对照 LOLDrivers 等**公开**列表；记驱动名/哈希/签名；分析**调用意图**；**不**展开 exploit 步骤 | E-driver-byovd | P1 |
详见 kernel-driver-reverse.md 流程；本表只补 agent 动作锚点。
## 6. DLL（AJ–AQ）— 与 A–T / #72 去重
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **AJ** | DLL 分析只看了导出/EP，忽略 TLS 或 DllMain | **TLS 回调 + DllMain 都必须看**；动态断点顺序仍遵循四级火箭（TLS→EP/DllMain→API→ExitProcess） | E-dll-tls-dllmain | P0 |
| **AK** | 导出名前良后恶、错名、或导出与行为不符 | 导出表与实际调用交叉；异常导出列表 | E-exports-anomaly | P0 |
| **AL** | 无导出或导出极少，仍被加载 | 从入口、字符串、xrefs、调用方定位；不要因「无导出」放弃 | E-dll-noexport | P0 |
| **AM** | 静态 IAT 缺 DLL、运行时才用 | **See A–T 补丁 R**（Delay-Load / E-delay-import），此处不双写长文 | E-delay-import | P0 指针 |
| **AN** | 需还原导出函数参数与调用约定 | 交叉引用 + 动态看寄存器/栈；标注 stdcall/fastcall 等 | E-dll-export-abi | P1 |
| **AO** | 疑 DLL 劫持/侧加载 | 查应用目录同名 DLL、搜索路径、KnownDLLs；合法程序+异常 DLL 组合 | E-dll-sideload | P1 |
| **AP** | 无文件映射/反射加载线索 | 内存特征、加载器行为、无路径模块；授权环境取证 | E-dll-reflective | P1 |
| **AQ** | 仅因导出名「不像恶意」降风险 | **禁止**只凭导出名判安全；结合段权限、入口、字符串、动态行为 | E-dll-export-priority | P1 |
DLL/SYS 硬门仍：E-imports + E-exports（见 re-agent-workflow）。
## 7. Android 格机 / 持久化（AR AS AT）→ apk-reverse
> **仅授权样本、镜像或测试设备。** 动作是检测、提取 IOC 与持久化路径，不是实施破坏。
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **AR** | Magisk 模块/脚本含删库、刷写、批量 rm 系统分区等**格机特征命令** | 特征命令表 + 模块路径；标高危破坏能力；不执行格机命令 | E-android-wiper-cmd | P0 |
| **AS** | 循环 curl|sh / 远程拉脚本、非常规 C2 URL | 提 URL；分析下载体是否含格机命令；记临时路径 | E-android-wiper-backdoor | P0 |
| **AT** | /data/adb/service.d、post-fs-data.d、可疑 /system/priv-app 等 | 列持久化脚本/APK；内容摘要入证 | E-android-persistence | P1 |
## 8. Android 透明/隐藏图标（AU AV）→ apk-reverse
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **AU** | LAUNCHER 图标全透明/空 label、Theme.NoDisplay、无 LAUNCHER category、组件 disabled | aapt dump badging + manifest；反编译查图标像素；异常项入证 | E-android-hidden-icon-manifest | P0 |
| **AV** | 已装但桌面无图标，后台流量/自启/高危权限/动态恢复图标 | pm list vs 桌面；dumpsys package；广播与 device_admin；行为入证 | E-android-hidden-icon-behavior | P1 |
---
> **以下 §9–§23 为 Issue #87 扩展规则（AW–DC）。**
> 已去除与现有文件重复的 ELF（→ elf-analysis.md）、Mach-O（→ platforms.md）、Python（→ languages.md）章节。
## 9. PDF 恶意文档（AW AX AY AZ）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **AW** | pdfid 检出 /JS、/JavaScript、/OpenAction、/AA、/Launch 计数 >0（含 hex 编码名称如 /4A#61#76#61... 的混淆计数） | pdfid -e 统计（对比 plain vs obfuscated 计数）；pdf-parser 提取可疑对象；peepdf 交互分析 + JS 仿真 | E-pdf-autoaction | P0 |
| **AX** | pdfid 检出 /EmbeddedFile >0；对象流含 FlateDecode/ASCIIHexDecode 级联过滤器链；或 /Annot 对象中隐藏编码载荷 | pdf-parser 提取流数据；peepdf 解码多层级联过滤器（含 AES 加密流 security handler r5/r6）；检查 Annotation 对象；file 识别解码结果类型 | E-pdf-embedded | P0 |
| **AY** | 提取的 PDF JS 含大量 eval、unescape、String.fromCharCode、atob | peepdf JS 仿真环境执行跟踪；逐层解码 Base64/Hex/ROT13；CyberChef 辅助 | E-pdf-js-deobf | P1 |
| **AZ** | PDF 结构异常：/JBIG2Decode、XREF 表被操纵、对象编号跳跃 | pdfid -d 重命名可疑关键字；检查已知 CVE 利用模式；提取 exploit 触发条件 | E-pdf-exploit | P1 |
## 10. Office OOXML / DDE / RTF（BA BB DK）→ 与 §3 VBA 互补
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BA** | docx/xlsx/pptx ZIP 解压后 word/_rels/ 或 xl/_rels/ 中含可疑外部关系（含 remote template injection） | 检查 *.rels 外部链接；检查 vbaData.xml；提取嵌入 OLE 对象；检查协议处理器滥用（ms-msdt: / search-ms: / ms-officecmd:） | E-office-ooxml | P0 |
| **BB** | 文档含 DDEAUTO 或 DDEEXEC 域代码，通过域执行外部命令 | olevba --dde 扫描；提取 DDE 命令参数；检查是否指向 PowerShell/外部 exe | E-office-dde | P0 |
| **DK** | .rtf 文件含嵌入 OLE 对象（非 OOXML、非经典 OLE 复合文档） | rtfobj 提取嵌入 OLE 对象；oleobj 分析对象类型；检查 Equation Editor 漏洞利用（CVE-2017-11882 等）；file 识别提取物类型 | E-rtf-ole | P0 |
## 11. WebAssembly（BC BD BE）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BC** | 文件以 \x00asm 魔术字节开头；或 JS 代码含 WebAssembly 实例化逻辑 | wasm2wat 转文本；检查 import 段识别宿主环境导入函数；wasm-decompile 生成伪代码；检查 Emscripten 胶水签名（__wasm_call_ctors）判断是否由 JS 编译而来 | E-wasm-struct | P0 |
| **BD** | WASM 函数数量多但逻辑简单、函数体被拆分为微小函数；或存在无意义 block/loop 嵌套 | diswasm 评估函数最小化级别；JEB Pro / IDA WASM 插件深度分析；动态追踪执行日志 | E-wasm-obfuscation | P1 |
| **BE** | WASM 模块通过 JS 导入/导出函数与浏览器交互，存在 WebSocket、fetch、WebGL 调用 | 同时分析 JS 胶水代码和 WASM 模块；浏览器 DevTools 追踪数据交换；提取网络通信 URL/域名 | E-wasm-c2 | P1 |
## 12. Java JAR/Class（BF BG BH BI）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BF** | JD-GUI/jadx 打开 JAR 时类名/方法名为无意义短字符（a.a.a / _0x 前缀 / 数字类名）；或大量 while/switch 控制流混淆 | 识别混淆器类型（ProGuard / Allatori / ZKM）；Java Deobfuscator 静态反混淆；高强度时动态调试追踪关键逻辑 | E-java-obfuscation | P0 |
| **BG** | 大量 Class.forName()、Method.invoke()、Constructor.newInstance()；或自定义 ClassLoader + defineClass() 从字节数组内存加载类；导入表无害但运行时动态加载恶意类 | javap -c -v 查看反射调用细节；追踪 Class.forName 参数字符串；检查 defineClass() 字节数组来源；动态对 Method.invoke 下断 | E-java-reflection | P0 |
| **BH** | JAR 含 .so（Linux/Android）或 .dll（Windows）；或 System.loadLibrary() 调用 | 提取原生库文件；file 识别格式；转入 ELF/PE 独立分析流程 | E-java-native | P1 |
| **BI** | JAR/ZIP 解压后含嵌套 JAR/WAR/EAR；/resources、/assets 中存在高熵 .dat/.bin/.img 文件 | 递归解压所有嵌套归档；熵值分析判断加密/压缩；检查 META-INF/MANIFEST.MF 和 pom.xml | E-java-nested | P1 |
## 13. AutoIt（BJ BK BL DM）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BJ** | PE 字符串含 AutoIt / AU3 / EA05 / EA06 签名；或资源节含 AutoIt 脚本资源（注意与 AutoHotKey 区分，MITRE T1059.010 两者共用） | autoit-ripper 提取编译脚本；识别编码家族（EA05 = AutoIt3.00 / EA06 = AutoIt3.26）；EA06 头部后提取 8 字节解密密钥用于解密载荷；还原源码 | E-autoit-extract | P0 |
| **BK** | 提取脚本含大量 StringEncrypt/_StringEncrypt；或 Execute 动态执行 + 无意义变量名 | myAutToExe 静态反编译；识别反调试技术；分析混淆后控制流 | E-autoit-deobf | P1 |
| **BL** | 脚本含 RegWrite（注册表持久化）、FileInstall（文件释放）、InetGet（网络下载）、Run/RunWait | 标记敏感 API 调用序列；分析 InetGet URL；追踪 FileInstall 释放路径 | E-autoit-malicious | P0 |
| **DM** | AutoIt 作为 loader 执行进程空洞注入（process hollowing）：CallWindowProc/EnumWindows 回调 + shellcode + 注入合法进程（regsvcs.exe 等），释放 .NET 载荷（DarkGate / Snake Keylogger / ArechClient2 模式） | 检查 DllCall/DllCallbackRegister 对 kernel32 注入 API 的调用链；提取 shellcode 数据；识别被注入目标进程；提取 .NET 载荷独立分析 | E-autoit-hollowing | P0 |
## 14. HTA / HTML Application（BM BN BO）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BM** | HTML 含 HTA:APPLICATION 标签、window.execScript 或 CreateObject 调用 | 检查 HTA:APPLICATION 属性（Application、WindowState）；提取 script 标签中的 VBS/JS | E-hta-bypass | P0 |
| **BN** | HTA 通过 mshta.exe 启动后 XMLHttpRequest / ActiveXObject 远程拉 Payload 执行 | 提取网络请求 URL；跟踪 ActiveXObject 创建（ADODB.Stream 等）；还原完整下载执行链 | E-hta-download-chain | P0 |
| **BO** | HTA 仅含单行极长混淆字符串，经 eval / execScript 执行 | 提取 Base64/Hex 编码载荷解码；CyberChef 递归检测编码类型；还原载荷 | E-hta-oneline | P1 |
## 15. WSF / JSE / VBE（BP BQ BR BS）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BP** | .wsf 含 \<job\> + \<script language="..."\> 标签，混合 JScript/VBScript/Python | 按 \<script language\> 分割代码块；分别按对应语言规则分析 | E-wsf-multi | P0 |
| **BQ** | .jse/.vbe 开头含 #@~^ 签名，Microsoft Script Encoder 编码 | screnc-decoder 解码；无工具时动态执行 + dump 解码脚本 | E-jse-decode | P0 |
| **BR** | WSF 多 \<script\> 块 + \<package\> 引用外部资源 + \<component\> 引用 COM 组件 | 建立跨块调用图；追踪 \<script\> 之间函数调用；还原完整执行流程 | E-wsf-call-chain | P1 |
| **BS** | WSF 含 WshShell.SendKeys 绕过 UAC、WshShell.Run 配合 0 窗口隐藏、WScript.Sleep 延时绕过 | 检查用户模拟操作是否用于绕过安全提示；记录隐蔽执行参数 | E-wsf-anti-detect | P1 |
## 16. MSI 安装包（BT BU BV）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BT** | MSI 文件含 CustomAction 表（Binary / Script / DLL 类型自定义操作） | msiexec /a 或 lessmsi 提取内容；检查 CustomAction 表；提取自定义操作二进制文件 | E-msi-custom-action | P0 |
| **BU** | MSI Binary 表含 VBScript/JScript 自定义操作脚本 | 从 Binary 表提取脚本二进制解码为可读脚本；按 VBS/JS 规则分析 | E-msi-script | P1 |
| **BV** | MSI 通过 /quiet /passive /qn 静默安装；ALLUSERS=1 提权 | 记录安装命令行参数；分析 Property 表权限设置；标记静默+提权组合 | E-msi-privilege | P1 |
## 17. REG 注册表脚本（BW BX BY）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BW** | .reg 写入 HKCU\...\Run 或 HKLM\...\Run 等自动启动路径 | 提取所有路径；标记 Run 路径条目为持久化；记录完整路径和值 | E-reg-persistence | P0 |
| **BX** | .reg 修改 HKCR\...\shell\open\command（文件关联）或 HKCR\CLSID\{...}\InprocServer32（DLL 注入） | 检查 shell\open\command 是否为非常规 exe；检查 InprocServer32 DLL 路径 | E-reg-hijack | P0 |
| **BY** | .reg 修改 HKLM\...\Policies\System（UAC 级别）、EnableLUA、ConsentPromptBehaviorAdmin | 检查修改前默认安全配置；分析对 UAC 的影响；标记降级行为 | E-reg-uac-bypass | P1 |
## 18. VBScript（BZ CA CB CC DN）
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **BZ** | .vbs/.js 同时被 VBScript 和 JScript 解析；条件编译（@_win32）或语言特性交叉执行 | 分离 VBScript/JScript 代码块；分别语法分析；识别混合执行逻辑 | E-vbs-mixed | P1 |
| **CA** | 脚本含 CreateObject("WScript.Shell") / CreateObject("Shell.Application") / Scripting.FileSystemObject | 标注高危 COM 对象调用；跟踪 Run/Exec 参数；追踪 FSO 创建的文件路径 | E-vbs-com-abuse | P0 |
| **CB** | 脚本开头 #@~^ 签名，Microsoft Script Encoder 编码（VBS 专有） | screnc-decoder 解码；无工具时动态执行 + dump 解码脚本 | E-vbs-encoded | P0 |
| **CC** | VBA/VBScript 含 WScript.Shell.Run + cmd /c + PowerShell，后续进程注入 | 追踪 CreateObject COM 对象链；分析 Run 参数中注入特征；记录完整进程创建链 | E-vbs-inject-chain | P0 |
| **DN** | VBScript/JScript 通过 WMI ActiveScriptEventConsumer 实现无文件持久化（无启动文件夹/注册表 Run 键） | 检查 WMI 事件订阅（__EventFilter + __FilterToConsumerBinding + ActiveScriptEventConsumer）；提取绑定脚本内容；标记无文件持久化 | E-vbs-wmi-persist | P0 |
## 19. BAT/CMD 高级混淆（CD–CI）→ 与 §1 U–W 互补
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **CD** | setlocal enabledelayedexpansion + !var! + 动态变量名（!var_%i%!） | 启用延迟扩展后逐条展开；Batch-Dump --expand 自动展开 | E-bat-delayed-expand | P0 |
| **CE** | 通过 type/more/findstr 读取自身或文件的 :stream ADS 替代数据流执行 | 检查 : 后缀引用（file.bat:payload）；dir /r 列 ADS；type file:stream 提取 | E-bat-ads-hidden | P0 |
| **CF** | 大量 echo 逐行写入 .tmp/.cmd 临时文件再 call 执行 | 提取所有 echo 重定向还原临时文件内容；监控临时目录生成的脚本 | E-bat-temp-gen | P1 |
| **CG** | for %%i in (...) do set var=%%i 累加变量；for /f 逐行解析命令输出 | 逐条展开 for 循环记录每次迭代赋值；序列化还原 for /f 结果 | E-bat-for-expand | P1 |
| **CH** | 主批处理通过 %1 %* 接收参数，由父进程/下载器传入混淆指令 | 检查调用上下文记录传入参数；Base64 参数解码还原；还原完整调用链 | E-bat-param-call | P1 |
| **CI** | certutil -decode / powershell -Command / echo \| findstr 组合解码执行 | 提取 Base64/Hex 字符串解码；检查解码结果是否为可执行脚本/PE | E-bat-encoded-exec | P0 |
## 20. PowerShell 高级绕过（CJ–CO, DL）→ 与 §2 X–Z 互补
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **CJ** | [Ref].Assembly.GetType('...AmsiUtils') / amsiInitFailed / GetTypes() 等 AMSI 绕过（含硬件断点绕过：CPU 调试寄存器，无内存写入/VirtualProtect） | 识别绕过模式（Patch / 注册表 / 环境变量 / 硬件断点）；动态确认生效；标记绕过技术类型 | E-ps-amsi | P0 |
| **CK** | [PSConstraintLanguage] 类型操作或通过 DefaultRunspace 修改会话状态绕过 CLM | 识别 CLM 绕过模式；标记 bypass-clm；分析绕过后执行上下文 | E-ps-clm-bypass | P0 |
| **CL** | [ScriptBlock]::Create / $ExecutionContext.InvokeCommand 构造器；或覆盖 ScriptBlock 日志设置 | 检查脚本是否禁用日志记录；动态验证日志是否被绕过 | E-ps-sb-log-bypass | P1 |
| **CM** | IEX (New-Object Net.WebClient).DownloadString(...) 或 [Reflection.Assembly]::Load(FromBase64...) 无文件执行 | 提取下载 URL 检查域名/IP 信誉；PS 日志捕获内存加载代码；隔离网络模拟提取载荷 | E-ps-reflect-load | P0 |
| **CN** | 三层以上编码嵌套：外层 Base64 → Gzip → XOR → 明文（超出 §2 X 的两层范围） | 递归解码到明文或无法继续；每层记中间状态；PowerDecode 自动化；每层结果入证 | E-ps-multi-decode | P0 |
| **CO** | Set-Alias 将 IEX 映射为单字符别名；Get-ChildItem variable: 动态获取变量值 | 展开所有别名映射替换回原始命令名；AST 分析还原变量 | E-ps-alias-decode | P1 |
| **DL** | 脚本含 ntdll.dll EtwEventWrite 补丁（stomping）静默遥测；常与 AMSI 绕过组合使用 | 检查是否存在 EtwEventWrite 地址获取 + 内存补丁（ret 0xC3）；与 CJ AMSI 绕过同时检查；标记双绕过组合 | E-ps-etw-bypass | P0 |
## 21. JavaScript 高级混淆（CP CQ DE DF）→ 与 §4 AD–AF 互补
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **CP** | JS 使用 Proxy 对象拦截属性访问 + Reflect API 动态调用方法绕过静态分析 | 识别 Proxy get/set/apply 陷阱函数；追踪 Reflect.get 实际目标；标记动态拦截行为 | E-js-proxy | P1 |
| **CQ** | JS 含 _0x... 十六进制字符串数组 + while(!![]) 死循环 + for+switch 控制流（obfuscator.io 特征） | 识别 obfuscator.io 特征（字符串数组+死循环）；de4js / jsnice 自动反混淆；还原后代码入证 | E-js-obfuscator | P0 |
| **DE** | JS 主体为大型字节码数组 + VM 解释器循环（多 while/switch），入口指向 eval/Function 构造器；业务逻辑完全不可读（§4 AD 的深化） | 识别 VM 入口函数跟踪 opcode→处理函数映射；浏览器动态执行 Hook eval 输出；JSimplifier AST 重构；记录 opcode 映射表 | E-jsvmp-deep | P0 |
| **DF** | JS 含 eval 动态生成新代码并立即执行、document.write 重写页面、或 Function 构造器动态构造函数体 | Hook eval 和 Function 构造函数记录生成的代码；浏览器动态执行捕获自修改内容 | E-js-selfmod | P1 |
## 22. Xposed/LSPosed 模块分析（CR–CX）→ apk-reverse
> 分析 Xposed/LSPosed **模块本身**作为逆向目标（非工具使用场景）。
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **CR** | AndroidManifest.xml 无 android:name 入口 Activity；meta-data 指定 xposedmodule=true | 检查 assets/xposed_init 确定入口类；搜索 IXposedHookLoadPackage/ZygoteInit/CmdInit 接口实现 | E-xp-entry | P0 |
| **CS** | 代码含 XposedHelpers.findAndHookMethod / XposedBridge.hookMethod / findClass | 提取 findAndHookMethod 第一参数（目标类）+ 第二参数（目标方法）；建立目标应用清单 | E-xp-hook-targets | P0 |
| **CT** | 模块含 DexClassLoader/PathClassLoader 动态加载；或 Runtime.exec / ProcessBuilder 执行命令 | 追踪 DexClassLoader 构造参数；提取动态加载 DEX 独立分析；检查 exec 命令参数 | E-xp-dynamic-load | P0 |
| **CU** | Hook 目标涉及支付/生物识别/短信/通讯录/位置/加密密钥等敏感 API | 对 Hook 目标类/方法进行敏感度分类；标记支付类/生物识别类/短信通讯录类；汇总威胁等级 | E-xp-sensitive-hooks | P0 |
| **CV** | 代码含 XposedBridge 检测规避 / Zygote 注入痕迹清除 / 自定义网络通信 | 检查 stacktrace 修改 / XposedBridge 类引用清除；检查独立网络请求（OkHttp/Socket）；识别 C2 目标 | E-xp-anti-detection | P1 |
| **CW** | 代码含 Resources 动态替换 / View 绘制拦截 / AccessibilityService 声明 | 检查 AssetManager 替换 / Resources.updateConfiguration；检查 AccessibilityService 配置；识别 UI 劫持 | E-xp-ui-hijack | P1 |
| **CX** | AndroidManifest.xml 声明 lsposed xposedscope meta-data；或代码含包名白名单检查 | 解析 xposedscope 目标应用范围；检查动态白名单绕过（反射修改 scope）；识别全局 Hook 越权 | E-xp-scope-bypass | P1 |
## 23. Magisk 模块深度分析（CY–DC, DG–DI）→ 与 §7 AR–AT 互补
> §7 聚焦格机/破坏行为。本节覆盖非破坏性但可疑的模块行为：安装脚本分析、文件释放、Zygisk 注入、反检测、持久化、提权、横向感染。
| ID | 触发 | 动作（摘要） | Evidence | 优先 |
|----|------|--------------|----------|------|
| **DG** | Magisk 模块 ZIP 根目录含 config.sh / install.sh；META-INF/com/google/android/update-binary 为非标准安装器 | 提取 config.sh/install.sh 中 on_install/print_modname/set_permissions 函数；检查 update-binary 是否含额外载荷；标记 pm install / dd 块设备 / mount -o remount,rw 操作 | E-mg-install-script | P0 |
| **DH** | ZIP 内含 system/ / vendor/ / data/ 目录结构；或 post-fs-data.sh / service.sh 等开机执行脚本 | 提取释放文件路径识别是否释放 APK 到 /system/priv-app/；检查 service.sh + post-fs-data.sh 内容识别开机自启/后台保活/C2 通信；标记所有写入系统分区操作 | E-mg-file-drop | P0 |
| **CY** | 模块含 zygisk/ 目录（arm64-v8a.so 等原生库）；或 config.sh 声明 IS_ZYGISK=true | 提取 zygisk/ 原生库分析 ZygiskModule 回调（onLoad / preAppSpecialize / postAppSpecialize）；检查 JNI Hook | E-mg-zygisk | P0 |
| **DI** | 模块脚本写入 /data/adb/service.d/ 或 /data/adb/post-fs-data.d/；或修改 crontab/init.rc（§7 AT 的深化） | 提取写入 service.d + post-fs-data.d 的脚本内容；检查卸载时自动感染其他模块的逻辑（post-uninstall.sh / 模块目录监控）；检查 magisk --remove-modules 触发保护机制 | E-mg-persistence | P0 |
| **CZ** | 模块脚本含 resetprop 修改系统属性 / magiskhide / DenyList；或集成 Shamiko（隐藏 Zygisk 本身）/ TrickyStore（篡改证书链）/ PlayIntegrityFork（伪造 Play Integrity API） | 提取所有 resetprop 调用识别被修改属性（ro.debuggable / ro.build.tags 等）；检查 DenyList 隐藏自身；识别 Shamiko/TrickyStore/PlayIntegrityFork 模块级反检测 | E-mg-anti-detect | P0 |
| **DA** | 模块脚本含 setenforce 0 / mount -o rw,remount /system / chmod 777 敏感目录 | 检查 SELinux 操作（setenforce/chcon/restorecon）；检查系统分区挂载 + dm-verity 禁用；标记高危提权 | E-mg-privilege | P0 |
| **DB** | 释放 APK/脚本含 curl/wget/HTTP 客户端；或释放 APK 申请 INTERNET + READ_CONTACTS/SMS 等敏感权限 | 提取网络请求目标 URL/IP；分析释放 APK 权限声明；识别数据外传逻辑 | E-mg-c2 | P0 |
| **DC** | 脚本遍历 /data/adb/modules/ 目录、修改其他模块文件、或写入自身副本到其他模块 | 检查 module.prop 注入恶意指令；检查其他模块 service.sh 追加恶意代码；识别"寄生"逻辑 | E-mg-cross-infect | P0 |
---
## 24. 约束（全局）
1. **不平行主流程**：阶段门闩仍以 re-agent-workflow / 各 skill 为准。
2. **Evidence 必记**：含失败、半还原、quality= 标注。
3. **与 A–T 去重**：PE 反调试不重复；AM→R；AJ 补 DLL 视角不推翻 TLS 火箭。
4. **工具缺失**：记 n/a + 手工等价，不假装已用商业套件。
5. **授权**：破坏性/注入/驱动漏洞类只做防御分析与取证表述。
6. **扩展规则去重**：ELF → elf-analysis.md；Mach-O → platforms.md；Python → languages.md。本表不重复这些格式的规则。
## 25. P0 最小勾选（类型命中时）
```text
□ bat/cmd → U（+ 需要时 V/W；高级 CD–CI）
□ ps1 → X（+ Z；高级 CJ–CO + DL ETW）
□ vba/xlm → AA + DD + DJ（+ AB/AC）
□ office ooxml/rtf → BA + DK（+ BB 若疑 DDE）
□ js 强混淆 → AD 或 AE（+ AF；高级 CP/CQ/DE/DF）
□ sys → AG + AH（+ AI 若疑 BYOVD）
□ dll → AJ + AK/AL；Delay-Load 走 R
□ apk 破坏/隐藏 → AR/AS 或 AU（+ AT/AV）
□ pdf → AW + AX（+ AY/AZ）
□ wasm → BC（+ BD/BE）
□ jar/class → BF + BG（+ BH/BI）
□ autoit → BJ + BL + DM（+ BK）
□ hta → BM + BN（+ BO）
□ wsf/jse/vbe → BP + BQ（+ BR/BS）
□ msi → BT（+ BU/BV）
□ reg → BW + BX（+ BY）
□ vbs → CA + CB + CC + DN（+ BZ）
□ xposed 模块 → CR + CS + CT + CU（+ CV–CX）
□ magisk 深度 → DG + DH + CY + DI + CZ + DA（+ DB/DC）
```

501
reverse-engineering/references/ollvm-deobfuscation.md Normal file
View File

@@ -0,0 +1,501 @@
# OLLVM 反混淆 / Obfuscator-LLVM Deobfuscation
> 面向 APK .so、ELF 二进制和控制流平坦化场景的 OLLVM 脱密工作流。
> 工具与变种信息基于 2026 年社区活跃项目调研，非训练记忆。
> 适用：Android NDK 加固、CTF 逆向、加壳 .so 分析、商业混淆器对抗。
---
## 0. 快速决策：我该用哪个工具？
根据你的环境和对目标混淆类型的判断，直接对号入座：
| 你的情况 | 首选工具 | 备选 | 说明 |
|---------|---------|------|------|
| 有 IDA Pro 7.5-7.7 + Hex-Rays，想一键去平坦化 | **obpo-plugin** | d810-ng | obpo 用 microcode + 数据流 + 混合执行，效果最强，但是云插件（需联网，核心闭源） |
| 有 IDA Pro（任意较新版本），想本地一站式反混淆 | **d810-ng** | D-810 原版 | 本地、开源、集成 Z3，支持 OLLVM/Tigress/Hodur/Approov 多变种 |
| 有 Binary Ninja | **ollvm-breaker** | — | 针对 Android .so 实战（libvdog 等加固样本） |
| 无 IDA/BN，纯脚本，目标 x86/x64 | **ollvm-unflattener** (Miasm) | angr deflat | 基于 Miasm 符号执行，BFS 多层处理 |
| 无 IDA/BN，纯脚本，目标 x86/x64 | **ollvm-unflattener** (Miasm) | angr deflat | 基于 Miasm 符号执行，BFS 多层处理 |
| 纯 Python 符号执行，CTF 场景 | **angr** Deobfuscator | Triton | 不依赖 GUI，脚本化 |
| 目标是 ARM64 .so，无 IDA | **deollvm** (Unicorn) | angr | 基于 Unicorn 的 ARM64 deflat |
| 遇到 BR 混淆（间接分支） | **DeObfBR** | 设置数据段只读 | Goron/Arkari 风格的 BR 混淆可被数据段只读简单对抗 |
| 遇到 Tigress 混淆 | d810-ng `UnflattenerSwitchCase`/`UnflattenerTigressIndirect` | — | d810-ng 内置 Tigress 专用 unflattener |
> **核心建议：** 优先用 **d810-ng**（本地、维护活跃、变种覆盖广）。云服务可用时 **obpo-plugin** 效果最好。两者都失败再上 **angr/Miasm** 符号执行做定制化处理。
---
## 1. 现代 OLLVM 变种生态（2026 社区调研）
OLLVM 早已不只是 2017 年那个原始仓库。以下是目前活跃的混淆器分支，**脱密前必须先判断目标是哪个变种**，因为不同变种的对抗手段差异很大：
### 1.1 混淆器分支谱系
| 变种 | 基准 LLVM | 相比原始 OLLVM 的新增特性 | 对抗要点 |
|------|----------|----------------------|---------|
| **Obfuscator** (原始) | 3.3~4.0 | sub + bcf + fla（三大基础 pass） | 标准工具均可处理 |
| **Hikari** | 6~8 | Anti Class Dump、Function Call Obfuscate、Function Wrapper、Indirect Branching、Split BB、String Encryption | 需先解密字符串 + 修复间接跳转 |
| **Hikari-LLVM15** | 15~19 | + Anti Debugging、Anti Hook、Constant Encryption | 已闭源；Constant Encryption 增加静态分析难度 |
| **goron** | 7~10 | Indirect Branch/Call/GlobalVariable | ⚠️ Goron 风格间接混淆可被「设置数据段只读」简单对抗 |
| **Arkari** (komimoe/Hikari) | 14~latest | 基于 goron，持续维护 | 同 goron，数据段只读即可部分对抗 |
| **Pluto** | 14 | MBA Obfuscation、Random CF、Split BB、**Trap Angr**（专门坑 angr） | ⚠️ Trap Angr pass 会让 angr 符号执行失效，需换工具或绕陷阱 |
| **Polaris** (原 Pluto) | 16 | Alias Access、Indirect Branch/Call、String Encryption、Merge Function、Linear MBA、Dirty Bytes Insertion、Function Splitting、Junk Insertion | 综合 Hikari+Pluto，最棘手，需分层处理 |
| **O-MVLL** | open-obfuscator | Python 驱动 pass manager；Anti Hooking、Arithmetic(MBA)、BB Duplicate、CF Breaking、Function Outline、Indirect Branch/Call、Opaque Constants | 现代 Android 加固常用，Python 配置易定制 |
| **amice** (Rust) | Rust 实现 | 全套 + VM Flatten、Instruction Virtualization、Delayed Offset Loading、Parameter Aggregation | 含 VM 化，需 VM handler 还原而非单纯 deflat |
| **VMP 系** (SmallVmp/VMPilot/xVMP/VMPacker) | — | 指令虚拟化 | **不属于 OLLVM 范畴**，需 VM 逆向，参考 VM 专用工具 |
### 1.2 关键判断线索
- **Trap Angr**（Pluto/Polaris）：如果 angr 跑着跑着爆炸或路径爆炸，怀疑目标用了 Trap Angr pass → 改用 d810-ng 或 Unicorn 动态方法
- **Goron/Arkari 间接跳转**：如果分发器用间接跳转（BR x8 而非 switch），先尝试把相关数据段设为只读，间接跳转目标常变成可静态求解
- **Constant Encryption**（Hikari-LLVM15/Polaris/O-MVLL）：常量在运行期解密，纯静态看不到真实值 → 需要 Unicorn 动态执行解密 stub
- **VM Flatten**（amice）：控制流变成 VM dispatch loop，**不要当普通 fla 处理**，需要先识别 VM handler 表
---
## 2. OLLVM 混淆类型检测
OLLVM 三大核心 pass 的识别特征：
### 2.1 控制流平坦化 (Control Flow Flattening / `fla`)
**IDA 视图特征：**
- 函数入口先跳转到唯一的分发器（dispatcher）块
- 主逻辑被拆成多个基本块，每个块末尾跳回分发器
- 分发器通过**状态变量**（state variable）决定下一个要执行的块
- 巨大的 `switch` 结构，各 case 之间无逻辑关联
```
Original: OLLVM flattened:
block_A entry -> dispatcher
block_B ↓
block_C state_machine:
switch(state):
0 → block_A
1 → block_B
2 → block_C
```
**变种形态（d810-ng 识别的多种 dispatcher）：**
- O-LLVM：switch / if-chain + 状态变量
- Tigress：`m_jtbl`（switch-case）或 `m_ijmp`（间接跳转，需 `goto_table_info` 配置）
- Hodur (PlugX)：嵌套 `while(1)` 状态机，`jnz state, #CONST`，**无 switch 分发器**
- Approov：`while(v8 != C)`，状态常量集中在 `0xF6000–0xF6FFF`
### 2.2 虚假控制流 (Bogus Control Flow / `bcf`)
- 每个真实分支间插入**不可达假分支**
- 假分支用**不透明谓词**保护（条件恒真/恒假，但静态分析无法直接证明）
- 大量死代码膨胀函数体积
```c
// 经典不透明谓词：x(x+1) 必为偶数，编译器证明不了
if (x * (x + 1) % 2 == 0) {
// 真实逻辑
} else {
// 不可达垃圾代码
}
```
### 2.3 指令替换 (Instruction Substitution / `sub`) → MBA
- 简单算术/位运算替换为等价复杂表达式（MBA, Mixed Boolean-Arithmetic）
```
a + b → (a ^ b) + 2*(a & b)
a ^ b → (a | b) - (a & b)
a - b → a + (~b) + 1
```
### 2.4 快速分类表
| 混淆类型 | IDA 特征 | 主要对抗手段 |
|---------|---------|------------|
| fla (平坦化) | 巨大 switch + 分发器 | obpo / d810-ng / deflat |
| bcf (虚假控制流) | 不可达分支 + 死代码 | d810-ng opaque predicate removal / 符号执行 |
| sub/MBA | 复杂算术表达式 | d810-ng MBA simplifier / SiMBA (Z3) |
| fla + bcf + sub | 全上，极大膨胀 | **分层去混淆（先 bcf 再 fla 再 sub）** |
---
## 3. 主流工具详解（社区活跃项目）
### 3.1 obpo-plugin — 最强效果，云插件
> [obpo-project/obpo-plugin](https://github.com/obpo-project/obpo-plugin) · 629⭐ · 2026-06 活跃
基于 Hex-Rays **microcode** 的伪代码优化器，使用**数据流跟踪 + 程序切片 + 混合执行（concolic）**重建平坦化控制流。效果是社区公认最强之一。
**关键特性：**
- 在 microcode 层操作，直接优化反编译输出（不是改 ASM）
- 支持 IDA 7.5.0 / 7.6.0 / 7.7.0 + Hex-Rays
- 架构：ARM, ARM64, x86, x86_64, PowerPC, PowerPC64, MIPS（7.6/7.5）
- **云插件**：目标函数二进制会上传到 obpo-server 处理（核心闭源，插件免费开源）
- 服务器自费维护，超时 600s，**禁止多线程/恶意调用**
**安装与使用：**
```text
1. 下载 obpo_plugin.py 和 obpoplugin 目录
2. 复制到 IDA plugins 路径
3. 重启 IDA，打开目标二进制
4. 在 CFG 中定位分发块（dispatcher），通常长这样：
[截图参考仓库 assets/dispatchblock.png]
5. 右键 → OBPO → Mark and process function
6. 处理完成后刷新反编译器
7. 可根据反编译变化继续标记新的分发块（迭代处理嵌套 fla）
```
**适用场景与限制：**
- ✅ 标准和嵌套 fla，效果好
- ⚠️ 需要联网，敏感样本（内部未公开漏洞、商业机密）慎用——二进制会上传
- ⚠️ 服务器可能宕机，依赖作者维护
- ❌ 不能解决所有混淆（作者明确声明）
### 3.2 d810-ng — 本地一站式首选
> [w00tzenheimer/d810-ng](https://github.com/w00tzenheimer/d810-ng) · 223⭐ · 2026-06-26 更新
D-810 的现代维护/重构版（Next Generation）。本地运行、开源、集成 **Z3 SMT** 求解器，变种覆盖最广。
**核心能力（按 d810-ng README 整理）：**
*指令级优化：*
| 类别 | 说明 |
|------|------|
| MBA simplification | `(a+b)-2*(a&b) => a^b`，Z3 验证的 DSL 规则 |
| Hacker's Delight | 位运算等价（来自 Hacker's Delight 一书） |
| O-LLVM patterns | Obfuscator-LLVM 专用 MBA 模式 |
| Constant folding | 22 条常量简化规则 |
| Predicate simplification | 不透明谓词去除（setz/setnz/lnot/smod） |
| Z3 rules | 模板匹配失败时用 SMT 求解 |
| Hodur-specific | PlugX (Hodur) 恶意软件的 MBA 模式 |
*控制流 Unflattener（按目标混淆分类）：*
| Unflattener | 目标 | 说明 |
|------------|------|------|
| `Unflattener` | O-LLVM | 标准 switch/if-chain + 状态变量 |
| `UnflattenerSwitchCase` | Tigress | Tigress switch-case 分发（`m_jtbl`） |
| `UnflattenerTigressIndirect` | Tigress | Tigress 间接跳转（`m_ijmp`），需 `goto_table_info` 配置 |
| `HodurUnflattener` | Hodur (PlugX) | 嵌套 `while(1)` + `jnz state, #CONST`，无 switch |
| `BadWhileLoop` | Approov | `while(v8 != C)`，状态常量在 0xF6000–0xF6FFF |
| `UnflattenerFakeJump` | 通用 | 去除恒真/恒假的条件跳转 |
| `SingleIterationLoopUnflattener` | 残留 | 清理 `INIT == CHECK` 且 `UPDATE != CHECK` 的单次循环 |
| `UnflattenControlFlowRule` (实验) | 通用 | 基于 path emulation 的 CFG unflattener |
**安装与使用：**
```text
1. clone d810-ng
2. 安装依赖（含 Z3）
3. 复制到 IDA plugins 目录
4. IDA 中按 Ctrl-Shift-D 加载插件
5. 在 GUI 中勾选要应用的规则集
6. 对目标函数应用
```
**为什么选 d810-ng 而非原版 D-810：**
- 原版 D-810 已较少维护
- d810-ng 有 CI 测试、重构代码、新增 Tigress/Hodur/Approov 专用 unflattener
- 集成 Z3，模板匹配失败时回退到 SMT 求解，成功率更高
### 3.3 ollvm-unflattener — Miasm 符号执行，纯脚本
> [cdong1012/ollvm-unflattener](https://github.com/cdong1012/ollvm-unflattener) · 265⭐ · 2026-06 活跃
基于 **Miasm** 符号执行引擎，不依赖 IDA/BN，纯 Python 命令行。
**特性：**
- 用 Miasm 符号执行恢复原始控制流（区别于 MODeflattener 的纯静态方法）
- **BFS 多层处理**：自动跟随目标函数的调用，递归去混淆
- 支持 Windows/Linux x86/x64
- 输出去混淆后的新二进制
**安装与使用：**
```bash
git clone https://github.com/cdong1012/ollvm-unflattener.git
cd ollvm-unflattener
pip install -r requirements.txt # miasm, graphviz, keystone-engine
# 基本用法
python unflattener -i <input.bin> -o <output.bin> -t <function_addr> -a
# -a: 自动跟随调用做多层处理
```
**适用：** 无 IDA、目标 x86/x64、需要批量脚本化处理。
### 3.4 ollvm-breaker — Binary Ninja 实战
> [amimo/ollvm-breaker](https://github.com/amimo/ollvm-breaker) · 441⭐
使用 **Binary Ninja** 去平坦化，仓库自带 Android 加固样本 `libvdog.so` 作为测试用例，已修复 JNI_OnLoad、crazy::GetPackageName、prevent_attach_one 等函数。
**适用：** Binary Ninja 用户、Android .so 实战。
### 3.5 deollvm — ARM64 Unicorn
> [GeT1t/deollvm](https://github.com/GeT1t/deollvm) · 34⭐ · 2026-04
基于 **Unicorn** 的 ARM64 OLLVM deflat。无 IDA 时处理 ARM64 .so 的备选。
### 3.6 DeObfBR — BR 混淆专项
> [Mrack/DeObfBR](https://github.com/Mrack/DeObfBR) · 96⭐ · 2026-06-25
专门去除 **BR 混淆**（间接分支混淆，Goron/Arkari 风格）。
**⚠️ 简易对抗技巧（来自 awesome-ollvm）：** Goron/Arkari 风格的间接相关混淆，可以通过**设置数据段为只读**来简单对抗——间接跳转目标常依赖运行期可写的数据段，设只读后变成可静态求解。
### 3.7 angr — 符号执行通用框架
```python
import angr
proj = angr.Project("target.so", auto_load_libs=False)
cfg = proj.analyses.CFGFast()
func = proj.kb.functions[0x12345]
# 内置 Deobfuscator
deob = proj.analyses.Deobfuscator(func=func)
deob.normalize()
```
**⚠️ Pluto/Polaris 的 Trap Angr pass：** 这两个变种专门写了 trap 来坑 angr 符号执行。如果 angr 路径爆炸或异常，怀疑目标用了 Trap Angr → 改用 d810-ng 或 Unicorn 动态方法。
---
## 4. 完整脱密工作流（按场景）
### 4.1 通用决策树
```
目标二进制
↓
1. 识别 OLLVM 变种（看 1.2 节线索）
├── 原始 OLLVM / Hikari / O-MVLL → 标准 fla/bcf/sub
├── Pluto / Polaris → 注意 Trap Angr，避开 angr
├── Goron / Arkari → 先试数据段只读，再处理 BR
├── Tigress → d810-ng Tigress unflattener
├── Hodur (PlugX) → d810-ng HodurUnflattener
└── amice (含 VM) → 不是单纯 fla，需 VM handler 还原
↓
2. 选择工具（看第 0 节决策表）
├── 有 IDA + 可联网 + 非敏感样本 → obpo-plugin
├── 有 IDA + 本地 → d810-ng
├── 有 Binary Ninja → ollvm-breaker
├── 无 GUI + x86/x64 → ollvm-unflattener (Miasm)
├── 无 GUI + ARM64 → deollvm (Unicorn) / angr
└── 纯符号执行 / CTF → angr
↓
3. 分层去混淆（顺序很重要）
a) 先去除不透明谓词 (bcf) → d810-ng opaque predicate removal
b) 再去除控制流平坦化 (fla) → unflattener
c) 最后简化 MBA (sub) → d810-ng MBA simplifier / SiMBA
↓
4. 验证
├── 函数体积显著减小？
├── CFG 从星形/放射状变为链形/树形？
└── Frida hook 关键函数验证逻辑正确？
```
### 4.2 Android NDK .so 脱密专项
Android NDK 编译的 .so 经 OLLVM 加固是 APK 逆向最常见的场景。
**Step 1 — 提取 .so：**
```bash
adb pull /data/app/~~/lib/arm64/libnative.so
# 或从 APK 直接解压：unzip target.apk -d out/ ; find out -name "*.so"
```
**Step 2 — 识别 OLLVM 与变种：**
```bash
readelf -a libnative.so | grep -E "Size|text" # .text 异常大但函数少 → 大概率 OLLVM
# IDA 打开看函数特征：
# 巨大 switch → fla
# 不可达分支 → bcf
# 复杂算术 → sub/MBA
# 间接跳转 BR x8 → Goron/Arkari，试数据段只读
# while(1) + jnz state → Hodur，用 d810-ng HodurUnflattener
```
**Step 3 — 脱密（分层）：**
```
a) bcf: d810-ng opaque predicate removal (或 obpo 自动处理)
b) fla: d810-ng Unflattener / obpo-plugin / deollvm(ARM64)
c) sub: d810-ng MBA simplifier
```
**Step 4 — Frida 动态验证：**
```javascript
// Trace OLLVM 状态变量，辅助 deflat 确定状态变量地址
const target = Module.findBaseAddress("libnative.so");
console.log("[+] libnative.so @", target);
// 在分发器入口下 hook，观察 state 变化序列
Interceptor.attach(target.add(0x1234), { // dispatcher offset
onEnter(args) {
// 读取状态变量（需根据反编译确定寄存器/栈位置）
console.log("[state]", this.context.x8); // 假设 state 在 x8
}
});
```
### 4.3 CTF 场景快速脱密
CTF 通常时间紧，优先用最快路径：
```python
#!/usr/bin/env python3
"""CTF OLLVM quick deflat with angr"""
import angr
proj = angr.Project("challenge", auto_load_libs=False)
cfg = proj.analyses.CFGFast()
# 找最大的几个函数（最可能是被混淆的）
funcs = sorted(cfg.functions.values(), key=lambda f: f.size, reverse=True)[:5]
for func in funcs:
print(f"[*] {func.name} @ {hex(func.addr)} size={hex(func.size)}")
try:
deob = proj.analyses.Deobfuscator(func=func)
deob.normalize()
print(f" [+] deobfuscated")
except Exception as e:
print(f" [-] failed: {e}")
# angr 失败 → 怀疑 Trap Angr → 换 d810-ng / Unicorn
```
---
## 5. MBA 表达式简化
### 5.1 常见 OLLVM MBA 模式
```python
# 这些等式是 OLLVM sub pass 生成表达式的化简目标
"(a | b) + (a & b)" # → a + b
"(a | b) - (a & b)" # → a ^ b
"(a ^ b) + 2*(a & b)" # → a + b
"(a | b) & ~(a & b)" # → a ^ b
"~(~a & ~b)" # → a | b (De Morgan)
```
### 5.2 工具选择
| 工具 | 方式 | 适用 |
|------|------|------|
| **d810-ng MBA simplifier** | IDA 内批量，Z3 验证 | 首选，集成在反编译流程 |
| **SiMBA** (`pip install simba-simplifier`) | 命令行/库 | 纯表达式化简，批量处理 |
| **Arybo** | 符号位向量 | 大量 MBA 表达式 |
| **Z3 直接求解** | SMT | 最通用，模板匹配都失败时 |
```python
# SiMBA 示例
from simba import simplify_mba
exprs = ["(a | b) + (a & b)", "(a ^ b) + 2*(a & b)"]
for e in exprs:
print(f"{e} → {simplify_mba(e)}")
```
---
## 6. 完整脱密案例脚本
```bash
#!/bin/bash
# OLLVM deobfuscation pipeline (2026 community tools)
# 适用标准 OLLVM / Hikari / O-MVLL 加固的 ELF/.so
BINARY=$1
echo "[*] Stage 0: 基本分析与变种识别"
file $BINARY
readelf -h $BINARY 2>/dev/null | head -5
echo " → 在 IDA 中确认变种（参考第 1 节）"
echo "[*] Stage 1: d810-ng 本地反混淆（首选）"
echo " IDA → Ctrl-Shift-D 加载 d810-ng"
echo " 勾选: MBA + Opaque predicate + Unflattener"
echo " Apply to target functions"
echo " 保存 IDB"
echo "[*] Stage 2: obpo-plugin（如 d810-ng 效果不足且可联网）"
echo " IDA → 右键 dispatcher → OBPO → Mark and process"
echo " ⚠️ 敏感样本勿用（二进制上传云服务）"
echo "[*] Stage 3: 无 IDA 备选（x86/x64）"
echo " python unflattener -i $BINARY -o deobf.bin -t <func_addr> -a"
echo "[*] Stage 4: ARM64 .so 无 IDA 备选"
echo " deollvm (Unicorn) 或 angr Deobfuscator"
echo "[+] Done. 在 IDA 中重新分析验证。"
```
---
## 7. 常见陷坑（社区实战总结）
| 问题 | 原因 | 解决办法 |
|------|------|---------|
| angr 路径爆炸/异常退出 | Pluto/Polaris 的 **Trap Angr** pass | 换 d810-ng 或 Unicorn 动态方法 |
| obpo-plugin 联不上 | 服务器自费维护，可能宕机 | 转用本地 d810-ng；可在 obpo 仓库提 issue |
| Goron/Arkari 间接跳转 deflat 失败 | 分发器用 BR x8 而非 switch | 先把数据段设只读，再用 DeObfBR |
| d810-ng 处理后函数仍乱 | OLLVM 自定义了 pass 参数/seed | 先符号执行去不透明谓词，再 unflatten |
| 嵌套 fla（多层平坦化）一次没清干净 | obpo/d810-ng 单次只清一层 | **迭代处理**：每次标记新出现的 dispatcher |
| ARM64 .so 用 deflat 报错 | 老 deflat 脚本只支持 x86 | 用 d810-ng / obpo（支持 ARM64）/ deollvm |
| Hikari 字符串看不到 | String Encryption pass | 用 Unicorn 模拟解密 stub，dump 解密后字符串 |
| amice 目标 deflat 完全无效 | 含 VM Flatten / Instruction Virtualization | **不是 OLLVM fla**，需 VM handler 还原（参考 VM 逆向） |
| Hodur(PugX) 样本没有 switch 分发器 | 嵌套 while(1) + jnz state | 用 d810-ng **HodurUnflattener**，别用普通 Unflattener |
| Approov 样本状态常量看不出规律 | 常量集中在 0xF6000–0xF6FFF | 用 d810-ng **BadWhileLoop** unflattener |
| 敏感样本误用 obpo | 二进制上传云服务 | 涉密/未公开漏洞样本**只用本地工具**（d810-ng/angr） |
| Frida hook OLLVM 函数卡死 | 状态变量被改导致无限循环 | 在分发器入口加条件断点限制执行次数 |
---
## 8. 工具速查表（2026 社区活跃度）
| 工具 | 平台 | 方式 | Stars/价 | 最近更新 | 开源 | 备注 |
|------|------|------|---------|---------|------|------|
| **obpo-plugin** | IDA | microcode+concolic（云） | 629 | 2026-06 | 插件开源/核心闭源 | 效果最强，需联网 |
| **ollvm-breaker** | Binary Ninja | BN API | 441 | 2026-06 | ✅ | Android .so 实战 |
| **ollvm-unflattener** | CLI | Miasm 符号执行 | 265 | 2026-06 | ✅ | x86/x64，BFS 多层 |
| **d810-ng** | IDA | microcode+Z3 | 223 | 2026-06 | ✅ | **本地首选**，变种覆盖广 |
| **DeObfBR** | — | BR 混淆专项 | 96 | 2026-06 | ✅ | Goron/Arkari 间接分支 |
| **IDA_Ollvm-unflattener** | IDA | Miasm 插件版 | 90 | 2026-04 | ✅ | ollvm-unflattener 的 IDA 插件封装 |
| **deollvm** | CLI | Unicorn | 34 | 2026-04 | ✅ | ARM64 专项 |
| **angr** | CLI | 符号执行 | — | 活跃 | ✅ | 通用，被 Trap Angr 克制 |
| **SiMBA** | CLI/库 | MBA 化简 | — | — | ✅ | 表达式化简 |
| **Triton** | CLI | 符号执行+污点 | — | 活跃 | ✅ | 动态符号执行 |
---
## 9. 参考链接
**混淆器（用于理解对抗目标）：**
- [obfuscator-llvm/obfuscator](https://github.com/obfuscator-llvm/obfuscator) — 原始 OLLVM
- [HikariObfuscator/Hikari](https://github.com/HikariObfuscator/Hikari) — Hikari
- [komimoe/Hikari](https://github.com/komimoe/Hikari) — Arkari (基于 goron, LLVM 14+)
- [amimo/goron](https://github.com/amimo/goron) — goron
- [bluesadi/Pluto](https://github.com/bluesadi/Pluto) — Pluto
- [za233/Polaris-Obfuscator](https://github.com/za233/Polaris-Obfuscator) — Polaris (原 Pluto)
- [open-obfuscator/o-mvll](https://github.com/open-obfuscator/o-mvll) — O-MVLL
- [fuqiuluo/amice](https://github.com/fuqiuluo/amice) — Rust 实现 OLLVM passes
- [lich4/awesome-ollvm](https://github.com/lich4/awesome-ollvm) — **变种生态总览（强烈推荐先读）**
**反混淆工具：**
- [obpo-project/obpo-plugin](https://github.com/obpo-project/obpo-plugin) — 最强云插件
- [w00tzenheimer/d810-ng](https://github.com/w00tzenheimer/d810-ng) — 本地首选
- [cdong1012/ollvm-unflattener](https://github.com/cdong1012/ollvm-unflattener) — Miasm 纯脚本
- [amimo/ollvm-breaker](https://github.com/amimo/ollvm-breaker) — Binary Ninja
- [GeT1t/deollvm](https://github.com/GeT1t/deollvm) — ARM64 Unicorn
- [Mrack/DeObfBR](https://github.com/Mrack/DeObfBR) — BR 混淆专项
- [maskelihileci/IDA_Ollvm-unflattener](https://github.com/maskelihileci/IDA_Ollvm-unflattener) — IDA 插件版
- [angr](https://angr.io/) — 符号执行框架
- [SiMBA](https://github.com/tech-srl/simba) — MBA 化简
**学术/博客：**
- [Quarkslab: Deobfuscation: Recovering an OLLVM-protected program](https://blog.quarkslab.com/deobfuscation-recovering-an-ollvm-protected-program.html) — deflat 经典原理
- [MODeflattener](https://github.com/mrT4ntr4/MODeflattener) — 静态 deflat（ollvm-unflattener 的对照）
> 关联文档：[[anti-analysis.md]] (反调试/反分析总表)、[[tools-advanced.md]] (高级工具集)、[[elf-analysis.md]] (ELF 文件分析)、[[ai-assisted-re.md]] (AI 辅助逆向)

240
reverse-engineering/references/re-agent-workflow.md Normal file
View File

@@ -0,0 +1,240 @@
# RE Agent 工作流门闩（静态↔动态）
> 来源启发：binary-re 阶段划分、社区 RE skill（Frida/r2/Ghidra/IDA 循环）、Cerberus 三头环（静/动/插桩）
> Issue #65 增量：IAT 修复铁律、六阶段映射、.NET/DLL·SYS 等价路径；用户指令可行性门闩；旁路补丁 6–10；反调试/混淆菜谱 A–T；非 PE 多格式菜谱 U–AV（2026-08-12）
> 适用：`reverse-engineering/`、`ida-reverse/`、`radare2/`、`malware-analysis/`、与 cre 角色交接
## 0. 启动
```text
□ scope.md：offline 样本路径 或 授权设备/靶机
□ tool-index：file/strings/r2/ida/frida 等实际路径
□ 角色：cre（ops/role-map）
```
## 0.1 Transition handoff（decision delta）
阶段之间不重新注入完整 case context。`scope.md` / `workitems.md` / Evidence 保持 authoritative；`timeline.md` 只承载 transition delta：
1. stage/turn 结束时只写真正改变后续动作的 `decision_delta`；无变化写 `[]`。
2. unchanged route/auth/scope/network profile/tool state/hypothesis 只放 `carry_forward_refs`，consumer 依引用读取，不重新 serialize / emit。
3. `decision_delta` 不是完整 state；consumer 必须先继承 refs，再应用 delta。
4. 只有两个或以上 evidence-supported 分支会导致不同下一动作时才停在 next-step menu；确定性 gate 直接推进。
例：Triage 完成且唯一合法下一步是 Static 时，transition 只需 `decision_delta: [phase=triage->static]` + `carry_forward_refs: [scope.md, evidence/E-triage.md]`。
## 0.5 用户指令可行性门闩（Issue #65）
**原则**：服从用户**目标**，不盲从用户**步骤顺序**。跳步前必须说明前提并请确认；确认后的强制步骤必须做，且诚实标注 Evidence 质量。
| 情况 | Agent MUST |
|------|------------|
| 用户要做 X，且当前状态可得到**有效** Evidence | 执行 X，更新 Evidence |
| 用户要做 X，但存在**已知阻塞前提**（例：已判定加壳且静态 IAT 不可读） | **禁止**假装完成有意义的 IAT；① 一句话说清阻塞；② 给出推荐顺序（先脱壳/修 IAT，或直接动态抓 API）；③ **请用户确认**是「仍强制看当前花表」还是「按推荐顺序」 |
| 用户明确**强制**当前步骤（如未脱壳也要看 IAT） | 执行并记 Evidence，MUST 标注 `quality=unreadable` / `packed`（或等价）；**禁止**据此下「无网络能力」等结论 |
| 用户接受推荐顺序 | 先做前提步骤；完成后自动或再应要求做 X；**禁止**用前提步骤（如脱壳）**冒充**「已完成导入表检查」 |
**与「重做 X」的关系**：重做 X 仍 = 重做被点名步骤（或其经确认的合法前提协商结果），禁止偷换成无关步骤。脱壳是导入表的**前提**，不是导入表的**替代品**。
典型冲突：用户在加壳样本上说「先不要脱壳，先看导入表」→ 壳常篡改导入目录/加密描述符，静态表花且无意义 → 走本表「阻塞前提」行，不得默默脱壳冒充，也不得默默交出花表当完成。
## 1. Triage（5–15 分钟 · 强制起点）
```text
□ 计算样本 Hash（MD5/SHA256）→ 唯一 ID
□ 识别文件类型：EXE / DLL / SYS / ELF / Mach-O / .NET / 脚本(bat/ps1/vba) / JS / APK 等
□ 非 PE/脚本/APK/驱动专项：见 §3.4 与 `references/nonpe-format-cookbook.md`（U–AV）
□ file / DIE / 熵 / 壳特征（PEiD / DIE / Exeinfo 等）
□ 架构：x86 / x64 / ARM；编译语言线索（VC++ / Delphi / .NET / Go / Rust）
□ 加壳类型线索：UPX / ASPack / VMProtect / Themida / 未知混淆
□ strings / rabin2 -z 捡漏
□ MUST 导入/导出锚点（见下方「导入表硬门与等价路径」）；若用户抢跑且有壳 → 先走 §0.5
□ 产出：E-triage（MUST 含 imports 或等价锚点分类摘要，含 quality 标注如适用）+ 假设清单
```
**阶段门闩（Triage → Static/Dynamic）**：E-triage 中未记录 imports **或** 合法等价锚点摘要前，MUST NOT 进入 Dynamic（除非已记录 IAT 修复失败并选择动态旁路，见 §1.2），也 MUST NOT 声称「基础分诊完成」。解析失败时仍 MUST 把失败输出写入 Evidence，不得跳过。用户要求「重做导入表检查」时 MUST 重做 imports/等价步骤本身（或先完成 §0.5 协商后的前提），禁止改换其他分析步骤冒充。
### 1.1 导入表硬门与等价路径
| 样本类型 | MUST 锚点（Evidence） | 说明 |
|----------|----------------------|------|
| 原生 PE/ELF/Mach-O（IAT 可读） | `E-imports` / `E-triage-imports`：导入分类摘要 | `rabin2 -i` / IDA imports / 等价 |
| DLL / SYS / 共享库 | **并列** `E-imports` + `E-exports`（`rabin2 -i` + `rabin2 -E`） | 导出表优先级等同导入表（对外入口） |
| .NET 托管（无传统 IAT） | **等价路径**：dnSpy/IL/元数据/程序集引用与敏感 API 摘要 → 仍写入 `E-imports` 或 `E-triage-imports` 语义槽 | **禁止** 因「没有 IAT」而空过硬门；dnSpy 查看 = 原生「查导入表」 |
| 导入表解析失败 / 为空 / 加壳花表 | 仍记失败或花表输出为 Evidence，并标 `quality` | 不得静默跳过；花表不得支撑能力否定结论 |
**干净导入表警告（MUST 提醒）**：若导入表「过干净」（仅 kernel32/ntdll 等基础 DLL、几乎无业务 API），高度怀疑 `LoadLibrary` + `GetProcAddress` 动态加载 → 在 Evidence 注明嫌疑，并 **SHOULD** 转入 Dynamic 抓取内存 API；不得仅凭静态 IAT 宣称「无网络/无文件能力」。
**高危 API 组合（补丁 8 · SHOULD）**：导入表过长时，优先输出**恶意组合聚类**，过滤纯系统基础调用。示例（非穷尽）：
- 高危簇：`FindWindowA/W` + `WriteProcessMemory` + `CreateRemoteThread`（注入）
- 高危簇：`CryptEncrypt` / `CryptAcquireContext` + 大量 `FindFirstFile` / `DeleteFile`（勒索倾向）
- 高危簇：`InternetOpen` / `WinHttp` / `URLDownloadToFile` + 持久化 API（`RegSetValue` / `CreateService`）
- 单独 `CreateFile` / `ReadFile` 等多为良性噪声，除非与上簇共现
### 1.2 脱壳与 IAT 处理（高风险分岔 · Issue #65）
```text
分支 A：无壳 / .NET 托管
→ 直接进入 §2 Static（.NET 走等价锚点）
分支 B：有壳 / 强混淆
Step 1：尝试脱壳（自动脱壳机 / 手动找 OEP）— 须在授权与隔离环境
Step 2：尝试修复 IAT
工具：x86 → ImportREC（或等价）；x64 → Scylla（或等价）。禁止 64 位样本死磕 ImportREC。
情况 B1：修复成功且可解析 → 记 E-imports（修复后）→ §2 Static
情况 B2：ImportREC/Scylla 报错、修复后无法运行、或 IAT 全乱码（VMP/加密壳）
→ 【IAT 修复铁律】立即终止继续静态 IAT 修复
→ MUST 记录 E-iat-repair-fail（命令、工具、失败现象、决定转动态）
→ 直接进入 §3 Dynamic：API 断点 / 硬件执行断点 / 内存搜索抓取导入
→ 这不算「跳过导入表」：导入表路径已尝试并记 Evidence
情况 B3（补丁 6）：脱壳并修 IAT 后双击闪退 / 蓝屏（疑文件 CRC/大小自校验）
→ 放弃继续静态修文件；记 E-self-check-crash 或并入 E-iat-repair-fail
→ 转入 §3 Dynamic：对 CreateFile / GetFileSize / 哈希相关 API 下断，定位校验绕过点
```
**IAT 修复铁律（MUST）**：优先尝试自动/半自动修复；一旦修复工具报错或修复后程序无法运行，**立即停止**在静态导入表上死磕，切换动态调试，用 API 断点（如 `bp CreateFile` / 关键网络 API）在运行时捕获导入函数。
## 2. Static（基础静态锚点 → 深挖）
| 工具 | 何时 |
|------|------|
| radare2 / rabin2 | 快速函数/导入/字符串（imports 已在 Triage MUST 完成或已记失败旁路） |
| IDA / Ghidra（MCP 或 headless） | 深挖、交叉引用、类型；survey 阶段复核 imports 分类 |
| jadx / dnSpy | Android / .NET |
| OLLVM 文档 | 控制流平坦化怀疑 |
```text
□ 确认 E-imports / E-triage 已含导入表或等价锚点 Evidence（缺失则先补，禁止后置）
□ 若 DLL/SYS：确认 E-exports 已记录
□ 敏感 API 分组 + 高危组合聚类（补丁 8）
□ 硬编码域名/IP/URL 字符串；资源节是否藏 Payload
□ 定位关键函数（加密/校验/网络/授权）→ 地址/符号写入 Evidence
□ 一条路不通 → 换工具（IDA↔r2↔Ghidra）
□ 时间盒（补丁 9 · SHOULD 默认）：静态深挖约 15 分钟仍无关键路径 → 强制转入 §3 Dynamic（用户/任务可覆盖时长）
```
**无 MCP 时**：可用导出反编译文本再分析（对照 P4nda0s reverse-skills / IDA-NO-MCP 思路），仍写 Evidence 路径。
## 3. Dynamic（交叉验证循环区）
核心理念：**静态提供线索 → 动态验证 → 验证卡壳 → 回静态重审**（无固定唯一顺序）。
### 3.0 断点起手式（补丁 7 + 10 · MUST 顺序）
在用户态调试器（x64dbg 等）启动样本前，按「四级火箭」预置断点（可因架构/工具略名不同，顺序不变）：
1. **TLS 回调**断点（调试器 EP 之前可能已跑）
2. **入口点 EP** 断点
3. **敏感 API** 断点（如 `CreateRemoteThread` / 网络 / 文件写）
4. **保底**：`ExitProcess` / 进程退出路径断点（补丁 10）— 一旦因反调试直接退出，**不急着重启**；立即 dump memory，将崩溃前镜像路径写入 Evidence，供字符串/数据恢复
```text
□ Frida / x64dbg / gdb / emulator：验证静态假设
□ 按 §3.0 预置断点后再跑；单步跟踪栈/寄存器（白盒）
□ 行为监控：沙箱 / Procmon / RegShot（黑盒）
□ IAT 修复失败 / 自校验闪退样本：硬件执行断点或内存搜索强行捕捉 API；CreateFile/GetFileSize 查 CRC
□ 反调试/反 Frida → reverse-engineering/anti-analysis
□ Android：root 检测 / SSL pinning 绕过脚本按需生成，**须在授权设备**
□ 崩溃日志驱动下一轮 hook（自适应循环）
□ 时间盒（补丁 9 · SHOULD 默认）：单步跟踪约 200 条指令仍无恶意行为线索 → 强制回静态搜字符串/换锚点（可覆盖）
```
### 3.1 沙箱 / 动态无行为应急分支（MUST）
```text
无行为或立刻退出 / 无限休眠
→ 检查反调试 / 反虚拟机例程（CPUID、高精度计时、沙箱特征等）
→ 尝试硬件断点绕过、修补检测点、或换物理机/更高保真环境
→ 将「无行为 + 疑似 anti-VM」写入 Evidence，禁止写成「样本无害」而不加条件
```
### 3.2 时间盒策略（补丁 9 · SHOULD）
| 阶段 | 默认阈值（可被用户/任务覆盖） | 动作 |
|------|------------------------------|------|
| Static 深挖无关键路径 | ~15 分钟 | 转入 Dynamic |
| Dynamic 单步无进展 | ~200 条指令 | 回 Static 字符串/交叉引用重锚 |
| 任一路径重复失败 | 记 Evidence 后换工具或旁路 | 禁止同一失败手法空转 |
### 3.3 反调试 / 混淆旁路速查（Issue #65 补丁 A–T · 高频）
完整索引与动作细节见 `reverse-engineering/anti-analysis.md`「Agent 响应菜谱 A–T」。此处只列 **P0 必查 + 常见转场**。默认 **授权隔离 lab**；patch/改标志位不是未授权生产动作。
| 触发特征 | 首选动作（摘要） | Evidence |
|----------|------------------|----------|
| `cpuid` 后 jz/jnz（A） | lab：改标志位或 patch 走真实分支；记检测点地址 | `E-anti-debug-cpuid` |
| `rdtsc` + sub/cmp（B） | bp rdtsc 或 hook 时间源；禁止无限空转等沙箱超时当「无害」 | `E-anti-debug-rdtsc` |
| PEB BeingDebugged / NtGlobalFlag（K） | ScyllaHide 或手改 PEB；patch 条件跳 | `E-anti-debug-peb` |
| `NtQueryInformationProcess` DebugPort/Flags/Object（P） | ScyllaHide / hook 返回值；记 class 参数 | `E-anti-debug-ntqip` |
| 导入极少但行为丰富 → API 哈希（N） | bp GetProcAddress；哈希反查回注 IDA | `E-api-hash` |
| strings 空但有网/文件行为 → 串加密（I） | 找 decode 例程 xref；解密后 dump 回注 | `E-string-decrypt` |
| 有签名但来源可疑（F） | SigCheck：有效/吊销/时间；**无效不降**威胁等级 | `E-sig-forge` |
| 标准 strings 无 IOC → 试宽字符（T） | `strings -el` / UTF-16LE；Alt+A unicode | `E-wide-strings` |
| 调试器名字符串 / Toolhelp 扫描（C） | bp CreateToolhelp32Snapshot 链 | `E-anti-debug-procscan` |
| AddVectoredExceptionHandler + 故意异常（D） | bp VEH 注册；分析 handler | `E-anti-debug-veh` |
| int3 / DR0–DR7（M） | patch int3；软断点或 ScyllaHide 藏硬件 BP | `E-anti-debug-bp` |
| 多 PE 头/重叠节（G） | 节表真实映射 + 熵；不信节名 | `E-pe-anomaly` |
| 文件尾 > 节总和 Overlay（J） | 提取 overlay；file/熵；找加载偏移 xref | `E-overlay` |
| .rsrc 异常大/高熵 RT_RCDATA（Q） | 提取资源；FindResource 链 + 解密 dump | `E-rsrc-payload` |
| 运行时才加载 DLL（R） | 查 Delay Import；bp delay-load helper | `E-delay-import` |
| while+switch 星形 CFG（H） | **See** `ollvm-deobfuscation.md`；插件失败则动态路径 | `E-cff` |
| 恒真/恒假分支（S） | **See** ollvm / 符号执行；动态为准 | `E-opaque-pred` |
| `/proc/self/status` TracerPid（L） | **Linux/ELF**；hook 或 patch；Windows 主路径不强制 | `E-anti-debug-tracerpid` |
**约束**：绕过失败也记 Evidence；禁止把「反调试触发退出」写成「样本无害」。完整 A–T 与 P2（E 编译时间、O 花指令）见 anti-analysis 菜谱节。
### 3.4 非 PE / 多格式旁路（Issue #65 补丁 U–AV · 路由）
完整索引：`reverse-engineering/references/nonpe-format-cookbook.md`。此处只列 **类型 → 入口**；动作细节在 cookbook / 对应 skill。
| 类型 | 跳转 | P0 Evidence 锚点（示例） |
|------|------|---------------------------|
| BAT/CMD | cookbook §1 + malware | `E-batch-deobf` |
| PowerShell | cookbook §2 + malware | `E-ps-decode-layer-N` |
| VBA 宏 | cookbook §3 + malware | `E-vba-pcode` |
| JS 强混淆 / JSVMP | **js-reverse** + cookbook §4 | `E-js-vmp` / `E-js-deobf` |
| SYS 驱动 | kernel-driver-reverse + cookbook §5 | `E-driver-irp-handlers` / `E-driver-ioctl` |
| DLL 侧重点 | cookbook §6（AM≡A–T **R**） | `E-dll-tls-dllmain` / `E-exports` |
| Android 格机/隐藏图标 | **apk-reverse** + cookbook §7–8 | `E-android-wiper-*` / `E-android-hidden-icon-*` |
**约束**：不另起「非 PE 六阶段」；与 §3.3 A–T 分工（PE 反调试 vs 多格式）。授权 lab；格机/BYOVD/反射 = 检测取证表述。
## 4. Synthesis（IOC / 攻击链 / 报告）
### Decision quality overlay (Issue #77)
Before closing Synthesis, apply [analysis-decision-framework.md](../../ops/analysis-decision-framework.md) **P0 checklist**: R41 grounded claims, R4* validated sufficiency, R1 confidence->dynamic, R2 hypothesis exit, R43 deadlock replan (under feasibility gate), R8/R23 no default malice/IOC. Multi-module -> R50; anti-analysis effort -> R51 + A-T cookbook.
Blindspots (Rust/Go/VMP/injection/OLE/PDF/agent-meta): [analysis-blindspot-cookbook.md](../../ops/analysis-blindspot-cookbook.md) R52-R81 — detection-oriented; not a parallel master flow.
```text
□ Finding：算法/校验逻辑/可利用点 / 行为结论
□ Path：callflow 或 solve 步骤挂 E-*
□ IOC：网络指纹 + 主机指纹（有则表；无则 n/a+原因）
□ 报告 docs-generator（malware/apt/null/vuln overlay 按任务选型）+ 可选图
□ 可选：YARA / Snort·Suricata 规则化沉淀
□ field-journal 脱敏
```
## 5. 六阶段实战映射（Issue #65 思维导图 → 本文件）
| 实战阶段 | 本文件章节 | 硬门 / 铁律 |
|----------|------------|-------------|
| 1 初步快速研判 | §0–§1 Triage | Hash、架构、文件类型、查壳；imports/等价锚点；§0.5 指令门闩 |
| 2 脱壳与 IAT | §1.2 | IAT 铁律；失败/自校验闪退 → Evidence → Dynamic |
| 3 基础静态锚点 | §2 Static | 高危 API 组合；时间盒 SHOULD |
| 4 深度交叉验证 | §3 Dynamic | 断点四级火箭；无行为应急；时间盒；§3.3 A–T；§3.4 U–AV 类型路由 |
| 5 提取 IoC 与攻击链 | §4 Synthesis | IOC + Kill Chain / Path |
| 6 归档与规则化 | §4 + docs-generator / YARA | 结构化报告；规则可选 |
## 6. 与「堆 RE skill 插件」的差异
- 本包用 **阶段门闩 + tool-index**，不默认启用 Hex-Rays「unsafe 全自动执行」类插件
- 动态插桩默认 **offline/lab** network_profile
- IAT/导入表：**尝试 + 记录** 优先于「无限静态死磕」或「静默跳过」
- 用户指令：**目标优先 + 前提协商**，禁止用无关步骤冒充被点名步骤

822
reverse-engineering/tools-advanced.md Normal file
View File

@@ -0,0 +1,822 @@
# CTF Reverse - Advanced Tools & Deobfuscation
Advanced tooling for commercial packers/protectors, binary diffing, deobfuscation frameworks, emulation, and symbolic execution beyond angr.
## Table of Contents
- [VMProtect Analysis](#vmprotect-analysis)
- [Recognition](#recognition)
- [Approach](#approach)
- [Tools](#tools)
- [CTF Strategy](#ctf-strategy)
- [Themida / WinLicense Analysis](#themida--winlicense-analysis)
- [Themida Recognition](#themida-recognition)
- [Approach for CTF](#approach-for-ctf)
- [Binary Diffing](#binary-diffing)
- [BinDiff](#bindiff)
- [Diaphora](#diaphora)
- [Deobfuscation Frameworks](#deobfuscation-frameworks)
- [D-810 (IDA)](#d-810-ida)
- [GOOMBA (Ghidra)](#goomba-ghidra)
- [Miasm](#miasm)
- [Qiling Framework (Emulation)](#qiling-framework-emulation)
- [Triton (Dynamic Symbolic Execution)](#triton-dynamic-symbolic-execution)
- [Manticore (Symbolic Execution)](#manticore-symbolic-execution)
- [Rizin / Cutter](#rizin--cutter)
- [RetDec (Retargetable Decompiler)](#retdec-retargetable-decompiler)
- [Custom VM Bytecode Lifting to LLVM IR (Google CTF 2017)](#custom-vm-bytecode-lifting-to-llvm-ir-google-ctf-2017)
- [Advanced GDB Techniques](#advanced-gdb-techniques)
- [Python Scripting](#python-scripting)
- [Brute-Force with GDB Script](#brute-force-with-gdb-script)
- [Conditional Breakpoints](#conditional-breakpoints)
- [Watchpoints](#watchpoints)
- [Reverse Debugging (rr)](#reverse-debugging-rr)
- [GDB Dashboard / GEF / pwndbg](#gdb-dashboard--gef--pwndbg)
- [Advanced Ghidra Scripting](#advanced-ghidra-scripting)
- [Patching Strategies](#patching-strategies)
- [Binary Ninja Patching (Python API)](#binary-ninja-patching-python-api)
- [LIEF (Library for Instrumenting Executable Formats)](#lief-library-for-instrumenting-executable-formats)
- [GDB Constraint Extraction with ILP/LP Solver (BackdoorCTF 2017)](#gdb-constraint-extraction-with-ilplp-solver-backdoorctf-2017)
- [GDB Position-Encoded Input with Zero Flag Monitoring (EKOPARTY 2017)](#gdb-position-encoded-input-with-zero-flag-monitoring-ekoparty-2017)
- [LD_PRELOAD to Dump Execute-Only Binary (BackdoorCTF 2017)](#ld_preload-to-dump-execute-only-binary-backdoorctf-2017)
---
## VMProtect Analysis
VMProtect virtualizes x86/x64 code into custom bytecode interpreted by a generated VM. One of the most challenging protectors in CTF.
### Recognition
```bash
# VMProtect signatures
strings binary | grep -i "vmp\|vmprotect"
# PE sections: .vmp0, .vmp1 (VMProtect adds its own sections)
readelf -S binary | grep ".vmp"
# Large binary with entropy > 7.5 in certain sections
```
**Key indicators:**
- `push` / `pop` heavy prologues (VM entry pushes all registers to stack)
- Large switch-case dispatcher (the VM handler loop)
- Anti-debug checks embedded in VM handlers
- Mutation engine: same opcode has different handlers per build
### Approach
```text
1. Identify VM entry points — look for pushad/pushaq-like sequences
2. Find the handler table — large indirect jump (jmp [reg + offset])
3. Trace handler execution — each handler ends with jump to next
4. Identify handlers:
- vAdd, vSub, vMul, vXor, vNot (arithmetic)
- vPush, vPop (stack operations)
- vLoad, vStore (memory access)
- vJmp, vJcc (control flow)
- vRet (VM exit — restores real registers)
5. Build disassembler for VM bytecode
6. Simplify / deobfuscate the lifted IL
```
### Tools
- **VMPAttack** (IDA plugin): Automatically identifies VM handlers
- **NoVmp**: Devirtualization via VTIL (open-source)
- **VMProtect devirtualizer scripts**: Community IDA/Binary Ninja scripts
- **Approach for CTF:** Often easier to trace specific operations (crypto, comparisons) than fully devirtualize
### CTF Strategy
```python
# Trace VM execution dynamically to extract operations on flag
# Hook VM handler dispatch to log opcode + operands
import frida
script = """
var vm_dispatch = ptr('0x...'); // Address of handler table jump
Interceptor.attach(vm_dispatch, {
onEnter(args) {
// Log handler index and stack state
var handler_idx = this.context.rax; // or whichever register
console.log('Handler:', handler_idx, 'RSP:', this.context.rsp);
}
});
"""
```
**Key insight:** Full devirtualization is rarely needed for CTF. Focus on tracing what operations are performed on your input. Hook comparison/crypto functions called from within the VM.
---
## Themida / WinLicense Analysis
Similar to VMProtect but with additional anti-debug layers.
### Themida Recognition
- Sections: `.themida`, `.winlice`
- Extremely heavy anti-debug (kernel-level checks, driver installation)
- Code mutation + virtualization + packing combined
### Approach for CTF
1. **Dump unpacked code:** Let it run, dump process memory after unpacking
2. **Bypass anti-debug:** ScyllaHide in x64dbg with Themida-specific preset
3. **Fix imports:** Use Scylla plugin for IAT reconstruction
4. **Focus on dumped code:** Once unpacked, analyze as normal binary
```bash
# x64dbg workflow for Themida:
1. Load binary
2. Enable ScyllaHide → Profile: Themida
3. Run to OEP (Original Entry Point) — may need several attempts
4. Dump with Scylla: OEP → IAT Autosearch → Get Imports → Dump
5. Fix dump: Scylla → Fix Dump
6. Analyze fixed dump in Ghidra/IDA
```
---
## Binary Diffing
Critical for patch analysis, 1-day exploit development, and CTF challenges that provide two versions of a binary.
### BinDiff
```bash
# Export from IDA/Ghidra first, then diff
# IDA: File → BinExport → Export as BinExport2
# Ghidra: Use BinExport plugin
# Command line diffing
bindiff primary.BinExport secondary.BinExport
# Opens in BinDiff GUI — shows matched/unmatched functions
```
**Key metrics:**
- Similarity score (0.0-1.0) per function pair
- Changed instructions highlighted
- Unmatched functions = new/removed code
### Diaphora
Free, open-source alternative to BinDiff, runs as IDA plugin.
```bash
# In IDA:
# File → Script file → diaphora.py
# Export first binary, then open second and diff
# Ghidra version: diaphora_ghidra.py
```
**Useful for CTF:** When challenge provides "patched" and "original" binaries, diff reveals the vulnerability or hidden functionality.
---
## Deobfuscation Frameworks
> OLLVM 脱密的完整工作流、变种生态和社区工具调研见 [ollvm-deobfuscation.md](references/ollvm-deobfuscation.md)。
### d810-ng (IDA) — 本地首选
D-810 的现代维护版（Next Generation），集成 **Z3 SMT** 求解器，覆盖 OLLVM / Tigress / Hodur(PlugX) / Approov 多种变种。
```text
Capabilities:
- MBA simplification (Z3-verified): (a ^ b) + 2*(a & b) → a + b
- Opaque predicate removal (Pred0/PredFF/PredSetz/PredSetnz)
- Constant folding (22 rules)
- Control flow unflattening (多种 unflattener):
* Unflattener → O-LLVM (switch/if-chain)
* UnflattenerSwitchCase → Tigress (m_jtbl)
* UnflattenerTigressIndirect → Tigress (m_ijmp)
* HodurUnflattener → Hodur/PlugX (while(1) + jnz state)
* BadWhileLoop → Approov (0xF6000–0xF6FFF 状态常量)
- Hacker's Delight 位运算等价
- PlugX (Hodur) 恶意软件专用 MBA 模式
Installation: clone → 复制到 IDA plugins 目录 → Ctrl-Shift-D 加载
Source: https://github.com/w00tzenheimer/d810-ng
```
### obpo-plugin (IDA) — 效果最强，云插件
基于 Hex-Rays **microcode** + 数据流跟踪 + 程序切片 + 混合执行（concolic）。社区公认效果最强之一。
```text
- 在 microcode 层优化反编译输出（不是改 ASM）
- 支持 IDA 7.5-7.7 + Hex-Rays，多架构 (ARM/ARM64/x86/x64/PPC/MIPS)
- 云插件：目标函数上传 obpo-server 处理（核心闭源，插件免费）
- ⚠️ 敏感样本慎用（二进制上传云服务）
- 用法：右键 dispatcher → OBPO → Mark and process function
Source: https://github.com/obpo-project/obpo-plugin
```
### ollvm-unflattener — Miasm 纯脚本
无 IDA 依赖，基于 Miasm 符号执行，BFS 多层处理，支持 x86/x64 Win/Linux。
```bash
git clone https://github.com/cdong1012/ollvm-unflattener
pip install -r requirements.txt # miasm, graphviz, keystone-engine
python unflattener -i <input> -o <output> -t <func_addr> -a # -a 自动多层
```
### ollvm-breaker (Binary Ninja)
Binary Ninja 去平坦化，仓库自带 Android 加固样本 `libvdog.so` 实战。Source: amimo/ollvm-breaker
### DeObfBR — BR 混淆专项
专门去除 Goron/Arkari 风格的 BR（间接分支）混淆。简易技巧：设置数据段只读可部分对抗。Source: Mrack/DeObfBR
### deollvm — ARM64 Unicorn
无 IDA 时处理 ARM64 .so 的备选，基于 Unicorn。Source: GeT1t/deollvm
### GOOMBA (Ghidra)
```text
GOOMBA (Ghidra-based Obfuscated Object Matching and Bytes Analysis):
- Integrates with Ghidra's P-Code
- Simplifies MBA expressions
- Pattern matching for known obfuscation
Installation: Copy .jar to Ghidra extensions
Usage: Code Browser → Analysis → GOOMBA
```
### Miasm
Powerful reverse engineering framework with symbolic execution and IR lifting.
```python
from miasm.analysis.binary import Container
from miasm.analysis.machine import Machine
from miasm.expression.expression import *
# Load binary and lift to Miasm IR
cont = Container.from_stream(open("binary", "rb"))
machine = Machine(cont.arch)
mdis = machine.dis_engine(cont.bin_stream, loc_db=cont.loc_db)
# Disassemble function
asmcfg = mdis.dis_multiblock(entry_addr)
# Lift to IR
lifter = machine.lifter_model_call(loc_db=cont.loc_db)
ircfg = lifter.new_ircfg_from_asmcfg(asmcfg)
# Symbolic execution
from miasm.ir.symbexec import SymbolicExecutionEngine
sb = SymbolicExecutionEngine(lifter)
# Execute symbolically, then simplify expressions
```
**Use case:** Deobfuscate expression trees, simplify complex arithmetic, trace data flow through obfuscated code.
---
## Qiling Framework (Emulation)
Cross-platform emulation framework built on Unicorn, with OS-level support (syscalls, filesystem, registry).
```python
from qiling import Qiling
from qiling.const import QL_VERBOSE
# Emulate Linux ELF
ql = Qiling(["./binary"], "rootfs/x8664_linux",
verbose=QL_VERBOSE.DEBUG)
# Hook specific address
@ql.hook_address
def hook_check(ql, address, size):
if address == 0x401234:
ql.arch.regs.rax = 0 # Bypass check
ql.log.info("Anti-debug bypassed")
# Hook syscall
@ql.hook_syscall(name="ptrace")
def hook_ptrace(ql, request, pid, addr, data):
return 0 # Always succeed
# Hook API (Windows)
@ql.set_api("IsDebuggerPresent", target=ql.os.user_defined_api)
def hook_isdebug(ql, address, params):
return 0
ql.run()
```
**Advantages over Unicorn:**
- OS emulation (file I/O, network, registry)
- Multi-platform (Linux, Windows, macOS, Android, UEFI)
- Built-in debugger interface
- Rootfs for library loading
**CTF use cases:**
- Emulate binaries for foreign architectures (ARM, MIPS, RISC-V)
- Bypass all anti-debug at once (no debugger artifacts)
- Fuzz embedded/IoT firmware without hardware
- Trace execution without code modification
---
## Triton (Dynamic Symbolic Execution)
Pin-based dynamic binary analysis framework with symbolic execution, taint analysis, and AST simplification.
```python
from triton import *
ctx = TritonContext(ARCH.X86_64)
# Load binary sections
with open("binary", "rb") as f:
binary = f.read()
ctx.setConcreteMemoryAreaValue(0x400000, binary)
# Symbolize input
for i in range(32):
ctx.symbolizeMemory(MemoryAccess(INPUT_ADDR + i, CPUSIZE.BYTE), f"input_{i}")
# Emulate instructions
pc = ENTRY_POINT
while pc:
inst = Instruction(pc, ctx.getConcreteMemoryAreaValue(pc, 16))
ctx.processing(inst)
# At comparison point, extract path constraint
if pc == CMP_ADDR:
ast = ctx.getPathConstraintsAst()
model = ctx.getModel(ast)
for k, v in sorted(model.items()):
print(f"input[{k}] = {chr(v.getValue())}", end="")
break
pc = ctx.getConcreteRegisterValue(ctx.registers.rip)
```
**Triton vs angr:**
| Feature | Triton | angr |
|---|---|---|
| Execution | Concrete + symbolic (DSE) | Fully symbolic |
| Speed | Faster (concrete-driven) | Slower (explores all paths) |
| Path explosion | Less prone (follows one path) | Major issue |
| API | C++ / Python | Python |
| Best for | Single-path deobfuscation, taint tracking | Multi-path exploration |
**Key use:** Triton excels at deobfuscation — run the program concretely, but track symbolic state, then simplify the collected constraints.
---
## Manticore (Symbolic Execution)
Trail of Bits' symbolic execution tool for native binaries. Its API is similar in spirit to angr.
```python
from manticore.native import Manticore
m = Manticore("./binary")
# Hook success/failure
@m.hook(0x401234)
def success(state):
buf = state.solve_one_n_batched(state.input_symbols, 32)
print("Flag:", bytes(buf))
m.kill()
@m.hook(0x401256)
def fail(state):
state.abandon()
m.run()
```
**Best for:** simpler Linux binaries and embedded targets. angr is generally more mature for complex RE tasks.
---
## Rizin / Cutter
Rizin is the maintained fork of radare2. Cutter is its Qt-based GUI.
```bash
# Rizin CLI (r2-compatible commands)
rizin -d ./binary
> aaa # Analyze all
> afl # List functions
> pdf @ main # Print disassembly
> VV # Visual graph mode
# Cutter GUI
cutter binary # Open in GUI with decompiler
```
**Cutter advantages:**
- Built-in Ghidra decompiler (via r2ghidra plugin)
- Graph view, hex editor, debug panel in one GUI
- Integrated Python/JavaScript scripting console
- Free and open source
---
## RetDec (Retargetable Decompiler)
LLVM-based decompiler supporting many architectures. Free and open-source.
```bash
# Install
pip install retdec-decompiler
# Or use web: https://retdec.com/decompilation/
# CLI
retdec-decompiler binary
# Outputs: binary.c (decompiled C), binary.dsm (disassembly)
# Specific function
retdec-decompiler --select-ranges 0x401000-0x401100 binary
```
**Strengths:** Multi-arch support (x86, ARM, MIPS, PowerPC, PIC32), free, produces compilable C. Good for architectures not well-supported by Ghidra.
---
## Custom VM Bytecode Lifting to LLVM IR (Google CTF 2017)
For complex custom VMs, transpile the VM bytecode to LLVM IR and use LLVM's optimization passes to simplify the code, then decompile the optimized IR.
```python
# Pipeline: VM bytecode → custom disassembler → LLVM IR → optimize → decompile
# 1. Write disassembler for the custom VM opcodes
# 2. Emit LLVM IR for each opcode:
# INC reg → %reg = add i32 %reg, 1
# CDEC reg → conditional decrement
# CALL fn → call void @fn()
# 3. Use MCJIT or llc to optimize:
# opt -O3 -S vm_lifted.ll -o vm_optimized.ll
# 4. Load optimized IR in IDA or decompile with RetDec
# Result: 1300 lines → 150 lines after inlining + constant folding
```
**Key insight:** LLVM's optimization passes (inlining, constant folding, dead code elimination) dramatically simplify lifted VM bytecode. A custom VM with 26 registers and 3 opcodes that produces 1300 lines of IL reduces to ~150 lines after `-O3`, revealing the underlying algorithm (e.g., Collatz sequence computation).
---
## Advanced GDB Techniques
### Python Scripting
```python
# ~/.gdbinit or source from GDB
import gdb
class TraceCompare(gdb.Breakpoint):
"""Log all comparison operations."""
def __init__(self, addr):
super().__init__(f"*{addr}", gdb.BP_BREAKPOINT)
def stop(self):
frame = gdb.selected_frame()
rdi = int(frame.read_register("rdi"))
rsi = int(frame.read_register("rsi"))
rdx = int(frame.read_register("rdx"))
# Read compared buffers
inferior = gdb.selected_inferior()
buf1 = inferior.read_memory(rdi, rdx).tobytes()
buf2 = inferior.read_memory(rsi, rdx).tobytes()
print(f"memcmp({buf1!r}, {buf2!r}, {rdx})")
return False # Don't stop, just log
# Usage in GDB:
# (gdb) source trace_cmp.py
# (gdb) python TraceCompare(0x401234)
```
### Brute-Force with GDB Script
```python
# Byte-by-byte brute force via GDB Python API
import gdb, string
def bruteforce_flag(check_addr, success_addr, fail_addr, flag_len):
flag = []
for pos in range(flag_len):
for ch in string.printable:
candidate = ''.join(flag) + ch + 'A' * (flag_len - pos - 1)
gdb.execute('start', to_string=True)
gdb.execute(f'b *{check_addr}', to_string=True)
# Write candidate to stdin pipe
# ... (setup input)
gdb.execute('continue', to_string=True)
rip = int(gdb.parse_and_eval('$rip'))
if rip == success_addr:
flag.append(ch)
break
gdb.execute('delete breakpoints', to_string=True)
return ''.join(flag)
```
### Conditional Breakpoints
```bash
# Break only when register has specific value
(gdb) b *0x401234 if $rax == 0x41
(gdb) b *0x401234 if *(char*)$rdi == 'f'
# Break on Nth hit
(gdb) b *0x401234
(gdb) ignore 1 99 # Skip first 99 hits, break on 100th
# Log without stopping
(gdb) b *0x401234
(gdb) commands
> silent
> printf "rax=%lx rdi=%lx\n", $rax, $rdi
> continue
> end
```
### Watchpoints
```bash
# Hardware watchpoint — break when memory changes
(gdb) watch *(int*)0x601050 # Break on write to address
(gdb) rwatch *(int*)0x601050 # Break on read
(gdb) awatch *(int*)0x601050 # Break on read or write
# Watch a variable by name (needs debug symbols)
(gdb) watch flag_buffer[0]
# Conditional watchpoint
(gdb) watch *(int*)0x601050 if *(int*)0x601050 == 0x42
```
### Reverse Debugging (rr)
```bash
# Record execution
rr record ./binary
# Replay with reverse execution support
rr replay
# In rr replay (GDB commands plus):
(gdb) reverse-continue # Run backward to previous breakpoint
(gdb) reverse-stepi # Step backward one instruction
(gdb) reverse-next # Reverse next
(gdb) when # Show current event number
# Set checkpoint and return to it
(gdb) checkpoint
(gdb) restart 1 # Return to checkpoint 1
```
**Key use:** When you step past the critical moment, reverse back instead of restarting. Invaluable for anti-debug that corrupts state.
### GDB Dashboard / GEF / pwndbg
```bash
# pwndbg (most popular for CTF)
# https://github.com/pwndbg/pwndbg
git clone https://github.com/pwndbg/pwndbg && cd pwndbg && ./setup.sh
# Key pwndbg commands:
pwndbg> context # Show registers, stack, code, backtrace
pwndbg> vmmap # Memory map (like /proc/self/maps)
pwndbg> search -s "flag{" # Search memory for string
pwndbg> telescope $rsp 20 # Smart stack dump
pwndbg> cyclic 200 # Generate De Bruijn pattern
pwndbg> hexdump $rdi 64 # Pretty hex dump
pwndbg> got # Show GOT entries
pwndbg> plt # Show PLT entries
# GEF (alternative)
# https://github.com/hugsy/gef
bash -c "$(curl -fsSL https://gef.blah.cat/sh)"
# Key GEF commands:
gef> xinfo $rdi # Detailed info about address
gef> checksec # Binary security features
gef> heap chunks # Heap chunk listing
gef> pattern create 100 # De Bruijn pattern
```
---
## Advanced Ghidra Scripting
```python
# Ghidra Python (Jython) — run via Script Manager or headless
# Batch rename functions matching a pattern
from ghidra.program.model.symbol import SourceType
fm = currentProgram.getFunctionManager()
for func in fm.getFunctions(True):
if func.getName().startswith("FUN_"):
# Check if function contains specific instruction pattern
body = func.getBody()
inst_iter = currentProgram.getListing().getInstructions(body, True)
for inst in inst_iter:
if inst.getMnemonicString() == "CPUID":
func.setName("anti_vm_check_" + hex(func.getEntryPoint().getOffset()),
SourceType.USER_DEFINED)
break
# Extract all XOR constants from a function
def extract_xor_constants(func):
"""Find all XOR operations and their immediate operands."""
constants = []
body = func.getBody()
inst_iter = currentProgram.getListing().getInstructions(body, True)
for inst in inst_iter:
if inst.getMnemonicString() == "XOR":
for i in range(inst.getNumOperands()):
op = inst.getOpObjects(i)
if op and hasattr(op[0], 'getValue'):
constants.append(int(op[0].getValue()))
return constants
# Bulk decompile and search for pattern
from ghidra.app.decompiler import DecompInterface
decomp = DecompInterface()
decomp.openProgram(currentProgram)
for func in fm.getFunctions(True):
result = decomp.decompileFunction(func, 30, monitor)
if result.depiledFunction():
code = result.getDecompiledFunction().getC()
if "strcmp" in code or "memcmp" in code:
print(f"Comparison in {func.getName()} at {func.getEntryPoint()}")
```
---
## Patching Strategies
### Binary Ninja Patching (Python API)
```python
import binaryninja as bn
bv = bn.open_view("binary")
# NOP out instruction
bv.write(0x401234, b"\x90" * 5) # 5-byte NOP
# Patch conditional jump (JNZ → JZ)
bv.write(0x401234, b"\x74") # 0x75 (JNZ) → 0x74 (JZ)
# Insert always-true (mov eax, 1; ret)
bv.write(0x401234, b"\xb8\x01\x00\x00\x00\xc3")
bv.save("patched")
```
### LIEF (Library for Instrumenting Executable Formats)
```python
import lief
# Parse and modify ELF/PE/Mach-O
binary = lief.parse("binary")
# Add a new section
section = lief.ELF.Section(".patch")
section.content = list(b"\xcc" * 0x100)
section.type = lief.ELF.SECTION_TYPES.PROGBITS
section.flags = lief.ELF.SECTION_FLAGS.EXECINSTR | lief.ELF.SECTION_FLAGS.ALLOC
binary.add(section)
# Modify entry point
binary.header.entrypoint = 0x401000
# Hook imported function
binary.patch_pltgot("strcmp", 0x401000)
binary.write("patched")
```
**LIEF advantages:** Cross-format (ELF, PE, Mach-O), Python API, can add sections/segments, modify headers, patch imports.
---
## GDB Constraint Extraction with ILP/LP Solver (BackdoorCTF 2017)
When a binary enforces linear arithmetic relationships between input bytes, extract constraints automatically via GDB and solve with an ILP solver.
**Technique:** Send position-encoded input (`input[i] = i`) so that when a comparison fires, you know exactly which positions are involved and what their sum/difference must equal. Collect all constraints from logged comparisons, then feed to PuLP or Gurobi.
```python
from pulp import *
n = 32 # flag length
prob = LpProblem("crackme", LpMinimize)
x = [LpVariable(f'x{i}', 32, 126, cat='Integer') for i in range(n)]
prob += 0 # dummy objective
# Constraints extracted via GDB automation (input[i]=i, monitor comparisons):
prob += x[3] + x[7] == 0xAB
prob += x[1] - x[5] == 0x0C
# ... add all extracted constraints ...
# Constrain to printable ASCII
for xi in x:
prob += xi >= 32
prob += xi <= 126
prob.solve(PULP_CBC_CMD(msg=0))
flag = ''.join(chr(int(value(xi))) for xi in x)
print("Flag:", flag)
```
**GDB automation to extract constraints:**
```python
# In GDB Python: set input[i]=i, run, log every CMP instruction result
import gdb
class CmpLogger(gdb.Breakpoint):
def stop(self):
frame = gdb.selected_frame()
# Read compared values, map back to input indices via position encoding
return False
```
**Key insight:** When a binary enforces linear arithmetic relationships between input bytes, ILP solvers directly find the satisfying assignment once constraints are extracted via GDB automation.
**References:** BackdoorCTF 2017
---
## GDB Position-Encoded Input with Zero Flag Monitoring (EKOPARTY 2017)
Send input where `input[i] = i` (position-encoded). Single-step through the binary monitoring the CPU zero flag (ZF). When ZF is set at a comparison involving a specific position's value, the comparison matched — log the expected value for that position.
```python
import gdb
# Script: single-step binary with position-encoded input, watch ZF
class ZFMonitor(gdb.Breakpoint):
def stop(self):
zf = (int(gdb.parse_and_eval('$eflags')) >> 6) & 1
if zf:
rip = int(gdb.parse_and_eval('$rip'))
# Disassemble at rip to find the compared immediate
disasm = gdb.execute(f'x/1i {rip-5}', to_string=True)
print(f"ZF set at {rip:#x}: {disasm.strip()}")
return False
# Run once with input b'\x00\x01\x02\x03...\x1f'
# ZF fires when comparison matches the position's own value -> that IS the key byte
```
Maps each input byte to its required value in one pass without manual reversing.
**Key insight:** Position-encoded input (`input[i]=i`) combined with zero flag monitoring reveals the full key/password in one pass — the zero flag fires when the expected value for position i equals i itself.
**References:** EKOPARTY CTF 2017
---
## LD_PRELOAD to Dump Execute-Only Binary (BackdoorCTF 2017)
A binary has execute-only permissions (mode `--x`, no read bit). The file cannot be read directly or with standard tools, but the kernel still maps it into memory on execution.
LD_PRELOAD a shared library with a constructor that runs inside the process and reads its own memory via `/proc/self/mem`:
```c
// dump_xo.c — compile: gcc -shared -fPIC -o dump_xo.so dump_xo.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
__attribute__((constructor)) void dump() {
FILE *maps = fopen("/proc/self/maps", "r");
char line[256];
unsigned long base = 0, end = 0;
// Find the execute-only binary's mapping (r-xp or --xp)
while (fgets(line, sizeof(line), maps)) {
if (strstr(line, "binary_name")) {
sscanf(line, "%lx-%lx", &base, &end);
break;
}
}
fclose(maps);
FILE *mem = fopen("/proc/self/mem", "rb");
fseek(mem, base, SEEK_SET);
size_t size = end - base;
void *buf = malloc(size);
fread(buf, 1, size, mem);
fclose(mem);
FILE *out = fopen("/tmp/dumped_binary", "wb");
fwrite(buf, 1, size, out);
fclose(out);
}
// Usage: LD_PRELOAD=./dump_xo.so ./binary_xo
```
**Key insight:** Execute-only prevents file reading but not execution. LD_PRELOAD constructors run inside the process where `/proc/self/mem` provides access to mapped memory regardless of file permissions.
**References:** BackdoorCTF 2017

846
reverse-engineering/tools-dynamic.md Normal file
View File

@@ -0,0 +1,846 @@
# CTF Reverse - Dynamic Analysis Tools
## Table of Contents
- [Frida (Dynamic Instrumentation)](#frida-dynamic-instrumentation)
- [Installation](#installation)
- [Basic Function Hooking](#basic-function-hooking)
- [Anti-Debug Bypass](#anti-debug-bypass)
- [Memory Scanning and Patching](#memory-scanning-and-patching)
- [Function Replacement](#function-replacement)
- [Tracing and Stalker](#tracing-and-stalker)
- [r2frida (Radare2 + Frida Integration)](#r2frida-radare2--frida-integration)
- [Frida for Android/iOS](#frida-for-androidios)
- [Frida Memoization for Recursive Function Speedup (hxp CTF 2017)](#frida-memoization-for-recursive-function-speedup-hxp-ctf-2017)
- [angr (Symbolic Execution)](#angr-symbolic-execution)
- [angr Installation](#angr-installation)
- [Basic Path Exploration](#basic-path-exploration)
- [Symbolic Input with Constraints](#symbolic-input-with-constraints)
- [Hook Functions to Simplify Analysis](#hook-functions-to-simplify-analysis)
- [Exploring from Specific Address](#exploring-from-specific-address)
- [Common Patterns and Tips](#common-patterns-and-tips)
- [Dealing with Path Explosion](#dealing-with-path-explosion)
- [angr CFG Recovery](#angr-cfg-recovery)
- [lldb (LLVM Debugger)](#lldb-llvm-debugger)
- [Basic Commands](#basic-commands)
- [Scripting (Python)](#scripting-python)
- [x64dbg (Windows Debugger)](#x64dbg-windows-debugger)
- [Key Features](#key-features)
- [Scripting](#scripting)
- [Common CTF Workflow](#common-ctf-workflow)
- [Qiling Framework (Cross-Platform Emulation)](#qiling-framework-cross-platform-emulation)
- [Qiling Installation](#qiling-installation)
- [Basic Usage](#basic-usage)
- [Anti-Debug Bypass via Emulation](#anti-debug-bypass-via-emulation)
- [Input Fuzzing with Qiling](#input-fuzzing-with-qiling)
- [Triton (Dynamic Symbolic Execution)](#triton-dynamic-symbolic-execution)
- [Intel Pin Instruction-Counting Side Channel (Hackover CTF 2015)](#intel-pin-instruction-counting-side-channel-hackover-ctf-2015)
- [Intel Pin Instruction Counting with Genetic Algorithm (hxp CTF 2017)](#intel-pin-instruction-counting-with-genetic-algorithm-hxp-ctf-2017)
- [Opcode-Only Trace Reconstruction (0CTF 2016)](#opcode-only-trace-reconstruction-0ctf-2016)
- [LD_PRELOAD time() Freeze for Deterministic Analysis (EKOPARTY 2017)](#ld_preload-time-freeze-for-deterministic-analysis-ekoparty-2017)
- [LD_PRELOAD memcmp Side-Channel for Byte-by-Byte Bruteforce (Blaze CTF 2018)](#ld_preload-memcmp-side-channel-for-byte-by-byte-bruteforce-blaze-ctf-2018)
---
## Frida (Dynamic Instrumentation)
Frida injects JavaScript into running processes for real-time hooking, tracing, and modification. Essential for anti-debug bypass, runtime inspection, and mobile RE.
### Installation
```bash
pip install frida-tools frida
# Verify
frida --version
```
### Basic Function Hooking
```javascript
// hook.js — intercept a function and log arguments/return value
Interceptor.attach(Module.findExportByName(null, "strcmp"), {
onEnter: function(args) {
this.arg0 = Memory.readUtf8String(args[0]);
this.arg1 = Memory.readUtf8String(args[1]);
console.log(`strcmp("${this.arg0}", "${this.arg1}")`);
},
onLeave: function(retval) {
console.log(` → ${retval}`);
}
});
```
```bash
# Attach to running process
frida -p $(pidof binary) -l hook.js
# Spawn and instrument from start
frida -f ./binary -l hook.js --no-pause
# One-liner: hook strcmp and dump comparisons
frida -f ./binary --no-pause -e '
Interceptor.attach(Module.findExportByName(null, "strcmp"), {
onEnter(args) {
console.log("strcmp:", Memory.readUtf8String(args[0]), Memory.readUtf8String(args[1]));
}
});
'
```
### Anti-Debug Bypass
```javascript
// Bypass ptrace(PTRACE_TRACEME) — returns 0 (success) without calling
Interceptor.attach(Module.findExportByName(null, "ptrace"), {
onEnter: function(args) {
this.request = args[0].toInt32();
},
onLeave: function(retval) {
if (this.request === 0) { // PTRACE_TRACEME
retval.replace(ptr(0));
console.log("[*] ptrace(TRACEME) bypassed");
}
}
});
// Bypass IsDebuggerPresent (Windows)
var isDbg = Module.findExportByName("kernel32.dll", "IsDebuggerPresent");
Interceptor.attach(isDbg, {
onLeave: function(retval) {
retval.replace(ptr(0));
}
});
// Bypass timing checks — hook clock_gettime to return constant
Interceptor.attach(Module.findExportByName(null, "clock_gettime"), {
onLeave: function(retval) {
// Force constant timestamp to defeat timing checks
var ts = this.context.rsi || this.context.x1; // x86 or ARM
Memory.writeU64(ts, 0); // tv_sec
Memory.writeU64(ts.add(8), 0); // tv_nsec
}
});
```
### Memory Scanning and Patching
```javascript
// Scan for flag pattern in memory
Process.enumerateRanges('r--').forEach(function(range) {
Memory.scan(range.base, range.size, "66 6c 61 67 7b", { // "flag{"
onMatch: function(address, size) {
console.log("[FLAG] Found at:", address, Memory.readUtf8String(address, 64));
},
onComplete: function() {}
});
});
// Patch instruction (NOP out a check)
var addr = Module.findBaseAddress("binary").add(0x1234);
Memory.patchCode(addr, 2, function(code) {
var writer = new X86Writer(code, { pc: addr });
writer.putNop();
writer.putNop();
writer.flush();
});
```
### Function Replacement
```javascript
// Replace a validation function to always return true
var checkFlag = Module.findExportByName(null, "check_flag");
Interceptor.replace(checkFlag, new NativeCallback(function(input) {
console.log("[*] check_flag called with:", Memory.readUtf8String(input));
return 1; // always valid
}, 'int', ['pointer']));
```
### Tracing and Stalker
```javascript
// Trace all calls in a function (Stalker — instruction-level tracing)
var targetAddr = Module.findExportByName(null, "main");
Stalker.follow(Process.getCurrentThreadId(), {
transform: function(iterator) {
var instruction;
while ((instruction = iterator.next()) !== null) {
if (instruction.mnemonic === "call") {
iterator.putCallout(function(context) {
console.log("CALL at", context.pc, "→", ptr(context.pc).readPointer());
});
}
iterator.keep();
}
}
});
```
### r2frida (Radare2 + Frida Integration)
```bash
# Attach radare2 to process via Frida
r2 frida://spawn/./binary
# r2frida commands
\ii # List imports
\il # List loaded modules
\dt strcmp # Trace strcmp calls
\dc # Continue execution
\dm # List memory maps
```
### Frida for Android/iOS
```bash
# Android (requires rooted device or Frida server)
adb push frida-server /data/local/tmp/
adb shell "chmod 755 /data/local/tmp/frida-server && /data/local/tmp/frida-server &"
# Hook Android Java methods
frida -U -f com.example.app -l hook_android.js --no-pause
```
```javascript
// hook_android.js — hook Java method
Java.perform(function() {
var MainActivity = Java.use("com.example.app.MainActivity");
MainActivity.checkPassword.implementation = function(input) {
console.log("[*] checkPassword called with:", input);
var result = this.checkPassword(input);
console.log("[*] Result:", result);
return result;
};
});
```
**Key insight:** Frida excels where static analysis fails — obfuscated code, packed binaries, and runtime-generated data. Hook comparison functions (`strcmp`, `memcmp`, custom validators) to extract expected values without reversing the algorithm. Use `Interceptor.attach` for observation, `Interceptor.replace` for modification.
**When to use:** Anti-debugging bypass, extracting runtime-computed keys, hooking crypto functions to dump plaintext, mobile app analysis, packed binary inspection.
### Frida Memoization for Recursive Function Speedup (hxp CTF 2017)
Hook a recursive function with Frida, memoize results, and replay cached values to skip redundant computation. Fibonacci-like recursive challenges with exponential complexity become instant with memoization.
```javascript
// memo_hook.js — memoize a recursive function to skip redundant calls
var memo = {};
var funcAddr = ptr("0x400abc"); // Address of the recursive function
var retAddr = ptr("0x400def"); // Address of the function's ret instruction
Interceptor.attach(funcAddr, {
onEnter: function(args) {
this.key = args[0].toInt32();
if (memo[this.key] !== undefined) {
// Skip computation entirely: set return value and jump to ret
this.context.rax = memo[this.key];
this.context.rip = retAddr;
}
},
onLeave: function(retval) {
// Cache the result for future calls with the same argument
memo[this.key] = retval.toInt32();
}
});
```
```bash
# Usage
frida -f ./binary -l memo_hook.js --no-pause
```
For multi-argument functions, build a composite key:
```javascript
Interceptor.attach(funcAddr, {
onEnter: function(args) {
this.key = args[0].toInt32() + "," + args[1].toInt32();
if (memo[this.key] !== undefined) {
this.context.rax = memo[this.key];
this.context.rip = retAddr;
}
},
onLeave: function(retval) {
memo[this.key] = retval.toInt32();
}
});
```
**Key insight:** Frida's `Interceptor` can both read and modify register state, allowing you to skip function execution entirely by setting `rax` (return value) and `rip` (to the `ret` instruction). This works on any recursive function where the same arguments produce the same result. Exponential-time recursive computations (Fibonacci, Ackermann, tree traversals) become linear with memoization.
**References:** hxp CTF 2017
---
## angr (Symbolic Execution)
angr automatically explores program paths to find inputs satisfying constraints. Solves many flag-checking binaries in minutes that take hours manually.
### angr Installation
```bash
pip install angr
```
### Basic Path Exploration
```python
import angr
import claripy
# Load binary
proj = angr.Project('./binary', auto_load_libs=False)
# Find address of "Correct!" print, avoid "Wrong!" print
# Get these from disassembly (objdump -d or Ghidra)
FIND_ADDR = 0x401234 # Address of success path
AVOID_ADDR = 0x401256 # Address of failure path
# Create simulation manager and explore
simgr = proj.factory.simgr()
simgr.explore(find=FIND_ADDR, avoid=AVOID_ADDR)
if simgr.found:
found = simgr.found[0]
# Get stdin that reaches the target
print("Flag:", found.posix.dumps(0)) # fd 0 = stdin
```
### Symbolic Input with Constraints
```python
import angr
import claripy
proj = angr.Project('./binary', auto_load_libs=False)
# Create symbolic input (e.g., 32-byte flag)
flag_len = 32
flag_chars = [claripy.BVS(f'flag_{i}', 8) for i in range(flag_len)]
flag = claripy.Concat(*flag_chars + [claripy.BVV(b'\n')])
# Constrain to printable ASCII
state = proj.factory.entry_state(stdin=flag)
for c in flag_chars:
state.solver.add(c >= 0x20)
state.solver.add(c <= 0x7e)
# Constrain known prefix: "flag{"
state.solver.add(flag_chars[0] == ord('f'))
state.solver.add(flag_chars[1] == ord('l'))
state.solver.add(flag_chars[2] == ord('a'))
state.solver.add(flag_chars[3] == ord('g'))
state.solver.add(flag_chars[4] == ord('{'))
state.solver.add(flag_chars[flag_len-1] == ord('}'))
simgr = proj.factory.simgr(state)
simgr.explore(find=0x401234, avoid=0x401256)
if simgr.found:
found = simgr.found[0]
result = found.solver.eval(flag, cast_to=bytes)
print("Flag:", result.decode())
```
### Hook Functions to Simplify Analysis
```python
import angr
proj = angr.Project('./binary', auto_load_libs=False)
# Hook printf to avoid path explosion in I/O
@proj.hook(0x401100, length=5) # Address of call to printf
def skip_printf(state):
pass # Do nothing, just skip
# Hook sleep/anti-debug functions
@proj.hook(0x401050, length=5) # Address of call to sleep
def skip_sleep(state):
pass
# Replace a function with a summary
class AlwaysSucceed(angr.SimProcedure):
def run(self):
return 1
proj.hook_symbol('check_license', AlwaysSucceed())
```
### Exploring from Specific Address
```python
# Start from middle of function (skip initialization)
state = proj.factory.blank_state(addr=0x401200)
# Set up registers/memory manually
state.regs.rdi = 0x600000 # Pointer to input buffer
state.memory.store(0x600000, b"AAAA" + b"\x00" * 28)
simgr = proj.factory.simgr(state)
simgr.explore(find=0x401300, avoid=0x401350)
```
### Common Patterns and Tips
```python
# Pattern 1: argv-based input
state = proj.factory.entry_state(args=['./binary', flag_sym])
# Pattern 2: Multiple find/avoid addresses
simgr.explore(
find=[0x401234, 0x401300], # Any success path
avoid=[0x401256, 0x401400] # All failure paths
)
# Pattern 3: Find by output string (no address needed)
def is_successful(state):
stdout = state.posix.dumps(1) # fd 1 = stdout
return b"Correct" in stdout
def should_avoid(state):
stdout = state.posix.dumps(1)
return b"Wrong" in stdout
simgr.explore(find=is_successful, avoid=should_avoid)
# Pattern 4: Timeout protection
simgr.explore(find=0x401234, avoid=0x401256, num_find=1)
# Or use exploration techniques:
simgr.use_technique(angr.exploration_techniques.DFS()) # Depth-first
simgr.use_technique(angr.exploration_techniques.LengthLimiter(max_length=500))
```
### Dealing with Path Explosion
```python
# Use DFS instead of BFS (default) for flag checkers
simgr.use_technique(angr.exploration_techniques.DFS())
# Limit symbolic memory operations
state.options.add(angr.options.ZERO_FILL_UNCONSTRAINED_MEMORY)
state.options.add(angr.options.ZERO_FILL_UNCONSTRAINED_REGISTERS)
# Hook expensive functions (crypto, hashing) to avoid explosion
import hashlib
class SHA256Hook(angr.SimProcedure):
def run(self, data, length, output):
# Concretize input and compute hash
concrete_data = self.state.solver.eval(
self.state.memory.load(data, self.state.solver.eval(length)),
cast_to=bytes
)
h = hashlib.sha256(concrete_data).digest()
self.state.memory.store(output, h)
proj.hook_symbol('SHA256', SHA256Hook())
```
### angr CFG Recovery
```python
# Control flow graph for understanding structure
cfg = proj.analyses.CFGFast()
print(f"Functions found: {len(cfg.functions)}")
# Find main
for addr, func in cfg.functions.items():
if func.name == 'main':
print(f"main at {addr:#x}")
break
# Cross-references
node = cfg.model.get_any_node(0x401234)
print("Predecessors:", [hex(p.addr) for p in cfg.model.get_predecessors(node)])
```
**Key insight:** angr works best on flag-checker binaries with clear success/failure paths. For complex binaries, hook expensive functions (crypto, I/O) and use DFS exploration. Start with the simplest approach (just find/avoid addresses) before adding constraints. If angr is slow, constrain input to printable ASCII and add known prefix.
**When to use:** Flag validators with branching logic, maze/path-finding binaries, constraint-heavy checks, automated binary analysis. Less effective for: heavy crypto, floating-point math, complex heap operations.
---
## lldb (LLVM Debugger)
Primary debugger for macOS/iOS. Also works on Linux. Preferred for Swift/Objective-C and Apple platform binaries.
### Basic Commands
```bash
lldb ./binary
(lldb) run # Run program
(lldb) b main # Breakpoint on main
(lldb) b 0x401234 # Breakpoint at address
(lldb) breakpoint set -r "check.*" # Regex breakpoint
(lldb) c # Continue
(lldb) si # Step instruction
(lldb) ni # Next instruction
(lldb) register read # Show all registers
(lldb) register write rax 0 # Modify register
(lldb) memory read 0x401000 -c 32 # Read 32 bytes
(lldb) x/s $rsi # Examine string (GDB-style)
(lldb) dis -n main # Disassemble function
(lldb) image list # Loaded modules + base addresses
```
### Scripting (Python)
```python
# lldb Python scripting
import lldb
def hook_strcmp(debugger, command, result, internal_dict):
target = debugger.GetSelectedTarget()
process = target.GetProcess()
thread = process.GetSelectedThread()
frame = thread.GetSelectedFrame()
arg0 = frame.FindRegister("rdi").GetValueAsUnsigned()
arg1 = frame.FindRegister("rsi").GetValueAsUnsigned()
s0 = process.ReadCStringFromMemory(arg0, 256, lldb.SBError())
s1 = process.ReadCStringFromMemory(arg1, 256, lldb.SBError())
print(f'strcmp("{s0}", "{s1}")')
# Register in lldb: command script add -f script.hook_strcmp hook_strcmp
```
**Key insight:** Use lldb for macOS binaries (Mach-O), iOS apps, and when GDB isn't available. `image list` gives ASLR slide for PIE binaries. Scripting API is more structured than GDB's.
---
## x64dbg (Windows Debugger)
Open-source Windows debugger with modern UI. Alternative to OllyDbg/WinDbg for Windows RE challenges.
### Key Features
```bash
# Launch
x64dbg.exe binary.exe # 64-bit
x32dbg.exe binary.exe # 32-bit
# Essential shortcuts
F2 → Toggle breakpoint
F7 → Step into
F8 → Step over
F9 → Run
Ctrl+G → Go to address
Ctrl+F → Find pattern in memory
```
### Scripting
```bash
# x64dbg command line
bp 0x401234 # Breakpoint
SetBPX 0x401234, 0, "log {s:utf8@[esp+4]}" # Log string arg on hit
run # Continue
StepOver # Step over
```
### Common CTF Workflow
1. Set breakpoint on `GetWindowTextA`/`MessageBoxA` for GUI crackers
2. Trace back from success/failure message
3. Use **Scylla** plugin for IAT reconstruction on packed binaries
4. **Snowman** decompiler plugin for quick pseudo-C
**Key insight:** x64dbg has built-in pattern scanning, hardware breakpoints, and conditional logging. For Windows CTF binaries, it's often faster than IDA/Ghidra for dynamic analysis. Use the **xAnalyzer** plugin for automatic function argument annotation.
---
## Qiling Framework (Cross-Platform Emulation)
Qiling emulates binaries with OS-level support (syscalls, filesystem, registry). Built on Unicorn but adds the OS layer that Unicorn lacks.
### Qiling Installation
```bash
pip install qiling
# Download rootfs for target OS:
git clone https://github.com/qilingframework/rootfs
```
### Basic Usage
```python
from qiling import Qiling
from qiling.const import QL_VERBOSE
# Linux ELF emulation
ql = Qiling(["./binary", "arg1"], "rootfs/x8664_linux",
verbose=QL_VERBOSE.DEFAULT)
ql.run()
# Windows PE emulation (no Windows needed!)
ql = Qiling(["rootfs/x86_windows/bin/binary.exe"], "rootfs/x86_windows")
ql.run()
# ARM/MIPS emulation (IoT firmware)
ql = Qiling(["rootfs/arm_linux/bin/binary"], "rootfs/arm_linux")
ql.run()
```
### Anti-Debug Bypass via Emulation
```python
from qiling import Qiling
ql = Qiling(["./binary"], "rootfs/x8664_linux")
# Hook ptrace syscall — return 0 (success)
def hook_ptrace(ql, ptrace_request, pid, addr, data):
ql.log.info("ptrace bypassed")
return 0
ql.os.set_syscall("ptrace", hook_ptrace)
# Hook specific address (e.g., anti-VM check)
def skip_check(ql):
ql.arch.regs.rax = 0 # Force success
ql.log.info(f"Skipped check at {ql.arch.regs.rip:#x}")
ql.hook_address(skip_check, 0x401234)
ql.run()
```
### Input Fuzzing with Qiling
```python
# Emulate binary with different inputs to find flag
import string
from qiling import Qiling
def test_input(candidate):
ql = Qiling(["./binary"], "rootfs/x8664_linux",
verbose=QL_VERBOSE.DISABLED, stdin=candidate.encode())
ql.run()
return ql.os.stdout.read()
for ch in string.printable:
output = test_input("flag{" + ch)
if b"Correct" in output:
print(f"Found: {ch}")
```
**Advantages over GDB/Frida:**
- No debugger artifacts (bypasses all anti-debug by default)
- Cross-platform without hardware (ARM, MIPS, RISC-V on x86 host)
- Scriptable with Python (faster iteration than GDB)
- Snapshot/restore for brute-forcing
**Key insight:** Qiling emulates the entire OS layer (syscalls, filesystem, registry), not just the CPU. This means anti-debug checks like `ptrace(TRACEME)` naturally return success without patching, and you can analyze ARM/MIPS binaries on an x86 host without QEMU or real hardware.
**When to use:** Foreign architecture binaries, IoT firmware, heavy anti-debug, automated testing of many inputs.
---
## Triton (Dynamic Symbolic Execution)
See [tools-advanced.md](tools-advanced.md#triton-dynamic-symbolic-execution) for full Triton reference. Quick usage:
```python
from triton import *
ctx = TritonContext(ARCH.X86_64)
# Symbolize input buffer
for i in range(32):
ctx.symbolizeMemory(MemoryAccess(0x600000 + i, CPUSIZE.BYTE), f"flag_{i}")
# Process instructions and collect constraints
# At comparison point, solve for flag
model = ctx.getModel(ctx.getPathConstraintsAst())
flag = ''.join(chr(v.getValue()) for _, v in sorted(model.items()))
```
**Key insight:** Triton excels at single-path DSE (Dynamic Symbolic Execution) where angr's path explosion is a problem. Feed it a concrete execution trace, symbolize specific inputs, and solve for constraints at comparison points. Faster than angr for linear code paths with known execution flow.
**Best for:** Single-path symbolic execution, deobfuscation, taint analysis. Faster than angr for linear code paths.
---
## Intel Pin Instruction-Counting Side Channel (Hackover CTF 2015)
**Pattern:** Brute-force input character-by-character against a binary using Intel Pin's `inscount0` tool. Each correct character causes deeper execution (more instructions) in the comparison logic.
```python
import string
from subprocess import Popen, PIPE
pin = './pin'
tool = './source/tools/ManualExamples/obj-ia32/inscount0.so'
binary = './target'
key = ''
while True:
best_count, best_char = 0, ''
for c in string.printable:
cmd = [pin, '-injection', 'child', '-t', tool, '--', binary]
p = Popen(cmd, stdout=PIPE, stdin=PIPE, stderr=PIPE)
p.communicate((key + c + '\n').encode())
with open('inscount.out') as f:
count = int(f.read().split()[-1])
if count > best_count:
best_count, best_char = count, c
key += best_char
print(f"Found: {key}")
```
**Key insight:** Movfuscated binaries (compiled with `movfuscator`) expand every instruction into sequences of `mov` operations, making static analysis impractical. However, character-by-character comparison still creates measurable instruction count differences. Pin's `inscount0.so` counts total executed instructions — the correct character at each position causes ~1000+ more instructions (proceeding further in the comparison). Also works for obfuscated binaries with sequential input checks.
---
### Intel Pin Instruction Counting with Genetic Algorithm (hxp CTF 2017)
For self-modifying code that decrypts the next chunk only after each character check passes, standard character-by-character Pin counting fails because the search space is too large and characters may interact. Use a genetic algorithm instead to explore the input space more efficiently.
```python
import subprocess
import random
import string
PIN_PATH = '/tmp/pin-3.5/pin'
TOOL_PATH = 'source/tools/ManualExamples/obj-intel64/inscount0.so'
def fitness(candidate):
"""Run binary under Pin and return instruction count as fitness."""
proc = subprocess.Popen(
[PIN_PATH, '-t', TOOL_PATH, '--', './binary'],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate(candidate.encode())
# inscount0 writes count to stderr or inscount.out
try:
with open('inscount.out') as f:
return int(f.read().split()[-1])
except:
return 0
def mutate(individual, rate=0.1):
"""Randomly mutate characters in the individual."""
result = list(individual)
for i in range(len(result)):
if random.random() < rate:
result[i] = random.choice(string.printable[:62])
return result
# Genetic algorithm parameters
FLAG_LEN = 40
POP_SIZE = 100
SURVIVORS = 20
# Initialize random population
population = [random.choices(string.printable[:62], k=FLAG_LEN) for _ in range(POP_SIZE)]
for generation in range(10000):
# Score each individual by instruction count
scored = [(fitness(''.join(p)), p) for p in population]
scored.sort(reverse=True)
best_score, best_individual = scored[0]
print(f"Gen {generation}: {best_score} {''.join(best_individual)}")
# Keep top survivors, mutate to refill population
survivors = [s[1] for s in scored[:SURVIVORS]]
population = survivors + [mutate(random.choice(survivors)) for _ in range(POP_SIZE - SURVIVORS)]
```
**Modified Pin for Go binaries (table-lookup flag checking):**
When standard `inscount` fails because counter increments don't correlate with correctness (e.g., table-lookup comparison), modify Pin's icount tool to only count executions at the success-branch address. Brute-force character-by-character with this targeted counter:
```cpp
// Modified inscount0.cpp — count only executions of a specific address
static ADDRINT target_addr = 0x401234; // success-branch address
static UINT64 target_count = 0;
VOID CountAtTarget(ADDRINT ip) {
if (ip == target_addr) target_count++;
}
VOID Instruction(INS ins, VOID *v) {
INS_InsertCall(ins, IPOINT_BEFORE, (AFUNPTR)CountAtTarget,
IARG_INST_PTR, IARG_END);
}
```
**Key insight:** When each correct character unlocks a new code section (self-modifying or multi-stage decryption), instruction count increases monotonically with correctness. A genetic algorithm explores the input space more efficiently than character-by-character brute-force because it can discover multiple correct characters simultaneously. Converges in approximately 30 minutes for 40-character flags. For table-lookup comparisons where total instruction count doesn't correlate, target a specific branch address instead.
**References:** hxp CTF 2017
---
## Opcode-Only Trace Reconstruction (0CTF 2016)
Given an execution trace with only opcodes (no register/memory values), reconstruct the program: sort/dedup trace by address, split into basic blocks, annotate functions. Sorting algorithms are particularly vulnerable -- branch decisions leak element ordering.
**Approach:**
1. Sort trace entries by address, deduplicate to recover code layout
2. Identify basic block boundaries (jumps, calls, returns)
3. Map branch taken/not-taken decisions from trace order
4. For sorting algorithms, partition comparisons reveal relative ordering of all input elements
**Key insight:** Execution traces without data values still leak information through branch decisions. Quicksort partition comparisons reveal which element is greater/lesser at each step, enabling full recovery of the sorted input from branch direction alone.
---
## LD_PRELOAD time() Freeze for Deterministic Analysis (EKOPARTY 2017)
Override `time()` via LD_PRELOAD to return a constant value, freezing any timestamp-seeded PRNG. Once the binary's cipher becomes deterministic, brute-force each output byte without understanding the VM or cipher internals.
```c
// freeze_time.c — compile: gcc -shared -fPIC -o freeze.so freeze_time.c
#include <time.h>
time_t time(time_t *t) {
if (t) *t = 1234567890;
return 1234567890;
}
```
```bash
# Build and use:
gcc -shared -fPIC -o freeze.so freeze_time.c
LD_PRELOAD=./freeze.so ./binary
# Byte-at-a-time oracle: run with frozen time, try each candidate byte,
# observe output — correct byte produces expected output character.
for byte in $(seq 0 255); do
output=$(echo -n "$(printf '\x%02x' $byte)" | LD_PRELOAD=./freeze.so ./binary)
# Check output against known/expected
done
```
If `srand()` or `rand()` is also involved, override `rand()` too:
```c
int rand(void) { return 42; }
```
**Key insight:** LD_PRELOAD function interception freezes non-determinism sources (time, rand). Once deterministic, even complex VMs become tractable byte-at-a-time oracles.
**References:** EKOPARTY CTF 2017
---
### LD_PRELOAD memcmp Side-Channel for Byte-by-Byte Bruteforce (Blaze CTF 2018)
**Pattern:** Replace `memcmp` with an LD_PRELOAD library that returns the number of matching bytes instead of the standard -1/0/1 result. This converts any memcmp-based validation into a byte-by-byte oracle. Automate with GDB Python scripting to bruteforce each character position.
```c
// memcmp_hook.c - compile: gcc -shared -fPIC -o hook.so memcmp_hook.c
int memcmp(const char *s1, const char *s2, int n) {
int cnt = 0;
for (int i = 0; i < n; ++i) {
if (s1[i] == s2[i]) cnt++;
else break;
}
return cnt;
}
```
```bash
# Use with GDB: LD_PRELOAD=./hook.so gdb ./binary
# Set breakpoint after memcmp, read return value to count matching bytes
# Iterate characters at each position to find the one that increases count
```
**Key insight:** Replacing memcmp via LD_PRELOAD to return match count converts any comparison-based validation into a byte-by-byte oracle. Combined with GDB scripting, this automates bruteforce of password/flag checks without reversing the validation algorithm.
**Detection:** Binary uses `memcmp` or `strcmp` for flag validation (visible in `ltrace` output or import table). The comparison function is called with user input and a computed/stored expected value.
**References:** Blaze CTF 2018

546
reverse-engineering/tools.md Normal file
View File

@@ -0,0 +1,546 @@
# CTF Reverse - Tools Reference
## Table of Contents
- [GDB](#gdb)
- [Basic Commands](#basic-commands)
- [PIE Binary Debugging](#pie-binary-debugging)
- [One-liner Automation](#one-liner-automation)
- [Memory Examination](#memory-examination)
- [Radare2](#radare2)
- [Basic Session](#basic-session)
- [r2pipe Automation](#r2pipe-automation)
- [Ghidra](#ghidra)
- [Headless Analysis](#headless-analysis)
- [Emulator for Decryption](#emulator-for-decryption)
- [MCP Commands](#mcp-commands)
- [Unicorn Emulation](#unicorn-emulation)
- [Basic Setup](#basic-setup)
- [Mixed-Mode (64 to 32) Switch](#mixed-mode-64-to-32-switch)
- [Register Tracing Hook](#register-tracing-hook)
- [Track Register Changes](#track-register-changes)
- [Python Bytecode](#python-bytecode)
- [Disassembly](#disassembly)
- [Extract Constants](#extract-constants)
- [Pyarmor Static Unpack (1shot)](#pyarmor-static-unpack-1shot)
- [WASM Analysis](#wasm-analysis)
- [Decompile to C](#decompile-to-c)
- [Common Patterns](#common-patterns)
- [Android APK](#android-apk)
- [Extraction](#extraction)
- [Key Locations](#key-locations)
- [Search](#search)
- [Flutter APK (Blutter)](#flutter-apk-blutter)
- [HarmonyOS HAP/ABC (abc-decompiler)](#harmonyos-hapabc-abc-decompiler)
- [.NET Analysis](#net-analysis)
- [Tools](#tools)
- [Two-Stage XOR + AES-CBC Decode Pattern (Codegate 2013)](#two-stage-xor--aes-cbc-decode-pattern-codegate-2013)
- [NativeAOT](#nativeaot)
- [Packed Binaries](#packed-binaries)
- [UPX](#upx)
- [Custom Packers](#custom-packers)
- [PyInstaller](#pyinstaller)
- [LLVM IR](#llvm-ir)
- [Convert to Assembly](#convert-to-assembly)
- [RISC-V Binary Analysis (EHAX 2026)](#risc-v-binary-analysis-ehax-2026)
- [Binary Ninja](#binary-ninja)
- [Decompiler Comparison with dogbolt.org](#decompiler-comparison-with-dogboltorg)
- [Useful Commands](#useful-commands)
For dynamic instrumentation tools (Frida, angr, lldb, x64dbg), see [tools-dynamic.md](tools-dynamic.md).
---
## GDB
### Basic Commands
```bash
gdb ./binary
run # Run program
start # Run to main
b *0x401234 # Breakpoint at address
b *main+0x100 # Relative breakpoint
c # Continue
si # Step instruction
ni # Next instruction (skip calls)
x/s $rsi # Examine string
x/20x $rsp # Examine stack
info registers # Show registers
set $eax=0 # Modify register
```
### PIE Binary Debugging
```bash
gdb ./binary
start # Forces PIE base resolution
b *main+0xca # Relative to main
b *main+0x198
run
```
### One-liner Automation
```bash
gdb -ex 'start' -ex 'b *main+0x198' -ex 'run' ./binary
```
### Memory Examination
```bash
x/s $rsi # String at RSI
x/38c $rsi # 38 characters
x/20x $rsp # 20 hex words from stack
x/10i $rip # 10 instructions from RIP
```
---
## Radare2
### Basic Session
```bash
r2 -d ./binary # Open in debug mode
aaa # Analyze all
afl # List functions
pdf @ main # Disassemble main
db 0x401234 # Set breakpoint
dc # Continue
ood # Restart debugging
dr # Show registers
dr eax=0 # Modify register
```
### r2pipe Automation
```python
import r2pipe
r2 = r2pipe.open('./binary', flags=['-d'])
r2.cmd('aaa')
r2.cmd('db 0x401234')
for char in range(256):
r2.cmd('ood') # Restart
r2.cmd(f'dr eax={char}')
output = r2.cmd('dc')
if 'correct' in output:
print(f"Found: {chr(char)}")
```
---
## Ghidra
### Headless Analysis
```bash
analyzeHeadless /path/to/project tmp -import binary -postScript script.py
```
### Emulator for Decryption
```java
EmulatorHelper emu = new EmulatorHelper(currentProgram);
emu.writeRegister("RSP", 0x2fff0000);
emu.writeRegister("RBP", 0x2fff0000);
// Write encrypted data
emu.writeMemory(dataAddress, encryptedBytes);
// Set function arguments
emu.writeRegister("RDI", arg1);
// Run until return
emu.setBreakpoint(returnAddress);
emu.run(functionEntryAddress);
// Read result
byte[] decrypted = emu.readMemory(outputAddress, length);
```
### MCP Commands
- Recon: `list_functions`, `list_imports`, `list_strings`
- Analysis: `decompile_function`, `get_xrefs_to`
- Annotation: `rename_function`, `rename_variable`
---
## Unicorn Emulation
### Basic Setup
```python
from unicorn import *
from unicorn.x86_const import *
mu = Uc(UC_ARCH_X86, UC_MODE_64)
# Map code segment
mu.mem_map(0x400000, 0x10000)
mu.mem_write(0x400000, code_bytes)
# Map stack
mu.mem_map(0x7fff0000, 0x10000)
mu.reg_write(UC_X86_REG_RSP, 0x7fff0000 + 0xff00)
# Run
mu.emu_start(start_addr, end_addr)
```
### Mixed-Mode (64 to 32) Switch
```python
# When a 64-bit stub jumps into 32-bit code via retf/retfq:
# - retf pops 4-byte EIP + 2-byte CS (6 bytes)
# - retfq pops 8-byte RIP + 8-byte CS (16 bytes)
uc32 = Uc(UC_ARCH_X86, UC_MODE_32)
# Copy memory regions, then GPRs
reg_map = {
UC_X86_REG_EAX: UC_X86_REG_RAX,
UC_X86_REG_EBX: UC_X86_REG_RBX,
UC_X86_REG_ECX: UC_X86_REG_RCX,
UC_X86_REG_EDX: UC_X86_REG_RDX,
UC_X86_REG_ESI: UC_X86_REG_RSI,
UC_X86_REG_EDI: UC_X86_REG_RDI,
UC_X86_REG_EBP: UC_X86_REG_RBP,
}
for e, r in reg_map.items():
uc32.reg_write(e, mu.reg_read(r) & 0xffffffff) # mu = 64-bit emulator from above
uc32.reg_write(UC_X86_REG_EFLAGS, mu.reg_read(UC_X86_REG_RFLAGS) & 0xffffffff)
# SSE-heavy blobs need XMM registers copied
for xr in [UC_X86_REG_XMM0, UC_X86_REG_XMM1, UC_X86_REG_XMM2, UC_X86_REG_XMM3,
UC_X86_REG_XMM4, UC_X86_REG_XMM5, UC_X86_REG_XMM6, UC_X86_REG_XMM7]:
uc32.reg_write(xr, mu.reg_read(xr))
# Run 32-bit, then copy regs/memory back to 64-bit
```
**Tip:** set `UC_IGNORE_REG_BREAK=1` to silence warnings on unimplemented regs.
### Register Tracing Hook
```python
def hook_code(uc, address, size, user_data):
if address == TARGET_ADDR:
rsi = uc.reg_read(UC_X86_REG_RSI)
print(f"0x{address:x}: rsi=0x{rsi:016x}")
mu.hook_add(UC_HOOK_CODE, hook_code)
```
### Track Register Changes
```python
prev_rsi = [None]
def hook_rsi_changes(uc, address, size, user_data):
rsi = uc.reg_read(UC_X86_REG_RSI)
if rsi != prev_rsi[0]:
print(f"0x{address:x}: RSI changed to 0x{rsi:016x}")
prev_rsi[0] = rsi
mu.hook_add(UC_HOOK_CODE, hook_rsi_changes)
```
---
## Python Bytecode
### Disassembly
```python
import marshal, dis
with open('file.pyc', 'rb') as f:
f.read(16) # Skip header (varies by Python version)
code = marshal.load(f)
dis.dis(code)
```
### Extract Constants
```python
for ins in dis.get_instructions(code):
if ins.opname == 'LOAD_CONST':
print(ins.argval)
```
### Pyarmor Static Unpack (1shot)
Repository: `https://github.com/Lil-House/Pyarmor-Static-Unpack-1shot`
```bash
# Basic usage (recursive processing)
python /path/to/oneshot/shot.py /path/to/scripts
# Specify pyarmor runtime library explicitly
python /path/to/oneshot/shot.py /path/to/scripts -r /path/to/pyarmor_runtime.so
# Save outputs to another directory
python /path/to/oneshot/shot.py /path/to/scripts -o /path/to/output
```
Notes:
- `oneshot/pyarmor-1shot` must exist before running `shot.py`.
- Supported focus: Pyarmor 8.x-9.x (`PY` + six digits header style).
- Pyarmor 7 and earlier (`PYARMOR` header) are out of scope.
- Disassembly output is generally reliable; decompiled source is experimental.
---
## WASM Analysis
### Decompile to C
```bash
wasm2c checker.wasm -o checker.c
gcc -O3 checker.c wasm-rt-impl.c -o checker
```
### Common Patterns
- `w2c_memory` - Linear memory array
- `wasm_rt_trap(N)` - Runtime errors
- Function exports: `flagChecker`, `validate`
---
## Android APK
### Extraction
```bash
apktool d app.apk -o decoded/ # Best - decodes XML
jadx app.apk # Decompile to Java
unzip app.apk -d extracted/ # Simple extraction
```
### Key Locations
- `res/values/strings.xml` - String resources
- `AndroidManifest.xml` - App metadata
- `classes.dex` - Dalvik bytecode
- `assets/`, `res/raw/` - Resources
### Search
```bash
grep -r "flag\|CTF" decoded/
strings decoded/classes*.dex | grep -i flag
```
### Flutter APK (Blutter)
```bash
# Run Blutter on arm64 build
python3 blutter.py path/to/app/lib/arm64-v8a out_dir
```
### HarmonyOS HAP/ABC (abc-decompiler)
Repository: `https://github.com/ohos-decompiler/abc-decompiler`
```bash
# Extract .hap first to obtain .abc files
unzip app.hap -d hap_extracted/
```
Critical startup mode:
```bash
# Use CLI entrypoint (avoid java -jar GUI mode)
java -cp "./jadx-dev-all.jar" jadx.cli.JadxCLI [options] <input>
```
```bash
# Basic decompile
java -cp "./jadx-dev-all.jar" jadx.cli.JadxCLI -d "out" ".abc"
# Recommended for .abc
java -cp "./jadx-dev-all.jar" jadx.cli.JadxCLI -m simple --log-level ERROR -d "out_abc_simple" ".abc"
```
Notes:
- Start with `-m simple --log-level ERROR`.
- If `auto` fails, retry with `-m simple` first.
- Errors do not always mean total failure; check `out_xxx/sources/`.
- Use a fresh output directory per run.
---
## .NET Analysis
### Tools
- **dnSpy** - Debugging + decompilation (best)
- **ILSpy** - Decompiler
- **dotPeek** - JetBrains decompiler
### NativeAOT
- Look for `System.Private.CoreLib` strings
- Type metadata present but restructured
- Search for length-prefixed UTF-16 patterns
### Two-Stage XOR + AES-CBC Decode Pattern (Codegate 2013)
**Pattern:** .NET binary stores an encrypted byte array that undergoes XOR decoding followed by AES-256-CBC decryption. The same key value serves as both the AES key and IV.
**Steps:**
1. Extract hardcoded byte array and key string from binary (dnSpy/ILSpy)
2. XOR each byte (may be multi-pass, e.g., `0x25` then `0x58`, equivalent to single `0x7D`)
3. Base64-decode the XOR result
4. AES-256-CBC decrypt with `RijndaelManaged` using the extracted key as both Key and IV
```python
from Crypto.Cipher import AES
from base64 import b64decode
# Step 1: XOR decode
data = bytearray(encrypted_bytes)
for i in range(len(data)):
data[i] ^= 0x7D # Combined XOR key (0x25 ^ 0x58)
# Step 2: Base64 decode
ct = b64decode(bytes(data))
# Step 3: AES-256-CBC decrypt (same value for key and IV)
key = b"9e2ea73295c7201c5ccd044477228527" # Padded to 32 bytes
cipher = AES.new(key, AES.MODE_CBC, iv=key)
plaintext = cipher.decrypt(ct)
```
**Key insight:** When `RijndaelManaged` appears in .NET decompilation, check if Key and IV are set to the same value — this is a common CTF pattern. The XOR stage often serves as a simple obfuscation layer before the real crypto.
---
## Packed Binaries
### UPX
```bash
upx -d packed -o unpacked
strings binary | grep UPX # Check for UPX signature
```
### Custom Packers
1. Set breakpoint after unpacking stub
2. Dump memory
3. Fix PE/ELF headers
### PyInstaller
```bash
python pyinstxtractor.py binary.exe
# Look in: binary.exe_extracted/
```
---
## LLVM IR
### Convert to Assembly
```bash
llc task.ll --x86-asm-syntax=intel
gcc -c task.s -o file.o
```
---
## RISC-V Binary Analysis (EHAX 2026)
**Pattern (iguessbro):** Statically linked, stripped RISC-V ELF binary. Can't run natively on x86.
**Disassembly with Capstone:**
```python
from capstone import *
with open('binary', 'rb') as f:
code = f.read()
# RISC-V 64-bit with compressed instruction support
md = Cs(CS_ARCH_RISCV, CS_MODE_RISCVC | CS_MODE_RISCV64)
md.detail = True
# Disassemble from entry point (check ELF header for e_entry)
TEXT_OFFSET = 0x10000 # typical for static RISC-V
for insn in md.disasm(code[TEXT_OFFSET:], TEXT_OFFSET):
print(f"0x{insn.address:x}:\t{insn.mnemonic}\t{insn.op_str}")
```
**Common RISC-V patterns:**
- `li a0, N` → load immediate (argument setup)
- `mv a0, s0` → register move
- `call offset` → function call (auipc + jalr pair)
- `beq/bne a0, zero, label` → conditional branch
- `sd/ld` → 64-bit store/load
- `addiw` → 32-bit add (W-suffix = word operations)
**Key differences from x86:**
- No flags register — comparisons are inline with branch instructions
- Arguments in a0-a7 (not rdi/rsi/rdx)
- Return value in a0
- Saved registers s0-s11 (callee-saved)
- Compressed instructions (2 bytes) mixed with standard (4 bytes) — use `CS_MODE_RISCVC`
**Anti-RE tricks in RISC-V:**
- Fake flags as string constants (check for `"n0t_th3_r34l"` patterns)
- Timing anti-brute-force (rdtime instruction)
- XOR decryption with incremental key: `decrypted[i] = enc[i] ^ (key & 0xFF) ^ 0xA5; key += 7`
**Emulation:** `qemu-riscv64 -L /usr/riscv64-linux-gnu/ ./binary` (needs cross-toolchain sysroot)
---
## Binary Ninja
Interactive disassembler/decompiler with rapid community growth.
**Decompilation outputs:** High-Level Intermediate Language (HLIL), pseudo-C, pseudo-Rust, pseudo-Python.
```bash
# Open binary
binaryninja binary
```
```python
# Headless analysis (Python API)
import binaryninja
bv = binaryninja.open_view("binary")
for func in bv.functions:
print(func.name, hex(func.start))
print(func.hlil) # High-Level IL
```
**Community plugins:** Available via Plugin Manager (Ctrl+Shift+P → "Plugin Manager").
**Free version:** https://binary.ninja/free/ (cloud-based, limited features).
**Advantages over Ghidra:** Faster startup, cleaner IL representations, better Python API for scripting.
---
## Decompiler Comparison with dogbolt.org
**dogbolt.org** runs multiple decompilers simultaneously on the same binary and shows results side-by-side.
**Supported decompilers:** Hex-Rays (IDA), Ghidra, Binary Ninja, angr, RetDec, Snowman, dewolf, Reko, Relyze.
**When to use:**
- Decompiler output is confusing — compare with alternatives for clarity
- One decompiler mishandles a construct — another may get it right
- Quick triage without installing every tool locally
- Validate decompiler correctness by cross-referencing outputs
```bash
# Upload via web interface: https://dogbolt.org/
# Or use the API:
curl -F "file=@binary" https://dogbolt.org/api/binaries/
```
**Key insight:** Different decompilers excel at different constructs. When one produces unreadable output, another often generates clearer pseudocode. Cross-referencing catches decompiler bugs.
---
## Useful Commands
```bash
# File info
file binary
checksec --file=binary
rabin2 -I binary
# String extraction
strings binary | grep -iE "flag|secret"
rabin2 -z binary
# Sections
readelf -S binary
objdump -h binary
# Symbols
nm binary
readelf -s binary
# Disassembly
objdump -d binary
objdump -M intel -d binary
```