vlad/reverse-skills-max
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

reverse-skills-max: initial public release (restored, redacted, hardened)

Browse Source
This commit is contained in:
baradok609hecl1pq6qi
2026-10-20 02:17:15 +03:00
commit da24c2bd20
2227 changed files with 443931 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

638
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-benchmark-l1-l3.md Normal file
View File

@@ -0,0 +1,638 @@
# Extension verification — benchmark B1/B2 against OWASP MASTG Crackmes
Covers the pass that turned the `tests/benchmark.md` rows **B1** (equal-length dex patch + repack +
re-sign installability, gates G2/G4) and **B2** (anti-debug / self-destruct race) from `pending`
into measured results.
Strength labels follow `docs/tool-verification/README.md`: **observed** (exact command and its
output reproduced here), **inferred** (follows from observed facts, the step itself not executed),
**unverified** (assumed or reported and not independently confirmed).
Reference environment: Windows 11 host, Python 3.14.0, host frida 16.7.19 (frida-tools 13.7.1),
JDK 17.0.4.1 at `C:\Program Files\Java\jdk-17.0.4.1` (not on `PATH`), `apksigner.bat` /
`zipalign.exe` at `E:\tools\bin`. Device: **`<DEVICE>`**, Android 11 / API 30, arm64-v8a, root via
`su`, Magisk denylist `enforced` with an **empty** list (no app is root-hidden). Package under test:
**`<PKG>`**.
All device operations ran under the shared device lock (`tools/_work/devlock.py`).
---
## 0. Headline
- **B1 is `observed` and positive.** An equal-length dex edit (2 bytes of a conditional branch
replaced by `nop`s), through `dex_patch_bytes.py` → `repack.py --no-sign` → `zipalign` →
`apksigner` → root `pm install`, installs and **demonstrably changes the app's behaviour on
screen**: the original and a zero-change control both show a blocking `Root detected!` dialog, the
patched build reaches `MainActivity` with the dialog gone.
- **`repack.py`'s default signing path did not work when this pass started, and the blocker was not
`apksigner`.** `apksigner`/`zipalign` are both present and both work; at that moment `repack.py`
signed only through an `uber-apk-signer` jar, which is not installed. **The repository changed
mid-pass** — another writer landed a `sign_with_apksigner()` route — and re-running the same
command then produced a fully signed, `[result] OK` APK in one invocation with byte-identical
output to the manual route (`sha256 ad6a51ce…7e56`). Both states are recorded below, because the
stale state is what `tools/_work/ENV.md` predicted and the new state is what a reader will meet.
This is also a reminder that a "the tool cannot do X" finding on a shared worktree has an expiry
date.
- **B2 is a split result.** The two scripts run and the probe writes its bytes (0.04–0.05 s after
resume, gated on `PATCHED`), so the *tooling* is verified. But the **patch does not prevent the
target's termination**: `libfoo.so` aborts through `goodbye()+12` at **3.22 s** after resume, and
three patched sites including `goodbyev` itself do not change that. Neutralising the obvious death
site is **not sufficient** on this target.
- **A finding that changes the whole B2 setup:** on this device the **unmodified, original APK
already reports `Rooting or tampering detected.`** and its tamper path runs before anything is
attached. L3's native CRC self-check fails against the pristine community APK on Android 11 /
arm64-v8a, so the "pristine baseline" the row assumed does not exist here.
---
## 1. Gate G2 and G4 — environment and control (`observed`)
`scripts/doctor.py`:
```
platform : Windows 11 / AMD64
python : 3.14.0
[OK ] static (dex/zip/strings) [OK ] repack + sign
[OK ] native ELF patching [-- ] jadx decompile
[-- ] apktool unpack [OK ] device work
[OK ] root on device [OK ] frida dynamic
[OK ] java 17.0.4.1 [OK ] adb 1.0.41 [OK ] frida 16.7.19
[-- ] apktool not on PATH
[-- ] unzip not on PATH
java jars found near this skill : (none)
device : <DEVICE> arm64-v8a Android 11 (sdk 30) root: yes
clock skew: 3.06 s
```
The `(none)` on the jar line matters for B1: no `baksmali`/`smali`/`uber-apk-signer` jar exists, so
every signing route that goes through a jar is unavailable before a single command is run.
**Keystore and tool locations** (recorded because `repack.py` resolves both off-`PATH`):
```
keytool : C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe
jarsigner : C:\Program Files\Java\jdk-17.0.4.1\bin\jarsigner.exe
apksigner : E:\tools\bin\apksigner.bat
zipalign : E:\tools\bin\zipalign.exe
```
`repack.py`'s `sibling_tool()` cannot find `keytool` from `C:\Program Files\Common Files\Oracle\Java\javapath\java.exe`,
because that directory holds only forwarders. Passing `--keytool` explicitly is required.
---
## 2. B1 — L1 equal-length dex patch, full chain (`observed`)
### 2.1 Source facts
```
$ python -c "... load_dex('UnCrackable-Level1.apk') ..."
classes.dex 5528 bytes
magic dex\n035\0
checksum field 7fb7d9fa
adler32(d[12:]) 7fb7d9fa -> matches
signature field b7fafe72cb521450c4470043caa332da61d1bec7
sha1(d[32:]) 5253237b04c2d002967bae8ef81daae08550c90c
-> MISMATCH
```
Two facts that a reader needs before trusting the tool's output:
- The **shipped dex's `signature` field is stale** while its `checksum` is correct. `dex_patch_bytes.py`
prints `header before: checksum_ok=True signature_ok=False` and explains it. If the tool had
written the two fields in the wrong order, that starting state would have hidden the bug.
- The class that owns the behaviour is
`Lsg/vantagepoint/uncrackable1/MainActivity;.onCreate(Landroid/os/Bundle;)V`, and the gate is the
first arm of a three-way root `OR`:
```
0x9c8 invoke-static Lsg/vantagepoint/a/c;.a()Z
0x9ce move-result
0x9d0 39000e00 if-nez v0 -> 0x9ec <-- the edit site
0x9d4 invoke-static Lsg/vantagepoint/a/c;.b()Z
```
`0x9ec` is `const-string "Root detected!"` followed by the private `a(String)` that builds the
blocking dialog. **The branch structure, not the method name, was read before editing** — the edit
target is the arm whose *fall-through* continues into the next check.
### 2.2 The patch
```json
[{
"name": "l1-first-root-check-fallthrough",
"class": "Lsg/vantagepoint/uncrackable1/MainActivity;",
"method": "onCreate", "desc": "(Landroid/os/Bundle;)V",
"match": {"kind": "if-testz", "bytes": "39000e00"},
"expect_next": {"kind": "invoke", "target_class": "Lsg/vantagepoint/a/c;",
"target_method": "b"},
"replace": {"kind": "nops"}
}]
```
```
$ python skills/apk-reverse/scripts/dex_patch_bytes.py tools/_work/bench/l1l3/dex/classes.dex \
--spec tools/_work/bench/l1l3/l1_root.spec.json --dry-run
-- [l1-first-root-check-fallthrough] Lsg/vantagepoint/uncrackable1/MainActivity;.onCreate(...)V
site 0x9d0 0x9d0: 39000e00 if-nez v0 -> 0x9ec
(before) 0x9c8: 710027000000 invoke-static Lsg/vantagepoint/a/c;.a()Z
(before) 0x9ce: 0a00 move-result
-> 0x9d0: 00000000 if-nez
(after) 0x9d4: 710028000000 invoke-static Lsg/vantagepoint/a/c;.b()Z
(after) 0x9da: 0a00 move-result
polarity: expect_next satisfied
== dry run: nothing written
```
`expect_next` is what pins the polarity: the instruction after the branch had to be the *next* root
check, so the fall-through is the "continue" side. Applied:
```
== header checksum 0x7fb7d9fa -> 0x86e1d928
signature b7fafe72cb521450 -> eb632328fd1ba03c
self-verify: checksum_ok=True signature_ok=True
== wrote classes.patched.dex (5528 bytes, delta 0)
```
**The header order question, answered by byte comparison rather than by the tool's own report**
(`observed`): the diff of the whole file against the source is exactly the two fields plus the patch
site.
```
sizes 5528 5528
diff offsets raw: ['0x8','0xa','0xb','0xc'...'0x1f', '0x9d0','0x9d2']
site 0x9d0 orig 39000e00 patched 00000000
8..12 fad9b77f -> 28d9e186 (checksum, bytes 8..12)
12..32 b7fafe72cb521450c4470043caa332da61d1bec7
-> eb632328fd1ba03c7ebc6df181e0708a761aa2e3 (signature, bytes 12..32)
selfverify checksum True signature True
```
Recomputing `zlib.adler32(d[12:])` and `hashlib.sha1(d[32:])` **on the written file** reproduces both
header fields, which is only possible if the signature was computed before the checksum. The patch
owns 4 bytes and the header owns 24; nothing else moved.
### 2.3 Repack — where the documented pipeline actually fails
`repack.py`'s default signing route:
```
$ python skills/apk-reverse/scripts/repack.py --apk UnCrackable-Level1.apk \
--dex classes.dex=classes.patched.dex --out out/l1_patched.apk \
--workdir out --ks out/bench.keystore \
--keytool ".../jdk-17.0.4.1/bin/keytool.exe" \
--jarsigner ".../jdk-17.0.4.1/bin/jarsigner.exe" \
--apksigner E:\tools\bin\apksigner.bat --zipalign E:\tools\bin\zipalign.exe
[zip] - META-INF/CERT.RSA (signature artifact)
[zip] - META-INF/CERT.SF (signature artifact)
[zip] - META-INF/MANIFEST.MF (signature artifact)
[zip] + classes.dex (5528 bytes <- .../classes.patched.dex)
[zip] unsigned written: 60967 bytes
[ks] generated E:\...\out\bench.keystore (alias=apkreverse)
error: signer jar not found: uber-apk-signer.jar
pass --signer-jar <path>, or set APK_SIGNER_JAR.
Any zipalign+apksigner based signer works here.
[exit code: 1]
```
Two separate facts, and the second is the one worth recording:
1. `--signer-jar` defaults to `uber-apk-signer.jar` and no such jar exists here, so the default run
cannot sign. (`ENV.md` predicted this.)
2. **The error message's second line is misleading in this environment.** "Any zipalign+apksigner
based signer works here" is true, but `repack.py` has no code path that *uses* a plain
`zipalign`+`apksigner` pair for signing — `sign_apk()` only ever invokes
`java -jar <signer_jar>`. `--apksigner`/`--zipalign` are used for **verification only**. So the
documented Route B works, but only if the caller performs the sign step outside `repack.py`.
The `--no-sign` flag is the supported seam.
Working route, both builds through the **same** pipeline:
```
$ python skills/apk-reverse/scripts/repack.py --apk UnCrackable-Level1.apk \
--dexdir tools/_work/bench/l1l3/dex_orig \
--out out/l1_control_unsigned.apk --workdir out/ctrl --no-sign
[zip] unsigned written: 60967 bytes
[zip] resources.arsc STORED offset=0xd4d4 aligned
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
[result] UNSIGNED (--no-sign): good for inspection, not installable as-is
$ (same, with --dex classes.dex=classes.patched.dex) -> out/l1_patched_unsigned.apk
[zip] resources.arsc STORED offset=0xd4d4 aligned
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
```
`AndroidManifest.xml` is reported `STORED ... NOT ALIGNED`, which is correct and not a defect: the
4-byte alignment requirement is `resources.arsc` (and uncompressed `lib/*.so`), not the manifest.
Signing, explicitly, both builds, same key:
```
$ E:\tools\bin\zipalign.exe -p -f 4 l1_control_unsigned.apk l1_control_aligned.apk ; rc=0
$ E:\tools\bin\apksigner.bat sign --ks bench.keystore --ks-pass pass:<pw> --key-pass pass:<pw> \
--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true \
--out l1_control.apk l1_control_aligned.apk ; rc=0
$ E:\tools\bin\apksigner.bat verify --print-certs --verbose \
--min-sdk-version 21 --max-sdk-version 34 l1_control.apk
Verifies
Verified using v1 scheme (JAR signing): true
Verified using v2 scheme (APK Signature Scheme v2): true
Verified using v3 scheme (APK Signature Scheme v3): true
Signer #1 certificate SHA-256 digest: ad13912308e0b6ded5d8be7da2868356348655033d1dde402d8f70d452fbf6db
```
Identical result for `l1_patched.apk`, with the same certificate digest, so the two builds differ
only in the dex bytes.
### 2.4 On-device result — the behaviour change, seen on the screen (`observed`)
Two builds were produced and both were checked, because "the pipeline works" and "the patch changed
the app" are different claims and only the second one is the deliverable.
**The one-command pipeline, re-run after the repository changed** (this supersedes §2.3's manual
route as the recommended invocation):
```
$ python skills/apk-reverse/scripts/repack.py --apk UnCrackable-Level1.apk \
--dex classes.dex=classes.patched.dex --out out/l1_patched_final.apk \
--workdir out/fin --ks out/bench.keystore \
--keytool "C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe" \
--apksigner E:\tools\bin\apksigner.bat --zipalign E:\tools\bin\zipalign.exe
[sign] route: apksigner
[out] ...\out\l1_patched_final.apk (70647 bytes)
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
Verified using v1 scheme (JAR signing): true
Verified using v2 scheme (APK Signature Scheme v2): true
Verified using v3 scheme (APK Signature Scheme v3): true
[result] OK
local sha256 ad6a51cef4699b73ff052d4e049ca9efd64e9570900d19509f0b7a7c20f87e56
```
The manual `zipalign`+`apksigner` route earlier in this section produced **the same sha256**, so the
two paths agree byte for byte and the result does not depend on which one a reader follows.
Install through the **root** path (`pm install -r -d`, which bypasses this ROM's streamed-install
interception), and the installed `base.apk` is hashed and compared to the local file so "installed"
cannot be confused with "the previous build is still there":
```
control device sha256 e5a9f335155e3e20219d89bc859ae89cc0a82f7135123397c526d94bfca4b3b4
local sha256 e5a9f335155e3e20219d89bc859ae89cc0a82f7135123397c526d94bfca4b3b4 identical
patched device sha256 ad6a51cef4699b73ff052d4e049ca9efd64e9570900d19509f0b7a7c20f87e56
local sha256 ad6a51cef4699b73ff052d4e049ca9efd64e9570900d19509f0b7a7c20f87e56 identical
```
`am start -W` for both builds:
```
LaunchState: COLD TotalTime: 241 ms WaitTime: 249 ms Status: ok (control)
LaunchState: COLD TotalTime: 232 ms WaitTime: 234 ms Status: ok (patched)
```
Window focus, sampled four times at 2 s intervals:
```
original mCurrentFocus=Window{a5d2ba7 u0 Root detected!} (all four samples)
control mCurrentFocus=Window{c4047e2 u0 Root detected!} (all four samples)
patched mCurrentFocus=Window{8c94fc1 u0 <PKG>/...MainActivity} (all four samples)
```
And the images, which are the evidence that actually counts —
`tools/_work/bench/l1l3/evidence/00_original/00_original_03.png` shows the modal
`Root detected! / This is unacceptable. The app is now going to exit. / OK` over a greyed-out input
screen. The same shot on the control build (`evidence/01_control/*.png`, and again on
`evidence/12_control_final/*.png` from the final one-command build) is byte-comparable to the
original's. `evidence/13_patched_final/13_patched_final.png` — the final patched build — shows the
screen **without any dialog**: input field live with the caret in it, `VERIFY` enabled, and
`mCurrentFocus` on `<PKG>/...MainActivity`.
**G4 is therefore cleared in the strong form**: the control is not just "the same APK" — it went
through the identical repack-and-sign pipeline with a byte-identical dex and still shows the old
behaviour. The behaviour change is attributable to the four patched bytes, and it reproduced on a
second, independently built APK from the final pipeline.
### 2.5 What the app's own check actually is (`observed`, and it contradicts the obvious reading)
The functional check was probed with a Frida hook rather than inferred from the crypto:
```
[a.b] input="8d127684cbc37c17616d806cf50473cc" -> 16 bytes: 8d127684cbc37c17616d806cf50473cc
[lib a.a] key=8d127684cbc37c17616d806cf50473cc
data=e5426215cb5b9a06c3a0b5e6a4bd769a49e8f074f82eff1d95ab7c17147618e7
-> 17 bytes 492077616e7420746f2062656c69657665 ascii="I want to believe"
[a.a] REAL input="" -> false
```
So: a **fixed 16-byte AES key** (`8d12…73cc`, a `const-string` in the dex) decrypts a fixed
base64 blob to `I want to believe`, and the user's input is compared against that plaintext. My first
reading — that the hex string was itself the secret — is **wrong**, and it was falsified by running
it: entering `8d127684cbc37c17616d806cf50473cc` produces `Nope...`.
`TextUtils.equals` is **not** the comparison site: hooking it captured only dialog-internal
comparisons (`("Nope...", "<null>")`). The comparison lives in the decrypted-bytes path of
`a.a(String)`.
Two device facts that cost time and are worth inheriting:
- **The input field cannot be driven on this ROM.** `adb shell input text "I%swant%sto%sbelieve"`
and a per-character `input keyevent` loop both landed `iwantto`: 7 of 17 characters survived and
every space was dropped. Three independent quoting routes gave the same result, so the field
itself rejects what the injection path sends. Reading the field back with `uiautomator dump` after
each attempt is what distinguished "the app said Nope" from "my input never arrived" — the first
two attempts looked like a wrong secret and were not.
- `EditText.setText(String)` is not reachable from outside (only
`setText(CharSequence, TextView$BufferType)` is declared, and `TextView.BufferType` is not exposed
as a property on the class object); an **empty** `EditText` also returns a **null** `Editable`, so
`getText().length()` fails with "not a function" and looks like a bridge limitation rather than an
empty field.
The success branch was therefore reached through the app's own method with the boolean forced, which
is honest as long as it is labelled — and it is:
`evidence/09_success_branch/09_success_branch.png` shows the app's `Success! / This is the correct
secret.` dialog. That image proves **the patched build reaches and renders the app's own success
branch**; it does not prove the secret was typed and accepted. The check itself is evidenced by the
libfoo/`a.b`/`a.a` log above.
### 2.6 B1 verdict
`observed`: the equal-length edit, the header recomputation in the correct order, the STORED+aligned
repack, v1+v2+v3 signing, the root-path install, the device-side hash match, and the on-screen
behaviour change all reproduce. G2 and G4 both clear.
---
## 3. B2 — L3 self-destruct race (`observed` for the tooling, `unverified`/negative for the patch)
### 3.1 What L3 actually does (read from the dex and the tombstone, not assumed)
`MainActivity.onCreate` calls `verifyLibs()` **first**, then `init("pizzapizzapizzapizzapizz".getBytes())`,
then an `AsyncTask`, then three root checks, then `isDebuggable()`, then `setContentView` — and
`setContentView` sits **after** the gate:
```
0x11a3ce invoke-static Lsg/vantagepoint/util/RootDetection;.checkRoot1()Z
...
0x11a406 sget Lsg/vantagepoint/uncrackable3/MainActivity;.tampered:I
0x11a40a if-eqz v0 -> 0x11a418 ; tampered == 0 -> skip the dialog
0x11a40e const-string "Rooting or tampering detected."
0x11a412 invoke-direct MainActivity;.showDialog(Ljava/lang/String;)V
0x11a418 ... setContentView
```
So when the tamper bit is set the app **never renders its UI**; it sits on a blocking dialog whose OK
button is wired to an `onClick` that exits. `verifyLibs()` is the source of that bit:
- it reads four ABI-keyed expected CRCs out of `resources.arsc` string resources
(`R.string` ids `0x7f0b002b`, `0x7f0b0029`, `0x7f0b0034`, `0x7f0b0035`),
- compares them to the zip entry CRCs of `lib/<abi>/libfoo.so`, and
- compares `classes.dex`'s zip CRC to the value the native `baz()` returns.
The native library exports exactly three JNI entries and one terminate routine:
```
Java_sg_vantagepoint_uncrackable3_CodeCheck_bar vaddr=0x33a4 size=256
Java_sg_vantagepoint_uncrackable3_MainActivity_init vaddr=0x3310 size=136
Java_sg_vantagepoint_uncrackable3_MainActivity_baz vaddr=0x3398 size=12
_Z7goodbyev vaddr=0x3080 size=24
_Z4randv vaddr=0xe4c size=36
```
and its strings name the detection inputs: `/proc/self/maps`, `frida`, `xposed`, `ptrace`,
`waitpid`, `getppid`, `pthread_create`, plus two log strings:
`"Tampering detected! Terminating..."` and `"Error opening /proc/self/maps! Terminating..."`.
`libfoo.so`'s single RX `PT_LOAD` is `off=0x0 vaddr=0x0 filesz=0x4700`, so **file offset == virtual
address == runtime offset from the module base** for everything in `.text`. That is what makes
`hook_patch_only.js`'s `FILE_OFFSET` meaningful here, and it is not true in general — a library whose
text segment has a non-zero `vaddr` would need `p_vaddr + delta`.
### 3.2 The baseline, and the finding that invalidates the assumed baseline
`observed`: the **unmodified, pristine APK** on this device shows the tamper dialog:
```
$ am start -n <PKG>/sg.vantagepoint.uncrackable3.MainActivity
833 ms pid=5377 5413 mCurrentFocus=Window{1d9ee56 u0 Rooting or tampering detected.}
1638 ms ... mCurrentFocus=Window{1d9ee56 u0 Rooting or tampering detected.}
...
10712 ms ... mCurrentFocus=Window{1d9ee56 u0 Rooting or tampering detected.}
```
and `logcat` shows the CRCs it compared (`V/UnCrackable3`), with `tampered` set. The process stays
alive in this state for at least 25 s: the tamper dialog alone does **not** kill the process, it waits
for the OK button.
`observed` — the **time-to-death baseline with a frida probe attached**, which is the number the row
needs:
```
$ python tools/_work/bench/l1l3/ttd_attach.py --package <PKG> --activity sg.vantagepoint.uncrackable3.MainActivity \
--js l3_read_crc.js --launch-delay 1.5 --seconds 20
[i] pre-attach main pid=7020 at 1.88s
[i] attached at +0.00s (probe loaded)
attach+ 0.75s main_pid=7020
... (alive)
attach+ 4.90s main_pid=7020
attach+ 5.34s main_pid=-
=== logcat ===
F/libc ( 7020): Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 7052 (tg.uncrackable3)
F/DEBUG ( 7203): #01 pc 000000000000308c .../lib/arm64/libfoo.so (goodbye()+12)
F/DEBUG ( 7203): #02 pc 00000000000031ac .../lib/arm64/libfoo.so
I/Zygote( 1007): Process 7020 exited due to signal 6 (Aborted)
```
**`TTD(frida attached) = 5.34 s`, and the death is a native `SIGABRT` originating in `goodbye()+12`.**
Note which process dies: `7020` is the main `<PKG>` process, `7052`/`tg.uncrackable3` is the failing
*thread*. That distinction matters — see §3.4.
### 3.3 The two scripts, run as shipped (`observed`)
Neither had ever been executed before this pass.
```
$ python skills/apk-reverse/scripts/spawn_patch_detach.py \
--package <PKG> --js skills/apk-reverse/scripts/hook_patch_only.js \
--captures 2 --interval 3 --no-screencap
[i] spawned pid=7442
[!] PATCH_BYTES is empty -- configure this probe before using it
[MSG] PATCHFAIL
[i] patched=False
[i] detached (memory writes persist, hooks are gone)
=== capture 0 ===
7567 00:01 sh -c ps -A -o PID,ETIME,ARGS | grep -i uncrackable3
7570 00:01 grep -i uncrackable3
mCurrentFocus=Window{7c35f28 u0 com.oppo.launcher/com.oppo.launcher.Launcher}
```
Three defects, all confirmed by later runs:
1. **`hook_patch_only.js` is a template, and its guard is correct but the guard is the whole
result.** `PATCH_BYTES = []` correctly refuses and reports `PATCHFAIL`, so an unconfigured run
cannot silently claim success. That is good behaviour; it just means the shipped file tests
nothing.
2. **`spawn_patch_detach.py` resumed the target before waiting for `PATCHED`.** The original order
was `resume(pid)` → then wait. That makes the wait meaningless: by the time the probe's
`Process.findModuleByName` poll sees the library, the target has been running.
3. **`--activity` defaults to `None` and the code then did nothing.** After `am force-stop` and a
spawn, omitting `--activity` left the app un-launched, so the capture set was of the launcher and
read exactly like "the app died". Fixed by resolving the launcher through
`cmd package resolve-activity --brief <pkg>` when `--activity` is absent.
**Correction to my own first fix.** I changed the order so the driver held the target **paused**
until `PATCHED`. That cannot work with a module-polling probe, and the run says so:
```
[i] probe loaded; target still PAUSED, waiting up to 15.0s for the patch
[5.00s][alive] [10.00s][alive] [15.00s][alive]
[!] the probe did not report PATCHED in 15.0s
[i] resumed pid=7666
[15.11s][i] libfoo.so base=0x764c0aa000 bytes@0x3080 = fd 7b bf a9 ...
[15.12s][PATCH] now = 00 00 80 52 d6 03 5f c0 ...
```
**While the process is frozen its libraries are not mapped**, so the poll can never succeed: 15 s
paused produced nothing, and `libfoo.so base=0x764c0aa000` appeared 0.11 s after resume. The shipped
order was right about *that*; what was wrong was detaching before the write landed. The corrected
driver resumes, then waits for `PATCHED` (reported loudly when it never arrives), then detaches.
### 3.4 The experiment, and its result
Sites were chosen from the tombstone and the ELF, not guessed. `goodbyev` at `0x3080`:
```
0x3080 a9bf7bfd stp x29, x30, [sp, #-16]!
0x3084 910003fd mov x29, sp
0x3088 321f07e0
0x308c 97fff725 bl 0x2ff0 <-- the call the tombstone names as goodbye()+12
0x3090 2a1f03e0
0x3094 97fff71f bl 0x2ff8
```
and the two helpers it tail-calls are trivial return-value stubs:
```
0x2ff0 52801a40 mov w0, #0xd2 0x2ffc d65f03c0 ret
0x2ff8 52800cc0 mov w0, #0x66 0x3000 321f0fe0 ...
```
So probe v2 patches the **return values the branch reads** (`0x2ff0`, `0x2ff8`) as well as
`goodbyev`'s entry, each rewritten to `mov w0, #0 ; ret` (+ nops, equal length), with an assertion on
the expected original bytes before each write.
Patched condition:
```
$ python tools/_work/bench/l1l3/b2_measure.py --js l3_hook_patch_v2.js --tag PATCHED --probe-seconds 25
[i] spawned pid=10000 (paused)
[i] resumed pid=10000
[0.04s][i] libfoo.so base=0x764b89d000 size=90112
[0.04s][i] goodbye entry = fd 7b bf a9 fd 03 00 91 e0 07 1f 32 25 f7 ff 97
[0.04s][i] goodbye+12 = 25 f7 ff 97
[0.04s][PATCH] 0x2ff0 (helper A -> return 0) 40 1a 80 52 c0 03 5f d6 -> 00 00 80 52 d6 03 5f c0
[0.05s][PATCH] 0x2ff8 (helper B -> return 0) c0 0c 80 52 c0 03 5f d6 -> 00 00 80 52 d6 03 5f c0
[0.05s][PATCH] 0x3080 (goodbyev entry -> no-op) fd 7b bf a9 ... -> 00 00 80 52 d6 03 5f c0 1f 20 03 d5 1f 20 03 d5
[0.05s][i] sites written=3 skipped=0
[i] patched=True
3.22s pid=10000 alive=False
V/UnCrackable3(10000): Tampering detected! Terminating...
F/libc (10000): Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 10032 (tg.uncrackable3), pid 10000
F/DEBUG (10036): #01 pc 000000000000308c .../lib/arm64/libfoo.so (goodbye()+12)
I/Zygote( 1007): Process 10000 exited due to signal 6 (Aborted)
[RESULT] tag=PATCHED spawned=10000 ttd_after_resume=3.22 survived=False
```
Control condition (identical reads and writes, original bytes written back):
```
$ python tools/_work/bench/l1l3/b2_measure.py --js l3_hook_control_v2.js --tag control_v2 --probe-seconds 22
[0.04s][CONTROL] 0x2ff0 unchanged (wrote back 40 1a 80 52 c0 03 5f d6)
[0.04s][CONTROL] 0x2ff8 unchanged (wrote back c0 0c 80 52 c0 03 5f d6)
[0.04s][CONTROL] 0x3080 unchanged (wrote back fd 7b bf a9 fd 03 00 91)
[0.04s][i] control sites ok=3
V/UnCrackable3(9312): Tampering detected! Terminating...
F/DEBUG ( 9348): #01 pc 000000000000308c .../lib/arm64/libfoo.so (goodbye()+12)
I/Zygote( 1007): Process 9312 exited due to signal 6 (Aborted)
```
**Both conditions terminate at `goodbye()+12` with `SIGABRT`.** The probes apply the patch
successfully, inside the race window (0.04–0.05 s after resume against a ~3.2 s deadline), and the
target still dies.
### 3.5 What this means, and what to do instead
`observed`:
- `spawn_patch_detach.py` **can** deliver an early memory patch: the write landed 0.05 s after resume
and survived the detach. The mechanism the row tests is real.
- `hook_patch_only.js` is a **template with correct guard rails**, not a working probe.
- The patch as configured is **insufficient**: three sites including the terminate routine's entry
and both of its branch inputs do not stop the abort.
`inferred` (labelled, not executed): the terminate path is not confined to `goodbyev` and its two
stubs. `goodbyev` is 24 bytes and its only two calls are the two stubs; the `bl` at `+12` is the
observed faulting frame, and yet patching both the callee and the caller changed nothing. The most
consistent explanation is that the tampering thread reaches the *same abort* through a second site
that this static pass did not map — the region around `0x2f80–0x3080` is a dense run of
`mov w0, #imm ; ret` pairs (values 0x11, 0x1b, 0x43, 0x5b, 0x67, 0x7b, 0x92, 0xa2, 0xa3, 0xae, 0xcc,
0xd2, 0xe3, 0xe7, 0xe8, 0xed, 0xf6, 0xfd, …), i.e. a jump-table-shaped cluster of tiny accessors
that a proper disassembly would resolve into a call graph, and `rand` (`_Z4randv`, 0xe4c, 36 B) is
part of the same obfuscated scheme. **A real fix needs a full AArch64 disassembly of `.text`
(~18 KB) rather than four hand-picked offsets.**
`observed` — the route that should be tried first next time: the abort is reachable from a **Java**
layer that is already fully mapped. `verifyLibs()` computes `tampered` and `onCreate` branches on it,
and the dialog's OK button is a plain `DialogInterface.OnClickListener` in `MainActivity$1`. Patching
the dex in that class (`if-eqz v0 -> 0x11a418` → nops, forcing the non-tamper side) removes the
dialog and lets `setContentView` run **without touching native code at all** — and unlike the native
patch it is a route this repository already has measured tooling for (see B1). This was not executed
in this pass; it is the recommended next experiment, not a result.
`observed` — a limitation the next pass must budget for: on this device the L3 **original** APK
already fails its own CRC self-check, so the tamper state is on before any instrumentation. Whatever
the follow-up tests must therefore be measured against *that* baseline, not against a clean one.
---
## 4. Task C — L2 (not run)
Not attempted: A and B consumed the pass. `UnCrackable-Level2.apk` is on disk
(`tools/_work/bench/apks/`, 901022 B, sha256 prefix `4c7980ef1f6cc295`) and the `dynamic-frida.md`
spawn route is untested for it. Recorded as **not run**, not as a negative result.
---
## 5. Defects found in the repository's own tooling
| Where | What | Evidence | Status |
|---|---|---|---|
| `scripts/repack.py` (`sign_apk`) | **Stale finding, re-opened and closed.** At the start of this pass it signed **only** via `java -jar <signer_jar>`; `--apksigner`/`--zipalign` were verification-only, so the "any zipalign+apksigner based signer works here" hint had no code path behind it | §2.3 exit 1 with `signer jar not found` | **fixed by another writer mid-pass**; re-measured in §2.4 — `[sign] route: apksigner`, `[result] OK`, byte-identical output |
| `scripts/repack.py` | The `signer jar not found` message is still emitted by the jar-only `sign_apk()` and still claims a zipalign+apksigner route works. When no `--apksigner` was supplied (the first run in §2.3) the message is the *only* guidance a reader gets, and it points at a function that does not exist | §2.3 | documented; the message should name the apksigner route it now has |
| `scripts/spawn_patch_detach.py` | Detached before the probe's write landed, so the write could be missed entirely while the run still reported `patched=False` and looked merely unlucky | §3.3 | **fixed** — resume, wait for `PATCHED`, report loudly, then detach |
| `scripts/spawn_patch_detach.py` | `--activity` absent left the target un-launched after a spawn, so the captures were of the launcher | §3.3 | **fixed** — resolve the launcher via `cmd package resolve-activity` |
| `scripts/spawn_patch_detach.py` | Unbounded `adb`/frida calls; `adb_run()`'s rc is never inspected for the launcher step | §3.3 | partially fixed (`--launch-timeout`, rc surfaced) |
| `scripts/hook_patch_only.js` | Guard rails are correct (`PATCH_BYTES` empty → `PATCHFAIL`); it is a template, and the docs imply it is a runnable probe | §3.3 | documented |
| `tools/_work/ENV.md` | Attributes the B1 signing problem to the apksigner/zipalign route; the blocker was the missing uber-apk-signer jar, and both apksigner and zipalign work | §2.3 | corrected here |
| `tests/benchmark.md` B2 | Describes L3 as a self-destruct race against a pristine baseline; on this device the original APK is already in its tamper state | §3.2 | corrected here |
---
## 6. Files produced by this pass
Workbench (not part of the skill; `tools/` is git-ignored):
- `tools/_work/bench/l1l3/l1_root.spec.json`, `dex/classes.patched.dex`, `dex_orig/classes.dex`
- `tools/_work/bench/l1l3/out/l1_control.apk`, `l1_patched.apk`, `l1_control_now.apk`,
`l1_patched_final.apk` (+ unsigned/aligned intermediates)
- `tools/_work/bench/l1l3/l1_probe.py`, `l1_type.py`, `l1_verify_field.py`, `l1_try_secret.py`,
`l1_drive.js`, `l1_probe_full.js`, `attach_probe.py`
- `tools/_work/bench/l1l3/l3_hook_patch_v2.js`, `l3_hook_control_v2.js`, `l3_read_crc.js`,
`elf_l3.py`, `dis_l3.py`
- `tools/_work/bench/l1l3/b2_measure.py`, `ttd.py`, `ttd_attach.py`
- `tools/_work/bench/l1l3/evidence/**` (PNG captures: `00_original`, `01_control`,
`02_patched_nosecret`, `09_success_branch`, `12_control_final`, `13_patched_final`)
A note on making §2.4 reproducible: `l1_probe.py` reuses the tag as the remote filename, so
`--tag` must be unique per run. Reusing a tag after a failed `adb push` installs the *previous*
remote file, which is exactly how the first attempt at the final patched verification showed
`Root detected!` — the control build's hash, from the control run's leftover remote path. The hash
comparison in the same script is what caught it; a screenshot alone would not have.

183
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-dart-aot-formats.md Normal file
View File

@@ -0,0 +1,183 @@
# EXTENSION — Dart AOT string-table formats, measured on a real dual-ABI app
Evidence file for the `dart-aot.md` changes in this pass. Every claim below carries a grade:
- **observed** — a command was run here and its output is quoted in this file.
- **inferred** — follows from an observed fact or from documented mechanism; the step was not executed.
- **unverified** — assumed or reported elsewhere, not reproduced here.
Target identity is replaced with `<PKG>`; the artifact is referred to as "the sample". The technique
(tool, library, function, hardening product) is kept, per
`skills/apk-reverse/references/desensitization-and-leak-scans.md`.
## 1. What was measured
| Claim under test | Grade | Result |
|---|---|---|
| The arm64 Dart AOT string table is `[0x80\|(len<<1)\|two_byte][payload]` and the entries chain | **observed** | confirmed on 4,980 entries |
| The armv7 (32-bit) snapshot uses the *same* format | **refuted** | the same extractor keeps 0 entries; the tag byte is not present before the literal |
| An armv7 string record is `[length u32le][UTF-16LE chars]`, so the length field reads `len*2` | **refuted** | the u32 before the payload reads the **byte** count, and the payload is UTF-8 |
| The engine version banner always yields `x.y.z (stable) (…)` | **refuted** | a real `libflutter.so` carries `0.0.1` with a blank build id and no `(stable)` |
| An equal-length replacement of a data key is legal on both ABIs without recomputing the tag | **observed** (record layout), **unverified** (end-to-end device run) | the record is self-contained on both ABIs |
Two of the five were assumptions that would have been written into the reference as fact. Both were
killed by a single command each, which is the whole argument for measuring before documenting.
## 2. Sample and preparation
The sample is the locally supplied "complex encryption and hardening" APK (360 shell + Flutter).
The two AOT snapshots were extracted from the zip without repacking:
```
$ python -c "import zipfile; z=zipfile.ZipFile(r'存在问题和例子\<sample>.apk'); \
[print(n, len(z.read(n))) for n in ('lib/arm64-v8a/libapp.so','lib/armeabi-v7a/libapp.so')]"
lib/arm64-v8a/libapp.so 16352152
lib/armeabi-v7a/libapp.so 17629776
```
Work copies live under the ignored workspace `tools/_work/dart/`; nothing from this evidence pass is
committed except the reference text.
## 3. The arm64 packed table — confirmed
```
$ python skills/apk-reverse/scripts/dart_pool_strings.py tools/_work/dart/app_arm64.so s64.tsv --min 6
candidates : 17352
kept (run >= 3) : 4980 (one-byte 4980 / utf16le 0)
below threshold : 12372 [--keep-isolated to keep them]
written s64.tsv
```
The tag arithmetic was checked directly against real literals in the image rather than only through
the script. For each literal, the byte immediately before it must equal `0x80|(len<<1)`:
```
$ python -c "... for m in (b'AdmobBanner', b'showSplashAd', b'AdPgl_Splash'): ..."
### app_arm64.so
AdmobBanner @0x4932e len=11 expected_tag=0x96
tag_byte_at_-1=0x96 matches=True
showSplashAd @0x39deb len=12 expected_tag=0x98
tag_byte_at_-1=0x98 matches=True
AdPgl_Splash @0x493ee len=12 expected_tag=0x98
tag_byte_at_-1=0x98 matches=True
```
**observed**: three independent literals of two different lengths, three matching tag bytes. This is
also the assertion a patch script should perform (`dart-aot.md` §10), and it is what rejects a hit
that landed mid-string.
## 4. The armv7 assumption — refuted, and what is actually there
The starting hypothesis (carried over from an external case) was that the 32-bit snapshot stores a
string as `[tag 0x81][len*2 u32le][UTF-16LE chars]`. Two scanners written against that hypothesis
found nothing, which is itself the evidence:
```
$ python tools/_work/dart/probe3.py app_armv7.so raw # [8-byte hdr ending 0x81][u32le len][chars]
hits=0 chained_runs=0 entries_in_runs=0
$ python tools/_work/dart/probe3.py app_armv7.so packed # the arm64 tag scheme
hits=0 chained_runs=0 entries_in_runs=0
```
The same extractor that keeps 4,980 entries on arm64 keeps **zero** on armv7:
```
$ python skills/apk-reverse/scripts/dart_pool_strings.py tools/_work/dart/app_armv7.so s7.tsv --min 6
candidates : 1355
kept (run >= 3) : 0 (one-byte 0 / utf16le 0)
```
A zero here is a **format mismatch, not an empty string table**, and that distinction is now stated in
`dart-aot.md` §7: reading it as "the strings were stripped" is exactly the wrong conclusion the same
section warns about for UTF-16.
What the record actually looks like, from bytes around a known literal:
```
$ python -c "... b.find(b'SystemChrome') ..."
0x1ffb3c b'\x00\x00\x00\x008\x06U\x00F\x00\x00\x00\xce-\x8a@SystemChrome.setEnab...'
0x204ce4 b'\x00\x00\x00\x008\x06U\x00H\x00\x00\x00\xa45jVSystemChrome.setSyst...'
0x2185c4 b' ere8\x07U\x00J\x00\x00\x00\x8c]\xb4GSystemChrome.setPref...'
```
Read as `[header u32][length u32le][payload]`: `0x00550638` then `0x00000046` = **70**, which is the
byte length of `SystemChrome.setEnabledSystemUIMode` (36 chars) immediately followed by
`SystemChrome.setSystemUIOverlayStyle` (34 chars) — **70 bytes, UTF-8, not UTF-16**. The second
literal starts at the exact byte where the first ends, and its own length field reads `0x00000048` =
72, matching its own continuation.
**observed**: `length == payload byte count`, payload is UTF-8, and in-place equal-length replacement
therefore keeps the record valid without recomputing anything. **refuted**: the `len*2` / UTF-16 form.
The header word is a tags value with full 4-byte alignment, not a constant to hard-code:
```
$ python -c "... alignment histogram over the image ..."
alignment mod 4: {0: 13397}
```
```
$ python tools/_work/dart/perf.py app_armv7.so
scanned 17629776 B in 0.94s using 4-byte stride + int.from_bytes
aligned-header string records: 582
stride-1 micro scan over 17629776 B took 2.80s (found 2762)
```
The literal search plus a backwards length assertion is the tool-independent method, and it costs
under a second over a 17.6 MB image — so validating every candidate is affordable and the armv7
limitation does not justify a hand-built table parser. **unverified**: whether a *different* armv7
snapshot uses a different header word; the reference tells the reader to read it from the file.
## 5. The engine version banner — a version-shaped lie
```
$ python -c "... rfind NUL before the ' on \"android_arm64\"' suffix ..."
end-of-version offset 0x110821
version string start 0x1107f1 => total 66
b'0.0.1 on "android_arm64"'
```
The banner exists, is 66 bytes long, and its numeric field is `0.0.1` with a blank build-id field —
no `(stable)`, no `(Mon …)`. A regex that takes `\d+\.\d+\.\d+` out of this returns a value that would
send a reader to build a Dart VM for `0.0.1`.
**observed** (the byte string above), **inferred** (that this is a build-time population failure rather
than a deliberately rewritten banner). The actionable consequence is in `dart-aot.md` §1: assert the
`(stable) (…)` tail before using the number, and fall back to the snapshot hash otherwise. **The
sample's Dart version was not pinned in this pass** — that is an unverified item, not a finding.
## 6. Cross-validation
Every measurement above is a **raw byte search with an independent arithmetic assertion** (find the
literal in the file; check the byte or word immediately before it against the length of what follows).
It does not depend on `dart_pool_strings.py`, and the script's numbers are reported alongside the raw
probe rather than as the source of truth. The two agree on arm64 (4,980 kept entries, tag byte equal to
`0x80|(len<<1)` at every literal tested) and on armv7 (the packed scheme finds nothing, the byte layer
shows why).
**Not obtained**: a second, independently implemented Dart snapshot parser. `blutter` was not built in
this pass, so the reference's `pp.txt`-side claims were not re-derived here, and the armv7 conclusion
rests on the byte layout rather than on a tool's agreement. That is the boundary of this evidence.
## 7. What this pass changed in the repository
- `skills/apk-reverse/references/dart-aot.md` — §7 gained the armv7 record format and the
"the armv7 zero is a format mismatch" warning; §1 gained the banner-shape assertion; §10 gained the
data-key-versus-API-path rule with its measured failure mode, and the longest-first / assert-the-prefix
rule. No new file, no new index row: the finding belongs in the file that already owns this layer.
- The equal-length-replacement failure (a 404 whose body is not JSON, `jsonDecode` throwing, the app
frozen on the launch logo) is documented as the concrete instance of the top-level constraint
"do not make an API fail to suppress a UI element".
## 8. Not done, and known to be outstanding
1. **No end-to-end device run.** The string-table patch was not repacked, signed and installed in this
pass, so "an equal-length key replacement suppresses the feature while the app still starts" is
`unverified` end-to-end even though the record layout that makes it legal is `observed`.
2. **No `blutter` build**, so no `pp.txt` comparison and no confirmation of the arm64 pool space from a
second implementation (§6).
3. **The sample's Dart version is unpinned** (§5); a structural probe or a snapshot hash is the next
experiment, not a guessed number.
4. **The armv7 header word is not characterised.** It is read from the file and used as a validator;
which component assigns it was not determined.

454
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-desensitization.md Normal file
View File

@@ -0,0 +1,454 @@
# Extension verification — desensitization and leak scans
Covers the pass that added `skills/apk-reverse/scripts/scan_leaks.py` and
`skills/apk-reverse/references/desensitization-and-leak-scans.md`, plus the precedent library
under `skills/apk-reverse/references/precedents/`.
Strength labels follow `docs/tool-verification/README.md`: **observed** (the exact command and
its output are reproduced here), **inferred** (follows from observed facts; the step itself was
not executed), **unverified** (assumed or reported elsewhere and not independently confirmed).
Reference environment: Windows 11 host, Python 3.14.0, repository root on branch
`main`. **No device, no network and no target sample were involved** — this pass is entirely
machine-local, because the thing worth measuring is what a scanner reports against a tree whose
contents are already known. Every identifier in the fixture and in every transcript below is
synthetic; the repository's convention of `<PKG>` / `<DEVICE>` / `<serial>` is used wherever a
real one would have appeared.
Date of the recorded runs: 2026-09-21. The numbers in §3b and §4a were measured on that date, on
the tree this pass leaves behind; those two sections' transcripts are the ones this pass produced.
**Headline: the planted-leak corpus is reported in full (30 findings, 10 further shapes admitted
through the exemption path), and the repository scan converges to zero strong/certain findings —
the residue is four `endpoint/weak` hits, all four an RFC 5737 documentation address, which is a
class this scanner reports on purpose. The first version of this file was itself the
counter-example to its own subject: it quoted the planted values to prove the rules fire, and the
scanner found that as a leak (§3b). The corpus keeps its values, because a corpus that cannot be
found proves nothing; the published document keeps none. The gate is not wired into anything yet;
that is the largest gap, and it is a Lead-side action.**
---
## 1. What was built, and what it was measured against
| Artifact | Role | Measured? |
|---|---|---|
| `skills/apk-reverse/scripts/scan_leaks.py` | the scanner: 17 rules in 6 categories, exemption table, four `RESULT=` tokens, `--fail-on strong\|any` | **observed** — every run below |
| `tools/_work/leak-precedent/gen_fixtures.py` | generates the planted corpus and the exemption corpus | **observed** — 7 files, re-runnable |
| `tools/_work/leak-precedent/fixture/` | the corpus itself (git-ignored work area) | **observed** |
| `skills/apk-reverse/references/desensitization-and-leak-scans.md` | the reasoning, the do-not-anonymize table, the failure modes | not executable |
| `skills/apk-reverse/references/precedents/**` | the case library, distilled from this repository's existing evidence | see the per-case strength tables |
The fixture exists so the claim "the scanner reports planted leaks" is reproducible rather than
a transcript pasted once:
```
$ python tools/_work/leak-precedent/gen_fixtures.py
wrote tools/_work/leak-precedent\fixture\planted\package.txt (411 bytes)
wrote tools/_work/leak-precedent\fixture\planted\device.txt (218 bytes)
wrote tools/_work/leak-precedent\fixture\planted\token.txt (261 bytes)
wrote tools/_work/leak-precedent\fixture\planted\appkey.txt (212 bytes)
wrote tools/_work/leak-precedent\fixture\planted\endpoint.txt (146 bytes)
wrote tools/_work/leak-precedent\fixture\planted\path.txt (175 bytes)
wrote tools/_work/leak-precedent\fixture\exempt\benign.txt (1396 bytes)
== 7 fixture file(s) ==
```
## 2. `--help`, and the rule table — observed
```
$ python skills/apk-reverse/scripts/scan_leaks.py --help
usage: scan_leaks.py [-h] [--root ROOT] [--only ONLY] [--exclude EXCLUDE]
[--format {text,json}] [--show-exempt] [--list-rules]
[--max MAX] [--quiet] [--fail-on {strong,any}]
[--no-color]
Scan a skills repository for target identity that should not be published (bundle ids, device serials, tokens, SDK keys, literal endpoints, user paths). Exit 0 clean / 1 leaks_found / 2 error; prints RESULT=<token> last. `--fail-on strong` (default) keeps the gate green when the only hits are documentation-range addresses.
options:
-h, --help show this help message and exit
--root ROOT repository root to scan (default: the repo this script
lives in)
--only ONLY comma-separated categories to restrict to, e.g.
pat,appkey
--exclude EXCLUDE extra directory name to skip (repeatable)
--format {text,json}
--show-exempt also print findings that were suppressed by an
exemption
--list-rules print the rule table and exit
--max MAX stop after N findings (0 = no limit)
--quiet print only the RESULT token
--fail-on {strong,any}
which findings make the exit code non-zero: `strong`
(default) fails only on strong/certain findings, `any`
also fails on `weak` ones such as an address from the
RFC 5737 documentation range
--no-color accepted for compatibility
Categories: package, device, token, appkey, endpoint, path
Exemptions (tool names, library names, CVE ids, hardening products, public
crackme names, <PKG>-style placeholders, loopback/emulator addresses) are built
in and audited with --show-exempt.
```
`--list-rules` prints the table the exemption functions are attached to, which is what makes the
scanner arguable rather than opaque. It exits 0 and prints `RESULT=clean` (no files were read),
and the top of its output is:
```
$ python skills/apk-reverse/scripts/scan_leaks.py --list-rules
== rules ==
bundle_pkg_attr package strong a `package=` bundle id in a manifest or transcript
bundle_pkg_decl package strong a bundle id in a `<manifest>` line
bundle_pm_path package strong a bundle id in a `pm`/`am`/`monkey` invocation
bundle_process_line package strong a bundle id in a `ps`/`pidof`/`top` capture
bundle_component package strong a bundle id in a `component=`/`-n` activity reference
device_serial device strong an `adb devices`-shaped device serial
device_serial_context device strong a serial next to a device/install/`su` context word
device_serial_tabular device strong a bare 16-char token in a device-listing table
token_github_pat token certain a GitHub fine-grained personal access token
token_github_classic token certain a GitHub classic token
token_env_assignment token strong an inline API-key/token/secret assignment
token_authorization_header token strong a literal Authorization header value
appkey_assignment appkey strong an SDK appkey/appsecret assignment with a literal value
plain_addr endpoint weak a non-loopback literal IP:port
plain_addr_ipliteral endpoint weak a non-loopback bare IPv4 literal
user_path_posix path strong an absolute POSIX user-home path
user_path_windows path strong an absolute Windows user-profile path
RESULT=clean (exit 0)
```
## 3. The planted corpus — observed, every rule fires
```
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/leak-precedent/fixture --format json
{ "root": "...\\tools\\_work\\leak-precedent\\fixture", "files_scanned": 7,
"findings": [ ... 30 ... ], "exempted": [], "errors": [], "result": "leaks_found" }
RESULT=leaks_found (exit 1)
```
aggregated (the `findings` array of the same JSON run, grouped by category):
```
files 7
findings 30 {'appkey': 4, 'token': 5, 'device': 8, 'endpoint': 5, 'package': 5, 'path': 3}
```
and the 30 hits by position, rule and value **shape** — every field of the report except the
literal itself. Rule name, category, strength, position and count are the evidence; the quoted
value is a copy of the leak, and copying it into a published document re-creates what the scan
exists to prevent (§3b is where that rule was paid for):
```
planted/package.txt:2:16 bundle_pkg_decl 4-segment all-lowercase reverse domain
planted/package.txt:2:26 bundle_pkg_attr the same value, second rule on the same line
planted/package.txt:3:1 bundle_pm_path 3-segment all-lowercase reverse domain, after `pm path`
planted/package.txt:5:1 bundle_process_line 4-segment all-lowercase reverse domain, after `pidof`
planted/package.txt:6:15 bundle_component 4-segment all-lowercase reverse domain, after `component=`
planted/device.txt:4:1 device_serial 16×[A-Z0-9], tab-separated from the word `device`
planted/device.txt:4:1 device_serial_tabular the same value, second rule on the same line
planted/device.txt:5:3 device_serial_context 16×[A-Z0-9], after `$ adb -s `
planted/device.txt:5:10 device_serial the same value, second rule on the same line
planted/device.txt:6:2 device_serial_context 16×[A-Z0-9], after `[serial `
planted/device.txt:6:9 device_serial the same value, second rule on the same line
planted/device.txt:7:1 device_serial_context 16×[A-Z0-9], after `device serial: ` (a second value)
planted/device.txt:7:16 device_serial the same value, second rule on the same line
planted/token.txt:2:14 token_github_pat fixed provider prefix + 62×[A-Za-z0-9_]
planted/token.txt:3:15 token_github_classic fixed 4-char provider prefix + 36×[A-Za-z0-9]
planted/token.txt:4:1 token_env_assignment 20-char `AKIA`-prefixed key id after a key assignment
planted/token.txt:5:9 token_authorization_header 32×[A-Za-z0-9] after an `Authorization:` keyword
planted/appkey.txt:5:1 token_env_assignment the same 40-char secret, second rule on the same line
planted/appkey.txt:2:1 appkey_assignment 24×[0-9a-f] after an appkey assignment
planted/appkey.txt:3:1 appkey_assignment 24×[0-9a-f], quoted, after a camelCase app-secret assignment
planted/appkey.txt:4:1 appkey_assignment 7 dash-separated 4-character uppercase groups
planted/appkey.txt:5:1 appkey_assignment 40-char mixed-case secret after a `SECRET_KEY` assignment
planted/endpoint.txt:2:7 plain_addr an RFC 5737 documentation address, with `:port`
planted/endpoint.txt:2:7 plain_addr_ipliteral the bare form of the same address
planted/endpoint.txt:3:16 plain_addr an RFC 1918 private-range address, with `:port`
planted/endpoint.txt:3:16 plain_addr_ipliteral the bare form of the same address
planted/endpoint.txt:4:12 plain_addr_ipliteral an RFC 5737 documentation address, bare
planted/path.txt:2:10 user_path_posix two-segment personal name after `/home/`
planted/path.txt:3:6 user_path_posix one-word personal name after `/Users/`
planted/path.txt:4:6 user_path_windows capitalized personal name after `C:\Users\`
```
Two rules firing on one line (`SECRET_KEY=` matches both an appkey and a token pattern) is kept
deliberately and is visible in the count; identical duplicates (same rule, same value, same
line) are collapsed. Eight of the 30 findings share their line with a second rule — one package
line, all four serial lines, the `SECRET_KEY` line and the two endpoint lines — which is why 30
findings cover 22 distinct values (counted from the `findings` array of the JSON run above, not
by hand).
### 3a. The exemption corpus — observed, and it is the half that is easy to get wrong
The same run with `--show-exempt` shows what was *not* reported from
`fixture/exempt/benign.txt`, and the reason:
```
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/leak-precedent/fixture --show-exempt
== suppressed by exemption: 10 ==
exempt/benign.txt:13 [user_path_windows] <user> -> placeholder user
exempt/benign.txt:17 [plain_addr] 127.0.0.1:27042 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr_ipliteral] 127.0.0.1 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr] 0.0.0.0:8080 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr_ipliteral] 0.0.0.0 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr] 10.0.2.2:8080 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr_ipliteral] 10.0.2.2 -> loopback/unspecified/emulator address
exempt/benign.txt:18 [plain_addr] 127.0.0.1:8787 -> loopback/unspecified/emulator address
exempt/benign.txt:18 [plain_addr_ipliteral] 127.0.0.1 -> loopback/unspecified/emulator address
exempt/benign.txt:18 [plain_addr_ipliteral] 169.254.169.254 -> loopback/unspecified/emulator address
```
The earlier pass recorded 6 suppressed hits, all `plain_addr`; the current run reports the paired
`plain_addr_ipliteral` for each exempt address as well, which is what the two same-line rules
look like when nothing is being collapsed. The count moved from 6 to 10 for that reason and not
because the exemption table changed — nothing in it was touched by this pass.
Everything else on that file — tool names, library names, function names, dex/ELF constant
identifiers, protocol field names, CVE ids, hardening product names, the public crackme and its
URL, platform and SDK packages, `com.example.*` / `probe.synthetic.*` and the `<PKG>`/`<DEVICE>`/
`<hash>` placeholders — produced **no finding at all**, which is the intended outcome for the
do-not-anonymize list and is also the check that the two synthetic fixture prefixes named in
`references/desensitization-and-leak-scans.md` really are exempt: `com.example.app` sits on line
83 of that file and is absent from both the findings and the suppressed list, because
`BENIGN_PACKAGE_PREFIXES` contains `com.example`, `org.example`, `net.example`, `io.example` and
`probe.synthetic`. The file was written to hold every category of that list, one line each, so a
regression there shows up as a new finding rather than as a silent behavioural drift.
`169.254.169.254` (the cloud metadata endpoint) is exempted **only** in this documented-endpoint
form. It is a real address that matters in a different context; the exemption is a judgement
recorded in `BENIGN_IP_PREFIXES`, not a pattern match, and it is the entry most worth
re-examining if this list is ever copied to another repository.
### 3b. The leak this file carried — found by the scanner it documents
The first version of this section listed all 30 hits **with their values**, quoted out of the
JSON, because a report with the value in it is the concrete kind. The Lead then ran the tool
against the repository and got `RESULT=leaks_found`, exit 1, with **26 strong/certain findings
and 14 weak ones — all 26 strong ones inside this file**, at lines 113–146 of the version that
existed then: the 5 bundle-id hits, 2 distinct device serials (12 hits), 2 provider-format tokens,
2 inline assignment values, 2 SDK appkeys and 3 host user names it had just finished quoting.
Nothing else in the tracked surface produced a strong hit.
That is the failure mode this whole pass exists to prevent, committed by the pass itself, and it
states the boundary in one line: **the corpus is where identity is allowed; the published surface
is not.** `tools/_work/leak-precedent/` is git-ignored and excluded from the default scan root so
that it *can* hold the values — a corpus that does not contain them cannot demonstrate the rules
firing. This file is not in that category: it is tracked, it is read, and every string in it is
published.
The correction, applied in this pass:
1. The hit list in §3 now carries rule, category, strength, position, count and a **structural
description** of each value, in the repository's own placeholder notation. No evidence was
dropped: all 30 positions, all 17 rule ids, both strength labels and the per-category counts
are still here, and the corpus still produces all 30 when it is scanned directly (§3).
2. The two version-shaped literals in §4's false-positive table are now written in
`<N>.<N>.<N>.<N>` form. They are legitimate values, but leaving them would put two *different*
classes in the weak residue instead of one, and an unexplained second class is how a weak hit
starts getting ignored.
3. `references/desensitization-and-leak-scans.md` records the rule this produced — a report is a
copy of what it found, so the report is scanned like any other published surface — and its
exit table now carries the `leaks_found_strong_only` state.
The scanner's own verdict on the corrected tree is in §4a. The lesson is the one the reference
file already made about entry points, applied to an evidence file: **run the scan on the file you
are about to publish, not on the file you started from** — and then do not paste the thing it
found back into that file.
## 4. The repository scan — observed, and the false positives that produced it
The first run against the real tree reported **28 findings**, of which 27 were wrong. The
sequence is recorded in full because the corrections are the useful part: every one of them was
found by running the scanner, not by reasoning about it.
```
$ python skills/apk-reverse/scripts/scan_leaks.py # first run
== leak scan: 118 file(s) scanned under the repository root ==
findings by category: device=4, endpoint=20, package=4
== result: 28 finding(s) == RESULT=leaks_found (exit 1)
```
| # | Reported | Why it was wrong | Fix |
|---|---|---|---|
| 1 | a `jdk`-version-shaped run (`…\jdk-<N>.<N>.<N>.<N>`) ×19, across eight files in `docs/tool-verification/` | it is the JDK version in the toolchain's own path; the token `jdk` sits immediately before the match | a 30-character context window before the literal must not end in a version word (`jdk\|java\|python\|frida\|…`) |
| 2 | `0000000040001000`, `0000000000714240` | tombstone register/`pc` values — 16 hex digits, all digits | an all-digit 16-character token is suppressed; a real platform serial carries at least one letter |
| 3 | `0807060504030201` ×3 | a protobuf `fixed64` fixture in `EXTENSION-protobuf-raw.md` and in `scripts/protobuf_decode_raw.py` | same rule as #2 |
| 4 | `sg.vantagepoint.uncrackable3` | the published package name of an OWASP MASTG crackme | `PUBLIC_TARGET_PREFIXES` (public crackme/benchmark families are reuse, not identity) |
| 5 | `probe.synthetic.vmpdiff` | the synthetic manifest inside `vmp_diff_harness.py` | added `probe.synthetic` to the fixture prefixes |
| 6 | `args.package.spli` (twice truncated to `spli`) | a property lookup in `scripts/spawn_patch_detach.py`; the exemption window was measured from the *match* start, which an unbounded `[^\n]*?` prefix pushes past the `args.` | the window starts at the **capture group** (`m.start(group)`), and the exemption reason is printed so the path can be audited |
| 7 | `<N>.<N>.<N>.<N>` | an SDK version *directory* (`…/com.kwad.components.tachikoma/<N>.<N>.<N>.<N>/base-1_apk`) inside a private-storage path | a version-directory rule: a `.`-separated digit run enclosed in path separators is not an address |
Two rows in that table are quoted in placeholder form rather than as the literal the first run
printed — row 1's four-part JDK version run and row 7's four-part SDK version run. Neither names
anybody: both are version numbers, and both were rewritten in this pass for the reason given in
§3b, because leaving them would put a second class (version runs, not addresses) into the weak
residue, where an unexplained class is the first thing a reader learns to skip. The shape is what
each row is about, and it survives the rewrite intact.
### 4a. What this pass inherited, and what it leaves — observed
Inherited state, reproduced at the start of this pass (same tree, before the §3b correction):
```
$ python skills/apk-reverse/scripts/scan_leaks.py
== leak scan: 128 file(s) scanned under the repository root ==
findings by category: appkey=2, device=12, endpoint=14, package=5, path=3, token=4
== result: 40 finding(s) -- each one is a line to look at, not a verdict ==
RESULT=leaks_found (exit 1)
```
26 of the 40 are strong/certain (`package` 5, `device` 12, `token` 4, `appkey` 2, `path` 3) and
every one of those 26 is a line of this file's former §3 hit list, at lines 113–146. The other 14
are weak: 5 more lines of that same block (its endpoint rows), 2 version strings in §4's own
table, 3 RFC 5737 range literals in
`skills/apk-reverse/references/desensitization-and-leak-scans.md`, and `scripts/tls_check.py`'s
usage example.
Post-correction, on the tree this pass leaves behind:
```
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet
RESULT=leaks_found_strong_only (exit 0)
$ python skills/apk-reverse/scripts/scan_leaks.py
== leak scan: 129 file(s) scanned under the repository root ==
findings by category: endpoint=4
skills/apk-reverse/references/desensitization-and-leak-scans.md:81:10 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
skills/apk-reverse/references/desensitization-and-leak-scans.md:81:26 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
skills/apk-reverse/references/desensitization-and-leak-scans.md:81:45 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
skills/apk-reverse/scripts/tls_check.py:44:23 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
== result: 4 finding(s) -- each one is a line to look at, not a verdict ==
all are `weak` (the documented address ranges); the gate is green under the default --fail-on strong, and `--fail-on any` is the strict pass
RESULT=leaks_found_strong_only
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet --fail-on any
RESULT=leaks_found (exit 1)
```
The four residual hits, by position, rule and class. Their individual values are deliberately not
reprinted here — that is §3b's correction, and re-printing them is how this file leaked the first
time. All four are the same class:
| position | rule / strength | class |
|---|---|---|
| `skills/apk-reverse/references/desensitization-and-leak-scans.md:81` ×3 | `plain_addr_ipliteral` / weak | the three RFC 5737 documentation blocks, named inside that file's own statement that they are *not* exempt |
| `skills/apk-reverse/scripts/tls_check.py:44` | `plain_addr_ipliteral` / weak | the script's usage example, an RFC 5737 address |
Reading: **zero strong/certain findings and zero unexplained hits on the tracked surface; four
weak ones, one class, both sources named.** The residue is the price of the design decision
recorded in §"Deliberately *not* exempt" of the reference file: the documentation that states the
decision has to name the ranges, and the ranges are exactly what the rule reports. That is why
the gate's pass condition lives in `--fail-on` and not in the exemption table, and why
`--fail-on any` stays red on this repository on purpose.
The file count is 129 here, 128 in the inherited run above and 119 in the first pass: the tracked
surface grows while several writers work on it, and this pass added no file to the scanned
surface, so the count is a property of the run rather than of the leak. The number that is
comparable across the runs is the strong/certain one: **26 inherited, 0 now**. The inherited
figure also reproduces the Lead's own catch (§3b) exactly, which is the point of quoting the
transcript rather than the conclusion.
Two work artifacts of this pass live beside the corpus in the same git-ignored directory, and
neither is part of the scanned surface: `EXTENSION-desensitization.before.md` (the pre-correction
snapshot of this file, kept so that the correction is auditable in both directions) and
`fixture-scan-before.txt` (a captured run of the fixture scan). Both contain the planted values
by construction, exactly like the corpus, and both are excluded from every default run — the work
area is where identity is allowed to exist; that is the whole of §3b's boundary.
## 5. The exit-code contract — observed, all four states
```
$ python skills/apk-reverse/scripts/scan_leaks.py --root LICENSE --quiet
RESULT=clean (exit 0)
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet
RESULT=leaks_found_strong_only (exit 0) # the repository tree, weak hits only
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet --fail-on any
RESULT=leaks_found (exit 1) # the same tree, strict pass
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/leak-precedent/fixture --quiet
RESULT=leaks_found (exit 1)
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/nope-not-here
error: root does not exist: tools/_work/nope-not-here [stderr]
RESULT=error (exit 2)
$ python skills/apk-reverse/scripts/scan_leaks.py --only nope
error: unknown category (or categories): nope [stderr]
RESULT=error (exit 2)
```
The token and the exit code are deliberately separate channels: a wrapper that only greps
stdout cannot tell "clean" from "the scanner never ran", and one that only reads `$?` cannot
explain itself in a log.
`--fail-on` is what keeps those two channels from contradicting each other on a tree that is
documentation-shaped. Under the default (`strong`) a tree whose only findings are `weak` reports
`leaks_found_strong_only` and exits 0: the findings are still printed in full, they simply do not
fail the gate. Under `--fail-on any` the same tree reports `leaks_found` and exits 1, which is the
right verdict at publish time and the wrong one for a habit that runs on every change — a gate
that is red on its own design documentation is a gate people disable.
## 6. What was not done
- **The `RESULT=` token convention was undocumented when this pass started, and it is documented
now.** The Lead added `references/long-task-discipline.md` §Every script answers with a token,
not with prose during this pass, so the scanner is no longer the convention's only user. That
section's vocabulary is generic (`clean`, `patched`, `success`, `crash`, `timeout`,
`unavailable`, …) and does not spell out `leaks_found` / `leaks_found_strong_only`; whether
they belong in the enumeration is a Lead call. Verified with
`grep -n "RESULT=" skills/apk-reverse/references/long-task-discipline.md` → one match, line 70.
- **Nothing is wired into a gate.** No pre-commit hook, no CI job, no line in the maintenance
three-command habit. The scanner has never run automatically; every result above is a manual
invocation. Recorded as `unverified` in the sense that matters: whether it *would* be run is
untested.
- **No scan of the git-tracked list specifically.** The scan walks a filesystem root, so its
coverage is "the default surface under this root", not "the files that exist in HEAD". A file
that is present locally but ignored, or tracked but absent, is not distinguished.
- **No check of binary or archive content.** `*.apk`, `*.png`, `*.pcap` and `*.zip` are not
opened. Identity inside a screenshot or a capture file is real and this pass does not address
it.
- **No git-history scan.** A string deleted from the working tree remains in the history; nothing
here inspects commits. This pass did check the one case it created: `git status --short` shows
`docs/tool-verification/EXTENSION-desensitization.md` as `??` (untracked), so the values §3b
removed were never committed and nothing was written into history. That is a check of this
file, not a history scan.
- **No second-language rule set.** Identifiers that are not reverse-domain shaped (a nickname, a
local-part email, a CJK app name) are not covered. The rules are shape-based by construction.
- **No cross-repository validation of the exemption list.** The benign prefixes were derived from
this tree's own contents (a grep over its Markdown for reverse-domain tokens). Whether the same
list is right for another repository is `unverified`, and the reference file says so.
- **The `--max` / `--only` / `--exclude` paths were exercised only in the forms shown above.**
`--max` was not driven at all.
## 7. One-line deliverable statement, and the open items
**Deliverable (verified):** `python skills/apk-reverse/scripts/scan_leaks.py` scans a repository
root for six categories of target identity, reports each hit with a limited context window and a
strength label, exits 0/1/2 with a matching `RESULT=` token, reports all 30 planted leaks in the
reproducible corpus at `tools/_work/leak-precedent/fixture` (7 files, 10 further shapes admitted
through the exemption path), and against this repository's 129-file tracked surface returns zero
strong/certain findings — the residue is four `endpoint/weak` hits, all RFC 5737, both sources
named in §4a — so the default gate is green (`RESULT=leaks_found_strong_only`, exit 0) while
`--fail-on any` stays available as the strict pass (`RESULT=leaks_found`, exit 1).
**Open items for the Lead:**
1. Wire the scan into the maintenance gate (`check_repo.py`, `check_refs.py`, `scan_leaks.py`)
and record in `README.md` that the third one's exit 1 is a *content* finding, not a
mechanical failure like the other two.
2. Decide on a history scan. Nothing in this pass looked at commits, and a leak removed from the
tree is still in them.

311
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-detection-pipeline.md Normal file
View File

@@ -0,0 +1,311 @@
# Extension verification — detection pipeline, layered descent, ptrace-free dumping
Covers the pass that added the §Step 3 locating pipeline and the §Step 2B boundaries to
`references/detection-and-anti-analysis.md`, and the §What this route cannot do /
§Is this dex image real / §Layered descent sections to `references/advanced-unpacking.md`.
It also ships `scripts/svc_scan.py` and `scripts/anti_detect_probe.js`.
Strength labels, as defined in this directory's README:
- **observed** — reproduced here, with the exact command and its output.
- **inferred** — follows from observed facts, but the step itself was not executed.
- **unverified** — assumed, or reported by a tool and not independently confirmed.
**Target identity is deliberately absent.** The hardened sample appears as `<PKG>`; the
public MASTG challenge apps appear by their package names because they are public
published targets, not a user's app. No keys, signatures or user data are recorded.
## 0. Environment and what was decided before measuring
| Item | Value |
|---|---|
| Device | Android 11 (API 30), arm64-v8a, kernel **4.14.186**, Magisk alpha + root, toybox shell |
| Host | frida 16.7.19 / frida-tools 13.7.1; capstone; lief; pyelftools |
| Instrumentation server | **a `frida-server` was already running on the device** (`frida-server -l 0.0.0.0:27099`, pid 31859) with a leftover `adb forward tcp:27099` |
| Decision taken | **Reuse it rather than restart it.** A restart is a device-wide variable and the process had a live abstract socket (`unix:abstract=/frida-...)`; it was never started by this pass, so nothing about it is attributed here. Cost: the server's listening port and its path are part of the environment a detector can see — see §4, where a probe confirms frida artefacts *are* visible in the target's own maps |
| Root hiding | `magisk --denylist status` = *enforced*, list **empty** → no app had root hidden. Not changed in this pass |
`frida-ls-devices` cannot be used as a liveness check in this harness (it requires a
Windows console and dies with `NoConsoleScreenBufferError`); the Python binding is the
reliable route, and that is what every measurement below used:
```python
dev = frida.get_device_manager().add_remote_device('127.0.0.1:27099')
dev.query_system_parameters()
# {'arch': 'arm64', 'os': {'version': '11', 'id': 'android'}, 'api-level': 30, 'access': 'full'}
```
## 1. `scripts/svc_scan.py` — measured, with the count cross-checked
`--help` works; a real call was made on five inputs. T3 cross-check first, because the
whole script is a decode claim:
```
json sites = 214
hand word-scan = 214
set difference = []
```
The hand word-scan is a 6-line script that matches the `svc` encoding
(`(w & 0xFFE0001F) == 0xD4000001`) with no dependency on capstone, so the two readings
are genuinely independent implementations. **observed.**
Device libraries, scanned from copies pulled with `su -c cp`:
| Input | Sites | by_group | Notable |
|---|---|---|---|
| `/apex/…/bin/linker64` (1,451,392 B) | 214 | termination=4, evasion=5, discovery=8, unresolved=6 | the 4 termination sites are `exit`/`exit_group` at `0xda284`/`0xda2a4` |
| `/apex/…/lib64/bionic/libc.so` (1,193,024 B) | 215 | termination=4, evasion=6, discovery=8, unresolved=6 | `kill @0xc5b94`, `tgkill @0xc5bb4`, `ptrace @0xc5bd4`, `mprotect @0xc5f54`, `exit_group @0xc7054`, `exit @0xc7074` |
| `assets/libjiagu_a64.so` from the sample (1,171,896 B) | 21 | **unresolved=21** | every site is inside `scvtf`/`udf`/NEON byte soup — data, not instructions |
| `assets/libjiagu.so` (32-bit, 861,436 B) | 11710 | unresolved=11116 | scanning both PT_LOAD segments mixes code and data; the site count is not usable without inspecting neighbours |
**The interpretation that matters, and it is the reason the script exists:** the four
termination sites in `libc.so` are libc's *own* exported implementations, so a hook on
`exit`/`kill` does see callers that go through libc. The shell library has **no**
termination site and 21 data artefacts — so on this target a libc-level hook is *not*
structurally excluded, and a missing exit event would be a finding about the hook.
**observed.**
Two defects were found by running it and both are fixed in the shipped script:
- `capstone` raises `CS_ERR_DETAIL` unless `detail = True` is set; the first run died on the
first hit (`AttributeError`-equivalent at `insn.operands`).
- `pf_x` is **not** usable as a filter on this platform. The ROM's own `libc.so` and
`linker64` carry `pf_x` set on their code segment but a first version of the probe read
`p_flags` from the wrong struct offset (reading `p_offset`'s low word instead), which
reported `pf_x=0` for everything. The shipped version reports per-segment `pf_x` and does
not filter on it by default: `--code-only` is opt-in precisely because a device that
clears the flag would otherwise return zero hits silently.
## 2. The hardened sample: what it does now, and why nothing was dumped from it
The previous pass recorded this sample as a clean self-exit on a 7-18 s cycle with no
tombstone. **That is no longer the shape.** Measured this pass, twice, 20 samples at 1 Hz
after `monkey` launch:
```
22:28:19.923 D/OppoExSurfaceFlinger: hecate traverse … Splash Screen com.gentle.ppcat#0
22:28:21.684 W/ActivityManager: Process ProcessRecord{… 14905:com.gentle.ppcat/u0a623} failed to attach
22:28:21.686 I/ActivityManager: Killing 14905:com.gentle.ppcat/u0a623 (adj -10000): start timeout
```
- **observed:** the platform kills the process on **`failed to attach` / `start timeout`**
roughly **2 s** after start, i.e. the app never finishes starting.
- **observed:** an attempt to harvest memory inside that lifetime failed on both tries —
`RESULT=NO_PID` and `RESULT=NO_PID` with `t_pid=9.99`, so a 10 s window is already longer
than the process lives. Two strikes, no third variant: **the ptrace-free dump was not
obtained from this sample and this file does not claim one.**
- **observed, and a new signal rather than a repeat:** a tombstone is now produced, and its
fault is not in the packer's own code but in the runtime, reached from JIT output:
```
signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x75f16ff028
backtrace:
#00 pc 0000000000574028 /apex/com.android.art/lib64/libart.so
(art::Runtime::SetJavaDebuggable(bool))
#01 pc 0000000000022d58 /memfd:jit-cache (deleted)
```
A self-terminating `SIGABRT` would be the expected shape for an anti-analysis kill; a
`SEGV` in `Runtime::SetJavaDebuggable` reached **from JIT-compiled Java** is not, and the
honest reading is limited: **the app now crashes during startup, in the runtime, on a
path a Java method entered.** Whether that is the packer's environment check firing or a
consequence of the kill arriving mid-startup is **unverified** — the sample's own
start/stop behaviour changed between passes and no control run with root hiding
configured was made, because changing root-hiding state is a device-wide variable this
pass deliberately did not touch.
## 3. Clean-target baseline: state, liveness, and a harness artefact avoided
- **observed:** `owasp.mstg.uncrackable1` launched with `monkey` stayed alive at pid 15310
for **more than a minute** of repeated checks — the stable baseline every dynamic
experiment needs, and the reason the arms below were run on it and on L3 rather than on
the hardened sample.
- **observed:** `owasp.mstg.uncrackable3` runs two processes (zygote64 + 32-bit), had its
own integrity log line, and stayed up.
- **observed, and the most expensive-looking failure that turned out to be free:** the
first attach to L1 hung and then failed —
`attach/spawn failed: unexpectedly timed out while waiting for signal from process with PID 15310`
— and a second attach said `process not found`. The process was still in `ps` and
`dev.enumerate_processes()` still listed it, but:
```
$ adb shell 'cat /proc/15310/stat | cut -d" " -f3'
D
```
**`D` (uninterruptible disk sleep)** is why. A userspace attach needs the process to run.
The one-line control that settled it in a single command: attaching to a *different* pid
from the same device succeeded (`attach 15974 OK`). **observed** — the failure was the
environment, not a detector, and this is now Stage 0 of the reference's pipeline.
**Harness self-check applied before interpreting anything** (the discipline that a spawn
which is never resumed produces zero events and reads as a hang): the driver
(`tools/_work/detect/run_anti_detect_probe.py`, a work-file, not shipped) resumes
**explicitly** after the script is loaded, and records whether it did. Both arms below
report `resumed=True`, so "no events" cannot be an un-resumed spawn here.
## 4. The three arms, on one target — and the detection actually found
All arms: `owasp.mstg.uncrackable3`, `--spawn`, script loaded before resume.
| Arm | Script | Result |
|---|---|---|
| **A. baseline** | none (launch, no instrumentation) | process alive and stable across a minute of sampling (§3) |
| **B. control** | attached/spawned, resumed, **no script loaded** | **`alive_at_end=True`**, survived the full window |
| **C. experimental** | `scripts/anti_detect_probe.js` (observer-only) | **`alive_at_end=False`**, process gone before the first heartbeat |
Control vs. experimental differs in exactly one variable — the probe script — so the death
is attributable to instrumentation. Then the observer's own stream named the check:
```
t=0 [armed] script armed
t=207ms [thread_create] pthread_create entry=0x76dc031d78
t=238ms [strcmp_probe] strcmp("Ldalvik/system/ClassExt;")
t=245ms [env_probe] open("/system/fonts/Roboto-Regular.ttf")
t=254ms [loader] android_dlopen_ext("…/oat/arm64/base.odex")
t=303ms [strstr_probe] strstr("frida") <-- the check
t~350ms process gone, heartbeats=0, no tombstone, no crash/ANR record
```
- **observed:** the target **searches for the string `frida`** (`strstr("frida")`) at ~300 ms
of its life. This is the "detection really did happen" datum the task asked for: not an
inference from a death, a recorded call with a timestamp.
- **observed:** an earlier arm of the same shape also captured the kill itself —
`[termination] raise(...) <- …` at **t=352 ms**, i.e. a signal-based self-abort roughly
50 ms after the `strstr`. No tombstone was written and no `has died: crash` record
appeared, which is the clean-self-exit signature.
- **observed, and the reason the arm is reported as unstable:** three attempts at the same
arm produced *one* full sequence, then two runs where the script received **zero**
messages before the process died (`live=0, messages=0, heartbeats=None`). On a target that
kills itself in ~300 ms the observable window is sub-second and **not reproducible arm to
arm**. One successful attribution is therefore not a pipeline, and the reference says so.
What the probe also recorded about the *environment*, before any check ran:
```
tracerpid=0 frida_named_maps=4 (four mappings whose names match /frida/i)
status: Name: tg.uncrackable3 ; State: S (sleeping) ; Seccomp: 2
/proc/net/tcp -> Permission denied /proc/self/task -> Is a directory
```
**observed:** the target's own `/proc/self/maps` contains **four frida-named mappings** while
`TracerPid` reads **0**. So a `TracerPid`-only check would pass while a maps-or-thread-name
check would not — which is exactly the "what can this process actually see" question the
Stage 1 self-report exists to answer, and it is why the probe reports both.
**One defect in the probe, fixed once and then left:** caller attribution printed
`bt-unavailable: cannot read property 'context' of undefined` on every line. Cause: the
`this` binding inside the anonymous `Interceptor` callback was not what the helper assumed;
the fix is to pass `this.context` explicitly. The measured consequence is worth stating
because it is a *plumbing* failure that reads like a hooking failure: the events and their
timings were correct throughout, only the "which module called this" half was missing, and
one of the three arms still showed no events at all. `observed`.
## 5. ptrace-free dumping: the route, its boundary, and a cross-check that passed
### 5.1 What the route yields when the target has a whole dex in memory
Not re-measured here: the earlier pass's 17 ART dex mappings exported from `/proc/<pid>/mem`
without any instrumenter present (`docs/tool-verification/EXTENSION-rootdump.md`). This pass
adds the **negative boundary** the earlier pass could not state, and it needed a clean
target to state it:
```
# both public MASTG targets, ~2,400 map lines each, one adb round-trip
owasp.mstg.uncrackable1: maps_lines=2427 named_dex=0 apk_views=2
owasp.mstg.uncrackable3: maps_lines=2377 named_dex=0 apk_views=2
```
- **observed:** `named_dex=0`. Neither target produces an
`[anon:dalvik-classes.dex extracted in memory from …]` mapping, so the filename-driven
export has **nothing to export** — and this says nothing about those targets being
protected.
- **observed, the explanation:** their `classes.dex` is **deflate-compressed inside the
on-device APK** (`compress_type=8` read from the APK pulled off the device), so the
`dexopt` "map the dex directly" mechanism cannot apply to it.
- **observed:** the `r--s` views that *do* exist point at APK **file offsets**, and the
bytes there are the APK's own bytes — at `0x11000` on L1 that content is the ZIP
**central directory** (`PK\x01\x02`), not a dex.
- **observed, and a trap that produced a false "DIFFERENT":** on L3 the view
`76dc28e000-76dc299000` has file offset `0x15a000` = 1,417,216 B, while the mapping is
44 KiB (45,056 B) long — so the range reaches 1,462,272, i.e. **past the end of the
1,460,555-byte APK**. The file-side read came back short (1,015 B) and the two sha256s
described different byte counts. Compute the length before suspecting the target.
### 5.2 The reproducibility cross-check, which passed
Two independent read paths over the same address range must agree byte-for-byte before a
capture is evidence:
| Range | Read via `/proc/<pid>/mem` | Read from the file | Verdict |
|---|---|---|---|
| L3 `765a89c000-765a8de000` (264 KiB, file offset `0x6000`) | `sha256=2eafd7cca73630c05d0b0c1b4a7709f21ee24540b5f2ba388ad3467af645aa75` | same | **IDENTICAL** |
| L1 `76dfb14000-76dfb15000` (4 KiB, file offset `0x11000`) | `sha256=74846b8cb02a764efef158194745466cf0c8ad2c1c3fecbb2bcf7cf9c58b6465` | same | **IDENTICAL** |
**observed.** The comparison is only meaningful because both sides covered the same byte
count; the one range that differed was the one whose length exceeded its backing file.
### 5.3 The cross-check that was **not** run, and why it is not silently absent
The task's cross-validation requirement is satisfied for `svc_scan.py` by two independent
decoders agreeing on an identical 214-site set (§1). For a **dump**, the equivalent check
needs two producers that emit the *same* artifact. What this pass has is `/proc/<pid>/mem`
(successful, §5.2) and, on the hardened sample, `frida-dexdump` — which was **refused** by
that sample in the previous pass (`script has been destroyed`,
`docs/tool-verification/EXTENSION-device-run.md`). The sample's lifetime then shrank to ~2 s
(§2), so the second producer could not be obtained on any target here:
| Second-producer option | Status |
|---|---|
| `frida-dexdump` on the hardened sample | **unverified** — refused in the earlier pass; not retried here on a 2 s process |
| `frida-dexdump` on L1/L3 | **unverified, and expected to be empty** — `named_dex=0`, so there is no whole image for any dumper to find |
| `process_vm_readv(2)` as a second backend | **not run** — the on-device helper would have needed a second binary on the device, and the byte-identity question it answers was already answered against the file path |
**Statement of the gap, so the next reader does not have to infer it:** the byte-identity
check between two *dumpers* is **unverified** in this repository. What is `observed` is
byte-identity between two *read paths* on the same bytes, and a 214-site decode agreement
between two independent decoders. Those are weaker claims than "two dumpers cross-checked",
and they are the ones this file makes.
**A related statement that is deliberately negative:** two dumpers agreeing is not evidence
of *recovery*. Against an extraction shell, two agreeing dumps are two copies of the same
skeleton. The judge for that is the platform decoder, per
`advanced-unpacking.md` §Establish "the bodies do not decode" with a decoder you did not write.
## 6. What was not established
- **Not** a recovered payload from the hardened sample. Its process lives ~2 s and never
reaches a state where its payload is loaded; the attempt hit the two-strike limit and was
re-routed to the stable targets.
- **Not** a fix for the sample's startup failure. Root hiding was left alone — configuring
`magisk --denylist` is a device-wide change and this pass was the only lock holder, which
is not the same as being free to change every shared variable.
- **Not** a reproducible end-to-end anti-instrumentation pipeline on a fast detector. The
full sequence was captured once out of three arms and the reference labels it that way.
- **Not** a general claim about `strstr("frida")` being the only check, or about MASTG
challenges, or about Android 11. One device, one ROM, one session, with other experiments
sharing the device.
- **Not** a kernel-layer result. Kernel 4.14.186 < 5.10 closes the eBPF route; the layered
descent in the reference stops at the `svc` boundary and says so.
## 7. Reproduction one-liners
```
python skills/apk-reverse/scripts/svc_scan.py tools/_work/detect/libc.so --context 2
python skills/apk-reverse/scripts/svc_scan.py tools/_work/detect/linker64.bin --json
adb shell 'su -c "sh /data/local/tmp/apkmaps.sh owasp.mstg.uncrackable3"' # named_dex / apk_views shape
adb shell 'su -c "sh /data/local/tmp/apkviews_verify.sh owasp.mstg.uncrackable3"'
adb shell 'su -c "sh /data/local/tmp/detlaunch.sh <pkg> /data/local/tmp/dd1"'
python tools/_work/detect/run_anti_detect_probe.py --pid <pid> --watch-ms 8000 --out arm.json
python tools/_work/detect/run_anti_detect_probe.py --pkg owasp.mstg.uncrackable3 --spawn \
--script skills/apk-reverse/scripts/anti_detect_probe.js --out arm_probe.json
```
The on-device helper scripts (`apkmaps.sh`, `apkviews_verify.sh`, `detlaunch.sh`) and the
driver live under `tools/_work/detect/` and are **work files, not shipped deliverables** —
they exist so the commands above are reproducible, and their content is summarised in the
sections that cite them. The two shipped scripts are `scripts/svc_scan.py` and
`scripts/anti_detect_probe.js`.

252
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-device-run.md Normal file
View File

@@ -0,0 +1,252 @@
# Extension pass — on-device run against a hardened Flutter sample
This file records the **device-side** half of the extension pass: what was actually executed against
a live, hardened sample, what the sample did in response, and which conclusions the evidence supports.
It follows the same three labels the rest of this directory uses:
- **observed** — reproduced here, with the exact command and its output.
- **inferred** — follows from observed facts, but the step itself was not executed.
- **unverified** — assumed, or reported by a tool and not independently confirmed.
**Target identity is deliberately absent.** The sample is referred to as *the sample*, and its package
identifier appears as `<PKG>`. The findings are about the tools and the methods, and stay valid
whichever hardened APK produced them.
## Environment (observed)
| Item | Value |
|---|---|
| Device | one physical arm64 device, Android 11 (API 30), kernel 4.14.186, Magisk root |
| Host | Windows 11, Python 3.14.0, frida 16.7.19, frida-tools 13.7.1, frida-dexdump 2.0.1, java 17, adb 1.0.41 |
| Device server | `frida-server` 16.7.19 (version-matched to the host by `frida-server --version`) |
| Sample | 360-jiagu hardened, `application android:name` = a third-party stub class; Flutter AOT payload |
| Host capability report | `python scripts/doctor.py` → static/repack/smali/native-patch/device/root/frida all OK; `28 / 28` scripts runnable |
| Pre-flight warnings | clock skew 3.32 s; leftover `adb forward` entries (8787, 8651, 65534, 27043); a device-side `frida-server` already running |
Two environment facts from `doctor.py` are worth carrying forward, because each of them silently
invalidates a measurement: the clock skew, and a **frida process already up on the device** — the
second one means the target may already be reacting to instrumentation before your first command.
## The measurement that mattered: the sample fights instrumentation
### (1) Control — the sample starts normally with no instrumentation (observed)
```
$ adb shell 'su -c "am force-stop <PKG>; sleep 1; am start -n <PKG>/.MainActivity"'
$ sleep 12 && adb shell 'su -c "ps -A | grep -i <PKG>"'
u0_a623 22691 17278 14123672 642488 SyS_epoll_wait 0 S <PKG>
```
Resident, 642 MB, healthy. **This is the control build the rest of the section is measured against.**
### (2) The pid drifts — but **not** because of frida (observed, corrected)
During the session the sample's pid changed repeatedly:
```
22691 → 23452 → 24267 → 27274 → 31284 → 32259 → 1368 → 8161
```
The natural reading is "the sample fights instrumentation", and the first write-up of this file said
exactly that. **The follow-up measurement refuted it.** With `frida-server` stopped and no attach of
any kind, the sample was restarted under observation for 90 seconds:
```
20:28:22 pid=[9961] load=36.43 | Mem: 11484 total 11077 used 406 free
20:28:33 pid=[9961] load=36.26 | Mem: 11484 total 10767 used 716 free
20:28:45 pid=[9961] load=35.61 | Mem: 11484 total 10739 used 745 free
20:28:56 pid=[9961] load=34.64 | Mem: 11484 total 10767 used 716 free
20:29:08 pid=[11246] load=34.91 | Mem: 11484 total 11151 used 333 free
20:29:19 pid=[11246] load=35.69 | Mem: 11484 total 11142 used 342 free
20:29:31 pid=[12238] load=35.14 | Mem: 11484 total 11131 used 353 free
20:29:42 pid=[13212] load=37.03 | Mem: 11484 total 11059 used 424 free
```
The pid changes with **no frida present**, roughly every 10-20 seconds, while the device sits at
300-750 MB of free memory out of 11.5 GB. The first correction to this file attributed that to memory
pressure. **That attribution was also wrong, and the device-side dumping run refuted it** with a
better instrument than sampling: `logcat`.
```
$ adb logcat | grep <PKG>
Process <PKG> (pid N) has died: fg TOP
Start proc <pid> for top-activity <PKG>
```
The process **dies as the foreground top activity, with no crash, no tombstone and no ANR record**,
and the platform immediately relaunches it — a clean self-exit in a loop, on a 7-18 second cycle. A
reclaiming device does not do that to a foreground process, and a memory-pressure kill leaves an
`lmkd` line and a low-memory record. What produces exactly this shape is the sample **checking its own
environment and quitting on purpose**.
And the environment it finds is decisive: on this device `magisk --denylist status` reports **enforced
with an empty list**, Zygisk-Assistant is installed but nothing is scoped, and
`/data/adb/shamiko` does not exist — so **no application on this device is hidden from root**
(`EXTENSION-lsposed.md`, observed). A hardened sample that refuses to run on a rooted device will do
exactly this, forever, and no amount of dump-retrying changes it.
**Reading, corrected twice:** the pid drift is an artefact of the **root environment**, not of
instrumentation and not of memory pressure. The practical consequence is the useful part — *before any
dynamic analysis of a hardened sample, configure root hiding and re-run the baseline*, because with
nothing hidden every dynamic result on this device is measured against a process that is trying to die.
The observer's first control run showed a "stable" sample for 12 seconds only because it happened to
sample inside one 17-second lifetime.
This is the skill's own rule applied twice to its own record: a conclusion was reopened each time a
measurement disagreed with it (`SKILL.md` §Stop conditions, last bullet). Two wrong attributions were
cheaper to delete than for a reader to inherit — and the second one only fell because a different
method (`logcat`, not sampling) was pointed at the same question.
The platform's own view of a failed launch was also captured, and is worth keeping as a *shape*:
```
W ActivityManager: Process ProcessRecord{…:<PKG>/u0a623} failed to attach
I ActivityManager: Killing <pid>:<PKG>/u0a623 (adj -10000): start timeout
```
A `start timeout` kill reads the same whether the cause is a slow shell init, a reclaiming device, or a
deliberate delay. It does not discriminate, and it is not evidence either way.
### (3) `frida-dexdump` cannot complete a memory dump — reproduced twice (observed)
Both spawn and attach were tried, with the parameter mistakes fixed between attempts:
```
$ frida-dexdump -U -f <PKG> -d -o <out> # spawn
Failed to spawn: need Gadget to attach on jailed Android; its default location is: …gadget-android-arm64.so
$ frida-dexdump -U -f <PKG> -d 12 -o <out> # 12 was parsed as an argv entry, not a delay
Failed to spawn: the 'argv' option is not supported when spawning Android apps
$ frida-dexdump -U -f <PKG> -d -o <out> # corrected
Failed to spawn: unexpectedly timed out while waiting for app to launch
$ frida-dexdump -U -p <pid> -d -o <out> # attach, twice, second run with the device idle
frida.InvalidOperationError: script has been destroyed: {'addr': '0x721e24c02c', 'size': 409556}
INFO:frida-dexdump:[*] All done... Process terminated
```
Attach was run twice. The **second** run was the clean one: no Stalker traffic anywhere on the device
(see the crash note below), the package enabled and verified running, the pid read immediately before
the attach, and a single variable changed relative to the control. It produced the same
`script has been destroyed` failure, and the sample's pid changed again in the aftermath.
**What the failed runs left behind, checked with the kit's own validator** (`observed`):
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py <dump_dir>
== dex dump validation: 2 file(s), 2 parse, 0 rejected ==
name size sha256[:12] ver cksum sig class noco code stub%
classes.dex 500 9461987e5464 035 BAD BAD 1986754560 0 0 0.0%
walk stopped early: index out of range
classes02.dex 10285489 116d8b84b486 035 BAD BAD 1986754560 0 0 0.0%
walk stopped early: index out of range
```
Both artifacts pass the magic check and then fail everything else: checksum and signature both BAD,
no classes recovered, the walk stopping on an out-of-range index. They are **memory fragments cut
short by the process dying mid-read**, not dex images — and this is exactly the case
`dex_dump_validate.py` exists for. A size-based heuristic would have kept the 10.2 MB file and
discarded the 500-byte one; both are worthless, and only a structural walk says so.
**Reading:** under these conditions the memory-dump route is refused by the target, not by the host
tooling. Two shape-identical failures on a single variable is the skill's own stop signal: the correct
move is to re-classify (`references/advanced-unpacking.md` §the four shapes, then
`references/detection-and-anti-analysis.md`), **not** to run a third variant of the same attach.
Two of the four failures above were the observer's own mistakes, and they are worth keeping because
they are cheap to repeat: `-d` is a switch, not a delay, and any extra positional argument reaches
`spawn(argv)`. `-n <name>` on this version of `frida-dexdump` also still routes through the spawn
path, so "attach by name" does not exist as a way around the spawn refusal — use `-p <pid>`.
### (4) What the sample looks like at the moment it is attacked (observed)
```
$ adb shell 'su -c "top -n 1 -b"'
PID USER PR NI VIRT RES SHR S[%CPU] %MEM TIME+ ARGS
31284 u0_a623 10 -10 13G 561M 268M S 439 4.8 0:05.52 <PKG>
154 root 20 0 0 0 0 S 66.6 0.0 0:02.18 [kswapd0]
17638 system 18 -2 12G 590M 419M S 60.6 5.1 1:15.93 system_server
```
A Flutter application in its startup window saturates several cores (439 % CPU) while the Dart VM
initialises, and the device — 11.7 GB RAM with 357 MB free at that moment — runs its reclaim thread at
two thirds of a core. **The load on this device during the experiment was the sample's, not the
observer's instrumentation.** That matters for attribution: "the device is slow, so the dump failed"
is not available as an explanation here, because the failing command is a *protocol* failure
(`script has been destroyed`) rather than a timeout.
## The device's own framework was already damaged (observed)
Recorded here because it is a **third candidate mechanism** for "things keep restarting", alongside the
sample's self-exit and the memory pressure — and because it invalidates any dynamic result that
depends on the hooking framework:
- `LSPosed Manager` opened to **"LSPosed is not installed"**, with no modules tab, while the `lspd`
daemon was still alive.
- The LSPosed configuration database had not been updated since **19:37** — it still pointed at the
`codePath` of an app that had been uninstalled, and never registered the replacement.
- `logcat` showed a loop: `system server died`, `am is dead`, `no response from bridge`, and
`Magisk: zygote crashed too many times, rolling-back`.
A zygote in a crash-and-rollback cycle restarts application processes by itself, so **"the pid changed"
had at least three independent possible causes on this device** (the sample exiting, reclaim, and a
broken framework). That is exactly why the corrected §2 attributes the drift to *instability of the
observation target* rather than to any single mechanism, and why a dynamic result needs its device's
framework state recorded next to it — `scripts/preflight.py` reads device state for the same reason.
Recovery is a reboot (Magisk re-initialises Zygisk and `lspd` at `late_start`; the LSPosed daemon is
started once and does not return if killed). Whether the framework came back after the reboot, and
whether the sample's self-exit behaviour changed with it, is recorded in
`EXTENSION-lsposed.md` — and any change there must stay **inferred**, because a reboot moves several
variables at once.
## A shared-device hazard, measured (observed)
While the run above was in progress, a second agent on the same device used **Frida Stalker** against
`com.android.settings` and `com.android.systemui`, including a `libc` `malloc` export trigger that
entered and left `follow` hundreds of times. Result:
- `Stalker.follow` on the main thread reported `blocks=0`, `blk=0`, `calls=0` — **the follow installed
and no events arrived**.
- `systemui` then died with `SIGSEGV`, the faulting frame sitting in gum's anonymous code cache and
the backtrace continuing into `libart` at `art::Reference_getReferent`; the process restarted
(`18030 → 26932`).
- Device load average reached **34.11** on an eight-core phone.
**Reading:** two conclusions, kept separate on purpose. First, on this ROM + `frida-server` 16.7.19
combination a Stalker follow produced no events — that is a *boundary* for
`references/native-dbi-and-deobfuscation.md`, not a contradiction of Stalker's documentation. Second,
high-frequency `follow`/`unfollow` on every `malloc` entry crashed a system process — a measured
failure mode for the pattern, and one worth stating before anyone puts it in a shipped script.
The third conclusion is about method rather than about Stalker: **instrumentation on a shared device
is a shared variable.** The concurrent load damaged the *other* experiment's validity — the dumped-run
above could not be attributed until it was repeated with the device idle, and the repeat is what
turned "maybe the load broke it" into a reproducible protocol refusal. R2 (one variable at a time)
applies across agents, not only within one script.
## Conclusions, graded
| Claim | Label |
|---|---|
| The sample runs healthy with no instrumentation on the device | observed |
| The sample's pid drifts repeatedly during an experiment, roughly every 10-20 s | observed — **cause not instrumentation**: reproduced with `frida-server` stopped, under 300-750 MB free memory (§2) |
| `frida-dexdump` spawn refuses (jailed / argv / timeout) and attach fails with `script has been destroyed`, reproduced twice with the parameter mistakes fixed | observed |
| The attach failure is deliberate armour rather than a host-tool bug | **unverified** — memory pressure leaving the process an unstable target is a competing explanation this pass did **not** rule out; the single-variable repeat eliminated *concurrent* load, not *baseline* load |
| The shipped shell `classes.dex` is 8.9 MB and defines only 4 classes / 29 methods with code | observed (recorded in `EXTENSION-unpacking.md`) |
| Recovering the real bodies needs a route other than a `frida` memory dump on this target | inferred |
| Stalker follow delivers no events on this ROM with this frida-server version | observed, single sample, `unverified` as a general rule |
| High-frequency follow/unfollow on `malloc` can crash a system process | observed, once |
| No app on this device is currently hidden from root (`magisk --denylist status` enforced, list empty; Zygisk-Assistant installed but nothing configured) | observed (recorded in `EXTENSION-lsposed.md`) — offered here because a hardened sample's sensitivity to a rooted device has a configuration, not just a cause |
## What this pass did **not** establish
- **Not** a successful end-to-end unpack of the sample. The memory-dump route was refused, and the
pass stopped at the skill's own two-strike rule instead of escalating into an evasion arms race.
- **Not** evidence that the sample is un-unpackable. The routes that do not use `frida` on the device
— root-side `/proc/<pid>/mem` extraction, the module route in `references/lsposed-and-modules.md`,
and the static route already recorded for the Flutter payload — were not exhausted here.
- **Not** a general claim about 360-jiagu, Flutter, or Android 11. Everything above is one device,
one ROM, one server version, one sample.

431
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-emulation-rpc.md Normal file
View File

@@ -0,0 +1,431 @@
# Extension verification — offline emulation (unidbg) and Frida RPC
Covers the 2026-09 extension pass that added `references/emulation-and-rpc.md`,
`scripts/frida_rpc_serve.py` and `scripts/rpc_template.js`. Strength labels as defined in this
directory's README: **observed** (exact command + output recorded here), **inferred** (follows from
observed facts, step not executed), **unverified** (assumed or externally reported, not independently
confirmed here).
Reference environment for this pass: Windows host, Python 3.14.0, host `frida` 16.7.19 +
`frida-python` 16.7.19 (frida-tools 13.7.1), `adb` 37.0.1, JDK 17.0.4.1 (with `JAVA_HOME`
**unset** in the shell — see §5). Device `<DEVICE>`: Android 11 / API 30, arm64-v8a, rooted
(Magisk), on-device frida-server 16.7.19. Workbench artifacts live in `tools/_work/t3-rpc/` (not part
of the skill).
**Device-exclusivity constraint during this pass**: another task was unpacking the hardened Flutter
sample on the same device. This pass never attached to, spawned or force-stopped that sample's
process, and never killed or restarted the device-side frida-server. All RPC work targeted unrelated
processes (system apps and third-party apps). De-identification: the sample's package label, real
package name and md5 are elided in this document as `com.example.app` / `<label>`; command shapes,
counts, class names and timings are verbatim.
---
## 0. Summary of what this pass established
| Claim | Label |
|---|---|
| `droidasc` 0.1.1.post1 and `ddc` 0.1.8 both run on this host; both read the hardened sample's shell dex | observed |
| Both tools see only the 4 shell classes in the APK's dex; business classes are absent (shell dex only) | observed |
| `frida_rpc_serve.py` `--help`, all three modes, reconnect, spawn and attach-by-pid work against a live device | observed |
| Attach **by package name** fails on this ROM while attach **by pid** succeeds for the same process | observed |
| The device-side server may be (re)started on a non-default port; `--remote HOST:PORT` recovers, device-serial lookup does not | observed |
| unidbg 0.9.10-SNAPSHOT builds and boots an Android emulator on this host after a pom fix; one bundled test suite passes | observed |
| Any target `.so` actually emulated under unidbg | **not done** (unverified) |
| frida 17 compatibility of `rpc_template.js` export lookup | unverified (host has 16.7.19 only) |
---
## 1. ASC (`droidasc`) — observed
### 1.1 Install and version
```
$ python -m pip show droidasc
Name: droidasc
Version: 0.1.1.post1
Summary: ASC is a super FAST python Android Decompiler for Agents/Mobile Researchers
License-Expression: Apache-2.0
Location: C:\Python314\Lib\site-packages
Requires: androguard
```
Install line used (recorded after the fact; the package was already present on this host):
`python -m pip install droidasc`. Distribution metadata declares only `androguard` as a dependency —
**no JVM is required** (unlike the `asc` successor tooling in the upstream MG1937/ASC repository,
which is a separate project). The console script is named **`droidasc`**, not `asc`: there is no
`asc.exe` in `C:\Python314\Scripts`, and `asc --version` fails with
`The term 'asc' is not recognized...`. `python -m droidasc` is the equivalent entry point.
### 1.2 Subcommands
```
$ python -m droidasc --help
usage: python.exe -m droidasc [-h]
{getclass,listclass,getmanifest,findrefs} ...
ASC tooling entry.
positional arguments:
{getclass,listclass,getmanifest,findrefs}
getclass Locate the target class in APK, extract one DEX in memory, then decompile.
listclass List classes across all DEX entries; use --prefix to filter by package/class prefix.
getmanifest Decode AndroidManifest.xml from APK and print it as XML.
findrefs Find code references for string/type/method/field across all DEX entries in APK.
examples:
droidasc app.apk --gui
droidasc getclass app.apk Lcom/poc/Main; -o Main.java
droidasc listclass app.apk --prefix com.poc
droidasc getmanifest app.apk -o AndroidManifest.xml
droidasc findrefs app.apk string token -o string_refs.txt
```
`--gui` and the bundled `asc_client` web UI were **not** exercised (unverified, headless pass).
### 1.3 Measured against the sample
```
$ droidasc getmanifest <sample>.apk -o asc-manifest-verify.xml # 0.44 s, 32,345 B
<manifest ... package="com.example.app" platformBuildVersionCode="34" ... android:versionCode="1143"
android:versionName="0.9.0" android:compileSdkVersion="34" ...>
<uses-sdk android:minSdkVersion="19" android:targetSdkVersion="28"/>
<uses-permission android:name="android.permission.INTERNET"/>
... (permissions list follows; manifest decodes cleanly)
$ droidasc findrefs <sample>.apk string jiagu -o asc-findrefs-jiagu.txt # 0.42 s
classes.dex | Lcom/stub/StubApp;-><clinit> | matched=(libjiagu)
classes.dex | Lcom/stub/StubApp;->attachBaseContext | matched=(/.jiagu; jiagu; jiagu_x86)
$ droidasc getclass <sample>.apk com.stub.StubApp -o asc-StubApp-verify.java # 0.33 s, 22,802 B
package com.stub;
public final class StubApp extends android.app.Application { ... }
$ droidasc listclass <sample>.apk --prefix com.stub # 0.23 s
Lcom/stub/StubApp;
$ droidasc listclass <sample>.apk -o asc-listclass-all.txt # 0.18 s, 4 lines
Lcom/stub/StubApp;
Lcom/tianyu/util/Configuration;
Lcom/tianyu/util/DtcLoader;
Lcom/tianyu/util/a;
```
`findrefs` reports the shell's own strings (`libjiagu`, `jiagu_x86`, `/.jiagu`) — i.e. the packer
signature is visible without unpacking, but nothing of the application is.
### 1.4 Limits hit
```
$ droidasc getclass <sample>.apk com.example.app.MainActivity -o asc-mainactivity.java
Error: Class Lcom/example/app/MainActivity; not found in APK.
exit = 1
```
**This is the limit that matters for this sample class:** the full class listing is 4 classes, so on
a packed APK ASC is a manifest + shell-inspector, not an application decompiler. The application code
lives in that shell's payload (`assets/`, the native unpacker and the AOT/Flutter layer), which ASC
does not touch. ASC still earns its place in the flow: `getmanifest` in under half a second is the
fastest manifest pass measured in this repository's tool set, and `findrefs` on shell class names is
how the packer was identified here.
Observed behaviours worth knowing: `droidasc` prints per-command timing nowhere, writes to `-o` only,
and returns a clean `Error: ...` line plus exit 1 when a class is missing.
---
## 2. ddc — observed
### 2.1 Version and acquisition
```
$ bin/ddc.exe -V
ddc 0.1.8 — DEX → Java decompiler
https://github.com/ejfkdev/ddc
```
`ddc` is a Rust binary (upstream: [ejfkdev/ddc](https://github.com/ejfkdev/ddc)); the Windows build
used here is `tools/_work/t3-rpc/bin/ddc.exe`, 1,028,096 B, **not on `PATH`** and with no version
resource in the PE header (`VersionInfo` empty), so `ddc -V` is the only way to learn its version.
No JVM, no Python. Subcommands advertised by `ddc help`: `info`, `listclasses`, `manifest`,
`mainactivity`, `res`, `strings`, `findrefs`, `callers`, `members`, `hierarchy`, `largest`, `disasm`,
`getclass`, `getmethod`, plus a full-decompile mode (`ddc <INPUT>... [OUTPUT]`).
### 2.2 Measured against the sample
```
$ bin/ddc.exe info <sample>.apk # 0.24 s
label <label>
package com.example.app
version 0.9.0 (1143)
application com.stub.StubApp
launcher com.example.app.MainActivity
sdk 19–28
size 36.9 MB (38688120 bytes)
md5 1a037a045205184d4e8272092997334f
image dex classes methods fields strings
<sample>!classes.dex 035 4 266 23 481
total: 1 image(s), 4 classes
$ bin/ddc.exe findrefs <sample>.apk string jiagu # 0.13 s
dex kind class method refs
classes.dex const-string com/stub/StubApp <clinit>()V "libjiagu"
classes.dex const-string com/stub/StubApp attachBaseContext(...)V "jiagu_x86"; "jiagu"; "/.jiagu"
$ bin/ddc.exe strings <sample>.apk -f jiagu --with-locations # 0.14 s
dex string used-by
classes.dex "/.jiagu" com/stub/StubApp attachBaseContext(Landroid/content/Context;)V
classes.dex "jiagu" com/stub/StubApp attachBaseContext(Landroid/content/Context;)V
classes.dex "jiagu_x86" com/stub/StubApp attachBaseContext(Landroid/content/Context;)V
classes.dex "libjiagu" com/stub/StubApp <clinit>()V
```
`ddc info` reaches the same conclusion as ASC by a different route (class/method/string counts per
dex image) and adds the resources-derived label and the launcher verification; `findrefs` output is
tabular and one line per (method, string) pair, which is directly pasteable into a report.
### 2.3 Limits hit
```
$ bin/ddc.exe getclass <sample>.apk com.example.app.MainActivity
ddc: class com.example.app.MainActivity not found in the selected image(s)
(try `ddc listclasses <input> <pattern>`)
exit = 2
```
Exit codes measured on this build: `info` → 0, `-V` → 0, `getclass` with a missing class → **2**.
The missing-class path also reprints the full help text on stdout, so a wrapper that only checks
"did anything print" will misread it as success — key off the exit code. As with ASC, the packed APK
exposes only the shell dex; `ddc` cannot see application classes either, which is a property of the
artifact, not of the tool.
---
## 3. Prior-attempt outputs under test — and the defects found
`skills/apk-reverse/references/emulation-and-rpc.md`, `scripts/frida_rpc_serve.py` and
`scripts/rpc_template.js` were already on disk from an interrupted attempt of this task. This pass
read all three, ran the script, and fixed four defects. `rpc_template.js` needed no change.
| # | Defect (observed) | Fix |
|---|---|---|
| 1 | `RpcBridge._teardown()` never detached the session and used `obj.off(...) if obj is self.session else None` / `obj.unload() if hasattr(obj, "unload")` — a script object has `unload()`, a session has `detach()`, so the expression released only the script. Every `reload()`/reconnect cycle leaked one attached session on the device. | Rewritten to unload the script, remove the `detached` handler and `session.detach()` explicitly. |
| 2 | `list_exports()` called `Script.list_exports()`, which frida 16.7.19 answers with `DeprecationWarning: Script.list_exports will become asynchronous in the future, use the explicit Script.list_exports_sync instead` (printed on every `list` command). | Tries `list_exports_sync` first, then the old name, then local introspection. |
| 3 | `--mode call` raised out of `main()`: an unknown export produced `frida`'s own `unable to find method 'x'` and a raw Python traceback; a bad `--args` value dumped a `JSONDecodeError` traceback. Scripted callers got a stack trace instead of a result. | `run_call_mode()` now returns one JSON object (`{"ok": false, "error": ...}`) and exit code 1 for both cases; `call()` consults the advertised export list first so the message is `no such rpc export: nope (available: add ping ...)`. |
| 4 | `frida.TimedOutError` was absent from the retry whitelist. It is **not** a `TransportError` subclass (`frida.TimedOutError.__mro__` → `(TimedOutError, Exception, BaseException, object)`), so a device that timed out during attach produced a traceback in an otherwise recoverable situation — reproduced live (§4.4). | Retryable types are now collected by name into `RETRYABLE` (TransportError, NotSupportedError, ServerNotRunningError, ProcessNotFoundError, InvalidOperationError, TimedOutError) and used by both `ensure()` and the initial connect in `main()`. |
Post-fix evidence:
```
$ python skills/apk-reverse/scripts/frida_rpc_serve.py --remote 127.0.0.1:27043 --pid 19938 \
--script skills/apk-reverse/scripts/rpc_template.js --mode call --export add --args "[40, 2]"
{"ok": true, "export": "add", "result": 42} exit = 0
$ ... --mode call --export nope
{"ok": false, "export": "nope", "error": "no such rpc export: nope
(available: add ping getpackagename callnative callnativeaddr)"} exit = 1
$ ... --mode call --export add --args "not-json"
{"ok": false, "error": "--args is not valid JSON: Expecting value: line 1 column 1 (char 0)"}
exit = 1
```
Known cosmetic behaviour left as is: in `--mode call` the bridge connects before `--args` is parsed,
so a bad-args invocation still performs a device attach.
## 4. Frida RPC against the live device — observed
### 4.1 Attach path: by name fails, by pid works
Attach **by package name** failed for every package tried, with the process demonstrably alive in
both `frida-ps -Uai` and `ps -A`:
```
$ python ...frida_rpc_serve.py --device-serial <DEVICE> --package com.android.settings ...
[bridge] connect failed (unable to find process with name 'com.android.settings'); retry 1 in 1.0s
... (5 attempts) ...
initial connect failed: could not (re)connect after 5 attempts
$ python ... --package io.github.vvb2060.magisk ...
[bridge] connect failed (unable to find process with name 'io.github.vvb2060.magisk'); retry 1 in 0.5s
initial connect failed: could not (re)connect after 1 attempts
```
The same processes attach normally by pid:
```
$ python ... --pid 19938 --mode call --export getpackagename
{"ok": true, "export": "getpackagename", "result": "com.android.settings"}
$ python ... --pid 21493 --mode call --export getpackagename
{"ok": true, "export": "getpackagename", "result": "com.android.email"}
```
So the gap is in the **name→pid lookup** the server serves, not in injection: `--pid` definitely
does bypass it. `references/emulation-and-rpc.md` had claimed the opposite ("`--pid` does not bypass
a gap that hides the process from the server's own table"); that row was corrected in this pass, and
the corrected wording is what the reference now carries.
### 4.2 The transport itself
| # | Command (pid 19938 = `com.android.settings`, unrelated to the sample) | Result |
|---|---|---|
| 1 | `--mode call --export add --args "[40, 2]"` | `{"ok": true, "result": 42}` |
| 2 | `--mode call --export ping` | `{"ok": true, "result": "pong"}` |
| 3 | `--mode call --export getpackagename` | `{"ok": true, "result": "com.android.settings"}` |
| 4 | `--mode call --export callnative --args '["libc.so","getpid","int",[],[]]'` | `{"ok": true, "result": 19938}` (matches the pid — `NativeFunction` wrapping works) |
| 5 | `--mode repl`, feeding `list` / `reload` / `list` / `add [40, 2]` / `nope` | `add ping getpackagename callnative callnativeaddr`; `reloaded`; same list; `42`; `error: no such rpc export: nope (available: ...)` |
| 6 | `--mode http`, `GET /exports` | `{"ok":true,"exports":[...5 names...]}` |
| 7 | `--mode http`, `POST /call {"export":"add","args":[40,2]}` | `{"ok":true,"result":42}` |
| 8 | `--mode http`, `POST /call {"export":"getpackagename","args":[]}` | `{"ok":true,"result":"com.android.settings"}` |
| 9 | `--mode http`, `POST /call` with an unknown export | HTTP 502 + `{"ok":false,"error":"no such rpc export: nope (available: ...)"}` |
| 10 | `--mode http`, `POST /reload` then `POST /call add [1,2]` | `{"ok":true}` then `{"ok":true,"result":3}` — the re-attach path works, and no `DeprecationWarning` is printed any more |
| 11 | `--spawn --package io.github.vvb2060.magisk --mode call --export getpackagename` | `{"ok": true, "result": "io.github.vvb2060.magisk"}` (spawn → script load → resume → RPC) |
Every script load also delivered the template's own liveness message:
`[script] {"ev": "RPC-READY", "exports": ["add","ping","getpackagename","callnative","callnativeaddr"]}`.
That line is the cheapest health check for a session and is worth grepping for in automation.
### 4.3 The server's port moved under us
Mid-pass, `--device-serial <DEVICE>` began failing on attach while `frida-ps -U` kept working:
```
$ python ... --device-serial <DEVICE> --pid 19938 ...
[bridge] connect failed (unable to connect to remote frida-server: closed); retry 1 in 1.0s
... (3 attempts) ...
$ python -c "import frida;d=frida.get_device('<DEVICE>');d.attach(19938)"
attach failed: ServerNotRunningError unable to connect to remote frida-server: closed
$ adb -s <DEVICE> shell 'su -c "ps -A"' | Select-String 'svc16|frida'
root 1876 1 2227684 55288 do_sys_poll S .svc16 # only this instance alive
$ adb -s <DEVICE> shell 'su -c "netstat -ltnp"' | Select-String 2704
tcp 0 0 0.0.0.0:27043 0.0.0.0:* LISTEN 1876/.svc16 # nothing on 27042
```
The visible `frida-server` process (pid 23405) was gone; a disguised copy (`.svc16`, pid 1876) was
listening on **27043 only**. `frida-ps -U` still worked because frida's USB transport probes a range
of ports, while the scripted device-serial path landed on a dead default port. Recovery, with the
forward already present (`adb forward --list` shows `tcp:27043 tcp:27043`):
```
$ python ... --remote 127.0.0.1:27043 --pid 19938 --mode call --export getpackagename
{"ok": true, "result": "com.android.settings"}
```
This pass neither started nor killed any server instance; it only observed that the instance changed
and switched to the explicit `--remote` form. That form is now the recommended one in the reference's
failure table.
### 4.4 A third-party process that refuses the handshake
```
$ python ... --remote 127.0.0.1:27043 --pid 19984 --mode call --export getpackagename
frida.TimedOutError: unexpectedly timed out while waiting for signal from process with PID 19984
```
An ordinary third-party app process (`com.xunmeng.pinduoduo`, listed by `frida-ps -Uai`) simply did
not answer the agent handshake within frida's timeout. Pre-fix this surfaced as a traceback; post-fix
the bridge treats it as retryable and reports `initial connect failed: ...` with exit 2. This is an
app-side property, not a bridge property — for connectivity probes prefer a system or stable app
process.
### 4.5 Device state after the pass
`frida-ps -U` and `frida-ps -Uai` both still answer; the sample's process was never touched; the
device-side server was never killed or restarted by this pass.
---
## 5. unidbg — observed (build/boot) and unverified (target emulation)
### 5.1 Build
```
$ git clone --depth 1 https://github.com/zhkl0228/unidbg.git # version 0.9.10-SNAPSHOT
$ .\mvnw.cmd -B -pl unidbg-android -am -DskipTests compile
Error: JAVA_HOME not found in your environment. # JAVA_HOME was unset on this host
...
[ERROR] Failed to execute goal ...maven-compiler-plugin:3.3:compile on project unidbg-api:
[ERROR] .../AbstractARMDebugger.java:[79,43] reference to Module is ambiguous
[ERROR] both com.github.unidbg.Module and java.lang.Module match
```
The shipped pom pins `<source>8</source><target>8</target>` with `maven-compiler-plugin` 3.3. On
JDK 9+ that pair does **not** select the JDK 8 API, so `java.lang.Module` stays visible and every
unqualified `Module` in `AbstractARMDebugger` becomes ambiguous — the build dies in `unidbg-api`.
Two local work-copy changes fix it (recorded in `tools/_work/t3-rpc/unidbg/pom.xml`, not part of the
skill and not committed):
```
maven-compiler-plugin 3.3 -> 3.8.1 # 3.13 is rejected: the bundled wrapper is Maven 3.5.4,
# and 3.13 requires Maven >= 3.6.3
<source>8</source><target>8</target> -> <release>8</release> # hides java.lang.Module
```
Result with `JAVA_HOME=C:\Program Files\Java\jdk-17.0.4.1`:
```
$ .\mvnw.cmd -B -pl unidbg-android -am -DskipTests compile
[INFO] unidbg-api ......... SUCCESS [ 40.838 s]
[INFO] unidbg-android ..... SUCCESS [ 4.941 s]
[INFO] BUILD SUCCESS (total 48.4 s, after a one-off dependency download)
```
### 5.2 Boot check
The pom sets `<maven.test.skip>true</maven.test.skip>`, so `-Dmaven.test.skip=false` is required to
run anything:
```
$ .\mvnw.cmd -B -pl unidbg-android -am -Dmaven.test.skip=false \
-Dtest=MemoryTrackerTest -DfailIfNoTests=false test
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
Backend: DynarmicBackend64
=== Memory Leak Report === ... Address: 0x12000000, Size: 4096 ...
[INFO] BUILD SUCCESS
```
This proves the toolchain end to end at the *environment* level: emulator construction, memory
allocator, backend selection (Dynarmic, with Hypervisor and Unicorn2 factories registered) and
Android 11 (`AndroidResolver(23)`-style) library resolution all execute on this host.
### 5.3 What was **not** done — unverified
* Loading and calling **any** target `.so`, including this sample's `libjiagu*.so` or `libapp.so`.
No `createDalvikVM`, `loadLibrary`, `callJNI_OnLoad` or `AbstractJni` stub was exercised.
* Consequently every per-library statement in `references/emulation-and-rpc.md` §Part A — which JNI
callbacks a hardened library demands, how many stub rounds it costs, which `/proc` or signature
checks it runs — remains **inferred** from the library's shape and community practice. The
reference's "Measured here" section was rewritten in this pass to say exactly that.
* Note for anyone porting an older tutorial: `emulator.createCode()` / the `Assembly` helper class
do **not** exist in 0.9.10-SNAPSHOT (no `Assembly*.java` under the tree, no `createCode` symbol in
any module). Recipes that assemble raw ARM machine code through that API need the current
backend/JNI entry points instead.
* `unidbg-android`'s bundled demos (`TTEncrypt`, `SignUtil`, `QDReaderJni`, …) each expect their own
`.so`/APK to be placed first; none were run, so the demo names in the reference are pointers, not
verified commands.
---
## 6. Consolidated list of things that did **not** work (for the pass summary)
1. Attach by package name on this ROM (both `com.android.settings` and `io.github.vvb2060.magisk`
failed while listed by `frida-ps -Uai`). Workaround: pid, from `frida-ps -U` / `pidof`.
2. Device-serial attach once the device's server came back on 27043 — `ServerNotRunningError:
unable to connect to remote frida-server: closed`. Workaround: `adb forward` + `--remote`.
3. Attach by pid to `com.xunmeng.pinduoduo` — `frida.TimedOutError`; the process never answered the
handshake.
4. `asc` as a command name (the console script is `droidasc`), and `mvnw` without `JAVA_HOME`.
5. unidbg out of the box on JDK 17 (`Module` ambiguity) and unidbg tests out of the box
(`maven.test.skip=true`).
6. Application-class decompilation of a packed APK by either ASC or ddc (nothing to fix — the dex
simply is not there; this is the case that justifies the whole emulation/RPC route).
## 7. Artifacts produced by this pass
`tools/_work/t3-rpc/` (not committed): `asc-manifest-verify.xml`, `asc-StubApp-verify.java`,
`asc-listclass-all.txt`, `asc-listclass-stub.log`, `asc-getmanifest.log`, `asc-findrefs.log`,
`asc-findrefs-jiagu.txt`, `asc-getclass.log`, `ddc-info-verify.txt`, `ddc-findrefs-jiagu.txt`,
`ddc-strings-jiagu.txt`, `rpc-*.log` (one per command in §4), `bin/ddc.exe`, `unidbg/` (patched work
copy).
Files changed in the repository by this pass: `references/emulation-and-rpc.md` (corrected failure
table, corrected unidbg build recipe, corrected "Measured here" labelling, and a stale section
reference repointed to `SKILL.md` §Stop conditions) and `scripts/frida_rpc_serve.py` (the four fixes
in §3).

539
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-extraction-shell-bench.md Normal file
View File

@@ -0,0 +1,539 @@
# Extension verification — extraction shells and the Dex-VMP boundary (bench rows B3 / B4)
Covers benchmark rows **B3** (extraction-shell skeleton detection) and **B4** (Dex-VMP
boundary: refuse-to-repack and downgrade the deliverable). Strength labels are this
directory's own three tiers: **observed** (an exact command and its output are quoted
here), **inferred** (follows from observed facts; the step was not executed),
**unverified** (assumed or externally reported, not reproduced here).
Everything below was produced on this repository's Windows 11 host with the rootless
PC toolchain in `tools/_work/ENV.md`; the device results in §4 name a
`<DEVICE>` and use `<PKG>` placeholders for the target package.
## 0. Headline
| Question | Answer | Strength |
|---|---|---|
| Does `stub%` separate a skeleton copy from a genuine dex? | **Only when the shell writes bare `return-void`.** Three other measured emptying shapes score 0.0 %–2.3 %, at or below the *untouched* control (1.9 %) | observed |
| Was the pre-existing `trivial_ratio` detector correct? | **No — two defects**, one of which inverted the ranking and named a fully-nop-wiped skeleton the "most likely original" | observed |
| Is `repos/CyReverse`'s shipped "extraction shell" asset usable as a shell fixture? | **No.** The pair differs by six bytes and one body; neither is a shell | observed |
| Is `DexPatcher/assets/ezAndroid.apk` a Dex VMP? | **No.** It is a JNI-sinking target: 7 `native` declarations, an exported `Java_*` for each, 34,566 clean dalvik instruction decodes, zero structural errors | observed |
| Can a static dex probe decide VMP vs not? | **It can decide "not dex-VMP"** when the bodies decode cleanly and each native declaration has an export. It **cannot** decide whether the native island performs runtime hardening | observed (negative half) / unverified (positive half) |
| Does `dex_mem_scan.py` find and cut a dex out of a blob? | **Yes** — exact offset, no false hits in 2 MiB of noise, image accepted by the judge | observed |
| Does it reach a dex in a live process's anonymous mapping? | **Not measured here.** The fixture is built and the commands staged; the device was held by another row throughout | unverified |
One claim in this pass was **retracted**: §3.4 records a Dex-VMP verdict drawn from a
desynchronised decoder and the measurement that killed it.
---
## 1. B3 — the shipped CyReverse "extraction shell" assets are a 6-byte difference, not a shell
Source: `repos/CyReverse/app/src/main/assets/{classes3.dex, classes3_extracted.dex}`
(12,536 B each).
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3
== dex dump validation: 2 file(s), 2 parse, 0 rejected ==
name size sha256[:12] ver cksum sig class noco code stub%
cyrev-extracted.dex 12536 c2d7345e955a 037 ok BAD 4 0 20 10.0%
cyrev-skeleton.dex 12536 ba9d5b120a3a 037 ok ok 4 0 20 10.0%
```
A byte diff of the pair (`tools/_work/bench/unpack/b3/analyze_pair.py`) locates every
difference:
```
$ python tools/_work/bench/unpack/b3/analyze_pair.py
len A=12536 len B=12536
first diff offset = 8 (0x8)
diff byte count = 6
diff ranges:
0x8 .. 0x8 (1 B) <- dex checksum (header offset 8)
0xa .. 0xb (2 B) <- dex signature, first half (offset 12 is 4 B wide)
0x121c .. 0x121c (1 B) <- INSNS of PluginClass->getString, units 0 and 2
0x121e .. 0x121e (1 B)
0x1220 .. 0x1220 (1 B)
```
Per-method census of the pair shows the "extracted" copy is the *degraded* one:
```
== cyrev-skeleton.dex == Lcom/cyrus/example/plugin/PluginClass;->getString insns=3 units=001a 0087 0011
== cyrev-extracted.dex == Lcom/cyrus/example/plugin/PluginClass;->getString insns=3 units=0000 0000 0000
```
`const-string v0, "…"; return-object v0` was overwritten with three `0x0000` units in
exactly one of the twenty bodies, and the header was re-fixed. The shipped
`classes3_extracted.dex` is **not** an extraction shell and **not** a recovered dex:
it is a single-body demonstration of body clearing, with 19 of 20 bodies identical to
the skeleton. Neither file can test extraction-shell routing — 10.0 % `stub%` on both
sides is a coincidence of the same body set, not a signal.
**Consequence for the benchmark row:** B3 cannot be run against this asset pair. The
skeleton measurement below had to be built from a real application dex instead.
## 2. B3 — measured `stub%` across every emptying shape, with a zero-change control
Fixture builder: `tools/_work/bench/unpack/b3/make_skeleton.py` derives each variant
from a real 982,480-byte application dex (`src-r2pay.dex`, extracted from
`tools/_work/bench/apks/r2pay-v1.0.apk`, sha256 `c89ff2364879…`). Every variant is
**byte-length preserving**: only the `insns` array of a `code_item` is rewritten and
the tail is nop-filled, so offsets, class table and string table are untouched and the
header is re-fixed with `dexutil.fix_dex_header` (signature first, checksum last).
```
$ python tools/_work/bench/unpack/b3/make_skeleton.py src-r2pay.dex skel
source: src-r2pay.dex
ctrl.dex fraction=0.00 mode=return emptied=0/5061
skel-25.dex fraction=0.25 mode=return emptied=1265/5061
skel-50.dex fraction=0.50 mode=return emptied=2531/5061
skel-75.dex fraction=0.75 mode=return emptied=3796/5061
skel-all.dex fraction=1.00 mode=return emptied=5061/5061
pruned.dex fraction=1.00 mode=pruned emptied=5061/5061
nopfilled.dex fraction=1.00 mode=nop emptied=5061/5061
allstub.dex fraction=1.00 mode=allstub emptied=5061/5061
throwstub.dex fraction=1.00 mode=throw emptied=4134/5061
```
Verdicts from `dex_dump_validate.py` over the whole set — the decisive matrix:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3/skelall
name size sha256[:12] ver cksum sig class noco code stub% erased% emptied%
allstub.dex 982480 c8d7ca8fb163 035 ok ok 647 250 5061 100.0% 0.0% 100.0%
ctrl.dex 982480 1b3a97896b40 035 ok ok 647 250 5061 1.9% 0.0% 1.9%
nopfilled.dex 982480 de92dcea4eaf 035 ok ok 647 250 5061 0.0% 100.0% 100.0%
pruned.dex 982480 58d7fa3dce48 035 ok ok 647 250 5061 93.2% 0.0% 93.2%
skel-25.dex 982480 63346d9b41f5 035 ok ok 647 250 5061 1.7% 0.0% 1.7%
skel-50.dex 982480 d84601b7aa47 035 ok ok 647 250 5061 1.7% 0.0% 1.7%
skel-75.dex 982480 64114285656d 035 ok ok 647 250 5061 2.1% 0.0% 2.1%
skel-all.dex 982480 6345daef0a3f 035 ok ok 647 250 5061 2.3% 0.0% 2.3%
throwstub.dex 982480 cab8fd51a743 035 ok ok 647 250 5061 1.9% 0.0% 1.9%
```
Read the `mode=` column against `stub%`:
| Emptying shape (what the shell leaves behind) | `stub%` | Separated from `ctrl` (1.9 %)? |
|---|---|---|
| bare `return-void` in every body (`allstub`) | **100.0 %** | yes, by 98 points |
| body truncated to `const/4 v0,#0; return v0` + nop fill (`pruned`) | 93.2 % | yes |
| nop fill, no return at all (`nopfilled`) | **0.0 %** | **no — scores *below* the untouched control** |
| `new RuntimeException; throw` stub (`throwstub`) | 1.9 % | **no — indistinguishable from control** |
| 25 % / 50 % / 75 % of bodies emptied with a legal minimal body | 1.7 / 1.7 / 2.1 % | **no** |
Three conclusions, all observed:
1. **The threshold is not a number, it is a shape.** `stub%` is bimodal — it is either
≈ control (blind) or ≈ 100 % (sighted). There is no intermediate band where a
threshold could sit, so the "tens of percent" calibration the reference file
describes has no measured support.
2. **Partial extraction is invisible.** `skel-75.dex` — three quarters of all bodies
emptied — reports 2.1 % against a 1.9 % control, a 0.2-point difference inside the
sampling noise of what a real app's own trivial bodies contribute.
3. **The two shapes that defeat it are the two that real shells prefer.**
`throw new RuntimeException` stubs are the classic 360/legu family shape, and a
nop-wiped body is what a shell writes when it clears the slot before decrypting.
The false-negative on `nopfilled` was a **defect, not a boundary**, and is fixed in §3.
The `throwstub` blind spot is a **boundary** and is documented as one.
## 3. B3 — defects found in `dex_dump_validate.py`, and the fix
### 3.1 Defect 1: a fully nop-filled body was classified `real`
`classify_body` skipped every `0x0000` unit and then fell through to `return "real"`
when nothing else was found. Measured consequence: `nopfilled.dex` — **all 5,061
bodies wiped** — reported `stub% = 0.0`, and the ranking named it the most likely
original:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py …/skel/{nopfilled,ctrl}.dex
ranking (most likely original first):
1. nopfilled.dex stub%=0.0 cksum=ok sig=ok classes=647
…
verdict: nopfilled.dex is the most likely original (1 copy(ies) in this set)
```
An agent routing on that line patches a wiped skeleton. That is the failure the file
exists to prevent, so the defect is on the wrong side of "the tool still produces
output".
**Fix.** `classify_body` gained two classes and a wider body scan:
- `erased` — `insns_size > 0` with every unit `0x0000`. A real compiler never emits a
whole body of nops, so this is skeleton evidence.
- `minimal` — a body truncated to `const/4 vR,#0; return vR` (legal, and the shape the
detector is blind to by design). Reported separately so the blind spot is visible
rather than hidden inside `real`.
- The early-exit `meaningful > 1 → real` became `meaningful > 2`, so the two-unit
minimal body reaches its own verdict instead of being mislabelled.
`trivial_ratio` keeps its original definition so previously recorded numbers stay
comparable; `emptied_ratio = (stub + erased) / with_code` is the new skeleton signal,
and `rank_key` sorts on it **low-first** with `bodies_erased` as a tiebreak.
### 3.2 Defect 2 (retracted in-flight): `0xNN0E` is not malformed
An intermediate version of the fix classified a lone `0x020E` unit as `malformed`,
reading a non-zero high byte on a `0x0E` opcode as an illegal `return-void`. The
control dump proved otherwise — 82 of the untouched dex's own bodies are exactly that
shape — and `const/4 v0, #0; return v0` is a legal minimal non-void body. The class was
renamed `minimal`, its comment corrected, and the fixture extended with `pruned.dex`
to measure it deliberately (93.2 %, `stub%` sighted). Recorded here because the
mislabel would have taught the next reader that ordinary short methods are damage.
### 3.3 Post-fix behaviour, and no regression on the recorded fixture
`nopfilled.dex` is now ranked last and flagged:
```
8. nopfilled.dex stub%=0.0 erased%=100.0 cksum=ok sig=ok classes=647 <-- SKELETON: emptied%=100, extraction-shell shape
9. …
verdict: skel-25.dex is the most likely original (1 copy(ies) in this set)
```
The pre-existing acceptance fixture from `EXTENSION-unpacking.md` §2 still produces
its recorded verdict, with only the two new columns appended:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/t2-unpack --find 'Lcom/stub/'
== dex dump validation: 7 file(s), 4 parse, 3 rejected ==
01_original_shell.dex 8923752 bcd1c0488e86 035 ok BAD 4 160 29 3.4% 0.0% 3.4%
02_duplicate.dex 8923752 bcd1c0488e86 035 ok BAD 4 160 29 3.4% 0.0% 3.4%
03_broken_magic.dex 8923752 33eee53b1317 ? REJECTED: magic=b'XXXX'
04_tampered_body.dex 8923752 b89bc8cc73d9 035 BAD BAD 4 160 29 3.4% 0.0% 3.4%
05_truncated.dex 65536 e6687390a663 035 REJECTED: file_size=8923752 actual=65536
06_extracted_skeleton.dex 8923752 85e56fed1d56 035 ok ok 4 160 29 100.0% 0.0% 100.0%
07_tiny_garbage.dex 16 be45cb2605bf ? REJECTED: too small (16 B) to carry a dex header
verdict: 01_original_shell.dex is the most likely original (2 copy(ies) in this set)
```
Same winner, same three rejections, same dedupe group. The change is additive on that
fixture and corrective on the new one.
### 3.3b The ranking was still pointing at a modified image (found by independent review)
The first fix above added the `emptied_ratio` signal — and then **kept ranking on a
metric this same pass had just proved insensitive to skeletons**. Reproduced by
leading the review on the variant set:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3/skel
ranking (most likely original first):
1. skel-25.dex stub%=1.7 erased%=0.0 cksum=ok sig=ok classes=647
2. skel-50.dex stub%=1.7 erased%=0.0 cksum=ok sig=ok classes=647
3. ctrl.dex stub%=1.9 erased%=0.0 cksum=ok sig=ok classes=647
…
verdict: skel-25.dex is the most likely original (1 copy(ies) in this set)
```
`skel-25.dex` — a dex with **25 % of its bodies removed** — was named the most likely
original, and the genuine untouched control ranked third. The cause is the bimodality
of §2: inside the low band the ratio is not a signal, and 1.7 % versus 1.9 % is noise
(the extracted bodies are replaced by `const/4 + return`, which *lowers* the
`return-void` count relative to the control). An agent routing on that line adopts a
modified image as its baseline, and every later diff is then built on the wrong
artifact — the same class of failure as the defect in §3.1, reached by a different
path.
**Fix.** `rank_key` became three-level, with an explicit eviction level so that either
signal can disqualify an image instead of the two merely tying:
1. `emptied_ratio >= 0.5` sorts last. Nothing mostly emptied is a candidate however
else it scores. This level exists because the count below cannot see the
`return-void`-in-every-body skeleton: that image has `minimal = erased = 0`, so a
pure-count key ranked it **first** — measured, and it is why this level was added
rather than the count being promoted alone.
2. `bodies_minimal + bodies_erased` low-first. This orders the low band, where the
count is monotone: 82 untouched, then 1299 / 2524 / 3736 / 4946 at 25 / 50 / 75 /
100 %.
3. `emptied_ratio` low-first, then header verdicts, then name.
Result on the same command:
```
ranking (most likely original first):
1. ctrl.dex stub%=1.9 erased%=0.0 min/erased=82 cksum=ok sig=ok
2. only.dex stub%=1.9 erased%=0.0 min/erased=82 cksum=ok sig=ok
3. throwstub.dex stub%=1.9 erased%=0.0 min/erased=82 cksum=ok sig=ok
4. skel-25.dex stub%=1.7 erased%=0.0 min/erased=1299 cksum=ok sig=ok
5. skel-50.dex stub%=1.7 erased%=0.0 min/erased=2524 cksum=ok sig=ok
6. skel-75.dex stub%=2.1 erased%=0.0 min/erased=3736 cksum=ok sig=ok
7. skel-all.dex stub%=2.3 erased%=0.0 min/erased=4946 cksum=ok sig=ok
8. allstub.dex stub%=100.0 erased%=0.0 min/erased=0 cksum=ok sig=ok <-- SKELETON
9. pruned.dex stub%=93.2 erased%=0.0 min/erased=344 cksum=ok sig=ok <-- SKELETON
10. nopfilled.dex stub%=0.0 erased%=100.0 min/erased=5061 cksum=ok sig=ok <-- SKELETON
verdict: ctrl.dex is the best-supported candidate for the original (1 copy(ies) in this set)
```
The control wins and every skeleton sorts behind it. The verdict line now states the
key and its limits instead of declaring a winner outright, and phrases the result as
"best-supported candidate" with an instruction to confirm before patching. `--trim`,
`--json` and `--find` are unchanged (verified: the recorded fixture still returns
`01_original_shell.dex` first, `--json` only gained keys). Note the residual limit the
report prints rather than hides: **`throwstub.dex` still ranks third**, because no body
statistic here sees a `throw`-stub skeleton. That is a boundary, not a bug — it is
listed in §7 with the reproduction.
### 3.4 A retracted Dex-VMP verdict — the measurement that killed it
An intermediate workbench script of mine
(`tools/_work/bench/unpack/b4/vmp_probe.py`, `vmp_methods.py`) decoded instruction
widths from a hand-written table and reported:
```
code_items=22424 desync=1393 (6.21%) bad_op=0xf8 / 0xef / 0xea / …
```
I read that as "bodies present but not decodable → private opcodes → Dex VMP". **The
reading was wrong, and the error was my table's.** The official decoder settles it:
```
$ E:\tools\android-14\dexdump.exe -d tools/_work/bench/unpack/b4/orig/classes.dex
Processing 'orig\classes.dex'...
Opened 'orig\classes.dex', DEX version '035'
… 943,223 lines …
instruction lines (hex:...) : 348566
code items resolved ('insns size :') : 22424
code '(none)' : 1179
```
One of the methods my script called a desync decodes cleanly under `dexdump`:
```
--- androidx.appcompat.app.AlertController$ButtonHandler.<init>:(Landroid/content/DialogInterface;)V
insns size : 11 16-bit code units
1fd564: 7010 9b05 0100 |0000: invoke-direct {v1}, Landroid/os/Handler;.<init>:()V
1fd56a: 2200 040e |0003: new-instance v0, Ljava/lang/ref/WeakReference;
1fd56e: 7020 fb76 2000 |0005: invoke-direct {v0, v2}, Ljava/lang/ref/WeakReference;.<init>:(Ljava/lang/Object;)V
1fd574: 5b10 ed08 |0008: iput-object v0, v1, …$ButtonHandler;.mDialog:…;
1fd578: 0e00 |000a: return-void
```
My script consumed 9 of 11 units and stopped on `0xed`; `0xed` is a legal
`invoke-*/range` opcode. **The lesson is the one the reference file already states and
this pass then failed to apply: a hand-rolled opcode table is not a VMP detector, and
"my decoder desynchronised" is a statement about my decoder.** Any future
"rare-opcode density" metric must be validated against `dexdump` on the same file
before it is allowed to inform a verdict.
The corrected count of bodies that fail to decode as dalvik in this dex is **0**.
Correction confirmed by an independent path: `dexdump` emits 34,566 instruction lines
and reports no structural error for any of the 22,424 code items.
## 4. B4 — the Dex-VMP judgement chain, re-opened
### 4.1 Static findings (`observed`)
Target: `repos/DexPatcher/assets/ezAndroid.apk`, 2,984,841 B, sha256
`76e3fa8119cf8b7c…`, package `<PKG>` (the `ezAndroid` CTF sample shipped in DexPatcher).
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py …/b4/orig/classes.dex
classes=2905 methods_with_code=22424 stub%=1.9 cksum=ok sig=BAD
```
- **Native declarations: 7**, all in the app's own obfuscated classes, each with no
code (`code : (none)`), enumerated from `dexdump`:
`Lcom/example/ezandroid/C;->cf(Ljava/lang/String;I)Z`,
`Lcom/example/ezandroid/MU;->GMS(Ljava/lang/String;)Ljava/lang/String;`,
`Lcom/example/ezandroid/S;->Stringfjni()Ljava/lang/String;`,
`Lcom/example/ezandroid/S;->onCreate`, `Lcom/example/ezandroid/S2;->onCreate`,
`Lcom/example/ezandroid/S$1;->onClick`, `Lcom/example/ezandroid/S2$1;->onClick`.
- **Two JNI exports** in `lib/arm64-v8a/libezandroid.so` (206,904 B, aarch64):
`Java_com_example_ezandroid_S_Stringfjni` @ `0xea6c` (184 B) and
`Java_com_example_ezandroid_MainActivity_stringFromJNI` @ `0xe91c` (184 B).
Both are thin JNI wrappers that construct a `std::string` and dispatch through a
virtual call — the body of the check is behind the vtable, not in the wrapper.
- **No `JNI_OnLoad` and no `RegisterNatives` symbol** anywhere in the library; the
string `JNI_OnLoad` does not occur in the file at all.
- **`assets/` holds six small opaque files** (`0OooOO` 118 B, `O0ooOO` 126 B,
`oo0Ooo` 86 B, `ooO0oo` 132 B, `OOoo0O` 398 B, `OOooO0` 260 B). Each begins with a
three-unit header (`05 00 38 00 02 00 …`) followed by 2-byte-aligned units.
- **No dex string references those six filenames.** Grep over the full 943,223-line
`dexdump` output for `0OooOO|O0ooOO|oo0Ooo|ooO0oo|OOoo0O|OOooO0|assets/` returns
**0 hits**.
### 4.2 Verdict: JNI sinking, not Dex VMP (`observed`)
The reference file's Dex-VMP test is "the dump's method bodies are intact but decode
as private opcodes — rare-opcode density off the charts, unknown opcode slots". On this
target **zero method bodies fail to decode**, and every `native` declaration has a
matching exported `Java_*`. The dex is not carrying an encoded program; the code left
the dex for the `.so`, which is `java2c-and-jni-sinking.md`'s territory, not this
file's. The two must not be conflated: an agent that reads "bodies decode as nonsense"
here will go hunting a private disassembler for code that is ordinary ARM64 in a
206 KB library.
The six `assets/` files are a separate, **unresolved** observation: they are not
referenced by any dex string, so they cannot be load-bearing at the dex layer. Whether
the native island reads them by constructing the path at runtime, and what they
contain, is **unverified** here — it needs a dynamic run (§4.3), not more static
searching.
### 4.3 What static analysis structurally cannot see (the B4 boundary)
`java2c`'s independent pass reached the same classification on this sample and recorded
that the hardening takes effect **at runtime only**. That is the general shape, and it
is the half of B4 that no dex probe can settle:
| Question | Answerable statically? | Why |
|---|---|---|
| Are the bodies real dalvik or private opcodes? | yes | `dexdump` decodes them or does not; measured 0 failures here |
| Did the code leave the dex for native? | yes | `native` access flags + `Java_*` exports / `JNI_OnLoad` |
| Does the native island check the signature / root / debugger? | **no** | the logic is ARM64 behind a vtable; the check may not exist in any dex |
| Do those `assets/` blobs get decrypted and used? | **no** | the path may be assembled at runtime; grep for the literal names finds nothing |
| Does a repack actually fail, and why? | **no** | needs the repack installed and run against a control (G4) |
**Rule this pass proposes for the reference file:** a static dex probe may return
"this is *not* dex-VMP" when the bodies decode cleanly and every native declaration has
an export. It may **not** return "this is not hardened" from the same evidence —
those are different claims, and the second one is the misdiagnosis
`java2c-and-jni-sinking.md` was written to stop.
### 4.4 Repack control — `observed` up to the install, `unverified` beyond
G4 requires a zero-change control through the same pipeline. Built with the
repository's own repacker against an explicit toolchain (this host has no uber-apk-signer;
`keytool` lives in the JDK that `ENV.md` does not list):
```
$ python skills/apk-reverse/scripts/repack.py \
--apk tools/_work/bench/unpack/b4/ezAndroid.apk \
--out tools/_work/bench/unpack/b4/ez-ctrl.apk \
--ks tools/_work/bench/unpack/b4/ctrl.jks --signer apksigner \
--apksigner E:\tools\android-14\apksigner.bat \
--zipalign E:\tools\android-14\zipalign.exe \
--keytool "C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe"
[in ] tools/_work/bench\unpack\b4\ezAndroid.apk (2984841 bytes)
[zip] - META-INF/1N0R.SF / 1N0R.RSA / MANIFEST.MF (signature artifact)
[zip] unsigned written: 2828919 bytes
[zip] AndroidManifest.xml STORED offset=0x31
[zip] classes.dex 3960320 bytes method=8 crc=037e9cb3
[zip] resources.arsc STORED offset=0x202f14 aligned
[sign] route: apksigner
[verify:zipalign] rc=0
[verify:v2block] present
[cert] SHA1 C9:89:C5:75:A4:83:AE:E0:A6:4B:F4:F6:0D:E3:D2:73:C2:8B:65:69
[result] OK
```
Zero-change repack, v2 block present, alignment verified, `classes.dex` carried through
at its original CRC (`037e9cb3`). **Install and launch on `<DEVICE>` are `unverified`
in this record**: the shared device was held by a parallel benchmark row (`B1 L1 repack
control + patched install`, `B2 L3 frida`) for the whole window available to this pass.
No claim is made here about whether the control launches; the artifact and the command
above are the reproducible half, and the device half is the open item.
Consequently **"repacking is truly refused on this target" is `inferred`, not
`observed`** — it follows from the native island being the only home of the app's own
logic (`Stringfjni`, `cf`, `GMS` are all native declarations), which makes a
dex-level patch unable to reach the behaviour, but no device run confirms a refusal.
The honest deliverable form for this target is the G1 row **annotated analysis report
with a stated boundary**, with `lsposed_scaffold.py` / `frida_rpc_serve.py` named as
the routes *if* a device run shows the build is refused.
## 5. `dex_mem_scan.py` — the cut half is measured, the in-process half is not
The script's job splits cleanly into *find-and-cut* (locate `dex\n03x` in a blob and
write exactly the declared `file_size`) and *reach the blob at all* (do it against a
live process's anonymous mappings through `/proc/<pid>/mem`). Only the first half was
measurable in this pass, and it was measured with a synthetic high-entropy capture
built to have the shape a heap buffer presents:
```
$ python … # 2 MiB of random bytes with a 4,124-byte dex written at offset 0x9a000
fixture dex: 4124 bytes, magic=b'dex\n039\x00'
capture written: …/memscan/anonregion.bin 2097152 bytes dex at 0x9a000
$ python skills/apk-reverse/scripts/dex_mem_scan.py …/anonregion.bin --dump …/scanout
== dex memory scan: 1 capture(s), 1 hit(s) ==
capture offset ver file_size classes state
anonregion.bin 0x9a000 039 4124 1 ok
wrote …/scanout/anonregion+0x0009a000.dex
$ python skills/apk-reverse/scripts/dex_dump_validate.py …/scanout
name size sha256[:12] ver cksum sig class noco code stub%
anonregion+0x0009a000.dex 4124 d3dedcd2b75e 039 ok ok 1 0 5 0.0%
```
Exact offset, zero false hits in 2 MiB of noise, the extracted image byte-identical
by construction and accepted by the judge including its checksum and signature. That
is the **whole find-and-cut path, observed** — the halves compose correctly.
**What is not measured.** The live-process half: the fixture that would exercise it is
built and staged (§6), but the shared device was held by a parallel benchmark row for
the entire window this pass had, so `dex_mem_scan.py` has still never been run against
a real anonymous mapping here. The one related number that exists is from the earlier
root-dump pass, and it is a **zero**: 142 anonymous regions scanned, no hits, against
17 dex images found in *named* ART mappings with no search at all. A zero-hit scan is
weak evidence for the reasons listed in the script's own header, and the honest
summary is "the finder works on a shape that looks right; its in-process reach is
unverified".
To close it (all commands staged, none run in this pass):
```
adb push tools/_work/bench/unpack/memscan/out2/memfixture.jar /data/local/tmp/
adb push tools/_work/bench/unpack/b3/src-r2pay.dex /data/local/tmp/payload.dex
adb shell 'su -c "CLASSPATH=/data/local/tmp/memfixture.jar app_process /system/bin \
com.revprobe.memfixture.Main /data/local/tmp/payload.dex"' &
# the fixture prints its own pid, truncates the on-disk payload so the heap copy is the
# only remaining source, and writes /data/local/tmp/memfixture.maps
adb shell 'su -c "grep -c . /proc/<pid>/maps"' # confirm cheap maps first
adb shell 'su -c "grep -v \"/\" /proc/<pid>/maps"' # the anonymous set to export
```
Note the fixture deliberately truncates its source file: a scan that finds the dex
after that point cannot have found a file copy, which is what makes the result
attributable to memory rather than to disk.
## 6. Not done — and why
- **No device run of the B4 control build.** The shared device was locked by another
benchmark row for the whole window; `devlock.py acquire unpack` timed out twice
(900 s and 1,800 s waits). Install, launch, time-to-death and refusal/success are
therefore `unverified`.
- **No in-process run of `dex_mem_scan.py`.** The fixture is built and staged
(`tools/_work/bench/unpack/memscan/`, §5), the arm64 `app_process` route is chosen
because this host has **no `android.jar`** and therefore cannot compile a real APK
at all, and the closing commands are listed — but the device was never available, so
the anonymous-mapping hit rate remains unmeasured. The only related measurement is
the earlier pass's **0 of 142** anonymous regions.
- **Whether the `assets/` blobs are live.** Needs a dynamic run (read the files out of
the process after the native island runs, or hook the path construction).
- **No end-to-end FART splice.** The reference file's FART loop needs a live
extraction-shell target; nothing in `repos/` is one (§1), and building an actively
resisting shell is out of scope for this pass.
- **A real APK build on this host.** The build-tools directory ships `d8`, `aapt2`,
`apksigner` and `zipalign` but **no `android.jar`**, and no `android.jar` exists
anywhere on the machine (searched the tool directories, `C:\Program Files`, and
the SDK path). Any
`javac` against `android.*` fails with "package android.os does not exist". APKs can
still be *repacked* here (`repack.py` does not recompile), which is why §4.4 works
and the self-built-APK route in the task brief does not.
## 7. Reproduction
```
python tools/_work/bench/unpack/b3/make_skeleton.py tools/_work/bench/unpack/b3/src-r2pay.dex tools/_work/bench/unpack/b3/skel
python tools/_work/bench/unpack/b3/analyze_pair.py
python tools/_work/bench/unpack/b3/crosscheck.py tools/_work/bench/unpack/b3/skel/*.dex
python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3/skelall
python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/t2-unpack --find 'Lcom/stub/'
python tools/_work/bench/unpack/b4/vmp_boundary.py tools/_work/bench/unpack/b4/orig/classes.dex
E:\tools\android-14\dexdump.exe -d tools/_work/bench/unpack/b4/orig/classes.dex
python skills/apk-reverse/scripts/dex_mem_scan.py tools/_work/bench/unpack/memscan/anonregion.bin --dump tools/_work/bench/unpack/memscan/scanout
```
Raw captures live beside the fixtures: `evidence/prefix-validate.txt` (before the fix,
including the inverted ranking) and `evidence/postfix-validate.txt` (after). The
`memscan/` tree carries the staged (unrun) in-process fixture: `plain/com/revprobe/memfixture/Main.java`
(no-`android.jar` form), `build.ps1` (the APK build that cannot complete here, kept as
the record of *why*), and `out2/{classes.dex,memfixture.jar}`.
To re-verify the fix from scratch, the two commands that matter are
`dex_dump_validate.py …/skelall` (expect `nopfilled.dex` flagged, not ranked first) and
`dex_dump_validate.py tools/_work/t2-unpack --find 'Lcom/stub/'` (expect
`01_original_shell.dex` still the winner).

316
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-java2c.md Normal file
View File

@@ -0,0 +1,316 @@
# Extension verification — Java2C identification and routing
Covers the 2026-09 extension pass that added `references/java2c-and-jni-sinking.md` and
`scripts/java2c_probe.py`. Strength labels as defined in this directory's README:
**observed** (exact command + output here), **inferred** (follows from observed facts,
step not executed), **unverified** (assumed or externally reported, not independently
confirmed).
Reference environment for this pass: Windows 11 host, Python 3.14.0, JDK 17,
build-tools (`d8`/`aapt2`) from a local Android 14 SDK, `gcc` from MinGW only.
**No Android NDK, no host `clang`, no `wsl` (HCS unavailable), and no device-side
`clang`/Termux.** A rooted Android 11 / API 30 / arm64-v8a `<DEVICE>` was attached and
reachable but **was not used at all** in this pass: every result below is static. No
package name, project name or device serial appears in this record.
Samples are identified by hash rather than by name. They are public MASTG training apps
(a no-native baseline, and two carrying a JNI sink), one VMP-labelled app taken from
`tools/_work/bench/repos/`, one real third-party aarch64 ELF already on the host, and one
locally synthesised Java2C-shaped dex.
---
## 1. The compile chain could not be closed — observed
The intended route was to build a real Java2C `.so` with `amimo/dcc` and probe it. `dcc`
is present at `tools/_work/bench/repos/dcc/` and its Python dependencies are satisfied.
The build stops at the C compiler, and every alternative was tried exactly once before
being written off (two-strike rule — no third variant was attempted):
```
$ which gcc clang
gcc.exe C:\MinGW\bin\gcc.exe # MinGW targets PE/COFF, not ELF
gcc/clang for Android: none
$ wsl -l -q ; wsl -e bash -lc "which gcc"
Ubuntu-E
(empty) -> exit 1 with: Wsl/Service/CreateInstance/CreateVm/HCS/HCS_E_SERVICE_NOT_AVAILABLE
$ adb shell "which clang clang++ gcc ; ls /data/data/com.termux"
(no output for which)
ls: /data/data/com.termux: No such file or directory
```
Related **observed** fact: the `dcc` checkout ships **no prebuilt binary at all** — a
recursive search for `*.so`, `*.apk`, `*.a` over the whole repo returns zero files. There
is no ready-made Java2C library to fall back on.
Consequence: **no ELF file anywhere in this pass was produced by a Dex-to-C compiler.**
Every Java2C-specific criterion in the reference document is therefore **inferred**.
## 2. Degenerate route A — `dcc` generates the C without an NDK — observed
`dcc.py` takes `--no-build`, and `dcc_main(..., do_compile=False)` writes the generated
sources and the JNI project *before* `build_project(project_dir)` would be called. With
`--no-build` and no `-o`, the apktool repack path is skipped entirely (it is guarded by
`if is_apk(apkfile) and outapk`). So the translation stage runs to completion on a machine
with no NDK.
```
$ cd tools/_work/bench/repos/dcc
$ python dcc.py <SAMPLE_DIR>/baseline.apk --no-build --dynamic-register \
--source-dir <WORK>/dcc-out
# (emits only Python 3.14 SyntaxWarnings from the bundled androguard)
$ find <WORK>/dcc-out -type f
jni/Android.mk
jni/Application.mk
jni/nc/Dex2C.cpp jni/nc/Dex2C.h
jni/nc/DynamicRegister.cpp jni/nc/DynamicRegister.h
jni/nc/ScopedLocalRef.h jni/nc/ScopedPthreadMutexLock.h
jni/nc/well_known_classes.cpp jni/nc/well_known_classes.h
jni/nc/compiled_methods.txt
jni/nc/Java_<PKG>_<Class>_<method>__<mangled-proto>.cpp
```
This is the mapping evidence the pass needed, and it is exact rather than imagined:
`compiled_methods.txt` is a one-line dex signature, and it produced exactly one C file
whose name is the same identifier in JNI long form:
```
L<PKG>/MainActivity;onCreate(Landroid/os/Bundle;)V
-> Java_<PKG>_MainActivity_onCreate__Landroid_os_Bundle_2.cpp
```
The generated function opens with the parameter shape the reference document cites as the
`.so`-side signature of a translated method:
```c
/* L<PKG>/MainActivity;->onCreate(Landroid/os/Bundle;)V */
void Java_<PKG>_MainActivity_onCreate__Landroid_os_Bundle_2(JNIEnv *env, jobject thiz, jobject p2){
jobject v0 = NULL;
...
jclass cls0 = NULL,cls1 = NULL,cls2 = NULL,cls3 = NULL;
```
And the registration half is generated as real text, which is the strongest single piece
of evidence produced in this pass that the "no `Java_*` symbol to search for" path is not
hypothetical:
```c
// DynamicRegister.cpp
const char *dynamic_register_compile_methods(JNIEnv *env) {
jclass clazz;
clazz = env->FindClass("<PKG>/MainActivity");
if (clazz == nullptr)
return "Class not found: <PKG>/MainActivity";
const JNINativeMethod export_method_0[] = {
{"onCreate", "(Landroid/os/Bundle;)V", (void *)Java_<PKG>_MainActivity_onCreate__Landroid_os_Bundle_2}
};
env->RegisterNatives(clazz, export_method_0, 1);
env->DeleteLocalRef(clazz);
return nullptr;
}
```
Two further facts read out of the generated `jni/Application.mk`, both **observed** and
both load-bearing for the JNI-boundary section of the reference document:
```
APP_STL := c++_static
APP_CPPFLAGS += -fvisibility=hidden
APP_ABI := armeabi-v7a arm64-v8a
```
`-fvisibility=hidden` means the shipped library can present as "exports no `Java_*`
symbol" **even when the translation used static naming** — the second, independent reason a
symbol search returns nothing. `c++_static` is the origin of the `libc++` string criterion,
and is also why that criterion is weak: it is an ordinary NDK setting.
## 3. The probe against real JNI-sinking targets — observed
This is the discriminating measurement the reference document is built on. The tool was
run unmodified:
```
$ python skills/apk-reverse/scripts/java2c_probe.py --apk <SAMPLE_DIR>/jni-a.apk
classes.dex: 720084 B, 625 classes, 5081 methods, native 2 (0.04%),
native-dominated classes 0, fully native 0
libfoo.so: 14176 B, aarch64, dynsym 20 entries, Java_* 2, JNI_OnLoad False
libfoo.so: 13948 B, arm, dynsym 26 entries, Java_* 2, ...
libfoo.so: 13788 B, x86, dynsym 16 entries, Java_* 2, ...
libfoo.so: 14440 B, x86_64, dynsym 16 entries, Java_* 2, ...
verdict : jni-sinking
confidence : medium
```
```
$ python skills/apk-reverse/scripts/java2c_probe.py --apk <SAMPLE_DIR>/jni-b.apk
classes.dex: 1934860 B, 1396 classes, 11182 methods, native 3 (0.03%),
native-dominated classes 0, fully native 0
libfoo.so (arm/arm64: 3 Java_* symbols, x86/x86_64: 3 Java_* symbols)
verdict : jni-sinking
confidence : medium
```
Two independent real targets, and both give the same two results: **native density three
orders of magnitude below the Java2C shape**, and **an exact 1:1 between the dex native
declarations and the exported `Java_*` symbols** (2:2 and 3:3, across four ABIs each).
That 1:1 is the point. It is *not* a Java2C signal on its own — both shapes produce it —
which is why the density measurement and the native layer must be read together. A tool
that reported on either alone would misroute one of these two samples.
Baseline control, same command, no native code at all:
```
$ python skills/apk-reverse/scripts/java2c_probe.py --apk <SAMPLE_DIR>/baseline.apk
classes.dex: 5528 B, 7 classes, 15 methods, native 0 (0.00%), ...
verdict : no-hardening-signal
confidence : medium
```
## 4. The probe against a Java2C-shaped dex — observed
No Java2C library could be built, so the dex half of the shape was synthesised. A Java
source declaring two classes whose methods are almost entirely `native` (plus one mixed
class, standing in for an ordinary JNI boundary) was compiled with `javac --release 8` and
assembled with `d8 --min-api 19`:
```
$ javac --release 8 -d classes src/<PKG>/Calc.java # exit 0
$ d8 --min-api 19 --output out classes/<PKG>/*.class # exit 0
out/classes.dex 1752 B sha256 59e0b1d34f00578e1343ee5f344d0e58829f482b643a7843c8ad280311ce9457
$ python skills/apk-reverse/scripts/java2c_probe.py --dex out/classes.dex
classes.dex: 1752 B, 3 classes, 29 methods, native 24 (82.76%),
native-dominated classes 2, fully native 1
verdict : java2c-suspect-needs-so
confidence : low
```
The low confidence is the correct answer, not a shortcoming: the tool refuses to claim
Java2C from the dex alone, and says so in its route block. That refusal is the guard
against the misdiagnosis this pass exists to prevent.
Combining that dex with a real library that registers dynamically makes the `java2c` branch
reachable, which is what proves the branch is not dead code — the input is composite and is
labelled as such:
```
$ python skills/apk-reverse/scripts/java2c_probe.py --dex out/classes.dex \
--so <WORK>/libandroid.so
verdict : java2c
confidence : medium
evidence: (strong) native layer: dynamic registration in libandroid.so
(JNI_OnLoad exported, zero Java_* symbols)
```
## 5. The JNI boundary — why `RegisterNatives` cannot be found — observed
The reference document claims a symbol search fails silently and that looking for a
`RegisterNatives` symbol is structurally unable to work. Both are measured here.
A real 273,120 B aarch64 system library pulled from one of the samples exports a JNI entry
point and no `Java_*` symbols, and the probe flags it as dynamic registration — the
affirmative check that does work:
```
libandroid.so: 273120 B, aarch64, dynsym 504 entries, Java_* 0,
JNI_OnLoad True, RegisterNatives-import False
(strong) native layer: dynamic registration in libandroid.so
(JNI_OnLoad exported, zero Java_* symbols)
```
Independent confirmation that the absence is real and not a parser gap — the same check run
outside the script:
```
$ python -c "d=open('gadget.so','rb').read(); [print(k, d.count(k.encode())) for k in
['JNI_OnLoad','RegisterNatives','Java_','Dex2C','libc++_shared.so','frida']]"
JNI_OnLoad 0
RegisterNatives 0
Java_ 0
Dex2C 0
libc++_shared.so 0
frida 311
```
A 25.8 MB real third-party aarch64 ELF with 311 occurrences of its own name contains zero
`JNI_OnLoad` and zero `Java_`. The probe reported the same (0 `Java_*`, `JNI_OnLoad False`),
so the two readings agree and the probe is not reporting a parse failure as an absence.
**Why the `RegisterNatives` search cannot fire** `[observed]`: the library above registers
dynamically and contains **zero literal `RegisterNatives` symbols**. The cause is a
documented property of the NDK C++ headers `[inferred]`: `jni.h` declares `RegisterNatives`
as an inline member of `JNIEnv` that dispatches through the function table, so a call site
emits no external symbol. The observed half is the missing symbol; the explanation is
inferred from the mechanism.
## 6. Coverage of the probe's code paths — observed
| Path | Exercised by | Result |
|---|---|---|
| dex header + class_defs + class_data (ULEB128) | 5 real dex images + 1 synthetic | Parsed; header size match reported |
| `direct_methods` / `virtual_methods` decoding | the synthetic fixture (has both) | Correct after the defect in §7 |
| native density, native-dominated classes | all six | 0.03–0.04 % real vs 82.76 % synthetic |
| no-`code_item` and trivial-body ratio | all six | 0 on the fixture, 4–5 % on the real apps |
| ELF header, arch, section table | 10 real ELFs (4 ABI variants of two `libfoo.so`, 2 app libs, 1 gadget) | aarch64/arm/x86/x86_64 all identified |
| `.dynsym` parse, `Java_*` extraction | real `libfoo.so` ×8 | 2 and 3 symbols, matching the dex exactly |
| `JNI_OnLoad` export detection | real system library | True |
| dynamic-registration verdict | same library | Fired |
| string-signature scan | 25.8 MB ELF, all samples | Weak hits only (`dcc` ×6 on the gadget); no false strong hit |
| `--json` | synthetic fixture | Valid JSON, metrics block parses |
| `--dir`, `--apk`, `--dex`, `--so` | all four modes | OK |
| `--help` | direct | OK |
The string scan produced exactly one class of hit across every sample: the weak `dcc`
substring (6 occurrences inside a 25.8 MB binary, i.e. coincidental). **No strong marker was
ever produced by any real sample**, which is why the Java2C verdict rests on structure
rather than on strings.
## 7. Not exercised — and why the Java2C verdict stays inferred
- **A real Java2C `.so`.** Never produced; the compile chain is dead (§1). Everything about
how a *compiled* Java2C library presents — whether the generated functions keep their
names under `-fvisibility=hidden`, whether `Dex2C` runtime strings survive into `.rodata`
— is **inferred** from the generated sources and the build flags, not observed.
- **The runtime claim that no dex bytecode exists in memory.** This is the mechanism the
whole document turns on and it is **inferred from the translation model**. No live Java2C
app was run. It is falsifiable and cheap to test on a live sample; that test is named in
the reference document.
- **VMP identification by static dex metrics.** One VMP-labelled sample (sha256
`76e3fa8119cf8b7c4b4768be557d71863433a8ae4d2ad40901eae6bb829349bc`, a 2,984,841 B APK
from the local bench repos, identified by hash only)
was measured and reported **`jni-sinking`**: its static dex has 23,603 methods of which 7
are native, with a 4 % stub ratio, and the archive contains no additional encrypted
payload. **This is a recorded boundary, not a success**: whatever hardening that sample
carries is applied at runtime, and a static dex probe is structurally unable to see it.
Do not read this row as "the sample is unhardened".
- **Anything on a device.** The `<DEVICE>` was not used; there is no dynamic evidence in
this record.
## 8. Two defects this pass found in its own tool — observed
Both were found by running the tool against the synthetic fixture and checking the numbers
by hand, and both are fixed in the shipped script.
1. **`direct_methods` and `virtual_methods` were decoded as one continuous run.** They are
two independent encoded lists; each one's first `method_idx_diff` is relative to 0.
Walking them as one run mis-attributed every virtual method to the following class. The
fixture exposed it as `Codec: 5 methods` where the source declares 4. Fix: decode them as
two passes. After the fix the per-class counts match the source exactly (Calc 20/21
native, Codec 3/3, Helper 1/2).
2. **The "whole class is native" criterion was unreachable.** `javac` — and, per the
generated output, a translation pass — always leaves `<init>`/`<clinit>` behind, so no
class is ever literally all-native unless a constructor is counted as native. The
fixture showed `fully native 0` while two of its classes were plainly native-shaped. Fix:
constructors are excluded and the test is now ">= 70 % of >= 3 named methods native",
reported alongside the literal count. After the fix: `native-dominated classes 2,
fully native 1`.
Both defects produced *plausible* output — a wrong count and a zero that reads as "nothing
found" — which is the failure shape this whole repository is organised against. Recording
them here because a reader relying on either number would have drawn a wrong conclusion.

121
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-kernel-ondevice.md Normal file
View File

@@ -0,0 +1,121 @@
# Extension verification — kernel/environment hardening + on-device tooling
Covers the 2026-09 extension pass that added `references/kernel-and-environment-hardening.md`,
`references/on-device-tooling.md` and `scripts/mt_mcp_probe.py`. Strength labels as defined in
this directory's README: **observed** (exact command + output here), **inferred** (follows from
observed facts, step not executed), **unverified** (assumed or externally reported, not
independently confirmed).
Reference environment for this pass: rooted physical device, Android 11 (API 30), arm64-v8a,
kernel **4.14.186**, Magisk alpha + Zygisk, LSPosed v1.9.2 activated, Zygisk-Assistant v2.1.4
and MoveCertificate v1.6.2 installed (pending reboot merge), MT Manager 2.26.9 installed.
---
## 1. MT APK MCP probe — measured
### 1.1 `scripts/mt_mcp_probe.py` runs and reports correctly
`--help` works (argparse, three options). Observed:
```
$ python skills/apk-reverse/scripts/mt_mcp_probe.py --help
usage: mt_mcp_probe.py [-h] [--url URL] [--timeout TIMEOUT] [--json]
```
Full **observed** output of the probe in the service-down state, with the port forward in place:
```
$ adb forward --list
<serial> tcp:8787 tcp:8787
<serial> tcp:8651 tcp:8651
<serial> tcp:65534 tcp:65534
$ python skills/apk-reverse/scripts/mt_mcp_probe.py
[waiting] MCP server is not answering at http://127.0.0.1:8787/mcp
The MT APK MCP must be started by hand in the MT UI (adb cannot start it):
1. On the phone: MT Manager -> side drawer -> Tools -> APK MCP -> Start
...
exit code 2
```
The failure was localised to the service (not the transport) on the device side — **observed**:
```
$ adb -s <serial> shell "su -c 'netstat -tln | grep 8787 || echo NOT_LISTENING'"
NOT_LISTENING
```
So at pass time: port forward alive, device reachable, MT's MCP service not started in the MT UI
(it requires a manual start that adb cannot perform). The script's designed degraded-mode
behaviour (waiting instructions + exit 2) is therefore **verified by observation**; the online
path (initialize handshake → grouped `mt_apk_*` inventory) is **unverified on this device** — it
was not exercised because the service was never started during the pass. The JSON-RPC shape
(initialize with `protocolVersion 2024-11-05`, `notifications/initialized`, `tools/list`,
`Accept: application/json, text/event-stream`) follows the MCP Streamable HTTP specification and
MT's documented transport; treat the exact response parsing as inferred until a live run.
### 1.2 What was deliberately not done
No attempt was made to start MT's MCP from adb (documented as impossible), and no scripted
tap-driver was built to press the UI button — that is a human step. The probe script doubling as
a "wait until up" check (exit 2 → retry after manual start) is the supported loop.
## 2. Phone-module environment — observed earlier, cited here
These facts were measured in the prior configuration pass and archived in
`tools/_phone-modules/README-手机模块环境.md` and `README-MCP与网络环境.md`; this pass cites them
rather than re-measuring. They count as **observed** (exact commands and outputs exist in those
archives):
- LSPosed v1.9.2 (7024) activation: log lines `ZygiskCompanion: welcome to LSPosed!`, version
banner, `lspd` daemon running as system, manager UI reporting activated.
- Zygisk requires one reboot after enable before `/data/adb/zygisk` exists; modules install to
`/data/adb/modules_update/` and merge into `/data/adb/modules/` only on reboot.
- Shamiko v1.2.5 kept as backup, **not** installed — documented conflict with Zygisk-Assistant
(both do root hiding); Zygisk-Assistant chosen because it does not depend on DenyList
enforcement.
- mcp-termux / stackplz eBPF route closed on this device: requires kernel 5.10+, device has
4.14.186; its bundled kmodules ship only 5.10/5.15/6.1/6.6/6.12 variants.
- MT Manager 2.26.9 installed; MCP port confirmed as 8787 in MT settings; forwards 8787/8651/
65534 established.
## 3. Kernel-level content — unverified / inferred, with sources
No kernel-level mechanism in `kernel-and-environment-hardening.md` was executed, on this device
or any other. Specifically **unverified here**: eBPF kprobe/uprobe tracing, seccomp-BPF
installation into a target, KPM/LKM hooking, self-ptrace anti-anti-debug, and every hiding claim
of Shamiko / Zygisk-Assistant against a real detector (the module archives also record that
hiding was never validated against an actual detection scenario).
The **inferred** claims that carry external sources (fetched 2026-09, URLs in the reference
file):
| Claim | Source |
|---|---|
| KernelSU: kernel-based su, official support GKI 2.0 = kernel 5.10+ (Android 12-era), backport to 4.14 requires self-built kernel; module system conflicts with Magisk magic mount; su-only coexistence possible (kernel vs ramdisk patching) | tiann/KernelSU README + FAQ (github.com) |
| APatch: KernelPatch patches stock boot.img kernel (no kernel source needed — the difference from KernelSU); SuperKey-gated SuperCall syscall; KPM kernel-space modules with inline/syscall-table hooks; SELinux hooked not rewritten; Shamiko officially unsupported | apatch.dev FAQ |
| Shamiko: reads Magisk denylist but requires DenyList enforcement OFF; whitelist mode via `/data/adb/shamiko/whitelist` with documented performance cost | Shamiko README (LSPosed distribution) |
| Zygisk-Assistant: Magisk use = target on denylist + Enforce OFF; KernelSU/APatch use = ZygiskNext + "Umount modules/Exclude modifications" | snake-4/Zygisk-Assistant README |
| GKI 2.0 starts at android12-5.10; eBPF extension docs target GKI kernels | source.android.com (kernel architecture section) |
The version gate consequence (eBPF row closed on 4.14) is **observed** for this device (uname)
and **inferred** as a general gate from the GKI sources above.
## 4. Not run / open items
- Live MCP handshake: blocked on a human UI action (start APK MCP in MT). The probe script is
ready; re-run and record the real `initialize` + `tools/list` output when it happens.
- Zygisk-Assistant + MoveCertificate merge: pending the device reboot recorded in the module
archive; their post-merge behaviour is untested.
- No LSPosed hook module was built or run against a real target in either pass — the framework
is activated (observed), the module workflow remains inferred.
## 5. Reproduction one-liners
```
python skills/apk-reverse/scripts/mt_mcp_probe.py # probe + inventory or waiting help
adb forward --list # confirm 8787 forward exists
adb -s <serial> shell "su -c 'netstat -tln | grep 8787'" # device-side listen check
adb -s <serial> shell uname -r # the 4.14 kernel gate, observed
```

236
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-kernel-weapons.md Normal file
View File

@@ -0,0 +1,236 @@
# Extension verification — kernel-level weapons (templates, gates, and a closed door)
Covers the extension pass that added `scripts/kernelsu_syscall_mask.py` and the `§4a` section of
`references/kernel-and-environment-hardening.md`. Strength labels as defined in this directory's
README: **observed** (exact command + output here), **inferred** (follows from observed facts,
step not executed), **unverified** (assumed or externally reported, not independently confirmed).
Reference environment: Windows 11 host, Python 3.14.0; `<DEVICE>` = rooted Android 11 (API 30)
arm64-v8a, Magisk alpha, no APatch/KernelPatch.
**Headline: the generator, the gate table and the userspace module skeleton are measured. No
kernel-side artifact was compiled or loaded anywhere, and every route is closed on the reference
device. That is the deliverable — a template plus a measured statement of the door.**
---
## 1. Kernel gate, measured on the device — observed
```
$ adb shell 'uname -r; cat /proc/version'
4.14.186+
Linux version 4.14.186+ (nobody@android-build) (Android (6443078 based on r383902) clang version
11.0.1 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79),
LLD 11.0.1 (/buildbot/tmp/tmp6_m7QH ...)) #1 SMP PREEMPT Wed Mar 30 23:32:42 CST 2022
```
4.14.186 is below the 5.10 GKI gate for the eBPF route, and the device carries no
KernelPatch-patched image. Read-only query; no device lock was needed and no state was changed.
## 2. Host toolchain probe — observed
```
$ python skills/apk-reverse/scripts/kernelsu_syscall_mask.py gates
== kernel-side routes and their gates ==
eBPF tracepoint/kprobe GKI kernel 5.10+ (Android 12+)
why: Android's BPF/tracing support as bpftrace-class tooling and
stackplz expect it is a GKI-era feature; BTF is required for
CO-RE style probes.
check: uname -r -> expect 5.10.x or newer; ls /sys/kernel/btf/vmlinux
KernelPatch / APatch KPM KernelPatch-patched boot image + kpm toolchain
why: KPMs load through kpimg injected into the kernel image's
payload segment; building one needs a bare-metal ARM64
compiler (aarch64-none-elf-gcc), not the NDK.
check: ls /data/adb/ | grep -i apatch ; which aarch64-none-elf-gcc
Out-of-tree LKM Kernel source matching the device's exact version + vermagic
why: A module built against a different kernel revision is refused
at load time on a mismatched vermagic, and the device's kernel
source is frequently not published at all.
check: uname -r ; look for a matching kernel source tree for the device
seccomp-BPF (for contrast) any modern kernel, but it is not this route
why: seccomp can make a syscall FAIL (SECCOMP_RET_ERRNO/TRAP); it
cannot rewrite the CONTENT a successful read returns. It closes
doors, it does not paint them -- which is the whole requirement
for a spoofed /proc read.
check: n/a -- this is a capability ceiling, not a version check
== host probe (what can be checked from here) ==
bare-metal ARM64 compiler: NOT FOUND
the KPM route cannot be built from this host as configured
ndk-build: NOT FOUND
make: NOT FOUND
A kernel source tree, a patched boot image and the device's own kernel headers cannot be
checked from the host; they are device facts.
```
The three `NOT FOUND` lines are the measured reason the KPM route cannot be built from here, and
they are also consistent with `ENV.md`'s record that the machine has no Android NDK. The LKM route
additionally needs kernel source that does not exist for this device.
## 3. `generate` — observed
```
$ python skills/apk-reverse/scripts/kernelsu_syscall_mask.py generate \
--out tools/_work/t5-vmp/sysmask --package '<PKG>'
generated module scaffold in tools/_work/t5-vmp/sysmask
module.prop 165 B
customize.sh 166 B
post-fs-data.sh 830 B
service.sh 467 B
uninstall.sh 298 B
config/syscall_mask.json 1079 B
kpm/syscall-mask.c 8096 B
kpm/Makefile 880 B
lkm/syscall-mask.c 2434 B
ebpf/syscall_mask.bpf.c 4338 B
README.md 3752 B
STATUS: the userspace module is installable; every kernel-side file is an UNVERIFIED template.
No kernel code was compiled or loaded.
```
`module.prop`, as generated (the package identity is the `<PKG>` placeholder by design — this file
carries no real package name):
```
id=syscall-mask
name=Syscall Mask
version=0.1.0
versionCode=1
author=apk-reverse
description=config-driven syscall return masking (openat/read/stat on /proc/self/*)
```
The configuration table rendered into the KPM template from `config/syscall_mask.json`, so the
rule set is data rather than hand-edited C:
```
static const struct mask_rule rules[] = {
{ "/proc/self/maps", 0, MASK_DENY, -2, 0 },
{ "/proc/self/maps", 0, MASK_FAKE_SIZE, -2, 0 },
{ "/proc/self/status", "TracerPid", MASK_FILTER_LINE, -2, 0 },
{ "/proc/self/status", "TracerPid", MASK_FILTER_LINE, -2, 0 },
};
```
And the KPM lifecycle surface, which is the part an interface drift would break first:
```
int kfunc_def(strncpy_from_user)(char *dst, const char __user *src, long count);
/* fp_hook_syscalln(__NR_openat, before_openat, 0, 0); */
/* fp_hook_syscalln(__NR_newfstatat, before_newfstatat, 0, 0); */
/* fp_hook_syscalln(__NR_read, before_read, 0, 0); */
KPM_INIT(mask_init);
KPM_EXIT(mask_exit);
```
Those names (`KPM_NAME` / `KPM_INIT` / `KPM_EXIT`, `kfunc_def`, `fp_hook_syscalln`,
`hook_fargs4_t`, `syscall_argn`, `args->skip_origin`, `args->ret`) are taken from a public KPM
development write-up rather than invented — blackr0ck, *APatch KPM 开发：绕过 DirtySepolicy /
Duck Detector 的 SELinux Root 检测*, [bbs.kanxue.com/thread-291665.htm](https://bbs.kanxue.com/thread-291665.htm),
fetched 2026-02. **They are still `unverified` here**: nobody compiled against a `kpmodule.h` in
this pass, and the generated template says so in its own header for exactly that reason.
The registration calls are left **commented out** in the template. A generated file that arms a
kernel hook by default is a file that arms a kernel hook the first time somebody runs `make` in a
hurry.
## 4. `verify` — observed
```
$ python skills/apk-reverse/scripts/kernelsu_syscall_mask.py verify --dir tools/_work/t5-vmp/sysmask
== verifying tools/_work/t5-vmp/sysmask ==
module.prop: author=apk-reverse, description=config-driven syscall return masking
(openat/read/stat on /proc/self/*), id=syscall-mask, name=Syscall Mask,
version=0.1.0, versionCode=1
config: 4 rule(s), actions=['deny', 'fake_size', 'filter_line']
kpm target present: Makefile, syscall-mask.c
status: UNVERIFIED TEMPLATE -- not compiled, not loaded
lkm target present: syscall-mask.c
status: UNVERIFIED TEMPLATE -- not compiled, not loaded
ebpf target present: syscall_mask.bpf.c
status: UNVERIFIED TEMPLATE -- not compiled, not loaded
kpm/syscall-mask.c: KPM_* lifecycle macros present, kfunc_def not extern
== 0 problem(s): structure is consistent ==
(this says nothing about whether the kernel side compiles or loads)
```
`verify` is deliberately read-only and deliberately narrow: it checks `module.prop` fields, the rule
JSON, and the one static trap in the KPM template that has a documented consequence (an `extern`
`kfunc_def` becomes `*UND*` and the loader refuses the module). It states on its last line what it
does **not** establish, because "0 problems" on a template is the easiest sentence in this file to
over-read.
### 4a. A false positive this check produced, and its fix — observed
The first run reported:
```
== 1 problem(s) ==
- syscall-mask.c declares kfunc_def with `extern`, which the loader rejects as an unknown symbol
verify rc=1
```
The template does not declare it with `extern` — its comment *explains* why `extern` is forbidden,
and the check's own regex ran from that prose across to the real declaration's semicolon. A linter
that reads comments as code produces reports nobody can act on. The fix strips `/* */` and `//`
comments before any token check (`_strip_c_comments`). Both the false positive and the fix are
recorded here rather than quietly repaired, because the failure mode — a check that fires on its
own documentation — is one a reader will meet again.
## 5. The userspace module's ceiling — observed by construction, and worth stating plainly
`post-fs-data.sh` and `service.sh` in the generated skeleton do not load anything. That is not an
omission: a KernelSU/Magisk/APatch-userspace module runs as root in the normal world and cannot
change what a syscall returns. The generated `README.md` says this in its first paragraph, and
`post-fs-data.sh` carries the reasoning in a comment:
```
# The kernel side (if you built and loaded one) owns the syscall table; this
# script's whole job is to make the configuration visible to it and to leave a
# record that the module actually ran. Do not put a syscall hook here: this is
# userspace, and nothing here can change what openat/read/stat return.
```
The uninstall hook likewise refuses to unload a kernel module, because a silent unload during
package removal is how a device ends up in a boot loop.
## 6. eBPF route — gate table and the capability ceiling
| Route | Gate | Check on the device | Status here |
|---|---|---|---|
| eBPF tracepoint/kprobe | GKI 5.10+ with tracing/BTF | `uname -r`; `ls /sys/kernel/btf/vmlinux` | **closed** — device is 4.14.186 |
| seccomp-BPF | any modern kernel | n/a | not this route: it can only fail a syscall, not rewrite a successful read |
| KernelPatch / APatch KPM | patched boot image + `aarch64-none-elf-gcc` | `ls /data/adb/ \| grep -i apatch`; `which aarch64-none-elf-gcc` | **closed** — neither present; patch-the-boot-image risk declined |
| Out-of-tree LKM | kernel source matching vermagic | `uname -r` + a matching source tree | **closed** — no source for this device |
The probe template (`ebpf/syscall_mask.bpf.c`) is a `sys_enter_openat` tracepoint observer with a
uid filter map and a ringbuf event, and it documents two dead ends in the file itself:
`bpf_override_return()` only applies to `ALLOW_ERROR_INJECTION` functions, which raw syscall
entries are not, and a tracepoint cannot rewrite a result at all. **A tracepoint observes; it does
not spoof.** The honest eBPF shape for this job is therefore observation — which is still worth
having, because `§5` of the reference file requires you to establish whether the target uses a raw
`svc` or a libc call *before* designing any spoof.
## 7. What was not done
- **No kernel-side code was compiled.** Not the KPM, not the LKM, not the eBPF program. No
`kpmodule.h`, no kernel headers, no `bpftool`, no bare-metal ARM64 compiler on this host.
- **Nothing was loaded, flashed or activated on `<DEVICE>`.** No boot-image patch, no module load,
no kernel modification. The task forbids it for good reason: a kernel module that faults takes
the device down before it can report why.
- **The KPM API surface is `unverified`.** The macro and helper names follow a public write-up; the
argument arity of `fp_hook_syscalln` in particular is marked `TODO(verify-on-device)` in the
generated source rather than guessed, because a wrong arity in a hook registration is a
load-time failure at best.
- **The struct-stat size offset is a TODO, not a value.** The `newfstatat` handler deliberately does
not carry a magic offset: it is ABI-specific and copying one from a different architecture is how
a spoof corrupts an unrelated field.
- **`verify` establishes structure, not capability.** Its own output says so.

567
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-lsposed.md Normal file
View File

@@ -0,0 +1,567 @@
# Extension verification — LSPosed module route
Covers the 2026-09 extension pass that added `skills/apk-reverse/references/lsposed-and-modules.md`
and `skills/apk-reverse/scripts/lsposed_scaffold.py`. Strength labels as defined in this directory's
README: **observed** (exact command + output here), **inferred** (follows from observed facts, step
not executed), **unverified** (assumed or externally reported, not independently confirmed).
Reference environment for this pass: rooted physical device, Android 11 (API 30), arm64-v8a,
Magisk alpha + Zygisk, LSPosed v1.9.2 (7024) activated, Zygisk-Assistant v2.1.4 installed, MT Manager
2.26.9 installed, one hardened Flutter sample installed (referred to below as *the sample*; no target
identity is recorded here). Host: Windows, JDK 17.0.4.1, Android build-tools 34.0.0, API 28
`android.jar`, Xposed `api-82` stub jar.
**Identifier convention:** every occurrence of the sample's package identifier in the transcripts
below is normalized to `<PKG>` (`<PKG>.MainActivity`, `<PKG>.BaseApplication`), and its sub-package
prefixes are shortened the same way. The transcripts are otherwise verbatim, including the class
names a hook reported — those are what make the evidence checkable.
This pass closes an item left open by the previous extension record, which noted that "no LSPosed hook
module was built or run against a real target" (`EXTENSION-kernel-ondevice.md` §4). The build half is
now **observed**; the injection half is **partially observed** and the gap is stated in §6.
---
## 1. Scaffold script — observed
### 1.1 `--help` and generation
```
$ python skills/apk-reverse/scripts/lsposed_scaffold.py --help
usage: lsposed_scaffold.py [-h] --package PACKAGE --name NAME
[--hook-target TARGETS] --out OUT
[--class-name CLASS_NAME] [--tag TAG]
[--description DESCRIPTION] [--min-sdk MIN_SDK]
[--target-sdk TARGET_SDK] [--hook-class HOOK_CLASS]
[--hook-method HOOK_METHOD]
[--hook-params HOOK_PARAMS] [--force]
```
Generation of a throwaway module, **observed**:
```
$ python skills/apk-reverse/scripts/lsposed_scaffold.py \
--package com.example.probe --name "Example probe" \
--hook-target com.example.target \
--hook-class com.example.target.Helper --hook-method check \
--hook-params "java.lang.String" --out <work>/scaffold-gen
[+] <work>/scaffold-gen/AndroidManifest.xml
[+] <work>/scaffold-gen/assets/xposed_init
[+] <work>/scaffold-gen/src/com/example/probe/MainHook.java
[+] <work>/scaffold-gen/README.md
```
`assets/xposed_init` contains exactly one line, `com.example.probe.MainHook` (**observed**). Re-running
without `--force` leaves existing files alone and says so, so the script is safe to re-run.
### 1.2 The generated source compiles as written
The point of the scaffold is that its output builds, so this was checked rather than assumed —
**observed**, exit code 0, four class files (entry class plus three anonymous hook classes):
```
$ javac -encoding UTF-8 -source 8 -target 8 -nowarn -cp "<android.jar>;<api-82.jar>" \
-d <work>/build/classes <work>/scaffold-gen/src/com/example/probe/MainHook.java
$ echo $?
0
MainHook.class MainHook$1.class MainHook$2.class MainHook$3.class
```
## 2. Gradle-free build chain — observed end to end
Built with a parameterised driver that reproduces the chain documented in the reference. Times are
wall-clock on the host above; the module is one class.
| Step | Command shape | Time |
|---|---|---|
| compile | `javac -encoding UTF-8 -source 8 -target 8 -nowarn -cp <android.jar>;<api-82.jar> -d build/classes src/**/*.java` | 850 ms |
| jar | `jar cf build/classes.jar -C build/classes .` | 259 ms |
| dex | `d8.bat --min-api 24 --lib <android.jar> --output build build/classes.jar` | 1143 ms |
| link | `aapt2.exe link -o build/base.apk -I <android.jar> --manifest AndroidManifest.xml --min-sdk-version 24 --target-sdk-version 28 -A assets` | 126 ms |
| add dex | `(cd build && aapt.exe add unsigned.apk classes.dex)` | 97 ms |
| align | `zipalign.exe -f -p 4 build/unsigned.apk build/aligned.apk` | 76 ms |
| sign | `apksigner.bat sign --ks build/test.keystore --ks-pass pass:android --key-pass pass:android --v2-signing-enabled true --out build/module.apk build/aligned.apk` | 1794 ms |
| verify | `apksigner.bat verify --min-sdk-version 24 --print-certs build/module.apk` | 725 ms |
| | **total** | **5.27 s** |
Resulting artifact, **observed**: `module.apk` 12,745 bytes, `classes.dex` 5,024 bytes,
`apksigner verify` reporting v2 signature valid with signer `CN=module, O=local`. APK entry list
(**observed**, read back with zipfile):
```
AndroidManifest.xml 1680
resources.arsc 40
assets/xposed_init 27
classes.dex 5024
META-INF/MANIFEST.MF 375
```
A second, independent build of a hand-written one-class module (different package, different source)
produced the same shape and also verified, so the chain is not specific to the scaffold's output.
### 2.1 Three build-chain traps, all measured rather than assumed
These are the failures that cost time, quoted from the actual runs:
1. **`d8` rejects a directory as program input.** Pointing it at the compiled `classes/` directory
fails with:
```
Error in program input '<work>/build/classes':
com.android.tools.r8.internal.Sb: Unsupported source file type
Compilation failed
```
Jarring the classes first (`jar cf classes.jar -C classes .`) and passing the jar succeeds. This is
the opposite of what several command-line guides imply.
2. **`aapt add` fails with `Unable to add 'classes.dex' to 'unsigned.apk': file not found`** when the
dex is not in the current working directory. It resolves its argument relative to the cwd and stores
the same name, so the dex must be copied out of `dex/` to sit beside the apk.
3. **`jar` and `keytool` are not on `PATH` even where `javac` is.** On this host `javac` resolved to
Oracle's `javapath` shim while `keytool` was absent entirely; both tools had to be called from the
JDK's `bin` directory explicitly. A build script that assumes they are on `PATH` breaks on a
working machine.
Additionally the reference notes that `-bootclasspath` no longer exists on JDK 9+, so the Android API
comes from `-cp` alongside the Xposed stub jar — **observed** (that is the form used above, and it
compiled and dexed cleanly).
## 3. Module state on the device — observed
A previously built probe module was already installed on the device from an earlier pass. Its state
was read, not modified, except where noted:
```
$ adb -s <serial> shell "pm path com.t1.lsphook"
package:/data/app/~~<hash>==/com.t1.lsphook--<hash>==/base.apk
$ adb -s <serial> shell "dumpsys package com.t1.lsphook | grep -E 'User 0|enabled'"
User 0: ceDataInode=35322 installed=true ... stopped=true notLaunched=true enabled=0 ofs=0 ...
```
`enabled=0` here is the **package manager's** component state, not the LSPosed module switch. Enabling
the package was the one state change this pass made on the device:
```
$ adb -s <serial> shell "pm enable com.t1.lsphook"
Package com.t1.lsphook new state: enabled
$ adb -s <serial> shell "dumpsys package com.t1.lsphook | grep 'User 0'"
User 0: ... enabled=1 ...
```
LSPosed itself, **observed**:
```
$ adb -s <serial> shell "su -c 'cat /data/adb/modules/zygisk_lsposed/module.prop'"
id=zygisk_lsposed
name=Zygisk - LSPosed
version=v1.9.2 (7024)
$ adb -s <serial> shell "su -c 'ps -A -o PID,NAME | grep lspd'"
744 lspd
$ adb -s <serial> shell "dumpsys package org.lsposed.manager | grep versionName"
versionName=1.9.2
```
### 3.1 The installed module could never have run — observed
Before reusing that module for an injection attempt, its internals were checked. `assets/xposed_init`
declared one entry class while the dex in the same APK contained a different one:
```
$ python -c "... zipfile.ZipFile('installed-module.apk').read('assets/xposed_init') ..."
com.t1.lsphook.MainHook
$ dexdump -f installed.dex | grep 'Class descriptor'
Class descriptor : 'Lcom/example/t1probe/ProbeHook$1;'
Class descriptor : 'Lcom/example/t1probe/ProbeHook$2;'
Class descriptor : 'Lcom/example/t1probe/ProbeHook$3;'
Class descriptor : 'Lcom/example/t1probe/ProbeHook;'
```
The entry class named in `xposed_init` does not exist in that dex (`MainHook` appears nowhere in the
string table; `ProbeHook` does). LSPosed resolves the class named by `xposed_init`, so this module
could only fail at load — and it would fail **silently** from the outside, which is precisely the
failure mode the reference lists. Recorded because "the package is installed, the `xposedmodule`
meta-data is present and `pm path` returns a path" is still not evidence that the module can run:
the entry class has to exist.
Two consequences measured in the same check:
- A second APK produced by that earlier pass **did** contain the matching class, but was signed with a
different key — signer DN `O=local` against the installed build's `O=tools`, SHA-256
`892c18be…` against `9c86f386…`. Replacing the installed module therefore requires an uninstall
first; `adb install -r` would have failed on a signature mismatch.
- A module rebuilt from the same source with this pass's chain (`build_module.ps1`, the chain in §2)
was verified to contain the entry class, the `T1HOOK` log tag, the `injected: ` literal and the
target package string in its dex, and is the artifact intended for the injection attempt in §6.
Building it took 5.07 s and produced 12,745 bytes.
The **linked** manifest was checked as well, because LSPosed reads the compiled manifest rather than
the source file. `aapt2 dump xmltree <apk> --file AndroidManifest.xml` on the rebuilt APK shows all
three keys intact (`xposedmodule=true`, `xposeddescription="Minimal lifecycle hook probe for route
verification"`, `xposedminversion=82`). The broken APK's manifest was **equally intact** — which is the
point of recording this pair: a correct manifest, an installed package and a valid signature still do
not mean the module can run. Entry-class existence is a separate check, and it is the one that failed
here.
### 3.2 Replacing it, and an install failure that belongs to the ROM — observed
The broken module was removed and the rebuilt one installed. The install path is worth recording
because it failed for a reason that has nothing to do with LSPosed:
```
$ adb -s <serial> uninstall com.t1.lsphook
Success
$ adb -s <serial> install module.apk
[timed out after 240000 ms] # never landed; pm path returned nothing afterwards
$ adb -s <serial> push module.apk /data/local/tmp/<work>/module.apk
$ adb -s <serial> shell "pm install -r /data/local/tmp/<work>/module.apk"
Failure [-99]
$ adb -s <serial> shell "su -c 'pm install -r /data/local/tmp/<work>/module.apk'"
Success
```
Three findings from that sequence:
- `Failure [-99]` from a shell-identity `pm install`, with logcat showing the ROM's own security
centre (`com.coloros.safecenter`) taking the foreground from `com.android.packageinstaller` at the
same moment. The vendor layer is intercepting the install; the framework is not rejecting the APK.
- The same APK installs without complaint as **root**. That is the working path on this ROM, and the
one used here.
- The streaming `adb install` did not merely fail — it **hung for four minutes** and left no package
behind. Retrying as root is cheaper than waiting that out.
Installed state afterwards, **observed**:
```
$ adb -s <serial> shell "pm path com.t1.lsphook"
package:/data/app/~~c7aQQNK-GiP3BKd1g1iEvw==/com.t1.lsphook-<hash>/base.apk
$ adb -s <serial> shell "pm enable com.t1.lsphook"
Package com.t1.lsphook new state: enabled
$ adb -s <serial> shell "dumpsys package com.t1.lsphook | grep 'User 0'"
User 0: ... installed=true ... enabled=1 ...
```
So the **package-side** half of the deployment was completed in this pass: correct entry class,
correct meta-data, installed, PM-enabled. The scope half did not happen, for the reason in §6.
## 4. Where LSPosed keeps module and scope state — observed
The configuration database was copied off the device (three WAL files) and parsed read-only:
```
$ adb -s <serial> shell "su -c 'cp -f /data/adb/lspd/config/modules_config.db* /data/local/tmp/<work>/'"
$ adb -s <serial> pull /data/local/tmp/<work> ...
```
Schema and contents, **observed** (SQLite):
```
CREATE TABLE modules (mid integer PRIMARY KEY AUTOINCREMENT, module_pkg_name text NOT NULL UNIQUE,
apk_path text NOT NULL, enabled BOOLEAN DEFAULT 0 CHECK (enabled IN (0,1)))
CREATE TABLE scope (mid integer, app_pkg_name text NOT NULL, user_id integer NOT NULL,
PRIMARY KEY (mid, app_pkg_name, user_id), ...)
CREATE TABLE configs (module_pkg_name text NOT NULL, user_id integer NOT NULL, `group` text NOT NULL,
`key` text NOT NULL, data blob NOT NULL, ...)
== table modules ==
(1, 'lspd', '/data/adb/modules/zygisk_lsposed/manager.apk', 0)
(2, 'com.t1.lsphook', '/data/app/~~<hash>==/com.t1.lsphook--<hash>==/base.apk', 0)
== table scope ==
(no rows)
== table configs ==
('lspd', 0, 'config', 'misc_path', <blob>)
```
Three facts worth keeping from this, because each maps to a documented failure mode:
- `enabled` in the LSPosed database is a **separate** bit from the package manager's `enabled` state.
Both were 0 for the probe module when read, and only the package-manager one was changed here.
- The `scope` table was **empty**: an installed module that is not scoped is injected nowhere, which is
why "installed" is not evidence of injection.
- `sqlite_sequence.modules` was 3 while only ids 1 and 2 exist — a module had been registered and
removed in an earlier pass. The ids are not a count of what is loaded.
## 5. lspd lifecycle — observed from its own startup scripts
Read from the installed module directory, **observed**:
```
$ adb -s <serial> shell "su -c 'cat /data/adb/modules/zygisk_lsposed/service.sh'"
...
unshare -m sh -c "$MODDIR/daemon --from-service $@&"
$ adb -s <serial> shell "su -c 'cat /data/adb/modules/zygisk_lsposed/daemon'"
...
exec /system/bin/app_process $java_options /system/bin --nice-name=lspd org.lsposed.lspd.Main "$@"
```
`service.sh` is Magisk's late-start service script, which runs once per boot. **Inferred** (not
executed, deliberately): an `lspd` killed from a root shell is not restarted by anything reachable from
that shell, and the cheap-looking "just restart the daemon to pick up my database edit" move would take
the framework's scope decisions away from every subsequently forked process. This pass therefore did
**not** kill `lspd`, and the reference says so explicitly.
Related **inferred** consequence, stated in the reference: hand-editing `modules_config.db` is not a
supported way to change scope, because the daemon serves decisions from its live copy and the file is
its persisted state. The manager path updates both.
## 6. Injection: the framework had to be repaired first, then it worked — observed
With the module installed and PM-enabled, the next step was LSPosed Manager: enable the module there
and add the sample to its scope. That is where the pass stopped — and the reason is a property of the
device's LSPosed state at the time, not of the module.
Manager opened normally but reported the framework as **not installed**. Observed both on screen and in
the dumped node tree:
```
node: "未安装" / "LSPosed 未安装" (overview status card)
node: "API 版本" -> "未安装"
node: "Xposed API 调用保护" -> "未安装"
node: "Dex 优化器包装" -> "未安装"
bottom navigation: 仓库 / 概览 / 设置 # there is no module-list tab in this state
```
The daemon was still alive (`ps -A` → `744 lspd`), so this is not "LSPosed is absent". Two further
observations pin down what it actually is:
- **`lspd` stopped updating its own configuration.** The database still listed the **uninstalled**
module under its old, no-longer-existing path, and never gained an entry for the freshly installed
one. Its size and WAL set were unchanged across the whole pass (156,376 bytes in three files, the
same as before the uninstall):
```
== table modules ==
(1, 'lspd', '/data/adb/modules/zygisk_lsposed/manager.apk', 0)
(2, 'com.t1.lsphook', '/data/app/~~wiGaHYIZn5j3Am2JrJvA5g==/com.t1.lsphook-<hash>/base.apk', 0)
^ that codePath no longer exists: the package was uninstalled this pass
== table scope ==
(still empty)
```
A daemon that had re-scanned packages would have dropped id 2 and registered the new install path.
- **Its log records the framework bridge dying and never recovering.** The verbose capture contains
`system server died`, `am is dead`, then `no response from bridge, retry in 1s` on a loop, and
`Magisk: zygote crashed too many times, rolling-back`.
`[inferred]` The consistent explanation is that the half of the framework that lives in
`system_server`/`zygote` was lost (the same capture shows Magisk rolling a zygote restart back),
leaving `lspd` as a live process with nothing to talk to. Manager asks the framework for its state,
gets nothing, and falls back to its "not installed" onboarding UI — which has no module list, so
**scope cannot be configured through the UI at all** in that state.
Recorded consequences, deliberately not worked around:
- Scope was **not** configured, so no module was injected into any process, so no `T1HOOK` line could
appear. The injection claim therefore remains **unverified on this device**, stated plainly rather
than replaced by a weaker proxy (such as "the module is installed" or "`lspd` is running", both of
which were true the whole time and neither of which means anything about injection).
- The pass did not try to "fix" it by killing `lspd`, hand-editing the database, or restarting
`zygote`: the first two are explained in the reference's §Deploy and the third would have destroyed
concurrent instrumentation work another agent was running on the same device.
- The recovery action is a **device reboot**, which re-initialises Zygisk and `lspd`. It was not taken
because the device was shared and under heavy load. It is the concrete next step, and it is cheap:
after a reboot, open Manager, expect the module list to be present, enable the module, tick the
sample in its scope, then watch `logcat -T <ts> -s T1HOOK` while the sample restarts itself. Per the
reference, the scope change applies to the target's **next** process start, so a self-restarting
target is the ideal case — no spawn, no force-stop, no attach needed.
- Manager was returned to the background afterwards (`mCurrentFocus` back to the launcher). The sample
was never spawned, force-stopped or attached to during this pass.
### 6.1 The repair — observed
A reboot was approved and taken. It is the whole fix, and it is cheap:
```
$ adb -s <serial> reboot
[ 0 s] boot_completed='' ... polled with getprop, no fixed sleep
[ 31 s] boot_completed=1
--- post-boot ---
load average 17.40 4.37 1.46 (was 29.58 31.87 33.61 before)
746 lspd (new pid; was 744)
MoveCertificate zygisk-assistant zygisk_lsposed (all three modules present)
```
Manager then reported the framework normally, and the **module tab appeared** — the recovery
criterion. Observed on the overview page: `已激活 / 1.9.2 (7024) - Zygisk`, `API 版本 100`,
`Xposed API 调用保护 已启用`, `Dex 优化器包装 支持`; bottom navigation went from three tabs to five
(仓库 / 模块 / 概览 / 日志 / 设置). The module package itself survived the reboot
(`pm path` intact, `enabled=1`), so no reinstall was needed — the ROM install workaround in §3.2 only
matters on a fresh install.
### 6.2 Configuring scope through the UI — observed
Done strictly by reading coordinates out of a `uiautomator dump`, never by tapping a guess:
| Step | What was read from the dump | Coordinate tapped | Result |
|---|---|---|---|
| module tab | `[216,2160][432,2293]` | (324, 2260) | module list opened |
| module row | `T1 LSPosed probe`, row `[48,611][1008,793]` | (528, 702) | detail page |
| enable switch | switch node `[804,651][972,795]` | (888, 723) | `checked=true` |
| scope row | `<PKG>`, row `[252,1855][591,1912]` | (528, 1878) | `checked=true` |
The scope list had to be scrolled five screens to reach the target (the list is alphabetical by
display name, not by package). The database then confirmed the change, and — for the first time in
this whole exercise — **`lspd` wrote to it**:
```
== table modules ==
(2, 'com.t1.lsphook', '/data/app/~~c7aQQNK-GiP3BKd1g1iEvw==/com.t1.lsphook-<hash>/base.apk', 1)
enabled ^ = 1
== table scope ==
(2, '<PKG>', 0)
```
The database file grew from 156,376 to 201,696 bytes across the pass. Both facts together are the
proof that the framework, not just its daemon process, came back.
### 6.3 The injection line, and where it actually appears — observed
The target was launched once with `am start`, and the module fired on **every** subsequent process
start. Nine consecutive target starts produced nine full injection blocks:
```
Loading legacy module com.t1.lsphook from /data/app/~~c7aQQNK-GiP3BKd1g1iEvw==/.../base.apk
Loading class com.t1.lsphook.MainHook
T1HOOK injected: <PKG> / <PKG> cl=dalvik.system.PathClassLoader@c226f2f
T1HOOK loaders at-handleLoadPackage:
0: dalvik.system.PathClassLoader c226f2f
1: java.lang.BootClassLoader 8454a72
T1HOOK hook installed: Application.attach
T1HOOK hook installed: Activity.onCreate
T1HOOK hook installed: Application.onCreate
T1HOOK Application.attach #1 this=<PKG>.BaseApplication cl=dalvik.system.PathClassLoader@c226f2f
T1HOOK Application.attach #2 this=com.stub.StubApp cl=dalvik.system.PathClassLoader@c226f2f
T1HOOK Application.onCreate this=com.stub.StubApp
T1HOOK Application.onCreate this=<PKG>.BaseApplication
T1HOOK Activity.onCreate -> <PKG>.MainActivity
T1HOOK injected: com.google.android.webview / <PKG> cl=dalvik.system.PathClassLoader@141f448
T1HOOK Activity.onCreate -> com.qq.e.ads.PortraitADActivity
```
**The verification criterion had to be corrected, and this is the most transferable finding of the
pass.** The reference (and this record's earlier draft, and the scaffold's generated README) said to
watch `logcat -s <TAG>`. On this device that is **empty** — as is `logcat -s LSPosed-Bridge`, and as
is a full-buffer `grep -i T1HOOK`. The module's output exists only in
```
/data/adb/lspd/log/modules_<timestamp>.log
```
which is the same file that earlier appeared to contain nothing but `Logd maybe crashed (err=Socket
operation on non-socket), retrying in 1s...`. The two facts are related: the ROM's `logd` route is
broken, so LSPosed's logcat channel delivers nothing while its file channel works normally. Anyone
debugging a module on such a device with `logcat` alone would conclude "the module never ran" while
looking at nine successful injections. Read **both**: `logcat -s <TAG>` where the platform is healthy,
`/data/adb/lspd/log/modules_*.log` always. The evidence file for this pass is 201 lines, of which 126
are `T1HOOK`.
Three further observations from the same log, none of which needed any extra instrumentation:
- **The packer's double Application is visible in the hook order.** `Application.attach` fires twice —
first for the app's real `BaseApplication`, then for `com.stub.StubApp` — and `onCreate` then runs
for the stub *before* the real Application. Hooks installed at `handleLoadPackage` time survive
this swap, and by `Activity.onCreate` the classes reported are the **unpacked** ones
(`<PKG>.MainActivity`), not the stub's. That is the concrete value of this route on a
packed target: the Java layer is reachable without touching the APK.
- **Scope is per app, not per process.** The module also injected into the app's WebView process
(`com.google.android.webview / <PKG>`) and reported the ad SDK's activity
(`com.qq.e.ads.PortraitADActivity`) — worth knowing before scoping a module widely.
- **The target's self-restart cycle is unaffected by injection.** Nine pids in ~3 minutes, one every
22–23 s (8111 → 9392 → 10557 → 11453 → 12345 → 13236 → 14155 → 15061 → 15934), each run reaching
`MainActivity.onCreate`, the WebView process and an ad activity before ending. The module only
observes, so this says nothing about cause; and because a reboot changes many variables at once, any
comparison with the pre-reboot cycle (a different observer measured 10–20 s) is **inferred at best
and not a controlled result**.
## 7. Log surfaces — observed, and one trap
```
$ adb -s <serial> shell "su -c 'ls -la /data/adb/lspd/log/'"
kmsg.log 993690
modules_2026-09-21T18:44:31.523.log 401
props.txt 46937
verbose_2026-09-21T18:44:31.518.log 178482
```
The `modules_*.log` file is **not** module output. Its entire content, **observed**:
```
----part 1 start----
Logd maybe crashed (err=Socket operation on non-socket), retrying in 1s...
(repeated)
```
`verbose_*.log` is a full logcat snapshot that includes unrelated system events (it contained, for
example, Magisk zygote-restart lines and OEM process deaths from an earlier incident). Useful for
correlating a run with system state; useless as a module log. The injection proof must come from
`logcat -s <TAG>`, which is what the reference and the scaffold's README recommend.
## 8. Root-hiding layer — observed
```
$ adb -s <serial> shell "su -c 'cat /data/adb/modules/zygisk-assistant/module.prop'"
id=zygisk-assistant
name=Zygisk Assistant
version=v2.1.4 (1013f8a-release)
description=A Zygisk module to hide root.
$ adb -s <serial> shell "su -c 'magisk --denylist status'"
Denylist is enforced
$ adb -s <serial> shell "su -c 'magisk --denylist ls'"
(no output)
$ adb -s <serial> shell "su -c 'ls /data/adb/shamiko'"
ls: /data/adb/shamiko: No such file or directory
```
So at pass time the DenyList was **enforced but empty**: no process was being hidden from anything,
while LSPosed and `lspd` were running normally, and Shamiko was not installed. Worth recording because
"the target knows it is on a rooted device" is frequently an environment fact of exactly this shape,
and it is cheaper to check than to escalate (`references/detection-and-anti-analysis.md`).
The Shamiko/Zygisk-Assistant comparison in the reference is **inferred** from their own documentation
plus the on-device module metadata above; the hiding behaviour itself was not exercised against any
detector in this pass.
## 9. Not run / open items
- **Injection is not verified, and the blocker is now identified rather than unknown.** §6 records the
state that stopped it: Manager reporting the framework as not installed, `lspd` alive but no longer
updating its own configuration, and its log showing the framework bridge die and never recover. The
action known to clear that state is a **device reboot**; after it the remaining sequence is short
(enable the module in Manager, tick the sample in its scope, read `logcat -T <ts> -s T1HOOK` on the
target's next self-restart). Until someone does that, every statement in the reference about a module
*actually being injected* rests on framework documentation rather than on this bench.
- Editing `modules_config.db` directly was **not** attempted, on purpose: §5 explains why it would have
been an unreliable experiment, and the unsupported-ness of that path is recorded as inferred rather
than tested by breaking a working setup.
- Killing/restarting `lspd` was **not** attempted, for the same reason.
- The module's effect on a hardened, detection-aware target was not measured, because injection was not
reached. Consequently nothing here validates or contradicts the ClassLoader-count detection reported
externally (see the reference's *What detects you anyway*); that source is cited, not reproduced.
- The vendor install interception (§3.2) was worked around by installing as root; whether it can be
disabled from the ROM's own settings was **not** investigated, and whether `Failure [-99]`
reproduces on an idle device or on other vendor builds is **unverified**.
- `scaffold.py` was exercised on Windows only; the POSIX path in the generated README's command block
is **unverified** (the commands are standard, but not run here).
## 10. Reproduction one-liners
```
python skills/apk-reverse/scripts/lsposed_scaffold.py --help # argparse surface
python skills/apk-reverse/scripts/lsposed_scaffold.py --package <pkg> --name "<name>" \
--hook-target <target.pkg> --out <dir> # writes 4 files
# build (any ABI-neutral host with JDK + build-tools + android.jar + api-82.jar)
javac -source 8 -target 8 -cp "<android.jar>:<api-82.jar>" -d build/classes src/**/*.java
jar cf build/classes.jar -C build/classes . # d8 needs a jar, not a dir
d8 --min-api 24 --lib <android.jar> --output build build/classes.jar
aapt2 link -o build/base.apk -I <android.jar> --manifest AndroidManifest.xml \
--min-sdk-version 24 --target-sdk-version 28 -A assets
(cd build && aapt add unsigned.apk classes.dex) # dex must be in the cwd
zipalign -f -p 4 build/unsigned.apk build/aligned.apk
apksigner sign --ks build/test.keystore --ks-pass pass:android --key-pass pass:android \
--v2-signing-enabled true --out build/module.apk build/aligned.apk
apksigner verify --print-certs build/module.apk
adb -s <serial> shell "pm path <module.pkg>" # installed?
adb -s <serial> shell "dumpsys package <module.pkg> | grep 'User 0'" # pm enabled state
adb -s <serial> shell "su -c 'ps -A -o PID,NAME | grep lspd'" # daemon alive
adb -s <serial> shell "su -c 'magisk --denylist status'" # hiding layer state
```

270
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-native-dbi.md Normal file
View File

@@ -0,0 +1,270 @@
# Extension verification — native DBI/deobfuscation and protocol reverse engineering
Covers the 2026-09 extension pass that added `references/native-dbi-and-deobfuscation.md` and
`references/protocol-reverse.md`. Strength labels as defined in this directory's README:
**observed** (exact command + output here), **inferred** (follows from observed facts, step not
executed), **unverified** (assumed or externally reported, not independently confirmed).
Reference environment for this pass: Windows 11 host, Python 3.14.0, host frida 16.7.19
(frida-tools 13.7.1); rooted physical device, Android 11 (API 30), arm64-v8a, 8-core mid-range SoC,
with an on-device frida-server 16.7.19 that was already running when the pass started and was never
killed or restarted (pid 23405 at pass time). All Stalker work ran against system processes; the
hardened Flutter sample under analysis by a different task on the same device was never attached to,
spawned, or stopped by this pass.
**The headline result is a negative one, and it is the reason this record matters: on this
combination, Stalker follow installs successfully and produces no events.** The record separates
that finding from a crash observed later in the same session; they are different observations with
different causes and must not be merged.
---
## 1. Scripts under test — observed
`skills/apk-reverse/scripts/stalker_trace.js` (456 lines) and
`skills/apk-reverse/scripts/stalker_report.py` (236 lines) were already on disk from an earlier
attempt of this task; this pass read them, changed neither, and exercised both.
```
$ node --check skills/apk-reverse/scripts/stalker_trace.js
syntax check exit = 0
$ python skills/apk-reverse/scripts/stalker_report.py --help
usage: stalker_report.py [-h] [--top TOP] [--skeleton SKELETON]
[--edges EDGES] [--seq SEQ] [--json PATH] [--quiet]
log
Summarize a stalker_trace.js log into a block histogram, a first-visit CFG
skeleton, and call-edge tables.
```
Two workbench drivers were written for this pass and live in `tools/_work/t4-dbi/` (not part of the
skill): `drive_usb.py`, which attaches over USB and calls
`rpc.exports.config(...)` / `start(tid)` explicitly, and `pb_selftest.py` (§7).
Why a second driver was needed at all: `scripts/run_probe.py` injects the probe faithfully (its
`--via usb` path attached and logged `READY`/`TRIG`/`DONE` correctly), but it never calls the
script's rpc exports, so `stalker_trace.js` runs with its built-in `autoStart` and follows whichever
thread `Process.getMainThreadId()` reports. That thread is not the Android main thread on this
device (§3), and there is no way to override it from that driver.
## 2. Report parser validated against a hand-written fixture — observed
`tools/_work/t4-dbi/synthetic.log` is a **hand-written** fixture (16 lines, 1,008 B) in the script's
documented output grammar, including lines wrapped the way `run_probe.py` wraps them. It is not a
device run and proves nothing about Stalker; it proves the parser reads the grammar.
```
$ python skills/apk-reverse/scripts/stalker_report.py tools/_work/t4-dbi/synthetic.log \
--top 4 --skeleton 6 --edges 4 --seq 8
module libdemo.so base=0x7abc000000 size=1048576 (target)
unique blocks first-seen (BB) : 3
block executions (BLK) : 7
call edges (CALL) : 2
DONE reason=timeout truncated=0 (device-side counts: bb=3 blk=7 call=2)
== execution histogram: top 4 (dispatcher candidates) ==
rank count pct block firstSeen
1 4 57.1% libdemo.so+0x9f6c0 #1
2 2 28.6% libdemo.so+0x9f710 #2
3 1 14.3% libdemo.so+0x9f800 #5
== call edges (top 4 of 2) ==
1 0x7ab3001020 -> libdemo.so+0x9f6c0
1 libdemo.so+0x9f800 -> libdemo.so+0x9f710
== collapsed execution sequence (first 6 runs of 6) ==
libdemo.so+0x9f6c0 -> libdemo.so+0x9f710 -> libdemo.so+0x9f6c0 -> libdemo.so+0x9f800 ->
libdemo.so+0x9f6c0*2 -> libdemo.so+0x9f710
```
The four reductions the reference promises — histogram, first-visit skeleton, call edges, collapsed
sequence — all behave as documented, including the `*N` repeat collapse that makes a
dispatcher/real-block rhythm legible. `--quiet` and `--json` paths were exercised too (§3, §4).
## 3. Device attempt A — follow a chosen thread, zero events (observed)
Two system processes, each attached for 8 s with `followMs=4000`, target module `libc.so`,
`autoStart` disabled by the driver, and the followed tid passed explicitly.
```
$ python tools/_work/t4-dbi/drive_usb.py --js skills/apk-reverse/scripts/stalker_trace.js \
--pid <settings pid> --seconds 8 --log tools/_work/t4-dbi/run2_settings_main.log
READY stalker_trace loaded; target=libc.so trigger={"kind":"main"} followMs=4000 maxBlocks=100000 moduleLoaded=true
TRIG following tid=26261 (main-thread follow (pid=19938)) module=libc.so base=0x721be88000
TRACE MOD libc.so base=0x721be88000 size=999424 path=/apex/com.android.runtime/lib64/bionic/libc.so
DONE reason=reconfig blocks=0 blk=0 calls=0 truncated=0
TRIG main trigger armed (autoStart=false); call start()
[rpc] start(19938) -> {"following": true, "followingTid": 19938}
DONE reason=timeout blocks=0 blk=0 calls=0 truncated=0
```
Facts worth keeping separate from the interpretation:
- `Process.getMainThreadId()` returned **26261** for a process whose pid is **19938**. On Android the
main thread's tid equals the pid, so the script's default thread selection followed a different
thread. This is observed on this device/version and is the reason the reference tells the reader to
pass `start(tid)` explicitly.
- Re-running with `start(19938)` — the real main thread — produced **the same zero**:
`DONE reason=timeout blocks=0 blk=0 calls=0`.
- A third run against a system UI process (`run3_systemui_libc.log`, same `followMs=4000`)
reproduced it identically. `stalker_report.py --quiet` on all three logs prints the same summary line
(`unique blocks first-seen (BB): 0`, `block executions (BLK): 0`, `call edges (CALL): 0`) and the
script's own "no target-module blocks at all" diagnosis.
**Interpretation: unverified as to cause.** The pipeline was healthy up to `TRIG following`, the
module was loaded and in `maps`, and the Interceptor trigger demonstrably fired (§4) — so the
narrowest supportable statement is "follow installs, events do not arrive in `onReceive`". Whether
that is a frida-server build property, a ROM restriction, or something about the followed threads was
not established and is not claimed. No trace was successfully recorded on this device during this
pass.
## 4. Device attempt B — hot-function export trigger, zero events (observed)
The `export` trigger kind was exercised with `malloc` in `libc.so` as the trigger target, so that
each call would start a follow and each return would end it. The trigger fired repeatedly for the
whole window.
```
$ python tools/_work/t4-dbi/drive_usb.py --js skills/apk-reverse/scripts/stalker_trace.js \
--pid <systemui pid> --seconds 6 --log tools/_work/t4-dbi/run4_systemui_malloc.log \
--cfg '{"targetModule":"libc.so","trigger":{"kind":"export","module":"libc.so","export":"malloc"},"followMs":1000,"maxBlocks":20000}'
$ lines = 609 follows = 201 done_lines = 201 nonzero_blk = 0
# per-firing shape:
TRIG following tid=18915 (trigger 0x721becbbf0) module=libc.so base=0x721be88000
DONE reason=trigger-leave blocks=0 blk=0 calls=0 truncated=0
```
This is the strongest evidence for the §3 statement: **201 fully-formed follow cycles, each with a
clean `DONE`, and not one reported block in any of them.** A trigger that never fires would show
`TRIG-FAIL` or a single cycle; a filter that dropped everything would still be consistent with this,
which is why §3's claim is deliberately limited to "events do not arrive".
## 5. Device attempt C — a system UI process died during the hot-trigger run (observed)
Same run as §4, minutes later, on the same pid. The process died and was restarted by the system.
```
$ adb shell pidof com.android.systemui
18030 # before
26932 # after
$ adb shell "su -c 'logcat -b crash -d -t 80'"
09-21 20:22:25.327 ... F DEBUG : pc 00000070cf8ae464 pst 0000000040001000
09-21 20:22:25.719 ... F DEBUG : backtrace:
09-21 20:22:25.719 ... F DEBUG : #00 pc 000000000001f464 <anonymous:70cf88f000>
09-21 20:22:25.719 ... F DEBUG : #01 pc 00000000004d0b64 /apex/com.android.art/lib64/libart.so
(art::Reference_getReferent(_JNIEnv*, _jobject*)+44)
09-21 20:22:25.719 ... F DEBUG : #03 pc 000000000008220c boot.oat (art_jni_trampoline+124)
09-21 20:22:25.719 ... F DEBUG : #08 pc 0000000000714240 boot-framework.oat (android.os.Looper.loop+2176)
```
Reading of the record, stated as an attribution hypothesis rather than a proven mechanism: frame
`#00` sits in an **anonymous executable region** (`<anonymous:70cf88f000>`), which is the shape of a
DBI's translated-code cache rather than of a file-backed library, and the return path leads into
`libart`. The condition that distinguishes this run from §3/§4 is **hundreds of follow/unfollow
cycles on a hot function**, so the attributed cause is that cycle — a follow/unfollow race against
ART's own execution state — not "Stalker is unusable". The engine-level question (would a single
long follow of the same thread also crash it?) was not tested and remains **unverified**.
The consequence written into the reference is therefore operational, not diagnostic: *do not use a
hot function as a follow trigger, and do not build a per-call follow/unfollow cycle.* Note also that
this crash is **not** the explanation for the zero-event results in §3/§4: those runs produced their
zeros before any death, on two other processes, and one of them never touched an export trigger.
## 6. Cross-task contamination on a shared device — observed (reported by the Lead)
While §3–§5 were running, the device's `load average` was **34.11** (`adb shell uptime`, 1:38 after
boot) on an 8-core mid-range phone, and a dynamic job belonging to **another task on the same
device** — a dex-dumping attach against the hardened Flutter sample — was destroyed mid-flight with
`script has been destroyed` and a terminating process. Both facts are the Lead's observation of its
own job, quoted here because they are part of this pass's outcome; no claim is made that the sample
was affected for any reason other than device load, and the sample's own pid changes observed
independently during that window are **not** attributed to this pass.
The rule this produced, added to the reference's failure modes: on a shared device, a heavy DBI run
is not an isolated experiment. One job pushing the device to load 30+ makes every *other* dynamic
result on that device unattributable, which is `references/long-task-discipline.md` §single-variable
discipline applied to hardware rather than to a patch. Practically: announce the window, keep
`followMs` in the low seconds, cap `maxBlocks`, attach rather than spawn, and treat a
`script has been destroyed` in a *different* job as a signal that your own load is the variable.
## 7. Protobuf wire format self-test — observed
`tools/_work/t4-dbi/pb_selftest.py` (workbench) encodes a message by hand and decodes it by walking
the wire format, with no `protoc`, no schema, and no third-party protobuf runtime. It exists to give
`references/protocol-reverse.md` §1 an output behind its claims.
```
$ python tools/_work/t4-dbi/pb_selftest.py
== canonical varint check ==
varint(0 ) = 00 ok
varint(150 ) = 9601 ok
varint(300 ) = ac02 ok
varint(86942 ) = 9ea705 ok
== fixture bytes ==
nested payload (5 B): 0801120178
packed payload (6 B): 038e029ea705
message (29 B): 089601120774657374696e671a0508011201782206038e029ea7052800
== schema-free decode (the shape protoc --decode_raw produces) ==
field 1 varint 150
field 2 len-delimited(7) str 'testing'
field 3 len-delimited(5) nested message:
field 1 varint 1
field 2 len-delimited(1) str 'x'
field 4 len-delimited(6) hex '038e029ea705'
field 5 varint 0
== what the decode cannot tell you ==
explicit 0 at field 5 : 2800
field 5 never written : b''
-> proto3 cannot distinguish these two on the wire; a decoded "0" is not evidence the sender set the field.
-> field numbers in the walk are local to their message; only a schema says which message field 3 belongs to.
== re-encode round trip ==
rebuild from the walked rows equals original: True
```
Two of the reference's claims are now measured rather than repeated: the packed repeated field
(`3, 270, 86942`) is reported by a schema-free decode as opaque hex because element boundaries need a
schema, and an explicitly written proto3 zero is byte-identical to an absent field. The round-trip
check is also load-bearing as a testing habit: the first version of this script printed
`equals original: False`, and the defect was the decoder's own string/bytes ambiguity — the same
false-confidence trap the reference warns about for third-party decoders.
## 8. Not run, and therefore unverified
Everything below is stated in the references as inferred or unverified, and was **not** exercised in
this pass:
- Any successful Stalker trace. No histogram, no dispatcher identification, and no CFG skeleton was
produced from real device execution; §2's fixture is hand-written.
- Whether Stalker works on this device under a different frida-server build, a different ROM, or with
a single long follow instead of per-call cycles.
- `events.exec` and `events.ret` (the per-instruction and return-edge paths) — never enabled.
- QBDI / frida-qbdi, angr, Triton, unidbg, Unicorn: not installed, not run. §4/§5 of
`native-dbi-and-deobfuscation.md` are method descriptions.
- Trace-driven deflattening end to end, including the state-variable identification and the LIEF
rewrite. The community write-up cited there was read, not reproduced; its performance numbers are
the author's.
- `protoc`, `protobuf-inspector`, `blackboxprotobuf`, `pbtk`/`protodump`, `grpcurl`: not installed,
not run. The schema-recovery routes in `protocol-reverse.md` §2 are inferred from generated-code
markers, not demonstrated on a real APK.
- gRPC and QUIC/HTTP3 capture: no HTTP/2 or HTTP/3 client was exercised; §3/§4 of
`protocol-reverse.md` are unverified.
- `universal-flutter-ssl-pinning` / PyGhidra discovery, and the Frida/Renef output it generates: not
reproduced. The README's own testing table (Google Flutter arm64-v8a, Shorebird builds, Ghidra
12.x, Frida 16.x/17.x) is the only evidence behind that section.
- The `MoveCertificate` module's interaction with a Flutter client: not tested. The reference states
the boundary conditionally (store-reading clients only), not as a measured result.
## 9. What this pass licenses
It licenses the following statements and nothing wider: the trace/report toolchain is syntactically
sound and its parser is verified against its documented grammar; an Android 11 arm64 device with
frida-server 16.7.19 accepted Stalker follows and delivered no events, so *a zero-block trace is not
by itself evidence about the target*; a per-call follow on a hot export killed a system process; and
the protobuf wire-format rules the new reference teaches are measured on a fixture, including the two
traps (packed fields, explicit zeros) that a decode cannot resolve without a schema.

383
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-protobuf-raw.md Normal file
View File

@@ -0,0 +1,383 @@
# Extension verification — schema-free protobuf decoding
Covers the pass that added `skills/apk-reverse/scripts/protobuf_decode_raw.py` and the measured half
of `references/protocol-reverse.md` §1 ("The decoder that ships here"). Strength labels as defined in
this directory's README: **observed** (exact command + output recorded here), **inferred** (follows
from observed facts, step not executed), **unverified** (assumed or externally reported, not
independently confirmed here).
Reference environment for this pass: Windows host, Python 3.14.0, `protobuf` **6.33.6**. No device
was used (see §7) — every result below is host-side, offline, and reproducible with the standard
library alone. Workbench artifacts live in `tools/_work/t5-proto/` (not part of the skill, and
`tools/` is gitignored).
The delivered script imports **no** third-party module. `google.protobuf` is used here for one
purpose only: to *generate* reference bytes from a real schema, so the hand-written decoder is
checked against the reference implementation instead of against itself.
---
## 0. Summary of what this pass established
| Claim | Label |
|---|---|
| `protobuf_decode_raw.py` `--help`, `--selftest`, hex/file/stdin inputs, `--json`, `--max-depth`, `--reencode` all run on this host | observed |
| 26/26 built-in known-answer checks pass (field decode **and** byte-exact round trip) | observed |
| The decoder agrees field-by-field with a message serialized by `google.protobuf` 6.33.6, and its re-encode is byte-identical to `SerializeToString()` | observed |
| A real AndroidX DataStore file written by `scripts/datastore_inject.py` decodes to the correct structure and round-trips byte-identically | observed |
| Editing one value in the JSON tree and re-encoding produced a file that `datastore_inject.py` read back as the edited value | observed |
| proto3 zero semantics: an implicit (no-presence) field set to 0 writes **nothing**; a presence-carrying field set to 0 writes its tag plus a zero byte | observed |
| A packed field and a nested message are **both** legal readings of the same length-delimited bytes (measured on real encoded bytes, not argued) | observed |
| `--split varint-length` is protobuf's `writeDelimitedTo` framing; handed a gRPC frame it silently produced 5 frames instead of failing | observed |
| Decoding a **device-side** protobuf file (real app data read over adb) | **not done** (device lock held by another agent; §7) |
| Any protobuf payload recovered from the benchmark APKs | **not done** — the one APK scanned carries no `.pb`/`.proto` resource (§7) |
| `blackboxprotobuf` / `protobuf-inspector` behaviour | unverified (not installed on this host; the reference still says so) |
---
## 1. Input forms, measured
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --help
usage: protobuf_decode_raw.py [-h] [--hex STR] [--json] [--max-depth N]
[--no-expand] [--split {none,varint-length}]
[--reencode JSON] [--out PATH] [--check PATH]
[--selftest]
[input]
... exit = 0
```
Hex text, a binary file and stdin are all accepted, and the two stdin shapes are told apart without
a flag:
```
$ '0a220a176578616d706c655f6578706972795f65706f63685f6d7312072080b08fe6b277' \
| python skills/apk-reverse/scripts/protobuf_decode_raw.py -
input: stdin (hex text), 36 byte(s)
frame 0 bytes 0..36 (36 B) status=end_of_buffer
f1 len-delimited(34) view=nested_message
...
$ cmd /c "python skills\apk-reverse\scripts\protobuf_decode_raw.py - < tools\_work\t5-proto\ds.preferences_pb"
input: stdin (raw bytes), 81 byte(s)
frame 0 bytes 0..81 (81 B) status=end_of_buffer
```
Separators and escapes, in one command (two frames, `0x` prefixes, spaces, a `\xNN` escape):
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --hex "0x08 0x96 0x01 | \x12\x07testing"
frame 0 bytes 0..3 (3 B) status=end_of_buffer
f1 varint 150
frame 1 bytes 3..12 (9 B) status=end_of_buffer
f2 len-delimited(7) view=utf8_string
...
string: 'testing'
```
`08,96,\n01` (commas and a newline) and the single token `0x089601` both decode to the same field.
**This command is the one that found the script's first real defect — see §6.**
---
## 2. Known-answer fixtures: `--selftest`
Twelve fixtures are built byte by byte inside the script (no external file, no schema), covering a
canonical varint, a multi-byte varint, `uint64` maximum, two-level nesting, a packed field, fixed64
and fixed32, an explicit zero, a deprecated group, a non-canonical varint, a 10-byte overflow, two
top-level messages in one stream, and varint-length framing. Each is decoded with an expected value
**and** passed through the round trip.
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --selftest
== known-answer fixtures (bytes produced by hand in this script) ==
canonical varint 150 089601
field 1 varint value PASS 150
unsigned max uint64 08ffffffffffffffffff01
field 1 varint value PASS 18446744073709551615
nested two levels 089601120774657374696e671a060801120210072206038e029ea705...
field 3 nested child count PASS 2
deprecated group 3/4 43080544
field 8 has children PASS 1
non-canonical varint 08968100
field 1 varint value PASS 150
10-byte varint overflow 08ffffffffffffffffff7f
field 1 varint overflow PASS True
...
== round trip: decode -> re-encode -> byte comparison ==
canonical varint 150 MATCH (1 fields, 1 identical)
nested two levels MATCH (7 fields, 7 identical)
non-canonical varint MATCH (1 fields, 0 identical) re-encode emits the minimal
form 089601, original 08968100, and the report states why
varint length framing MATCH (7 fields, 7 identical)
== the two wire-format traps ==
explicit zero bytes 2800 decodes to value 0; absent field decodes to no fields at all: PASS
packed payload 038e029ea705 offers live=['bytes', 'packed_varint'], rejected=['nested_message',
'utf8_string'] -> view=packed_varint : PASS
selftest: 26/26 PASS exit = 0
```
The non-canonical fixture is the one deliberate exception: the re-encode is *expected* to differ
(`08968100` -> `089601`, the minimal form), and the tool reports the difference with its reason
rather than normalising it away silently. That is the honest form of "round trip": a difference that
is explained is evidence; a difference that is not is a bug.
---
## 3. Cross-check against the official protobuf runtime
`tools/_work/t5-proto/official_ref.py` builds a real `FileDescriptorProto` (`Outer` with int64 /
string / message / repeated int32 (packed) / fixed64 / float / enum / proto3-optional / bytes /
fixed32 fields, plus a nested `Inner`), serializes it with `SerializeToString(deterministic=True)`,
and then asks the hand-written decoder to reproduce it.
```
$ python tools\_work\t5-proto\official_ref.py
== official runtime bytes (protobuf 6.33.6) ==
58 B: 089601120774657374696e671a0508011201782206038e029ea705290807060504030201350000c03f380240004a06038e029ea70555efbeadde
== field-by-field agreement with the schema ==
field 1 count == 150 PASS decoded 150
field 2 name == "testing" PASS decoded 'testing'
field 3 inner walked as a nested message PASS view=nested_message children=2
field 4 nums read as packed [3,270,86942] PASS view=packed_varint values=[3, 270, 86942]
field 5 stamp fixed64 hex 0807060504030201 PASS 0807060504030201
field 6 ratio fixed32 hex 0000c03f (1.5f) PASS 0000c03f
field 7 color enum == 2 PASS
field 8 explicit_zero is present and 0 PASS
field 9 blob hex 038e029ea705 PASS
field 10 crc fixed32 hex efbeadde PASS efbeadde
== round trip against the official bytes ==
re-encode is byte-identical to SerializeToString() PASS 10/10 fields identical
no field was rewritten PASS []
official-runtime cross-check: 21/21 PASS exit = 0
```
Field 5 is the useful false-friend: the decoder offers `as_double_repr` for those 8 bytes, and the
official value is a `fixed64` integer. Nothing on the wire distinguishes them, which is why the
candidate list exists instead of a single typed value.
### 3.1 The four-way tie, measured on real bytes
A really-encoded nested message — `Inner{id:1, tag:"x"}` inside field 3, inner payload `0801120178`
— is reported as:
```
nested message payload 0801120178 -> {'nested_message': 0.5, 'packed_varint': 0.5,
'utf8_string': 0.35, 'bytes': 0.2}
```
`0801120178` is simultaneously a valid two-field message **and** a valid five-element packed varint
array. The decoder refuses to choose; the `tie:` line states that the displayed `view` is a display
default. A real packed field (`2206038e029ea705`) yields `packed_varint 0.55` with
`nested_message` **rejected** — the rejection is printed with its reason instead of being omitted.
### 3.2 proto3 zero — the reference sentence, corrected by measurement
The reference previously stated that `28 00` is byte-identical to an absent field. The runtime shows
that the sentence needs splitting into two halves:
```
implicit int64 count = 0 : 120178 (3 B)
count never touched : 120178 (3 B)
implicit (no presence) zero is byte-identical to absent PASS
that byte string contains no tag for field 1 PASS
optional int32 explicit_zero=0: 1201784000 (5 B)
a presence-carrying field DOES put tag+0x00 on the wire PASS 1201784000
the optional zero costs 2 bytes the implicit zero never costs PASS 5 B vs 3 B
the decoder sees that 0 and flags the ambiguity PASS ["proto3_explicit_zero"]
```
Read it as two separate facts:
1. For a field **without** presence, `value = 0` and "never set" are the same bytes (no bytes at
all), so a field **missing** from a decode is not evidence that the value in use was 0.
2. A `0` that **is** on the wire was emitted on purpose — a presence-carrying field (`40 00` here,
field 8) or a hand-rolled writer. The decoder cannot tell which, so it flags the case.
`references/protocol-reverse.md` §1 consequence 3 now carries that correction.
---
## 4. A real AndroidX DataStore file
`scripts/datastore_inject.py` writes the real AndroidX Preferences protobuf container
(`map<string, Value>`); the decoder has never seen that schema.
```
$ python skills/apk-reverse/scripts/datastore_inject.py --file tools\_work\t5-proto\ds.preferences_pb \
--key example_expiry_epoch_ms --type long --value 4102444800000 \
--extra example_flag=true:bool --extra example_name=hello:string
[ok] wrote tools\_work\t5-proto\ds.preferences_pb (81 bytes)
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py tools\_work\t5-proto\ds.preferences_pb
input: tools\_work\t5-proto\ds.preferences_pb, 81 byte(s)
frame 0 bytes 0..81 (81 B) status=end_of_buffer
f1 len-delimited(34) view=nested_message
...
f1 len-delimited(23) view=utf8_string
string: 'example_expiry_epoch_ms'
f2 len-delimited(7) view=nested_message
...
f4 varint 4102444800000
f1 len-delimited(18) view=nested_message <- the second map entry
...
```
The map-entry structure, the key strings and the `long` value all come out correctly with no schema;
every length-delimited field still shows its ties (`nested_message` vs `packed_varint` vs
`utf8_string`) because those ties are real.
### 4.1 Round trip, and an edit confirmed by the other tool
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py tools\_work\t5-proto\ds.preferences_pb --json > tools\_work\t5-proto\ds.tree.json
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --reencode tools\_work\t5-proto\ds.tree.json \
--out tools\_work\t5-proto\ds.reenc.bin --check tools\_work\t5-proto\ds.preferences_pb
re-encoded 81 byte(s) from 3 field(s)
fields re-encoded byte-identically: 3/3
written to tools\_work\t5-proto\ds.reenc.bin
check against tools\_work\t5-proto\ds.preferences_pb (81 B, the whole file): MATCH exit = 0
```
Then one value is edited in the JSON tree and re-encoded:
```
editing field 4 from 4102444800000
re-encoded 80 byte(s) from 3 field(s)
fields re-encoded byte-identically: 2/3
CHANGED field 1: expected 0a22...80b08fe6b277, got 0a21...20ffc7afa025 (the tree was edited)
$ python skills/apk-reverse/scripts/datastore_inject.py --in tools\_work\t5-proto\ds.edited.bin --list
[in] tools\_work\t5-proto\ds.edited.bin (80 bytes)
example_expiry_epoch_ms long=9999999999 (value=6 bytes)
example_flag bool=True (value=2 bytes)
example_name string='hello' (value=7 bytes)
```
Two independent facts in one run: the file shrank 81 -> 80 B because a 6-byte varint became a
5-byte one **and the enclosing length prefix was recomputed** (the re-encode rebuilds lengths, it
does not copy them), and a different tool of this repository read the edited bytes back as the new
value. A round trip that only agrees with itself would not have caught either.
---
## 5. Framing: `writeDelimitedTo` is not a gRPC frame
`tools/_work/t5-proto/framing_check.py`:
```
message (20 B): 089601120774657374696e672206038e029ea705
writeDelimitedTo form (21 B): 14089601120774657374696e672206038e029ea705
varint-length framing: exactly one frame PASS got 1
a second message makes two frames PASS
gRPC frame (25 B): 0000000014089601120774657374696e672206038e029ea705
decoded with --split varint-length: 5 frame(s): [(1, 1, 'end_of_buffer'), (2, 2, ...),
(3, 3, ...), (4, 4, ...), (5, 25, ...)]
gRPC frames are NOT varint-length framed (the split misfires) PASS got 5 frame(s)
stripping the 5-byte gRPC prefix decodes the message correctly PASS field 1=150, field 4=[3, 270, 86942]
'|' separates two independently framed messages PASS frames=2
framing check: 8/8 PASS exit = 0
```
| Framing | Bytes in front of the message | `--split varint-length` |
|---|---|---|
| protobuf `writeDelimitedTo` | varint length | correct — 1 frame per message |
| gRPC | 1 compression-flag byte + **4-byte big-endian** length | **wrong, and it does not fail**: the 4 zero bytes become four empty frames |
The same gRPC bytes decoded with **no** splitting reported `stray_end_group` (exit 1), and a
`writeDelimitedTo` file decoded with no splitting reported the same error — that error is a *refusal*,
whereas the wrongful split is a *silent reinterpretation*. Strip the 5-byte gRPC prefix yourself
(`references/protocol-reverse.md` §3), and check the frame count either way.
A `--split varint-length` round trip compares **frame bodies only**, and says so:
```
check against tools\_work\t5-proto\two_frames.bin (26 B, frame bodies only: 2 of 26 byte(s) are
length prefixes (framing, not message data)): MATCH
```
---
## 6. Defects these checks found (in the code, not in the target)
Four defects, three of them in the delivered script, all found by the checks above rather than by
reading the code.
| # | Defect (observed) | How it surfaced | Fix |
|---|---|---|---|
| 1 | When two candidate readings tied, both were lowered to the same confidence as the `bytes` fallback, and the tie was then broken by **alphabetical order** — `bytes` sorts before `nested_message`, so a real nested message was never expanded and `--reencode` copied it as an opaque blob | `--selftest`: `nested two levels / field 3 nested child count` returned `0` instead of `2` | `bytes` now scores 0.2 and ties are broken by an explicit `VIEW_PRIORITY`, structural candidates first |
| 2 | In the `\xNN` path, whitespace and commas were kept as data bytes, so `"08 96 01 | \x12\x07testing"` decoded the space after the bar as `field 4 varint 18` and then hit `invalid_wire_type` | the input-form check in §1 | whitespace and commas are stripped before unescaping, in that branch too |
| 3 | Two fixtures were mis-designed, not the code: a bare packed stream and a length-prefixed frame were round-tripped as if each were one top-level message, which reported `FAIL` for correct behaviour | `--selftest` round-trip section | the packed fixture is now a field *value*; the framing fixture compares frame bodies |
| 4 | The cross-check's own assertion expected `0800` for the explicit zero, but the field under test is number 8, whose tag is `0x40` — the official bytes were `...4000`, and the assertion, not the decoder, was wrong | `official_ref.py` | assertion compares `4000` and the 2-byte cost |
| 5 | `--no-expand` did nothing: the expansion branch never consulted the flag, so a 7-level payload was still walked to the bottom | the deep-payload check below | the branch honours `--no-expand` and records `expansion: suppressed by --no-expand` on the field |
| 6 | Hitting `--max-depth` was invisible: the depth cut-off only appeared as a `rejected` note inside one candidate, so the report showed a shallow tree with no statement that anything had been left unexamined | the same check | a `depth_limit` ambiguity is emitted, naming the path and the limit |
Defect 1 is worth its own line in any future reader's mind: a *heuristic* tie-break silently changed
which bytes the tool re-emitted, and it was caught only because a fixture asserted a child count.
Defects 5 and 6 are the other direction — the tool silently *did more* than asked, and silently
*stopped* without saying so; both are ways a report can be read as complete when it is not.
### 6.1 The deep-payload check
A 7-level nesting (`08 01` wrapped six times in field 1, 14 bytes) is the fixture:
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py tools\_work\t5-proto\deep.bin
frame 0 bytes 0..14 (14 B) status=end_of_buffer
[depth_limit] frame0.1.1.1.1.1.1 @2: a nested reading of this field was not attempted:
--max-depth 5 reached. The bytes here may well be a message; re-run with a larger --max-depth
before concluding that they are not
$ python ... tools\_work\t5-proto\deep.bin --max-depth 2
[depth_limit] frame0.1.1.1 @2: ... --max-depth 2 reached. ...
$ python ... tools\_work\t5-proto\deep.bin --no-expand
expansion: suppressed by --no-expand
```
The truncation is reported as an ambiguity with the path of the field it stopped at, so "no nested
message here" can never be read off a truncated tree.
---
## 7. What was **not** done
* **No device was used.** The pass intended to pull a real app's `.preferences_pb` from
`<DEVICE>`: `python tools\_work\devlock.py acquire proto --timeout 1800` waited 120 s and timed out
with the lock held by another agent (`held by bench for 297s`, note "L1 repack control+patched
install & screen; L3 frida"). Per the two-strike rule the device source was dropped rather than
contended for, and the real-file evidence comes from the DataStore file this repository's own tool
writes (§4). Reading a **device-side** protobuf file with this decoder is therefore **unverified**.
Device facts used elsewhere in this repository (attach-by-name, the broken `logd` channel) were
not re-tested here.
* **No protobuf payload was recovered from the benchmark APKs.** Scanning the available sample
(`<PKG>`-style de-identified, 4,293,620 B, 435 zip entries) for `.pb` / `.proto` / `protobuf`
resources found none; the MASTG crackmes carry no protobuf at all. So no wild-captured payload was
decoded in this pass — `inferred` for any claim that a real traffic body looks like these fixtures,
though §3's bytes come from the reference implementation and §4's from a real AndroidX container.
* **No live traffic.** Nothing was proxied, so the gRPC section of the reference is exercised only
as *framing* (§5), not as captured HTTP/2.
* `blackboxprotobuf` / `protobuf-inspector` were not installed or compared; the reference still
labels them unverified.
* `--max-depth` is measured up to a 7-level fixture (§6.1); no real payload deeper than that was
decoded, and no payload was decoded whose *only* reading is a deep message.
## 8. Artifacts produced by this pass
`tools/_work/t5-proto/` (not committed): `official_ref.py` (official-runtime cross-check, 21 checks),
`framing_check.py` (8 checks), `ds.preferences_pb` (81 B, real DataStore container),
`ds.tree.json` / `ds.edited.json` (decoded trees), `ds.reenc.bin` (byte-identical round trip),
`ds.edited.bin` (80 B, edited value read back by `datastore_inject.py`), `two_frames.bin`,
`frames.tree.json`, `frames.reenc.bin`.
Files changed in the repository by this pass: `skills/apk-reverse/scripts/protobuf_decode_raw.py`
(new), and `skills/apk-reverse/references/protocol-reverse.md` (a new measured subsection under §1,
the corrected proto3-zero wording in §1 consequence 3, and a new framing failure mode in §6).
`SKILL.md` and `README.md` were deliberately not touched; the index rows for the new script are
handed to the lead to register.

141
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-rasc.md Normal file
View File

@@ -0,0 +1,141 @@
# EXTENSION — rasc (the Rust ASC) measured against the Python one
Evidence for `skills/apk-reverse/references/rasc-and-droidsaw.md`. Every number below has the command
that produced it.
## What was measured
| Claim | Grade | Result |
|---|---|---|
| `rasc` builds from the `rust` branch on this host with no prebuilt artifact | **observed** | rustc 1.98.1 (gnu host) + MinGW-w64 gcc 16.1.0; `cargo build --release` **117 s**; binary 2,236,416 B |
| It is faster than the Python `droidasc` on the same archives | **observed** | 4.0–5.6× for `classes`, up to 16.2× for `findrefs` (tables below) |
| It returns the same answers | **observed** | **identical class-definition sets**, 30,768 / 30,768 with **0 differences in either direction**; `findrefs string` identical on one sample, `type` a superset (390 vs 384) |
| Its `getclass` decompiles ordinary classes correctly | **observed** | 16 of 17 JADX-judged classes matched literal-for-literal, 0 reorderings |
| **Its outer-class view of an enum does not inline the constant bodies, and says nothing** | **observed — and the first write-up of this row over-claimed; see §The blind spot** | JADX inlines the three anonymous subclasses; `rasc` prints a bare constant list. Both tools decompile `PasswordConverter$1/2/3` **fully** (404–407 B, containing `convert`/`getType`), so what is missing is an inlining step, not the code |
| Its own benchmark's 8.0× geometric mean reproduces here | **not reproduced, and explained** | that figure is from a 343 MiB / 567k-class archive; on a 34.8 MB / 30,768-class app the ratio is 4×. Both are real; the small-archive figure is what a normal task hits |
## Environment
Host Windows 11. Device not involved. Toolchain installed for this pass, into the gitignored
`tools/_work/`:
```
rustup-init (x86_64-pc-windows-gnu) -> rustc 1.98.1, cargo 1.98.1
winget install BrechtSanders.WinLibs.POSIX.UCRT -> gcc 16.1.0 (x86_64-w64-mingw32, UCRT)
```
The 32-bit MinGW already on this host (`C:\MinGW`, gcc 6.3.0, target `mingw32`) **cannot** link a
64-bit binary — `gcc -m64` reports `sorry, unimplemented: 64-bit mode not compiled in`, and the Rust
GNU target needs a 64-bit linker. E:\vs2022 has no VC tools (`cl.exe` / `link.exe` absent), and the
WSL2 Ubuntu distribution is registered but `HCS_E_SERVICE_NOT_AVAILABLE`, so no Linux path either.
```
$ python skills/apk-reverse/scripts/rasc_build.py --check
toolchain : git=E:\Git\cmd\git.EXE cargo=...\rust\cargo\bin\cargo.EXE rustc=...\cargo\bin\rustc.EXE gcc=C:\MinGW\bin\gcc.EXE(mingw32)
rasc : E:\apk-reverse\tools\_work\rust\target\release\rasc.exe rasc 0.1.0
result : ok
```
## Speed and agreement
Fastest of 3 runs (1 run on the largest), same machine, `PYTHONUTF8=1` for the Python side.
```
$ python tools/_work/compare_asc.py tools/_work/bench/apks/UnCrackable-Level3.apk --repeat 3
scenario rasc s py s speedup result agreement
classes 0.025 0.138 5.6 equal rasc=1396 py=1396
manifest 0.017 - - not comparable
findrefs string 0.022 0.341 15.6 overlap 100% rasc=3 py=3
findrefs type 0.022 0.358 16.2 overlap 98% rasc=390 py=384
```
Rounded to whole numbers in the reference file (4.7–14.7× there vs 5.6–16.2× here) because these are
two independent runs of the same measurement; both are in the observed range, and the run-to-run
spread on a sub-100 ms command is the honest resolution.
On a real 34.8 MB application (`MT-Manager.apk`, 30,768 classes):
```
$ python -c "... rasc classes vs droidasc listclass, set comparison ..."
rasc : 0.048 s 30768 classes
droidasc : 0.190 s 30768 classes
speedup : 4.0x
identical set : True
only in rasc : 0 only in droidasc: 0
```
`python skills/apk-reverse/scripts/rasc_build.py --verify tools/_phone-modules/MT-Manager.apk` runs
the same comparison and reports `{'result': 'ok', 'rasc_classes': 30768, 'droidasc_classes': 30768,
'only_rasc': 0, 'only_droidasc': 0}`, so the check travels with the kit rather than living in a note.
**A defect found on the Python side during this work:** `droidasc listclass` exits 1 with
`'gbk' codec can't encode character` on an app whose classes contain non-Latin identifiers, when the
console encoding is not UTF-8. With `PYTHONUTF8=1` it completes. `rasc` was unaffected. That is a
one-variable difference in the environment, not a property of either tool, and it is recorded because
it cost a round here.
## The blind spot, verified rather than reported
The `rasc` repository ships `bench/jadx_parity.py`, which treats a string literal JADX has and `rasc`
does not as lost code, and refuses to judge a class JADX itself could not decompile cleanly. Its own
class sampling takes every-nth entry from `rasc classes`, which on a real app lands mostly on
obfuscated names where JADX produces nothing — 8 of 10 picked that way were unjudgeable here, so the
sample measures the sampler. `tools/_work/parity_fair_sample.py` filters to app-like classes first
(ASCII, dotted, no inner class, not `R`/`BuildConfig`) and feeds those to the unmodified harness:
```
$ python tools/_work/parity_fair_sample.py tools/_phone-modules/MT-Manager.apk 20
classes in APK: 30768
app-like candidates: ...
judged 17 class(es) where JADX was clean; literals rasc lost: 1; reorderings: 0
```
The one finding, checked by hand with `tools/_work/verify_parity_finding.py`:
```
class : org.bouncycastle.crypto.PasswordConverter
jadx literals: ['ASCII', 'PKCS12', 'UTF8']
rasc literals: []
missing : ['ASCII', 'PKCS12', 'UTF8']
'ASCII' appears in rasc output as an identifier: True
...
VERDICT: form difference (enum constants printed as a list, literals not re-emitted)
```
**The automated verdict understates it, and the hand check then overstated it. Both corrections are
kept.** The harness counts literals, and the literals in JADX's output belong to the anonymous
subclasses it inlines; `rasc`'s outer-class view prints a bare constant list. The first write-up here
said "the method bodies are gone". **That was wrong, and the way it was wrong is worth recording: the
subclasses were never queried.** They exist as their own classes and both tools decompile them fully:
```
$ rasc getclass <apk> 'Lorg/bouncycastle/crypto/PasswordConverter$1;' -> 404 B, has convert(/getType(
$ droidasc getclass <apk> 'Lorg/bouncycastle/crypto/PasswordConverter$1;' -> 460 B, has both
$ ... $2 and $3 the same shape, 406-407 B vs 462-463 B
```
So the accurate finding is narrower than the first version and narrower than the harness's phrasing:
**`rasc` does not inline enum-constant bodies into the outer class view, and it does not say so** —
the bodies are one `getclass` away, on the subclass JADX itself names in a comment
(`// from class: ...PasswordConverter.1`). Neither does the Python tool inline; `droidasc`'s outer
listing is just richer (2,170 B vs 314 B, with `$VALUES`, `$values()` and the constructors explicit).
The general lesson this left behind, and the reason the episode stays in the record: a harness that
counts one artefact (literals) can point at a real difference while misdescribing it, and a hand
check that stops at the first comparison can convert that into an overstated claim. The corrective was
to query the classes the tool we were comparing against had named — which cost one command.
## Not established
- **No whole-APK parity run.** The comparison is on class sets, sampled `getclass` output and one
`findrefs` family per sample. `bench/corpus_parity.py` and `bench/jadx_parity.py` exist for a fuller
run and were used on one archive, not on a corpus.
- **`findrefs` on a large corpus was not run to completion.** On the 34.8 MB app, `findrefs string
http` returned 0 rows on both sides — a query with no hits is not a parity result, and a query that
does hit was only compared on the small archive.
- **`getclass` was judged on 17 classes of one app.** The enum blind spot is confirmed; whether other
class shapes (inner classes, synthetic bridges, `invokedynamic`) diverge the same way is untested.
- **No device or Android-side use.** `rasc` reads archives on the host; nothing here exercises it as a
device tool, and the reference file makes no claim about that.
- **The 8.0× upstream figure was not reproduced**, because the archive it was measured on (343 MiB,
567k classes) is not available here. The direction and the smaller ratio were measured.

585
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-reconstruction.md Normal file
View File

@@ -0,0 +1,585 @@
# Extension verification — truthful environment gates, documented-command validation, and APK rebuild safety
Covers the P0 pass that replaced the optimistic half of `skills/apk-reverse/scripts/doctor.py`
with a probed capability registry, added `check_commands.py` at the repository root, and — in the
same round, by teammate `zip-safety` — replaced the APK-rebuild writer inside
`skills/apk-reverse/scripts/so_constpatch.py`. Strength labels are the ones this directory's
README defines: **observed** (reproduced here, exact command and output recorded), **inferred**
(follows from observed facts, the step itself was not executed), **unverified** (assumed or
reported elsewhere, not independently reproduced).
Two gates depend on the material below, which is why the report was allowed to become worse
looking: G2 (environment gating) reads the capability verdicts, and the documented-command check
is what keeps a reader from following a page instead of the tool. "Documented command" below means
a script invocation written in a code block or inline backtick in a markdown file — the shape
`check_commands.py` extracts and validates.
Reference environment for this pass: Windows 11 / AMD64, Python **3.14.0**, JDK **17.0.4.1**
(`java`/`javac` only, on the Oracle launcher shim path), `adb` 1.0.41 with one attached
`arm64-v8a` device at API 30, Frida **16.7.19** host package, capstone present, Android
build-tools unpacked at a local directory but **not** on `PATH`. Workbench artifacts live in
`tools/_work/truth-gates/` and `tools/_work/zip-safety-out/` (not part of the skill; `tools/` is
gitignored, so those paths are an operation record rather than something a reader can open).
---
## 0. Summary of what this pass established
| Claim | Label |
|---|---|
| The committed `doctor.py` registered **28** scripts against **51** files actually in `scripts/` (46 `.py` + 5 `.js`) | observed |
| Its capability table derived `repack + sign` and `smali round-trip` from `java` alone; on a JDK-only `PATH` it reported **both as OK** while `keytool`, `jarsigner`, `zipalign`, `apksigner` and all eight smali jars were absent | observed |
| The rebuilt `doctor.py` scans the directory, so `registered/checked = N / N` equals the file count; on this host `52 / 52` (47 `.py` + 5 `.js`, the new `capabilities.py` included) | observed |
| Capability verdicts now come from a real `requires` closure with an executable next action and a cost estimate; 16 capabilities are resolved, and the same JDK-only host reports `repack_signed` and `smali_roundtrip` **BLOCKED** | observed |
| `check_commands.py` reads each script's `argparse` interface with `ast` (no script executed to learn its interface) and validates **139** documented commands across 76 markdown files against 39 interfaces, with 0 drift after the one known drift was fixed | observed |
| The drift the checker was written for — `sig_probe.py --live --pkg <pkg>` — is reproduced **from `git show HEAD:`** of the pre-fix page and is reported at the right line | observed |
| The pre-existing `so_constpatch.py` APK writer dropped per-entry zip metadata and lost `resources.arsc` alignment; the replacement keeps metadata byte-for-byte and the alignment gate passes | observed |
| `zipalign -c -v 4` from the Android build-tools agrees with that verdict independently (old writer `rc=1 FAILED`, new writer `rc=0`), re-checked in this pass on artifacts produced by the other teammate | observed |
| `repack.py:_needs_alignment` uses `^lib/[^/]+\.so$`, which never matches the real `lib/<abi>/x.so` layout, so its alignment filler is dead code on real APKs | observed (reported, **not** fixed — outside every current write scope) |
| A real `pm install` of any rebuilt artifact | not done — no install was attempted in this pass |
---
## 1. The doctor was optimistic, and a gate trusted it
### 1.1 A hand-written inventory that could only go stale
The committed version carried `script_dependencies()` returning a literal dict of 28 entries. The
directory held 51 files. The 23 absent from the table were not peripheral: they included
`dex_patch_bytes.py`, `dex_find_insn.py`, `dex_dump_validate.py`, `dex_mem_scan.py`, `dexutil.py`,
`protobuf_decode_raw.py`, `java2c_probe.py`, `vmp_diff_harness.py` and `scan_leaks.py` — nine of the
scripts this skill exists for. The summary line read `scripts runnable here: 28 / 28`, so the report
looked like complete coverage of a set that was less than half the set.
The replacement cannot drift in that direction, because the list *is* the directory:
```
$ python skills/apk-reverse/scripts/doctor.py --scripts
--- scripts ---
directory: E:\apk-reverse\skills\apk-reverse\scripts
registered/checked = 52 / 52 (.py=47, .js=5)
ok=33 partial=13 blocked=6 unknown=0
```
A file whose dependency set cannot be decided is printed as `UNKNOWN` and still counted in the
denominator; `unknown=0` here is a measurement, not an assumption. Dependencies come from an `ast`
pass over imports checked against `sys.stdlib_module_names`, plus a narrow scan for host tools
(`resolve_tool('java')`, the program of a `subprocess` list) against a known-tool set. Modules are
never imported to probe them, because importing the probe target is itself a side effect — `frida`
starts threads and enumerates devices at import time.
### 1.2 The formula that ignored the toolchain — decisive reproduction
`doctor.py:319-320` (committed):
```python
caps['repack + sign'] = bool(tools['java']['path']) and py['ok']
caps['smali round-trip'] = bool(tools['java']['path']) and py['ok']
```
Neither ability follows from `java`. To show that the mechanism was wrong rather than merely lucky,
the committed script was extracted from git and run with a `PATH` holding a JDK and nothing else:
```
$ git show HEAD:skills/apk-reverse/scripts/doctor.py > tools/_work/truth-gates/doctor-HEAD.py
$ set PATH=C:\Program Files\Common Files\Oracle\Java\javapath;C:\Windows\System32;C:\Windows;C:\Python314
$ python tools/_work/truth-gates/doctor-HEAD.py
--- capabilities ---
[OK ] static (dex/zip/strings)
[OK ] repack + sign <-- no keytool, no zipalign, no signer on this PATH
[OK ] smali round-trip <-- none of the eight jars exist on this host
[OK ] native ELF patching
[-- ] jadx decompile
...
```
The rebuilt script, on the **same** restricted `PATH`:
```
[BLOCKED] repack_signed Repack, align and sign an APK
missing : tool:keytool -- not on PATH or APKREV_TOOLS
next: the directory on PATH (C:\Program Files\Common Files\Oracle\Java\javapath)
is a launcher shim, not a full JDK bin/: put %JAVA_HOME%\bin first, or
install a JDK - a JRE ships no keytool [~2 min, unverified]
missing : tool:zipalign -- not on PATH or APKREV_TOOLS
next: build-tools are already unpacked here but are not on the search path:
set APKREV_TOOLS=E:\tools\android-14;E:\tools\bin (found: ...) [~0 min, observed]
missing : artifact:signer -- tool:apksigner (not on PATH or APKREV_TOOLS);
jar:uber-apk-signer (uber-apk-signer.jar not found)
reduced : tool:jarsigner -- not on PATH or APKREV_TOOLS
[BLOCKED] smali_roundtrip Smali disassemble / assemble round-trip
[OK ] repack_unsigned Repack an APK without signing (--no-sign path)
ok=5 partial=1 blocked=10 (of 16 capabilities)
```
`repack_unsigned` staying OK is deliberate and true: `repack.py --no-sign` is `zipfile` plus the
STORED-entry rules and needs no Java tooling at all. A gate that reported the whole file unusable
would have been a pessimism of its own.
### 1.3 The closure is the point, not the atom list
`capabilities.py` is the single registry: 16 capabilities, each a `name`, a `note`, the scripts it
serves, `required` atoms, `optional` atoms, and `depends_on` capabilities that are expanded
recursively. `repack_signed` depends on `repack_unsigned`, so it cannot be OK while the repacking it
is built on is not. Atoms cover the four kinds that actually decide something here: an importable
module, a tool resolvable on `PATH` or `APKREV_TOOLS`, a jar set, a device property (present,
rooted, frida-server running), a repository file, and one **input artifact** that no host can
install — the Dart AOT snapshot dump, which is why `dart_aot_full` reports BLOCKED with
`build blutter … or run the pinned aotopsy front end` rather than a download link.
`status ∈ ok | partial | blocked`. A missing *required* atom blocks. A missing *optional* atom
degrades: `static_native` stays usable without `capstone` and names the part that is not
(`svc_scan.py` refuses to run without it; the caller index in `dart_disasm.py` works).
Two probes were worth getting right rather than guessing:
- A JDK whose launcher shim is on `PATH` (`...\Oracle\Java\javapath`) exposes `java` and `javac`
but not `keytool` or `jarsigner`. The registry probes those separately and, on failure, says the
shim directory is a shim instead of reporting a missing JDK.
- Build-tools present but unexposed are found by searching the usual SDK locations plus a
conventional `tools/` directory, and the next action becomes the exact assignment
(`set APKREV_TOOLS=E:\tools\android-14;E:\tools\bin`) with a measured cost of `~0 min`, rather
than "install build-tools".
That last hint was verified end to end: with `APKREV_TOOLS` set, `repack_signed` loses the
`zipalign` and `signer` blockers and keeps only `keytool` — which is the honest residual on this
host, because the JDK really is a shim install.
### 1.4 Cost estimates carry their own strength
Every install hint carries `est_minutes` with a `basis`. `observed` means measured in this
repository: the `--unsafe-rebuild`-independent blutter build is **≈78 s** end to end
(`TOOL-VERDICTS.md`), and exposing an already-unpacked build-tools directory is 0 minutes because
the search found it. Everything else is labelled `unverified` and is a planning aid only: a guessed
ten minutes printed as a measurement is the same class of error as a capability verdict derived
from the wrong tool.
Per-capability `est_basis` is `mixed` when the missing atoms disagree, which is what makes
`repack_signed` read `~12 min, mixed` rather than an authoritative-looking single number.
### 1.5 Exit codes and the token
`doctor.py` and `capabilities.py` both follow this round's convention: `0` = nothing blocked
(`partial` does not fail a gate, and the counts are printed so it cannot hide), `3` = at least one
capability blocked, `2` = usage, `4` = internal. The last line is `RESULT=env_ok` /
`env_partial` / `env_blocked` / `usage_error` / `internal_error`, and the usage-error path keeps the
token too — `argparse` would otherwise exit 2 with nothing a caller can branch on.
`--json` carries `{status, exit_code, capability?, evidence[], warnings[], next_action?}` plus, for
compatibility, the keys the old report used (`python`, `tools`, `jars`, `device`, `capabilities`,
`scripts`). `capabilities` changed shape on purpose — the old flat `{name: bool}` map *was* the
optimism — and a `capabilities_legacy` map gives the same information as `{id: "ok"|"partial"|"blocked"}`
for any consumer that only wants the verdict.
---
## 2. `check_commands.py` — the commands a reader is told to run
### 2.1 What it reads, and from where
`check_repo.py` proves that a referenced path resolves and every script answers `--help`. Neither
can see a documented command whose flags no longer exist, and the repository shipped exactly one for
a long time. `check_commands.py` closes that gap by reading each script's real interface with `ast`:
- `ArgumentParser()` / `add_argument()` calls are parsed statically, so **no script is executed to
learn its interface**;
- `add_subparsers()` / `add_parser('name')` are followed, and a sub-parser's flags are attributed
to that sub-command rather than merged into the parent (one variable reused across several
sub-parsers is handled, and the parameters are then attributed to the group as a whole);
- `action=store_true|store_false|count|help|version` and `nargs=0` mean the flag consumes no value;
- a script with no `argparse` at all (a hand-rolled `sys.argv` parser: `smtool.py`,
`dex_strpatch.py`) falls back to parsing its own `--help` output, and `smtool.py`'s `--cp` is
recovered that way. The probe runs in a **throwaway directory**, because a probe must not leave
anything behind — a script that reads its first argument as an output directory would otherwise
create one named after the flag.
Documented commands are extracted from fenced code blocks and inline backticks in `README.md`,
`skills/**/*.md` and `docs/**/*.md`. A documented line is split on `&&`, `||`, `;` and `|`, and a
redirection truncates the segment: `python check_repo.py && python check_refs.py` is two commands,
not one with two extra positionals. `<PKG>`-style placeholders are kept as single tokens, because
reading `<pkg>` as a shell input redirect truncates the command at `--pkg` — and a truncated
command checks clean, which is how the checker would have missed the one drift it was built for.
### 2.2 Allowance rules that keep it honest rather than noisy
- argparse accepts a **unique prefix** of a long option, so `--max` passes for `--max-depth`; an
*ambiguous* prefix is reported.
- `-h` / `--help` are always accepted.
- A flag that needs a value does **not** consume an option-looking token; argparse reports
"expected one argument" instead. This is the rule that makes `--live --pkg <pkg>` report `--pkg`
rather than swallow it as the value of `--live`.
- Positional counts are computed from the declared positionals (with `nargs=None|int|'?'|'*'|'+'`)
and are reported as **warnings**, never as drift: a static count can be thrown off by a
placeholder or an elision, so it does not get to fail a gate. It is skipped entirely when any
option has a variable `nargs` or the interface came from `--help`.
### 2.3 The drift it exists for, reproduced from git
The page was already fixed by the time this checker ran, so the pre-fix revision is taken from git
and checked as a file:
```
$ git show HEAD:skills/apk-reverse/references/signature-derived-keys.md > tools/_work/truth-gates/sig-derived-HEAD.md
$ python check_commands.py --fix-report --path tools/_work/truth-gates/sig-derived-HEAD.md
tools\_work\truth-gates\sig-derived-HEAD.md:61 sig_probe.py unknown flag --pkg closest known: --apk cmd: python scripts/sig_probe.py --live --pkg <pkg>
RESULT=commands_drift
```
Line 61, the right script, and the correct suggestion (`sig_probe.py` takes `--live PKG` and has
never had `--pkg`). A second, synthetic fixture proves the check is not a single-rule special case:
```
$ python check_commands.py --fix-report --path tools\_work\truth-gates\NEGATIVE-unknown-flag.md
...:4 sig_probe.py unknown flag --pkg closest known: --apk cmd: python skills/apk-reverse/scripts/sig_probe.py --live --pkg <pkg>
...:5 dex_classdiff.py unknown flag --original closest known: --max-list cmd: ... --original a.dex --patched b.dex
...:5 dex_classdiff.py unknown flag --patched closest known: --max-list cmd: ... --original a.dex --patched b.dex
...:7 skills/apk-reverse/scripts/does_not_exist.py unknown flag missing-script closest known: - cmd: python .../does_not_exist.py --x 1
RESULT=commands_drift
```
Exit code 1, one machine-readable line per finding, with the document line, the command, the
unknown flag and the closest known flags — which is what `--fix-report` is for.
### 2.4 Coverage on the real repository
```
$ python check_commands.py
docs : 76 markdown file(s)
scripts : 39 argument table(s) read, 2 without a readable table
commands : 139 checked (139 clean, 0 drift, 0 warning)
skipped : 24 workbench, 1 external, 1 quoted, 0 unverifiable, 30 not a script invocation
RESULT=commands_ok
```
`139 checked` counts every invocation resolved to a readable interface, not merely the ones with
findings; a pass that reports only its findings cannot tell "clean" from "nothing looked at".
Everything skipped is printed with its reason, and the reasons are of four kinds:
- **workbench** (24): the script lives under `tools/`, which `.gitignore` excludes, so a reader
cannot open it and a missing file there is not drift. A bare filename that only exists under
`tools/` is caught too, because those evidence records refer to workbench scripts by name.
- **not a script invocation** (30): `python -m …`, `python -c "…"`, an elided `python ... --flag`,
or an output line that merely starts with `python`.
- **external** (1): `python scripts\init_env_win.py` in the blutter build record is a path inside
*another* repository's checkout — the record's preceding line is `git clone --depth 1
worawit/blutter`. Each such entry carries its reason in an auditable table that the report
prints, because a silent allow-list is indistinguishable from a missed drift.
- **quoted** (1): `FINDINGS.md` quotes `python scripts/sig_probe.py --live --pkg <pkg>` *as the
defect it records*. It is evidence, not an instruction; the page's own sentence says the script
has no `--pkg`. Same auditable-table treatment.
The two scripts with no readable table are listed by name. A script with no flags has nothing that
can drift, so its invocations pass without being checkable — stated so the edge of the check is
visible instead of implied.
### 2.5 Defects the negative tests found in the checker itself
Each of these produced a *false clean* on the pre-fix page, and each was found by testing the
checker against a known-bad document rather than by reading its code.
| # | Defect | How it surfaced | Fix |
|---|---|---|---|
| 1 | `<pkg>` was read as a shell input redirection, truncating the command at `--pkg`; the truncated command then checked clean | the git-HEAD reproduction reported no drift at all | `<...>` with no interior space is treated as a placeholder token; `>`/`>>` still truncate |
| 2 | A flag that takes a value consumed the *next* token unconditionally, so `--live --pkg` made `--pkg` the value of `--live` | the same reproduction, after fix 1 | an option-looking next token is not consumed, matching argparse's "expected one argument" |
| 3 | The whole documented line was read as one argv, so `&&`, the second `python`, and a redirect target entered the positional count | 3 warnings about `check_repo.py && python check_refs.py` and `… --json > out.json` | lines are split on `&&`/`\|\|`/`;`/`\|` and truncated at a redirection |
| 4 | `docs/**` records reach workbench scripts as `../../tools/_work/…`, which resolves to a real file, so they were reported as unverifiable repository scripts | 17 spurious entries | a resolved path under `tools/` is classified workbench, and paths are `realpath`-normalised so `dexpatch/../smtool.py` is one entry, not two |
| 5 | `commands_checked` counted only commands *with findings*, so a clean run reported `3 checked` instead of 139 | comparing the summary against the extraction dump | every resolved invocation is emitted as a `clean` record and counted |
| 6 | `--help` fallback probes ran with `cwd` at the repository root | reasoning about probe side effects, then confirmed | the probe runs in a temporary directory |
Defect 1 is the instructive one: a checker whose parser silently shortens a command reports success
for exactly the inputs it was written to reject, and nothing in its own output looks wrong. The
defence is not more careful reading, it is a fixture that is known to be bad.
### 2.6 A second blind spot, found by a reviewer reading this gate's output
A reviewer reported seven "drifting" commands. Reproducing them split the report three ways, and the
middle category is the one worth reading.
**Not reproducible.** `python tests/run_tests.py -q` runs: `run_tests.py` forwards what it is given
to pytest (`args = list(argv[1:]) or ["-q"]`), and its usage block documents exactly that. `dcc.py`
is an external checkout, not a script this repository ships — the form the record uses is:
```
$ cd tools/_work/bench/repos/dcc
$ python ./dcc.py <SAMPLE_DIR>/baseline.apk --no-build --dynamic-register
```
**Real, and fixed.** Seven commands in two evidence records named a **workbench** script with no path
at all (`$ python analyze_pair.py`, `b2_measure.py`, `ttd_attach.py`, `make_skeleton.py`). Those files
exist under `tools/`, which `.gitignore` excludes, so a reader copying the command from the
repository root gets `No such file`. Two of those documents wrote the correct path elsewhere, so they
disagreed with themselves.
**The part that matters: this gate had been scoring those `ok`.** `resolve_script()` tried the
*document's own directory* first, so a bare `analyze_pair.py` inside `docs/tool-verification/`
resolved to `tools/_work/bench/unpack/b3/analyze_pair.py` — a real file — and passed. The fix is not
a stricter regex but a corrected resolution order plus a corrected classification:
1. Resolution now follows the order a **reader** would try: repository root, then the skill's script
directory, then the document's own directory (which is what `../x.py` in a nested README means).
2. A bare name that only resolves under `tools/` is `unqualified-workbench`, and it counts as
**drift, not a skip** — a one-token fix that a reader cannot run. A first attempt classified it as
a low-confidence warning, which would have left all seven in place.
3. The one honest exception is pinned too: a block that opens with `cd tools/_work/bench/repos/dcc`
tells the reader where to stand, so a bare `python dcc.py` after it is correct. That is skipped as
`workbench-after-cd`.
### 2.6a The same gate failed on CI and passed locally, and the reason is worth keeping
The first version of the fix above shipped and CI went red on `check_commands.py` — in both the 3.11
and 3.13 jobs — while the identical command passed on the machine that wrote it. The difference is
`tools/`: it is gitignored, so a clean checkout does not have it, and **recognition of a workbench
artifact by filename required the tree to exist**. On the developer's box `dcc.py` matched a name
under `tools/` and was skipped as `workbench-after-cd`; on CI nothing matched, it fell through to
`missing-script`, and the gate reported drift for three quoted lines in this very file.
Two lessons, both now enforced rather than remembered:
- **A check whose verdict depends on a gitignored directory is not a check.** The classification is
now computed from the **document** as well: `qualified_basenames()` collects every `.py` the text
itself writes *with* a path, and a bare name that appears in that set is a shorthand the document
already explains — so it skips identically on a clean checkout and on a populated one.
- **A gate that cannot be run in the same shape as CI will disagree with CI.** Reproduce the CI
condition locally before believing a green run: `mv tools /tmp/hidden && python check_commands.py`
is the whole experiment, and it takes seconds. That is what found this, after the fix had already
been pushed.
**Two defects of this pass's own, found while fixing it.** A `--json` run printed the `RESULT=` token
*after* the JSON document, so `--json | jq` failed with `Extra data`; the token is gone from that path
and the status is a field in the document instead. And the fix's own substitution left four literal
backslashes before `$` in the two files — a reader would have copied them.
`tests/integration/test_check_commands_gate.py` pins the resolution order, the drift classification,
the `cd` exception, and a clean pass over the committed tree, so this blind spot cannot return
silently.
### 2.7 What this check does not cover
- It validates **flags and (weakly) positional counts**, not semantics. A command with correct flags
and a wrong path still passes; so does a documented `--duration 14` where the useful value is
different.
- A script whose interface is neither declarative nor printable is outside the check; the two
current cases are listed by name rather than folded into the pass count.
- It does not verify that a command *works* — only that the script would not reject it as unknown
arguments. Running the 139 commands remains a benchmark-pass activity.
---
## 3. APK rebuild safety in `so_constpatch.py` (teammate `zip-safety`; re-checked here)
### 3.1 What the committed writer did
```python
# so_constpatch.py:204-218, before this round
with zipfile.ZipFile(out_path, 'w', zipfile.ZIP_DEFLATED) as dst:
for item in src.infolist():
blob = new_data if item.filename == entry else src.read(item.filename)
dst.writestr(zipfile.ZipInfo(item.filename, date_time=item.date_time),
blob, item.compress_type)
```
A new `ZipInfo` built from a filename and a timestamp carries nothing else, so `extra` and
`external_attr` are lost; `create_system`, `internal_attr` and the version fields are reset; and
every entry is re-compressed as DEFLATED, so a STORED `resources.arsc` may stop being STORED and
lose its 4-byte data alignment. There was no `extractNativeLibs` check, no alignment check, and no
pointer at `scripts/repack.py`.
### 3.2 What replaced it
The writer now emits the local headers and the central directory itself, because alignment is a
property of the local header length that `zipfile` cannot express. Per entry it reproduces the raw
local name bytes, both `extra` fields, `external_attr`, `internal_attr`, `create_system`,
`create_version`, `extract_version`, the comment, the DOS time/date words and the general-purpose
flags (the data-descriptor bit is cleared, since the size is known up front). An untouched STORED
entry's compressed stream is copied verbatim in 1 MiB chunks, so its CRC and `compress_size` are
identical by construction rather than by luck. Only the named entry is ever re-compressed, and only
if it was DEFLATED. Alignment padding goes into a private local extra record (`0xCA`) so the
original extra field stays a byte-for-byte prefix.
### 3.3 The offset that matters — a correction to state plainly
The alignment contract is the **data offset** of the entry, not its local-header offset. This
matters because the two differ by the header length plus the name, and real archives routinely have
an unaligned header offset:
```
r2pay (input, 435 entries): resources.arsc STORED data offset 0x3cec08 header_offset 0x3cebce
data offset % 4 == 0 OK header_offset % 4 == 2
```
`zipalign`, `apksigner` and `repack.py`'s own `check_alignment` all test the data offset. So the
rule this tool enforces is: `resources.arsc` must be STORED and its **data offset must be a
multiple of 4**; any other STORED entry that arrived aligned must still be aligned after the
rebuild. The alignment predicate differs from `repack.py`'s in one substantive way —
```
always : resources.arsc (Android R+ install gate)
always : STORED lib/**/*.so (extractNativeLibs=false maps them in place)
preserved: any other STORED entry already at a 4-byte data offset
never : DEFLATED entries, and STORED entries that arrived unaligned
```
— and it uses `^lib/.+\.so$` where `repack.py` uses `^lib/[^/]+\.so$`. The latter matches
`lib/x86/libfoo.so` but **not** `lib/arm64-v8a/libfoo.so`, because the ABI directory is a second
path segment: on a real APK that pattern matches nothing, so the alignment filler it guards would
never fire. That is a live defect in `repack.py` (`_needs_alignment`, and the
`check_alignment`/`zip_report` paths that inherit the predicate), reported rather than fixed
because `repack.py` is in no one's write scope this round. It is worth fixing before the refusal
message tells users to switch to that tool.
### 3.4 Evidence, fixture level
```
arsc_aligned.apk (7 entries)
OLD writer vs INPUT NEW writer vs INPUT
extra differ 7/7 (nothing but offset)
eattr differ 7/7
csys differ 7/7
ver_made differ 7/7
untouched entries with changed CRC/size : 2 untouched entries with changed CRC/size : 0
resources.arsc: STORED @0x3a3 UNALIGNED resources.arsc: STORED @0x3e4 ALIGNED
zipalign -c -v 4 -> rc=1 zipalign -c -v 4 -> rc=0
```
### 3.5 Evidence, a real 435-entry package, and an independent re-check
Same sample, same patch, old writer versus new writer versus the input:
```
extra differ 205/435 (nothing but offset)
eattr differ 435/435
iattr, csys, ver_made differ 272/435
flags differ 164/435
untouched entries with changed CRC/size : 132 untouched entries with changed CRC/size : 0
file size 4,293,620 -> 3,897,949 file size 4,293,620 -> 4,289,340
resources.arsc: @0x36f931 UNALIGNED resources.arsc: @0x3cec08 ALIGNED
```
The old writer also *shrinks* the archive by ~396 KB: re-compressing entries that were already
compressed changes the container, not the payload.
This pass re-checked the decisive claim with the Android build-tools' own verifier — a tool in
neither teammate's write scope — on the artifacts the other teammate produced:
```
$ E:\tools\android-14\zipalign.exe -c -v 4 <file>
plain.old.apk rc=1 Verification FAILED plain.new.apk rc=0 Verification succesful
arsc.old.apk rc=1 Verification FAILED arsc.new.apk rc=0 Verification succesful
r2pay.old.apk rc=1 r2pay.new.apk rc=0
```
`zipalign -c 4` checks every uncompressed entry, which is stricter than Android's install gate. The
bare-`.so` path (no container) was re-hashed as well and shows the intended zero regression:
```
0790faeaec0ed4ea8bd3e859db23840dd9d1c2635330b48c4d2e3a3f68fdd371 bare.old.so
0790faeaec0ed4ea8bd3e859db23840dd9d1c2635330b48c4d2e3a3f68fdd371 bare.new.so
```
### 3.6 The refusal gate
Rebuilding is declined by default when the package cannot be rebuilt safely, with the measurement
and a pointer at the tool that owns a full resource-safe repack:
```
$ python skills/apk-reverse/scripts/so_constpatch.py tests/fixtures/apk/noextract.apk \
--entry lib/arm64-v8a/libfoo.so --replace apkhuan=android -o out.apk
extractNativeLibs=false
UNSAFE: AndroidManifest.xml declares android:extractNativeLibs="false": ...
REFUSED: this package cannot be rebuilt safely by this tool (see the UNSAFE line(s) above).
* If you want a resource-safe repack, use scripts/repack.py: ...
* ... re-run with --unsafe-rebuild to proceed anyway.
RESULT=refused_unsafe_rebuild exit=1 output written: no
```
Refusal is exit `1`, not `3`: `3` means the tool lacks the capability, whereas here the tool is
capable and the *package* is the reason nothing was written, so it is a negative finding about the
target. `--unsafe-rebuild` proceeds and still re-measures the STORED/alignment gate on its own
output. Exit codes follow the round's set — `0` success, `1` negative finding, `2` usage (including
an equal-length violation), `3` unsupported container (zip64, a non-deflate target, not a zip),
`4` internal error with the output deleted — and `--json` carries
`{status, exit_code, capability, evidence[], warnings[], next_action?}` plus `result`, `audit`,
`verify` and `patch`, with the final line `RESULT=<token>`.
### 3.7 What section 3 does not establish
- **No device, no install.** No `adb` install or cold start was run. The `[-124]` install failure is
argued from the bytes and from `zipalign`, not observed on a device. `inferred`.
- **No signing run.** The rebuild output is unsigned by design; `repack.py` owns signing, so no
`apksigner verify` result exists. `unverified`.
- **No genuine zip64 sample.** zip64 is refused (`exit 3`); the refusal path was exercised with a
hand-built non-zip file, not with a >4 GiB archive.
- **Page alignment (4 KB/16 KB) is out of scope.** This work guarantees 4-byte alignment, the
install gate. 4 KB/16 KB alignment for `extractNativeLibs=false` builds is `zipalign -p`'s job and
stays with `repack.py`.
- `--unsafe-rebuild` cannot repair an already-compressed `resources.arsc`: the method is preserved,
the self-check reports it as a warning, and the output stays non-compliant.
---
## 4. What this pass did not do
- **No `pm install` of any artifact**, and therefore no cold-start or behavioural verification of a
rebuilt or re-signed APK. Everything in §3 is container-level plus an external alignment verifier.
- **No signing.** On this host `repack_signed` is genuinely blocked (a JDK launcher shim, no
build-tools on `PATH`, no signer jar), so the signed-repack path was **not** executed; the
capability verdict is a probe result, not an observation of a failed run.
- **No smali round-trip**, for the same reason: the eight jars are absent. `smali_roundtrip` is
reported BLOCKED and was not exercised.
- **`dart_aot_full` was not exercised**: no snapshot dump exists in this session, and that
capability reports an artifact the host cannot install rather than a missing package.
- **`check_commands.py` was not run against a benchmark target**, only against this repository. Its
false-clean rate on a foreign documentation set is unknown.
- **The 30 skipped invocations were not individually confirmed** to be non-scripts; they are
classified by the reason printed for each (`-m`, `-c`, an elision, or a prose line beginning with
`python`), and a misclassification there would be a missed check rather than a false finding.
- **One workbench nuisance observed and not cleaned up** (not in any write scope): the tree carries
an untracked `--help/` directory containing `exempt/benign.txt` and `planted/{appkey,device,
endpoint,package,path}.txt`, i.e. the planted leak-fixture layout. It is not produced by
`scan_leaks.py --help` (reproduced in a scratch directory: usage only, nothing written) and not
by this pass's tools, which now probe in a temporary directory. The generator is
`tools/_work/leak-precedent/gen_fixtures.py`, whose output directory is taken from an argument, so
a probe that passed `--help` as that argument would create exactly this. It is worth removing by
whoever runs the final commit, because the directory holds deliberately leak-shaped files.
## 5. Files changed by this pass
New: `skills/apk-reverse/scripts/capabilities.py`, `check_commands.py`.
Rewritten: `skills/apk-reverse/scripts/doctor.py` (its documented CLI is unchanged; `--scripts` and
`--json` still work, and the JSON keeps its old keys).
Merged here from teammate `zip-safety`'s work (their write scope, not this one):
`skills/apk-reverse/scripts/so_constpatch.py`, `tests/test_so_constpatch.py`,
`tests/fixtures/make_fixture_apk.py`, `tests/fixtures/apk/{plain,arsc_aligned,noextract,services}.apk`
and `tests/fixtures/apk/.gitignore` (the repo-wide `*.apk` ignore rule needs a `!*.apk` reverse rule
for fixtures to be stageable; `git check-ignore -v` confirms it).
Neither teammate committed or pushed. Nothing in §1 or §2 was measured on a device; §3's device
work did not happen either.
## 6. Index lines handed to the Lead
- `skills/apk-reverse/SKILL.md` (and `README.md` where the tool table lists scripts):
**`scripts/capabilities.py`** — new. "Capability registry: the single source of truth for what
this host can actually do. Resolves a capability's real `requires` closure (tools, modules, jar
sets, device properties, input artifacts) into `ok | partial | blocked`, with an executable next
step and a cost estimate that names its own strength. `--list`, `--json`; used by `doctor.py`."
- **`check_commands.py`** (repository root, sibling of `check_repo.py` / `check_refs.py`) — new.
"Validates every documented script invocation (`python …`) against the script's own `argparse`
table, read with `ast` so no script is executed to learn its interface. `--fix-report` prints
`doc:line`, the command, the unknown flag and the closest known flags. Exit 0 clean / 1 drift /
2 usage." It is a natural fourth entry in the publish-gate list wherever those are described.
- **`skills/apk-reverse/scripts/doctor.py`** — the existing row's wording can gain: "the script
inventory is scanned from the directory (`registered/checked = N / N`, undecidable entries counted
as `unknown`), and capabilities are probed rather than inferred from `java`".
- **`skills/apk-reverse/scripts/so_constpatch.py`** — the existing row can gain the clause from
`zip-safety`'s §7: an APK rebuild preserves each entry's zip metadata (STORED stays STORED,
`extra`/`external_attr`/`create_system` intact, `resources.arsc` kept STORED and 4-byte aligned)
and is **refused by default** when `extractNativeLibs="false"` or `resources.arsc` is not
STORED + aligned — `scripts/repack.py` is the route for that package, or `--unsafe-rebuild` to
override deliberately.
- **`repack.py:_needs_alignment` is a live defect** (`^lib/[^/]+\.so$` cannot match
`lib/<abi>/x.so`), reported by `zip-safety` and confirmed here by reading the predicate. Someone
needs a write scope for `repack.py`, or it should be recorded as a known gap.

256
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-rootdump.md Normal file
View File

@@ -0,0 +1,256 @@
# Extension verification — root-side memory dump (the route that does not use frida)
Covers the 2026-09 extension pass that added the §Dumping when frida is refused section
to `references/advanced-unpacking.md`. It is the follow-up to
`docs/tool-verification/EXTENSION-device-run.md`, which established that
`frida-dexdump` is refused by this target (attach: `script has been destroyed`, twice,
single-variable repeat) — that file is the background, not repeated here. This pass
asks the next question: **can root read the process's memory directly, with no
instrumentation present at all?**
Strength labels, as defined in this directory's README:
- **observed** — reproduced here, with the exact command and its output.
- **inferred** — follows from observed facts, but the step itself was not executed.
- **unverified** — assumed, or reported by a tool and not independently confirmed.
**Target identity is deliberately absent**; the package appears as `<PKG>`. The
findings are about the route, the device shell and the tooling, and stay valid
whichever hardened APK produced them.
## Environment (observed)
| Item | Value |
|---|---|
| Device | the same physical arm64 device as `EXTENSION-device-run.md`: Android 11 (API 30), kernel 4.14.186, Magisk root |
| Instrumentation | **none** — the device-side `frida-server` was left stopped for the whole pass; `ps -A \| grep -i frida` returned nothing before and after |
| Sample | 360-jiagu hardened, Flutter AOT payload, installed and untouched (no reinstall, no repack) |
| Root access | `adb shell 'su -c "…"` — full root, no `avc: denied` in any read attempt below |
| Device shell | toybox (`grep`, `dd`, `ps`, `stat`, `awk`, `sed`); no `findstr`, no `ls --time-style`, no `map_files` entries |
Two shell facts were measured before anything else, because both silently corrupt a
naive dump and neither announces itself:
```
$ adb shell 'su -c "grep --help"' # → -a -b -o -z -E -F available
$ adb shell 'su -c "dd --help"' # → iflag=skip_bytes,count_bytes available
```
## 1. The sample does not stay up, with or without frida (observed)
The control assumption going in was that the sample is stable when nothing is
attached. It is not: over a 60-second sample at 1 Hz the pid moved four times, and
logcat explains every move.
pid sequence, `ps -A -o PID,NAME | grep -w <PKG>` once per second:
```
20:31:02 16729 … 20:31:07 16729
20:31:08 17951 … 20:31:23 17951
20:31:26 20021 … 20:31:42 20021
20:31:43 21632 … 20:32:00 21632
20:32:01 (none) 20:32:02 22770 …
```
The platform's own record of two of those transitions:
```
09-21 20:31:43.327 I/ActivityManager: Process <PKG> (pid 20021) has died: fg TOP
09-21 20:31:43.394 I/ActivityManager: Start proc 21632:<PKG>/u0a623 for top-activity {<PKG>/.MainActivity}
09-21 20:32:01.463 I/ActivityManager: Process <PKG> (pid 21632) has died: fg TOP
09-21 20:32:01.555 I/ActivityManager: Start proc 22770:<PKG>/u0a623 for top-activity {<PKG>/.MainActivity}
```
- **observed:** each process lives **7-18 s**, then dies as a *foreground, top*
process, and is started again by `ActivityManager` because the activity stack is
still on top. Lifetime is not constant.
- **observed:** there is **no** crash, `SIGSEGV`, `SIGABRT`, ANR or tombstone entry
for these transitions — filtered logcat for `died|kill|Force|ANR|crash|Timeout|
SIGSEGV|SIGABRT` returned only the two `has died: fg TOP` lines and the restarts.
A clean self-exit, not a platform kill.
- **observed:** the restarts stop when the app is no longer the top activity; after
the last death in the 22-sample trace the pid stayed empty for 14 s until
`monkey` relaunched it.
- **inferred:** this is the packer's own environment check reacting to the device
(root), not a reaction to instrumentation — no instrumenter was present. It also
explains the earlier "stable" control reading: a 12-second window fits inside a
17-second lifetime.
- **inferred:** the practical consequence is the design rule of the whole route —
every step must complete inside one lifetime, and any batch routine must re-read
`/proc/<pid>/maps` after a restart instead of trusting its first pid.
## 2. The map is the cheapest source of dex images (observed)
`/proc/<pid>/maps` for the live sample: 522,795 bytes, 5,617 regions. ART names the
dex images it holds, so no search is needed for those:
```
7129d8c000-712a60f000 r--p [anon:dalvik-classes.dex extracted in memory from /data/app/~~…/base.apk]
6f9ba03000-6f9be2b000 r--p [anon:dalvik-classes.dex extracted in memory from /data/user/0/<PKG>/app_e_qq_com_onlp_…/1ced034d9b425c3061036c1cc9b634]
6f75899000-6f75cae000 r--p [anon:dalvik-classes.dex extracted in memory from /data/user/0/<PKG>/app_sodler/com.kwad.components.tachikoma/5.6.10.1/base-1_apk]
… 17 such regions in total
```
Export command actually used (decimal offsets; see §4 for why not hex):
```
dd if=/proc/<pid>/mem iflag=skip_bytes,count_bytes skip=<DEC_START> count=<SIZE> \
of=/data/local/tmp/t2dump/dexNN.bin bs=64k
```
All 17 regions exported; 16 of 17 completed inside one lifetime, two of the trailing
regions were cut short by the restart described in §1 and came back as 0-byte files —
which is the failure mode to expect, and the reason the batch must be re-entrant.
## 3. Validation: what the exported images actually are (observed)
The dumps are page-aligned and therefore 0..4095 bytes longer than the image they
carry, so all 17 were **rejected on first validation**:
```
== dex dump validation: 7 file(s), 0 parse, 7 rejected ==
dalvik-classes.dex_…base.apk.dex 8925184 REJECTED: file_size=8923752 actual=8925184
```
Trimming each file to the `file_size` field at dex-header offset 32 (dropped 68 …
3,724 bytes per file) then gave a clean run:
```
== dex dump validation: 17 file(s), 17 parse, 0 rejected ==
name size sha256[:12] ver cksum sig class noco code stub%
dex16.bin 122460 3fc5581d6eb3 035 ok BAD 103 207 8974 1.1% (ksad dynamic apk)
dex08.bin 1069516 613cfef811ab 035 ok BAD 1285 432 8929 0.9% (kwad ksdp)
dex14.bin 8923752 1497590d0a24 035 ok BAD 4 160 29 3.4% (from /data/app/…/base.apk)
dex01.bin 3212480 fb61a3f085f5 035 ok ok 3169 785 15924 1.0% (pangle live-lite classes6)
… 17 images, stub% 0.9 – 3.4, 17/17 unique by sha256
ranking (most likely original first):
1. dex08.dex stub%=0.9 cksum=ok sig=BAD classes=1285
…
17. dex14.dex stub%=3.4 cksum=ok sig=BAD classes=4
```
**What this means, and what it does not:**
- **observed:** the route produces *real, parseable dex* from a hardened app's process
memory without any instrumentation — 17 images, one class count 103 up to 10,148,
`stub%` in the low single digits. This is the strongest evidence that the
`/proc/<pid>/mem` route is viable where the frida route was refused.
- **observed:** the image that came from the app's own `base.apk` defines **4
classes** on an 8.9 MB file (`sig=BAD` on the untouched image). File size and class
count move independently under a packer — this is the landing-shell skeleton shape,
matching the shipped `classes.dex` recorded in `EXTENSION-unpacking.md`.
- **observed:** across 22 s of 1 Hz sampling of a live process, the count of
`extracted in memory from /data/app` mappings was **always 1** (`appBase=1`,
`appPrivate=7`, `dexmaps=17`, `VmRSS` 674-746 MB). No second application dex mapping
ever appeared.
- **inferred:** while the observer watched, the packer never placed its real payload
in memory as a *loaded ART dex*. Combined with the clean self-exit in §1, the
reading is that the shell's environment check fires before (or independently of)
payload loading on this device.
- **unverified:** whether a longer-lived or differently-triggered instance would load
a second dex. No attempt was made to defeat the check, by task instruction.
## 4. Searching memory for a decrypted dex: what the device shell does to you
Two blind searches were run for the pattern the payload would leave if it were sitting
in a native or heap buffer rather than a loaded dex.
**Anonymous `rw-p` regions** (everything except ART internals, per-thread stacks, cfi
shadow and dart regions), 143 regions exported, 40,670,268 bytes → **0 dex magic
hits** (`scan_magic.py`, `dex\n03x` with `file_size` cross-check).
**ART main space (`dalvik-main space (region space)`, 1 GB reservation)** sampled in
64 MB windows from the start of the space → **0 dex magic hits** in the 192 MB
covered, at **~10 MB/s**:
```
maps_line=12c00000-52c00000 rw-p … [anon:dalvik-main space (region space)]
region_space_base=314572800
off=0 abs=314572800 secs=6 hits=[]
off=134217728 abs=448790528 secs=7 hits=[]
PID_DEAD before off=268435456
```
**inferred:** at 10 MB/s a full 1 GB heap sweep is ~100 s — several process
lifetimes — so heap search is not a plan for a sample with this restart cadence.
Named ART dex mappings are free; a heap sweep is not.
Four device-shell traps cost real time, each one producing a plausible-looking wrong
answer rather than an error. All four are **observed**:
| Trap | Observed behaviour | What worked |
|---|---|---|
| mksh arithmetic is 32-bit | `$((16#7129d9a000))` → `702128128` (the low 32 bits); `dd skip=702128128` then read an unrelated address and the "dump" was zeros | convert with `awk` (`48-bit` values exact in double): `awk 'BEGIN{…n=n*16+v…; printf "%d", n}'` |
| VMA page alignment | 17/17 dumps rejected with `file_size=… actual=…`, excess 68…3724 B | trim to header `file_size` before validating |
| `grep` is line oriented | `grep -F` with a real newline in the pattern matched only `035` at the wrong offset; `-E` with a literal newline matched nothing | `grep -z -a -o -b -E 'dex.035'` → offset `4` on a synthetic probe containing the magic at offset 4 |
| toybox `awk` regex filter | `awk '/ rw-p / && $0 !~ /dalvik-\|…/'` selected all **808** map lines, including `r--p` system libraries, so the candidate list was meaningless | filter with `grep -aE '^[0-9a-f]+-[0-9a-f]+ rw-p '` plus `grep -av`, keep `awk` for arithmetic only |
**`/proc/<pid>/map_files/` is not a shortcut on this ROM** (observed): the directory
exists, is `dr-x------ root root`, and lists **no entries**, so
`dd if=/proc/<pid>/map_files/<start>-<end>` yields 75 bytes of `can't open … No such
file or directory`. The `mem` + decimal-offset form is the one that works.
**A quoting trap in a different layer** (observed): driving the same commands through
`subprocess` with a list argv stripped the inner quotes, so only the first statement
reached `su -c` and the rest ran as the `shell` user — the visible symptom was
`can't create /data/local/tmp/…: Permission denied` on a file owned by root. Root
commands with embedded quoting should go through one shell layer, not two.
## 5. Leftover dumps in the work directory — what they are (observed + inferred)
`/data/local/tmp` carried `.so` files from earlier sessions. They were verified rather
than trusted:
| File | Size | sha256[:16] | ELF | Sections |
|---|---|---|---|---|
| `L_libapp.so` = `d_libapp.so` = `libapp.bak` | 13,501,344 | `2275dd4398f0b34e` | ELF64 AARCH64, `ET_DYN`, phnum 7 | 11 |
| `rt_libapp.so` | 13,501,344 | `8b490512258ba1c8` | ELF64 AARCH64, `ET_DYN`, phnum 7 | 11 |
| `libapp.so` | 12,256,160 | `4cfec2762c25539b` | ELF64 AARCH64, `ET_DYN`, phnum 7 | 11 |
| `ll.so` | 6,230,040 | `20f2d12b5049f238` | ELF64 AARCH64, `ET_DYN`, phnum 9 | 26 |
- **observed:** all four are structurally valid ELF64/AARCH64 shared objects with a
**complete section header table**, and the three `libapp` variants contain the Dart
AOT `snapshot` marker (3 occurrences each).
- **observed:** the `libapp.so` shipped inside the reference APK is **16,352,152 B**
uncompressed (`lib/arm64-v8a/libapp.so`), i.e. ~2.9 MB larger than any leftover
copy, and no leftover file matches it byte for byte.
- **inferred:** the leftovers are copies of a *Dart AOT library from some build*, not
a range lifted from this process's memory. The signature of a memory-lifted image
is the opposite of what these show: `PT_LOAD` contents with **no** section header
table. They are therefore recorded as **reference material of uncertain
provenance**, not as "the dump this pass succeeded in taking" — and they were not
used for any conclusion above.
## 6. Conclusions, graded
| Claim | Label |
|---|---|
| Root can read `/proc/<pid>/maps` and `/proc/<pid>/mem` on this device with no `avc` denial and no instrumenter present | observed |
| 17 ART dex mappings were exported from a live hardened process and, after trimming to header `file_size`, all 17 parsed as valid dex | observed |
| The dex mapping sourced from the app's `base.apk` defines 4 classes on 8.9 MB (landing-shell skeleton); the other 16 are real SDK dexes, `stub%` 0.9-3.4 | observed |
| The sample self-exits every 7-18 s with no frida present (`has died: fg TOP`, no crash/tombstone) and is restarted by the platform while it is the top activity | observed |
| The earlier "stable with no instrumentation" control reading was an artefact of a 12 s window inside a 17 s lifetime | inferred |
| The packer's real payload never appeared as a loaded ART dex during 22 s of sampling (`appBase` constant at 1) | observed (single sample) |
| Anonymous `rw-p` memory (143 regions / 40.7 MB) and the first 192 MB of the ART heap contain no dex magic | observed |
| A full 1 GB heap sweep is impractical at the measured ~10 MB/s against a 7-18 s lifetime | observed (throughput), inferred (planning consequence) |
| `map_files` is unusable on this ROM; hex parsing in mksh is 32-bit; toybox `grep -F` cannot match the dex magic and toybox `awk` regex filtering is unreliable | observed |
| The leftover `libapp*.so` files are unrelated to this process's memory; provenance unverified | observed (structure/size), inferred (provenance) |
## 7. What this pass did not establish
- **Not** a recovered payload. The route recovered every dex the process had loaded
— SDK plugins and the shell skeleton — but no business dex, because none was loaded
while the process was observed. The Flutter payload itself (`libapp.so`, Dart AOT)
is a separate target and is not reached by a dex-oriented dump.
- **Not** an evasion. The sample's self-exit cadence was *not* fought: no attempt was
made to defeat the environment check, per the task's two-strike discipline. A
longer-lived instance might load more; that is stated as **unverified** rather than
attempted.
- **Not** a general claim about 360-jiagu, Flutter or Android 11 — one device, one
ROM, one sample, one session, with a shared device in use by other experiments at
the same time.
- **Not** a memory-completeness claim: the 102 MB unnamed anonymous block that appears
once the Flutter engine is up was never captured (it was absent during the short
lifetimes observed and the process died before the chunked export finished). It is
recorded here as an uncovered region, not as a scanned-clean one.

337
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-split-apk.md Normal file
View File

@@ -0,0 +1,337 @@
# Extension verification — split APK / App Bundle sets
Covers the pass that added split-set support to `scripts/repack.py` and wrote
`references/split-apk.md`. Strength labels as defined in this directory's README:
**observed** (exact command + output reproduced here), **inferred** (follows from observed
facts, step not executed), **unverified** (assumed or externally reported, not independently
confirmed).
Reference environment for this pass: Windows 11 host, Python 3.14.0, Java 17.0.4.1
(`keytool` NOT on PATH — it was passed explicitly as
`<jdk>/bin/keytool.exe`), `apksigner` 0.9 run from `lib/apksigner.jar`, `zipalign.exe` from the
same build-tools layout. **No `uber-apk-signer` anywhere on the machine**, which is the condition
that made the signing-route fallback a requirement rather than a nicety. Target: one rooted physical
Android 11 (API 30) arm64-v8a device, written `<DEVICE>` throughout.
All package names, application directory names and file names are replaced with placeholders
(`<PKG>`, `<SET>`, `<SET_A>`, `<SET_B>`). The measurements, byte counts, entry counts and outputs
are unchanged; only the identifiers are.
Fixtures: two **real split sets already installed on the device**, copied out read-only
(`adb pull` of the system directory; nothing on the device was uninstalled, replaced or modified).
They were chosen because one exercises each side of the merge/resign decision:
| | Members | Shape |
|---|---|---|
| Set A | 8 | base + 7 density splits, every density split carrying its own resources |
| Set B | 9 | base + 1 ABI split (native libraries only) + 7 density splits that carry a placeholder table and nothing else |
Neither set is third-party: `pm list packages -3` reported **zero** split sets among ~40 installed
third-party packages, while four preinstalled Google components were split. That distribution is
recorded because "pick a third-party split app off the device" is not reliably available.
## 1. Reading a set: what the members actually contain — observed
`pm path <PKG>` on each package returned one line per member (8 and 9 lines respectively), which is
how the targets were identified in the first place:
```
$ adb shell 'pm path <PKG>'
package:/system/priv-app/<DIR>/<SET>.apk
package:/system/priv-app/<DIR>/<SET>-arm64_v8a.apk
package:/system/priv-app/<DIR>/<SET>-hdpi.apk
... (one line per member)
```
`scripts/repack.py --split-mode analyze` over set A (identifiers replaced):
```
== split set: 8 apk(s), 6629510 bytes total
density <SET_A>-hdpi.apk 49623 B split=config.hdpi res=39 entries arsc=9392
density <SET_A>-ldpi.apk 45501 B split=config.ldpi res=39 entries arsc=9392
density <SET_A>-mdpi.apk 45501 B split=config.mdpi res=39 entries arsc=9392
density <SET_A>-tvdpi.apk 74227 B split=config.tvdpi res=78 entries arsc=15124
density <SET_A>-xhdpi.apk 53754 B split=config.xhdpi res=39 entries arsc=9432
density <SET_A>-xxhdpi.apk 57878 B split=config.xxhdpi res=39 entries arsc=10324
density <SET_A>-xxxhdpi.apk 57876 B split=config.xxxhdpi res=39 entries arsc=11200
base <SET_A>.apk 6245150 B split=- dex=classes.dex res=581 entries arsc=1924488 assets=1
[plan] base = <SET_A>.apk (package=<PKG>)
[plan] MERGE BLOCKED: merging is not legal for 7 of 7 splits: config.hdpi, config.ldpi, config.mdpi,
config.tvdpi, config.xhdpi, config.xxhdpi, config.xxxhdpi carry their own resources, and folding those in
means merging resources.arsc (a resource-compiler job). Use --split-mode resign, or pass
--drop-split-resources to accept losing exactly those resources
[plan] resign = legal: one keystore for all 8 members, install the set with
[result] analyze only: nothing was written. Pass --split-mode merge|resign|auto to act on this set.
```
And over set B:
```
== split set: 9 apk(s), 1411272 bytes total
abi <SET_B>-arm64_v8a.apk 255861 B split=config.arm64_v8a libs=arm64-v8a
density <SET_B>-hdpi.apk 16797 B split=config.hdpi arsc=40
... (6 more density members, all 16797 B, all arsc=40)
base <SET_B>.apk 1037832 B split=- dex=classes.dex res=2 entries arsc=71316 assets=1
[plan] base = <SET_B>.apk (package=<PKG>)
[plan] note: every non-base split carries only code/native libraries, so a merged single APK is legal
[plan] note: config.arm64_v8a provides libs for arm64-v8a -- keep only the ABIs the target device runs,
selected with --abi
[plan] merge = legal: 8 member(s) fold into the base
```
Four facts this pass established with those numbers, and that the reference file now depends on:
- **The ABI member carries no `resources.arsc` at all** (set B: `libs=arm64-v8a`, no arsc column).
Its two `.so` files are the whole payload. This is the member that folds cleanly.
- **Each density member carries its own table and its own `res/` entries** (set A: 39-78 entries,
9.0-15.1 KB tables). `resources.arsc` is per member, not per set — the base's 1.88 MB table does
not contain the density variants.
- **A `resources.arsc` can be a 40-byte placeholder.** All seven density members of set B carry
`arsc=40` and zero `res/` entries, while still being legally required to carry a table. A
"has an arsc → owns resources" test would have blocked a merge that is entirely legal here, which
is why the script's test is *`res/` entries or a table over 1 KB*.
- **Base and split share one package name**, read from the binary AXML `package` attribute of every
member, and the base is identifiable as the member whose manifest carries **no `split` attribute**
(set A: `split=-`).
## 2. Unified re-signing — observed
`--split-mode resign` over set A with one keystore, run through the new `apksigner` route:
```
$ python skills/apk-reverse/scripts/repack.py --split-dir <SET_A>/ --split-mode resign \
--split-out-dir signed_set/ --ks key.jks \
--keytool <jdk>/bin/keytool.exe --apksigner <build-tools>/lib/apksigner.jar \
--zipalign <build-tools>/zipalign.exe
[resign] signer route: apksigner
[resign] <SET_A>-hdpi.apk -> signed_set/<SET_A>-hdpi.apk (41298 bytes)
[resign] <SET_A>-ldpi.apk -> signed_set/<SET_A>-ldpi.apk (37202 bytes)
[resign] <SET_A>-mdpi.apk -> signed_set/<SET_A>-mdpi.apk (37202 bytes)
[resign] <SET_A>-tvdpi.apk -> signed_set/<SET_A>-tvdpi.apk (61642 bytes)
[resign] <SET_A>-xhdpi.apk -> signed_set/<SET_A>-xhdpi.apk (41337 bytes)
[resign] <SET_A>-xxhdpi.apk -> signed_set/<SET_A>-xxhdpi.apk (49563 bytes)
[resign] <SET_A>-xxxhdpi.apk -> signed_set/<SET_A>-xxxhdpi.apk (49577 bytes)
[resign] <SET_A>.apk -> signed_set/<SET_A>.apk (4172089 bytes)
[result] OK: 8 members, one certificate (sha256 1eb31d9c…2c44)
```
Every member's independent `apksigner verify --print-certs --verbose --min-sdk-version 21
--max-sdk-version 34` returned `Verifies` with:
```
Verified using v1 scheme (JAR signing): true
Verified using v2 scheme (APK Signature Scheme v2): true
Verified using v3 scheme (APK Signature Scheme v3): true
Signer #1 certificate DN: CN=apkreverse, OU=dev, O=dev, L=NA, ST=NA, C=NA
Signer #1 certificate SHA-256 digest: 1eb31d9c9c81104367a66a0201125aa91d28a6a09eb1dcb5029c36bc0bde2c44
```
All eight printed the **same** SHA-256 — that, not per-file `Verifies`, is the acceptance
criterion for a set, and it is the check the run performs explicitly.
**The signing-route fallback is the load-bearing part of this section.** With no
`uber-apk-signer` on the machine, the pre-change script could not sign anything at all: it exited
with `signer jar not found`, naming a jar the user never had. The new `--signer auto` resolves to
`zipalign -p -f 4` + `apksigner sign --v1 --v2 --v3`, which produced the output above. `--apksigner`
accepts `apksigner.bat` (verified: `rc=0`, `--version` → `0.9`), a `lib/apksigner.jar` value run
through `java -jar` (verified: same), and the extensionless shell wrapper (which is a shell script
and does not run on Windows — that is a fact about the wrapper, not about this script).
### The signing route this pass actually used — and why it had to change
`uber-apk-signer` is `repack.py`'s default signer, and it is **not installed on this machine**
(no `uber-apk-signer.jar`, `APK_SIGNER_JAR` unset). Before this change that made every signed build
impossible here: the script exited with `signer jar not found` and named a jar the user never had.
What actually ran in all four measured routes:
| Item | Value used |
|---|---|
| signer | `zipalign.exe` + `apksigner`, the latter run from `lib/apksigner.jar` through `java -jar` |
| `--apksigner` | `<build-tools>/lib/apksigner.jar`. A `.bat` value was also tested and works (`rc=0`, `--version` → `0.9`); the extensionless wrapper in the same layout is a shell script and does not run on Windows |
| `--zipalign` | `<build-tools>/zipalign.exe` |
| `--keytool` | `<jdk>/bin/keytool.exe` — **not on PATH**, and signing aborts without a way to create the keystore, so this flag is mandatory on such a machine |
| scheme flags | `--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true` |
Alignment is handled **per member, and in this order**:
1. the script writes the archive itself, keeping `AndroidManifest.xml` and `resources.arsc` STORED
and 4-byte-aligning `resources.arsc` plus uncompressed `lib/**`, then re-reads the file and
prints the gate (`== alignment gate: resources.arsc STORED and 4-byte aligned (OK)`);
2. `zipalign -p -f 4` runs over that output (`-p` adds the page alignment the loader wants for
`extractNativeLibs="false"` libraries);
3. `apksigner sign` runs **last** — it adds signature entries without reshuffling the archive,
whereas a `jarsigner`-style rewrite after aligning destroys the alignment and the install is then
refused with `[-124]`.
For a split set this is repeated for every member, which is why `resign_split_set()` de-signs and
re-aligns each member individually before signing it: a single member whose `resources.arsc` is
compressed or unaligned is refused by the installer on its own, and the resulting error names that
member's package rather than the set.
## 3. Merging — observed, including the refusal
Set B, where every non-base member is code or native libraries:
```
$ python skills/apk-reverse/scripts/repack.py --split-dir <SET_B>/ --split-mode merge \
--out merged.apk --abi arm64-v8a --ks key.jks --apksigner <...>/lib/apksigner.jar
[merge] + lib/arm64-v8a/lib<name>_compat_libc++.so (18944 bytes <- <SET_B>-arm64_v8a.apk)
[merge] + lib/arm64-v8a/lib<name>utilsjni.so (206680 bytes <- <SET_B>-arm64_v8a.apk)
[merge] folded: dex=0 libs=2 assets=0 conflicts=0
[merge] DROPPED 7 entry/ies. This is a DOWNGRADED build -- say so when you deliver it:
[zip] classes.dex 872160 bytes method=8 crc=<...>
[zip] resources.arsc STORED offset=0x581fc aligned
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
[sign:zipalign] rc=0
[sign:apksigner] rc=0
[out] merged.apk (603824 bytes)
[verify:apksigner] rc=0 Verifies / v1 true / v2 true / v3 true
[result] OK
```
`dex=0` is correct rather than suspicious: the base owns `classes.dex` and no member of this set
carries a second one, so no renumbering was needed. The `libs=2` fold is the whole point — a
one-file APK that now carries the native libraries the ABI member used to provide separately.
Set A, where merging is **not** legal, refuses rather than producing a broken artifact:
```
$ python skills/apk-reverse/scripts/repack.py --split-dir <SET_A>/ --split-mode merge \
--out merged.apk --no-sign
error: refusing to merge <SET_A>-hdpi.apk -- it carries its own resources (39 res entries,
arsc=9392 bytes).
Folding it in would require merging resources.arsc, which this script does not do.
Either use --split-mode resign (recommended for a resource split), or pass
--drop-split-resources to accept that the resources it owns are lost.
exit code 1
```
This is the enforcement the reference file promises: the refusal happens **before** any output is
written (the `--out` path does not exist afterwards), and the escape hatch is explicit, printed, and
required. The mechanism is **observed**; whether a build produced through that hatch merely loses
its density variants or misrenders is **inferred** — it was not exercised, because the only sets
available were a legal-merge set and a refused set, and manufacturing the third case would have
meant hand-editing a resource table.
## 4. Single-APK behaviour is unchanged — observed
Both pre-existing paths were re-run through the same new signing route, on an unmodified sample
APK from `tools/_work/bench/apks/`:
- **zero-change roundtrip** (`--apk … --out … --ks …`, no `--dexdir`): the run dropped exactly the
three original signature artifacts (`META-INF/CERT.RSA`, `META-INF/CERT.SF`, `META-INF/MANIFEST.MF`),
wrote a 60964-byte unsigned archive, signed to 70647 bytes, reported `== alignment gate:
resources.arsc STORED and 4-byte aligned (OK)` and `[result] OK` with v1/v2/v3 all `true`;
- **explicit dex replacement** (`--dexdir` supplying the same `classes.dex`, 5528 bytes):
`[repl] classes.dex<-…/classes.dex`, the entry re-added at the same size and CRC, alignment gate
OK, `[result] OK`.
Both were run with `--no-sign` as well, which exits 0 and reports `UNSIGNED (--no-sign)`. `--help`
works, exits 0, and carries the split examples and the route-decision text; `--split-mode analyze`
exits 0; a merge refusal exits 1; a failed verification would exit 2.
The output fix worth recording: `AndroidManifest.xml` used to be printed as `NOT ALIGNED` in the zip
layout (it only has to be STORED, unlike `resources.arsc`). It now prints `-` for entries that need
no alignment, so the report cannot be misread as a failing gate.
## 5. Install and launch on the device — **unverified**
**Not executed.** The device was held by a concurrent device experiment for the whole window
available to this pass (`python tools/_work/devlock.py status` reported the lock held by another
teammate for 1417 s, note naming its own install/launch and Frida work), and the lead's decision was
to ship this gap labelled rather than wait for it. Heavy device work must be serialised
(`tools/_work/ENV.md`), so the honest label for everything below is **unverified** — a claim about
installability that was never installed is exactly the failure `references/verification.md` exists
to prevent.
Prepared, and not run — the sequence is the one in `references/split-apk.md` §4, against the
re-signed renamed fixture:
```
adb install-multiple -r <signed_set>/*.apk
adb shell cmd package resolve-activity --brief <PKG>
adb shell am start -n <PKG>/<launcher-activity>
adb shell 'dumpsys activity activities | grep -m2 ResumedActivity'
adb exec-out screencap -p > launch.png
```
What the pass *did* establish without a device, and what each fact does not cover:
- **The artifacts are installable in principle, not in fact.** All eight members of the re-signed set
verify with v1/v2/v3 `true` and one identical SHA-256 (§2), and the merged single APK verifies the
same way (§3, §4). Uniformity of the certificate and a passing `apksigner verify` are necessary
conditions for a set install; the platform installer accepting the set is the step that was not run.
- **The negative control was not run either.** The signature-conflict text quoted in
`references/split-apk.md` §6 (a pulled set reinstalled under its own package name) is the
platform's documented shape and the shape a competing tool reports; no captured output from
`<DEVICE>` backs it.
- Every statement about **what the archives contain** — split roles, arsc ownership, the 40-byte
placeholder tables, the fold set, the refusal — is **observed** and device-independent.
- Every statement about **what the package manager does with them** is **inferred**.
**What closing it needs:** one device window that is not shared, then the sequence above (about three
minutes of device time). The two things most likely to show up there, and already accounted for in
the script: an OEM installer interception on `adb install-multiple` (there is a root
`pm install-multiple` path for it) and a launcher activity that needs resolving rather than guessing.
## 6. Maintenance gate — observed
Run at the repository root with the change in place:
```
$ python check_repo.py
== result: 1 problem(s)
- skills/apk-reverse/scripts/dex_strpatch.py --help crashed:
$ python check_refs.py
== section references: 242 reachable ==
== result: 0 dangling, 0 warning(s)
```
`check_refs.py` is clean, which covers the three `references/repack-and-sign.md` §-references the new
file adds, and `check_repo.py` reports one problem in a **different** file belonging to a different
task; `repack.py` parses and answers `--help` with exit 0.
`check_repo.py` requires every `references/*.md` to be named in `SKILL.md` or the root `README.md`,
and this task was not permitted to edit either, so the reachability step was pre-checked in a **copy**
of the repository (`tools/_work/repo-check/`, workbench only, never in the tree) by running the gate
twice on one snapshot with the index lines as the only variable:
```
without the index lines: == result: 5 problem(s) ... UNLISTED references/split-apk.md
with the index lines: == result: 4 problem(s) ... (the UNLISTED line is gone)
```
The new file therefore contributes nothing to the gate once listed. The listing itself was added to
`README.md` by the lead, and the run quoted above is the result.
## 7. Deliberately not done — and what stays unverified
- **No `.aab` was processed.** A bundle input would need `bundletool` (absent here, and it is the
tool that owns the correct resource merge). This pass only ever handled maps of an *installed* set.
- **No fixture was hand-built with `aapt2`/`d8`.** Those became available on this machine only after
the fixtures were chosen (a build-tools archive was unpacked late in the round), and the two real
sets already exercised both branches of the decision. Building a minimal split set by hand remains
the cheapest way to reproduce this topic on a machine with no installed bundle and no device, and
the route is **inferred** — it was not attempted here. It would not change the resource-merge
boundary either: `aapt2 link` merges resources when it *builds* an APK from sources, not when it is
handed two finished ones.
- **No resource merge was implemented or attempted**, by design
(`references/split-apk.md` §5). The consequence is that a bundle whose resources live in splits
cannot be reduced to one faithful APK by anything in this repository.
- **`apktool` was not exercised, because it is not usable on this machine.** Measured:
`apktool.bat d …` fails with `Error: Unable to access jarfile <missing-path>`, while
`lib/apksigner.jar` and `zipalign.exe` in the same layout work. Any manifest edit that needs
`apktool` is therefore **unverified** here, and AXML attributes are read-only in this pass (the
script parses the string pool and the root element's attributes; nothing writes AXML).
- **Feature splits (`isFeatureSplit`) were not measured.** Neither available set has one. The
handling of `isFeatureSplit` / `configForSplit` in the reference file is **inferred** from the
platform's documented semantics plus the fact that both attributes are readable through the same
AXML reader that was verified against real manifests.
- **Split APKs that are not App-Bundle-derived** (the older per-density `-hdpi.apk` naming style,
or an OEM's own module set) were not available; both sets here use the standard `config.<qualifier>`
split names, and the reader accepts either because it reads the manifest attribute rather than the
file name.
- **A XAPK/APKS archive** (a store bundle of bundles, usually a zip holding the split set plus
`manifest.json`) was not processed. The script's `--split-dir` recursion handles a *directory* of
APKs, so an unpacked XAPK works; the archive itself is not read.

89
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-stalker-exclude.md Normal file
View File

@@ -0,0 +1,89 @@
# Stalker `exclude` — what it fixes, and what it does not
**Question.** Community practice says a followed thread that runs through `libc`/`libart` on arm64
gives deadlocks, watchdog `SIGABRT`s and zero-event traces, and that `Stalker.exclude()` is the
remedy. This pass measured which of those two claims survives contact with a device.
**Strength summary:** the *crash* half is `observed` with a clean control; the *zero-event* half is
`observed` as a **negative** — exclusion did not change it. Every command and its output are below.
## Design
Three arms, one device, one package, one module, one follow window. Exactly one variable moves
between `baseline` → `control` → `treatment`.
| Arm | What it does | Why it exists |
|---|---|---|
| `baseline` | spawn, attach, inject the script, **resume, and never follow** | Without this arm a dead process cannot be attributed: "following killed it" and "the app dies under frida anyway" look identical |
| `control` | follow the main thread for 6 s with `excludeModules: []` | The shape that produced the earlier zero-event / `SIGSEGV` results |
| `treatment` | same follow, with the script's built-in list of 20 system modules excluded | The proposed remedy |
The harness also neutralises one trap in the script itself: `CONFIG.autoStart` defaults to `true`, so
loading `stalker_trace.js` starts a follow with *default* options before a runtime configuration can
land. Left alone, that run is neither the control nor the treatment and contaminates both. The
harness patches `autoStart` to `false` in the injected source and asserts the patch count.
For the same reason the harness resumes explicitly: `frida` spawns a process **suspended**, so
without `device.resume(pid)` the followed thread executes nothing and both arms report a zero-event
trace *manufactured by the harness*. That failure mode is indistinguishable from the one under
investigation, which is exactly why it has to be engineered out before the measurement means
anything.
## Result
```
baseline alive=True (no follow installed)
control alive=False err=script has been destroyed
EXCL|excluded=0/0 []
treatment alive=True DONE|reason=timeout blocks=0 blk=0 calls=0 truncated=0
EXCL|excluded=20/20 [libm.so,libdl.so,libart.so,libartbase.so,libnativehelper.so,…] not-loaded=3
WARN|zero events 1500ms after follow (blocks=0 blk=0 calls=0) -- the pipeline is NOT proven
```
## What this settles
1. **A follow with no exclusion killed the target process, and the baseline arm rules out the
alternatives.** The process survives an attach plus resume with no follow, and dies only when the
follow is installed. That is `observed`, not inferred.
2. **Exclusion kept the process alive.** The treatment arm reached its normal `DONE|reason=timeout`
with the process still running. Excluding the system libraries from a follow is not a style
preference.
3. **Exclusion did not restore event delivery.** The treatment arm still reported
`blocks=0 blk=0 calls=0` after a full 6-second window. The zero-event trace is therefore a
**separate defect from the load problem**, and `Stalker.exclude()` must not be presented as its
fix. This partially contradicts the community framing, which lists "no events" among the symptoms
exclusion cures.
4. **The zero-event warning added to `stalker_trace.js` this pass fires correctly** — `treatment`
emitted `WARN|zero events 1500ms after follow`, while `control` could not, because its process was
already gone. A dead pipeline now announces itself instead of being read as "the code did not run".
## What this does not settle
- **Why events do not arrive.** This measurement says it is not the exclusion list. It does not
identify the cause, and it does not distinguish "no events" from "the followed thread genuinely ran
nothing in that window" — the per-arm control that would separate those (following a thread
executing a known loop) was not run.
- **Whether the crash is reachable without a follow.** Only one package and one module were used, and
the harness kills the process at the end of each arm, so nothing is claimed about a long-lived follow.
- **Anything about other ROMs, other frida versions, or armv7.** Single-device result.
- **The multiplier.** Nothing here measures the 20-50x cost figure; this is about survival and
delivery, not throughput.
## Environment
`<DEVICE>` (Android 11 / API 30 / arm64-v8a, Magisk root), frida host 16.7.19, on-device
frida-server 16.7.19 reached over `adb forward tcp:27099`. Target: the MASTG `UnCrackable-Level1`
sample, package `<PKG>`, module `libc.so` followed on the main thread for 6 s per arm. Device
`load average` was ~30 throughout — high for an 8-core phone, and worth recording because the
earlier incident in `native-dbi-and-deobfuscation.md` §6 happened on a loaded device.
## Reproducing
```
python tools/_work/b6/stalker_ab.py --package <PKG> --module libc.so \
--mode baseline|control|treatment --seconds 6 --device 127.0.0.1:27099 \
--out tools/_work/b6/r_<mode>.json
```
The driver is a work artefact under `tools/_work/` (git-ignored), not part of the skill. Its three
arms and the two harness traps it neutralises are the reusable part.

361
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-test-harness.md Normal file
View File

@@ -0,0 +1,361 @@
# EXTENSION — executable test harness, CI, and the exit-code work order
This file records what the regression harness in `tests/` actually does, what it measured on
this machine, and what it does **not** cover. Strength labels follow the repository rule in
`references/long-task-discipline.md`: `observed` means an exact command and its output are
below, `inferred` means reasoned from a neighbouring measurement, `unverified` means nobody
has put a command behind it here.
The distinction this file exists to keep honest: `tests/benchmark.md` is a **record** of routes
run once by hand against real samples and a real device. `tests/` is a **gate** that runs on
every commit with the standard library and pytest and nothing else. Neither replaces the
other, and a green suite is not evidence that any route works on a hardened target.
---
## 1. How to run it, and what it printed
```
$ python -m pytest -q
596 passed, 153 skipped, 26 xfailed in 49.67s observed
$ python -m pytest -q -m unit
163 passed, 612 deselected in 4.50s observed
$ python -m pytest -q -m cli
342 passed, 153 skipped, 255 deselected, 25 xfailed in 38.26s observed
$ python -m pytest -q -m integration
65 passed, 709 deselected, 1 xfailed in 1.33s observed
$ python tests/run_tests.py -q -m unit
163 passed, 612 deselected in 5.08s observed
```
`tests/run_tests.py` is the one-command entry point. It refuses to fail obscurely when pytest
is absent — measured by putting a `pytest.py` that raises `ImportError` first on
`PYTHONPATH`:
```
$ python tests/run_tests.py
pytest is not installed; this suite cannot run.
install: python -m pip install pytest
the suite is stdlib-only otherwise (no device, no network, no adb)
exit=3 observed
```
Exit 3 is the repository's "environment capability insufficient" code, so a wrapper can tell
"no pytest here" apart from "tests failed". The suite is stdlib-only apart from pytest itself.
CI targets Python 3.11 and 3.13; this machine measured on 3.14.0 with pytest 8.4.1.
---
## 2. Coverage
| Area | What runs | Strength |
|---|---|---|
| dex format table | `insn_units` asserted opcode by opcode against 29 named traps (payload widths, `0x32`-`0x3D` being two units, `const-string` vs `/jumbo`, the `goto` family at `0x28`-`0x2A`); `OP_UNITS` and the accessor must agree for every listed opcode; unallocated opcodes still terminate | observed |
| dex header | magic/`file_size`/table-pointer checks against a synthesised consistent header; the verified offset table asserted by value; signature-before-checksum order, including the failure mode where a checksum taken over a zeroed signature never verifies | observed |
| branch arithmetic | `if-test`/`if-testz`/`goto`/`goto16`/`goto32` targets and register extraction, on a synthetic buffer | observed |
| the sample regression | `dex_patch_bytes.py` + `l1_root.spec.json` → sha256 `67ae1e58…` and byte-identical to `classes.patched.dex`; the diff window is pinned (see §6) | observed |
| scheme-less protobuf | built-in fixture set 26/26, per-fixture expectations, decode→re-encode byte-identical for every canonical fixture, the non-canonical fixture required to *report* its mismatch, both wire-format traps | observed |
| blob rotation search | outer encodings round-trip, raw/zlib/gzip identified by name, the rotation and prefix (`skip`) parameters recovered from a payload built with known parameters | observed |
| leak scanner | synthesised decoys in a temporary directory are reported with the right rules and strengths; the exemption list (`com.android.*`, `com.google.*`, `org.example.*`, `<pkg>`, loopback) produces no finding; an RFC 5737 address stays `weak` and keeps the default gate green while `--fail-on any` fails it; JSON report keys and the last-line `RESULT=` token | observed |
| CLI contract | every shipped script, four invocation shapes where they apply: `--help`, unknown flag, no arguments, all inputs pointed at a missing file | observed |
| adb argv construction | `devsh.py`, `coldstart.Device`, `install_test.adb` build an **argv list**; `\|`, `>`, `$()` and quotes survive as literal text inside one argument; serial injection as `-s`; env-var precedence | observed |
| screenshot validation | `coldstart.Device.screencap` judges the bytes on disk: PNG magic accepted, non-PNG rejected, a sub-8-byte body rejected, a timeout reported as a failed capture | observed |
| launch-health verdict | `install_test.main` driven with a scripted `pidof`: stable pid + no fatal signature is the only pass; a changed pid, a dead app, or a stable pid with a `FATAL EXCEPTION` line all fail; two samples are actually taken | observed |
| log/tombstone parsing | `coldstart.signal_counts` over a logcat fixture (9 patterns, exact counts); `native_crash.parse` signal/code/pid/uptime/frames/registers; `is_system_lib` classification; the `verdict_hint` for an arranged fault | observed |
---
## 3. Not covered — the honest gaps
### 3.1 Device work: excluded by design, because hardware is present
This harness must not touch a device, and after one measured incident that is now enforced
rather than assumed. It has to be: `adb devices` on the machine this was written on reports an
online device, so a naive "run every script with its arguments filled in" test drives real
hardware.
```
$ adb devices
<DEVICE> device observed
```
**What happened.** An early version of the CLI contract test executed `coldstart.py` with its
requirements filled in. The script found the single online device, adopted it, started
`adb logcat -v threadtime`, wrote a 927,893-byte `logcat.txt`, captured four frames, and left
**five orphaned `adb logcat` processes** behind when the 12 s timeout killed the parent. The
whole mess landed in `skills/apk-reverse/scripts/`, which is another owner's write scope.
**What was cleaned up.** The five orphaned `adb logcat` processes were terminated, and the
directory the script created (`skills/apk-reverse/scripts/coldstart/`, containing `logcat.txt`
and `shots/f0*.png`) was removed. Verified: `git status` on `skills/apk-reverse/scripts` no
longer lists it.
**What changed in the design.**
- Every child process now runs in a private temporary working directory, so a script that
defaults an output path relative to cwd cannot write into the shipped tree. This is the
general fix; the incident was one instance of the class.
- `contract.DEVICE_OR_NETWORK_BOUND` — 15 scripts — is never executed with complete arguments.
`--help` and an unknown flag are still exercised for all of them, because argparse answers
both before any device call is reached, and that was verified for every script individually:
`--help` returned in well under a second and an unknown flag returned 2 for each.
**Residual risk, stated plainly.** This is a blocklist, not a sandbox. A script not on the
list that grows device access would be executed. The list is checked against the tree on every
run (a script that appears without being classified becomes a new pass or a new xfail rather
than a silent gap), but it is maintained by hand.
Fifteen scripts are device- or network-bound and their argument-complete behaviour is
`unverified` here. What that means concretely: this harness cannot tell you that a repack
installs, that an anti-debug patch survives, that a hook fires, or that a tracer reports
blocks. Those live in `tests/benchmark.md`, and the `unverified` marks there are unchanged by
anything in this directory.
### 3.2 CI has never executed: `unverified`
`.github/workflows/ci.yml` exists and is syntactically valid — verified locally:
```
$ python -c "import yaml,io; yaml.safe_load(io.open('.github/workflows/ci.yml',encoding='utf-8'))"
ci.yml: valid YAML observed
```
and every command it runs was executed by hand on this machine:
```
$ python check_refs.py exit 0 observed
$ python check_budget.py exit 0 ("0 failure(s), 0 warning(s), 3 note(s)") observed
$ python check_repo.py exit 0 (with PYTHONUTF8=1; see §5.1) observed
$ python -m pytest -q exit 0 ("596 passed, 153 skipped, 26 xfailed") observed
```
It is still `unverified` as a workflow, and the blocker is specific: there is no GitHub Actions
runner reachable from this machine, the workflow has not been pushed (this work item is
forbidden from committing), and no local runner (`act`) is installed. What cannot be claimed
until the first push: that `actions/setup-python` resolves both matrix versions, that the gates
behave identically under a Linux UTF-8 locale, and that the suite's timing stays inside any
runner limit. The last two matter because the gates behave *differently* on this Windows host —
see §5.1 — and because the CLI suite spawns roughly 200 subprocesses.
A remote does exist (`origin` → `github.com/newliver666/apk-reverse`), so the workflow will run
on the first push; nobody has observed it do so.
### 3.3 Agent behaviour: `unverified`, and deliberately out of scope
`skills/apk-reverse/evals/evals.json` is a skeleton: three cases with expected outputs and
grading assertions, and an explicit statement **in the file** that nothing in it has ever been
executed. No with-skill arm and no without-skill arm has been run. Running them means running
the agent twice per case and grading transcripts, which was excluded from this pass by decision
(`REBUILD-BRIEF.md` §2, the do-not-do list). The file carries its own run procedure.
### 3.4 The 153 skips, by category
| Count | Reason | Reading |
|---|---|---|
| 60 | device/network bound, not executed | 15 scripts × 4 scenarios. Deliberate; see §3.1. |
| 40 | requires no argument, so the contract does not apply | 10 scripts × 4 scenarios. Requiring exit 2 from `doctor.py` or `scan_leaks.py` with no arguments would invent a rule the contract does not state. |
| 34 | does not advertise `--json` | the 34 of 46 scripts with no `--json` flag. That is the §1.6 problem, not a harness gap. |
| 11 | `--json` advertised but the path needs a device or a sample | includes `capabilities.py`, added during this pass. The shape is not asserted because the run that would reach it needs a target. |
| 8 | hand-written usage, argv not derivable | `dexutil.py` and `smtool.py`, 4 scenarios each. Their usage line is prose, so the missing-input argv is not machine-derivable. `devsh.py` and `patch_smali.py` share the trait but are already covered by row 1. |
Total 153, which is why the headline is "passed, skipped, xfailed" and not "passed".
---
## 4. The exit-code work order — 26 xfails
These are not flaky tests. Each is a script on the tree that does not meet the repository
contract in `REBUILD-BRIEF.md` §4 (`0` success, `1` target result failed, `2` bad
arguments/input, `3` environment capability missing, `4` internal error). The xfail reason
carries the measured value, so `pytest -q -ra` prints the work order at any time.
**A. `--help` does not exit 0 — 4 scripts.** A hand-written parser rather than argparse, so
`--help` falls through to "unknown argument". None of them prints a traceback, so this is a
contract gap and not a crash.
`devsh.py`, `dexutil.py`, `patch_smali.py`, `smtool.py` — measured `2`, expected `0`.
**B. A missing input file exits 1 instead of 2 — 13 scripts.** The distinction matters to a
calling agent: 1 means "your target failed", 2 means "your invocation was wrong". Reporting a
typo in a path as a target failure sends the next agent to debug the sample.
`apk_diff.py`, `dart_disasm.py`, `dart_pool_strings.py`, `dex_classdiff.py`,
`dex_dump_validate.py`, `dex_find_insn.py`, `dex_mem_scan.py`, `dex_strpatch.py`,
`elf_plt.py`, `native_crash.py`, `patch_smali.py`, `stalker_report.py`, `svc_scan.py` —
measured `1`, expected `2`.
**C. An uncaught exception instead of a clean report — 7 scripts.** The user-visible
difference: a traceback instead of a sentence.
`dart_pool_strings.py`, `dex_classdiff.py`, `dex_find_insn.py`, `dex_mem_scan.py`,
`dex_strpatch.py`, `patch_smali.py`, `stalker_report.py` — all `FileNotFoundError` (on Windows,
`dex_mem_scan.py` reports `WinError 3`).
**D. `--json` without a `status` field — 1 script.**
`preflight.py --json` returns top-level keys `rows` and `worst`, with no `status`. Per §4 the
document must carry `status` (and ideally `exit_code`, `capability`, `evidence`, `warnings`,
`next_action`). For contrast, `doctor.py --json` was measured during this pass and *does* carry
`status`, and now exits 3 rather than 0 when a capability is missing — the unified convention
is arriving script by script.
**E. `su` quoting — 1 item, outside the exit-code list.** `devsh.su` wraps in double quotes and
escapes only `"`, so `;`, `$()`, backticks and `|` from a `--pkg`-style value still reach the
device shell. `coldstart.Device.shell` uses the POSIX single-quote escape and does neutralise
them. Two helpers, two strategies — the `device_shell()` unification item. The contrast is
asserted in `tests/integration/test_adb_command_construction.py::TestDevshSuQuotingGap`.
Nothing in this list was fixed by this work item: every one of those scripts belongs to another
owner, and a test that quietly repaired its subject would stop being a test.
---
## 5. Defects found and registered, not fixed
### 5.1 `check_repo.py` misreports a script as crashing under a non-UTF-8 console (`observed`)
```
$ python check_repo.py
- E:\apk-reverse\skills\apk-reverse\scripts\dex_strpatch.py --help crashed:
== result: 1 problem(s) ==
exit=1
```
with, above it, `UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 in position 63` raised
inside `subprocess`'s reader thread. The gate spawns each script with `text=True` and no
explicit encoding, so on this host the child's output is decoded with GBK. One script's
`--help` text is not GBK-decodable, and the gate attributes the failure to that script. The
script itself is fine:
```
$ python skills/apk-reverse/scripts/dex_strpatch.py --help ; echo $?
0 observed
```
Pinning the codec fixes it:
```
$ PYTHONUTF8=1 python check_repo.py
== result: 0 problem(s) ==
exit=0 observed
```
`PYTHONUTF8: "1"` is therefore set in `ci.yml`. Two caveats: CI runs on Linux, where the default
codec is already UTF-8, so this is a portability fix for anyone running the gates on Windows
rather than a fix for the CI run; and `check_repo.py` is in the Lead's write scope, so the
underlying `text=True` call is not changed here.
### 5.2 `native_crash.py` overwrites `regs['pc']` with a backtrace frame (`observed`)
Register extraction scans every line of a crash block with one permissive pattern, so a
`#NN pc <hex>` backtrace line matches too and the last one wins. On the fixture, the register
dump's own `pc 00000000007a111128cc` is not what `regs['pc']` holds; `0x55488` — the last
frame's pc — is. The verdict is unaffected (the arranged-fault hint keys off the fault address
and the registers holding it, and neither is `pc`), but a reader who trusts `regs['pc']` as the
faulting instruction pointer is misled. Pinned in
`tests/integration/test_no_device_flows.py::TestTombstoneParsing`.
### 5.3 `coldstart.installed_facts` `versionCode` swallows the rest of the line (`observed`)
The extraction is `r'\s*(versionCode|...)=(.*)'`, so on a real `dumpsys package` line —
`versionCode=42 minSdk=24 targetSdk=34` — the value is the whole remainder, not `42`. It still
works as the before/after fingerprint the caller uses it for, but the field name overstates
what it holds. Pinned in `tests/integration/test_adb_command_construction.py`.
### 5.4 The leak scanner reports one `ip:port` twice (`observed`)
`192.0.2.10:8080` satisfies both `plain_addr` (`ip:port`) and `plain_addr_ipliteral` (bare
IPv4), because the literal rule's trailing `(?![\w.])` is not violated by a colon. Both hits
are `weak`, so the default gate stays green; the effect is report volume, not a wrong verdict.
Pinned, with the reasoning, in `tests/unit/test_scan_leaks.py`.
### 5.5 `usb_net_proxy.py` does something with no arguments (`observed`)
With no arguments it neither prints usage nor exits: it ran past both a 12 s and a 20 s timeout
during probing. Every other script either requires arguments or prints its own documentation.
Not diagnosed further here — the fix belongs to whoever owns that script. It is classified as
device/network bound, so the suite does not execute it.
### 5.6 Stray artifacts in the tree that this work item did not create (`observed`)
Found while checking its own side effects, left in place because they are not this work item's
to delete and may belong to a concurrent one:
- `--help/` at the repository root, containing `planted/{package,device,appkey,endpoint,path,token}.txt`
and `exempt/benign.txt`. The names say "leak-scanner decoy set", so some scanner test wrote
its fixtures into a directory literally named `--help` — most likely an argument-parsing bug
in that test, not a scan result.
- `skills/apk-reverse/scripts/example.preferences_pb` and `skills/apk-reverse/scripts/snapshots/`
(containing `snap.png`, 3.97 MB, and `_snap_ui.xml`). Timestamps 01:17 and 01:18, unchanged
across this work item's later runs, so they are not its output.
---
## 6. The established regression marker
The red line recorded in `REBUILD-BRIEF.md` §2 is reproduced by the suite rather than
re-invented:
```
$ python skills/apk-reverse/scripts/dex_patch_bytes.py \
tools/_work/bench/l1l3/dex/classes.dex \
--spec tools/_work/bench/l1l3/l1_root.spec.json \
-o <tmp>/classes.patched.dex
== header checksum 0x7fb7d9fa -> 0x86e1d928
signature b7fafe72cb521450 -> eb632328fd1ba03c
self-verify: checksum_ok=True signature_ok=True
== wrote <tmp>/classes.patched.dex (5528 bytes, delta 0)
$ sha256sum <tmp>/classes.patched.dex
67ae1e58e7a44777e5ac8f9f823ab21e55e56c6122d2d64c3627232d902a2bba observed
```
Identical to the stored `tools/_work/bench/l1l3/dex/classes.patched.dex`, byte for byte. The
suite asserts the hash **and** where the difference is allowed to be: bytes 8..32 (checksum and
signature, recomputed) plus the four bytes at `0x9d0`. Two of those four keep their original
value — `39000e00` → `00000000` clears two bytes — and a recomputed checksum can keep a byte by
coincidence, so the assertion is containment plus "these must have moved", not equality of the
diff set. That is the honest form: an equality assertion would fail on a correct build.
The test copies the bench input into `tmp_path` before patching, so a test run cannot overwrite
the stored expectation.
`tools/_work/bench/**` is git-ignored, so on a fresh checkout and in CI this test **skips** with
the printed reason instead of turning red. A regression marker that fails because a sample is
absent is a marker people learn to ignore.
---
## 7. Files added by this work item
```
pytest.ini repository root
tests/README.md how to run it, and the ground rules
tests/run_tests.py entry point; exits 3 without pytest
tests/conftest.py fixtures, subprocess runner, isolated cwd
tests/unit/test_dexutil.py
tests/unit/test_protobuf_decode_raw.py
tests/unit/test_blob_decode.py
tests/unit/test_scan_leaks.py
tests/unit/test_dex_patch_bytes_regression.py
tests/cli/contract.py usage parsing and the contract constants
tests/cli/test_cli_contract.py
tests/integration/test_adb_command_construction.py
tests/integration/test_no_device_flows.py
tests/fixtures/tombstone_sample.txt synthetic, 1.3 KB
tests/fixtures/logcat_sample.txt synthetic, 1.0 KB
.github/workflows/ci.yml
skills/apk-reverse/evals/evals.json skeleton, never executed
```
Not touched: `skills/apk-reverse/scripts/**`, `tests/test_so_constpatch.py`,
`tests/fixtures/apk/**`, `SKILL.md`, `README.md`, the root `check_*.py` scripts. No commit, no
push. The only write this work item made inside the shipped tree was its own accident (§3.1),
and it was reverted.
Fixture budget: two text fixtures, 2.3 KB total, well under the 32 KB per-file limit. The leak
decoys are synthesised at run time and are not committed.

142
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-unpacking.md Normal file
View File

@@ -0,0 +1,142 @@
# Extension verification — advanced unpacking (extraction shells, active invocation, VMP boundary)
Covers the 2026-09 extension pass that added `references/advanced-unpacking.md` and
`scripts/dex_dump_validate.py`. Strength labels as defined in this directory's README:
**observed** (exact command + output here), **inferred** (follows from observed facts,
step not executed), **unverified** (assumed or externally reported, not independently
confirmed).
Reference environment for this pass: Windows 11 host, Python 3.14.0, host frida
16.7.19 + frida-dexdump 2.0.1; rooted physical device, Android 11 (API 30),
arm64-v8a, with a frida-server process running (the environment snapshot recorded in
`tools/_work/t2-unpack/doctor-baseline.txt`, taken at pass time). The analyzed sample
is a 360-jiagu hardened Flutter app; its business logic lives in `libapp.so`, so the
shell dex recovered below is the outer layer of a two-layer target — recorded here
precisely because it exercises the layering that whole-dex dumping cannot cross.
---
## 1. Test fixture — observed
`tools/_work/t2-unpack/make_fixture.py` (workbench, not part of the skill) extracts
the shell `classes.dex` from the sample APK and derives every shape the validator
must distinguish:
```
01_original_shell.dex the real shipped shell dex, untouched (expected winner)
02_duplicate.dex byte copy of 01 (dedupe must group it)
03_broken_magic.dex magic bytes destroyed (must be rejected)
04_tampered_body.dex one byte flipped mid-file (checksum+signature BAD)
05_truncated.dex first 64 KB only (file_size mismatch -> rejected)
06_extracted_skeleton.dex every method body stubbed to nop-fill + return-void,
header re-fixed (extraction-shell signature)
07_tiny_garbage.dex 16 random bytes (too small -> rejected)
```
Facts **observed** about the shipped shell dex itself (file `01_original_shell.dex`,
sha256 `bcd1c0488e861ea804fcbb08f90993eb3258f5c8b2e49780aeca4102dedd5af7`, 8,923,752 B,
dex version 035):
- `class_defs_size == 4` on an 8.9 MB dex — under a packer, file size and class
count move independently: the bytes are encrypted payload, not code.
- 160 of 189 defined methods carry no `code_item` (`code_off == 0`): the shell's
class data is mostly declarations, normal for a stub image.
- `checksum ok` while `signature BAD` **on the untouched shipped image** — the
vendor's build wrote an adler32 consistent with a sha1 signature field that does
not match the file contents. Consequence recorded in `advanced-unpacking.md`:
treat checksum/signature verdicts as evidence about *who touched the image*, not
as a pass/fail gate on usability; a naive "signature must verify" filter would
have rejected the genuine article.
## 2. `scripts/dex_dump_validate.py` — observed
`--help` works (argparse). Run over the fixture directory:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/t2-unpack --find 'Lcom/stub/'
== dex dump validation: 7 file(s), 4 parse, 3 rejected ==
name size sha256[:12] ver cksum sig class noco code stub%
01_original_shell.dex 8923752 bcd1c0488e86 035 ok BAD 4 160 29 3.4%
02_duplicate.dex 8923752 bcd1c0488e86 035 ok BAD 4 160 29 3.4%
03_broken_magic.dex 8923752 33eee53b1317 ? REJECTED: magic=b'XXXX'
04_tampered_body.dex 8923752 b89bc8cc73d9 035 BAD BAD 4 160 29 3.4%
05_truncated.dex 65536 e6687390a663 035 REJECTED: file_size=8923752 actual=65536
06_extracted_skeleton.dex 8923752 85e56fed1d56 035 ok ok 4 160 29 100.0%
07_tiny_garbage.dex 16 be45cb2605bf ? REJECTED: too small (16 B) to carry a dex header
dedupe: 3 unique image(s) among 4 valid file(s)
bcd1c0488e861ea8 01_original_shell.dex, 02_duplicate.dex
ranking (most likely original first):
1. 01_original_shell.dex stub%=3.4 cksum=ok sig=BAD classes=4
2. 02_duplicate.dex stub%=3.4 cksum=ok sig=BAD classes=4
3. 04_tampered_body.dex stub%=3.4 cksum=BAD sig=BAD classes=4
4. 06_extracted_skeleton.dex stub%=100.0 cksum=ok sig=ok classes=4 <-- SKELETON: stub%=100, extraction-shell shape
verdict: 01_original_shell.dex is the most likely original (2 copy(ies) in this set)
exit code 0
```
Every fixture behaved exactly as designed — **observed**, and this is the acceptance
test for the script:
- the byte-identical pair grouped by sha256 (never by size);
- the three broken shapes rejected with the discriminating reason as the error string;
- the tampered body still ranked (structure intact) but demoted behind both intact
copies — checksum is ranking evidence, not a gate;
- the skeleton ranked last and flagged: `stub%` alone separated it from the genuine
image (100.0 vs 3.4) with identical class/method counts, which is exactly the
signal an extraction shell leaves in a whole-dex dump;
- `--find 'Lcom/stub/'` scored 1 string-table hit per shell image, demonstrating the
business-dex discrimination hook described in `recon.md` §Unpacking a dex-level
packer.
`--json` emits a deterministic full report (`profiles` + `ranking`, regenerated at
`tools/_work/t2-unpack/validate_report.json`, byte-consistent with the earlier run of
the same fixture). Exit code 1 only when nothing in the input parses as a dex.
The stub detector's definition (skip `0x0000` nop units; a lone `return*` remains)
matches how the fixture and how field-reported extraction shells degrade a body;
note the detector intentionally does **not** treat `const/4 + return` two-unit
bodies as stubs — a genuinely tiny method is real code.
## 3. Device-side dump of the hardened sample
The device pass (spawn + frida-dexdump against the 360-jiagu sample on the Android 11
device) was executed by the Lead in a parallel session; its artifacts and verdicts
are merged here from that pass's records rather than re-measured.
- The sample app was already installed; a frida-server process was up on the device
at pass time (**observed** in `doctor-baseline.txt`: `root 11628 ... S frida-server`).
- Dump artifacts were staged under `tools/_work/t2-unpack/` by that pass; the
validator run in §2 is the structural verdict over them. For this target the
recoverable dex layer is the shell/skeleton layer only: as recorded in the handoff
brief, the business logic sits in `libapp.so` (Flutter/Dart AOT, 16.3 MB), so no
dex — dumped or repaired — contains it. That layering is the pass's second
headline finding: a "successful" unpack of a hardened Flutter app yields the
Java/plugin layer, and the analysis boundary then moves to the Dart snapshot
(`references/dart-aot.md`), not deeper into dex repair.
## 4. FART loop, Youpk, Android 12-16 analysis, VMP recovery — inferred
None of the recovery machinery was executed in this pass: the sample's shell is a
first-generation landing shell (real bodies present, `stub%` near baseline), so
active invocation had nothing to recover. The FART three-step loop, the
`code_item` length trap details, the five Android 12-16 failure roots and their
repairs, the Youpk description, and the differential-hardening VMP recovery plan in
`references/advanced-unpacking.md` are **inferred** from public work (the FART/Youpk
releases and the 2026-08 Kanxue `thread-292312` analysis of why classic
active-invocation hooks die on modern ART). What *is* **observed** is the detection
metric those sections rely on (§2) and the fact that this Android 11 device sits in
the version band where the classic interpreter entry points still exist — the pass
did not modify ART (cost decision, recorded in the task brief).
## 5. Deliberately not done
- No ART modification / custom ROM build (Youpk route): cost exceeds the task's
value on this sample, whose dex layer is a landing shell.
- No live extraction-shell target was available, so the splice/repair half of the
FART loop (bin merge, `fix_dex_header`, jadx acceptance) has no run here; the
skeleton fixture covers the detection side only.
- No real Dex VMP sample was in scope; the VMP section stays a research plan and is
labelled as such in the reference file.

293
vendor-new/apk-reverse/docs/tool-verification/EXTENSION-vmp-diff.md Normal file
View File

@@ -0,0 +1,293 @@
# Extension verification — differential hardening for Dex-VMP
Covers the extension pass that added `references/vmp-differential-analysis.md` and
`scripts/vmp_diff_harness.py`. Strength labels as defined in this directory's README:
**observed** (exact command + output here), **inferred** (follows from observed facts,
step not executed), **unverified** (assumed or externally reported, not independently
confirmed).
Reference environment: Windows 11 host, Python 3.14.0, JDK 17.0.4.1
(`C:\Program Files\Java\jdk-17.0.4.1\bin\javac.exe`), Android build-tools 2.19 / D8
8.2.2-dev off-PATH. **No device and no network were involved in this pass** — unlike the
unpacking and rootdump passes, this one is deliberately machine-local, because the thing
worth measuring here is the comparison logic, not the platform.
**Headline: the local half of the route is measured and closes exactly; the
platform half was not exercised at all, and the cost table says why it cannot be.**
---
## 1. Environment discovery — observed, and one correction to `tools/_work/ENV.md`
`ENV.md` records `E:\tools\android-14\` as a complete build-tools directory. At pass time
it held **three** entries:
```
$ Get-ChildItem 'E:\tools\android-14\' -Name
apksigner.bat
lib
zipalign.exe
```
`aapt2.exe`, `d8.bat` and `dexdump.exe` were absent, so the `build` subcommand had no
toolchain. The complete build-tools tree was sitting unextracted in `E:\tools\bt.zip`:
```
$ (bt.zip entries) | Where-Object { $_.Name -match '^(aapt2|d8|dexdump|split-select)' }
android-14/aapt2.exe 4566296
android-14/d8.bat 3156
android-14/dexdump.exe 986392
android-14/split-select.exe 1589016
android-14/lib/d8.jar 14929869
```
After `Expand-Archive -Path 'E:\tools\bt.zip' -DestinationPath 'E:\tools\' -Force`:
```
$ E:\tools\android-14\aapt2.exe version
Android Asset Packaging Tool (aapt) 2.19-10229193
$ E:\tools\android-14\d8.bat --version
D8 8.2.2-dev (build facedf41bbd28b563d1e9e09c5f72d7c5ca598d8 from go/r8bot (luci-r8-custom-ci-focal-14-zjz7))
```
Two further tool facts worth recording, both **observed**:
- **The apktool jar the wrapper points at does not exist**, so `<tools>\bin\apktool.bat` (which is
`java -jar "<tools>\agb_apktool.jar" %*`) resolves to nothing. apktool is unavailable on this
machine, and a bare `apktool d` fails with a jar-not-found error rather than a
recognisable "tool missing" message.
- `javac` is on PATH only through the Oracle `javapath` forwarder; the real JDK is at
`C:\Program Files\Java\jdk-17.0.4.1`. The harness takes `--javac` explicitly for this
reason.
## 2. `build` — observed
```
$ python skills/apk-reverse/scripts/vmp_diff_harness.py build \
--build-tools E:\tools\android-14 --out tools/_work/t5-vmp/fixture \
--javac "C:\Program Files\Java\jdk-17.0.4.1\bin\javac.exe" --apk
javac: C:\Program Files\Java\jdk-17.0.4.1\bin\javac.exe
d8: E:\tools\android-14\d8.bat
aapt2: E:\tools\android-14\aapt2.exe
generated: ...\src\OpCoverBig.java (37196 B)
$ ... javac -g --release 11 -d ...\classes ...\OpCoverProbe.java ...\OpCoverBig.java
javac ok: 9 class files
$ E:\tools\android-14\d8.bat --min-api 26 --no-desugaring --output ...\dex <9 .class paths>
d8 ok: ...\dex\classes.dex (35028 B)
== ...\classes.dex (35028 bytes) ==
classes=9 methods_with_code=75 instructions=3065 desynced=0
opcode coverage: 218/224 (97.3%)
not emitted by this fixture (6):
0x09 move-object/16 move-object/16 (32x) needs both registers >= 256; the object
frame reaches 256 on the source side only, which d8 encodes as 0x08
0x1b const-string/jumbo needs a string_ids index > 65535, i.e. a fixture carrying
65536+ string constants
0x2a goto/32 needs a >32767-code-unit backward jump; javac refuses the method
first with 'code too large' (its own 64 KB per-method bytecode cap)
0xfd invoke-custom/range d8 emitted the 35c form for every lambda shape tried, including
a six-parameter one
0xfe const-method-handle no Java-language literal; reachable only by hand-written smali
or direct dex construction
0xff const-method-type same as 0xFE
$ E:\tools\android-14\aapt2.exe link --manifest ...\apk\AndroidManifest.xml -o ...\apk\base.apk
aapt2 ok: ...\probe.apk (20482 B, unsigned)
build rc=0
```
Structural facts behind those numbers: **0 desynced methods** (every `decode_all` walk
ended exactly on `insns_off + insns_size*2`, i.e. the format table and the emitted stream
agree), 3,065 instructions across 75 method bodies, 9 classes.
### 2a. The two toolchain traps this measured — observed
Both are encoded in the script so they are not re-derived:
- **`d8` refuses a non-existent `--output` directory.** First run:
`Error: Invalid output: ...\dex / Output must be a .zip or .jar archive or an existing
directory / Compilation failed` and rc=1. Same class of trap as `frida-dexdump -o`.
`build` creates the directory first.
- **`aapt2 link` needs `android.jar` for every `android:` attribute.** A manifest carrying
`android:versionCode` / `android:versionName` / `android:minSdkVersion` produced five
`error: attribute android:... not found` lines and rc=1. A manifest using only the
un-namespaced `package` attribute links cleanly with no `-I` flag at all — which is what
the harness ships, at the cost of an APK with no version attributes. That is acceptable
for a dex carrier and is stated in the output.
### 2b. `goto/32` is unreachable from Java — observed
Driving the generator to 33,000 loop-body statements to push the backward jump past the
20t range:
```
$ python tools/_work/t5-vmp/gen_big_probe.py 33000
generated ...\OpCoverBig.java: 34442 lines, 766468 bytes
javac rc=1
OpCoverBig.java:1427: error: code too large
public static int gotoFar(int x) {
1 error
```
The JVM's own 64 KB per-method bytecode limit fires before the dex branch-distance
threshold can be crossed. `goto/32` therefore cannot be covered by any javac-built
fixture, at any generator size — this is a structural ceiling, not a tuning problem.
## 3. Closed loop — observed, and this is the load-bearing result
`simulate` relabels every opcode through a bijection whose ground truth it writes to disk.
`compare` is then given only the two dex files and must re-derive that table.
```
$ python .../vmp_diff_harness.py simulate .../fixture/dex/classes.dex \
--out tools/_work/t5-vmp/sim.dex --seed 7 --table tools/_work/t5-vmp/truth.json
simulated: ...classes.dex -> ...sim.dex
mode=relabel, 3065 method body(ies)/instruction(s) touched (seed 7)
header recomputed: checksum_ok=True signature_ok=True
ground-truth table written to ...truth.json (compare must re-derive it)
$ python .../vmp_diff_harness.py compare .../fixture/dex/classes.dex tools/_work/t5-vmp/sim.dex
aligned=75 resized=0 stripped=0 missing=0 added=0
aligned slots compared: 3065
verdict: USABLE
(map: 218 rows, every one verdict=high)
compare rc=0
```
Scoring the derivation against the ground truth:
```
ground-truth bijection size : 256
opcodes the fixture emits : 218
compare rows : 218
compare high-confidence rows : 218
compare undetermined : 6
compare non-injective : 0
compare run verdict : usable
derived entries that are WRONG : 0 (none)
derived entries with no truth : 0 []
emitted opcodes NOT derived : 0 []
EXACT RECOVERY: 218/218 emitted opcodes (100.0%)
CLOSED-LOOP VERDICT: IDENTICAL
```
Why the denominator is 218 and not 256: the ground-truth bijection covers all 256 byte
values, but 32 of those are unallocated dex slots and 6 are opcodes this fixture never
emits — and an opcode that never appears leaves no known plaintext. The six are reported
as `undetermined` with their reasons, never guessed. **Every entry the run did produce was
correct, and every opcode that could be read was read.**
## 4. Negative shapes — observed
The decision table's failure branches were driven on purpose, because a detector that has
never been seen to fire is not a detector.
**Stub shape** (`--mode stub`: every body replaced by an equal-length `return-void` fill —
the extraction shape):
```
$ python .../vmp_diff_harness.py compare .../classes.dex tools/_work/t5-vmp/stub.dex --quiet
aligned=75 resized=0 stripped=0 missing=0 added=0
aligned slots compared: 3065
verdict: NOT-USABLE
! 1 private byte(s) are claimed by up to 218 original opcodes each. No injective
relabelling can do that: it is the signature of method bodies replaced by one
common stub, and any substitution read off such a pair describes the stub, not a
private opcode space.
rc=1
```
This is the failure mode worth reading twice. In the per-opcode direction the reading is
**perfect** — 218 rows, all `high`, zero conflicts — because every original opcode maps to
the same private byte. Only the reverse (`private byte → set of originals`) check exposes
it. An implementation without that check would have returned a confident, wholly
meaningless table, and the first version of this one did exactly that: measured before the
fix, the same input returned `verdict: PARTIALLY-USABLE` alongside a note reading
"0 of 218 opcodes saw more than one candidate byte" — a warning that contradicted itself.
**Length-changing shape** (`--mode shrink`: `insns_size` rewritten without moving bytes, a
deliberately inconsistent image that exists only to drive the detector):
```
$ python .../vmp_diff_harness.py compare .../classes.dex tools/_work/t5-vmp/shrink.dex --quiet
aligned=0 resized=75 stripped=0 missing=0 added=0
aligned slots compared: 0
verdict: NOT-APPLICABLE
! 75 method(s) kept a body but changed its length. Boundaries cannot be projected
across a length change; a private opcode stream with its own length table is not
reachable this way.
rc=1
```
Regression after both fixes: the relabel loop still reports `USABLE` and the closed-loop
check still reports `IDENTICAL` — recorded above, re-run after the verdict-logic change.
**Not driven:** the `stripped` shape (`code_off == 0`) needs a class-data rewrite rather
than a byte edit, so its detector path is **inferred** from the shape counters, not
observed firing. The counters themselves (`stripped=0` in every run above) are observed.
## 5. `emit-smali` — observed
```
$ python .../vmp_diff_harness.py emit-smali tools/_work/t5-vmp/sim.dex \
--table tools/_work/t5-vmp/cmp.json --class LOpCoverProbe --method arithInt
; instructions restored: 3065 ; unmapped-opcode stops: 0
; THIS IS A READING SKELETON, NOT ASSEMBLABLE SMALI.
...
.class public LOpCoverProbe;
.super Ljava/lang/Object;
.source "OpCoverProbe.java"
# ---- arithInt(II)I registers=4 insns_size=29
.method public static arithInt(II)I
.registers 4
# 0x0000 90000203 add-int
# 0x0004 d000e803 add-int/lit16
# 0x0008 d8000007 add-int/lit8
# 0x000c b130 sub-int/2addr
...
# 0x0038 0f00 return
.end method
```
Two checks that the restore is real rather than plausible: the stream decodes to the
**original fixture's** `arithInt` body (compare it against `OpCoverProbe.java`'s 17
operations), and the payload-bearing `switchPacked` method also restores cleanly —
`packed-switch` followed by its payload block, stepped over by its own layout rather than
mistaken for opcodes. `unmapped-opcode stops: 0` is the honest coverage measure, and it is
zero only because the table is complete for this fixture.
Three defects were found and fixed by this run, each one a silent-wrong-answer shape:
- `--class LOpCoverProbe;` matched **zero** classes because the filter accepted only one of
the three spellings (`LOpCoverProbe;` / `LOpCoverProbe` / `OpCoverProbe`) — output read
exactly like a class that is not in the file.
- `.source` printed `"<string_idx 34748 out of range>"`: `source_file_idx` sits at
`class_def_item + 16`, not `+ 20` (that is `annotations_off`).
- Methods rendered without access flags, so the skeleton was not readable as smali.
## 6. What was not done
- **No hardening platform was contacted.** The entire submission link of the chain is
`unverified`, and its cost estimate in the reference file is reasoning about how those
services are distributed (accounts, rate limits, download-page delivery, possible refusal
of a synthetic fixture), not a measured upload. This is the pass's largest gap and it is
not closeable from this machine.
- **No real VMP sample was used.** `tools/_work/bench/repos/DexPatcher/assets/ezAndroid.apk`
is recorded in `ENV.md` as a real Dex-VMP challenge sample and was **not** opened in this
pass; the differential route needs a *matching hardened counterpart of the same build*,
which no available artifact provides.
- **The engine-behaviour assumption is inferred**: that a real VMP substitutes per-opcode
at stable instruction length. `simulate` implements the weakest form of that assumption
deliberately — it is the only form this harness can check itself against.
- **Table stability across inputs is unverified.** Whether one fixture's table applies to
another build of the same app, or survives a vendor version change, was not tested and is
the reason the reference file says to re-derive per target.
- **The `stripped` detector path did not fire** (see §4).
- `--mode shrink` writes a structurally inconsistent dex by design; `dexutil.Dex.check()`
will not reject it because it does not range-check `insns` against the file. It is a
detector probe, is labelled as such in `--help`, and must not be treated as a build.

249
vendor-new/apk-reverse/docs/tool-verification/FINDINGS.md Normal file
View File

@@ -0,0 +1,249 @@
# Findings about the skill, from measuring it
Everything here is a result about **the skill**, not about the target. Each item names the evidence
that produced it and its strength.
---
## F1 — The Coverage claim was partly aspirational, and the pass identified exactly which part
`SKILL.md`'s Coverage section lists as *covered by verified mechanisms*:
> Flutter / Dart AOT: pinning the engine version, decoding the object pool, locating and patching
> business logic inside `libapp.so` (`dart-aot.md`).
Measured against a real Dart 3.6.0 `libapp.so`, that sentence splits into three claims of very
different standing:
| Sub-claim | Standing after this pass |
|---|---|
| pinning the engine version | **verified** — the banner route (§1) is correct and 3.6.0 is confirmed three ways |
| decoding the object pool | **false as written** — the kit cannot do it; there is no resolver |
| locating business logic inside `libapp.so` | **true, but conditional** — it works, on top of an external snapshot dumper that the skill neither ships nor installs |
The worked example in `TOOL-VERDICTS.md` shows both halves at once: the chain
`"showRewardAdProvider"` → `pp+0x23548` → ref site `0x7a9464` → `ShowRewardAdFamily.call_2f28ac
@ 0x7a93ec` succeeded, and step 1 was performed by blutter, not by the skill.
The reason is structural, not a shortage of effort: `dart_pool_strings.py` reports **file offsets**
while `pp.txt` and `dart_pprefs.py` work in **pool offsets**. Over the 4,241 strings present in both
spaces there are **4,237 distinct deltas** (min −1,361,429, max +277,654). The mapping is a property
of the reconstructed pool and is not a constant, so no amount of care inside the script recovers it.
Capabilities the kit does not have, stated plainly: snapshot/object-pool deserialisation; symbol and
name recovery; a real call graph (the `--build-index` index holds raw `B`/`BL` targets only); a
decompiler; a function boundary/size table.
---
## F2 — Two scripts fail in the way that reads as a target finding (the P26 class)
The handover singled out `pitfalls.md` P26: *"a self-built analysis script fails in a way that reads
as a target finding."* This pass produced two live instances of exactly that, which is why the P26
warning earns its place.
**`elf_plt.py` returned a confident false negative.** It reports 3,437 stubs where `.rela.plt` holds
3,438, and answers `0x587060 -> not a resolved stub` for an entry that is a legitimate stub for
`dl_iterate_phdr`, with a real caller at `0x576a2c`. Nothing in the output indicates a problem. For
the script's stated purpose — deciding whether a hardening symbol exists in a library — the wrong
answer is indistinguishable from the right one.
**`find_refs.py` returned a clean empty result for input it could not read.** Its docstring advertised
"a smali tree or a directory of dex files"; `iter_files()` accepted only `.smali`. Given the dex
directory it printed `[total refs] 0`, exited **1**, and emitted no warning. This is worse than a
crash: it is the exact input to "nothing references this, safe to patch", produced by the very tool
recommended for judging blast radius before patching.
Re-measured later: the exit code recorded in this section was **0** and it is **1** — the `total == 0`
path has always returned 1. The correction matters because it moves the defect from "no signal at
all" to "the signal was there and the text contradicted it": the note diagnosed the *needle* and never
named the *input form*, so a caller reading the message was still sent the wrong way. Fixed and
re-verified; see F9 for what measuring it turned up underneath.
Both, plus `dart_pprefs.py`'s silently dropped double constants, share one property: **the failure is
one-sided and quiet.** None of the three reports reduced confidence in its own output.
---
## F3 — Defects found, ranked by consequence
| # | Script | Defect | Consequence | Status |
|---|---|---|---|---|
| 1 | `find_refs.py` | `iter_files()` matched only `.smali`; a dex dir yielded 0 results and a direct dex was regex-scanned as text | silent false "no references" on the tool whose job is blast-radius | **fixed, re-verified against a fixture and the real target** |
| 2 | `elf_plt.py` | loop bound `off < fsz - 16` misses the final stub slot | false negative for one real symbol per library, conditionally | **fixed on a copy, re-verified** |
| 3 | `dart_pprefs.py` | only the GP register file is decoded; `ldr dD,[x27,#imm]` (`0xFD400000`) and all double constants lost | under-reports pool references (302 displacements / 1,802 sites here), which breaks the "count refs to judge importance" advice in `dart-aot.md` §9/§10 — one site went `[0 refs]` → `[79 refs]` | **fixed on a copy; agrees with capstone 0/0** |
| 4 | `so_constpatch.py` | rebuild drops the zip `extra` field, destroying 4 KB alignment | for `extractNativeLibs=false` APKs the output will not load; masked here because this target is DEFLATED + `extractNativeLibs=true` | **cause identified, proven by fixture; not fixed** |
| 5 | `so_constpatch.py` | `--diff --name-regions` compares file offset with vaddr | feature completely inert on libraries where `p_vaddr != p_offset` (2 of 36 here), with no warning | cause identified |
| 6 | `so_constpatch.py` | `external_attr` rewritten on 1982/1982 entries | metadata-based integrity comparison detects the rebuild | cause identified |
| 7 | `dexutil.py` | switch payloads decoded as instructions; `decode()` does not resolve operands | caused a spurious "desync likely" verdict on `saveToLocal` | cause identified |
| 8 | `so_constpatch.py` | "N byte(s) changed" is the overwrite window, not bytes changed | overstates change count (13 reported vs 7 actual) | cause identified |
| 9 | `lib_map.py` | `?` means both "read failed" and "not probed" | a real read failure is invisible | cause identified |
| 10 | `dart_pool_strings.py` | TSV escaping is lossy for LF/CR/TAB | 7 of 18,363 rows ambiguous; 19 entries genuinely contain those bytes | cause identified |
| 11 | `elf_plt.py` | `size = 16 if ... else 16` | dead code | cause identified |
| 12 | `dex_strings.py` | `--per-file` prints only files with hits | can be misread as "only this dex was scanned" | cause identified |
Items 1-3 are the ones that changed a conclusion during this pass rather than merely being noticed.
---
## F4 — Documentation contradictions
Independently found while mapping the skill and while using it. The executable cross-reference
checkers (`check_refs.py`, `check_repo.py`) pass, so these are invisible to the repo's own gates.
| # | Location | Problem |
|---|---|---|
| 1 | `dart-aot.md` §2 | Claims blutter's `asm/` has *no instructions* — "Do not plan around it." **Wrong.** One plugin file under `asm/<plugin>/src/ads/reward/` is 1,327 lines carrying class layouts, signatures, a full instruction listing, pool annotations and resolved call targets. It is the richest artifact in the chain and the doc says to ignore it. |
| 2 | `dart-aot.md` §2 | Budgets "tens of minutes" for the first blutter build; measured **≈78 s** (~30× pessimistic). Also demands a very recent C++ compiler; **MSVC 19.34 (Nov 2022) suffices**. |
| 3 | `dart-aot.md` §7 | Presents two-byte hit chaining as an acceptance test. On this sample, 9,840 two-byte CJK candidates yield 2,081 chained runs, all garbage. |
| 4 | `signature-derived-keys.md:61` | Documents `python scripts/sig_probe.py --live --pkg <pkg>` — the script has no `--pkg`; it takes `--live PKG`. Line 54 and `pitfalls.md` P27 give the correct form. |
| 5 | `dart-aot.md:204` | `dart_disasm.py ... --index callers.json --build-index` omits the required argument value; argparse errors out. |
| 6 | `byte-level-patching.md:186` | Says different-length strings "need `dex_strpatch`-class tooling", while `dex_strpatch.py` requires `len(new) == len(old)` — it is precisely the tool that cannot do this. Contradicts `dex-patching.md:123`. |
| 7 | Coverage vs `toolchain.md` | Coverage lists Unity/IL2CPP logic recovery as **not covered**, but `toolchain.md` Tier 4 names concrete tooling without pointing back at that disclaimer, so a reader of `toolchain.md` alone would take IL2CPP for a supported route. |
| 8 | Script count | `SKILL.md`/`README.md` index 35 rows, which is 34 scripts plus the `scripts/dexpatch/` directory row. |
| 9 | `dart_pool_strings.py` docstring | Says two-byte auto-detection is unreliable (confirmed here); `dart-aot.md` §7 treats UTF-16LE hits as a reliable rule. |
| 10 | `verification.md` rung 7 | Re-downloading a distributed file to compare hashes has no landing place in `SKILL.md`'s six "done" criteria, so the two sections disagree on strength. |
Item 1 is the most damaging because it steers a reader away from the best available evidence.
---
## F5 — "verified mechanisms" left no reproducible trace in the repo
The Coverage preamble frames its list as *verified mechanisms*, and `references/long-task-discipline.md`
defines **Observed** as *"reproduced it, with the exact command and output"*. The repository contains
no sample, no fixture, no captured log or screenshot, no patch record, no test and no CI, and its
`README.md` states it holds no target-specific data.
So the strongest support the repository itself can exhibit for those claims is *Inferred*. This is
**not** an accusation that the claims are false — several were confirmed here on the first attempt,
and `dart-aot.md` §1 proved exactly right. It is a statement about evidentiary standing: the repo
asks its readers to label their own claims and does not leave itself a way to meet its own bar.
The concrete measurement figures that *are* reproducible are the 14 sample-scoped numbers listed in
the skill map (e.g. dex 4.32 MB → 7.73 MB; `.RSA` 1199 B vs 777 B at runtime; a log tag going `8 → 0`).
---
## F6 — Boundary evidence: where the skill's method held, and where it ran out
This is the observation the pass was built to produce, so it is stated as a sequence rather than a
verdict.
**Held, unchanged:** manifest and packer triage; multi-dex string/class recon at 27.6 MB across four
dex files (`dex_strings.py`, `dexutil.py`); native ELF import mapping at scale (36,974 relocations);
live-process library mapping; equal-length `.so` constant patching; the launch-timeline methodology
(`coldstart.py`); the P26 discipline itself — checking whether a tool was ever able to answer before
reading its silence as a finding, which is what turned the blutter crash from a false "unusable" into
a corrected "works after a relink".
**Ran out:** from the Dart snapshot inward. Everything past `pp.txt` is fine; getting *to* `pp.txt`
from the binary requires a container resolver the kit does not have. That is one dependency, not a
subject area.
**Not exercised at all:** packers, code virtualization, custom linkers, integrity-check redirection,
tamper-triggered suicide. This target has none of those, and the unmodified app's own startup failure
also removes the repack loop those scenarios need.
---
## F7 — A trap in this specific environment that will produce false confirmations
`androguard` 4.x (the version installed here) has **four independent reference-analysis failures on
this sample**: `get_xref_from()` returns 0 for everything, including `AdLoader.loadAd()V` which has
20+ callers by other means; `get_ref()` raises; a full `get_operands()` sweep returns 0 strings.
Anyone who reaches for androguard to double-check a "zero references" conclusion will get **zero, and
therefore agreement**. The zero-reference result in `TARGET-FACTS.md` does not come from androguard;
it comes from `dexutil.py` plus raw-byte re-checks of the bodies that would not decode cleanly. This
is worth a line in the environment documentation because it is a tool that lies in the direction of
the hypothesis being tested.
---
## F8 — Reconciling the two failures: a genuine but unproven loader signal
Recorded as a boundary between what was established and what was guessed, because collapsing them
would be its own error.
The launch does show non-standard startup plumbing (observed):
- `E LoaderLog: 10026 / 101004` on the device, `10020 / 101003` on the emulator — an SDK-side
loader logging numeric status codes
- `E Instrumentation: Uninitialized ActivityThread, likely app-created Instrumentation, disabling
AppComponentFactory` — the process is not taking the ordinary `ActivityThread` path
- `E <PKG-truncated>: Unknown bits set in runtime_flags: 0x40000000` (logcat truncates the tag to 23
characters, so the package name appears clipped in this line)
And yet the dex-side packer verdict is clean: no hardening library in the native set, an ordinary
`application android:name` (the app's own class, not a packer stub), no known packer markers. Those two
observations sit in tension.
**Not resolved.** The `LoaderLog` emitter was not attributed to a specific class — the string exists
in `classes2.dex` and `classes3.dex` but the classes that declare it are third-party, and this pass
did not trace the call chain. Whether this plumbing is a benign logging SDK, part of the Flutter
plugin bootstrap, or a light-weight loader contributing to the abort is **unverified**. Stating it
as established would be exactly the kind of overreach this report is trying to document.
---
## F9 — Fixing F2 exposed a deeper one: the dex opcode table was shifted, and nine widths were wrong
Fixing `find_refs.py`'s dex path means depending on `dexutil.decode()`'s boundaries being right, so
that was measured rather than assumed. It was not right: on a real 8.9 MB dex, **65 of 50,791 method
bodies did not decode to their declared boundary**, and **32 carried an operand index outside the
table it indexes**.
The cause is a run of `OP_NAMES` and `insn_units` shifted by one from `0x16` to `0x2C`:
| opcode | specification says | the repository had | width: spec vs repo |
|---|---|---|---|
| `0x17` | `const-wide/32` (31i) | `const-wide/16` | **3 vs 2** |
| `0x18` | `const-wide` (51l) | `const-wide/32` | **5 vs 3** |
| `0x1B` | `const-string/jumbo` (31c) | `const-class` | **3 vs 2** |
| `0x1C` | `const-class` (21c) | `monitor-enter` | **2 vs 1** |
| `0x1E` | `monitor-exit` (11x) | `check-cast` | **1 vs 2** |
| `0x20` | `instance-of` (22c) | `array-length` | **2 vs 1** |
| `0x21` | `array-length` (12x) | `new-instance` | **1 vs 2** |
| `0x23` | `new-array` (22c) | `filled-new-array` | **2 vs 3** |
| `0x26` | `fill-array-data` (31t) | `throw` | **3 vs 1** |
| `0x2C` | `sparse-switch` (31t) | *(absent)* | **3 vs 2** |
Plus `0xFA`-`0xFF` (`invoke-polymorphic`, `invoke-custom`, `const-method-handle`,
`const-method-type`), absent from the width logic and defaulting to one unit each where the
specification has 4, 4, 3, 3, 2, 2.
The `goto` family went with it: `branch_target()` computed a branch target for `0x27`, which is
`throw`, and read the offset width one opcode early for the rest. `dex_find_insn.py`'s kind map
carried the same shift in its `goto` and `switch` entries, so a semantic search for a branch matched
the wrong instructions.
That this survived an earlier "fix instruction widths" pass is not surprising in hindsight:
`insn_units` ended with `if op == 0x29: # measured: this slot is goto/16 (20t)` — the shape of a
repair made one observation at a time, where the width of `0x29` was corrected without checking the
opcode's *name* against a specification. Name and width were wrong together and consistently, which
is exactly why the decode still looked plausible.
**How it was settled.** Not by reading the table again, but by two measurements that do not depend on
this project's code:
1. **Self-consistency at scale.** A real dex is a stream ART can execute, so a correct decoder must
land every method body exactly on `insns_off + insns_size*2`. 65 did not. After the fix,
**50,791 of 50,791 do, with 0 illegal operand indices.**
2. **A second decoder.** `androguard` 4.1.4 decoded the same dex independently. Aligned by method
order, the per-instruction width sequences are **identical for all 50,791 bodies — 0 differing.**
Names and widths now come from the format specification's table, and width is a lookup (`OP_UNITS`)
rather than a chain of range tests, which is the shape that made this class of error possible.
**Two things this pass did not fix, recorded so they are not mistaken for clean:**
- **`dexutil.string()` decodes MUTF-8 with `errors="replace"`.** Identifiers containing non-BMP
characters decode to U+FFFD — this sample has them, and one crashed a `print` with a
`UnicodeEncodeError` on a non-BMP codepoint. A class name read from the dex may therefore not
compare equal to the same name read from another tool. It does not affect instruction decoding or a
descriptor-based needle, but it is why the cross-check above had to align by order rather than by
key.
- `OP_NAMES` had no entry for roughly 150 valid opcodes (`aget`/`aput`, the `cmp` family, the
`neg`/`not`/conversion unops, the arithmetic and `lit8`/`lit16` families). They decoded correctly by
fallback but printed as `op_XX`. The regenerated table covers 224 opcodes.

95
vendor-new/apk-reverse/docs/tool-verification/README.md Normal file
View File

@@ -0,0 +1,95 @@
# Tool verification record
This directory is the **evidence record for the `apk-reverse` skill measured against real
targets**, kept separate from the skill itself so the Coverage statement in `SKILL.md` can be
checked against the runs behind it, and so a later reader can tell which claims were *observed*,
which were *inferred*, and which are still *unverified*.
It is **not** a set of target-specific notes kept for reuse, and it deliberately carries **no target
identity**. The measurements are about the **tools and the skill**; they stay valid whichever APK
produced them. Where a concrete figure was needed to make a point (a class count, a file size, a
build time) the figure is kept and the target is named generically.
Targets measured: Flutter AOT applications, multi-dex, mid-size native library sets, no packer.
The APKs themselves are **not** in this repository (`.gitignore` excludes `*.apk`, and `tools/`).
**Workbench paths are an operation record, not a pointer.** A record may say a run happened *at*
`tools/_work/...` so the command can be judged; `.gitignore` excludes `tools/`, so that path is not
something a reader can open. Anything a reader is *meant* to follow is named under
`docs/tool-verification/` or inside the skill — that rule is why a reference document never cites a
`tools/` path as its evidence, even when the work happened there.
## Strength labels
Every claim carries one of three labels, used the same way `references/verification.md` uses them:
- **observed** — reproduced here, with the exact command and its output.
- **inferred** — follows from observed facts, but the step itself was not executed.
- **unverified** — assumed or reported by a tool and not independently confirmed.
## Index
| File | Covers |
|---|---|
| `TOOL-VERDICTS.md` | One verdict per tested script and external toolchain, with the independent cross-check behind it |
| `FINDINGS.md` | Findings about the skill itself: defects, contradictions, boundary evidence |
| `REPO-DECISIONS.md` | What changed in the repo as a result, and what deliberately did not |
## Extension-pass record
The extension pass (the six coverage gaps named in the external review: module ecosystem, extraction
shells and VMP, emulation/RPC, native DBI, kernel-level environment, protocol layers) keeps its
evidence here, one file per topic, in the same three labels:
| File | Covers |
|---|---|
| `EXTENSION-device-run.md` | The on-device run against a hardened Flutter sample: the two control runs, the frida refusals, the **twice-corrected** pid-drift attribution, the failed-dump artefacts checked with `dex_dump_validate.py`, and a shared-device hazard measured with Stalker |
| `EXTENSION-unpacking.md` | Extraction-shell diagnosis by trivial-body ratio, `dex_dump_validate.py` against a fixture derived from the sample's own shell dex, and the tool's own defect found during the run |
| `EXTENSION-rootdump.md` | The root-side `/proc/<pid>/maps` + `mem` route that does not use frida: 17 real dex images recovered, the shell skeleton they are not, four device-shell traps, and the negative result on anonymous/ART-heap scanning |
| `EXTENSION-lsposed.md` | Module route: the gradle-free build chain with timings, three build traps, LSPosed config-DB anatomy, the module that could never have worked, and the root-hiding configuration the device actually has |
| `EXTENSION-emulation-rpc.md` | ASC/`ddc` measured on a hardened APK, Frida-RPC exercised end to end on a live device, and the unidbg build repair |
| `EXTENSION-native-dbi.md` | Stalker: the zero-event boundary, the crash from following a hot libc export, and an offline protobuf round-trip that caught a defect in its own decoder |
| `EXTENSION-kernel-ondevice.md` | The MT Manager APK MCP probe, the module-environment facts it could verify, and the kernel routes that this device's 4.14 kernel puts out of reach |
## Benchmark-pass record
The benchmark pass put **public targets** under the documented routes — `tests/benchmark.md` is the
matrix, and these files are the evidence behind its rows. Same three labels. Several of these rows
came back negative, and four of them overturned a conclusion this repository had already recorded;
those are called out in the matrix rather than buried here.
| File | Covers |
|---|---|
| `EXTENSION-benchmark-l1-l3.md` | The first two matrix rows: the L1 equal-length patch -> repack -> re-sign chain with its on-screen behaviour change and its zero-change control, and the L3 time-to-death baseline, the two spawn/patch/detach orderings that cannot work, and the terminate path that patching the obvious death site does not stop |
| `EXTENSION-extraction-shell-bench.md` | `dex_dump_validate.py` against an 11-variant skeleton set: the trivial-body ratio as a bimodal result rather than a threshold, the three script defects the variants exposed, and the ranking that pointed at a modified image until independent review caught it |
| `EXTENSION-stalker-exclude.md` | Stalker exclusion measured across three arms: the no-follow baseline, the follow that kills the process, and the follow with 20 modules excluded that survives but still delivers zero events |
| `EXTENSION-java2c.md` | Java2C against JNI sinking: native-declaration density roughly 2000x apart, why a `Java_*` symbol search fails silently, and the Dex-to-C build chain no toolchain on this host could complete |
| `EXTENSION-vmp-diff.md` | The known-plaintext differential: a 218-of-224-opcode coverage fixture, a closed-loop check that recovered 218/218 mappings with zero fabrications, and the one link that cannot be automated |
| `EXTENSION-kernel-weapons.md` | The generated KernelSU/APatch scaffold and its three kernel-side templates, all shipped unbuilt, plus the structural correction that a userspace module cannot change a syscall return value |
| `EXTENSION-split-apk.md` | Two real split sets through both branches: unified re-signing, the merge that is refused by design, the merge that succeeds, and the signer chain this host actually needs |
| `EXTENSION-reconstruction.md` | The reconstruction pass: the doctor capability closure (a JDK-only PATH used to report "can re-sign", now `BLOCKED` with a next action), the zip rebuild that stops dropping per-entry metadata (`zipalign -c -v 4` fails the old output and passes the new), and `check_commands.py` catching a documented flag that does not exist |
| `EXTENSION-test-harness.md` | The executable half of the evidence: what `tests/` asserts and what it measured, the `dex_patch_bytes` regression reproduced byte-for-byte, the 153 skips and 26 xfails that form the exit-code work order, the device-bound exclusion forced by an incident where the suite drove a real phone, and CI written but never run |
| `EXTENSION-rasc.md` | The Rust ASC measured against the Python one: identical class-definition sets on two archives, the speedup per scenario, the build (no prebuilt artifact exists, and a 32-bit MinGW cannot link it), and **the enum shape whose method bodies it drops with no warning** |
| `EXTENSION-protobuf-raw.md` | A schema-free decoder cross-checked against the official runtime and a real DataStore container, with the packed-boundary and proto3-zero ambiguities reproduced on real bytes |
## Absorption-pass record
A later pass absorbed mechanics from external skill repositories (`reverse-skill`,
`awesome-game-security`, `android-reverse-engineering-claude-skill`, `MobileRE-Skill`), keeping each
project's own discipline: only the phase, the checklist and the command pattern cross over, with the
source URL and the access date recorded, and nothing vendored as a runtime dependency. These files
live here rather than in the skill because two of them corrected a claim the repository had already
written down.
| File | Covers |
|---|---|
| `EXTENSION-desensitization.md` | The leak scanner's rules and exemptions, its four `RESULT=` states and exit codes, the escaping false positives, and **the pass that found its own evidence file leaking 26 strong hits** — the fixture's literal match values, quoted to prove the rules fire |
| `EXTENSION-dart-aot-formats.md` | Two string-table claims tested against a real dual-ABI Flutter app: the arm64 packed scheme confirmed against literal tag bytes, and the assumed armv7 `len*2`/UTF-16 form **refuted** — the 32-bit record is `[header u32][byte-count u32le][UTF-8]`, and the extractor's zero is a format mismatch rather than an empty table |
| `EXTENSION-detection-pipeline.md` | Naming the check that fires: three arms on a public MASTG target (control alive, probe arm dead in ≈300 ms via `strstr("frida")`, with the `TracerPid=0` versus 4-frida-mappings asymmetry), `svc_scan.py`'s two decoders agreeing on an identical 214-site set, the layered Java-to-svc descent, and the ptrace-free dump boundary on targets whose dex is deflated inside the APK |
## Adding a run
Record the tools, versions and exact commands, and label each claim. Keep target-specific facts out:
what belongs here is what a future reader can apply to a *different* APK — a defect, a measured
cost, a method that did or did not work, and the cross-check that settled it.

305
vendor-new/apk-reverse/docs/tool-verification/REPO-DECISIONS.md Normal file
View File

@@ -0,0 +1,305 @@
# Repository decisions, and the evidence behind them
The pass was framed as a choice between two ways to split the skill. The measurement rejected the
framing, and this file records why, then states exactly what changed in the repository.
---
## The decision: **no second skill.** Fix skill A instead.
This is the "暂不建" outcome the brief explicitly permits, and it is supported by the evidence rather
than by preference.
### Why the framing did not survive contact with the target
The two candidate splits were:
- **by entry point** — everything reached from an APK (including Flutter/IL2CPP) stays in A; only
standalone ELF goes to B. Cost: A keeps growing.
- **by logic location** — A keeps only logic genuinely in dex; Flutter, IL2CPP, hardened dex, native
and standalone ELF all move out. Cost: four references (~962 lines) relocate.
The measured target is exactly the case that was supposed to discriminate between them: Flutter
(machine-code side) whose entry point is an APK. It produced a third answer.
**The problem is not where the content lives. It is one missing dependency.** Everything the kit does
*after* a snapshot dumper has produced `pp.txt` works correctly — the reference index, the
disassembly windows, the caller index, all verified against capstone. The single thing it cannot do
is get *from the binary to* `pp.txt`, because it has no snapshot container resolver, and that gap is
not bridgeable from inside the scripts (4,241 shared strings, 4,237 distinct deltas, no constant).
A second skill would relocate Flutter documentation without adding the capability that is missing.
The gap would travel with the documents. Worse, it would split the Coverage claim in two and leave
the user of either skill less able to tell what actually works.
### The supporting evidence, in order of weight
1. **The kit works where it was tested.** `dart_disasm.py` decoded identically to capstone (32/32,
96/96); the caller index re-derived independently agrees with symmetric difference **0**;
`dart_pool_strings.py` was byte-exact on 18,356 of 18,363 rows with all 7 residuals explained as
TSV escaping, and 12/12 random offsets re-located from raw bytes. These are not the results of a
tool that needs replacing.
2. **The blocker is a front end, and free front ends exist.** aotopsy ran the full pipeline out of
the box — 30,586 functions, 39,202 pool entries, 175,120 call edges, 95.7% BLR annotation — with
**no toolchain**: pure Go, no compiler, no Dart SDK. That is the "decoding the object pool"
capability, delivered as a binary.
3. **A split would contradict the Coverage section's own structure.** Coverage already lists Flutter
under *covered by verified mechanisms* and Unity/IL2CPP under *not covered*. The real defect is
that the Flutter line overstates what ships. The fix for an overstated claim is to correct the
claim, not to relocate it.
4. **The cost of the split lands on the wrong people.** Moving `dart-aot.md`,
`native-and-so.md`, `native-tamper-and-suicide.md` and `code-virtualization-and-custom-linkers.md`
would break 162 cross-references that currently resolve, and would move material that this pass
could not exercise at all (packers, VMP) on the strength of evidence gathered about something
else.
### What the second skill *would* have been, stated so the decision can be revisited
If a future pass ever does need one, the boundary the evidence points at is **"standalone machine-code
targets with no APK"** — a plain `.so`/ELF analysed on its own. That is the split the entry-point
option described, and it remains untested: this target is an APK, so no evidence for or against it
was produced here. The pass therefore declines to build it and says so plainly rather than inventing
a skeleton.
### Selective installation, measured — and what it implies for shared tooling
The brief asked for the shared-tooling question to be settled by the actual install product rather
than by preference. Measured with `npx skills add newliver666/apk-reverse --skill apk-reverse -y
--copy` in an empty directory:
- The skill is copied **whole and self-contained** into each detected agent directory — `.agents/`,
`.claude/`, `.kiro/`, `.pi/`, `.qwen/`, `.zcode/`, six in total on this host — plus a
`skills-lock.json` at the root.
- Every file lands under `<agent>/skills/apk-reverse/`: `SKILL.md`, all 28 `references/*.md`, all 34
`scripts/*` including the `dexpatch/` subdirectory. **Nothing outside the skill directory is
installed.** The repository's root-level `docs/` and the three maintenance checkers are absent from
the product, as intended.
- There is **no cross-skill sharing mechanism** in the installed layout.
Consequence for this decision: had the skill been split, the two halves would each have needed their
own copy of any shared script, or a path that points outside the skill directory — and the latter
does not survive installation, because it is not part of the product. So the "shall we put shared
tools in a root `shared/`" question resolves itself: it cannot work that way, and with one skill there
is nothing to share. The record is kept here because the observation is what makes the reasoning
checkable rather than asserted.
---
## Changes made
### `references/dart-aot.md`
| Change | Reason |
|---|---|
| Documented a **pinned front end** (aotopsy v1.6.0, sha256 `5a90bcf2…fdc1`; Dart 2.10–3.13; ARM64 + x86_64) as the entry point for the workflow, with blutter as the alternative when a compiled Dart VM is acceptable | The workflow's step 1 was not performable with what the skill shipped. Measured: the kit cannot go from a string to a pool offset. |
| Corrected the build budget: **≈78 s measured**, not "tens of minutes" | Measured end to end, log timestamps 04:45:59 → 04:47:17 |
| Corrected the compiler requirement: **MSVC 19.34 (Nov 2022) suffices** | Verified by compiling and running a `std::format` probe with it |
| Removed the claim that `asm/` contains no instructions | Measured false: the file carries class layouts, signatures, a full instruction listing, pool annotations and resolved call targets |
| Recorded the version-probe trap: `aotopsy doctor` labels the snapshot **3.6.2**, the engine banner and snapshot hash say **3.6.0**, and 3.6.0 is correct | Three independent facts agree on 3.6.0; aotopsy's number is a structural profile label and must not be passed to a VM-compiling tool |
| Noted that the two-byte chaining test in §7 did not hold here | 9,840 two-byte CJK candidates → 2,081 chained runs, all junk |
### `references/references-index` / script defects
`dart_pprefs.py` is fixed in the repository. The defect — only the GP register file decoded, so
`ldr dD,[x27,#imm]` (`0xFD400000`) and an entire class of double constants were dropped — was
re-verified on the patched script against capstone over the whole `.text`: **0 displacements and 0
sites of difference**, against 302 and 1,802 before.
`find_refs.py`, `elf_plt.py` and the `so_constpatch.py` alignment defect are **documented but not
fixed**. The repository's rule is that nothing ships unrun, and the fixes for these were not
exercised against the real target in this pass:
- `find_refs.py` — accepting `.dex` input means deciding how to read dex without baksmali/jadx
present, which is a design decision, not a one-line patch.
- `elf_plt.py` — the three-bound fix was verified on a **copy** (`work/native/tools/elf_plt_fixed.py`),
not in the repository, so the repository version is untouched.
- `so_constpatch.py` — the alignment fix needs a padding implementation and a zipalign story.
Each is recorded in `docs/tool-verification/FINDINGS.md` with its root cause and the fix sketch,
which is the honest form: the knowledge is preserved and the unrun code does not enter the tree.
### `SKILL.md` — Coverage
Rewritten so the Flutter line states what was measured:
- **In coverage:** Flutter/Dart AOT analysis *given a snapshot dump* — pool-reference counting,
disassembly windows, caller indexing and patching — all verified on a real Dart 3.6.0
`libapp.so`, with the front end named as a documented dependency.
- **Explicit new dependency line:** the kit does not decode the object pool; it requires a snapshot
dumper (aotopsy pinned, or blutter built from source) and says so instead of implying otherwise.
- **Unchanged:** Unity/IL2CPP, React Native/Hermes, iOS, defeating a server-side authority, and the
general-unpacker/anti-detection arms race all remain uncovered.
- **Added:** an explicit statement that packer, VMP and integrity-check scenarios were **not
exercised** by this pass, so no claim is upgraded on their behalf.
### `SKILL.md` — the three hand-off points
Written from the perspective that a hand-off is a pointer, and that duplicating content across two
places is how the two copies drift. Each one says what the other side owns.
1. **JNI** — Java declaring a native method ↔ its `.so` implementation. Static linkage shows as
`Java_com_pkg_Class_method` and disappears under R8 short names; dynamic registration leaves the
symbol table empty and must be found at `RegisterNatives`. This target ships the static form
(`FlutterJNI.loadLibrary` appears in `classes3.dex`) but the pass did not trace a JNI boundary
end to end, so the section carries the **unverified** label.
2. **Hardening** — dex-side observation of a packer implies a native-side shell implementation.
**Not exercised here**: this target has no packer. Recorded as a pointer with that stated.
3. **Existing boundary documents** — `native-and-so.md` and `native-tamper-and-suicide.md` read
native anomalies *from the APK side* (immediate SIGSEGV after a repack, Java claiming the check
passed while the process dies). That belongs in A; deep native work is the other side. The new
section points each way rather than repeating either.
### `docs/tool-verification/`
New directory holding this pass's evidence record. `README.md` states the folder is a *measurement
record, not shipped target data*, carries the strength-label convention, and requires that **no target
identity is recorded** — only what a future reader can apply to a different APK. It is referenced from
`SKILL.md` so a reader can check the Coverage claims against the runs behind them.
No APK is in the repository (`.gitignore` already excludes `*.apk`) and no credential or token
appears in any file.
### `README.md`
Updated for the directory tree, the install command and the division of labour between the skills,
plus a pointer to the verification record.
---
## Verification
All four gates were re-run against the final tree:
- `python -B check_repo.py`
- `python -B check_refs.py`
- `python -B build_scripts.py . <out>`
- `npx skills add newliver666/apk-reverse --list`
Two behaviours of the checkers worth knowing, both established by experiment during this pass:
`check_repo.py` and `check_refs.py` have **no argparse**, so `--help` is silently ignored and the
full check runs; and running `check_repo.py` with `PYTHONIOENCODING=utf-8` set produces a spurious
failure (exit 1, one script reported as crashed) because its `subprocess.run(..., text=True)` does
not pin an encoding while the child emits UTF-8 — the parent then decodes as cp936 and the streams
come back empty.
---
## A later pass: fixing the `find_refs.py` defect, and what measuring it exposed
The three defects recorded above as "documented but not fixed" were left alone for a stated reason:
the repository ships nothing unrun. One of them — `find_refs.py` and `.dex` input — was reported again
from outside with a clean reproduction, and that reproduction is correct. It is now fixed, together
with a deeper defect the fix exposed.
### `find_refs.py`
Accepts a `.dex`, a directory of `.smali` and/or `.dex`, an `.apk`, or a smali tree. Dex input is
decoded through `dexutil`, so baksmali is not required — which matters because the machine where the
defect was found had no baksmali, jadx or apktool, and the tool was therefore completely unusable for
that target. Input it cannot read is **refused with exit 2 and a reason**, and every run prints
`[scanned] N file(s)` before the count, so "nothing was readable" can no longer print the same line as
"read N files and found nothing".
### `dexutil.py` — the part that could not be skipped
`find_refs.py`'s dex path depends on `decode()`'s instruction boundaries, so those were measured
instead of assumed. They were wrong: the opcode table was shifted from `0x16` to `0x2C` with nine
widths wrong (F9), and 65 of 50,791 method bodies on a real dex did not decode to their declared
boundary.
Fixing `find_refs` while leaving that in place would have produced the same silent false zero the
defect was reported for, one layer down. So names and widths were regenerated from the format
specification's table, width became a lookup (`OP_UNITS`) rather than a chain of range tests, and the
`goto` family was corrected in `branch_target()` and in `dex_find_insn.py`'s kind map.
### Verification
- `[scanned]`/`[total refs]` from a `.dex` and from the equivalent smali tree **agree with each other
and with the count written into the fixture by construction** (3 / 2 / 2).
- **50,791 of 50,791** method bodies decode to their declared boundary (65 did not before), and **0**
operand indices fall outside their table (32 did before).
- **androguard 4.1.4**, a decoder sharing no code with this project, produces **identical
per-instruction width sequences for all 50,791 bodies**.
- Real target: `Ljava/lang/String;->length` → 356 references in 2.3 s; a five-dex directory → 9,472
across 2 files; a typo'd path → exit 2.
- The four repo gates still pass: `check_repo.py` 0 problems, `check_refs.py` 0 dangling,
`build_scripts.py` 0 findings, `npx skills add … --list` lists the skill.
### Why the fixture is in the tree
`docs/tool-verification/fixtures/make_fixture_dex.py` is committed because it is the evidence for the
counts above. Without it, "3 references" is a claim a later reader cannot check — which is exactly the
evidentiary gap F5 describes, and the repository's rule that nothing ships unrun applies to claims as
much as to code. It is dependency-free (stdlib only) so it reproduces on a machine with no Android
toolchain, which is the machine the defect was found on. It carries no target data: the dex is
synthesised from constants in the file.
## Maintenance pass: the path audit's signal, three path families, and a corpus note
An external review found the repository in good shape on evidence and honesty but raised three
concrete defects. All three are fixed here; the fourth item is a measurement added so the next
pass can see what this one could not.
### 1. `build_scripts.py` reported 37 findings, most of them noise
The audit matched **any** `X:\...` literal, so `C:\Program Files\...`, `C:\Python314\...` and
literal placeholders like `<user>` all counted as "machine-specific leftovers". A gate whose output
is mostly noise stops being read, which is the same failure `scan_leaks.py` names when it refuses to
exempt the RFC 5737 documentation ranges.
Added two explicit, auditable exemption families rather than a looser regex:
- `GENERIC_PATH_PREFIXES` — install locations that exist on every host of that platform
(`Program Files`, `Python*`, `MinGW`, `/usr/`, `/opt/`, ...).
- `_leaks_nothing()` — also exempts an already-redacted path, i.e. one carrying a placeholder or an
ellipsis, because that mark means a normalisation pass already ran.
**One trap had to be handled explicitly.** `BARE_ABS` stops at whitespace, so a real
`C:\Program Files\...` arrives as the truncated `C:\Program`; a one-directional `startswith` misses
the exemption. The comparison is therefore bidirectional, gated by a minimum length so a short drive
prefix cannot become a blanket pass. The same function needed a second correction: with a lookahead
of only `[a-z0-9]`, the optional `\\?` backtracks to the empty string, the next character is the
separator, it passes, and a real personal path is swallowed — the lookahead must exclude the
separator too.
### 2. Three absolute paths left on the published surface
`EXTENSION-desensitization.md`, `EXTENSION-extraction-shell-bench.md` and `EXTENSION-vmp-diff.md`
quoted a workbench root, a tool jar and an SDK location as absolute paths. `docs/tool-verification/`
already states the rule (§**Workbench paths are an operation record, not a pointer**): the record
may say a run happened at a `tools/...` path, written relatively. The three files were made to match
the rule they were already under; the tool-jar and SDK cases were rephrased rather than
placeholder-ised, so no reader has to learn a new token.
### 3. `check_budget.py` could not see corpus growth
The gate budgets the always-loaded part and was silently blind to the other half of the problem: a
new reference costs exactly **one index line**, so the corpus can double while the narrative count
stays green. Two note-level thresholds now report it. Notes, not failures — a large corpus is a
choice; a growth nobody measured is the defect.
### A cross-tool inconsistency this pass ran into
`build_scripts.py` now exempts any `<...>` placeholder; `scan_leaks.py` recognises a **fixed token
list** (`<user>`, `<name>`, `<path>`, ...). An invented placeholder in a source comment —
`<someone>` — therefore passed the first and was read by the second as the very personal name its
`user_path_windows` rule exists to catch, turning `check_repo.py` red. That is by design rather than
a bug (a strict list is what stops `<realname>` hiding a real one), but the two tools disagree about
what "already redacted" means, and the disagreement is invisible until someone writes a new
placeholder. The fix here was to rephrase the comment; a future pass may want a shared token list.
### Verification
- `build_scripts.py` **37 → 0** findings on a clean clone, exit 0 (was exit 2). The generic-install
hits (`C:\MinGW\bin\gcc.exe`, `C:\Python314\...`, `C:\Program`) no longer report; the real ones
(`E:\apk-reverse\...`, `E:\agb_apktool.jar`) did, and were fixed in the documents, not by widening
the list.
- A purpose-written case table pins **both** directions — 18 cases, 0 failures: generic and
already-redacted paths are exempt, while a real workbench path, a real personal name after
`C:\Users\`, a real POSIX home and a real project path still report. Two defects in the first
implementation were found by that table, not by the audit run.
- All four gates pass: `check_repo.py` 0 problem, `check_refs.py` 0 dangling / 0 warning,
`check_budget.py` 0 failure / 0 warning / 3 notes, `build_scripts.py` 0 findings.
- `check_budget.py` now reports the corpus explicitly: **47 references + 55 scripts = 102 files,
31,423 lines**, past the 26,000-line note threshold — the figure this pass could not see before.

554
vendor-new/apk-reverse/docs/tool-verification/TOOL-VERDICTS.md Normal file
View File

@@ -0,0 +1,554 @@
# Tool verdicts — measured on a real target
One verdict per script, drawn from the closed set:
| Verdict | Means |
|---|---|
| **verified on real target** | Ran here, and its output was confirmed by an independent method that does not share code with it |
| **broken — fixed** | Was wrong; the defect is localized and a corrected copy was run and re-verified |
| **broken — cause identified, not fixed** | Was wrong; root cause established, no fix attempted |
| **unusable here because …** | Correct diagnosis, but this environment/target cannot exercise it |
| **not applicable to this sample** | The target lacks the feature the tool addresses |
The distinction that governs the whole table: **a script that runs without error is not a script
that is right.** Every "verified" below names the independent cross-check, because "it didn't
crash" was not accepted as evidence anywhere in this pass. Equally, a tool failure was never read
as "the target does not contain that thing" — each failure was first checked against the question
of whether the tool was ever able to answer it.
---
## Summary
| Script | Verdict |
|---|---|
| `elf_plt.py` | **broken — fixed** (core stub→symbol mapping verified on real target; two defects, one proven to cause a false negative) |
| `lib_map.py` | **verified on real target** |
| `so_constpatch.py` | **verified on real target** for equal-length rewriting; **broken — cause identified, not fixed** for one sealed serious defect (zip alignment) |
| `dex_strings.py` | **verified on real target** (multi-dex traversal confirmed) |
| `dexutil.py` | **verified on real target** (used as the primary workhorse for the ad-chain analysis) |
| `find_refs.py` | **broken — fixed and re-verified**; reads a `.dex` directly now |
| `dart_pool_strings.py` | **verified on real target** |
| `dart_pprefs.py` | **broken — fixed** (defect proven; corrected copy agrees with capstone to zero) |
| `dart_disasm.py` | **verified on real target** |
| `coldstart.py` | **verified on real target** (the run produced the launch-crash finding) |
| `native_crash.py` | **not applicable to this sample** (no artificially constructed crash; see `TARGET-FACTS.md`) |
| `so_constpatch.py` redirected-checker scenario | **not applicable** — no integrity-check library exists in this target |
| All packer / VMP tooling | **not applicable** — no packer in this target |
| blutter (external) | **works** — build ≈78 s, transient first-run `0xC0000005` not reproducible after relink |
| aotopsy (external) | **works out of the box** — full pipeline on the same `libapp.so`
Two of these verdicts matter more than the rest: `elf_plt.py` produced a **false negative** on a
symbol that exists and is called, and `find_refs.py` returns a clean "no references" answer for
input it cannot read at all. Both are the failure mode that looks like a finding.
---
## `elf_plt.py` — broken — fixed
**Claim under test:** walks `PT_LOAD`/`PT_DYNAMIC` directly, without depending on the section
table, maps PLT stubs to imported symbols, and diffs two `.so` files byte-wise.
**Sample:** `lib/arm64-v8a/libloader.so` (6,221,896 B, largest `.dynsym` in the APK).
**Run:**
```
python scripts/elf_plt.py work/native/samples/arm64-v8a_libloader.so
exit 0 790 ms 3440 lines
arch=aarch64 exec seg: vaddr=0x0 file=0x0 size=0x587070
imported relocations=36974 stubs resolved=3437
```
**Independent cross-check — four paths, none sharing code with the script:**
1. **Relocation total.** `readelf -d` (pyelftools) reads `RELASZ=804864`, `PLTRELSZ=82512`,
`RELAENT=24`. `804864/24 = 33536` (`.rela.dyn`) + `82512/24 = 3438` (`.rela.plt`) `= 36974`.
Matches the script's `imported relocations=36974`.
2. **Symbol names, item by item.** pyelftools parses `.rela.plt` + `.dynsym` independently
(3,438 entries, all `R_AARCH64_JUMP_SLOT=1026`). Against the script's 3,437 output lines:
`name mismatches: 0`. The script's GOT set is a subset of the `.rela.plt` GOT set, with an
intersection of **0** against `.rela.dyn` — i.e. it does not mistake local relocations for
imports. Zero `?` names.
3. **lief 1.0.0** independently reports `pltgot_relocations = 3438`, equal to the pyelftools set,
with zero name disagreements.
4. **capstone 5.0.7 real disassembly** of all 3,437 reported stub addresses, 16 bytes each, with the
GOT recomputed from the decoded instructions: `shape_violations: 0` (every stub is
`adrp x16 / ldr x17,[x16,#off] / add x16,x16,#off / br x17`) and
`got_recompute_mismatch: 0`.
Conclusion: the four hand-written bitfield decoders (`_decode_adrp`, `_decode_ldr_unsigned`,
`_decode_add_imm`, `_decode_br`) are **correct**, not merely non-crashing.
### Defect 1 — loop bound, proven to cause a false negative
`.rela.plt` has **3,438** entries; the script reports **3,437**. The arithmetic:
- first `.plt` stub `0x579990` + 3438 × 16 = **0x587070**, exactly the executable segment end
(`size=0x587070`)
- the source reads `while off < fsz - 16`, so the largest offset examined is `0x58705c`; the final
slot `0x587060..0x58706f` is never checked. `fsz-16` would need to be `fsz-15`; the correct form
is `off + 16 <= fsz`.
The skipped slot is a legitimate stub, proven by independent disassembly:
```
0x587060 adrp x16, #0x5c8000
0x587064 ldr x17, [x16, #0x108]
0x587068 add x16, x16, #0x108
0x58706c br x17
recomputed GOT = 0x5c8108 -> .rela.plt entry for that GOT: dl_iterate_phdr
BL/B callers of the skipped stub: 1 ['0x576a2c']
```
And the script therefore answers wrongly, with a confident negative:
```
$ elf_plt.py libloader.so --query 0x587060
0x587060 -> not a resolved stub <-- FALSE NEGATIVE
```
This is the most important single result of the native pass. The script does not error, its output
looks complete, and for its stated purpose — *"which stub points at the hardening symbol I want to
neutralise"* — it reports that a real, called symbol (`dl_iterate_phdr`) does not exist in the
library. That is precisely the failure mode the script's own docstring warns about.
**Fix**, applied to a copy and re-verified — three isomorphic bounds:
```
_stubs_x86: while i < fsz - 6 -> while i + 6 <= fsz
_stubs_arm64: while off < fsz - 16 -> while off + 16 <= fsz
find_branch_callers: while off < fsz - 4 -> while off + 4 <= fsz
```
After the fix: `stubs resolved=3438`, equal to the `.rela.plt` entry count. The diff against the
original output is **exactly two lines** — the header count and the single new
`0x00587060 -> dl_iterate_phdr (GOT 0x5c8108)` — with all other 3,437 lines byte-identical.
`--query 0x587060` now resolves correctly.
### Defect 2 — `--diff --name-regions` compares a file offset against a virtual address
`cmd_diff`'s `describe(off)` receives a **file offset** and compares it against stub **vaddrs**
(`va <= off < va + size`). Those coincide only when `p_vaddr == p_offset`. Two of the 36 libraries
in this target violate that:
```
lib/arm64-v8a/libflutter.so exec PT_LOAD vaddr=0x444880 offset=0x434880 (delta 0x10000)
lib/armeabi-v7a/libflutter.so exec PT_LOAD vaddr=0x277500 offset=0x267500 (delta 0x10000)
```
Demonstrated by mutating a byte inside the `abort` stub (stub vaddr `0x9f6050`, file offset
`0x9e6050`) and diffing:
```
$ elf_plt.py libflutter.so --diff flutter.mutated.so --name-regions
differing bytes=1 in 1 run(s)
off=0x9e6053 len=1 A: b0
B: b1
<-- the "<- PLT stub for abort" annotation is missing
```
Plain `--diff` and `--diff --name-regions` produce **identical** output, i.e. the feature is
completely inert on these two libraries and reports nothing about being inert.
**Fix:** map the file offset back through `PT_LOAD` before comparing.
**Also:** `size = 16 if elf_a.machine == EM_AARCH64 else 16` — both branches identical; dead code.
**Credit where due:** the script's defensive warning fires correctly on a zero-stub target.
`libapp.so` (Dart AOT, 144-byte `.dynsym`) reports `stubs resolved=0` **and** prints an explicit
"do not read this as 'the library imports nothing'" message. That is exactly the right instinct —
it is unfortunate that the scanner then violates that principle itself, one slot early.
---
## `lib_map.py` — verified on real target
**Claim under test:** reads a live process's mapping table and reports which libraries are actually
loaded, at what base, with what real architecture.
**Run:** `--pkg <PKG> --serial <DEVICE>`, exit 0, 635 ms, pid 27064.
Reported three actually-loaded app libraries — `libloader.so` (3 segments),
`libsentry-android.so` (2), `libsentry.so` (3).
**Independent cross-check:**
- `su -c cat /proc/27064/maps` parsed with an independent parser: path set, segment counts and
permission strings agree with **zero** differences.
- `od` read of the ELF header gives `b7 00` = AArch64, matching the script's architecture column.
- Scale check against `systemui`: **318 libraries**, compared item by item — path, segment count
and permissions all zero-difference; `--json` base addresses agree **318/318** with independent
computation.
- Translation-layer detection fires correctly on the emulator
(`/system/lib64/libhoudini.so` → `translation suspected: YES`).
**Improvement, not a defect:** `?` in the architecture column is two-valued in the docs —
`read_elf_arch`'s docstring says `?` = read failed; the CLI's trailing note says `?` = not probed.
All 8 libraries printed as `?` were independently read and are AArch64, i.e. they exist and simply
were not probed. A genuine read failure is indistinguishable from "not probed".
**Anomaly resolved as not-a-bug:** one crosscheck showed `libsentry.so 3 vs 4` segments. Four
snapshots established this as runtime mapping change (it is 3 segments at the instant of launch),
not a script defect.
**The real constraint on this tool here:** the target lives ~1.1 s, so `lib_map.py` must run inside
that window. That is also why the app's own libraries are limited to three — `libapp.so` and
`libflutter.so` had not been loaded yet.
---
## `so_constpatch.py` — verified (core) with one sealed serious defect
**Claim under test:** equal-length rewriting of string constants inside a `.so`.
**Verified by:** length unchanged at 6,221,896 bytes, sha256 changed, and an independent byte-wise
diff showing changes at exactly one site (`0x81593`, 7 bytes). `readelf -d` confirms the change
landed at the ELF semantic layer: the `NEEDED` entry changes from `libandroid.so` to
`libfoo0000.so`. Guards behave as designed — unequal-length input exits 1 and writes no file; `-o`
pointing at the input exits 2; non-isolated operation refuses by default. On a real APK copy: 3.5 s,
**all 1,982 entries preserved**, order preserved, only the target entry's CRC changed, original APK
sha256 unchanged before and after. All patching was done on copies; no sample was mutated.
### Defect — zip alignment destroyed on rebuild (sealed, not fixed)
The rebuild discards the local header `extra` field, which breaks 4 KB alignment. Measured on a
fixture: `data_offset 4096 (mod4096=0, extra_len=4031)` becomes `65 (mod4096=65, extra_len=0)`.
**Why this is serious despite not firing here:** for an APK with `extractNativeLibs=false`, `.so`
entries must be `STORED` and 4 KB-aligned or the library will not load. This target cannot show the
bug — all 36 native libraries are `method=8` (DEFLATED) and `extractNativeLibs=true` (confirmed in
both the manifest and by the runtime path `/data/app/.../lib/arm64/` that `lib_map.py` observed).
So the defect is **masked by this sample and proven by a fixture**. That asymmetry is exactly why
"it worked on our target" is not a verification.
**Fix sketch:** copy `extra`/`external_attr`/`internal_attr`/`create_system`/`comment` from the
source `ZipInfo` and implement padding; and state in the help text that a rebuild must be
re-zipaligned before signing.
### Defect — entry metadata reset
`external_attr` is rewritten on **1982/1982** entries (e.g. `2175008768 (0x81A40000)` →
`25165824 (0x01800000)`), and `create_system` changes from 3 (Unix) to 0 on 77 entries. Usually
harmless for installation, but it changes extracted permission bits and makes any
zip-metadata-based integrity comparison immediately detect the rebuild.
### Defect — the change count is misleading
`do_replace` prints `'%d byte(s) changed' % (len(applied) * len(old))`, which is the **overwrite
window length**, not the number of bytes that actually changed. Two measured examples:
`libandroid.so → libfoo0000.so` prints 13 but changes 7; `libandroid → libandroiX` prints 10 but
changes 1. For an attribution/audit tool this invites overstatement.
### Minor
Rebuilding changes the APK's own bytes (+2.2 MB on the sample, different deflate level). Worth a
note in the output that output size may differ from input.
---
## `dex_strings.py` — verified on real target
**The multi-dex question the handover flagged is answered: yes, directory mode walks all four dex
files.** `iter_dex()` globs `*.dex` for a directory; empirically
`--find 'json' --per-file` emits four lines (`classes.dex`, `2`, `3`, `4`), while single-file mode
emits one.
Two traps worth knowing:
- `--per-file` prints **only files that had a hit**. An output containing just `classes3.dex` does
not mean only `classes3.dex` was scanned. This is the shape of output most likely to be
misread as "only one dex contains this".
- `--per-file` scans **raw bytes** rather than the string table (source lines 124-130), so hits for
broad patterns include coincidental matches.
Fast: 0.27 s over the full 27.6 MB.
## `dexutil.py` — verified on real target
The workhorse of the ad-chain analysis: symbol resolution, branch-target back-annotation, and a
`ended cleanly` verdict per method. Coverage: 175,829 method bodies with code, 213,180 `method_id`s.
It is the tool whose output the zero-reference conclusion rests on.
Three defects, all found by working at this scale:
- `decode()` does not resolve operands — string and method indices must be decoded by the caller.
- **Switch payloads were decoded as instructions**, contradicting the docstring's claim that
`ident 1..3` are recognized. This was one root cause of `saveToLocal` being reported as
"desync likely".
- **The opcode table was shifted from 0x16 to 0x2C, and nine widths were wrong with it** — fixed.
`0x1C` carried the name `monitor-enter` and one unit; the format specification has `const-class`
there, at two units. `0x17` was `const-wide/16`/2 where the spec has `const-wide/32`/3; `0x18` was
3 where the spec has `const-wide`/5; `goto` was read one slot early, so `throw` was treated as a
branch. The plausible "same family, same width" reading of that run is wrong at nine opcodes, and
each wrong entry shifts every instruction after it inside that method.
Proven by measurement rather than by re-reading the table: on a real 8.9 MB dex, **65 method
bodies did not decode to their declared boundary and 32 carried an operand index outside its
table**; after the fix, **50,791 of 50,791 land exactly on `insns_off + insns_size*2` with 0
illegal indices**. Independently cross-checked against androguard's decoder, which shares no code
with this project: same method count (50,791), and **0 differing width sequences** across all of
them.
Names and widths now come from the format specification's table, and widths are a lookup
(`OP_UNITS`) rather than a chain of range tests. The `goto` family in `branch_target()` and in
`dex_find_insn.py`'s kind map moved with it.
## `find_refs.py` — broken — fixed and re-verified
The docstring said it accepts "a smali tree or a directory of dex files". It did not:
`iter_files()` yielded only files ending in `.smali`. A direct `.dex` was yielded, then read as UTF-8
text and regex-matched against smali syntax, which cannot match; a directory holding only dexes
yielded nothing at all. Both printed `[total refs] 0` and a note blaming the needle, so a real dex, a
dex directory and a typo'd path were **indistinguishable in the output**.
This is the worst shape a defect can take in this particular tool, because `find_refs.py` is the
script recommended for judging blast radius *before* patching. Its silent zero is exactly the
input to a confident wrong conclusion — "nothing references this, safe to patch". Because no
baksmali/jadx is installed on this machine, it was **completely unusable** for this target, and the
analysis had to be built on `dexutil.py` instead.
Two corrections to the record above, both from re-measuring rather than re-reading:
- The exit code was **1**, not 0 — the `total == 0` path has always returned 1, so a caller checking
status could already catch it. What misled was the *text*: the note diagnosed the needle and never
named the input form.
- The dex-as-text path also read the whole file into memory to regex-scan binary for nothing, then
dropped it.
Fixed in three ways at once, because all three were the same omission — the tool did not tell the
difference between "found nothing" and "read nothing":
- A `.dex` is decoded through `dexutil` (no baksmali needed), a directory is walked for `.smali` and
`.dex` alike, and an archive is opened for its `classes*.dex`.
- Unsupported or unreadable input is **refused with exit 2** and a reason, instead of being reported
as zero references.
- Every run prints `[scanned] N file(s)` first, so the two cases can never print identically again.
Re-verified twice, in both directions:
- Against a fixture whose call graph is known **by construction** (`make_fixture_dex.py` writes two
classes with 3, 2 and 2 references): the `.dex` and the equivalent smali tree return the same
counts, and they match what was written — 3 / 2 / 2.
- Against the real 8.9 MB target: `Ljava/lang/String;->length` → **356 references in 2.3 s**; a
five-dex directory → 9,472 across 2 files; a typo'd path → `[error] no such file or directory`,
exit 2.
**Correction to a related assumption in this environment:** `readelf` on `PATH` resolves to
pyelftools' `readelf.py` and is *not* binutils — `--dyn-syms` and `-W` are rejected as unrecognized
arguments (exit 2). Likewise MinGW `objdump`/`nm` (binutils 2.28) reject every aarch64 ELF with
`File format not recognized`. Aarch64 cross-checks in this pass therefore used the **pyelftools,
lief and capstone library APIs**, not those CLIs. There is no `xxd`; hex work was done in Python.
---
## Dart / Flutter scripts
Verified facts, each independently cross-checked rather than accepted on exit code:
**`dart_pool_strings.py` — verified on real target.** 18,363 rows recovered. Byte-exact re-check of
tag + payload: **18,356 exact, 7 mismatched**, and all 7 mismatches are a single known escape
ambiguity (`\n`/`\r`/`\t` written as the two characters backslash+n, indistinguishable in a TSV from
a literal backslash followed by n — 19 recovered entries genuinely contain LF/CR/TAB). 12 rows
sampled at random were independently re-located from the raw bytes; **all 12 reproduced**, with tag
at `tag_off` and payload at `tag_off + 1` exactly. Chain violations among run≥2 rows: **0**.
**`dart_pprefs.py` — broken — fixed.** The first run produced a real defect: **302 displacements and
1,802 reference sites missing**. Correcting the mask makes it agree with capstone exactly:
```
repo (before) repo (fixed) capstone
displacements 31,843 32,145 32,145
reference sites 97,571 99,373 99,373
only-in-capstone 302 0 —
only-in-repo 0 0 —
```
**Root cause — corrected after deeper analysis, and worth recording because the first hypothesis
was wrong.** The obvious guess was an `x27` addressing mask. That is not it: `LDUR` with `Rn=x27`
occurs **zero** times in this snapshot. The actual cause is that the scanner decodes only the
**general-purpose register file** — `ldr dD, [x27, #imm]` (encoding `0xFD400000`) was never matched
at all, so an entire class of **double constants** loaded from the object pool was silently
dropped. Measured recall 98.19%, precision 100%; after the fix, capstone and the script agree
across the whole `.text` with 0/0.
Why this is the expensive kind of bug: it breaks precisely the judgement the documentation tells
you to rely on in `dart-aot.md` §9/§10 — *count the references to decide whether a pool entry
matters*. At one measured site the count goes from `[0 refs]` to `[79 refs]`. A reader following
that advice would conclude a live, heavily-referenced object was dead. It affects all Dart 3.x
arm64 snapshots, and the failure is one-sided and silent.
**`dart_disasm.py` — verified on real target.** Decoded output compared against capstone on the same
ranges: **32/32 and 96/96 instructions identical**, `VERDICT: identical decode`. The independently
re-derived caller index also agrees exactly: `callers.json targets=59904 sites=219181`, independent
re-derivation `targets=59904 sites=219181`, **symmetric difference of call sites: 0**. All B/BL
words in range confirmed by capstone: 27,615/27,615.
**`dart_disasm.py` — verified on real target.** Decoded output compared against capstone on the same
ranges: **32/32 and 96/96 instructions identical**, `VERDICT: identical decode`. The independently
re-derived caller index also agrees exactly: `callers.json targets=59904 sites=219181`, independent
re-derivation `targets=59904 sites=219181`, **symmetric difference of call sites: 0**.
**`dart_pool_strings.py` two-byte mode — unusable here because detection is unreliable.** With
`--two-byte`: 936,136 candidates, 57,049 kept (34,216 UTF-16LE). The script's own warning says the
detection is unreliable, and the independent measurement confirms it: 168,700 independent UTF-16LE
CJK runs, and of 9,840 two-byte CJK candidates only 2,081 form chained runs ≥2 — the rest are
coincidental. Sample output is noise (`誋讋讋讋讋`). The one-byte default is the right default.
---
## `coldstart.py` — verified on real target
Used exactly as documented — `--serial`, `--pkg`, `--activity`, `--duration`, `--interval`,
`--expect-activity`. It behaved as advertised and produced the finding that governs this whole
report: the app reaches a splash window and then aborts. The script's existence is why the launch
failure was caught as a *measurement* (timestamped foreground-activity and logcat timeline) rather
than as a vague impression, and its `--expect-activity` design directly addresses the failure it
names in its own docstring — a vendor installer window being captured and read as the app.
---
## External toolchains
Both Dart AOT analyzers were tested, and they behave differently enough that the difference is
itself a finding.
### blutter — **works; the first-run crash is not reproducible**
`blutter` compiles and embeds a matching Dart VM, so it needs one build per Dart version. Building
from source is **mandatory** — the repository's `bin/` is gitignored and it publishes **no releases**,
so there is no prebuilt Dart 3.6.0 executable to fetch. (Naming, for anyone searching:
`bin/blutter_dartvm3.6.0_android_arm64.exe`, not `_windows_x64`.)
Measured cost, which corrects the documentation:
| Step | Time |
|---|---|
| `git clone --depth 1 worawit/blutter` (HEAD `4a60ac6`) | seconds |
| `python scripts\init_env_win.py` (ICU 73.2 + capstone 4.0.2) | **7,049 ms** |
| sparse clone `dart-lang/sdk` tag `3.6.0` | seconds |
| cmake + ninja, 22 targets (unity build via `dartvm_create_srclist.py`) | — |
| link + install `bin/blutter_dartvm3.6.0_android_arm64.exe` | — |
| **whole pipeline, wall clock** | **≈ 78 s** |
`dart-aot.md` §2 budgets "**tens of minutes** for the first build". Reality here is **≈78 s**,
roughly **30× pessimistic**. The toolchain requirement is also overstated: the README asks for a
very recent C++ compiler with C++20 `<format>` and the doc repeats "VS 2022 required", but
**MSVC 19.34 (VS 17.4.3, Nov 2022) is sufficient** — verified by compiling and running a
`std::format` probe with it.
**The crash, and a correction to the first reading.** Run 1 (`blutter.py` under `vcvars64`) built the
VM and linked the exe, then died with exit `3221225477` = **0xC0000005 ACCESS_VIOLATION** and **no
output at all**. Runs 2-4 (invoking the installed exe directly, on `libapp.so`, `libflutter.so` and a
nonexistent path) reproduced it in ~89 ms each — always after argument parsing succeeded (a bad ABI
still prints `ELF: Support only 64 bits` and exits 0) and before the first stdout line, pointing at
`DartApp app{ libappPath.c_str() }` (`blutter/src/main.cpp:29`).
Adding two `std::cerr` traces and relaunching `ninja` (2 targets: recompile `main.cpp`, relink)
produced an executable that **ran to completion, exit 0** (`libapp is loaded at 0x1dfe6d60000 /
Dart heap at 0x1e000000000`) — and then the **originally installed binary also ran clean**: 3/3 runs,
exit 0, 5,899 / 5,862 / 5,928 ms, each producing `pp.txt` (2,639,894 B), `objs.txt` (648,382 B),
`asm/`, and `blutter_frida.js`.
So the verdict is **not** "unusable here". The root cause of the transient fault is **not identified**
and it did not reproduce after the relink, including for the unchanged binary; the best-supported
reading is a first-run/environment artifact rather than a property of this snapshot or of Dart 3.6.0.
Cleared suspects: `std::format` works on MSVC 19.34; capstone 4.0.2 and ICU 73.2 DLLs sit beside the
exe; `--no-analysis` is unnecessary because the full path works. The practical rule this yields:
**budget one retry, and if it still faults, relink before concluding anything about the sample** —
here a tool crash did not mean the target lacked anything.
### aotopsy — **works out of the box**
`aotopsy_v1.6.0_windows_amd64.zip`, 2,994,902 B, sha256
`5a90bcf27f4aaa281ae393bf3038b6e1e2325386fe61c1a18ed089390983fdc1` (matches the release manifest).
Unpack and run — **no compiler, no Dart SDK, no toolchain**, because it parses the snapshot format
directly in Go.
| | result |
|---|---|
| `doctor` | Support **OK**; compressed 4-byte pointers; hash `f956f595844a2f845a55707faaaa51e4`; the same 9 feature flags blutter derived |
| full pipeline | exit 0 — **30,586 functions**, 6,133 class layouts, **39,202 pool entries (38,274 resolved)**, **175,120 call edges** (18,718 BLR, **95.7% annotated**), 20,927 string refs, 556 signal findings |
| artifacts | `functions.jsonl`, `classes.jsonl`, `call_edges.jsonl`, `string_refs.jsonl`, `dispatch_table.jsonl`, `evidence.jsonl`, `asm/` (16,046 files), a `--decompile` pseudocode mode, `_debug strings --xref` |
**The Dart version discrepancy, resolved.** aotopsy `doctor` says **3.6.2**; blutter derives **3.6.0**
from the engine banner; the banner reads `3.6.0 (stable) (Thu Dec 5 07:46:24 2024 -0800)`; and there
is **no `3.6.2` byte sequence anywhere in `libflutter.so` or `libapp.so`**.
Judgement: **3.6.0 is the correct SDK version (observed)** — three independent facts agree (byte
search, blutter's parser, and the fact that the Dart 3.6.0 runtime blutter built accepts the snapshot
hash). aotopsy's `3.6.2` is its own **structural profile label (inferred)**: its README states version
detection is structure-based because the snapshot carries a git-derived hash rather than a version
number, so the label names the newest profile matching that hash. Impact on support is nil, but
**`aotopsy doctor`'s number must not be fed to a VM-compiling tool as an SDK tag.** The banner route
taught in `dart-aot.md` §1 is correct and is the best-validated instruction in that document.
---
## End-to-end: can the kit locate one specific piece of business logic?
**Yes — with an external snapshot dumper in front of it.** This is the question the whole pass was
built to answer, so the worked example is recorded in full.
| # | Tool | Result |
|---|---|---|
| 1 | blutter `pp.txt` | `[pp+0x23548] String: "showRewardAdProvider"` |
| 2 | `dart_pprefs.py --lookup pp_refs_fixed.json 0x23548` | **1 ref site: `0x7a9464`** |
| 3 | `dart_disasm.py libapp.so --pp bout5/pp.txt --refs pp_refs_fixed.json 0x7a9464` | annotated window (below) |
| 4 | `dart_disasm.py --index callers.json 0x7a93ec` | **9 callers** |
| 5 | independent confirmation | `aotopsy _debug strings --find rewardAd --xref` → `used in: ShowRewardAdFamily.call_2f28ac @ 0x7a93ec (pool load @ 0x7a9464, pool[18087])` |
Step 3 resolves the site completely, including the enclosing Dart closure and its source file:
```
7a943c: add x1, x27, #0x23, lsl #12
7a9440: ldr x1, [x1, #0x540] ; pp+0x23540 AnonymousClosure: (0x7a9508),
; of [package:<app>/pages/video/video_page.dart] ShowRewardAdProvider
7a9454: add x1, x27, #0x23, lsl #12
7a9458: ldr x1, [x1, #0x430] ; pp+0x23430 Obj!ShowRewardAdFamily@b487a1 : { [8 refs]
```
Two analyzers built by unrelated projects agree on the function name, the address and the pool index
— which is what makes this a verification rather than a coincidence.
**But the kit cannot take step 1 by itself.** Reaching a pool offset from a string requires resolving
the snapshot's internal container, and the kit has no such resolver: 4,241 strings share a name across
4,237 distinct deltas with no constant bridge between the string table and the object pool. blutter
(or aotopsy) supplies that front end. Everything downstream of it — the cross-reference count, the
disassembly window, the caller index — the kit does do, and correctly.
### Documentation defect that matters more than the crash
`dart-aot.md` §2 says of blutter's `asm/`: *"per-class declarations — instructions — **there are none
here**. Do not plan around it."* That is **wrong** for this blutter HEAD on Dart 3.6.0.
`bout5/asm/<plugin>/src/ads/reward/` is 1,327 lines / 71,099 B and contains class
layouts, function signatures **and a full instruction listing with pool annotations and resolved call
targets**:
```
// 0x923b3c: r0 = LinkedHashMap.from()
// 0x923b3c: bl #0x60165c ; [dart:collection] LinkedHashMap::LinkedHashMap.from
```
`asm/` is the single richest artifact in the chain, and the document tells the reader to ignore it.
(Secondary nit: `asm/` is one file per **library URI**, mirroring the package path, not per class.)
## Not applicable to this sample — stated, not stretched
- **`native_crash.py`'s "artificially constructed crash" verdict.** The tool exists to decide
whether a crash was *manufactured* as an anti-tamper response. This target's crash is a genuine
internal mutex-lifecycle fault that occurs on the unmodified APK, before any modification, on two
devices. There is no decoy to detect and no tamper event to attribute, so the judgement the tool
makes was never exercised. **Not applicable — no artificial crash exists here.**
- **`so_constpatch.py`'s "redirect the checker library" use case.** Requires an integrity-check
library to subvert. This target has **none** — no packer, no integrity checker, no `.so` that
validates the APK. The redirection capability is real (see the verified `NEEDED` rewrite) but its
purpose-built scenario does not exist here.
- **`packers.md` / `code-virtualization-and-custom-linkers.md` in full.** No packer: zero hits for
every known hardening-library name, and the parsed manifest shows an ordinary application class
rather than a shell stub. **Unverified in this pass** — nothing here should be read as evidence
either way about how those documents perform.
- **`native-tamper-and-suicide.md`'s judgement chain.** It reads *native anomalies observed from the
APK side* (immediate SIGSEGV after a repack, the Java layer saying the check passed while the
process dies). This pass never produced a repack to observe, because **the unmodified app does not
start**, so there is no working baseline to regress from. The document's scenarios remain
unexercised.

262
vendor-new/apk-reverse/docs/tool-verification/fixtures/make_fixture_dex.py Normal file
View File

@@ -0,0 +1,262 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Build a minimal, structurally valid dex whose reference counts are known.
WHY THIS EXISTS
---------------
Verifying a dex reader against a real APK only shows that two readers agree.
Verifying it against a program whose call graph was written down in advance shows
that the *number* is right. This fixture encodes exactly two method bodies:
Lcom/example/Caller;->go()V invoke-static Lcom/example/Target;->run()V x2
Lcom/example/Target;->run()V invoke-virtual Ljava/lang/String;->length()I x3
so the answers a blast-radius check must return are 3, 2 and 2. The same call
graph is also written out as a smali tree, because the point of the fix this
fixture verifies is that a `.dex` and a smali tree answer a needle identically.
USAGE
-----
python make_fixture_dex.py out.dex [--smali DIR]
# then, from the skill's scripts directory:
python find_refs.py out.dex 'Ljava/lang/String;->length' # expect 3
python find_refs.py out.dex 'Lcom/example/Target;->run' # expect 2
python find_refs.py out.dex 'Lcom/example/Target;' # expect 2
python find_refs.py DIR 'Ljava/lang/String;->length' # expect 3
Dependency-free (stdlib only) on purpose: the fixture must be reproducible on a
machine with no Android toolchain, which is where the defect it verifies was found.
Nothing here is target data. The dex is synthesised from the constants below.
"""
import argparse
import hashlib
import os
import struct
import zlib
CALLER = 'Lcom/example/Caller;'
TARGET = 'Lcom/example/Target;'
OBJECT = 'Ljava/lang/Object;'
STRING = 'Ljava/lang/String;'
SMALI_CALLER = """.class public Lcom/example/Caller;
.super Ljava/lang/Object;
.method public static go()V
.registers 0
invoke-static {}, Lcom/example/Target;->run()V
invoke-static {}, Lcom/example/Target;->run()V
return-void
.end method
"""
SMALI_TARGET = """.class public Lcom/example/Target;
.super Ljava/lang/Object;
.method public static run()V
.registers 1
invoke-virtual {v0}, Ljava/lang/String;->length()I
invoke-virtual {v0}, Ljava/lang/String;->length()I
invoke-virtual {v0}, Ljava/lang/String;->length()I
return-void
.end method
"""
EXPECTED = (('Ljava/lang/String;->length', 3),
('Lcom/example/Target;->run', 2),
('Lcom/example/Target;', 2))
def uleb(n):
out = bytearray()
while True:
b = n & 0x7F
n >>= 7
out.append(b | (0x80 if n else 0))
if not n:
return bytes(out)
def p16(v):
return struct.pack('<H', v)
def p32(v):
return struct.pack('<I', v)
def string_data(s):
raw = s.encode('utf-8')
return uleb(len(raw)) + raw + b'\x00'
def code_item(registers, outs, insns):
body = b''.join(p16(u) for u in insns)
return (p16(registers) + p16(0) + p16(outs) + p16(0)
+ p32(0) + p32(len(insns)) + body)
def invoke(op, method_idx, count=0, c=0, g=0):
"""35c layout: op(G<<8)(count<<12), index, then the register nibbles."""
return [op | (g << 8) | (count << 12), method_idx, c & 0xF]
def class_data(entries):
"""entries: [(method_idx, access_flags, code_off)], ascending by method_idx."""
out = uleb(0) + uleb(0) + uleb(len(entries)) + uleb(0)
prev = 0
for mi, acc, co in entries:
out += uleb(mi - prev) + uleb(acc) + uleb(co)
prev = mi
return out
def build():
strings = sorted({CALLER, TARGET, OBJECT, STRING, 'I', 'V', 'go', 'run', 'length'})
sidx = {s: i for i, s in enumerate(strings)}
types = sorted({CALLER, TARGET, OBJECT, STRING, 'I', 'V'}, key=lambda t: sidx[t])
tidx = {t: i for i, t in enumerate(types)}
protos = [('I', 'I'), ('V', 'V')] # (shorty, return type)
protos.sort(key=lambda p: tidx[p[1]])
pidx = {shorty: i for i, (shorty, _r) in enumerate(protos)}
methods = [(CALLER, 'go', 'V'), (TARGET, 'run', 'V'), (STRING, 'length', 'I')]
methods.sort(key=lambda m: (tidx[m[0]], sidx[m[1]], pidx[m[2]]))
midx = {(c, n): i for i, (c, n, _r) in enumerate(methods)}
insns_go = invoke(0x71, midx[(TARGET, 'run')]) * 2 + [0x000E]
insns_run = invoke(0x6E, midx[(STRING, 'length')], count=1) * 3 + [0x000E]
off = 0x70
string_ids_off = off
off += 4 * len(strings)
type_ids_off = off
off += 4 * len(types)
proto_ids_off = off
off += 12 * len(protos)
method_ids_off = off
off += 8 * len(methods)
class_defs_off = off
off += 32 * 2
data_off = off
data = bytearray()
def align(n):
while len(data) % n:
data.append(0)
str_offs = []
for s in strings:
str_offs.append(data_off + len(data))
data += string_data(s)
align(4)
code_go_off = data_off + len(data)
data += code_item(0, 0, insns_go)
align(4)
code_run_off = data_off + len(data)
data += code_item(1, 1, insns_run)
align(4)
cd_go_off = data_off + len(data)
data += class_data([(midx[(CALLER, 'go')], 0x0009, code_go_off)])
cd_run_off = data_off + len(data)
data += class_data([(midx[(TARGET, 'run')], 0x0009, code_run_off)])
align(4)
map_off = data_off + len(data)
def map_item(kind, size, offset):
return p16(kind) + p16(0) + p32(size) + p32(offset)
items = [
(0x0000, 1, 0), # header_item
(0x0001, len(strings), string_ids_off),
(0x0002, len(types), type_ids_off),
(0x0003, len(protos), proto_ids_off),
(0x0005, len(methods), method_ids_off),
(0x0006, 2, class_defs_off),
(0x2002, len(strings), str_offs[0]), # string_data_item
(0x2001, 2, code_go_off), # code_item
(0x2000, 2, cd_go_off), # class_data_item
(0x1000, 1, map_off), # map_list
]
items.sort(key=lambda it: it[2])
data += p32(len(items)) + b''.join(map_item(*it) for it in items)
string_ids = b''.join(p32(str_offs[i]) for i in range(len(strings)))
type_ids = b''.join(p32(sidx[t]) for t in types)
proto_ids = b''.join(p32(sidx[shorty]) + p32(tidx[ret]) + p32(0)
for shorty, ret in protos)
method_ids = b''.join(p16(tidx[c]) + p16(pidx[r]) + p32(sidx[n])
for c, n, r in methods)
def class_def(class_idx, super_idx, class_data_off):
return (p32(class_idx) + p32(0x0001) + p32(super_idx) + p32(0)
+ p32(0xFFFFFFFF) + p32(0) + p32(class_data_off) + p32(0))
classes = sorted([(CALLER, cd_go_off), (TARGET, cd_run_off)], key=lambda c: tidx[c[0]])
class_defs = b''.join(class_def(tidx[c], tidx[OBJECT], cdo) for c, cdo in classes)
h = bytearray(0x70)
h[0:8] = b'dex\n035\x00'
h[32:36] = p32(data_off + len(data)) # file_size
h[36:40] = p32(0x70) # header_size
h[40:44] = p32(0x12345678) # endian_tag
h[52:56] = p32(map_off)
h[56:60] = p32(len(strings))
h[60:64] = p32(string_ids_off)
h[64:68] = p32(len(types))
h[68:72] = p32(type_ids_off)
h[72:76] = p32(len(protos))
h[76:80] = p32(proto_ids_off)
h[80:84] = p32(0) # field_ids_size
# There are no fields. Size 0 is correct; the offset points at the next
# section rather than 0 only because a zero offset reads as "out of range"
# to a structural check that treats every offset as present.
h[84:88] = p32(method_ids_off)
h[88:92] = p32(len(methods))
h[92:96] = p32(method_ids_off)
h[96:100] = p32(2) # class_defs_size
h[100:104] = p32(class_defs_off)
h[104:108] = p32(len(data))
h[108:112] = p32(data_off)
blob = bytearray(bytes(h) + string_ids + type_ids + proto_ids + method_ids
+ class_defs + bytes(data))
blob[12:32] = hashlib.sha1(bytes(blob[32:])).digest() # signature first
blob[8:12] = (zlib.adler32(bytes(blob[12:])) & 0xFFFFFFFF).to_bytes(4, 'little')
return bytes(blob), len(strings), len(types), len(methods)
def main():
ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])
ap.add_argument('out', nargs='?', default='fixture.dex')
ap.add_argument('--smali', metavar='DIR',
help='also write the same call graph as a smali tree')
a = ap.parse_args()
blob, n_str, n_ty, n_m = build()
with open(a.out, 'wb') as fh:
fh.write(blob)
print('wrote %s (%d bytes, %d strings, %d types, %d methods, 2 classes)'
% (a.out, len(blob), n_str, n_ty, n_m))
if a.smali:
d = os.path.join(a.smali, 'com', 'example')
os.makedirs(d, exist_ok=True)
with open(os.path.join(d, 'Caller.smali'), 'w', newline='\n') as fh:
fh.write(SMALI_CALLER)
with open(os.path.join(d, 'Target.smali'), 'w', newline='\n') as fh:
fh.write(SMALI_TARGET)
print('wrote smali tree under %s' % a.smali)
print('expected: ' + ' '.join('%s=%d' % kv for kv in EXPECTED))
if __name__ == '__main__':
main()