reverse-skills-max: initial public release (restored, redacted, hardened)

vendor-new/apk-reverse/docs/tool-verification/FINDINGS.md

@@ -0,0 +1,249 @@
+# Findings about the skill, from measuring it
+
+Everything here is a result about **the skill**, not about the target. Each item names the evidence
+that produced it and its strength.
+
+---
+
+## F1 — The Coverage claim was partly aspirational, and the pass identified exactly which part
+
+`SKILL.md`'s Coverage section lists as *covered by verified mechanisms*:
+
+> Flutter / Dart AOT: pinning the engine version, decoding the object pool, locating and patching
+> business logic inside `libapp.so` (`dart-aot.md`).
+
+Measured against a real Dart 3.6.0 `libapp.so`, that sentence splits into three claims of very
+different standing:
+
+| Sub-claim | Standing after this pass |
+|---|---|
+| pinning the engine version | **verified** — the banner route (§1) is correct and 3.6.0 is confirmed three ways |
+| decoding the object pool | **false as written** — the kit cannot do it; there is no resolver |
+| locating business logic inside `libapp.so` | **true, but conditional** — it works, on top of an external snapshot dumper that the skill neither ships nor installs |
+
+The worked example in `TOOL-VERDICTS.md` shows both halves at once: the chain
+`"showRewardAdProvider"` → `pp+0x23548` → ref site `0x7a9464` → `ShowRewardAdFamily.call_2f28ac
+@ 0x7a93ec` succeeded, and step 1 was performed by blutter, not by the skill.
+
+The reason is structural, not a shortage of effort: `dart_pool_strings.py` reports **file offsets**
+while `pp.txt` and `dart_pprefs.py` work in **pool offsets**. Over the 4,241 strings present in both
+spaces there are **4,237 distinct deltas** (min −1,361,429, max +277,654). The mapping is a property
+of the reconstructed pool and is not a constant, so no amount of care inside the script recovers it.
+
+Capabilities the kit does not have, stated plainly: snapshot/object-pool deserialisation; symbol and
+name recovery; a real call graph (the `--build-index` index holds raw `B`/`BL` targets only); a
+decompiler; a function boundary/size table.
+
+---
+
+## F2 — Two scripts fail in the way that reads as a target finding (the P26 class)
+
+The handover singled out `pitfalls.md` P26: *"a self-built analysis script fails in a way that reads
+as a target finding."* This pass produced two live instances of exactly that, which is why the P26
+warning earns its place.
+
+**`elf_plt.py` returned a confident false negative.** It reports 3,437 stubs where `.rela.plt` holds
+3,438, and answers `0x587060 -> not a resolved stub` for an entry that is a legitimate stub for
+`dl_iterate_phdr`, with a real caller at `0x576a2c`. Nothing in the output indicates a problem. For
+the script's stated purpose — deciding whether a hardening symbol exists in a library — the wrong
+answer is indistinguishable from the right one.
+
+**`find_refs.py` returned a clean empty result for input it could not read.** Its docstring advertised
+"a smali tree or a directory of dex files"; `iter_files()` accepted only `.smali`. Given the dex
+directory it printed `[total refs] 0`, exited **1**, and emitted no warning. This is worse than a
+crash: it is the exact input to "nothing references this, safe to patch", produced by the very tool
+recommended for judging blast radius before patching.
+
+Re-measured later: the exit code recorded in this section was **0** and it is **1** — the `total == 0`
+path has always returned 1. The correction matters because it moves the defect from "no signal at
+all" to "the signal was there and the text contradicted it": the note diagnosed the *needle* and never
+named the *input form*, so a caller reading the message was still sent the wrong way. Fixed and
+re-verified; see F9 for what measuring it turned up underneath.
+
+Both, plus `dart_pprefs.py`'s silently dropped double constants, share one property: **the failure is
+one-sided and quiet.** None of the three reports reduced confidence in its own output.
+
+---
+
+## F3 — Defects found, ranked by consequence
+
+| # | Script | Defect | Consequence | Status |
+|---|---|---|---|---|
+| 1 | `find_refs.py` | `iter_files()` matched only `.smali`; a dex dir yielded 0 results and a direct dex was regex-scanned as text | silent false "no references" on the tool whose job is blast-radius | **fixed, re-verified against a fixture and the real target** |
+| 2 | `elf_plt.py` | loop bound `off < fsz - 16` misses the final stub slot | false negative for one real symbol per library, conditionally | **fixed on a copy, re-verified** |
+| 3 | `dart_pprefs.py` | only the GP register file is decoded; `ldr dD,[x27,#imm]` (`0xFD400000`) and all double constants lost | under-reports pool references (302 displacements / 1,802 sites here), which breaks the "count refs to judge importance" advice in `dart-aot.md` §9/§10 — one site went `[0 refs]` → `[79 refs]` | **fixed on a copy; agrees with capstone 0/0** |
+| 4 | `so_constpatch.py` | rebuild drops the zip `extra` field, destroying 4 KB alignment | for `extractNativeLibs=false` APKs the output will not load; masked here because this target is DEFLATED + `extractNativeLibs=true` | **cause identified, proven by fixture; not fixed** |
+| 5 | `so_constpatch.py` | `--diff --name-regions` compares file offset with vaddr | feature completely inert on libraries where `p_vaddr != p_offset` (2 of 36 here), with no warning | cause identified |
+| 6 | `so_constpatch.py` | `external_attr` rewritten on 1982/1982 entries | metadata-based integrity comparison detects the rebuild | cause identified |
+| 7 | `dexutil.py` | switch payloads decoded as instructions; `decode()` does not resolve operands | caused a spurious "desync likely" verdict on `saveToLocal` | cause identified |
+| 8 | `so_constpatch.py` | "N byte(s) changed" is the overwrite window, not bytes changed | overstates change count (13 reported vs 7 actual) | cause identified |
+| 9 | `lib_map.py` | `?` means both "read failed" and "not probed" | a real read failure is invisible | cause identified |
+| 10 | `dart_pool_strings.py` | TSV escaping is lossy for LF/CR/TAB | 7 of 18,363 rows ambiguous; 19 entries genuinely contain those bytes | cause identified |
+| 11 | `elf_plt.py` | `size = 16 if ... else 16` | dead code | cause identified |
+| 12 | `dex_strings.py` | `--per-file` prints only files with hits | can be misread as "only this dex was scanned" | cause identified |
+
+Items 1-3 are the ones that changed a conclusion during this pass rather than merely being noticed.
+
+---
+
+## F4 — Documentation contradictions
+
+Independently found while mapping the skill and while using it. The executable cross-reference
+checkers (`check_refs.py`, `check_repo.py`) pass, so these are invisible to the repo's own gates.
+
+| # | Location | Problem |
+|---|---|---|
+| 1 | `dart-aot.md` §2 | Claims blutter's `asm/` has *no instructions* — "Do not plan around it." **Wrong.** One plugin file under `asm/<plugin>/src/ads/reward/` is 1,327 lines carrying class layouts, signatures, a full instruction listing, pool annotations and resolved call targets. It is the richest artifact in the chain and the doc says to ignore it. |
+| 2 | `dart-aot.md` §2 | Budgets "tens of minutes" for the first blutter build; measured **≈78 s** (~30× pessimistic). Also demands a very recent C++ compiler; **MSVC 19.34 (Nov 2022) suffices**. |
+| 3 | `dart-aot.md` §7 | Presents two-byte hit chaining as an acceptance test. On this sample, 9,840 two-byte CJK candidates yield 2,081 chained runs, all garbage. |
+| 4 | `signature-derived-keys.md:61` | Documents `python scripts/sig_probe.py --live --pkg <pkg>` — the script has no `--pkg`; it takes `--live PKG`. Line 54 and `pitfalls.md` P27 give the correct form. |
+| 5 | `dart-aot.md:204` | `dart_disasm.py ... --index callers.json --build-index` omits the required argument value; argparse errors out. |
+| 6 | `byte-level-patching.md:186` | Says different-length strings "need `dex_strpatch`-class tooling", while `dex_strpatch.py` requires `len(new) == len(old)` — it is precisely the tool that cannot do this. Contradicts `dex-patching.md:123`. |
+| 7 | Coverage vs `toolchain.md` | Coverage lists Unity/IL2CPP logic recovery as **not covered**, but `toolchain.md` Tier 4 names concrete tooling without pointing back at that disclaimer, so a reader of `toolchain.md` alone would take IL2CPP for a supported route. |
+| 8 | Script count | `SKILL.md`/`README.md` index 35 rows, which is 34 scripts plus the `scripts/dexpatch/` directory row. |
+| 9 | `dart_pool_strings.py` docstring | Says two-byte auto-detection is unreliable (confirmed here); `dart-aot.md` §7 treats UTF-16LE hits as a reliable rule. |
+| 10 | `verification.md` rung 7 | Re-downloading a distributed file to compare hashes has no landing place in `SKILL.md`'s six "done" criteria, so the two sections disagree on strength. |
+
+Item 1 is the most damaging because it steers a reader away from the best available evidence.
+
+---
+
+## F5 — "verified mechanisms" left no reproducible trace in the repo
+
+The Coverage preamble frames its list as *verified mechanisms*, and `references/long-task-discipline.md`
+defines **Observed** as *"reproduced it, with the exact command and output"*. The repository contains
+no sample, no fixture, no captured log or screenshot, no patch record, no test and no CI, and its
+`README.md` states it holds no target-specific data.
+
+So the strongest support the repository itself can exhibit for those claims is *Inferred*. This is
+**not** an accusation that the claims are false — several were confirmed here on the first attempt,
+and `dart-aot.md` §1 proved exactly right. It is a statement about evidentiary standing: the repo
+asks its readers to label their own claims and does not leave itself a way to meet its own bar.
+
+The concrete measurement figures that *are* reproducible are the 14 sample-scoped numbers listed in
+the skill map (e.g. dex 4.32 MB → 7.73 MB; `.RSA` 1199 B vs 777 B at runtime; a log tag going `8 → 0`).
+
+---
+
+## F6 — Boundary evidence: where the skill's method held, and where it ran out
+
+This is the observation the pass was built to produce, so it is stated as a sequence rather than a
+verdict.
+
+**Held, unchanged:** manifest and packer triage; multi-dex string/class recon at 27.6 MB across four
+dex files (`dex_strings.py`, `dexutil.py`); native ELF import mapping at scale (36,974 relocations);
+live-process library mapping; equal-length `.so` constant patching; the launch-timeline methodology
+(`coldstart.py`); the P26 discipline itself — checking whether a tool was ever able to answer before
+reading its silence as a finding, which is what turned the blutter crash from a false "unusable" into
+a corrected "works after a relink".
+
+**Ran out:** from the Dart snapshot inward. Everything past `pp.txt` is fine; getting *to* `pp.txt`
+from the binary requires a container resolver the kit does not have. That is one dependency, not a
+subject area.
+
+**Not exercised at all:** packers, code virtualization, custom linkers, integrity-check redirection,
+tamper-triggered suicide. This target has none of those, and the unmodified app's own startup failure
+also removes the repack loop those scenarios need.
+
+---
+
+## F7 — A trap in this specific environment that will produce false confirmations
+
+`androguard` 4.x (the version installed here) has **four independent reference-analysis failures on
+this sample**: `get_xref_from()` returns 0 for everything, including `AdLoader.loadAd()V` which has
+20+ callers by other means; `get_ref()` raises; a full `get_operands()` sweep returns 0 strings.
+
+Anyone who reaches for androguard to double-check a "zero references" conclusion will get **zero, and
+therefore agreement**. The zero-reference result in `TARGET-FACTS.md` does not come from androguard;
+it comes from `dexutil.py` plus raw-byte re-checks of the bodies that would not decode cleanly. This
+is worth a line in the environment documentation because it is a tool that lies in the direction of
+the hypothesis being tested.
+
+---
+
+## F8 — Reconciling the two failures: a genuine but unproven loader signal
+
+Recorded as a boundary between what was established and what was guessed, because collapsing them
+would be its own error.
+
+The launch does show non-standard startup plumbing (observed):
+
+- `E LoaderLog: 10026 / 101004` on the device, `10020 / 101003` on the emulator — an SDK-side
+  loader logging numeric status codes
+- `E Instrumentation: Uninitialized ActivityThread, likely app-created Instrumentation, disabling
+  AppComponentFactory` — the process is not taking the ordinary `ActivityThread` path
+- `E <PKG-truncated>: Unknown bits set in runtime_flags: 0x40000000` (logcat truncates the tag to 23
+  characters, so the package name appears clipped in this line)
+
+And yet the dex-side packer verdict is clean: no hardening library in the native set, an ordinary
+`application android:name` (the app's own class, not a packer stub), no known packer markers. Those two
+observations sit in tension.
+
+**Not resolved.** The `LoaderLog` emitter was not attributed to a specific class — the string exists
+in `classes2.dex` and `classes3.dex` but the classes that declare it are third-party, and this pass
+did not trace the call chain. Whether this plumbing is a benign logging SDK, part of the Flutter
+plugin bootstrap, or a light-weight loader contributing to the abort is **unverified**. Stating it
+as established would be exactly the kind of overreach this report is trying to document.
+
+---
+
+## F9 — Fixing F2 exposed a deeper one: the dex opcode table was shifted, and nine widths were wrong
+
+Fixing `find_refs.py`'s dex path means depending on `dexutil.decode()`'s boundaries being right, so
+that was measured rather than assumed. It was not right: on a real 8.9 MB dex, **65 of 50,791 method
+bodies did not decode to their declared boundary**, and **32 carried an operand index outside the
+table it indexes**.
+
+The cause is a run of `OP_NAMES` and `insn_units` shifted by one from `0x16` to `0x2C`:
+
+| opcode | specification says | the repository had | width: spec vs repo |
+|---|---|---|---|
+| `0x17` | `const-wide/32` (31i) | `const-wide/16` | **3 vs 2** |
+| `0x18` | `const-wide` (51l) | `const-wide/32` | **5 vs 3** |
+| `0x1B` | `const-string/jumbo` (31c) | `const-class` | **3 vs 2** |
+| `0x1C` | `const-class` (21c) | `monitor-enter` | **2 vs 1** |
+| `0x1E` | `monitor-exit` (11x) | `check-cast` | **1 vs 2** |
+| `0x20` | `instance-of` (22c) | `array-length` | **2 vs 1** |
+| `0x21` | `array-length` (12x) | `new-instance` | **1 vs 2** |
+| `0x23` | `new-array` (22c) | `filled-new-array` | **2 vs 3** |
+| `0x26` | `fill-array-data` (31t) | `throw` | **3 vs 1** |
+| `0x2C` | `sparse-switch` (31t) | *(absent)* | **3 vs 2** |
+
+Plus `0xFA`-`0xFF` (`invoke-polymorphic`, `invoke-custom`, `const-method-handle`,
+`const-method-type`), absent from the width logic and defaulting to one unit each where the
+specification has 4, 4, 3, 3, 2, 2.
+
+The `goto` family went with it: `branch_target()` computed a branch target for `0x27`, which is
+`throw`, and read the offset width one opcode early for the rest. `dex_find_insn.py`'s kind map
+carried the same shift in its `goto` and `switch` entries, so a semantic search for a branch matched
+the wrong instructions.
+
+That this survived an earlier "fix instruction widths" pass is not surprising in hindsight:
+`insn_units` ended with `if op == 0x29: # measured: this slot is goto/16 (20t)` — the shape of a
+repair made one observation at a time, where the width of `0x29` was corrected without checking the
+opcode's *name* against a specification. Name and width were wrong together and consistently, which
+is exactly why the decode still looked plausible.
+
+**How it was settled.** Not by reading the table again, but by two measurements that do not depend on
+this project's code:
+
+1. **Self-consistency at scale.** A real dex is a stream ART can execute, so a correct decoder must
+   land every method body exactly on `insns_off + insns_size*2`. 65 did not. After the fix,
+   **50,791 of 50,791 do, with 0 illegal operand indices.**
+2. **A second decoder.** `androguard` 4.1.4 decoded the same dex independently. Aligned by method
+   order, the per-instruction width sequences are **identical for all 50,791 bodies — 0 differing.**
+
+Names and widths now come from the format specification's table, and width is a lookup (`OP_UNITS`)
+rather than a chain of range tests, which is the shape that made this class of error possible.
+
+**Two things this pass did not fix, recorded so they are not mistaken for clean:**
+
+- **`dexutil.string()` decodes MUTF-8 with `errors="replace"`.** Identifiers containing non-BMP
+  characters decode to U+FFFD — this sample has them, and one crashed a `print` with a
+  `UnicodeEncodeError` on a non-BMP codepoint. A class name read from the dex may therefore not
+  compare equal to the same name read from another tool. It does not affect instruction decoding or a
+  descriptor-based needle, but it is why the cross-check above had to align by order rather than by
+  key.
+- `OP_NAMES` had no entry for roughly 150 valid opcodes (`aget`/`aput`, the `cmp` family, the
+  `neg`/`not`/conversion unops, the arithmetic and `lit8`/`lit16` families). They decoded correctly by
+  fallback but printed as `op_XX`. The regenerated table covers 224 opcodes.