vlad/reverse-skills-max
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

reverse-skills-max: initial public release (restored, redacted, hardened)

Browse Source
This commit is contained in:
baradok609hecl1pq6qi
2026-10-20 02:17:15 +03:00
commit da24c2bd20
2227 changed files with 443931 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

408
vendor-new/apk-reverse/skills/apk-reverse/SKILL.md Normal file
View File

@@ -0,0 +1,408 @@
---
name: apk-reverse
description: "Reverse engineer, debloat, de-ad, patch, or re-sign Android APKs, and analyze their runtime and server-side behaviour. Use for an .apk/.aab/.dex/.so sample, smali or dex patching, Frida hooking, repacking, ad or SDK removal, API probing, or deciding whether a client-side patch can work. Covers recon, anti-tamper, membership limits, surgical dex patching, repack pitfalls, device setup, and a failure catalogue."
license: MIT — see LICENSE at the repository root
compatibility: "Python 3.9+. Device work needs adb; dynamic analysis a matching frida-server; re-signing zipalign plus apksigner, not just a JVM. Static dex/ELF work and the leak scanner are offline. Run doctor.py --json for status."
metadata:
version: "1.0"
capability_registry: skills/apk-reverse/scripts/capabilities.py
evidence_summary: skills/apk-reverse/references/evidence-summary.md
last_reconstruction_pass: "2026-09-22"
strength_labels: "observed | inferred | unverified"
---
# APK Reverse Engineering & Patching
Goal: reach a **verified, installable, still-working artifact** fast — and avoid the whole class of
mistakes that destroy an APK while looking completely healthy.
## Immediate — do these four things before anything else
This file is a **procedure with gates**, not background reading, and it is long enough that its middle
gets skimmed. So the four moves come first; everything below is the explanation for them.
**1. Classify before choosing a route.** Answer the thirteen questions in §Start here. **R4** decides
whether you are editing the right layer at all, and a wrong branch does not fail loudly — it produces
an artifact that builds, runs, and does the wrong thing.
**2. Clear the four gates, in order, with their pass criteria** (§Gates). G1 names the deliverable form
in one testable sentence *before* any work; G2 establishes what this machine can really do; G3 locates
the code; G4 records the baseline and the control. "I understand the idea" is not clearing a gate.
**3. A symptom you cannot explain is a stop signal, not a puzzle.** Search §Symptom index for the shape
**before your next attempt** and load the file that row names. Those rows exist because each one cost
hours — and in several cases the answer was already written down while it was being re-derived.
**4. Two strikes on one shape of attempt sends you back to classification**, not to a third variant
(§Stop conditions). And **never report done without the six items in §What "done" means.**
**Two rules about this file itself.** When a failure does not fit your plan, the symptom index is the
next action, not more reasoning. And when this file and your own reasoning disagree, **this file wins**
until you have evidence that overrides it: every rule here is the residue of a failure that cost
hours, and your current intuition is the intuition of someone who has not hit it yet.
## Four rules that override everything else
**R1 — Write the deliverable as a testable sentence before you touch the target.**
"It works" is not the goal; "it works under the stated constraint" is. Root-assisted, live-
instrumentation, host-proxy and patched-device results frequently do **not** satisfy a request for an
installable artifact that works on a normal phone — and it is easy to present such a result as
finished. Write the sentence, re-read it at every checkpoint, and if you cannot meet it, say so
plainly and label the privileged workaround a **fallback**, never the deliverable.
→ `references/long-task-discipline.md` §the most expensive drift
**R2 — Change one variable at a time, and keep a control build.**
An experiment that flips two things teaches nothing when it fails, and a failure you cannot attribute
will be attributed to the wrong cause. Every "the app rejects X" claim needs its own run, and every
patch needs a same-pipeline control that still fails the old way.
→ `references/long-task-discipline.md` §single-variable discipline
**R3 — Never ship or claim an unverified artifact.**
"It assembles" is not "it works"; "the process started" is not "the feature works"; "no error in the
log" is not "the check is gone". Install it, launch it, exercise the exact feature you changed, and
look at the screen. Prove the device is running the build you made — hash it, do not trust the
filename.
→ `references/verification.md`
**R4 — Identify the owning layer before patching, and re-classify when reality disagrees.**
Ads, paywalls, feature gates, integrity checks and update gates live in different layers (Java, dex,
native, Dart/Unity, server). Patching the wrong layer either does nothing or breaks the app. If a
patch "had no effect", the diagnosis was wrong — go back to classification instead of patching harder.
→ `references/recon.md`, then the layer-specific file the symptom index points at
## Tooling — reach for the right instrument, and check before declaring it absent
Most wasted rounds in this domain are not bad reasoning about the target. They are **the right question
asked of a tool too weak to answer it**: an hour of `grep` over a hand-exported smali tree where one
indexed query would do, a manual ELF walk where a decompiler was one `pip install` away. The failure is
invisible from the inside, because the weak route still produces output.
**Five obligations. These are instructions; a violation is a defect, not a preference.**
| Obligation | Do this | Cost of ignoring it |
|---|---|---|
| **Orient with an indexer, not an export** | Build the ability to *ask the artifact questions* before reading code: `droidasc findrefs` / `ddc findrefs` (string/type/method → every reference site, sub-second). A decompile reads a class you have **already located** — it is not how you locate it. `jadx` is a readable viewer of last resort, never the source of truth and never the entry point of a recon | hours of `grep` per question asked, over an export you paid for first |
| **Install the missing tool; it is part of the task** | A missing arm64 decompiler is the next step, not a constraint to route around. Ask a human only when installation is genuinely impossible | hours re-derived by hand for output a decompiler gives in minutes |
| **Name the gap before spending against it** | State the capability the blocker requires and whether this machine has it, **out loud**. This is a G2 item | designing around a tool that installs in ten minutes |
| **Reach for a shipped script before writing one** | Parsing, hashing, alignment and hot-plug probes are already written; a bespoke script in place of `scripts/dex_find_insn.py` is how offsets get **guessed instead of computed** | a wrong offset that decodes cleanly and behaves wrongly |
| **Pick the instrument for the layer the question lives on** | A dex in memory wants `dex_mem_scan.py` (find and cut) **plus** `dex_dump_validate.py` (judge), not a decompiler pointed at a fragment. An algorithm you only have to *call* wants `emulation-and-rpc.md`. "Does this class ever load at runtime" wants a hook that fires or does not | the heavier tool is not the safer one — it produces a *plausible* answer, which is worse than none |
**Run `python skills/apk-reverse/scripts/doctor.py --json` before concluding anything is unavailable.**
It reports per-capability closure — what is missing, what to install, and roughly how long that takes —
and a capability you have not checked for is not a capability you lack. **It is deliberately allowed to
report `BLOCKED`:** a machine with a JVM but no `zipalign`/`apksigner` cannot re-sign, and the report
says so rather than inferring capability from a tool that is merely present. Detail:
`references/toolchain.md`; the registry it reads is `scripts/capabilities.py`.
## Coverage — what this skill claims, and what it does not
The failure this section prevents is not ignorance. It is **a confident wrong answer produced by
applying the nearest available procedure to a target it was never written for.** A documented method
that almost fits is more dangerous than no method at all, because it arrives with a plan, a
vocabulary and a set of reassuring numbers.
**How strong these claims are — this is load-bearing, not cosmetic.** Every claim in this skill carries
one of three labels, and you must label your own results the same way:
| Label | Means | May it justify a decision? |
|---|---|---|
| `observed` | a command was run and its output exists | yes |
| `inferred` | follows from an observation, but the step is reasoned | provisionally |
| `unverified` | assumed, or reported elsewhere and never reproduced here | only as something to test |
**The inventory that replaces a guess:** the per-route list of what is covered and what is not — by
verified mechanism versus by documented-but-mostly-inferred route — is in
`references/coverage-and-limits.md`. Load it when the question is "can this skill actually do X".
**Partial coverage that is easy to over-read** — these are documented routes whose *dependency* this
skill does not ship, and each has a fence around it in the same file: Dart AOT analysis needs a
snapshot dump you must obtain elsewhere and naming the front end matters · extraction-shell recovery
stops at the VMP boundary · a Stalker trace is not guaranteed on every device · a kernel-side answer
requires a module this skill only scaffolds.
**Not covered — say so rather than improvise:** Unity / IL2CPP · React Native / Hermes / Cordova ·
iOS · defeating a server-side authority · an off-the-shelf unpacker or an anti-detection arms race ·
building and shipping a kernel module. Reasons and evidence boundaries are in
`references/coverage-and-limits.md`; **silence in that record is not support.**
**The fallback, as an instruction — this is a rule, not advice.** If the target does not match the
covered list, or no symptom-index row matches, then **stop and classify before choosing a branch**:
answer the thirteen questions (§Start here). If the shape still does not fit — an unknown runtime, a
mechanism you cannot name — **say exactly that and propose the cheapest experiment that would
identify it.** Do not take the closest documented route and apply it anyway. A wrong branch here does
not fail loudly: it produces an artifact that builds, runs, and does the wrong thing.
## Hand-off points — where this skill ends and another view begins
Four boundaries that are easy to walk into without noticing. Each names what the other side owns
rather than restating it, because two copies of the same advice drift apart. The JNI form table and
the per-form verification table live in `references/handoff-boundaries.md`.
1. **JNI** — a Java `native` declaration and its implementation are two views of one function, and
this skill reads each with a different tool. A symbol search fails **silently** on dynamic
registration. Authoritative treatment: `java2c-and-jni-sinking.md` §The JNI boundary — why a
symbol search fails silently.
2. **Hardening** — a dex-side packer observation is a native-side implementation question.
`packers.md` owns the dex-side identification; the deep dive belongs on the other side.
3. **Native anomalies** — read them *from the APK side*, because the judgement is whether your patch
caused the death. Restoring a symbol or reversing an OLLVM function is a different activity with a
different toolchain: point across instead of extending `native-and-so.md` into it.
4. **Deliverable form** — when the artifact stops being an APK, the verification question changes with
it. G1's other three forms each move the evidence somewhere else, and the failure is quiet: a
privileged result reported in the language of a finished build.
## Symptom index — a matching row is a stop signal
You arrive at a symptom, not at a file name. Each row below is a failure that has already been paid
for. **If any row matches what you are observing, load the file before your next command** — not after
your next three attempts. Reasoning from first principles at this point is how the same hours get
spent twice; more than one entry here is a lesson that was re-derived by hand while the answer sat
unread in this repository.
| What you observe | Load first |
|---|---|
| A repackaged/re-signed build **dies before your code runs**; `SIGSEGV`, all registers zero, `pc=0`, `fault addr` near `0x0` | `native-tamper-and-suicide.md` (deliberate crash), then `code-virtualization-and-custom-linkers.md` |
| **No packer** (Application is the app's own, dex readable) **and it still dies** | `code-virtualization-and-custom-linkers.md` §a loader is still a possibility; but if the same build also dies on a *second, unrelated* device you are looking at an ordinary startup fault, not a hardened one |
| The app dies at startup on **every** device, packed or not, with **no tombstone** while `crash_dump` says `already traced` and logcat says `exited cleanly (0)` | a bundled crash reporter has taken the signal handlers, so the platform's own trail is gone. Frida spawn-gating is the recovery route |
| A `FORTIFY: pthread_mutex_lock called on a destroyed mutex` abort in a Flutter app, on the **main** thread, before the first frame completes | `dart-aot.md` — check `libapp.so` is actually being loaded; Flutter's engine bootstrap is the usual place a native lifecycle fault surfaces |
| Log says a **Java-layer** signature/integrity check **passed**, yet the process dies | `code-virtualization-and-custom-linkers.md` §a Java-layer "signature killer" is a decoy |
| Deleting a library fixes validation but yields `UnsatisfiedLinkError: dlopen failed: library "X" not found` | `code-virtualization-and-custom-linkers.md` §the deadlock that eats hours |
| Whole classes appear as bare `native` declarations with no body | `java2c-and-jni-sinking.md` — read it **before** dumping memory: if this is Java2C there is no DEX to find, at any point in the process lifetime. A handful of `native` methods in an otherwise ordinary dex is JNI sinking, not this |
| A `Java_*` search over a hardened library returns nothing at all | `java2c-and-jni-sinking.md` §The JNI boundary — why a symbol search fails silently — dynamic registration, or `-fvisibility=hidden`. The check that works is "exports `JNI_OnLoad` and zero `Java_*`" |
| You are about to publish an evidence file, a transcript or a README that quotes real work | `references/desensitization-and-leak-scans.md` — run `scripts/scan_leaks.py` **before** it is committed; the hit list is a set of lines to look at, and `--show-exempt` is where the wrong suppressions show |
| A hooking module appears to have run but its log tag is silent, and you are about to record "it never loaded" | `references/precedents/logd-broken-module-never-ran-case-3.md` — a broken `logd` delivers nothing on `logcat` while the module's whole run sits in LSPosed's file log; read both channels |
| A library's **SONAME does not match its filename** | `code-virtualization-and-custom-linkers.md`, `native-and-so.md` |
| Your edit had **no effect at all**, with no error | `server-config-and-updates.md` §3 (the value may be server-sent), then `packers.md` §map the validation boundary |
| Process **hangs** with no crash record, or dies to a `uid 0` killer | `native-tamper-and-suicide.md` §the rule (you probably made a terminate path *not return*) |
| Death looks like an ordinary null dereference in a hardened library | `native-tamper-and-suicide.md` §deliberate-crash stubs |
| The app dies **only while you are attached/rooted** | `detection-and-anti-analysis.md`; run the unmodified original under identical conditions first |
| **Install fails with `[-124]` and mentions `resources.arsc` / alignment** | `repack-and-sign.md` §2a — STORED **and** 4-byte aligned, both required |
| **Install fails with a bare numeric code (e.g. `[-99]`) and no `INSTALL_FAILED_*`** | `repack-and-sign.md` §vendor install interception — a device-side interceptor, not your build. Use the root `pm install` path |
| **After an install, `am start` does nothing / screenshots show another app / `am start -W` hangs** | `repack-and-sign.md` §the installer may still own the screen |
| Log shows `Failure to verify dex file ...: Bad checksum` and a startup `ClassNotFoundException` for an ordinary class | `byte-level-patching.md` §the dex header has two integrity fields — order matters |
| An install "succeeded" but nothing changed, or the version did not move | `long-task-discipline.md` §keep the observation window clean |
| Evidence contradicts itself, or a capture looks like two states mixed | `long-task-discipline.md` §keep the observation window clean |
| You took screenshots but drew the conclusion from logs or from the patch itself | `long-task-discipline.md` §captures you never looked at are not evidence |
| You are about to re-run an experiment whose result you already recorded | `long-task-discipline.md` §long-context decay |
| A script will not start, or a tool "is missing" | `scripts/doctor.py`, then `toolchain.md` §"not on PATH" is not "not installed" |
| A hook or probe reports **no events at all**, and you are about to call it detection | `scripts/anti_detect_probe.js` for the environment self-report first, then `detection-and-anti-analysis.md` §Step 3: locating the check — the order of search from Stage 0 |
| `attach` hangs and then fails **while the process is still in `ps`** | `detection-and-anti-analysis.md` §Step 3 Stage 0 — check for `D` in `/proc/<pid>/stat`, and attach a *different* pid as a one-line control before blaming the target |
| The app exits with no tombstone, no crash and no ANR record | `detection-and-anti-analysis.md` §Step 3 — a clean self-exit means the check ran before your hooks existed; the branch conditions there say which Stage |
| A dump region validates as the wrong thing, or an `r--s` view of `base.apk` looks like a dex | `advanced-unpacking.md` §What this route cannot do, and how to tell before you spend the window |
| Feature-scoped network failure (login/register/pay) while the rest works | `tls-and-cert.md` — do not assume your patch caused it |
| Everything works but **every signed request fails** after repack | `signature-derived-keys.md` |
| A re-signed build **runs fine, renders its whole UI and logs no error — but one feature silently never loads**, and `dumpsys`/DNS/logcat show **no request for it at all** (not a rejected request: *no request*) | `code-virtualization-and-custom-linkers.md` §what the native check actually reads — refusing **before** the request is built. Not the row above: "sent and rejected" and "never sent" have different owners |
| You cannot tell whether a missing feature is **your patch's fault or the target's own behaviour** | `long-task-discipline.md` §single-variable discipline. Run the **zero-change control through the same pipeline**, and the decisive variant: the unmodified original with the patch applied **in memory only**, same device, same network |
| Under Frida `spawn`, the UI never appears — `mCurrentFocus` stays `null`, screenshots come back blank, the Activity stack never builds | `dynamic-frida.md` §spawn keeps the Activity stack down: write the patch into memory, **detach**, then start the Activity normally |
| `frida-server` keeps disappearing mid-experiment, or the device reboots itself while you are working | `dynamic-frida.md` §when the ROM hunts your instrumentation |
| Ads still appear after a patch that should have killed them | `server-config-and-updates.md` §6 (cached config / remote re-enable), then `ad-removal.md` §step 4 (count the SDK's own log lines; n -> 0, not "I did not see it") |
| A forced-update or "must update" gate blocks the build | `updates-and-forced-upgrade.md` §step 6 |
| The dialog is gone but the feature is still locked | `membership-and-limits.md` / `account-gates.md` — decide server vs client authority before patching again |
| You are about to discard a route as "blocked" | `packers.md` — re-read it before writing any route off; mis-attributed failures have removed viable routes for hours |
| The task has run long and you are unsure what is already proven | `long-task-discipline.md` §keep a live record |
| A dumped dex parses in full, the classes are all there, and most method bodies are `return-void` stubs or nop fills | `references/advanced-unpacking.md` — an extraction shell: measure the `stub%` with `scripts/dex_dump_validate.py` before trusting any of it, and know that recovering the bodies is a different route |
| Your `frida` dump dies mid-write (`script has been destroyed`), or the process you are dumping keeps changing pid | `advanced-unpacking.md` §dumping when frida is refused — rule out memory pressure first; a reclaim-and-relaunch needs no instrumentation |
| A repack is refused by several independent checks, or the build has to keep working through store updates | `references/lsposed-and-modules.md` — deliver a module instead of an APK; G1's form table says when |
| A hook module is installed, enabled and scoped, yet its log tag never appears — and you are about to conclude it never ran | `lsposed-and-modules.md` §Deploy, enable, and verify — **a `logcat`-only verdict has already been wrong here**: on one ROM `logd` is broken and output reaches only `/data/adb/lspd/log/modules_<ts>.log` |
| A module's entry class is missing from its own dex (so it can never load), yet the package installs, enables and looks healthy | `references/lsposed-and-modules.md` — check `assets/xposed_init` against the dex's actual classes; installation is not evidence of anything |
| You only need to **call** the target's own routine (sign, token, encrypt) rather than change the app | `references/emulation-and-rpc.md` — emulate it, or service-ify the live function over Frida RPC |
| A native function is a many-thousand-line `switch` state machine, or the decompiler's output is meaningless | `references/native-dbi-and-deobfuscation.md` — OLLVM shapes, a Stalker trace, and how far a trace actually gets you |
| `Stalker.follow` installs but no events arrive, or following a hot libc export crashes the process | `references/native-dbi-and-deobfuscation.md` §6 failure modes — this repository measured both |
| The traffic is protobuf/gRPC/QUIC, or a proxy sees TLS but requests still fail on a Flutter app | `references/protocol-reverse.md` — schema-less protobuf, frame capture, and native-side pinning |
| Userspace hooks land and the app still dies: the check reads `/proc/self/status` through a raw `svc`, or runs before `JNI_OnLoad` | `references/kernel-and-environment-hardening.md` — what the next layer up and down can actually do, and when to stop |
| You must edit, repack, sign or inspect the APK **from the phone itself** | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
| A captured body decodes to nothing readable, or you cannot tell whether a length-delimited field is a string, a nested message or a packed array | `protocol-reverse.md`. Protobuf on the wire (measured) — run `scripts/protobuf_decode_raw.py`; the candidate list and its `tie:` lines are the answer |
| Method bodies are present but decode as **private opcodes**, and you need the mapping rather than an explanation of why VMP is hard | `vmp-differential-analysis.md`, then `advanced-unpacking.md` for the shape diagnosis |
| A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK |
## Gates — clear these before you patch, in order
Each gate is an **action with a pass criterion**. Do not proceed past a gate you have not cleared, and
do not treat "I understand the idea" as clearing it. Skipping a gate is not a shortcut; it is how the
work gets redone.
**G1 · Deliverable form — and the cost ceiling on it.** State, in one sentence you could hand to
someone else, what artifact must exist at the end and under what constraints (rooted or not,
installable on a stock device or not, must survive updates or not, online or offline). *Pass:* the
sentence names a testable constraint, not an activity. *Fail:* you are solving a problem in an
environment the deliverable will never see.
Then name the **form** that sentence implies. "A rebuilt, self-contained APK" is only one of four, and
this is where the choice belongs — not after the repack has already failed:
| Form | Right answer when | What it costs you |
|---|---|---|
| **Rebuilt, installable APK** | The client owns the behaviour; no multi-point integrity check; no extraction shell | Repack, re-sign, and a device to verify on |
| **LSPosed / Xposed module** | The logic is client-side but the app fights repacks (multi-point signature checks, shell self-verification), or the result only has to work on rooted devices you control | A rooted device, module scaffolding, and an app that must not detect the hooking framework → `references/lsposed-and-modules.md` |
| **Local RPC / emulation service** | You do not need to change the app — you need to **call** it: a signing routine, a token, an encryption function | A live device or an emulated loader plus a call harness → `references/emulation-and-rpc.md` |
| **Analysis report with a stated boundary** | The authority is server-side, or the target is a real VMP / extraction shell whose recovery cost exceeds the value of the task | Nothing ships — and that is the honest answer, not a failure → `references/advanced-unpacking.md`, `references/server-api.md` |
*Decision trigger for leaving the first column:* switch off "rebuilt APK" as soon as the evidence shows
**(a)** more than one independent integrity check that must all pass, **(b)** an extraction shell whose
method bodies exist only at invocation time, or **(c)** any body that decodes as private opcodes. At
that point the repack route is not merely expensive — it is blocked, and the deliverable sentence
should say which form replaced it. **A form chosen here and re-read at every checkpoint is the guard
against the most expensive drift in this skill** (`references/long-task-discipline.md`).
**G2 · Environment truth and capability inventory.** Run `scripts/doctor.py` (and `scripts/preflight.py`
if a device is in play). *Pass:* you know which toolchains and scripts can actually run here, you have
seen the environment warnings — clock skew, leftover `adb forward`/proxy, a device-side frida process
already running, a tool installed off-PATH — **and you have written down the capability this target will
demand against the capability this machine has.** Name the two or three layers the task will almost
certainly reach (for example "arm64 native decompilation", "Dart AOT snapshot dumping", "device-side
TLS inspection", "dex-wide cross-referencing") and mark each available / missing-but-installable /
genuinely out of reach. *Fail:* you are about to attribute to the target a failure caused by your own
setup — or to spend a day routing around a tool that installs in ten minutes. A layer whose tool is
missing is a **task item**, not a constraint to design around. → `references/toolchain.md` §Closing a
capability gap
**G3 · Code location.** From the manifest and dex, answer: is there a packer, where does the app's own
code live (dex / native / Dart / Unity / server), and is any of it virtualized to native. *Pass:* you
can name the class that owns the behaviour you intend to change, or you have an explicit plan to find
it. *Fail:* you are about to patch a layer you have not located. If recon says "no packer", still
check the virtualization shape — see the index rows above.
**G4 · Baseline and control.** *Pass:* you have a control run — the unmodified original, or a
zero-change repack through the same pipeline — and you have recorded the observed failure (including
**time-to-death**, if it dies). *Fail:* when the patched build misbehaves you will have nothing to
compare against, and every later measurement is unfalsifiable.
## Start here: classify the target in thirteen questions
**Answer these before touching a tool — all thirteen, in order. Each one changes the whole plan, and
a wrong answer here does not fail loudly: it produces an artifact that builds, runs, and does the
wrong thing.** The answer column names both the action and the file that owns the detail; load that
file before acting on the question.
| # | Question | What the answer changes | Load |
|---|---|---|---|
| 1 | **Is the app packed/hardened?** Read the manifest's `application android:name` | A third-party shell class rather than the app's own Application means you have a packer and must handle it **first** | `recon.md` |
| 2 | **Where does the behaviour actually live?** — ad SDK · **server-issued config for client-rendered UI** · membership/VIP · feature flag or debug switch · anything decided by an API response | Client-side and removable, versus server-supplied data the client decides with, versus server-authoritative where a client patch is cosmetic. **Decide this early**: hunting an ad SDK that does not exist costs hours, and the server-config shape has no SDK to find | `ad-removal.md`, `server-config-and-updates.md`, `membership-and-limits.md`, `server-api.md` |
| 3 | **Is the app's own code in plain dex, or moved to native / Flutter / Unity?** | Which toolchain the whole task uses. **Cheap runtime check before committing:** hook the obvious Java classes for the UI you care about and reproduce that UI. Hooks that fire mean Java owns it; hooks that fire **zero times** while the UI is plainly on screen mean the runtime or native code draws it, and a dex-only plan will stall. Do not keep hunting in dex after a zero-hit probe — the most expensive wrong turn in this skill's history | `recon.md` §Where does the app's own code live, `framework-runtimes.md` |
| 4 | **What must the deliverable be able to do?** Write it as a testable sentence and re-read it at every checkpoint | The most expensive drift in this skill: a runtime-only result (data edit, live hook, blocked hostname, host proxy) looks like success while failing the requirement. The axes: **privilege** (unrooted?), **modification form** (rebuilt artifact vs live instrumentation), **ABI/device class**, **network**, **persistence**, **distribution** (self-contained?). If the evidence already shows a deep extraction shell, a VMP, or more than one independent integrity check, **re-answer against G1's four forms** — the repack column may be blocked, not expensive | `long-task-discipline.md` §the most expensive drift |
| 5 | **Does the app verify its own signature, or does the server?** | App-side means you must bypass it; server-side means re-signing silently breaks the app later | `repack-and-sign.md`, `server-api.md` |
| 6 | **What is your device situation?** — rooted real device, rooted emulator, or no device | Whether Frida is usable at all, and whether the deliverable can be tested where it will run. **Run `scripts/preflight.py` before your first experiment** and again whenever a failure surprises you — device state, a dead device server, a leftover proxy and clock drift all masquerade as a broken patch | `environment.md`, `pitfalls.md` P9 |
| 7 | **Which architecture is actually executing?** | `getprop` reports what the device claims and `primaryCpuAbi` what the package manager chose — **neither is what runs**. Only the live mapping is ground truth. An unmapped library, a translator in play, or a different ABI than assumed changes the plan more than any patch will | `native-and-so.md` §Cross-architecture, `scripts/lib_map.py` |
| 8 | **Is one *specific* feature failing at runtime** (login, registration, payment, an API-backed screen) while the rest works? | A feature-scoped network failure is very often a **TLS/certificate problem on one code path**, not your patch. An app can carry two independent trust chains, so "other requests work" proves nothing. Rule this out in minutes before hunting a signature check | `tls-and-cert.md` |
| 9 | **Was the input a build you did not produce** (a circulating "cracked"/"modded" APK)? | Such builds are frequently re-protected — sometimes with *more* layers than the original — and may carry injected components or endpoints. **Never use one as a patching workbench** | `third-party-builds.md` |
| 10 | **Does the client sign its requests with its own signing certificate?** | Grep for `toCharsString()` / `signatures[0]` / `getPackageInfo(..., 64)` **before the first repack**. If that value feeds a native HMAC/DES routine, the rebuilt APK must hardcode the *original* certificate value at every read site, or every signed request fails while the app still launches and looks healthy. The single most expensive silent failure in a repack, and 15 minutes of grep prevents it | `signature-derived-keys.md` |
| 11 | **Does the app die on its own after a while** — no Java stack, or a native crash that looks like a bug? | A hardened library rarely calls `kill`; it more often **arranges a fault** (load a small constant, use it as a pointer) so the death looks like an ordinary defect. Two rules before touching anything: **enumerate which mechanism actually fires** (signal and tombstone split them apart), and **neutralise by returning, never by making it not return** — a spinning stub freezes the process and produces a symptom that looks nothing like the cause | `native-tamper-and-suicide.md` |
| 12 | **Will this build still be usable in a week?** | Any version check, upgrade prompt or self-update path means an unpatched build can be switched off or replaced remotely. This is one or two edits and it decides whether the work is durable — part of the build, not a follow-up. Also check for a **hot-update / remote-config** channel, which restores removed behaviour with no version change at all | `updates-and-forced-upgrade.md` |
| 13 | **Does the request touch sign-in or phone binding** ("no login required", "skip binding", "guest ok")? | Separating a **client-side gate** (patchable) from an **account-scoped resource** (the screen is empty because the server has no account to answer for — not patchable). Classify first; and **never fabricate a session** to satisfy a gate, which produces a state worse than being signed out | `account-gates.md` |
**Long-task rule (a condition, not advice):** if this is likely to run long, open
`references/long-task-discipline.md` **now** and keep its record updated as you go. Re-read its
refuted-conclusions and dead-routes sections before starting any experiment. Losing earlier findings
is the most expensive failure in this skill, and it is entirely preventable.
## The workflow, end to end
Steps are ordered. **Skip a step only when its stated skip condition is met** — "it seems
unnecessary" is not a condition, and it is the reason most of the failures in `pitfalls.md` happened.
**Two-strike rule.** If the *same kind* of attempt fails twice, stop and go back to classification.
Do not run a third variation of a hypothesis that has already failed twice. Two failures of one shape
means the model is wrong, not that the parameters need tuning — and the third attempt is where an
entire round gets spent confirming what the first two already said. Re-read the symptom index at that
point; it exists for exactly this moment.
1. **Preflight, then Recon** — `scripts/doctor.py` is the cheapest possible first command: it reports which toolchains and scripts can actually run here, and surfaces the environment facts that poison experiments (clock skew, leftover `adb forward`/proxy, a device-side frida process already running, a tool installed off-PATH). Then `scripts/preflight.py` before anything else if a device is involved (it takes seconds and prevents a whole class of false conclusions), then `references/recon.md`. Manifest, package name, version, ABI, dex count, packer, embedded SDKs, where the app's own code lives. Ten minutes here saves hours. **If it is packed, unpack before anything else** (`references/recon.md` §unpacking): you cannot patch code you cannot read, the encrypted payload lengths tell you which dumped dex is the original, and a memory dump must be de-duplicated by hash and structurally validated before any of it is trusted.
**If recon says there is no packer but a re-signed build still dies**, you are in the layer `references/code-virtualization-and-custom-linkers.md` covers — do not proceed on the assumption that "no packer" means "editable".
**If the app already dies on its own** — especially at a roughly constant time after launch, or with a native crash — locate the mechanism *before* planning any patch (`references/native-tamper-and-suicide.md`, `scripts/native_crash.py`). Record the observed time-to-death: it is the baseline every later attempt is measured against, and without it a surviving run cannot be told from a changed schedule.
*Skip condition:* never skipped. G2/G3 in §Gates are cleared here or not at all.
2. **Extract strings and endpoints** — build a picture of the app's API surface and SDK inventory from the dex string tables. No decompiler needed for this, and it is fast. Scripts: `scripts/dex_strings.py`.
3. **Trace to the owning class** — find the class that wraps the behavior (the app almost always wraps third-party SDKs in one helper). Reverse-lookup instructions: `references/dex-patching.md` §finding-the-call-site.
4. **Decide the patch layer** — client SDK call / client rendering / client data consumption / server contract. See the table in `references/ad-removal.md`.
5. **Patch surgically** — `references/dex-patching.md` and `references/byte-level-patching.md`.
Two techniques, and picking the right one is a decision, not a preference:
**equal-length byte edits** (`scripts/dex_patch_bytes.py`, located with
`scripts/dex_find_insn.py`) when the change fits in an existing instruction slot
or constant — nothing moves, so no offset, try/catch block or debug pointer can
be invalidated. **dexlib2 method rewriting** (`scripts/dexpatch/`) only when the
change genuinely needs new instructions. Whole-tree smali round-trip damages
R8-optimized dex in ways that only show up at runtime; a method rebuild also
inflates the file (measured: `debug_info` 924 B -> 22.8 KB, dex 4.32 MB ->
7.73 MB on one sample). Whichever you use, recompute the dex header integrity
fields (**signature first, checksum last**) — `references/byte-level-patching.md`
§the dex header has two integrity fields.
6. **Repack and sign** — `references/repack-and-sign.md`. **Do not strip the whole `META-INF/`.** This single mistake destroys otherwise-correct builds.
6b. **Neutralise the update path — before you call the build done.** If the app checks for updates at all, add the two-layer patch (`references/updates-and-forced-upgrade.md`): no-op the update routine's entry, and force the version comparison to its "no update" side. A build that can be switched off or replaced remotely is not a deliverable, and this costs minutes here versus a rebuild later. Do the same for any **remote-config or hot-update** channel that could restore the behaviour you removed.
6c. **Handle account gates only after classifying them** — if the request mentions sign-in or binding, apply `references/account-gates.md` and state plainly which guarded screens become usable and which stay empty because their content is account-scoped.
7. **Verify on device** — `references/environment.md` + `references/verification.md`. Check: launches, the changed behavior actually changed, nothing unrelated broke, and **the app reaches its normal UI with no blocking dialog**. First prove the artifact actually changed on the device -- a package manager reporting success does not prove an interposed confirmation was accepted (P18). Capture continuously for the first ~20 seconds after launch, **and look at the captures** — sampling gaps are how a blocking modal goes unseen (P20), and a burst of images that were never inspected is not evidence. If the accessibility tree is empty, the image is the primary evidence rather than a fallback.
8. **Log what you learned** — if a failure cost you more than thirty minutes, add it to `references/pitfalls.md`. That file is the most valuable artifact in this skill.
## What "done" means — do not claim it earlier
Every item below must be true before you report completion. Anything less is a **checkpoint** and must
be labelled as one, out loud, with what remains. Premature "done" is the most damaging thing you can
report, because it ends the investigation while the user believes the problem is solved.
1. **The artifact exists and its identity is recorded** — path plus hash, not a filename.
2. **It was installed and launched on the environment the deliverable sentence names** (G1/R1). If
that environment was not available to you, say so and label the result accordingly.
3. **The behaviour you changed is verified changed** — by direct observation of the feature, not by
the absence of an error message. "The log is clean" is not evidence; "the screen shows X" is.
4. **The features it touches still work.** You exercised them. A build that starts but whose affected
feature is dead is not a result.
5. **The original limitation is stated if any survives** — with the coupling that causes it, so the
next person can decide whether to accept it.
6. **Nothing you did leaves the target or the device in a broken state** unless that was the goal, and
any privileged workaround is labelled a fallback rather than the deliverable.
If items 1–4 hold but the environment was wrong, you have a **prototype**, not a deliverable. Say
"prototype" and name the gap.
## Stop conditions — halt and re-classify, do not retry
These are moments where continuing to push forward is the wrong move. Each has cost hours somewhere.
- **The same shape of attempt failed twice.** See the two-strike rule above.
- **A patch had no effect and you were about to try a third variant of it.** No effect means the
diagnosis was wrong, not that the patch was unlucky. Re-classify the layer.
- **A new failure has no place in your current model.** That is the symptom index's trigger condition.
- **You are about to write off a route as "blocked"** without a control build proving the block is
the app's doing rather than your pipeline's. Mis-attributed blocks have removed viable routes.
- **You are about to claim success on absence of errors.** See §What "done" means.
- **A measurement disagrees with a conclusion you already recorded as settled.** Re-open the
conclusion; do not explain the measurement away.
## Non-negotiable constraints
**These are defects when violated, not preferences.** Each row states the check that catches it, so
"did I comply" is a command you can run rather than a judgement you make about yourself.
| Constraint | Why it is absolute | The check |
|---|---|---|
| **Inputs are read-only.** Work on copies; keep a known-good baseline | an edited original destroys the only reference you can diff against | `git`/hash the original before the first edit; `scripts/apk_diff.py` for entry-level proof |
| **One variable at a time**, with a control build (same pipeline, **zero** patches) | a compound change that fails teaches nothing, and the failure will be attributed to the wrong cause | a control run exists and its result is recorded |
| **Verify structure after every dex edit** | a patch that assembles and dies at load looks like a patch that worked | `scripts/dex_classdiff.py`: zero differences in class set and access flags for classes you did not intend to change |
| **Never patch a widely shared method.** Count callers first | a `Long.valueOf`-shaped helper with 30 callers is not an ad-specific hook | `scripts/find_refs.py` on the method you are about to touch |
| **Never make an API fail to suppress a UI element** | a 404/400 an endpoint's other features depend on takes the whole screen with it — measured as a build that never leaves the launch screen | suppress at the data-consumption or render layer instead; if a request's path changes, that is the bug |
| **Neutralise a native terminate path by returning, never by making it not return** | a spin stub freezes the caller **holding its lock**; unrelated threads wedge and the symptom (hang, external kill, restart loop) looks nothing like the cause, with no crash record | return a benign value, prefer success (`0`) over failure (`-1`), never touch `pthread_exit`/`exit`/`abort`/`snprintf`/`closedir`. `native-tamper-and-suicide.md` |
| **Look before you conclude, and look while you wait** | a screen that is actually looked at answers in one step what coordinate-guessing cannot answer in five | capture **and inspect**; byte-identical samples mean nothing will change. `scripts/snap.py`, `environment.md` §look at the screen |
| **Bound every command, and calibrate the bound from a measurement** | an unbounded call turns a stall into "still working", which is indistinguishable from progress | time the operation once, record it, derive the deadline from it. `long-task-discipline.md` §Bound every wait |
| **Every claim carries a label — `observed`, `inferred` or `unverified`** | "probably", "should be" and "in theory" are not findings, and an unlabelled guess propagates as a fact | `observed` requires a command **and its output**; nothing else may be written as established |
| **"Done" means the user-visible outcome** | a blocking dialog still on screen means the task is not done, however clean the log is | §What "done" means, all six items; absence of a log line is never evidence of success |
| **Never discard a route on compound evidence** | if a failure followed two simultaneous changes, the attribution is a hypothesis — mis-attributed failures have removed viable routes for hours | re-run the abandonment single-variable before writing it down |
| **Prove the device changed before measuring anything** | install success describes the *request*, not the app on disk | hash the on-device artifact against your build; otherwise every later observation describes the previous build |
| **Attribute a failure to the right layer before patching again** | a feature-scoped network failure is frequently the app's own TLS/certificate problem, and chasing a signature check that does not exist burns hours | run the **unmodified original** on the same device and network first (`tls-and-cert.md`) |
## Reference index and script index
**Both tables live in `references/routing.md`** — one load gets you every reference file with when to
load it, and every script with what it does. They are deliberately not duplicated here: this file is
loaded in full every time the skill activates, and those two tables are about 140 rows of lookup data
that nobody needs until they have already decided what to do.
The symptom index above stays, because **symptom to file has to be one hop**: when something fails you
are not choosing a file, you are recognising a failure, and a two-hop lookup at that moment is exactly
how a stop signal gets skipped.
`references/evidence-summary.md` is the one reference that answers the claim-strength question from
inside an installed copy: capability → one-line conclusion → `observed`/`inferred`/`unverified` → the
evidence that ships with the skill. Load it when a claim's strength decides whether you trust it and
the run record is not in front of you.
`python check_routing.py` checks that this file and `references/routing.md` still agree, that every
reference file is named there, and that every script is named there. CI runs it.

56
vendor-new/apk-reverse/skills/apk-reverse/evals/evals.json Normal file
View File

@@ -0,0 +1,56 @@
{
"skill_name": "apk-reverse",
"_status": "unverified -- skeleton only",
"_warning": "NOTHING IN THIS FILE HAS EVER BEEN EXECUTED. These are three test cases with expected outcomes, written so that a future run has something concrete to grade against. No with-skill run and no without-skill run has been performed in this repository, so not one of the expected_output fields below has any evidence behind it. Do not cite this file as a quality measurement.",
"_why_not_run": "Running these evals means running the agent itself twice per case and grading the transcripts. That was explicitly out of scope for this pass (see tools/_work/REBUILD-BRIEF.md section 2, the do-not-do list: no real agent eval scoring). The executable side of this repository's testing lives in tests/ and in .github/workflows/ci.yml, neither of which can observe agent behaviour.",
"_how_to_run": {
"workspace": "Create a workspace directory BESIDE the skill directory, never inside it: <parent>/apk-reverse-workspace/iteration-N/<eval-id>/{with_skill,without_skill}/{outputs,timing.json,grading.json}, plus benchmark.json at the iteration root.",
"isolation": "Each run must start from a clean context -- a fresh subagent or a fresh session -- so that only skills/apk-reverse/SKILL.md and its references influence the with-skill arm. A run that inherits the skill author's reasoning grades the author, not the skill.",
"arms": [
"with_skill: the skill directory is mounted and the prompt is the eval prompt verbatim.",
"without_skill: identical prompt, identical tools, no skill mounted. This is the baseline; a case where both arms do equally well tests nothing."
],
"recording": "Per run, capture the produced files under outputs/, token and wall-clock totals in timing.json, and one grading.json holding the per-assertion verdicts.",
"grading": "Grade against the _assertions list on each case, not against the prose in expected_output. Mark an assertion as ungradable rather than guessing: a wrong pass is worse than a recorded gap.",
"aggregation": "Write benchmark.json with, per case, the pass count in each arm and the delta. Report a case whose with-skill arm does not beat the baseline as a skill defect, not as a flaky run."
},
"evals": [
{
"id": 1,
"prompt": "I patched one method inside classes.dex, repacked the APK and signed it, and now the install fails. adb says INSTALL_FAILED_INVALID_APK, and on another device I got INSTALL_PARSE_FAILED_NO_CERTIFICATES. Is the hardening on this app blocking my build? Should I switch to a Frida hook instead?",
"expected_output": "The answer routes to the repack layer, not to the anti-tamper layer, and says why. It should name the concrete repack-time causes and the check that distinguishes them: resources.arsc losing its STORED method and 4-byte alignment because the whole archive was re-compressed, ZipInfo fields (extra, external_attr, create_system) being dropped when entries are rebuilt, and extractNativeLibs=false packages refusing a re-compressed native library. It should point at the repository's own repack path rather than hand-rolled zipfile writing, and it should state that an install failure after an unsigned or badly signed rebuild is not evidence of hardening. Answering \"yes, it is the anti-tamper layer, use Frida\" is a failure.",
"_assertions": [
"Names ZIP metadata loss and/or resources.arsc re-compression plus 4-byte alignment as the leading cause.",
"Names at least one concrete check the user can run (compression method of resources.arsc, alignment, ZipInfo field preservation, or extractNativeLibs).",
"Explicitly declines to attribute the install failure to hardening without evidence.",
"Routes to the repository's repacking route rather than recommending a Frida hook as the first move."
],
"_targets_reference": "references/repack-and-sign.md, references/byte-level-patching.md"
},
{
"id": 2,
"prompt": "I decompiled the dex and most of the method bodies are empty or just throw an exception. That means this app uses dex VMP (virtualization), right? Write me the section of the report that says it is protected with dex VMP so I can put it in the deliverable.",
"expected_output": "It refuses to declare dex VMP and explains what would have to be measured first. A hand-written opcode table or a hand-counted stub ratio is not sufficient evidence, because a decode desync produces exactly the same appearance; the failure count has to come from an official disassembler on the same image, and the empty-body ratio needs a zero-change control for comparison. It should also separate the neighbouring diagnosis -- a Java2C / JNI-sinking shell also shows 'everything moved out' while containing no virtualized bytecode at all -- and offer to run the repository's dex-side tooling to produce the numbers before writing any conclusion.",
"_assertions": [
"Does not produce the requested VMP verdict.",
"Requires an official disassembler's structural failure count (dexdump / baksmali) as the deciding evidence.",
"Requires a zero-change or alternative-hypothesis control before reading a stub ratio as extraction or virtualization.",
"Distinguishes dex VMP from Java2C / JNI sinking, or asks which of the two is being claimed.",
"Offers a concrete command sequence that would produce the missing evidence."
],
"_targets_reference": "references/advanced-unpacking.md, references/java2c-and-jni-sinking.md, references/detection-and-anti-analysis.md"
},
{
"id": 3,
"prompt": "In this classes.dex, find every string containing 'http' and tell me which methods reference them.",
"expected_output": "It actually runs the repository's tooling and reports real output: the strings route through the dex string table (dex_strings.py) and the referencing-method route through the instruction/field-index readers (find_refs.py, dexutil.py, or dexdump for an independent second opinion). The deliverable contains the exact commands, the counts, and at least one concrete string with its referencing method rendered from real output. Describing what one could do, naming generic tools, or listing strings from memory with no command behind them is a failure.",
"_assertions": [
"At least one repository script is executed and its command line appears in the answer.",
"The reported strings come from real output, not from memory or paraphrase.",
"Referencing methods are identified, not only the strings.",
"At least one cross-check with an independent decoder is performed or explicitly flagged as not done."
],
"_targets_reference": "scripts/dex_strings.py, scripts/dexutil.py, scripts/find_refs.py, references/byte-level-patching.md"
}
]
}

420
vendor-new/apk-reverse/skills/apk-reverse/evidence/capability-matrix.json Normal file
View File

@@ -0,0 +1,420 @@
{
"schema": "apk-reverse/capability-matrix",
"version": 1,
"purpose": "One machine-readable row per capability this skill claims: how strong the claim is, whether the route itself was ever measured, where the evidence lives inside an installed copy, and what the claim does not cover. An installed copy of this skill must be able to answer 'is this proven, how strongly, and where' without the repository it was extracted from.",
"read_with": "references/evidence-summary.md (the prose condensation of this file, and the only place that lists the non-shipped record)",
"status_labels": {
"ok": "The documented route was carried out on a real target and the result is recorded.",
"partial": "Part of the route was measured; at least one decisive step (usually install, launch, or a second independent producer) was not.",
"blocked": "Either the route depends on something this skill does not ship, or nothing in this record is evidence about it either way. Do not treat 'blocked' as 'impossible' -- treat it as 'no footing here'."
},
"strength_labels": {
"observed": "A command was run and its output exists behind the claim.",
"inferred": "Follows from a documented mechanism or a neighbouring measurement; the step itself was not executed.",
"unverified": "Assumed, or reported by someone else, and not reproduced in this record."
},
"evidence_rules": [
"Every path in `evidence` is relative to the installed skill root and resolves in an installed copy.",
"Paths in `evidence_not_shipped` sit at the repository root, which `npx skills add` does not install. Every one of them is marked 'repository root, not shipped'.",
"Rows are traceable to an existing evidence file or to a row of the repository's benchmark matrix (B1-B13). Nothing here was written from expectation.",
"Where a source document and a script disagree, the script's behaviour was taken as the measurement and the row says so."
],
"capabilities": [
{
"capability": "dex-equal-length-surgical-patch",
"group": "dex",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B1"],
"evidence": ["references/byte-level-patching.md", "references/dex-patching.md", "references/patch-audit.md", "scripts/dex_patch_bytes.py", "scripts/dex_find_insn.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-benchmark-l1-l3.md section 2 (repository root, not shipped)", "tests/benchmark.md row B1 (repository root, not shipped)"],
"limits": "Equal-length edits only. The end-to-end measurement is one 2-byte branch rewrite inside a 5,528-byte dex, whose whole-file diff is those 4 bytes plus dex header bytes 8..32; the patched file self-verifies. Constraints that cannot be expressed as an equal-length edit need the dexlib2 route, which has no public-target measurement."
},
{
"capability": "dex-method-level-rewrite-dexlib2",
"group": "dex",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/dex-patching.md", "references/patch-audit.md", "scripts/dexpatch/"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "The rewriter and the smali round-trip tools (scripts/smtool.py, scripts/patch_smali.py) carry no measurement against a public target in this record. The round-trip blind spot is documented: a whole-tree smali round trip can pass every table check and still crash with IncompatibleClassChangeError. Treat the route as documented, not proven."
},
{
"capability": "dex-header-integrity-and-verifier-legality",
"group": "dex",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B1"],
"evidence": ["references/patch-audit.md", "references/byte-level-patching.md", "scripts/dexutil.py", "scripts/dex_check_verifier.py", "scripts/dex_classdiff.py"],
"evidence_not_shipped": ["tests/benchmark.md row B1 (repository root, not shipped)"],
"limits": "Header recompute order (adler32 over d[12:], sha1 over d[32:]) is proven on one written file, which reproduces both fields. Passing scripts/dex_classdiff.py is necessary, not sufficient: it cannot detect code-item damage, and the checks that look further (instruction-length audit, the equal-length blind spot, move-result adjacency) live in references/patch-audit.md."
},
{
"capability": "dex-string-constant-patch",
"group": "dex",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/byte-level-patching.md", "references/dex-patching.md", "scripts/dex_strpatch.py"],
"evidence_not_shipped": ["docs/tool-verification/TOOL-VERDICTS.md (repository root, not shipped)"],
"limits": "Equal-length only: the shipped scripts/dex_strpatch.py enforces equal length and refuses a substring of a longer identifier. A different-length string edit is therefore not covered by that script; it needs the dexlib2 route, whose build is documented and unmeasured. No public-target measurement for the string path."
},
{
"capability": "repack-sign-install-single-apk",
"group": "packaging",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B1", "B9"],
"evidence": ["references/repack-and-sign.md", "references/verification.md", "scripts/repack.py", "scripts/install_test.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-benchmark-l1-l3.md section 2 (repository root, not shipped)", "tests/benchmark.md rows B1 and B9 (repository root, not shipped)"],
"limits": "Measured chain on this class of host: zip, then zipalign -p -f 4, then apksigner.jar --v1 --v2 --v3; the order cannot be reversed (the installer refuses with [-124]). B1 confirms installability by matching the on-device base.apk sha256 against the local build, and its control build through the identical pipeline still fails the old way. Signing has no uber-apk-signer here. The isolated-constant rewrite script writes an APK through a different code path whose zip-metadata and alignment handling was recorded as a defect; that path is not covered by this row."
},
{
"capability": "split-apk-and-app-bundle-sets",
"group": "packaging",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B9"],
"evidence": ["references/split-apk.md", "scripts/repack.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-split-apk.md (repository root, not shipped)", "tests/benchmark.md row B9 (repository root, not shipped)"],
"limits": "Analyze, unified re-sign and merge were run on two real sets: after resign every member shares one certificate with v1+v2+v3 true, and merge correctly refuses (exit 1, before writing) a set whose members carry their own resources.arsc while succeeding on the code/native-only set. adb install-multiple and launch confirmation were NOT executed, so installability of a re-signed set is unverified, and the split mode's install path is exactly where a wrong alignment hides."
},
{
"capability": "third-party-build-audit",
"group": "packaging",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/third-party-builds.md", "scripts/apk_diff.py"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "scripts/apk_diff.py is listed among the scripts the first verification pass never ran, and no benchmark row exercises it. Absence from that record is not a verdict on it, but nothing here supports trusting its output either."
},
{
"capability": "extraction-shell-detection-trivial-body",
"group": "unpacking",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B3"],
"evidence": ["references/advanced-unpacking.md", "scripts/dex_dump_validate.py", "scripts/dex_mem_scan.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-extraction-shell-bench.md (repository root, not shipped)", "tests/benchmark.md row B3 (repository root, not shipped)"],
"limits": "The ratio is bimodal, not thresholded: a real skeleton reads about 100 percent, a zero-change control 1.9 percent, nop-cleared bodies 0.0 percent and throw stubs 1.9 percent, so the shapes real shells use are invisible to it. The earlier 'tens of percent means a skeleton' rule was wrong and was deleted. Measured on an 11-variant synthetic set derived from one real dex, not on a live extraction shell, and three script defects were found by that set."
},
{
"capability": "dex-vmp-declaration-boundary",
"group": "unpacking",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B4"],
"evidence": ["references/advanced-unpacking.md", "references/code-virtualization-and-custom-linkers.md", "references/vmp-differential-analysis.md"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-extraction-shell-bench.md section 3.4 and section 4 (repository root, not shipped)", "tests/benchmark.md row B4 (repository root, not shipped)"],
"limits": "Static criteria can rule a dex-VMP out; they cannot rule runtime-applied hardening out. Never declare a VMP without an official disassembler's failure count: a hand-written opcode table reported 1,393 of 22,424 bodies as undecodable and produced a VMP verdict on a sample that the platform decoder showed to be ordinary dalvik with zero structural errors. Install, launch and refusal for that sample remain untested."
},
{
"capability": "vmp-differential-opcode-map",
"group": "unpacking",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": [],
"evidence": ["references/vmp-differential-analysis.md", "scripts/vmp_diff_harness.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-vmp-diff.md (repository root, not shipped)"],
"limits": "The fixture covers 218 of 224 opcodes and the closed loop re-derived that table exactly: 218 of 218 emitted opcodes, zero wrong entries, zero fabricated entries. What was not exercised is the other half of the premise -- no third-party hardening platform was contacted, and the assumption that a real engine substitutes per opcode at stable instruction length is inferred from public write-ups. No benchmark row covers this route."
},
{
"capability": "java2c-vs-jni-sinking-discrimination",
"group": "native",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B5", "B11"],
"evidence": ["references/java2c-and-jni-sinking.md", "scripts/java2c_probe.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-java2c.md (repository root, not shipped)", "tests/benchmark.md rows B5 and B11 (repository root, not shipped)"],
"limits": "The discriminating measurement is observed: native-declaration density separates the shapes by roughly 2000x (0.03-0.04 percent on three real JNI samples against 82.76 percent on a Java2C-shaped fixture), and Java_* symbols matched dex native-method counts 1:1 per ABI. No Dex-to-C compiler output was ever built here (no NDK, no host or device clang), so every Java2C-specific criterion is inferred. A symbol search fails silently on real Java2C: dcc emits -fvisibility=hidden and C++ jni.h inlines RegisterNatives."
},
{
"capability": "native-terminate-path-neutralisation",
"group": "native",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B2"],
"evidence": ["references/native-tamper-and-suicide.md", "references/native-and-so.md", "scripts/spawn_patch_detach.py", "scripts/hook_patch_only.js"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-benchmark-l1-l3.md section 3 (repository root, not shipped)", "tests/benchmark.md row B2 (repository root, not shipped)"],
"limits": "The tooling works; the patch does not. Writes landed 0.04-0.05 s after resume and survived detach, and the patched run and a read-and-write-back control both still died with SIGABRT at the same site: neutralising the terminate routine plus both of its branch inputs was insufficient, and the region is a dense cluster of mov w0,#imm ; ret stubs. Holding the process paused until the patch lands structurally cannot work, because a frozen process has no libraries mapped. On that target the pristine original already reports tamper, so no clean baseline exists there. No surviving-target result is claimed."
},
{
"capability": "native-instruction-tracing-stalker",
"group": "native",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B6"],
"evidence": ["references/native-dbi-and-deobfuscation.md", "scripts/stalker_trace.js", "scripts/stalker_report.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-stalker-exclude.md (repository root, not shipped)", "tests/benchmark.md row B6 (repository root, not shipped)"],
"limits": "Split result: following a hot libc export with no exclusions kills the process, and excluding 20 system modules keeps it alive -- but delivery is still zero events (blocks=0 blk=0 calls=0). Exclusion is therefore not the fix for a trace that reports nothing, which contradicts the framing that lists 'no events' among the symptoms exclusion addresses. Two harness traps that manufacture a false zero-event result were fixed in the same pass. No real OLLVM trace was produced, and no trace pipeline is verified on the test device."
},
{
"capability": "native-library-mapping-and-plt-resolution",
"group": "native",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": [],
"evidence": ["references/native-and-so.md", "references/toolchain.md", "scripts/lib_map.py", "scripts/elf_plt.py"],
"evidence_not_shipped": ["docs/tool-verification/TOOL-VERDICTS.md (repository root, not shipped)"],
"limits": "scripts/lib_map.py is verified on a real target. scripts/elf_plt.py was broken and fixed: one defect was proven to cause a false negative on a symbol that exists and is called, and a second compared a file offset against a virtual address under --diff --name-regions. Run it before patching any stub, and treat its 'no callers' answer as a claim about its own input parsing until you have seen it scan something you know references the symbol."
},
{
"capability": "native-crash-triage-and-swallowed-stacks",
"group": "native",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/native-tamper-and-suicide.md", "references/environment.md", "scripts/native_crash.py", "scripts/grab_crash.py"],
"evidence_not_shipped": ["docs/tool-verification/TOOL-VERDICTS.md (repository root, not shipped)"],
"limits": "scripts/native_crash.py was recorded as 'not applicable to this sample' in the first pass, and scripts/grab_crash.py was never run -- including on the one target whose crash-reporter SDK hid exactly the stack it claims to recover, so that claim is unverified rather than wrong. The general rule that survives without a measurement is narrower: a process that dies with only 'uncaughtException time: ...' and no stack has crashed."
},
{
"capability": "runtime-analysis-with-frida",
"group": "runtime",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B2", "B6", "B13"],
"evidence": ["references/dynamic-frida.md", "references/environment.md", "references/native-tamper-and-suicide.md", "scripts/run_probe.py", "scripts/frida_probe.js", "scripts/preflight.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-device-run.md (repository root, not shipped)", "tests/benchmark.md rows B2, B6 and B13 (repository root, not shipped)"],
"limits": "Instrumentation is itself a measured variable. Attaching is what kills some targets (the probe arm of B13 dies about 300 ms after resume while the no-script control stays alive); exclusion is what keeps others alive (B6); and a spawn ordering that holds the process paused cannot work, because no libraries are mapped yet (B2). Reading a device's refusal as a fact about the target is the failure mode this row exists to prevent -- the preflight check comes before blaming a patch."
},
{
"capability": "runtime-data-local-state",
"group": "runtime",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B8"],
"evidence": ["references/runtime-data.md", "scripts/datastore_inject.py", "scripts/blob_decode.py"],
"evidence_not_shipped": ["tests/benchmark.md row B8 (repository root, not shipped)"],
"limits": "The DataStore encode/decode path is proven on one real 81-byte container: an edited value re-encoded (81 to 80 bytes, length prefix recomputed) was read back by the injector. SharedPreferences and SQLite edits have no equivalent measurement here, and an edit that keeps reverting is a routing signal -- something else owns the value -- not a data-editing problem."
},
{
"capability": "anti-instrumentation-triage-naming-the-check",
"group": "runtime",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B13"],
"evidence": ["references/detection-and-anti-analysis.md", "scripts/anti_detect_probe.js"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-detection-pipeline.md sections 1, 4 and 5 (repository root, not shipped)", "tests/benchmark.md row B13 (repository root, not shipped)"],
"limits": "The check was named and the asymmetry recorded: strstr(\"frida\") fired at 303 ms of process life, then the process was gone with no tombstone, no crash and no ANR record, while the probe's own self-report showed TracerPid=0 with four frida-named mappings visible in /proc/self/maps. Reproducibility is honestly bounded: 3 same-shape arms, 1 delivered the full sequence, because a target that dies in about 300 ms leaves a sub-second window. The probe is observer-only by contract and patches nothing."
},
{
"capability": "emulation-and-frida-rpc",
"group": "runtime",
"status": "partial",
"route_measured": true,
"strength": "inferred",
"benchmark_rows": [],
"evidence": ["references/emulation-and-rpc.md", "scripts/frida_rpc_serve.py", "scripts/rpc_template.js"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-emulation-rpc.md (repository root, not shipped)"],
"limits": "The rpc.exports bridge was exercised end to end on a live device and the indexer pair (droidasc / ddc) was measured on a hardened APK, but the reference classifies the route itself as inferred: emulated execution costs environment-filling work, the unidbg build needed repair before it ran, and no target .so had been emulated at the time of writing. The tool was measured; the route was not."
},
{
"capability": "schemaless-protobuf-decode",
"group": "protocol",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B8"],
"evidence": ["references/protocol-reverse.md", "scripts/protobuf_decode_raw.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-protobuf-raw.md (repository root, not shipped)", "tests/benchmark.md row B8 (repository root, not shipped)"],
"limits": "26 of 26 built-in fixtures, 21 of 21 against the official runtime (re-encode byte-identical to SerializeToString()), 8 of 8 framing checks. Two ambiguities are inherent to the wire format rather than defects: 0801120178 is a valid nested message and a valid packed array, and a non-presence 0 is byte-identical to 'never assigned'. Four script defects were found by these checks and fixed."
},
{
"capability": "server-api-probing-and-tls-feature-scope",
"group": "protocol",
"status": "partial",
"route_measured": false,
"strength": "inferred",
"benchmark_rows": ["B7"],
"evidence": ["references/server-api.md", "references/tls-and-cert.md", "references/protocol-reverse.md", "scripts/probe_api.py", "scripts/tls_check.py"],
"evidence_not_shipped": ["tests/benchmark.md row B7 (repository root, not shipped)"],
"limits": "Client-scoped by design: this determines who owns a gate, not how to break an authorization the server performs. The native-side certificate-pinning row (a Cronet sample) was not run -- it needs a gradle+NDK build this class of host cannot do -- and QUIC/HTTP3 is stated as a limit rather than a route. Certificates and pinning are asserted from documented behaviour, so label a feature-scoped TLS failure as inferred until you have reproduced the failure yourself."
},
{
"capability": "dart-aot-analysis-given-a-snapshot-dump",
"group": "dart",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": [],
"evidence": ["references/dart-aot.md", "scripts/dart_disasm.py", "scripts/dart_pprefs.py", "scripts/dart_pool_strings.py"],
"evidence_not_shipped": ["docs/tool-verification/TOOL-VERDICTS.md (repository root, not shipped)", "docs/tool-verification/EXTENSION-dart-aot-formats.md (repository root, not shipped)"],
"limits": "Measured on a real Dart 3.6.0 build: the disassembler decoded identically to capstone (32 of 32 and 96 of 96), the caller index was re-derived independently with a symmetric difference of 0, and a specific business-logic site was located end to end. This row assumes a snapshot dump already exists -- producing one is a separate, blocked capability (see dart-aot-snapshot-dump-production)."
},
{
"capability": "dart-aot-string-table-format",
"group": "dart",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B12"],
"evidence": ["references/dart-aot.md", "scripts/dart_pool_strings.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-dart-aot-formats.md (repository root, not shipped)", "tests/benchmark.md row B12 (repository root, not shipped)"],
"limits": "Two claims were tested and both were wrong before this row. The arm64 packed form is confirmed at the byte level (tag byte equals 0x80|(len<<1) at every literal checked, 4,980 chained entries). The previously documented armv7 form (len*2 plus UTF-16) is refuted: the 32-bit record is [header u32][byte-count u32le][UTF-8 payload], and the extractor's zero for that ABI is a format mismatch rather than an empty table. An end-to-end patched build was never repacked and installed, so no device patch is claimed. A third claim, that the engine banner gives x.y.z (stable), is also refuted on that target."
},
{
"capability": "dart-aot-snapshot-dump-production",
"group": "dart",
"status": "blocked",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/dart-aot.md", "references/coverage-and-limits.md"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "This skill ships no snapshot container resolver and cannot synthesize one, so the Dart AOT workflow cannot start from an APK alone. The two offset spaces are not a constant offset apart: dart_pool_strings.py reports file offsets while the pool index speaks pool offsets, and a measured run found 4,237 distinct deltas over 4,241 shared strings. Ship a pinned front end (aotopsy, pure Go, no toolchain) or build blutter (about 80 s, needs a C++ toolchain) -- and say which one, because the two report different Dart version labels for the same binary."
},
{
"capability": "packer-custom-loader-identification",
"group": "hardening",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/packers.md", "references/code-virtualization-and-custom-linkers.md", "references/coverage-and-limits.md"],
"evidence_not_shipped": ["docs/tool-verification/TOOL-VERDICTS.md (repository root, not shipped)", "docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "The first verification pass did not exercise the packer, code-virtualization, custom-linker, integrity-check-redirection or tamper-suicide scenarios: its target has none of them, and its unmodified build already fails to start, which removes the repack-and-regress loop those scenarios need. The packer tooling was recorded as 'not applicable to this sample'. Nothing in the record is evidence either way about a protected target -- the benchmark pass measured shape discrimination only."
},
{
"capability": "module-delivery-instead-of-repack-lsposed",
"group": "hardening",
"status": "partial",
"route_measured": false,
"strength": "inferred",
"benchmark_rows": [],
"evidence": ["references/lsposed-and-modules.md", "scripts/lsposed_scaffold.py", "references/precedents/logd-broken-module-never-ran-case-3.md"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-lsposed.md (repository root, not shipped)"],
"limits": "Documented route, not a measured one: the generated project was built end to end with the gradle-free chain and its timings recorded, but the delivery route itself -- a module standing in for a refused repack -- has no public-target measurement. It needs a device with an active LSPosed, and confirming that a module actually ran takes more than one log surface; a logcat-only check has already produced one confidently wrong conclusion here."
},
{
"capability": "kernel-side-syscall-answer-forging",
"group": "kernel",
"status": "blocked",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": ["B10"],
"evidence": ["references/kernel-and-environment-hardening.md", "scripts/kernelsu_syscall_mask.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-kernel-weapons.md (repository root, not shipped)", "tests/benchmark.md row B10 (repository root, not shipped)"],
"limits": "The generator is measured -- an 11-file userspace module skeleton plus KPM/LKM/eBPF templates, with verify reporting 0 problems -- but no kernel-side artefact was compiled or loaded: the test kernel is 4.14.186 (eBPF needs 5.10+), the host has no aarch64 cross-compiler, make or ndk-build, and there is no kernel source. Structural correction with teeth: an ordinary KernelSU module is a userspace module and cannot change a syscall return value, so a template that claims to do so is wrong regardless of the toolchain."
},
{
"capability": "svc-site-scanning",
"group": "kernel",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B13"],
"evidence": ["references/kernel-and-environment-hardening.md", "scripts/svc_scan.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-detection-pipeline.md section 5 (repository root, not shipped)", "tests/benchmark.md row B13 (repository root, not shipped)"],
"limits": "Two independent decoders (a hand-written word scan and a capstone-based scan) returned an identical 214-site set on a device linker64, with an empty set difference, which is what makes the output usable as evidence about a library rather than about the scanner. A byte scan also matches data, so neighbour context decides whether a site is code: the hardened shell library's 21 svc sites are all data, and an ordinary module cannot act on them."
},
{
"capability": "on-device-tooling-mt-manager",
"group": "device",
"status": "partial",
"route_measured": true,
"strength": "observed",
"benchmark_rows": [],
"evidence": ["references/on-device-tooling.md", "scripts/mt_mcp_probe.py"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-kernel-ondevice.md (repository root, not shipped)"],
"limits": "The 'service is down' path is measured: the probe prints start-it-by-hand instructions and exits 2. The connected path needs MT Manager's APK MCP started by hand, so it is documented rather than measured, and the MCP surface itself was inventoried from the tool's own listing rather than exercised against a target."
},
{
"capability": "client-side-ads-and-server-issued-ui-config",
"group": "deliverable",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": ["B1"],
"evidence": ["references/ad-removal.md", "references/server-config-and-updates.md", "references/membership-and-limits.md"],
"evidence_not_shipped": ["docs/tool-verification/TOOL-VERDICTS.md (repository root, not shipped)"],
"limits": "Verified on a real mid-size Flutter-AOT-shaped target and its ad chain, where the dex reader was the primary workhorse of the analysis and a repacked build's on-screen behaviour change was confirmed. A splash, popup or tab set that arrives as server-issued UI config is a routing decision, not an SDK hunt, and the deliverable is the check that stops it, not the string that names it."
},
{
"capability": "membership-paywall-gate-client-enforceability-decision",
"group": "deliverable",
"status": "ok",
"route_measured": false,
"strength": "inferred",
"benchmark_rows": [],
"evidence": ["references/membership-and-limits.md", "references/account-gates.md", "references/handoff-boundaries.md"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "The decision framework is documented and its purpose is to say plainly when a gate is not client-enforceable; no gate-shaped public target was put through it in this record, so the framework is itself inferred rather than measured. A gate the server enforces is out of scope by design (see blocked-server-side-authority), and promising an unlock before classifying the owning layer is the specific failure this row prevents."
},
{
"capability": "update-and-forced-upgrade-neutralisation",
"group": "deliverable",
"status": "partial",
"route_measured": false,
"strength": "inferred",
"benchmark_rows": [],
"evidence": ["references/updates-and-forced-upgrade.md", "references/signature-derived-keys.md"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "Documented route with no public-target measurement in this record. The shape to expect is not a crash: a rebuilt build installs, runs, and fails every signed request, so a version check and a signature-derived key must be classified before the artifact is called finished. This is an every-deliverable concern, not an optional one."
},
{
"capability": "publishing-sanitisation-leak-scan",
"group": "governance",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": [],
"evidence": ["references/desensitization-and-leak-scans.md", "scripts/scan_leaks.py", "references/precedents/manual-grep-finds-what-nobody-grepped-case-5.md"],
"evidence_not_shipped": ["docs/tool-verification/EXTENSION-desensitization.md (repository root, not shipped)"],
"limits": "Measured against a planted corpus (30 findings across 6 categories, every rule firing, zero false positives on the do-not-anonymize list) and against this repository (26 strong hits, all inside its own evidence file, reduced to 0 strong and 4 weak across 129 files after correction). One thing it cannot do: the scanner was never run against the pre-correction historical tree, so 'the scan would have caught it' is inferred. A strong hit is a prompt to look, not a verdict, and over-redaction is the failure on the other side."
},
{
"capability": "signature-derived-keys",
"group": "deliverable",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/signature-derived-keys.md", "scripts/sig_probe.py"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "The offline candidate path from an APK and the authoritative live read are both documented in the reference; neither has a public-target measurement here. The failure shape is a rebuilt APK that installs, launches, and then fails every signed request, which is easy to report as a server problem and is not one."
},
{
"capability": "blocked-unity-il2cpp-react-native-ios",
"group": "boundary",
"status": "blocked",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/framework-runtimes.md", "references/coverage-and-limits.md"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "Unity / IL2CPP logic recovery, React Native / Hermes bytecode and Cordova internals, and every iOS or .ipa workflow are outside this skill. references/framework-runtimes.md identifies the runtime and establishes that the dex is not the battlefield; there is no verified recipe here for locating a method inside libil2cpp.so plus global-metadata.dat. Say so instead of improvising."
},
{
"capability": "blocked-server-side-authority",
"group": "boundary",
"status": "blocked",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": ["references/server-api.md", "references/handoff-boundaries.md"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "Determining who owns a gate is covered; breaking an authorization the server performs is not. There is no client-side patch that reaches it, and the honest deliverable is the residual: which authority decides, and why the artifact cannot change it."
}
]
}

131
vendor-new/apk-reverse/skills/apk-reverse/evidence/known-limitations.md Normal file
View File

@@ -0,0 +1,131 @@
# Known limitations — what an installed copy cannot do, and what was never measured
This file exists because of how the skill is delivered. `npx skills add` installs `skills/apk-reverse/`
and nothing else, so the evidence record this skill was measured against — which sits at the
repository root — is **not present in your install**. Every claim in `SKILL.md` and in `references/`
still carries a strength label; this file and `references/evidence-summary.md` are what let you read
that label without the record it came from.
Read this before treating any "this was measured" statement as covering the route you are about to
take. Silence in this list is not support for a route; it means nobody here paid for the counters yet.
## Strength labels
- **observed** — a command was run and its output exists behind the claim.
- **inferred** — follows from a documented mechanism or a neighbouring measurement; the step itself
was not executed.
- **unverified** — assumed, or reported by someone else, and not reproduced in this record.
A row whose strength is `unverified` is a statement about *this skill's evidence*, not about the
mechanism's truth. It means nobody has yet put a command and its output behind it here.
## Dependencies this skill does not ship — name them before the workflow starts
- **Dart AOT analysis needs a snapshot dump.** The workflow begins at a snapshot text file; producing
it needs a snapshot container resolver this skill does not contain and cannot synthesize. Ship a
pinned front end (`aotopsy`, pure Go, no toolchain) or build `blutter` (about 80 s, needs a C++
toolchain) — and **say which one**, because the two report different Dart version labels for the
same binary. The two offset spaces are not a constant offset apart: one tool reports file offsets
while the pool index speaks pool offsets, and a measured run found 4,237 distinct deltas over 4,241
shared strings. Never describe the object pool as something this skill decodes on its own.
- **The Android build-tools are not on PATH here.** `aapt2`, `D8`, `apksigner`, `zipalign`, `dexdump`
and `split-select` all live in a build-tools directory and must be passed explicitly. The measured
signing order is zip, then `zipalign -p -f 4`, then `apksigner.jar --v1 --v2 --v3`; reversing it is
refused by the installer. There is no `uber-apk-signer` on this class of host, and no `apktool`,
`jadx`, `gradle`, NDK, host or device clang, `android.jar`, or smali/baksmali jar.
- **The runtime routes need a real device.** Hooking, dumping, install-then-launch verification and
the module route all require a rooted device; several of them additionally require a framework the
device must have active.
- **The evidence record and the benchmark matrix do not ship.** Their paths appear in older
documents as if they were openable files. They resolve only in the repository they were written in.
## Capabilities with no footing in an installed copy
These are not "hard"; they are paths where nothing in this record is evidence either way, so a plan
built on them has nothing to check itself against.
- **Producing a Dart AOT snapshot dump.** This skill ships no resolver for it. See the dependency
section above.
- **Kernel-side syscall answer forging.** The generator is measured; **no kernel-side artefact was
ever compiled or loaded**. The tested kernel is 4.14.186 (eBPF needs 5.10+), the host has no
`aarch64` cross-compiler, `make` or `ndk-build`, and there is no kernel source. One structural
correction survives without a measurement and is worth more than the templates: **an ordinary
KernelSU module is a userspace module and cannot change a syscall return value.**
- **Unity / IL2CPP logic recovery, React Native / Hermes bytecode and Cordova internals, and every
iOS or `.ipa` workflow.** The runtime can be identified; the logic recovery is not covered, and
there is no verified recipe for locating a method inside `libil2cpp.so` plus `global-metadata.dat`.
- **Defeating a server-side authority.** This skill determines *who owns a gate*, not how to break an
authorization the server performs. Report it as a residual instead of patching harder.
## Routes that were never exercised
- **The packer, code-virtualization, custom-linker, integrity-check-redirection and tamper-suicide
scenarios were never run end to end.** The target used for the first verification pass has none of
those features, and its unmodified build already fails to start — which removes the
repack-and-regress loop those scenarios need. The packer tooling was recorded as *not applicable to
that sample*. The benchmark pass measured **shape discrimination** on public targets; it did not run
a live extraction shell. Anything you read here about a protected target is `inferred` unless it
says otherwise.
- **Scripts from the first pass that carry no measurement** (absence is not a verdict on them):
the crash tools, the APK differ, the screenshot tool, the installer test, the repacker, the
equal-length patcher, the instruction finder, the verifier checker, the class differ, the string
patcher, the smali tools, the DataStore injector, the API prober, the TLS checker, the USB network
helper, and the device shell helper. The benchmark pass has since run some of these; the rest are
still unmeasured, and the references say so where it matters. `grab_crash.py` is the sharpest
example: it claims to recover a stack a crash-reporter SDK swallowed, and it was never tried — on
the one target that presented exactly that situation.
- **Steps that a route needs but that were not executed**, so the route is not end-to-end:
installing a re-signed split set and confirming it launches; installing, launching or getting a
refusal from the VMP-boundary sample; the native certificate-pinning row (it needs a gradle+NDK
build this host cannot do); an end-to-end Dart string patch repacked and installed; and byte
agreement between two independent dumpers.
- **No real OLLVM trace and no trace pipeline verified on a device.** Exclusion keeps a target alive
but does not restore event delivery, and a zero-event trace is a boundary to identify rather than a
recipe to follow.
## Conclusions that changed — an older copy may still state the old one
Each of these was a claim this skill used to make and no longer does. If you are reading a copy or a
cached page that states the old version, the old version is wrong.
- **"Tens of percent means an extraction-shell skeleton."** Deleted. The ratio is **bimodal, not
thresholded**: a real skeleton reads about 100 %, a zero-change control 1.9 %, nop-cleared bodies
0.0 %, throw stubs 1.9 %.
- **A VMP verdict withdrawn.** A hand-written opcode table reported 1,393 of 22,424 bodies as
undecodable and that was read as private opcodes; the platform decoder found 34,566 instruction
decodes and **zero structural errors** on the same file. The surviving rule: never declare a VMP
without an official disassembler's failure count.
- **"Exclusion fixes the zero-event trace."** It does not. Exclusion is what keeps the target alive;
delivery stayed at zero in the treatment arm.
- **"`repack.py` has no apksigner signing path."** That conclusion expired mid-pass when the path was
added; the signing chain was then measured end to end.
- **The armv7 Dart string-table form is UTF-16 with a `len*2` length.** Refuted at the byte level:
the 32-bit record is `[header u32][byte-count u32le][UTF-8 payload]`, and the extractor's zero for
that ABI is a format mismatch rather than an empty table.
- **A documented tool flag that does not exist, and a documented tool class that does not match the
shipped script.** Where a reference names a command or a tool class, check it against
`scripts/` in this install before running it — the script is the measurement, and a reference line
can lag behind it.
## What an installed copy can answer, and what it cannot
| Question | Answerable here | Not answerable here |
|---|---|---|
| Is this capability proven, and how strongly? | `references/evidence-summary.md`, `evidence/capability-matrix.json` | The exact commands and captured output behind it |
| Which capability is outright blocked? | `evidence/capability-matrix.json` rows with status `blocked` | Whether the mechanism would work on your target |
| Which tool versions was this measured on? | `evidence/tested-tool-versions.json` | Versions this skill was never measured on — that list is there too, but it is a list of gaps, not of results |
| What does the skill deliberately not cover? | `references/coverage-and-limits.md` | — |
| What is the exact output of a past run? | — | The record at the repository root, which does not ship |
## Failure modes this file exists to prevent
- Reading "a command was run" as "the command was correct". A tool that runs without error is not a
tool that is right: one script here produced a false negative on a symbol that exists and is called,
and another returns a clean "no references" answer for input it cannot read at all.
- Reading `inferred` as `observed` because the surrounding prose is confident.
- Reading `unverified` as a refutation. It is an absence of evidence here, not evidence of absence.
- Reading silence in the "never exercised" list as support for a route.
- Calling a route verified when the install step was never run. Several routes in this skill stop
exactly there, and an artifact that installs is not the same artifact as one that launched.
- Treating a documented version requirement as a measured one. The version table lists what was read
off the tools, and separately lists the versions nothing here has ever run on.

103
vendor-new/apk-reverse/skills/apk-reverse/evidence/tested-tool-versions.json Normal file
View File

@@ -0,0 +1,103 @@
{
"schema": "apk-reverse/tested-tool-versions",
"version": 1,
"purpose": "Every tool and library version this skill's claims were measured on, with the command that produced each number, so a reader can tell 'this was verified' from 'this was verified on a version I do not have'. An installed copy carries this file; the full run records do not ship.",
"how_to_re_probe": [
"Windows / PowerShell host, as used for every measurement below.",
"Python libraries: python -c \"import importlib.metadata as m; print(m.version('frida'))\"",
"CLI tools: <tool> --version (or -V).",
"PATH presence: Get-Command <tool>. Anything reported NOT ON PATH must be passed explicitly by path, and scripts here do that."
],
"host": {
"os": "Windows 10.0.26100 (x86_64)",
"python": "3.14.0 (CPython, MSC v.1944 64-bit AMD64)",
"shell": "PowerShell 7 (pwsh)",
"strength": "observed"
},
"measured_on": {
"note": "Every entry below was probed on the host described above during the packaging pass; each carries the command that produced it. Listing a tool here means its version was read, not that every capability of it was exercised -- see capability-matrix.json for the latter.",
"languages_and_runtimes": [
{"name": "python", "version": "3.14.0", "probe": "python -V", "strength": "observed", "used_by": "all scripts under scripts/"},
{"name": "java", "version": "17.0.4.1 (2022-08-18 LTS)", "probe": "java -version", "strength": "observed", "used_by": "scripts/dexpatch/, jar-based tools"},
{"name": "javac", "version": "17.0.4.1", "probe": "javac -version", "strength": "observed", "used_by": "scripts/lsposed_scaffold.py build notes"},
{"name": "node", "version": "v24.11.1", "probe": "node -v", "strength": "observed", "used_by": "no script in this skill requires it"},
{"name": "sqlite3", "version": "3.50.6 (32-bit build)", "probe": "sqlite3 -version", "strength": "observed", "used_by": "local-state work"},
{"name": "git", "version": "2.46.2.windows.1", "probe": "git --version", "strength": "observed", "used_by": "repository gates"}
],
"device_toolchain": [
{"name": "adb", "version": "1.0.41 (platform-tools 37.0.1)", "probe": "adb version", "strength": "observed", "used_by": "scripts/install_test.py, scripts/coldstart.py, scripts/preflight.py, scripts/devsh.py"}
],
"android_build_tools": {
"package": "Android SDK build-tools 34.0.0",
"probe": "source.properties: Pkg.Revision=34.0.0",
"strength": "observed",
"not_on_path": true,
"note": "None of these resolve through PATH; the working calls pass them explicitly. The measured signing order is zip, then zipalign, then apksigner -- reversing it is refused by the installer.",
"members": [
{"name": "aapt2", "version": "2.19-10229193", "probe": "aapt2 version"},
{"name": "D8", "version": "8.2.2-dev (build facedf41bbd28b563d1e9e09c5f72d7c5ca598d5)", "probe": "java -cp <build-tools>/lib/d8.jar com.android.tools.r8.D8 --version"},
{"name": "apksigner", "version": "0.9 (jar)", "probe": "java -jar <build-tools>/lib/apksigner.jar --version"},
{"name": "zipalign", "version": "no version string emitted", "probe": "zipalign (prints the 'Zip alignment utility' banner)"},
{"name": "dexdump", "version": "no version string emitted", "probe": "dexdump (prints usage and exit)"},
{"name": "split-select", "version": "same package (34.0.0)", "probe": "package revision"}
]
},
"frida_stack": [
{"name": "frida", "version": "16.7.19", "probe": "frida --version", "strength": "observed", "used_by": "scripts/run_probe.py, scripts/spawn_patch_detach.py, scripts/hook_patch_only.js, scripts/anti_detect_probe.js, scripts/stalker_trace.js, scripts/frida_rpc_serve.py"},
{"name": "frida-tools", "version": "13.7.1", "probe": "python -c \"import importlib.metadata as m; print(m.version('frida-tools'))\"", "strength": "observed"},
{"name": "objection", "version": "1.12.5", "probe": "python -c \"import importlib.metadata as m; print(m.version('objection'))\"", "strength": "observed"}
],
"python_libraries": [
{"name": "androguard", "version": "4.1.4", "strength": "observed", "used_by": "dex reading paths"},
{"name": "capstone", "version": "5.0.9", "strength": "observed", "used_by": "scripts/dart_disasm.py comparison, scripts/svc_scan.py second decoder"},
{"name": "unicorn", "version": "2.1.4", "strength": "observed", "used_by": "emulation route"},
{"name": "lief", "version": "1.0.0", "strength": "observed", "used_by": "ELF work"},
{"name": "pyelftools", "version": "0.33", "strength": "observed", "used_by": "ELF work; also supplies the 'readelf' script that PATH resolves to (see notes)"},
{"name": "protobuf", "version": "6.33.6", "strength": "observed", "used_by": "the official-runtime cross-check behind the schema-free decoder"},
{"name": "networkx", "version": "3.6.1", "strength": "observed"},
{"name": "pydot", "version": "4.0.1", "strength": "observed"},
{"name": "asn1crypto", "version": "1.5.1", "strength": "observed"},
{"name": "cryptography", "version": "48.0.1", "strength": "observed"},
{"name": "lxml", "version": "6.0.2", "strength": "observed"},
{"name": "mitmproxy", "version": "12.2.3", "strength": "observed", "notes": "the library version"}
],
"proxy": [
{"name": "mitmdump (standalone binary in the local tools directory)", "version": "11.1.2 binary, bundling Python 3.13.1 and OpenSSL 3.4.0", "probe": "<binary> --version", "strength": "observed", "notes": "This is NOT the same version as the mitmproxy library above. A local workbench note recorded 12.2.3 against this binary's path; the binary reports 11.1.2. If a captured-flow shape matters, read the version from the binary you actually invoke."}
],
"readelf_note": {
"resolved_to": "a pyelftools-supplied Python script earlier on PATH than any GNU binutils readelf",
"version": "not obtainable from the resolved executable (its --version prints nothing)",
"strength": "observed",
"consequence": "A local workbench note recorded 'readelf 2.28'. That number does not match what PATH resolves to here. Treat any readelf-derived figure as produced by the pyelftools script until proved otherwise."
}
},
"external_toolchains_measured_elsewhere": {
"note": "These were exercised in earlier passes and their versions are recorded in files at the repository root, which an installed copy does not contain. Listed here so the claim is not silently version-free.",
"record_location": "docs/tool-verification/TOOL-VERDICTS.md and docs/tool-verification/EXTENSION-*.md (repository root, not shipped)",
"entries": [
{"name": "blutter", "reference": "HEAD 4a60ac6; init_env downloads ICU 73.2 + capstone 4.0.2", "result": "works; the first-run crash was not reproducible after relink; build about 78 s", "strength": "observed"},
{"name": "aotopsy", "reference": "pure Go, no toolchain needed; hash recorded in the verdict file", "result": "works out of the box; the same feature flags blutter derived", "strength": "observed"},
{"name": "devsh / device shell", "reference": "toybox shell, one rooted Android 11 / API 30 / arm64-v8a device", "result": "the device facts that shaped several traps", "strength": "observed"}
]
},
"not_available_here": [
{"name": "apktool", "state": "not on PATH", "probe": "Get-Command apktool", "notes": "the local launcher was recorded as a dead link in the workbench record (not shipped), so no apktool version has ever been exercised here"},
{"name": "jadx", "state": "not on PATH", "notes": "documentation treats it as a readable viewer of last resort, not as a verified measurement source"},
{"name": "unzip, tshark, rabin2 (rizin), gdb", "state": "not on PATH"},
{"name": "gradle", "state": "not installed", "notes": "the module route uses a gradle-free chain"},
{"name": "Android NDK / host clang / device clang", "state": "not installed", "notes": "this is why no Dex-to-C output was ever built and why the Java2C-specific criteria stay inferred"},
{"name": "smali / baksmali jars, android.jar", "state": "absent", "notes": "scripts/smtool.py carries its own classpath expectations"},
{"name": "uber-apk-signer", "state": "absent", "notes": "the measured signing chain is zipalign plus apksigner.jar"},
{"name": "keytool, jarsigner", "state": "not on PATH", "notes": "present inside the JDK installation and must be passed by path"}
],
"not_tested_at_these_versions": [
"frida: only 16.7.19 was measured. No claim here has evidence against frida 17.x or any other major.",
"Android device: only Android 11 / API 30 / arm64-v8a was measured. The documented analysis of Android 12-16 behaviour (why classic active-invocation hooks stopped working) is inferred from public work, not measured.",
"Android build-tools: only 34.0.0. aapt2, D8, apksigner, zipalign and dexdump have no measurement on any other revision.",
"Python: only 3.14.0. JDK: only 17. Node: only v24.11.1 (and nothing here needs it).",
"Kernel side: no kernel module was compiled or loaded at any version, on any kernel. The generator was exercised; the artefact was not.",
"Gradle and the Android NDK: never used, at any version.",
"Dart: the string-table formats were checked on two ABIs of one build (a Dart 3.6.0 snapshot), through a front end this skill does not ship.",
"Mitmproxy: the library and the standalone binary measured different versions (12.2.3 and 11.1.2); no capture in the record is tied to a mitmproxy release."
]
}

160
vendor-new/apk-reverse/skills/apk-reverse/references/account-gates.md Normal file
View File

@@ -0,0 +1,160 @@
# Account gates: sign-in walls, forced binding, guest mode
Load this when the task mentions "no login required", "don't force me to sign in", "skip phone
binding", "guest mode", or when a screen is unreachable without an account — and also **before**
promising that any account- or membership-derived feature will work offline.
This file is mostly about **telling apart two things that look identical in the UI and are completely
different in reality**:
- a **client-side gate** — code that refuses to proceed. Patchable.
- an **account-scoped resource** — the screen is empty because the server sent nothing, because the
request was made without a valid identity. **Not** patchable.
Confusing the two is how a task ends up "done" while the user stares at an empty page.
## Step 1: classify what you are looking at
| Symptom | Most likely cause | Can a client patch help? |
|---|---|---|
| A dismissible dialog / bottom sheet asking to sign in, then the app works | client-side gate | **yes** |
| A full-screen wall with no dismiss, app unusable | client-side gate | **yes** |
| Screen renders its frame but the list is empty and a "log in to see" hint is drawn | **account-scoped data** | no — the data does not exist locally |
| A specific action (comment, favourite, follow, download) refuses | client-side gate | usually yes |
| The action succeeds locally but reverts on restart / never reaches the server | **server-side identity** | no |
| "Binding required" before an action | client-side gate | **yes** |
| An entitlement appears then disappears | server-side authority | no |
**The deciding question is never "is there a login screen?" — it is "does the thing I want exist
without an identity?"**
Ask it before patching:
- If the content is **fetched per-account** (subscriptions, favourites, reading history, purchased
items), a patch changes the gate and nothing else. The list stays empty because the server was asked
for *this account's* items and there is no account.
- If the content is **public** and merely hidden behind a prompt (a reader, a catalogue, a settings
page), the patch is real and the feature becomes usable.
Reading the code tells you which: follow the "sign in" prompt to what it guards, then look at whether
that guarded path issues a request carrying a token, or touches only local state.
## Step 2: find the convergence point
Sign-in checks are usually concentrated, because a UI cannot meaningfully enforce login at thirty
call sites. Look for a small number of helpers with names in this family:
```
checkLogin* ensureLogin* requireLogin* needLogin* isLogined* isLoggedIn*
checkBind* ensureBind* requireBind* needBind*
showLogin* openLogin* loginSheet* loginDialog*
```
and, on the state side, a **login-state field or stream** that every helper reads
(`isLogin`, `loginedStream`, an `Rx<bool>`, an auth-state enum, a credential store wrapper).
**Rule: patch the predicate, not the prompt.** Making the sheet not appear is cosmetic — the caller
still treats the answer as "not signed in" and takes its else-branch, which is usually where the real
feature got skipped. The durable fix is to make the *verdict* come back as "signed in / no sign-in
needed", so the caller takes its happy path.
Two useful shapes, both better than editing the dialog:
| Target | Change | Effect |
|---|---|---|
| the gate predicate used by many callers | return "ok / already signed in" unconditionally | every caller proceeds; no new code path |
| the "should I prompt?" decision inside the helper | return "nothing to do" | the prompt never appears and callers proceed |
Prefer the first when callers branch on the result; prefer the second when the helper is fire-and-forget.
**Count the callers before choosing** (`scripts/find_refs.py`). A predicate with one caller is a
surgical target; a predicate with forty callers is a behavioural change you must reason about — it will
also release flows you were not asked to touch (settings, purchase, upload). That may be fine, but it is
a decision, not a side effect.
### Forced binding specifically
Phone binding is the same pattern with its own helper and usually **two** state checks in series
(bound-phone? and email-or-other-recovery?), because either satisfies the requirement. Patch the
combined verdict, not the first condition — a patch that only satisfies one branch still prompts on the
other.
Watch for a **"skip"/"later" path the app already has**, and for **which actions the binding actually
gates**. Binding is often required for a subset (password change, deletion, payment) while the rest of
the app is unaffected; releasing it globally changes what the app believes about account safety.
## Step 3: what NOT to do
These are the tempting moves that produce a worse artifact than doing nothing:
- **Do not fabricate a session.** Writing a dummy token, a fake credential blob, or a placeholder
identity into the credential store makes the client *believe* it is signed in. It then calls
identity-scoped endpoints with garbage, and the app enters a state that is **worse than signed out**:
screens show an authentication-failure message, or silently return empty data with no explanation.
A signed-out app that says "please sign in" is better than one that claims a session and fails.
*(If you touch stored credentials at all, keep a backup and expect to have to restore it.)*
- **Do not delete the sign-in UI.** Removing the entry point does not change the verdict; it only makes
the failure unexplainable when a caller does branch on it. Keep the entry point working — it is
normally an explicit requirement ("signing in must still be possible").
- **Do not release gates that protect server-side operations.** If the action's result is an
entitlement the server must grant (a purchase, a quota, a subscription), releasing the client gate
only lies to the UI. See `references/membership-and-limits.md`.
- **Do not assume "guest can browse" means "guest can do everything".** Establish per feature which of
the three states you are in: public, gate-only, or identity-required.
## Step 4: the session-loss problem (this one is unavoidable)
Any change to the signing identity — including a plain reinstall of *your own* build — can invalidate
stored credentials, because they may be sealed with a key held by the platform keystore rather than by
the app.
**What you observe:** the app opens fine, then one subsystem reports it cannot decrypt or unwrap
something, or the user is silently signed out.
**What it means:** the app's private data was carried across an uninstall/reinstall (or restored from a
backup), but the encryption key was destroyed with the app. The ciphertext is now unopenable by anyone.
**What to do:** treat "the user signs in once on the new build" as the expected, documented outcome — not
as a defect to fix. Put it in the delivery notes in one line, together with the fact that a later
upgrade over the *same* signing key can be installed without uninstalling, and therefore will not lose
the session.
**Do not** chase this into the keystore. Restoring the old key is not part of packaging a patched build,
and any workaround that keeps the *old* ciphertext alive while the key is gone is a dead end.
Related and worth knowing: an app may keep the same logical account in **two places** — a plain
preference blob and a sealed store. After a reinstall the plain one survives and the sealed one does
not, which shows up as an account that is half-present (identity fields populated, token empty). That
half-state is a useful diagnostic: it tells you the app reads its identity from the sealed store, which
is exactly the thing you must not try to forge.
## Step 5: verify
The claim to support is narrow and should be stated narrowly.
1. **The prompt does not appear** on a cold start and on the screens that used to show it.
2. **The feature behind it actually works**, not merely "no longer blocked": open the reader, the
catalogue, the settings page — whatever the gate guarded — and confirm real content or real function.
3. **The sign-in entry still exists and is usable.** Signing in with a real account must still work; that
is usually part of the request, and it is the thing that proves you patched a gate rather than broke
authentication.
4. **Signed-in behaviour still works** for a real account, so you can tell the difference between
"released the gate" and "broke the auth path".
5. **No fake-session artefact is left behind** — check that you did not write anything into the
credential store or preferences. If you did, remove it and re-verify from a clean install.
State the result as: *"the gate no longer blocks X, from a signed-out state; **the content behind it is
public / the content behind it is per-account and therefore still empty when signed out**."* That second
clause is what stops a client patch from being mistaken for an account.
## Reporting template
```
Gate: <sign-in sheet | binding prompt | action-scoped check>
Class: client-side gate / account-scoped data (not patchable)
Patch: <predicate or decision, not the prompt>
Signed-out state: <what works now, per screen>
Signed-in state: <still works, listing what still requires an account>
Not achievable: <per-account lists, entitlements — with the reason>
Session note: sign in once after install; upgrade over the same key keeps the session
```

180
vendor-new/apk-reverse/skills/apk-reverse/references/ad-removal.md Normal file
View File

@@ -0,0 +1,180 @@
# Ad removal
Goal: ads stop appearing, and **nothing else breaks**. Read `pitfalls.md` P5 and P6 before patching anything here.
**Load this when:** an ad, promo, splash or rewarded video must stop appearing, or an ad-related SDK was found in recon. It gives the enumeration step that decides *which* layer the ad lives on, then the removal per layer.
## Step 1: enumerate what "ads" means in this app
Do not assume "ads" is one thing. Enumerate first.
**SDK-integrated ads** (the app calls an SDK):
- Search dex strings for SDK markers: `openadsdk`, `TTAdSdk`, `TTAdNative`, `Pangle`, `pangolin`, `com.qq.e`, `GDTAd`, `gdt_plugin`, `anythink`, `ATSDK`, `ATRewardVideoAd`, `bdxadsdk`, `sigmob`, `ksad`, `mobads`, `beizi`.
- **Aggregators** (AnyThink, TopOn, and similar) are wrappers: the ad networks underneath are *their* adapters, not app code. Patching an individual network's class is usually pointless; the aggregator still runs and still reaches the network.
- The app almost always funnels every call into **one wrapper class**. That class is the patch surface (Step 2).
**Dynamic-plugin ad SDKs** (the SDK is not in the APK). Aggregators increasingly ship as a runtime plugin: the app downloads an APK on first launch into its private directory and loads it through a plugin classloader.
```
/data/data/<app.package>/files/<sdk>_p/<plugin.pkg>/version-*/apk/base-1.apk
/data/data/<app.package>/files/<sdk>_p/<plugin.pkg>/version-*/lib/<abi>/*
```
- Consequence: **deleting bundled `assets/`, `lib/*.so`, or the SDK's classes from the dex does not kill this SDK.** The loader re-downloads on the next launch, and a half-deleted plugin is worse than an intact one.
- The kill switch is the **init gate** (Step 2). Killing init also prevents the working directory from ever being created, which is the strongest verification signal available (Step 4).
- The plugin may also be bundled. Two copies still means one init gate; patch that, not the payload.
**Server-driven ads / sponsored content** (the server returns the ad, the client renders it):
- **This is the most common shape in a modern app, and it is usually the EASIEST to
remove — not the hardest.** Go to `server-config-and-updates.md` for the full
procedure. The short version: the client keeps a complete "do not show it" branch
so the operator can turn the slot off, so neutralising that one branch is a
small, local edit.
- Look for endpoints like `/adverts`, `/adv`, `/banner`, `/config`, and DTOs named
`Advertisement*`, `Advert*`, `Banner*`, `Promotion*`, or — more common in practice
— a generic `*Config` payload carrying per-feature blocks (`splash`, `noticePopup`,
`updatePopup`, `tabbar`, `banner`).
- The client renders whatever the list contains. If the list is empty there is nothing
to show, but see `pitfalls.md` P5 on **how not** to empty it: never make a shared
request fail.
- Sponsored cards that look like content (a VPN promo, a network-accelerator card, a
third-party product) are usually exactly this.
**A note on what "no SDK found" means.** If you have grepped the dex, enumerated the
loaded classes at runtime, and counted the SDK log tags in a capture, and all three
are zero (see Step 1), then there is no SDK, and continuing to search for one is a
dead end rather than a thorough approach. Re-classify against this list instead. A
zero result on all three signals is a strong positive finding about where the
behaviour lives, and it should move you to the server-config path immediately.
**Legit content that looks like an ad.** Verify before acting. A `/adverts?position=banner` response containing anime titles and poster images is the home-page carousel, not an advertisement. Removing it removes real functionality.
## Step 2: find the single convergence point
Almost every ad-enabled app wraps one SDK in a **single singleton helper**: `init`, `loadSplash`, `loadBanner`, `loadFullScreen`, `loadReward` all live in the same class. Find that class, and patch nothing else if you can.
```
helper.init(Application) -> Sdk.init(appId, appKey); Sdk.start() // plus plugin download
helper.showSplash(Activity, ViewGroup, onClose: () -> Unit)
helper.showInterstitial(Activity)
helper.showReward(Activity, onStart: () -> Unit, onFinish: (Boolean) -> Unit)
helper.preload*(Activity)
helper.canShow*(Activity): Boolean // == isReady / isLoaded / isAdReady
```
**Preferred play: readiness always false + init is a no-op.**
1. Make the readiness predicate constant `false` (`isReady` / `canShow*` / `isLoaded` / `isAdReady`).
2. Make `init` an empty method (`return-void`) so the SDK never builds a network stack and never requests a plugin.
3. Only then touch show/load methods, and only for slots that bypass the readiness predicate.
Why this order works:
- Callers of a readiness predicate **already own a "no ad available" branch**, and the app ships and tests it. Returning `false` routes every slot through that existing path: no crash, no hang, no new code path.
- Killing init removes the SDK's network, cache, and download activity entirely. That is the difference between "the ad is hidden" and "the ad subsystem never started", and only the second claim is provable (Step 4).
- Per-slot patching (splash this week, interstitial next) leaves the SDK alive and downloading. Avoid it unless a slot ignores the predicate.
Method classification, once the helper is located:
| Method | Contains | Patch |
|---|---|---|
| Init | `Sdk.init(...)` + `Sdk.start()`, plugin download | body -> `return-void` |
| Readiness / gate | `isAdReady`, `canShow`, `isLoaded` | return `false` / `0` |
| Show splash / interstitial | constructs an ad object, `loadAd()` / `show()` | `return-void`, **but preserve the completion callback** |
| Show reward | same, plus success/failure lambdas | `return-void`; decide deliberately whether to grant the reward |
| Preload / warm-up | `RewardVideoAutoAd.init(...)`, cache priming | `return-void` |
### The callback trap (this one breaks startup)
A splash/loading ad usually takes a **completion lambda** as a parameter, and the app's startup state machine waits for that lambda before dismissing the splash screen.
**If you replace the method with a bare `return-void`, the lambda never fires and the app hangs on the splash screen forever.** A patch that "removed the ad" and produced a frozen splash is this bug, not a repackaging bug.
Correct shape, skip the ad and still complete:
```
.method public final showSplash(Landroid/app/Activity;Landroid/view/ViewGroup;Lkotlin/jvm/functions/Function0;)V
.registers N
invoke-interface {p3}, Lkotlin/jvm/functions/Function0;->invoke()Ljava/lang/Object;
return-void
.end method
```
**Before patching, prove the contract.** Read the listener class the method constructs and find which callback triggers the completion lambda (`onAdDismiss`? `onAdError`? `onAdLoadTimeout`?). That tells you whether to call it on success, on failure, or unconditionally.
For a reward ad whose lambdas are `(onStart, (Boolean) -> Unit)`, granting the reward client-side is a choice: `onFinish(true)` makes the app treat the reward as earned. Check whether the reward is validated server-side before promising it.
### Global timestamp gates (one write, many slots)
Some "free reading + ads" apps do not gate ads per slot. They store one local timestamp and every slot, cooldown, and interval check reads it: splash, chapter-break interstitial, unlock prompt, "next ad in N minutes". Patching call sites one by one always misses some.
- **Find it:** dump the app's preference keys and filter for ad-ish names (`ad`, `free`, `cooldown`, `interval`, `end_time`, `expire`, `next_`). Then hook `SharedPreferences.getLong/getInt/getString` and watch which key is read immediately before a slot appears. The key read across several screens is the gate.
- **Patch it:** in `Application.onCreate()`, write a far-future value as early as possible (for example `4102444800000L`). Avoid `Long.MAX_VALUE`: implementations often compute `end - now` and can misjudge an extreme value. One write disables every reader at once, which is far more stable than editing each consumer.
- **If the app overwrites it later:** observe what it writes back under different start sequences. A single startup write is not automatically permanent for keys the app re-asserts; then patch the writer or the reader instead.
- **Same technique, two key semantics:** "ad-free until `<date>`" keys and "cooldown/interval" keys both respond to this, but the value differs. For a cooldown key, a far-future value means "next ad is never due", which is what you want, while a progress/check-in flow reading the same key may behave oddly.
- **Side effects are mandatory in the delivery notes.** The same key is often reused as a business field (the anchor for a new-user ad-free window, a remaining-quota counter). Forcing it to a far-future value changes those business judgements. Usually harmless for a de-ad goal, but state it explicitly and confirm the related screens do not show a broken state.
- With DataStore instead of SharedPreferences, a write inside `Application.onCreate` needs a coroutine or a blocking write; see `runtime-data.md`, and never block the main thread on persistence.
### "Ads" vs ad-gating UI
Some blocking dialogs contain no ad slot at all ("watch an ad to unlock", skip/continue prompts, cooldown notices). If the goal is "the app is usable", they count as ads and must be handled too. Handle them by letting the gate pass, not by deleting the dialog view.
- Where the gate reads the timestamp key above, the timestamp patch already covers it.
- Where it reads a boolean or a `canUnlock()` method, patch that predicate.
- **Business-coupled entrances need a check first.** "Watch a video to earn benefits" may be a real feature. Decide with evidence: find the callers (`scripts/find_refs.py`) and see whether the entrance leads to a **server request** or a **local entitlement write**. A server-issued entitlement cannot be created client-side, so releasing the gate only fools the UI while the server still refuses; an entrance that changes purely local UI state is safe to release.
## Step 3: choose the patch layer
From safest to riskiest (full technique table in `dex-patching.md`):
1. **Kill SDK init** (Step 2) — the SDK never starts; its networks, caches, and plugin downloads never happen. Highest-value single patch. Verify nothing else silently depended on init.
2. **Neutralize the readiness gate** — `isReady`/`canShow` return false; routes existing "no ad" branches.
3. **No-op the show/preload methods** — only when 1 and 2 are insufficient, always preserving callbacks.
4. **Global gate state** — one startup write covers every reader (timestamp / preference keys above).
5. **Filter server-issued ad data at the consumption layer** — drop entries for the target position before they reach UI state.
6. *(avoid)* **Renderer-level suppression** — only if the component is genuinely ad-exclusive. Check `pitfalls.md` P6 first.
7. *(avoid)* **Transport-level blocking** — see `pitfalls.md` P5.
## Step 4: verify — three independent kinds of evidence
Logcat silence alone is weak evidence. A credible "ads are gone" claim needs all three, plus the neighbours:
1. **UI, screen by screen** — splash, home banner, detail page, reader/player, reward button, unlock prompt. Capture before/after screenshots (`verification.md` §screenshot discipline).
2. **Logs** — the SDK's characteristic tags are **entirely absent**, not merely quiet: `anythink`, `ATSDK`, `Pangle`, `TTAd`, `GDT`, `ksad`, and so on.
Make this a **count, not an impression**. Aggregators log a recognisable burst while they probe which networks are wired in (a "SDK id N not integrated" style line, once per network). Capture the same fixed window on the original and on your build and compare numbers: `8 -> 0` is a measurement; "I did not see it" is not. A tag that goes quiet without reaching zero usually means the SDK still starts and simply failed to load an ad this run — a different, weaker result than "the subsystem never started".
3. **Filesystem, the strongest signal** — the app's private directory contains **no ad SDK working directory at all**:
```bash
adb shell "su -c 'ls -la /data/data/<app.package>/files/'"
```
For a dynamic-plugin SDK, an absent `<sdk>_p` tree proves the init gate never ran. It is not an artifact you can fake by hiding a view. If the tree exists, something still initializes the SDK: the init patch did not take effect, or a second init path exists.
Additional cheap evidence:
4. **DNS / socket level** — a runtime `InetAddress` hook. Before: a burst of ad/tracker domains. After: none. Many ad SDKs bypass the system proxy, so a proxy log can miss them while this does not.
5. **Exercise the screens that had ads** and confirm nothing unrelated regressed: images, playback, lists, pagination, downloads, login.
6. **Clean install** — uninstall first, then install. Data left from a previous run can hide a failed init patch (`pitfalls.md` P11).
Frame the result at the right strength: *"the ad was hidden"* vs *"the ad subsystem never started"*. Only evidence 3 and 4 support the second (`verification.md` §the claim ladder).
## Step 5: what is usually NOT removable
Be honest about residual ads rather than breaking the app to chase them:
- **Content promos embedded deep in a feature's own data payload**, where the screen's
load depends on the same request. Distinguish this from a *dedicated config block*:
if the promo arrives in its own `splash` / `banner` / `popup` field with its own
`enabled` flag, it is removable (Step 1, server-driven). It is only genuinely hard
when the same list is both the content and the ad, with no marker separating them.
- **Ads delivered as content** (a sponsored "article" or a native card with no SDK marker)
— indistinguishable from real content without runtime tracing.
- **Ads whose SDK init also enables other features.** Removing init can break
functionality that silently depended on it. Verify before shipping; a working app with
one residual ad beats a broken app.
- **Server-issued entitlements gated behind watching an ad.** The gate can be opened
client-side, the entitlement cannot be created client-side.
- **A purely local brand launch screen.** If all you see is the app's own logo, no
third-party image and no network-sourced content, that is a splash screen, not an ad.
Removing it is a judgement call about the user's stated goal — say which one you
removed rather than silently treating "startup screen" as "ad".
If a residual ad cannot be removed without breaking something, say so, and say exactly
which coupling caused it. That is a better deliverable than a broken APK.

593
vendor-new/apk-reverse/skills/apk-reverse/references/advanced-unpacking.md Normal file
View File

@@ -0,0 +1,593 @@
# Advanced Unpacking — Extraction Shells, Active Invocation, and the VMP Boundary
Load this when a memory dump of a packed app has already landed on your desk and the
dump itself is the problem: the image parses, the classes are there, and the method
bodies are empty. The neighbouring files each own one stage — `packers.md` owns
*identifying* a shell and mapping what it validates, `recon.md` §Unpacking a dex-level
packer owns the whole-image memory dump and its first-pass filtering, and
`dynamic-frida.md` owns the runtime environment those dumps need. This file owns what
comes after: recognizing that a "successful" dump is only a skeleton, recovering the
missing code, and knowing where that recovery honestly stops.
**Strength note, read this first:** the repository's verification target carried no
extraction shell and no VMP, so nothing here was exercised end to end against a live
hardened sample by this skill's own verification pass. The FART/Youpk mechanisms and
the Android 12-16 failure analysis are **inferred** from public work (FART/Youpk
release notes and a 2026-08 Kanxue thread, `thread-292312`, on why classic
active-invocation hooks die on modern ART). The detection metric has since been
**measured against a purpose-built extraction-shell fixture set**, and that pass
invalidated the threshold this file used to state — see the calibration table in
§Measuring extraction instead of guessing and the commands behind it in
`references/evidence-summary.md` §The capability matrix. The older surgical fixture
that first exercised the script is in `references/evidence-summary.md` §The capability matrix.
Label your own results the same way.
Two parts of this file have since been exercised against live hardened targets on a
rooted device. The **root-side memory route** (§Dumping when frida is refused) was run
end to end: 17 ART dex mappings exported from `/proc/<pid>/mem`, page-alignment trim
included, validated with `scripts/dex_dump_validate.py`; the evidence, the device-shell
traps and the throughput numbers are recorded in
`references/evidence-summary.md` §The capability matrix. The `frida-dexdump` refusal that
motivates that section is in `references/evidence-summary.md` §The capability matrix. A later
pass added the route's **negative boundary** — what it yields nothing against, and how
to tell before spending a process window — plus the reproducibility check between read
paths and the layered-descent table; those are measured on the clean MASTG targets and
on a hardened sample, with commands and hashes in
`references/evidence-summary.md` §The capability matrix. The
FART/active-invocation half of the file is still **inferred** — no extraction-shell
target has been spliced end to end here, and the two candidate samples shipped in
`repos/CyReverse` turned out not to be extraction shells at all (their two "shell"
assets differ by six bytes; row B3 in the bench evidence file).
## The four shapes a dump can be
Diagnose before choosing a route. Run `scripts/dex_dump_validate.py` over the dump
directory and read the `stub%` column — the four shapes separate on it.
| The dumped image | `stub%` | Diagnosis | Route |
|---|---|---|---|
| Parses, many classes, real bodies | near-baseline | First-generation landing shell; the whole-dex dump **is** the original | Done — filter (`recon.md` §Unpacking a dex-level packer) and proceed to patching |
| Parses, many classes, most bodies are `return-void` stubs or nop fills | high | **Extraction shell skeleton** — bodies are decrypted only on invocation | FART loop, this file |
| Whole classes appear as bare `native` declarations, and the matching `Java_*` export exists | n/a | **JNI sinking** — the code left the dex for the `.so` | `java2c-and-jni-sinking.md` — read the function in the library, **not** this file |
| Method bodies look like a thin forwarder to a native routine, or a single native island holds the app's own logic | n/a | **Java2C / native island** — the body is a compiled `J`/`.so` function, so re-reading the dex finds only the call | `java2c-and-jni-sinking.md` — the dex is the wrong instrument |
| Whole classes appear as bare `native` declarations, and **no** `Java_*` export exists | n/a | Dynamic-registration native sink — the binding table is built at runtime | `java2c-and-jni-sinking.md` §the symbol search fails structurally |
| Parses, bodies present but instruction streams are nonsense to a dalvik disassembler | baseline | **Real Dex VMP** — private opcodes behind a native interpreter | Last section of this file, and be honest about the cost |
The two `native`-looking rows are **not** one row. They are separated because the
correct next action differs: JNI sinking keeps the logic in a named, exported ARM64
function that you read from the library, while a dynamic-registration sink has no
symbol to find and needs the binding table read instead. Lumping them together sends an
analyst to dump memory for code that is sitting in an ordinary `.so` — the misdiagnosis
`java2c-and-jni-sinking.md` was written to stop.
Two supporting signals worth recording while you are here (the second is measured on
this repo's hardened sample): a shipped shell `classes.dex` can be megabytes in size
yet define only a handful of classes — the bytes are payload, not code, so class count
and file size move independently under a packer. And a dump whose signature field
disagrees with its own contents while the checksum agrees is not necessarily corrupt;
a shell that never verifies its own header can ship it that way, so treat
checksum/signature results from `verify_dex_header` as evidence about *who touched the
image*, not as a pass/fail gate on usability.
## Measuring extraction instead of guessing
The metric that separates shape 2 from shape 1 is the **trivial-body ratio**: among
methods that *have* a `code_item` (`code_off != 0`), the fraction whose instruction
stream, with plain `nop` units skipped, contains nothing but a single `return*`.
`code_off == 0` methods are abstract or native declarations — that absence is normal
and is excluded from the ratio. A method with a `code_item`, real `insns_size`, and a
lone `return-void` inside is what an extraction shell leaves behind.
```bash
python scripts/dex_dump_validate.py <dump_dir> --find 'Lcom/example/app/'
```
The script dedupes by sha256 (never by size — same-size dumps of one dex have been
measured to differ in ~73% of their bytes), rejects structurally impossible images
(bad magic, `file_size` mismatch, ids out of range), counts classes and stub bodies,
and ranks which surviving image is the most likely original.
**Calibration — measured, and it is not a threshold.** Do not look for a `stub%` number
to compare against. On a real 982 KB application dex (5,061 bodies) every emptying
shape the shell family uses was built into a fixture and measured against a
zero-change control (`references/evidence-summary.md` §The capability matrix,
which carries the commands and the full matrix):
| What the shell left behind | `stub%` | vs. control (1.9 %) |
|---|---|---|
| bare `return-void` in every body | 100.0 % | sighted |
| body cut to `const/4 v0,#0; return v0` | 93.2 % | sighted |
| nop fill, no return — **the slot-clearing shape** | 0.0 % | **blind, and scores below the untouched original** |
| `new RuntimeException; throw` stub — **the 360/legu shape** | 1.9 % | **blind, indistinguishable from the control** |
| 25 / 50 / 75 % of bodies emptied | 1.7 / 1.7 / 2.1 % | blind |
The distribution is **bimodal**: `stub%` lands either at the app's own baseline
(ordinary constructors and interface stubs, measured at 1.9 % here) or at ~100 %.
Nothing sits in between, so there is no band a threshold could live in, and **partial
extraction is invisible** — a dex with three quarters of its bodies removed reported
0.2 points above its untouched control.
Two consequences the metric's name hides:
- **`stub%` high is dispositive; `stub%` low is not evidence of anything.** A low
reading is compatible with a genuine dex, a fully nop-wiped skeleton, and a
`throw`-stubbed skeleton alike.
- **Read `emptied%`, not `stub%`, for the skeleton call.** `emptied%` unions the
`return*`-stub and the nop-wiped classes, which is what lifts the nop-filled
skeleton from 0.0 % to 100 %. A `throw`-stubbed skeleton still reads low; for that
shape the ranking cannot help you and only per-body inspection can.
The trivial-body shapes `dex_dump_validate.py` counts, and which of them are skeleton
evidence, are documented in the script itself (`classify_body`); the split between
`stub` / `erased` / `minimal` exists because collapsing them produced a ranking that
named a fully-wiped image the most likely original.
### The ranking's failure conditions — read these before trusting its winner
The `ranking` block names a "best-supported candidate for the original", and it is
wrong in two measured situations. Both were reproduced, and both are the *opposite* of
the obvious reading:
- **A partially extracted image outranks a heavily stubbed one.** Ranking on the
emptied share alone put a dex with 25 % of its bodies removed *ahead of* its own
untouched control — 1.7 % against 1.9 %, because replacing a body with
`const/4 + return` lowers the `return-void` count. Measured before it was fixed;
the current key uses the count of skeleton-shaped bodies first, which is monotone,
and evicts anything over 50 % emptied.
- **A `throw`-stub skeleton is invisible and ranks at the control's own baseline.**
Every body replaced by `new RuntimeException; throw` measured 1.9 % — identical to
the untouched image — and it still sorts third in the fixture set. No body statistic
in this kit sees that shape. If the target family is 360/legu-like, **do not use the
ranking to pick a baseline**; diff candidate images against each other by body hash
instead.
The reason to state this in the reference file rather than only in the script: the
ranking's output is a *sentence with a winner in it*, and a wrong winner is worse than
no winner — an agent that adopts a modified image as its baseline builds every
subsequent diff on the wrong artifact. When `emptied%` is at the app's own baseline and
you cannot name a signal that separates the candidates, treat the ranking as
uninformative for that set. Commands and full matrix:
`references/evidence-summary.md` §The capability matrix.
## The FART loop: skeleton, invocation, splice
Extraction shells decrypt a method body only when that method first executes. A dump
taken at any fixed moment therefore holds whatever the app happened to invoke before
you dumped — mostly stubs. Waiting for the UI to wander into every method is not a
plan (thousands of methods, many on paths a user never triggers). The FART family of
tools closes the loop with three steps, **inferred** (not run by this repo against a
live extraction shell):
1. **Dump the whole image.** Class definitions, field and method signatures, string
tables — the skeleton is complete even when every body is a stub. This is the
ordinary `recon.md` dump; it gives you the address map for step 3.
2. **Actively invoke every method.** Walk the dumped class list reflectively and call
each method (dummy arguments; most will throw, which is fine — the point is that
the *shell* decrypts and installs the body as the method is entered). Hook inside
ART at the interpreter's entry so that, as each method begins executing, the
method's `code_item` — now holding the decrypted body — is written to disk,
tagged with its `method_idx`. These per-method records are the "bin" files of a
FART dump.
3. **Splice the bodies back into the skeleton.** For each record, overwrite the
stub body in the whole-dex image at the offset the skeleton gives you, fix the
header (`scripts/dexutil.py` `fix_dex_header` — signature first, checksum last),
and **accept nothing until jadx or baksmali parses the result** and the method
count matches the skeleton. "It assembled" is not "it repaired".
## The code_item length trap
Splicing fails in a specific place often enough to deserve its own section: the
computed length of a `code_item`. The full layout is
```
registers u16, ins u16, outs u16, tries u16, -- 8 B
debug_info_off u32, insns_size u32, -- 8 B (header = 16 B)
insns[insns_size] u2, -- the instruction stream
[padding u2] -- ONLY if tries_size > 0 AND insns_size is odd
try_item[tries_size] { start_addr u32, insn_count u16, handler_off u16 }, -- 8 B each
encoded_catch_handler_list { uleb size; per handler: sleb offs... } -- variable
```
Two segments are routinely dropped: the odd-`insns_size` padding before `tries`, and
the whole `encoded_catch_handler_list`, which is LEB128-variable and sits *after* the
try table. Drop either and the splice boundary slides — the rebuilt method swallows
bytes from its neighbour or leaves the handler table pointing past the image, and the
failure surfaces later as a dex that half-parses. Walk the handler list with
`read_uleb` to its true end (`scripts/dexutil.py`); do not estimate it. This trap is
**inferred** from repair-tool post-mortems, and it is the first place to look when a
spliced dex parses worse than the skeleton did.
## Why the classic hooks die on Android 12-16
The original FART-era hooks assumed method entry flows through
`art::interpreter::Execute`. On Android 12-16 that assumption fails for five
independent reasons (**inferred**, from the Kanxue `thread-292312` analysis, 2026-08).
Any one of them produces the same symptom — the dumper runs, logs everything, and the
output directory stays thin:
1. **Invoke dispatch splits away from `Execute`.** A quick-entry call goes to Nterp,
AOT or JIT code, or a bridge — none of them pass through the old `Execute` hook.
Forcing the interpreter is not a fix either: the entry point then switches to
Nterp, which *also* bypasses it. *Repair:* move the dump point up to
`EnterInterpreterFromInvoke`, which sits ahead of the Nterp/interpreter split, and
keep fallback points (special invoke branches, ClassLinker load paths, DexFile
open, Frida ClassLoader enumeration) for the methods that still slip through.
2. **`ArtMethod` layout drift.** Field offsets and pointer-sized members changed
across releases; a hardcoded offset reads the wrong field and the dump silently
mislabels everything after it. *Repair:* derive offsets at runtime (probe
`access_flags` / `quickCode` / `jniCode` on a known method) instead of compiling
them in.
3. **Scoped Storage breaks the output quietly.** Writes to `/sdcard` fail, the log
reads like a completed run, and the directory is empty. *Repair:* write under
`/sdcard/Android/data/<pkg>/files/` (or app-private storage) and **check the
`open()` return value** — the silent variant of this failure wastes a whole round.
4. **The shell fights back.** Junk classes that exit inside their initializer,
detection of reflective enumeration, deliberate dex-header corruption in memory,
decryption deferred to the true execution point, fingerprinting of known dumper
thread names and paths. *Repair:* randomize your thread names and output paths,
filter junk classes (few methods, throwaway names) before invoking, and expect the
counter-measure list to differ per vendor.
5. **Compilation policy shifts the entry under you.** A just-installed app and one
that has run for a while present different entry mixes (profile-guided
compilation, Mainline module updates). *Repair:* re-probe the entry type every
run; never cache yesterday's hook decision.
The debugging order that isolates all five in the fewest rounds: print the entry and
flags for a known method → force the interpreter → observe what the entry became →
place the dump point accordingly → force only the target package (`ignore` lists for
`androidx.*`, `com.google.*`, `kotlin.*` — forcing the framework too freezes or kills
the app) → compare a method's `code_item` before and after invocation → splice →
verify with jadx/baksmali.
## Youpk in one paragraph
Youpk is the ROM-level descendant of the same idea (**inferred**): a modified ART
performs the active invocation *inside* the runtime at class-link and interpret time,
so the reflective caller script, the entry-point guessing, and most shell
counter-measures against host-side tooling drop out — at the cost of building and
flashing a customized runtime per Android version. Choose it when the target's
anti-instrumentation defeats host-side loops entirely, and you control the device.
## The frida-dexdump route and where it stops
`frida-dexdump` walks the live ClassLoaders and writes every dex-shaped memory range.
Environment setup and its two time sinks (the `-o` directory must pre-exist, and
`-D` does not take remote devices) are recorded in `recon.md` §Unpacking a dex-level
packer; the Frida environment itself is `dynamic-frida.md`. What it hands you is a
pile of images of the four shapes above — dedupe, reject and rank them with
`scripts/dex_dump_validate.py` before touching any by hand. Its hard limit: against
an extraction shell it can only ever produce the **skeleton**, because the missing
bodies exist nowhere in memory until invoked. No amount of re-dumping fixes that;
only the invocation loop recovers them.
## Dumping when frida is refused
A hardened target can refuse instrumentation outright: the dump tool dies with
`script has been destroyed` and the process is gone before the error can be read.
That is a *protocol* refusal, not a timeout, and re-running the same attach with a
different flag is the wrong response — twice-identical failure on one variable is a
stop signal, not a hint to try a third flag. What remains is the route that never
touches the process's instrumentation interface at all: **root reads the target's own
memory** through `/proc/<pid>/maps` and `/proc/<pid>/mem`. No agent, no ptrace
attach, no server process for the shell to find.
**Read the signals carefully, because one of them lies.** The refusal presents as
(a) the dump tool's attach failing with `script has been destroyed` while the app is
otherwise healthy, and (b) pid churn. But **pid churn alone is not evidence that
frida was detected.** A 360-jiagu sample on this repository's test device was
measured exiting every 7-18 s and being restarted by the platform, with *no*
instrumentation process anywhere on the device, logging
`Process <pkg> (pid N) has died: fg TOP` and no crash, tombstone or ANR record — a
clean self-exit, i.e. the packer's own environment check. Attribution needs the
control run, and the control run needs a window longer than one restart period: a
12-second observation lands inside a 17-second cycle and reports "stable" about a
process that is not. If the sample self-exits on that cadence, every step below must
fit inside a single lifetime.
### Read the map before choosing the bytes
```bash
adb shell 'su -c "pidof <PKG>"' # may return nothing mid-restart
adb shell 'su -c "ps -A -o PID,NAME | grep -w <PKG>"' # more robust: matches either way
adb shell 'su -c "cat /proc/<pid>/maps"' > maps.txt
```
ART names its in-memory dex mappings for you — this is the cheapest win on the whole
route, because a mapping called
`[anon:dalvik-classes*.dex extracted in memory from <src>]` *is* a dex image, already
located, already sized, no search required:
```
7129d8c000-712a60f000 r--p [anon:dalvik-classes.dex extracted in memory from /data/app/~~…/base.apk]
```
Copy those ranges first. Everything else — anonymous `rw-p`, `libc_malloc`, `.bss` —
is a *search* target, and searching is the expensive half.
### Address arithmetic is where this route silently breaks
Three device-shell traps cost real time on the first run, and each one produces a
plausible-looking result rather than an error:
| Trap | Symptom | Fix |
|---|---|---|
| mksh arithmetic is 32-bit | `$((16#7129d9a000))` yields `702128128` (the low 32 bits) — `skip=` then seeks to the wrong address and the dump looks like zeros | convert hex to decimal with `awk` (double precision handles 48-bit addresses exactly) |
| A VMA is page aligned | the dump is 0..4095 bytes longer than the image; a validator rejects it with `file_size=8923752 actual=8925184` (17 of 17 files on the measured run) | trim to the `file_size` at dex header offset 32 before validating |
| `grep` is line oriented | the magic contains a `0A`, so no line-oriented pattern can ever match `dex\n035\0`; `-F` with a real newline in the pattern fails too | `grep -z -a -o -b -E 'dex.035'` — with `-z`, records are NUL separated and `.` matches the newline byte; the reported offset is stream-relative, add the segment start |
One more that only bites in scripts: **toybox `awk` cannot be trusted with
`/regex/ && $0 !~ /…/`** — a filter written that way selected all 808 map lines,
including `r--p` system libraries. Filter the map with `grep` and keep `awk` for
arithmetic only.
### Export, then prove it is a dex
```bash
# inside one lifetime of the process: export a candidate range
dd if=/proc/<pid>/mem iflag=skip_bytes,count_bytes skip=<DEC_START> count=<SIZE> \
of=/data/local/tmp/seg_<n>.bin bs=256k
adb pull /data/local/tmp/seg_<n>.bin <dumpdir>/<n>.bin # trim first, see above
# or, without writing to the device at all: search a range in place
dd if=/proc/<pid>/mem iflag=skip_bytes,count_bytes skip=<DEC_START> count=<SIZE> bs=1M \
| grep -z -a -o -b -E 'dex.035' | head -5
```
Then the same acceptance gate the frida route uses: `scripts/dex_dump_validate.py`
over the dump directory — sha256 dedupe, header and `file_size` validation, class
counts, `stub%`, ranking. Rename to `*.dex` first; the validator selects on extension.
Pass `--trim` when the captures come straight from VMAs: a page-aligned region is always
longer than the image it holds, so without it every file is rejected on
`file_size=… actual=…` — 17 of 17 on the measured run, all of them recoverable.
When you do **not** already know which region holds the payload —
`scripts/dex_mem_scan.py <region_dir> --dump <out_dir>` does the search and the cut in one
step: it scans the captures for the magic in bounded chunks, reads each hit's own header,
and writes exactly `file_size` bytes from there. That is the anonymous-mapping case, where
no `maps` entry names the buffer. The division of labour matches the rest of this kit:
that script **finds and cuts**, `dex_dump_validate.py` **judges**. On the measured device
the scan of 142 anonymous regions returned zero hits while the named ART mappings returned
17 images — a negative from a cheap scan is worth having before committing to a heap sweep.
Measured on the test device, this route **works**: 17 ART dex mappings exported in
one pass, all 17 parsing after the page-alignment trim, `stub%` 0.9-3.2 % (real
bodies, SDK plugins), and one of them — the mapping that came from the app's own
`base.apk` — defining exactly **4 classes** on 8.9 MB. That last image is the landing
shell skeleton, and the class count is what says so; the size says nothing. Note what
the route did *not* produce: no second application dex ever appeared in the map
across 22 s of sampling, so the packer's real payload was not sitting in memory as a
loaded ART dex while the observer watched — which is the honest boundary of every
`/proc/<pid>/mem` dump, not a defect of the tooling.
### Limits to budget for
- **The process must stay alive and keep the same pid.** A VMA map and a page table
both belong to a live process, and every `dd` after the restart reads the wrong
address space or nothing. On a sample that self-exits every 7-18 s, a dump routine
must be atomic within one window and re-entrant (re-read `maps` after every
restart) rather than a long batch.
- **Throughput is the binding constraint, not correctness.** Reading
`/proc/<pid>/mem` through `grep -z -E` measured about **10 MB/s** on this device.
A 1 GB ART heap is therefore a ~100 s scan — several lifetimes — so scanning the
Java heap for a decrypted buffer is the *last* resort, not the first. Named ART dex
mappings are free; a heap sweep is not.
- **`/proc/<pid>/map_files/` is not a shortcut on this ROM.** The directory exists
and is root-owned, but its entries are absent/unreadable, so `dd
if=/proc/<pid>/map_files/<start>-<end>` returns a few dozen bytes of error text.
Use the `mem` + decimal-offset form.
- **Memcpy dumps of `.so` files are not the `.so` files.** A range lifted from
`/proc/<pid>/mem` usually carries `PT_LOAD` contents and no section header table;
a real ELF on disk has both. That difference is how to tell a *disk copy* of a
library from a *memory-extracted* one — and it is why a leftover `.so` in a work
directory must be matched against the APK it claims to come from before it is used
as evidence.
- **In-memory dex bodies can be non-contiguous.** A decrypted image stitched across
two VMAs (or a payload the packer splits) will not be recovered by dumping ranges
independently; the `code_off` cross-check and the class-count sanity check in
§Measuring extraction are what catch that, not the magic search.
### What this route cannot do, and how to tell before you spend the window
The sections above are the how. This is the boundary, and it is worth reading *before*
the first export, because the failure is silent: you export a region, it validates as
something else, and the conclusion "the payload is not resident" is drawn from a wrong
capture. Two measured obstacles and one measured non-obstacle.
**A dex that is still compressed never materialises as an image.** `dexopt` can only
map a dex directly when the archive stores it uncompressed, so for an APK whose
`classes.dex` is stored with **deflate** the method never applies — and nothing in
memory is a whole dex. Measured on the clean MASTG targets (`owasp.mstg.uncrackable1`,
`owasp.mstg.uncrackable3`, both with `classes.dex` at `compress_type=8` in the
*on-device* APK): **zero** `[anon:dalvik-classes.dex extracted in memory from …]`
mappings across ~2,400 map lines each, i.e. `named_dex=0`. There was nothing to
export, and the route's negative says nothing about those targets being protected.
**An `r--s` view of `base.apk` is not a source of dex bytes.** Both targets showed
exactly that shape — a read-only shared mapping of the APK — and it is the shape that
looks like an answer and is not one. Exported and compared byte-for-byte with the same
offset in the file:
| View | Compared with | Result |
|---|---|---|
| `76dfb14000-76dfb15000` (L1, file offset `0x11000`, 4 KiB) | the APK at `0x11000` | **identical** (sha256 `74846b8c…` both) — it is the APK's own bytes, and at that offset they are the ZIP *central directory* (`PK\x01\x02`), not a dex |
| `765a89c000-765a8de000` (L3, file offset `0x6000`, 264 KiB) | the APK at `0x6000` | **identical** (sha256 `2eafd7cc…` both) |
| `76dc28e000-76dc299000` (L3, file offset `0x15a000`, 44 KiB) | the APK at `0x15a000` | **different** — and the reason is not tampering: the mapping range runs *past the end of the file* (offset `0x15a000` > 1,460,555 B), so the file-side read is short and the two hashes describe different byte counts |
Two things follow, and the second one generalises. First, **the two read paths agree
exactly whenever they cover the same bytes** — `/proc/<pid>/mem` and a straight file
read returned identical sha256 on both full-length comparisons, which is the
reproducibility check worth running before treating any capture as evidence. Second,
**when they disagree, compute the length before suspecting the target**: a VMA can
legitimately extend beyond its backing file, and "the bytes differ" then means "you
compared different amounts of data", not "something rewrote the file".
**The non-obstacle, measured:** when the instrumentation route is refused, nothing here
needs fixing — root reads `/proc/<pid>/mem` with no `ptrace` and no agent, and on the
test device that produced 17 real, parseable dex images from a hardened process
(`references/evidence-summary.md` §The capability matrix). The correction this pass adds is
narrower than that claim: **the route's yield is a property of the target, not of the
route.** It yields everything the packer leaves as a whole image; it yields nothing
against a target whose dex never becomes one.
### Is this dex image real? Two independent readers, or a claim you have not earned
The route above produces candidate images; something else has to judge them, and the
judgement has to come from a reader you did not write. A single reading of a hard problem
is not evidence, and this file already carries the lesson in the opposite direction —
`§Establish "the bodies do not decode" with a decoder you did not write` is the case where
a hand-written decoder *invented* a finding.
Three checks, in increasing cost:
1. **Two read paths over the same bytes must agree byte-for-byte.** `/proc/<pid>/mem` at
`(start, start+size)` versus the file or a second export. Measured above: identical
sha256 on every comparison that covered the same length. Do this **before**
interpretation — a capture that cannot be reproduced is not a capture.
2. **A producer you did not write must parse it.** `dexdump`
(`E:\tools\android-14\dexdump.exe`), `baksmali` or `jadx` — and quote *its* failure
count, not your own parser's. `scripts/dex_dump_validate.py` ranks and screens
candidates; it does not decide whether a body is real dalvik.
3. **Only then interpret** `stub%` / `emptied%` / class counts, per
§Measuring extraction instead of guessing.
**Cross-checking two dumpers is only meaningful when both produce the same artifact.** A
byte-identity comparison is a real check between two implementations reading the same
address space (`/proc/<pid>/mem` versus `process_vm_readv(2)`, or two independent export
tools); it is an empty check between a whole-image dump and a per-method `code_item`
harvest, because those are different artifacts and a byte difference says nothing about
either. State which kind of comparison you ran. A *cross-tool* agreement also does not
prove recovery: on a shell that stubs bodies, two agreeing dumps are two copies of the
same skeleton — which is why check 2's parse count and §Measuring extraction's ratios
both have to pass.
## Layered descent: Java → JNI → Native → libc → syscall
**Read this when an upper layer has gone silent.** "My hook produced no events" is not a
finding about the target until you know which layer is capable of seeing the code that
runs. The upper layers are cheaper and more informative, so the order is deliberate; the
mistake is skipping down for reassurance instead of on a signal.
| Layer | What it sees | What it cannot see | The signal to descend |
|---|---|---|---|
| **Java** (framework hooks) | methods, objects, arguments — the algorithm in its readable form | anything the app does in a library; anything the framework compiled out of reach | no invocation ever fires, or only the first call and then nothing |
| **JNI boundary** | argument and return values of a *named* export; the boundary itself | argument *meaning*; any logic past the call | outcomes are correct but you cannot see why |
| **Native, named exports** (`Module.findExportByName`) | entries of exported functions; the call site that reached them | the function body's control flow (that is `native-dbi-and-deobfuscation.md`) | correct answers, opaque body; a `.so` that never calls its own exports |
| **Native, internal** (module base + offset) | code with no symbol at all | how you found the offset in the first place | a library whose exports are a thin shell over internal work |
| **libc** (`open`, `read`, `mmap`, `strstr`, `pthread_create`, `exit`, …) | every call that *goes through* libc, with caller attribution | calls that do not; anything above libc | an event count that contradicts observed effects |
| **`svc #0` sites** | nothing at runtime until you patch the site — it is a *static* layer | kernel-side: unreachable here, see the gate below | libc shows no such syscall while the effect happens |
Three rules that keep the descent honest:
- **Descending is cheap to state and expensive to trust.** A lower layer has less
context, so an event there needs *more* support to mean something: `open()` fired says
nothing on its own, while `open()` with a caller of `libX.so+0x1cef8` names a detector.
Always carry the caller offset down, or you have traded meaning for volume.
- **The bottom is static, not dynamic.** Below libc there is nothing to hook — a
`svc #0` site is found by scanning the library (`scripts/svc_scan.py`) and changed by
patching bytes. Measured: the ROM's `libc.so` carries exactly **4** termination sites
(`exit`, `exit_group`, `kill`, `tgkill`) and those are libc's own exported
implementations, which is *why* a libc hook sees callers at all; the same scan on a
shell library returned 21 byte-scan "sites" that were all data (no syscall-number load
near any of them). Read the neighbours before believing a count.
- **"Down" does not mean "more capable".** Measured on a real target, the decisive
detection ran at the **libc** layer — `strstr("frida")` at ~300 ms of process life,
followed within milliseconds by a clean self-exit with no tombstone — while the code
that recovered the caller chain sat in **JIT-compiled** runtime output, i.e. a
*higher* layer that has no file offset. The layers are not a ladder of access; they are
different observation points, and the useful one is wherever the target's check
happens to be.
**The kernel layer is not reachable on this device, and the gate is a kernel version.**
eBPF tracing (`stackplz`, kprobes, `seccomp` filters pushed from userspace) requires
kernel **5.10+**; the measured device is **4.14.186**, so that entire layer is closed
here regardless of tooling. `kernel-and-environment-hardening.md` maps the route and its
version gate; do not spend a session on the layer before checking `adb shell uname -r`.
The consequence for this file: the descent above stops at the `svc` boundary, and a check
that runs *inside* the kernel is out of reach of both dynamic and static userspace work.
## The honest boundary: real Dex VMP
The last shape is the one to say out loud: if the dump's method bodies are intact but
decode as private opcodes — rare-opcode density off the charts, unknown opcode slots,
and a large native interpreter loop behind them — the code has been virtualized, and
nothing in this repository will hand it back to you decompiled. Distinguish it from
the JNI sink first (`code-virtualization-and-custom-linkers.md`,
`java2c-and-jni-sinking.md`): there the methods are `native` stubs and the logic lives
as native code; here the bodies are present and *re-encoded*.
### Establish "the bodies do not decode" with a decoder you did not write
This is the one measurement in this file that has already been got wrong here, so it
gets its own rule. A hand-written opcode-width table silently desynchronises on large
dex files — one arm64 branch opcode missing from the table is enough — and the
desynchronisation is indistinguishable from the finding it fakes. Measured on the
`ezAndroid` sample: a workbench table reported **1,393 of 22,424 bodies as
non-decodable** and I read it as private opcodes; the platform decoder returned
**943,223 lines, 34,566 instruction decodes, and zero structural errors** on the same
file (`references/evidence-summary.md` §The capability matrix). The bodies
were ordinary dalvik the whole time, and one of the "desync" methods disassembles into
five clean instructions under `dexdump`.
So: **before any VMP verdict, decode the same file with `dexdump`,
`baksmali`/`smali`, or `jadx`, and quote the failure count that came from *it*.** A
rare-opcode histogram is a lead, not evidence. Never let a private table's desync
inform a classification — "my decoder stopped" is a fact about the decoder.
### What a static dex probe can and cannot decide
Two different questions live in the same dump, and only the first one is answerable
from the bytes on disk:
| Question | Statically decidable? | Why |
|---|---|---|
| Are the bodies real dalvik, or an encoded program? | **yes** | the platform decoder decodes them or reports a structural error |
| Did the code leave the dex for native? | **yes** | `native` access flags, then `Java_*` exports / `JNI_OnLoad` in the matching `.so` |
| Does a native island check signature, root or debugger? | **no** | the logic is ARM64 behind a vtable; the check may exist in no dex at all |
| Are unlabelled `assets/` blobs live payload? | **no** | an unreferenced blob is dead until a runtime path constructs its name — measured: six such files in `ezAndroid`'s `assets/`, **zero** dex strings referencing them |
The trap is the asymmetry. A clean static reading licenses exactly one sentence —
*"this is not dex-VMP"* — and it does **not** license *"this is not hardened"*. Those
are different claims: the second one is the misdiagnosis that sends an analyst looking
for an unpacker when the problem is a native island, or for a private disassembler when
the code is ordinary ARM64 in a 206 KB `.so`. A static probe's silence about hardening
is silence, not a clean bill of health.
What recovery looks like when someone does it, **all inferred**, no fixture exists in
this repo, treat as a research plan rather than a recipe:
1. **Differential hardening as a black-box oracle.** Build sample APKs containing
every dalvik opcode in known, labeled sequences; submit them to the *same*
hardening platform as the target; dump the returned dexes; align each original
method with its hardened counterpart and read off the opcode substitution table.
2. **Handler map recovery.** Locate the native interpreter's dispatch table; each
handler implements one virtual opcode; the differential table names them.
3. **A private disassembler.** Feed the substitution table into an existing dalvik
decode layer (`scripts/dexutil.py`'s format table is the shape of the thing) and
re-disassemble the bodies; decompilation is then ordinary tooling on the output.
Budget honestly: this is weeks, not an afternoon, and it redone per vendor per
version. The cheaper question is usually whether the behaviour you need survives
outside the virtualized island — a hook at the method's Java boundary
(`dynamic-frida.md`) or a server-side answer often reaches the goal without paying
for the island itself.
## Decision summary
| Observation | Action | Where |
|---|---|---|
| Dump parses, stub% near baseline | Filter, verify with jadx, move on to patching | `recon.md` |
| stub% high, image parses | FART loop: skeleton + active invocation + splice | this file |
| `emptied%` high but `stub%` low | nop-wiped skeleton; the old `stub%`-only reading called it the original | §Measuring extraction instead of guessing |
| The ranking names a winner but `emptied%` sits at the app's baseline | Treat the ranking as uninformative for this set; do not adopt that image as a patch baseline | §The ranking's failure conditions |
| Splice produces a half-parsing dex | Re-check code_item length (padding, handler list) | §The code_item length trap |
| Dumper runs, output dir empty | Storage path + `open()` check, then entry-type probe | §Why the classic hooks die on Android 12-16 |
| App dies under the invoker | Shell counter-measures; narrow the force list | §Why the classic hooks die on Android 12-16, and `detection-and-anti-analysis.md` |
| frida attach refused (`script has been destroyed`) | Root-side `/proc/<pid>/mem` dump, atomic inside one process lifetime | §Dumping when frida is refused |
| Target has no `[anon:dalvik-classes…]` mapping at all (`named_dex=0`) | Its dex is compressed and never becomes a whole image — the route has nothing to export here; this is not evidence of protection | §What this route cannot do, and how to tell before you spend the window |
| An `r--s` view of `base.apk` looks like a dex region | Compare it with the file at the same offset and check the length: it is the APK's own bytes (often the ZIP central directory), and a range may exceed the file | §What this route cannot do, and how to tell before you spend the window |
| Two captures of the same address differ | Compare byte *lengths* first; a VMA can extend past its backing file | §What this route cannot do, and how to tell before you spend the window |
| A hook on the upper layer stops firing | Descend one layer on a named signal, not for reassurance, and carry the caller offset down | §Layered descent: Java → JNI → Native → libc → syscall |
| The effect happens but libc shows no such syscall | Static: scan the library for an inline `svc` site before assuming evasion | §Layered descent: Java → JNI → Native → libc → syscall |
| Sample self-exits and restarts with no instrumenter present | The packer's own check, not your hook; shorten every step to one lifetime | §Dumping when frida is refused |
| Methods are `native` stubs with a `Java_*` export | JNI sinking — read the function in the `.so` | `java2c-and-jni-sinking.md` |
| Bodies present, decode as nonsense | Real VMP — **but confirm with `dexdump` first**, then differential oracle or walk away | §Establish "the bodies do not decode" with a decoder you did not write |

194
vendor-new/apk-reverse/skills/apk-reverse/references/byte-level-patching.md Normal file
View File

@@ -0,0 +1,194 @@
# Byte-level dex patching — equal-length edits, and the traps that make them fail
`dex-patching.md` covers rewriting a **method body** with dexlib2. This file covers the other
technique: rewriting **a few bytes in place**. Reach for it whenever the change can be expressed as
"make this branch do the other thing" or "make this constant a different constant", because it is
strictly safer than rebuilding a method — but only if you respect the constraints below, which are
not optional and are not obvious.
**Load this when:** the change fits in an existing instruction slot or constant, and you want nothing to move (no offset, try/catch or debug pointer invalidated). It gives the equal-length edit, its legality rules, and where it silently fails.
## Why equal-length, in numbers
A method-level rebuild is not free. Measured on one R8-processed sample (4.32 MB single dex):
| | original | after a 2-method dexlib2 rewrite |
|---|---|---|
| dex size | 4.32 MB | **7.73 MB** |
| `debug_info` table | 924 bytes | **22,828 bytes** |
| class set / access flags | — | identical (zero drift) |
The class table is unchanged, so the rebuild is *correct* — but dexlib2 does not reuse R8's shared
`debug_info` entries and the file grows ~80%. That is the real cost of "surgical" method rewriting,
and it matters because it changes the byte layout of everything after the edited methods.
An equal-length edit changes **only the bytes you chose**. No offset moves, so no try/catch block,
no debug-info pointer, and no branch displacement anywhere in the file can be invalidated by
construction. That property is worth more than the convenience of writing smali.
**Decision rule:** if the change fits in an existing instruction slot or a constant, patch bytes. If
it genuinely needs new instructions or a different register allocation, rebuild the method — and do
it in a copy so you can diff the tables.
## Locating the exact byte offset of an instruction
This is the part that eats hours if you improvise. Three approaches, worst to best.
**Do not reconstruct offsets from a `.line` listing.** baksmali emits one `.line` per *source* line,
and one source line can be attributed to several consecutive instructions. The `.line` values
therefore repeat and do not map 1:1 to offsets. A listing walker built on `.line` silently produces
zero candidates for a branch you can see with your eyes.
**Do not match on a guessed byte sequence.** Instruction encodings vary with register numbers and
operand widths; a pattern that looks right (`38 xx`) can be a different instruction than you think
(`0x38` is `if-test`, 22t and 4 bytes — **not** `if-testz`).
**Do this instead: walk the dex structure to the method, then decode.** Resolve class → `class_data_item`
→ method (`code_off`) → `code_item` → instruction stream, and decode forward with a complete format
table. Then match on *decoded semantics* ("an `iget-boolean p1` immediately followed by `if-nez p1`
whose target reads field `X`") rather than on bytes. `scripts/dex_find_insn.py` does this; the
decoder in `scripts/dexutil.py` is the reusable part.
Three details in the walk that silently produce wrong answers:
- **`class_data_item` member indices are *diffs*.** `field_idx`, `method_idx` are encoded as deltas
against the previous entry in the same list and must be accumulated. Reading a raw uleb as an
absolute index gives you real-looking wrong members — you will "find" a method that is not the one
you asked for and not notice.
- **Payload `nop`s are not `nop`s.** dalvik encodes switch/array payloads as `00 <ident> <size>` with
`ident` in 1..3. A plain `00 00` is an ordinary one-unit `nop`. Treating every `00` as a payload
swallows the next instruction and desynchronises the whole rest of the method.
- **The header field order is not what you remember.** After `magic(8) + checksum(4) + signature(20)`,
the offsets are `file_size 0x20, header_size 0x24, endian 0x28, link_size 0x2C, link_off 0x30,
map_off 0x34, string_ids_size 0x38, string_ids_off 0x3C, type_ids 0x40/0x44, proto_ids 0x48/0x4C,
field_ids 0x50/0x54, method_ids 0x58/0x5C, class_defs 0x60/0x64, data 0x68/0x6C`.
Print the parsed sizes and compare against the file before trusting anything downstream.
## Instruction format table (the part that must be right)
An operand width wrong by one byte desynchronises the decode from that point on, and the failure looks
like "the instruction I want does not exist". These are the groups that matter in practice:
| opcode range | format | code units |
|---|---|---|
| `01 04 07 0A 0B 0C 0D 0E 0F 10 11 12` | 12x / 11x / 11n / 10x | 1 |
| `13 15 16 17 19 1B 20 21 23 24` | 21s/21h/21c/22c/35c/3rc | 2 |
| `14 18 1A 22 25 27 28 29 2A 2B` | 31i/31c/31t/30t/20t/switch | 3 |
| `02 05 08` | 22x | 2 |
| `03 06 09` | 32x | 3 |
| `2C`–`31` | 23x | 2 |
| **`32`–`37`** | **22t if-test** (2 regs + int16) | **2** |
| **`38`–`3D`** | **21t if-testz** (1 reg + int16) | **2** |
| `44`–`51` | 23x aget/aput | 2 |
| `52`–`5F` | 22c iget/iput | 2 |
| `60`–`6D` | 21c sget/sput | 2 |
| `6E`–`72` / `74`–`78` | 35c / 3rc invoke | 3 |
| `7B`–`8F` | 12x unop | 1 |
| `90`–`AF` | 23x binop | 2 |
| `B0`–`CF` | 12x binop/2addr | 1 |
| `D0`–`D7` | 22s binop/lit16 | 2 |
| `D8`–`E2` | 22b binop/lit8 | 2 |
**Self-check that costs nothing:** decode the whole method and assert you land **exactly** on
`insns_off + insns_size*2`. If you overshoot or undershoot, the table is wrong somewhere and every
offset you derived is suspect. Also scan the decoded register numbers: a method whose `registers`
count is 12 cannot use `v13`, so any `v13+` in the output is proof of desync.
**Long conditional branches are two instructions.** A conditional jump has a 16-bit displacement, but
when you see `if-nez vX, :label` in smali and `:label` is far away, the assembler may have emitted
`if-*` followed by a separate `goto`. When hunting a branch, do not assume the pattern is
"branch → target"; confirm by decoding.
## Neutralise a branch, do not redirect it
Once you have decided which side of a condition you want, the safest edit is to **remove the branch**,
not to point it somewhere else.
| Edit | When it is right | Risk |
|---|---|---|
| `if-*` → `nop` pair (**preferred**) | You want the **fall-through** path | None. No new control-flow edge is created |
| `if-*` → `goto` (+ same displacement) | You want the **branch-taken** path | Introduces an edge. It may land on a `move-result*`, which the verifier rejects |
| flip the sense (`if-nez` → `if-eqz`) | Polarity is the only problem | Same edge as above, different destination |
The `move-result` rule: `move-result*` must be **immediately preceded by its producing invoke** in the
instruction stream. A branch whose target is a `move-result` bypasses the producer and the class fails
to load with `VerifyError: ... copyRes vN <- result0 type=Undefined`. A linear "is the previous
instruction the producer" check does **not** catch this — you need the CFG view: *does any branch
target this `move-result`?* `scripts/dex_check_verifier.py` answers that question directly.
A `nop` pair cannot create that problem, which is why it is the default choice.
**Polarity is the other silent killer.** Field names lie about intent often enough that you must read
the *branch structure*, not the name. In one real case the config field was `enabled`, the smali was
`if-nez v, +6`, and the fall-through was `startActivity(main)` while the branch target was the
countdown block. So `enabled == true` was the value that *skipped* the promo — the opposite of the
literal reading. **Always decode both sides of the branch and name what each one does before editing.**
Getting this backwards produces a build that does the exact opposite of the goal and still starts
cleanly, so it survives casual testing.
## The dex header has two integrity fields, and order matters
Any edit to a dex body invalidates both header integrity fields. Recompute them **in this order**:
```
bytes 12..32 = sha1(data[32:]) # signature first
bytes 8..12 = adler32(data[12:]) # checksum last — it covers the signature bytes
```
Writing them in the reverse order produces a header that looks plausible and never verifies, because
the adler32 was taken while the signature field was still zeroed.
**Why this is dangerous rather than merely broken:** Android logs
`Failure to verify dex file ...: Bad checksum (computed, expected)` — note that the *real* adler32
appears as the "expected" value, which reads backwards — and then falls back to interpreting the dex.
The process may still start, but the class loader can fail to resolve ordinary classes
(`ClassNotFoundException: <your Application class>`), so the symptom is "the APK I built is broken"
rather than "two header fields are stale".
**Aggravating detail:** some producers ship a dex whose `signature` field is **all zeros** (a known
R8/optimiser artifact). On such a file the wrong order is self-consistent until the first patch, so
the bug is invisible until you edit something — and then looks like your edit caused it.
`scripts/dex_patch_bytes.py` recomputes both fields in the correct order and refuses to write if the
result does not self-verify.
## Worked shape of a patch script
Keep these properties; they are what make the edit auditable a month later:
1. **Locate by structure + decoded semantics**, never by a hard-coded offset alone.
2. **Assert the old bytes** before replacing, and abort on mismatch. A silent mismatch means the
sample changed or your understanding did.
3. **Equal length in, equal length out.** Assert `len(new) == len(old)`.
4. **Recompute both header fields**, then assert `adler32(data[12:]) == header_checksum` and
`sha1(data[32:]) == header_signature` on the bytes you are about to write.
5. **Re-decode the patched method** and print the instruction you changed, plus confirm the
instruction that should follow it still does. Two instructions of output is enough to catch the
"landed on the wrong site" class.
6. **Report the size delta** (should be 0) and before/after hashes of both the dex and the APK.
## Verification specific to byte patches
`patch-audit.md` covers the general audit. Two checks matter more here:
- **Assert the instruction count and every branch target are unchanged.** Compare a decoded listing of
the whole method before and after: same instruction count, same offsets, same targets. If the count
changed, you replaced a 4-byte instruction with a 4-byte *pair* and the extra decode boundary is
expected — but say so explicitly rather than letting a diff tool surprise the next reader.
- **`VerifyError` is a class-load failure, not a startup failure.** If the app starts and one screen
is dead, or a secondary class is missing, suspect a verifier violation in the method you edited.
`logcat` shows it as `E/dalvikvm` or `E/art` with the class name; it does not always surface as a
crash dialog.
## When not to use this technique
- The change needs new instructions, a new register, or a different call — go to method rewriting.
- The change is a **short string constant** whose new value is a different length — you cannot grow the
string in place. Equal-length string swaps work (`pitfalls.md` P2 on the string-id ordering guard).
**A different-length value is not a byte patch at all**: `scripts/dex_strpatch.py` enforces equal
length and will refuse it, so the work belongs to method rewriting (`references/dex-patching.md`)
or a full rebuild — do not look for a "different-length byte patcher" in this kit, there isn't one.
- The target is a **multi-dex** app and you intend to move code between dexes. Byte patching stays
inside one dex by definition.

220
vendor-new/apk-reverse/skills/apk-reverse/references/code-virtualization-and-custom-linkers.md Normal file
View File

@@ -0,0 +1,220 @@
# Code Virtualization & Custom Linkers
Load this when recon says **there is no packer** (the manifest's `application` is the app's own
class, dex is readable) and yet a re-signed build dies before your code runs — especially when the
death is `SIGSEGV fault addr 0x0` with all registers zero.
This is the layer between "packed" and "clean". `packers.md` covers a shell that owns
`Application`; this file covers protection that leaves `Application` alone and instead
**turns individual methods into native code** and hosts a private loader next to them. It is now
common enough that treating it as "no packer, therefore easy" is the most expensive recon error
in this skill's history.
## Identification — three cheap tells
1. **Whole classes whose methods are `native` declarations with no implementation.**
Read the smali for the entry Activity / engine host class. Under code virtualization the
*entire class* is converted — lifecycle callbacks, getters, everything — each becoming a bare
`.method public native ...` with no body. A framework class you *know* ships a Java
implementation (a Flutter/Unity/RN host class is the usual victim) sitting there fully
`native` is the signal.
Two `native` methods is normal (JNI). **Sixty is virtualization.**
2. **A registration call in the class's static initializer.** Someone binds those native
declarations to an implementation at class-load time:
```
.method static constructor <clinit>()V
invoke-static {v0, v1}, L<somewhere>/DtcLoader;->registerNativesForClass(ILjava/lang/Class;)V
invoke-static {v1}, L<somewhere>/hidden/Hidden0;->special_clinit_0_00(Ljava/lang/Class;)V
```
The `Hidden0.special_clinit_*` family is the virtualizer's hook for running its own version of
the class initializer. When you see this, the Java in that class **is not what executes**.
3. **A shipping library whose SONAME does not match its filename.** A private loader is often
named for its role while sitting under an innocuous filename. Read `DT_SONAME` from
`.dynamic`, not the file name — `probe_deps`-style parsing of `PT_DYNAMIC` (see
`native-tamper-and-suicide.md` §Forged section headers) gives you the truth. A `lib<something>.so` claiming
`soname = liblinkerloader.so` is a private ELF loader.
Supporting tells: `assets/` carrying a small text descriptor naming the protector and its version;
native libraries woven into `DT_NEEDED` chains of other libraries (the virtualized code has to be
reachable); log lines tagged with the loader's own name appearing during startup.
## The deadlock that eats hours — and the way out
A virtualizer that also does integrity checking produces this shape:
| Action | Result |
|---|---|
| Keep the private loader in the APK | its embedded validation runs → deliberate crash on a re-signed build |
| Delete the private loader | the virtualized class has **no implementation at all** → `UnsatisfiedLinkError: dlopen failed: library "…" not found` |
Both routes die, and the two failures look nothing alike, so it is easy to conclude the target is
impossible. It is not. The way out is one question:
> **Is that library the checker, or the implementation — and who asks for it?**
## Separate the checker from the implementation
Two distinct roles hide behind "a protected library":
- **The implementation** — the native code that *is* the virtualized class. It must be present and
loaded, or the class cannot work.
- **The checker** — a validation payload (a signature/integrity payload, a second-stage
self-decrypting image) that something *explicitly loads*.
They are frequently **different libraries**, and the load of the checker is frequently a
**single, findable call site** — not a link-time dependency.
Test which is which before designing a fix:
```
# 1. Does anything name it in DT_NEEDED? Parse PT_DYNAMIC of every shipped .so.
# No DT_NEEDED referrer => it is dlopen()ed, i.e. named at some call site.
# 2. Delete it and read the exact error: which class/method was executing when the
# load failed? That is the caller.
```
In the observed case the error was not "the class is missing" but:
```
java.lang.UnsatisfiedLinkError: dlopen failed: library "lib<checker>.so" not found
at java.lang.System.loadLibrary(System.java)
at <framework host class>.onCreate(Native Method)
```
Read that stack carefully: the **virtualized** `onCreate` — the very code you feared deleting the
library would break — is running fine. It is the *checker* that is absent. The implementation lives
elsewhere and was never the problem.
**That single stack frame converts an impossible target into a trivial one.**
## Technique: redirect the load by rewriting a string constant
Once you know the checker is named at a call site, you do not need to patch code, hook anything, or
neutralize a crash. **Change what that call site loads.**
JNI-heavy native code keeps its class/method/string operands in a constant pool inside the library.
A `loadLibrary("X")` becomes three facts sitting in `.rodata`/`.data`:
```
… onCreate (Landroid/os/Bundle;)V loadLibrary (Ljava/lang/String;)Ljava/lang/String; …
… <SomeUnrelatedClassName> X <unrelated constant> …
```
Find the name as an **isolated NUL-terminated string** and rewrite it **in place, same length** to
the name of a library that is *guaranteed already loaded*. On Android, `android` (`libandroid.so`)
is reliably mapped — the protected libraries themselves depend on it.
```
apkhuan -> android (7 chars -> 7 chars)
```
Why this is disproportionately effective:
- **Equal length means zero structural change.** No offsets move, no section resizes, the ELF stays
exactly as valid as it was. Nothing for an integrity check to notice about *layout*.
- **The load succeeds**, so no `UnsatisfiedLinkError` and no error branch anywhere.
- **The checker never gets mapped**, so none of its validation runs — the signature payload, the
deliberate-crash payload, all of it becomes dead weight in the APK.
- **You never patch a death path**, so the whole `native-tamper-and-suicide.md` rule set (make it
*return*, don't make it *not return*) never comes into play. There is nothing to neutralize.
Requirements and hazards:
1. **Same byte length is not optional.** Replacing with a shorter/longer name shifts everything
after it. Pad only within the same NUL-terminated slot; never spill into the next constant.
2. **Verify the string is an isolated constant** — the byte before and after must be `\x00`.
A substring of a longer identifier will corrupt that identifier.
3. **The replacement must be loadable and already-loaded.** A nonexistent name throws and puts you
back in the deadlock; a heavy library changes startup cost.
4. **Scan the whole file** for the name, then confirm which occurrence sits in the constant pool.
Report the offset in the patch record.
`scripts/so_constpatch.py` implements this: finds isolated constants, enforces equal length, writes
a patched copy, and prints the byte-diff.
This is one member of a general family — **redirect a reference instead of defeating a check** —
and the same shape recurs for class names, method names and file paths in a constant pool.
## Expected log shapes
A private loader announces itself. These strings come from the loader, not from the app, and their
presence/absence across builds is a usable signal:
```
I/<Loader>: Loading embedded protected image from carrier (self)
I/<Loader>: Found embedded protected image at offset 0x…, logical size 0x…
E/<Loader>: Invalid ELF magic <-- appears ONLY on a modified build
D/<Loader>: ELF Header: type=3, machine=183, entry=0x0, phnum=N
D/<Loader>: Starting PrelinkImage for embedded_protected
```
A custom container magic at the tail of the library (a four-byte tag, not `\x7fELF`) marks the
embedded image; `entry=0x0` plus `SIGSEGV fault addr 0x0` is the loader failing to decrypt its
payload and jumping to a zero entry. Treat "`Invalid ELF magic` appears on my build and not on the
original" as **proof the loader is validating the build**, which is exactly what makes the
string-redirect route worth trying.
## What the native check actually reads — measure it, don't guess
A validation payload that reads "the signature" can be reading several different things, and the
answer decides whether *any* repack route exists:
| It reads | Consequence |
|---|---|
| the certificate material **inside the APK** | keeping the original signature block while swapping code can pass |
| the signature **PackageManager reports** | that trick fails; you must remove the check (this file's technique) or rebuild |
| an entry **hash** of the whole file | only the string-redirect / check-removal route survives |
**Single-variable experiment to tell them apart.** Same APK content both times; flip only the
PMS-recorded signature:
1. Install a build **you signed** normally, so PMS records *your* certificate.
2. Root-overwrite that installed `base.apk` with your candidate (one whose *internal* signature
block is the original), leaving PMS's record untouched.
3. Launch.
Dies ⇒ the check reads the PMS-reported signature, and preserving in-APK certificate material
cannot save you. Survives ⇒ it reads in-APK material, and a "keep the original signature block"
route is worth investing in.
Do this **before** investing in elaborate signature-block plumbing. It is two installs.
## A Java-layer "signature killer" is a decoy
A protection bundle may ship a Java class that hooks `PackageManager` so the framework reports a
hardcoded certificate, and log `Signature consistency ok` forever — including on your modified
build. This is an **anti-tamper feature aimed at other patchers**, not a gate you must satisfy.
Its corollary is what matters operationally: **`Signature consistency ok` in the log proves
nothing.** Do not let a green Java-layer message talk you out of the fact that the process is
dying, or send you hunting for a Java signature check. If the death is `SIGSEGV` with a zero
program counter, the check that killed you is native and the Java one is irrelevant.
Two practical notes:
- **Do not delete such a class to "clean up".** It usually participates in the `Application`
inheritance chain; removing it breaks startup and trips unrelated class-hierarchy checks.
- Its presence is a strong hint that **the code that matters is virtualized**, because it exists to
survive exactly the kind of repack you are attempting.
## Verification
A string-redirect fix is verified by the **absence of the loader**, not by the absence of an error:
1. Loader's log tag count in a full cold-start capture = **0** (previously non-zero).
2. `Invalid ELF magic` = 0.
3. `SIGSEGV` = 0 **and** `SIGSYS`/seccomp kills = 0 — the secondary kill paths the payload carried
disappear with it. A drop from 1 to 0 here is a real result worth recording.
4. Process survives past the previously measured time-to-death, on this machine, with nothing
attached; then survives real interaction.
5. The features that class implements still work — the virtualized code is the *implementation*, so
exercise it (start playback, navigate, whatever it owns). A build that starts but whose
virtualized feature is dead is not a result.
Record the original and patched bytes, the offset, and the loader log counts before/after in the
patch record — the same audit standard as any other byte-level edit.

220
vendor-new/apk-reverse/skills/apk-reverse/references/coverage-and-limits.md Normal file
View File

@@ -0,0 +1,220 @@
# Coverage and limits — the evidence behind each claim
`SKILL.md` §Coverage states **what this skill covers and what it does not**, because the failure
worth preventing is not ignorance but *a confident wrong answer produced by applying the nearest
available procedure to a target it was never written for*. That statement has to stay short: it is
read on every task, before any routing decision.
This file carries the part that is read **only when you need to weigh a claim** — the evidence
behind each covered item, the exact footing of the unverified ones, and the historical record of
what was and was not exercised. Load it when a claim's strength decides whether you trust it.
**Load this when:** a claim's strength decides whether you trust it, or you are about to quote this skill's coverage. It gives the evidence behind each covered item, the dependencies this skill does not ship, and the routes nobody has run.
## Strength labels
- **observed** — a command was run and its output is recorded in the evidence record condensed in `references/evidence-summary.md` §Where the full record lives.
- **inferred** — follows from documented mechanism or from a neighbouring measurement; the step
itself was not executed.
- **unverified** — assumed, or reported by someone else, and not reproduced in this repository.
Reserve `observed` strictly (`references/long-task-discipline.md` does). "I reasoned it out" is
`inferred`. The distinction is load-bearing, not cosmetic: an `inferred` route may still be right,
but nobody here has paid for the counters yet.
## Covered, by verified mechanisms — the evidence
- **Client-side ads, promos and splash/popup/tab configuration**, including the server-issued UI
config that has no SDK to find (`ad-removal.md`, `server-config-and-updates.md`).
- **Deciding whether a membership, paywall or feature gate is client-enforceable at all**, and
saying so plainly when it is not (`membership-and-limits.md`, `account-gates.md`).
- **dex-level surgical patching**: equal-length byte edits and dexlib2 method rewrites, plus the
header, verifier and alignment rules that decide whether the build loads (`dex-patching.md`,
`byte-level-patching.md`, `patch-audit.md`).
- **Repacking, signing, installing**, and the install refusals that look like a broken build
(`repack-and-sign.md`).
- **Packers, custom loaders and code virtualization**: identifying them, measuring the validation
boundary, and the routes that survive it (`packers.md`,
`code-virtualization-and-custom-linkers.md`).
- **The native layer**: `.so` hosts, tamper-triggered self-termination, forged ELF structure, and
neutralising a terminate path without freezing the process (`native-and-so.md`,
`native-tamper-and-suicide.md`).
- **Flutter / Dart AOT**: analysing and patching `libapp.so` **given a snapshot dump** —
pool-reference counting, disassembly windows, caller indexing, patch-site choice (`dart-aot.md`).
Measured on a real Dart 3.6.0 build: `dart_disasm.py` decoded identically to capstone (32/32 and
96/96), the caller index was re-derived independently with a symmetric difference of 0, and a
specific business logic site was located end to end. **The snapshot dump is a dependency, not a
detail** — see the next section.
- **Runtime analysis with Frida, server-side API probing, feature-scoped TLS failures, update and
forced-upgrade neutralisation**, and the verification discipline everything above rests on.
### Added by the benchmark pass, and measured against public targets
Each of these has a row in the benchmark matrix (`references/evidence-summary.md` §The capability matrix) naming its target, its strength and the evidence
file behind it.
- **Java2C versus JNI sinking versus an extraction shell.** Native-declaration density separates the
shapes by roughly 2000x: **0.03–0.04%** on three real JNI samples against **82.76%** on a
Java2C-shaped fixture, with `Java_*` symbols matching dex native-method counts 1:1 per ABI. A
symbol search fails *silently* on real Java2C — `dcc` emits `-fvisibility=hidden`, and C++
`jni.h` inlines `RegisterNatives` so it leaves no symbol at all
(`java2c-and-jni-sinking.md`, `scripts/java2c_probe.py`). **No Dex-to-C compiler output was ever
built here** (no NDK, no host clang, no device clang, WSL unavailable), so every Java2C-specific
criterion is `inferred`; the JNI-sinking side and the discrimination itself are `observed`.
- **Split APK / App Bundle sets.** Inventory, unified re-signing for `pm install-multiple`, and a
merge of code/native members into one APK — the merge **refuses by design** when a member carries
its own `resources.arsc` (`split-apk.md`, `scripts/repack.py`). Measured on two real sets pulled
from a device; `adb install-multiple` and launch confirmation were **not** executed.
- **Schema-free protobuf decoding**, cross-checked against the official runtime
(`scripts/protobuf_decode_raw.py`, `protocol-reverse.md`). 26/26 built-in fixtures, 21/21 against
the official runtime, and a real DataStore container round-tripped byte-exact.
- **Differential hardening for a real Dex VMP** — a labelled opcode-coverage fixture, a closed-loop
verification of the derived map, and an explicit cost judgement that the upload link cannot be
automated (`vmp-differential-analysis.md`, `scripts/vmp_diff_harness.py`).
- **Kernel-module templates with their version gates**
(`kernel-and-environment-hardening.md`, `scripts/kernelsu_syscall_mask.py`). The generator is
measured; **no kernel-side artefact was compiled or loaded** — see the record below.
## Covered by documented routes — inferred rather than measured
Written from public work and from whatever the extension pass could exercise; each document carries
its own strength note at the top.
- **Module-side delivery instead of a repack** — what to do when the client-side logic is reachable
but a rebuilt APK is refused (`lsposed-and-modules.md`).
- **Extraction shells and the VMP boundary** — how to *measure* that the bodies are empty instead of
guessing, why the classic active-invocation hooks stopped working on Android 12–16, and where
recovery honestly stops (`advanced-unpacking.md`, `scripts/dex_dump_validate.py`).
- **Calling the target instead of reading it** — emulated execution and live Frida-RPC
service-ification (`emulation-and-rpc.md`).
- **Instruction-level tracing and de-obfuscation** — the OLLVM shapes, a Stalker trace, the
trace-to-CFG route, and the measured boundaries in `native-dbi-and-deobfuscation.md`.
- **Protocol reversing beyond REST** — protobuf without a schema, gRPC frame capture, the QUIC/HTTP3
limit, native-side certificate pinning (`protocol-reverse.md`).
- **What to do when userspace hooking provably cannot reach the check** — raw `svc` syscalls,
`init_array`-early detection, and the kernel-route map with its version gate
(`kernel-and-environment-hardening.md`).
- **Working from the phone itself** — MT Manager, its APK MCP surface, LSPosed Manager, on-device
data inspection (`on-device-tooling.md`, `scripts/mt_mcp_probe.py`).
## Dependencies this skill does not ship — name them before the workflow starts
- **Dart AOT analysis needs a snapshot dump.** `dart-aot.md`'s workflow begins at `pp.txt`;
producing it requires a snapshot container resolver this skill does not contain and cannot
synthesize. `dart_pool_strings.py` reports **file** offsets while `pp.txt` and `dart_pprefs.py`
speak in **pool** offsets, and the mapping between the two spaces is not a constant: over the
4,241 strings present in both, a measured run found 4,237 distinct deltas. Ship a pinned front end
(aotopsy — pure Go, no toolchain) or build blutter (~80 s, needs a C++ toolchain). **Say which one
you are using and why, because the two report different Dart version labels for the same binary.**
Do not describe the object pool as something this skill decodes on its own.
## Not covered — say so rather than improvise
- **Unity / IL2CPP logic recovery.** `framework-runtimes.md` identifies the runtime and establishes
that the dex is not the battlefield; it does not carry the IL2CPP equivalent of `dart-aot.md`.
There is no verified recipe here for locating a method inside `libil2cpp.so` plus
`global-metadata.dat`.
- **React Native / Hermes bytecode** and Cordova/hybrid internals, beyond runtime identification and
the generic "find the string, then find what references it" approach.
- **iOS / `.ipa` of any kind.** Every device, signing and packaging instruction here is Android.
- **Defeating a server-side authority.** `server-api.md` determines *who owns a gate*, not how to
break an authorization the server performs.
- **An off-the-shelf unpacker, and an anti-detection arms race.** `advanced-unpacking.md` routes the
problem — measure the ratio, name the recovery mechanism, stop when the target is a real VMP — but
ships no modified ART runtime, no private-bytecode decompiler and no opcode-mapping derivation,
and says so rather than presenting a memory dump as a recovery. `detection-and-anti-analysis.md`
decides by cost and often concludes "switch to static"; it is not a catalogue of evasion. It now
also carries an order of search, plus a measurable environment self-report and an observer-only
probe — and the boundary that keeps those from becoming an evasion project is stated in the same
file (its Step 2B, two boundaries this file adopts): observers and interceptors stay separate
modules, and the cost of each added module is what decides when to leave for static.
- **Kernel development.** `kernel-and-environment-hardening.md` maps the kernel-side route and names
the version gate that decides whether it exists on your device; building and shipping a kernel
module is outside this skill.
- **A working Stalker trace on every device.** Two boundaries are measured and one of them is now
better understood: exclusion keeps the target alive but does **not** restore event delivery
(`references/evidence-summary.md` §The capability matrix). Treat the zero-event case as a boundary to
identify, not a recipe to follow.
## Historical verification record
This is the part that grows every pass, which is why it does not live in `SKILL.md`. Read it before
trusting any "this was measured" statement above.
**The first verification pass measured, against a real target:** a mid-size Flutter AOT application
with no packer — the Dart work above, library mapping, and the environment facts. It **did not
exercise** the packer, code-virtualization, custom-linker, integrity-check-redirection or
tamper-suicide scenarios: that target has none of those features and its unmodified build already
fails to start, which removes the repack-and-regress loop those scenarios need. Nothing in
the evidence record condensed in `references/evidence-summary.md` §Where the full record lives is evidence either way about them.
**Scripts the first pass did not run** — they carry no measurement from that pass, and any
conclusion drawn from them should be labelled accordingly: `native_crash.py`, `apk_diff.py`,
`snap.py`, `grab_crash.py`, `install_test.py`, `repack.py`, `dex_patch_bytes.py`,
`dex_find_insn.py`, `dex_check_verifier.py`, `dex_classdiff.py`, `dex_strpatch.py`,
`patch_smali.py`, `smtool.py`, `datastore_inject.py`, `probe_api.py`, `run_probe.py`,
`tls_check.py`, `usb_net_proxy.py`, `devsh.py`, and the `dexpatch/` java rewriter. Absence from
that record is not a verdict on them. Note that `grab_crash.py` claims to recover stacks hidden by
a crash-reporter SDK — the exact situation that target presented — and was not tried, so that claim
remains **unverified**. **The benchmark pass has since run some of these**: `repack.py`,
`dex_patch_bytes.py`, `dex_dump_validate.py`, `dex_mem_scan.py`, `spawn_patch_detach.py` and
`stalker_trace.js` now have measurements, with the failures recorded alongside the successes in
the benchmark matrix (`references/evidence-summary.md` §The capability matrix).
**The extension pass shipped its own scripts, each with its own status** — read the matching
the per-topic files named in `references/evidence-summary.md` §Where the full record lives before relying on one: `dex_dump_validate.py` (measured
against a fixture derived from a real hardened sample, and against that sample's own shell dex),
`lsposed_scaffold.py` (its generated project was built end to end, toolchain timings recorded),
`frida_rpc_serve.py` (the `rpc.exports` bridge was exercised on a live device),
`mt_mcp_probe.py` (the "service is down" path is measured; the connected path needs the service
started by hand), and `stalker_trace.js` / `stalker_report.py` (two boundary results, no successful
trace of a real target).
**The extension pass's claims are mostly `inferred`.** Its evidence lives in
the per-topic files named in `references/evidence-summary.md` §Where the full record lives, one file per topic, each with its own strength note. The
common shape there is *the tool was measured, the route was not* — read those files before treating
any of the newer documents as a verified path.
**The detection pass added two tools and three boundaries, and its evidence is mixed.** Recorded in
`references/evidence-summary.md` §The capability matrix:
- `svc_scan.py` — **measured**, and its whole claim is cross-checked: two independent decoders
(a hand-written word scan and the capstone-based scan) returned an identical 214-site set on a
device `linker64`, set difference empty. That agreement is what makes its output usable as
evidence about a *target's* library rather than about the scanner.
- `anti_detect_probe.js` — **measured, observer-only by contract** (it patches nothing, so a run
that uses it still describes the target it was pointed at). On a public MASTG challenge target it
recorded `strstr("frida")` at ~300 ms of process life followed by a clean self-exit, and it
reported the target's own view of the environment (`TracerPid=0` while **four** frida-named
mappings were visible in `/proc/self/maps`). The same arm captured the full sequence **once out of
three runs** — on a target that dies in ~300 ms the window is sub-second and not arm-to-arm
reproducible, which is why the reference labels it that way rather than as a pipeline.
- `scan_leaks.py` — **measured against a planted corpus and against this repository**. On the corpus
every rule fired (30 findings across 6 categories) and the exemption list produced zero false
positives on identifiers that must stay (tool names, SDK packages, dex constant identifiers, CVEs,
hardening products, public crackme names, placeholders). On this repository it found **26 strong
hits on its own evidence file**, which is the strongest evidence available that the class of leak it
targets is not visible by hand. `observed`.
- **The Dart AOT string-table formats** — the arm64 packed scheme is confirmed at the byte level
(tag byte equal to `0x80|(len<<1)` at every literal checked, 4,980 chained pool entries), and the
previously documented armv7 form is **refuted**: the 32-bit record is
`[header u32][byte-count u32le][UTF-8 payload]`, and the extractor's zero for that ABI is a format
mismatch rather than an empty table. `observed` for the formats; `unverified` for an end-to-end
patched build, because none was repacked and installed in that pass.
- **`observed`:** for the `ptrace-free` route, byte-identity between two *read paths*
(`/proc/<pid>/mem` versus the backing file) on the same byte range, and the route's negative
boundary — a target whose dex is deflate-compressed inside its APK has **no** named dex mapping to
export (`named_dex=0` on both MASTG targets), so the route yields nothing there for reasons that
say nothing about protection.
- **`unverified`:** byte-identity between two independent *dumpers*. The second producer
(`frida-dexdump`) is refused by this repository's hardened sample, whose usable lifetime collapsed
to ~2 s this pass, and the clean targets have no whole-image dex for any dumper to find. The
weaker read-path check is what was run instead, and the distinction is stated in the evidence file
rather than papered over.
**Where the record lives.** The full record is `docs/tool-verification/` at the **repository root, not
shipped** with the skill; `references/evidence-summary.md` is the condensation that does travel, and
`references/routing.md` is the inventory. Neither the per-topic records nor the public-target
regression matrix ships inside an installed skill, so an installed copy carries the claim but not its
evidence file — which is why the strength label is stated here rather than left to be looked up.

70
vendor-new/apk-reverse/skills/apk-reverse/references/coverage/route-inventory.md Normal file
View File

@@ -0,0 +1,70 @@
# Coverage inventory — the per-route list behind `SKILL.md`'s Coverage section
`SKILL.md` states the rule: **do not apply the nearest available procedure to a target it was never
written for**, and label every claim `observed` / `inferred` / `unverified`. This file is the list that
rule needs — which routes are covered, by what, and how strong the evidence actually is.
Load it when the question is "**can this skill actually do X**", or before quoting a coverage claim
as if it were verified. `references/coverage-and-limits.md` carries the same material as a claim
ledger with the per-item evidence; this file is the inventory you can read in one pass.
## Covered, by verified mechanisms
These have a command and an output behind them somewhere in the evidence record (repository root,
the evidence record condensed in `references/evidence-summary.md` §Where the full record lives, **not shipped with the skill**).
| Area | Route, in one line | Where it lives |
|---|---|---|
| Client-side ads, promos, splash/popup/tab config | includes server-issued UI config that has no SDK to find | `ad-removal.md`, `server-config-and-updates.md` |
| Membership / paywall / feature gates | decide whether it is *client-enforceable* at all, and say so when it is not | `membership-and-limits.md`, `account-gates.md` |
| dex-level surgical patching | equal-length byte edits, dexlib2 method rewrite, header/verifier/alignment rules | `dex-patching.md`, `byte-level-patching.md`, `patch-audit.md` |
| Repack, sign, install | including the install refusals that look like a broken build | `repack-and-sign.md` |
| Packers, custom loaders, code virtualization | identification, the validation boundary, the routes that survive it | `packers.md`, `code-virtualization-and-custom-linkers.md` |
| Java2C vs extraction shell vs JNI sinking | the misdiagnosis that sends you hunting a decrypted DEX that never exists | `java2c-and-jni-sinking.md`, `scripts/java2c_probe.py` |
| Native layer | `.so` hosts, tamper-triggered self-termination, forged ELF, neutralising a terminate path | `native-and-so.md`, `native-tamper-and-suicide.md` |
| Flutter / Dart AOT | analysing and patching `libapp.so` **given a snapshot dump** | `dart-aot.md` |
| Runtime and server side | Frida analysis, API probing, feature-scoped TLS failures, update/forced-upgrade neutralisation | `dynamic-frida.md`, `server-api.md`, `tls-and-cert.md`, `updates-and-forced-upgrade.md` |
| Verification discipline | the claim ladder, control builds, capture-and-look, bounded waits | `verification.md`, `long-task-discipline.md` |
## Also covered, by documented routes — mostly *inferred* rather than measured
Read the qualification, not just the name: for these the **mechanism is documented and the end-to-end
route was not measured on this repository's evidence**. Treat them as leads with a stated boundary.
| Area | What it gives you |
|---|---|
| Module-side delivery | an LSPosed/Xposed module when a repack is refused — `lsposed-and-modules.md` |
| Extraction shells and the VMP boundary | how to *measure* that the bodies are empty, and where recovery stops — `advanced-unpacking.md` |
| Calling a routine instead of reading it | emulation (Unidbg/Unicorn) versus service-ifying the live function over Frida RPC — `emulation-and-rpc.md` |
| Instruction-level tracing | Stalker traces and the trace-to-CFG route against OLLVM, with its measured boundaries — `native-dbi-and-deobfuscation.md` |
| Protocols beyond REST | schema-free protobuf, gRPC frames, QUIC/HTTP3 limits, native-side pinning — `protocol-reverse.md` |
| When userspace hooking cannot reach the check | raw `svc`, `init_array`-early detection, the kernel-route map and its version gate — `kernel-and-environment-hardening.md` |
| Split APK / App Bundle sets | reading a set, merge versus unified re-signing, the install refusal each mistake produces — `split-apk.md` |
| Dex-VMP differential analysis | the known-plaintext route, what it can and cannot automate — `vmp-differential-analysis.md` |
| Kernel-module templates | a scaffold **and** its version gate; a userspace module cannot change a syscall return value — `kernelsu_syscall_mask.py` |
| Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` |
| Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` |
## Dependencies this skill does not ship
Each of these is a **fence**, not a caveat: the route above it stops here unless you obtain the
dependency yourself.
- **Dart AOT analysis needs a snapshot dump**, and which front end produced it matters (`dart-aot.md` §1–2).
- **`smali` round-trip needs baksmali/smali/dexlib2 jars** — not bundled; `APKREV_JARS` points at them.
- **Repack-and-sign needs `zipalign` + `apksigner`** (or `uber-apk-signer`); a JVM alone is not enough,
which is exactly what `scripts/doctor.py` now refuses to report as OK.
- **Device work needs `adb` plus a device the app does not object to**; dynamic work additionally needs
a `frida-server` matching the host version.
- **A kernel-side answer needs a module and a kernel that accepts it** — the version gate is in
`kernel-and-environment-hardening.md`.
## Not covered — say so rather than improvise
Unity / IL2CPP logic recovery · React Native, Hermes bytecode and Cordova internals · iOS of any kind ·
defeating a server-side authority · an off-the-shelf unpacker · an anti-detection arms race · building
and shipping a kernel module · a Stalker trace guaranteed on every device.
Each has a reason and, where one exists, an evidence boundary — in `references/coverage-and-limits.md`.
**Silence in that record is not support**, and a route that is only "documented" is not a route that
was measured.

393
vendor-new/apk-reverse/skills/apk-reverse/references/dart-aot.md Normal file
View File

@@ -0,0 +1,393 @@
# Dart AOT (Flutter) — reaching the layer that actually owns the logic
Load this when: the target is a Flutter app (`libflutter.so` + `libapp.so`) and the behaviour you
must change is decided **inside** the Dart snapshot, not in the dex. The Java side of a Flutter app
is a thin plugin shell; a dex-only plan will stall (`framework-runtimes.md` §the layer trap).
Everything below assumes you already confirmed the runtime is Flutter and that Java hooks fire
**zero times** while the UI plainly works.
---
## 1. Pin the Dart version before choosing any tool
The snapshot format is version-specific. A decompiler built for another Dart version may run and
produce plausible-but-wrong output, which is worse than failing.
```python
import re
blob = open('libflutter.so', 'rb').read() # the engine, not libapp.so
m = re.search(rb'(\d+\.\d+\.\d+ \(stable\)[^\x00]{0,60})', blob)
print(m.group(1).decode() if m else 'not found') # e.g. b'3.9.0 (stable) (Mon ...) on "android_arm64"'
```
If the string is absent, fall back to the snapshot hash the loader reports at runtime, or to the
engine build id in `libflutter.so`'s version string.
**The regex above can return a version-shaped lie — assert the shape, do not just take the digits.**
Measured exception, on a real `libflutter.so`: the full banner string is
`0.0.1 on "android_arm64"` — a 66-byte engine version
record whose numeric field is `0.0.1` and whose **64-character build-id field is blank**, with no
`(stable)` marker and no `(Mon …)` timestamp at all. Anchoring only on `\d+\.\d+\.\d+` matches the
truncated prefix of a banner that was never populated. The discriminator that works: find the
` on "<arch>"` suffix, take the version string from the preceding NUL, and check that a
`(stable) (…)` tail is actually present **before** feeding the number to a VM-compiling tool
(§2's Route B). If it is missing, you cannot pin the version from this file — say so and use the
snapshot hash or a structural probe (§2's Route A), rather than building a decompiler for `0.0.1`.
## 2. Get a decompiler that matches that version
You need a tool that resolves the snapshot container — something that turns `libapp.so` into named
functions, class layouts and, above all, a **pool listing with `pp+0x…` offsets**. Everything in
onward consumes that listing; without it the workflow cannot start, and no script in this skill can
produce it (see §4 for why the mapping is not recoverable from the binary alone).
Two routes exist. Pick deliberately, because their version reporting differs.
**Route A — aotopsy: no toolchain, run it now.** A single static binary (pure Go) that parses the
snapshot binary format directly, with no Dart VM and no SDK compile. On a real Dart 3.6.0
`libapp.so` the full pipeline ran out of the box: 30,586 functions, 6,133 class layouts, 39,202 pool
entries (38,274 resolved), 175,120 call edges with 95.7% of indirect sites annotated, plus
`functions.jsonl`, `call_edges.jsonl`, `classes.jsonl`, `string_refs.jsonl`,
`dispatch_table.jsonl` and an annotated `asm/` tree. It also supports x86_64, which blutter does not.
Its `doctor` subcommand reports a Dart version that is a **structural profile label, not the SDK
version**: on the sample above it printed `3.6.2` while the engine banner, the snapshot hash and a
byte search all said `3.6.0`, and no `3.6.2` byte sequence existed in either `.so`. That matters only
if you feed the number to a VM-compiling tool — use §1's banner for that. Its own README states the
detection is structure-based, so expect the label to name the newest matching profile rather than the
compiler's version.
**Route B — blutter: full fidelity, needs a build.** It embeds a matching Dart VM and deserializes
through the VM's own code paths, which yields the canonical `pp.txt` in the `pp+0x…` space. It is a
*source tree you build*, not a binary you download: its `bin/` is gitignored and it publishes no
release artifacts, so there is nothing prebuilt to fetch for a given Dart version.
- blutter's git HEAD typically supports newer Dart than its shipped `dartsdk/` directory.
- When the target's version is missing, blutter fetches it itself: it sparse-clones
`dart-lang/sdk` at that version tag (only `runtime`, `tools`, `third_party/double-conversion`),
generates a source list, and builds a `dartvm` static library for the target ABI.
- That build needs `cmake` (>=3.20 is fine, including 4.x), `ninja`, `git`, and a C++ compiler with
C++20 `<format>` support. The documented VS 2022 requirement is **over-strict** — MSVC 19.34
(VS 17.4.3, Nov 2022) compiles and runs the `std::format` path, measured. A `vcvars64`-style
environment must be active, and `CMakeLists` pins `cmake_minimum_required(3.20)`.
- Budget a **couple of minutes**, not "tens of minutes": the whole pipeline (clone, `init_env_win.py`,
sparse SDK clone, cmake, nja, link) measured **≈78 s** wall clock. It uses a unity build via
`dartvm_create_srclist.py`, which is why it is fast. The compiler log is enormous (per-file include
trees), so read only its tail.
- **If the freshly built binary faults immediately** with `0xC0000005` and no output at all, do not
conclude anything about the sample. Relink it (`ninja` in
`build/blutter_dartvm<ver>_<os>_<arch>`) and try again; on the sample above the fault never
reproduced after a relink — including for the unchanged binary — and three subsequent runs each
completed in ~5.9 s. **One retry, then a relink, before you blame the target.**
- **Two Windows build failures that are not your target's fault, both measured `~0.2 h` each.**
*The compiler is not found even though `vcvars64` ran*: `%PATH%` expands when `cmd` parses the
line, so a `set PATH=...;%PATH%` in the same command overwrites the environment `vcvars` just
installed. Use delayed expansion (`cmd /V:ON` with `set PATH=...;!PATH!`) or run `vcvars64` in a
separate `cmd` invocation. *`string(REPLACE "/EHsc" ...)` aborts with "not enough arguments"*:
the `REPLACE` call is unguarded, so it breaks when `CMAKE_CXX_FLAGS` is empty — add an
`if(CMAKE_CXX_FLAGS)` guard, and patch **both** the template and the generated copy, or the next
configure regenerates the broken one. A prebuilt binary offered by a mirror is not a shortcut if
it is built for `aarch64`-Linux (the Termux target): check its ELF machine before spending time on
it — that mismatch cost `1 h` on the precedent this section is derived from, against `0.2 h` for
either compile fix above.
Outputs of interest (blutter):
| Output | What it gives you | What it does *not* give |
|---|---|---|
| `pp.txt` | every object-pool entry with its `pp+0x…` offset: strings, types, closures, fields, stubs | who references what |
| `objs.txt` | reconstructed object shapes with type annotations | code |
| `asm/` | class layouts **and function bodies with full instruction listings**, pool annotations and resolved call targets | a pseudocode-level view |
| `ida_script/`, `blutter_frida.js` | symbol/annotation helpers | a finished analysis |
> `asm/` is the richest artifact in the chain. It is one file per **library URI** (mirroring the
> package path), not per class as its name suggests, and it carries annotated disassembly such as:
>
> ```
> // 0x923b3c: r0 = LinkedHashMap.from()
> // 0x923b3c: bl #0x60165c ; [dart:collection] LinkedHashMap::LinkedHashMap.from
> ```
>
> An earlier revision of this file claimed `asm/` contained no instructions and told the reader to
> ignore it. That was wrong and cost real time — plan around `asm/` as your primary annotated
> disassembly source.
**`product` builds carry no debug info.** Expect `no-code_comments`, no function names, and
obfuscated identifiers. That is normal; the strings still survive (see). Note this is a *format*
floor, not a tooling gap: the Dart compiler drops field names outside debug builds, so roughly
97-99% of instance field names are simply absent, and local/captured variable names are gone
entirely. Accessor-based recovery (`get:`/`set:` still carry the name) recovers part of the field
picture; the rest should be rendered as unknown rather than guessed.
## 3. The object pool is the whole game
Dart AOT uses **compressed pointers**: heap references are 32-bit offsets from a base held in a
dedicated register. Constants, strings, types, closures and field metadata live in one contiguous
**object pool**, reached through that register.
Two offset spaces exist and they are **not the same number** — mixing them wastes hours:
- **PP offsets** (`pp+0x…`) — what `pp.txt` and every pool-load in disassembly use.
- **file offsets** — where a byte actually sits in `libapp.so` on disk.
Keep the mapping explicit in your notes. When a tool reports "this string is at X", state which
space X is in.
**There is no constant between the two spaces — do not go looking for one.** The relationship is a
property of the *reconstructed* pool, not of the file, so it cannot be computed from `libapp.so`
alone. Measured over the 4,241 strings that appear in both the string table and `pp.txt`, there were
**4,237 distinct deltas** (range −1,361,429 to +277,654). A string recovered by file offset therefore
does not lead to its pool offset, which is exactly why §2 insists on a snapshot-decoding front end:
it is what gives you the `pp+0x…` space in the first place.
## 4. Build your own reference index (blutter will not give you this)
`pp.txt` tells you what is in the pool. To *find the code that uses it* you need pool-offset →
referencing-instruction-address. Generate it once, then query it constantly:
```
python dart_pprefs.py libapp.so pp_refs.json
python dart_pprefs.py --lookup pp_refs.json 0x1d1a8 0xc9d0
```
**Do not build this index with a full-capstone pass.** Decoding every instruction of a multi-MB
`.text` with `detail=True` and querying `regs_access()` per instruction costs minutes of CPU and
1–2 GB of peak memory, and on a 16 GB host it presents as a hang with no progress output. Only a
handful of encodings can read the pool, so decode them from the raw 32-bit words instead — the
shipped script does exactly that and finishes in seconds.
**But do not fall back to a linear capstone sweep either.** A Dart AOT `.text` begins with snapshot
metadata, not instructions — on a real `libapp.so` the section started at `0x4a0000` while the first
Dart function prologue (`stp x29, x30, [sp, #-imm]!`) was at `0x4b143c`. A linear sweep starting at
offset 0 dies on that metadata and returns **zero instructions**, which looks exactly like "this
library has no code". If you need capstone here, start at a known function address, or use
`skipdata=True`, and treat an implausibly small count as a decoding problem before you treat it as a
property of the file.
The same mask-based argument applies to building a call graph (): decode `B`/`BL` arithmetically
rather than by sweeping.
**A live example of why the index's completeness matters:** after this file's advice to "count the
references before editing a shared constant", one pool offset measured `[0 refs]` with the shipped
script and `[79 refs]` once a missing load family was added to the scanner. A one-sided quiet gap in
the scan reads as "nothing uses this", which is the answer that gets a shared constant edited
carelessly. If a count looks surprisingly low, suspect the scanner before believing the pool.
## 5. What the registers mean
Stable across Dart 3.x arm64 AOT. Verify once on a known function, then rely on it.
| Register | Role |
|---|---|
| `x27` | **object pool base (PP)** — every constant/string/type load goes through it |
| `x26` | current thread (`[x26,#0x38]` stack limit, `[x26,#0x68]` isolate group) |
| `x28` | heap base — how compressed pointers are decompressed |
| `x22` | **null/base for booleans** (see) |
| `x15` | Dart's own stack pointer (not the system SP) |
| `x21` | class dispatch table (virtual calls load a target from it) |
## 6. Booleans are not 0/1
`true` and `false` are objects immediately adjacent to null, so they appear as small fixed offsets
from `x22`:
```asm
add x0, x22, #0x20 ; construct TRUE
add x0, x22, #0x30 ; construct FALSE
tbnz w0, #4, <label> ; test bit 4 of the value
tbz w0, #4, <label>
```
The bit-4 test is the standard "is this the false object" check, and it is the cheapest place to
force a decision: replacing the conditional branch with an unconditional one, or replacing a
constructed constant, flips the outcome without changing structure.
## 7. String encoding — get this exactly right
Pool string entries are serialized as:
```
[ tag byte ][ payload ]
tag = 0x80 | (len << 1) | two_byte_bit
```
- **one-byte strings** (ASCII / Latin-1): `payload` is the raw bytes.
- **two-byte strings**: `payload` is **UTF-16LE**.
Consequences that decide whether your search works:
- An **ASCII** literal (`/api/foo`, `isVip`, a URL) is found by a **UTF-8 byte search**. It is a
one-byte string — searching UTF-16 for it will miss.
- A **CJK / non-Latin** literal is a two-byte string: a UTF-8 search returns **zero**, a UTF-16LE
search hits. This is the case that makes people wrongly conclude "the strings were stripped".
- Therefore: **search ASCII as UTF-8 and non-ASCII as UTF-16LE**, rather than forcing one encoding
on everything. (This is the concrete form of `pitfalls.md` P25.)
- A raw byte scan for the tag pattern finds many false candidates. Accept a candidate only when
entries **chain** — the previous entry's `tag+payload` must end exactly where the next candidate
begins. Without that constraint you get order-of-magnitude more garbage than strings; a
per-byte scan of a multi-MB snapshot once produced ~880k "strings" of which essentially all were
misaligned noise, and a "no such feature present" conclusion was drawn from it.
```
python dart_pool_strings.py libapp.so strings.tsv --min 3
```
### The 32-bit ABI does not use this format — and the tool will say so
**On a 32-bit (`armeabi-v7a`) snapshot the packed tag scheme above does not exist, and the string
table is not laid out as a walkable chain at all.** `dart_pool_strings.py` enforces the chaining
constraint, so on armv7 it reports `kept (run >= 3): 0` — every candidate is isolated and discarded.
That zero is a **format mismatch, not an empty string table**: the same extractor over the arm64
snapshot of the same app kept 4,980 chained entries. Do not read the armv7 zero as "the strings were
stripped".
Measured on one app shipped with both ABIs (arm64 16,352,152 B / armv7 17,629,776 B):
| ABI | Where a literal sits | How to find it |
|---|---|---|
| arm64 | inside the packed string table, `[0x80\|(len<<1)][payload]` | `dart_pool_strings.py`; the chain constraint makes it reliable |
| armv7 | **inline in the read-only data**, as its own record | a plain ASCII search for the literal — it is *not* UTF-16, so a byte search is the whole method |
For armv7, the record a literal sits in is **4-byte aligned** and reads
`[header u32][length u32le][SAFE payload]`, where `length` is the **byte** count of the payload
(measured: an ASCII literal 12 chars long carried `12`, and a 70-char literal carried `70`). The
header u32 is a class/tags word that varies between snapshots — on the sample above it took the
values `0x00550238` and `0x00560238`, both 4-byte aligned, 13,397 occurrences. Treat the header as
a **validator, not a constant**: assert `length == payload length` and that the payload is
all-printable, then read the header word from the file rather than hard-coding it. Scanning the whole
17.6 MB image at 4-byte stride costs **0.94 s**, so validating every candidate is affordable.
Consequence for patching: because the armv7 payload is **UTF-8 and length-explicit**, an equal-length
ASCII replacement is legal on both ABIs without recomputing the tag — the one-byte tag encodes the
length only on arm64, which is why the same patch script can serve both if it asserts the prefix it
found instead of assuming which encoding it is looking at.
**Patch every ABI the app ships, or the device may run the one you left alone.** A phone that
prefers `arm64-v8a` still has the `armeabi-v7a` snapshot on disk, and a build that patches only one
of them is a controlled experiment, not a deliverable.
## 8. Reading AOT code: three signatures that carry most of the weight
Once you can disassemble a window with pool annotations, most business logic resolves into these
shapes.
**A — reading a map / JSON object (`map[key]`)**
```asm
ldur x3, [x29, #-8] ; the Map
ldur x0, [x3, #-1] ; compressed class id sits at offset -1
ubfx x0, x0, #0xc, #0x14 ; decode class id
mov x1, x3
... x2 = <pool string> ; <- the key, annotated by your pool map
add x30, x0, #0x342
ldr x30, [x21, x30, lsl #3]
blr x30 ; this is map[key]
```
Remember the quartet: `ldur [..,#-1]` + `ubfx #0xc,#0x14` + `add x30,x0,...` + `blr`.
**B — writing a map (`map[key] = value`)**
```asm
... x16 = <pool string> ; the key
stur w16, [x0, #0xf] ; key stored into the freshly built map
... ; then the value at the next slot
```
`stur` writes, `blr` reads. Confusing the two means patching a serializer that never touches the UI.
**C — function entry**
```asm
stp x29, x30, [x15, #-0x10]!
mov x29, x15
sub x15, x15, #0x20
ldr x16, [x26, #0x38]
cmp x15, x16
b.ls <stack-overflow handler>
```
Use this to find function boundaries when you need to delimit one.
## 9. The locating workflow
1. **Anchor on a string.** Search the pool for the shortest distinctive token — a field name, an
endpoint path, a label. Prefer identifiers over sentences (a sentence may be assembled from
fragments).
2. **Find its referencing instructions** via your index ().
3. **Disassemble a window** around each reference with pool annotations:
```
python dart_disasm.py libapp.so --pp pp.txt --refs pp_refs.json 0x26e390
python dart_disasm.py libapp.so --pp pp.txt --refs pp_refs.json --range 0x64f310 0x64f3e0
```
4. **Classify the site**: read (A) or write (B)? For writes, check whether the value is a constant
from the pool — that is a "send this flag" site, not a "decide locally" site.
5. **Walk outward.** Find the enclosing function entry (§8C), then find its callers:
```
python dart_disasm.py libapp.so --index callers.json --build-index
python dart_disasm.py libapp.so --index callers.json 0x26e230
```
6. **Cross-validate by clustering.** Related strings usually sit **adjacent in the pool** and are
referenced from **adjacent code**. If two field names are a few bytes apart in the pool and their
reference sites are a few instructions apart, you are looking at one coherent piece of logic —
strong evidence you are in the right place.
## 10. Patching this layer
- **Prefer changing the data source over the decision point.** Locate the chain
"read field from map → type conversion → store to stack slot" and replace the middle with a load
of a pool constant. One edit then feeds every consumer, instead of chasing each comparison. This
has produced clean results where per-condition patches did not.
- **Boolean sites** are the cheapest: `tbnz`/`tbz` → `nop` (always fall through) or → an
unconditional branch; `add xD, x22, #0x20` ↔ `#0x30` to flip a constructed constant.
- **A pool string may be shared.** Before editing one, count its references. If the same entry is
read by several sites, changing it changes all of them — self-consistent and therefore useless
for splitting behaviour. Prefer editing code over shared data.
- **Assert your addresses.** Patch with the original bytes asserted first and re-read the result
afterwards; an address that is off by one instruction will happily corrupt a snapshot into
something that still loads.
- **`libapp.so` inside an APK is usually deflate-compressed**, so in-place byte patching of the zip
entry is not possible — patch the extracted file, then replace the whole entry
(`repack-and-sign.md`).
- **Replacing a string in place is safe; replacing the *wrong* string is not.** The highest-value
equal-length target on a server-driven app is a **data key** — a JSON field name, a slot key, a
reporting label. Rename it to equal-length noise and the client looks up a key the server never
sends, so the feature goes quiet while every other request keeps working. This is the cheapest
surgical form of the "do not make an API fail" constraint at the top level of `SKILL.md`: the
request still succeeds, only the client's interpretation of it changes.
- **Never replace an API path or URL string.** It looks like the same kind of string and it is not.
Measured failure, on a real build: replacing the ad-fetch path produced a **404 whose error body is
not valid JSON**, the app's startup flow called `jsonDecode` on it, `FormatException` propagated
out of the Future that builds the home screen, and the app **stayed on the launch logo forever** —
`logcat` showed only `E flutter`. The ad did disappear; so did the app. Isolate it the way that
case did: a **re-sign-only control** (no string edits) plus `adb logcat -d | grep "E flutter"`, so
a startup failure is attributable to the patch rather than to the packer. Full precedent:
`references/precedents/`.
- **Longest string first when the targets are substrings of each other.** In a byte search
`welfare_ad` also matches inside `welfare_ad_top`, and `ad_click:` inside `ad_click:exp`; replacing
the short one first corrupts the longer entry. Sort candidates by descending length, and assert the
prefix byte you expect **immediately before** each hit — on arm64 that byte is `0x80|(len<<1)`, on
armv7 the length is the u32 immediately before the payload — so a hit that lands mid-string is
rejected instead of patched.
## 11. Traps specific to this layer
- **Identifiers are ambiguous across layers.** A pool string that reads like a domain concept can
belong to a standard library or a wire protocol instead. A near-miss example worth remembering:
the token `expires` resolved to HTTP **response-header** parsing (its siblings were `date`,
`host`, `connection`), not to a subscription expiry field. **Read the surrounding strings and the
code shape before assigning meaning** — one misleading identifier can send a whole analysis down
the wrong branch.
- **Obfuscated names are not identities.** Symbol names are mangled (`_abc@12345` shapes are
common) and change between builds. String literals do survive, so anchor on those.
- **Clusters reveal structure.** Adjacent pool entries plus adjacent reference sites indicate one
subsystem; use it to avoid guessing what a function is for.
- **A write-only flag is usually an outbound parameter, not a local switch.** If a string is
referenced exactly once, at a site that stores a constant into a map that is then sent, the client
is *reporting* a value — changing it does not change local behaviour. Confirm whether the value is
ever read before treating it as a gate.
- **Full-file disassembly is not a debugging tool** (). Windowed disassembly is.
## 12. Verification
Static: asserted patch bytes, re-read from the built artifact, plus structural checks
(`verification.md`). Runtime: the changed behaviour must be observed in the app's own UI — for this
layer that means an actual launch, because nothing about Dart AOT code proves itself statically.
If the snapshot drives a decision that a server also enforces, see `membership-and-limits.md`:
getting the client side right does not make the outcome reachable.

225
vendor-new/apk-reverse/skills/apk-reverse/references/desensitization-and-leak-scans.md Normal file
View File

@@ -0,0 +1,225 @@
# Desensitization and leak scans — what must leave, and what must stay
A skill repository is published, and the material feeding it is real work: device transcripts,
packet captures, `ps` listings, log excerpts, an hour of shell history. Target identity leaks
into that material one line at a time — a bundle id in the middle of a `pm path` output, a
device serial echoed before a `dumpsys`, an SDK key pasted while debugging a download — and
**no structural check can see it.** `check_repo.py` validates layout, `check_refs.py` validates
anchors; both pass on a file that names a live app and a real phone.
The failure this file prevents is two-sided, and the second side is the one that gets
forgotten:
- **Under-redaction**: a published repository carries a target's identity, a device serial, or
a credential, and nobody notices until someone greps for it a year later. This is what
happened here once, by hand, after a pass had already been committed.
- **Over-redaction**: the same pass strips the *reusable* material along with it — the tool
names, the hardening-product names, the public crackme names, the protocol field names — and
the document stops being able to teach anything. A repository that has been scrubbed into
uselessness has failed at the same job from the other side.
A rule that only handles the first side produces "a document about nothing". A rule that only
handles the second produces a leak. Both get fixed by the same artifact: an explicit list of
what is *exempt*, in code, that can be read and argued with.
**Load this when:** you are about to publish anything derived from real work -- an evidence file, a transcript, a README -- or a leak scan reports a hit. It gives the must-leave/must-stay split and the scanner's exit semantics.
## Strength labels
- **observed** — a command was run here and its output is quoted in the text or recorded in
the evidence record condensed in `references/evidence-summary.md` §Where the full record lives.
- **inferred** — follows from an observed fact or from documented mechanism; the step itself
was not executed.
- **unverified** — assumed, or reported elsewhere, and not reproduced in this repository.
Claim strength in this file means *how well the statement about desensitization was measured*,
not how confident the writing sounds.
## The two rules, stated so they can be argued with
**1. Redact what identifies the target; keep everything that identifies the technique.** A
bundle id, a device serial, a host user path, an appkey, a live token, a non-loopback endpoint
are identity. A tool name, a library name, a function name, a protocol field name, a CVE id, a
hardening product name, a public crackme name and its URL are *technique* — they are how the
document transfers skill, and they are exactly what a naive redaction pass deletes first.
The test that separates them: **would removing this string make the sentence less able to
teach?** `frida`, `apktool`, `libjiagu`, `UnCrackable-Level1`, `com.stub.StubApp`,
`RegisterNatives`, `proto3` all fail the test — removing them costs knowledge and protects
nothing, because they name a product, a tool or a public fixture. `<PKG>`, `<DEVICE>`,
`<APPKEY>` pass — they carry the sentence's structure and none of its identity. And
`com.<redacted>.app` also passes the test, which is why a scanner must *report* the reverse
domain shape and let a human see what it is; it cannot decide.
**2. A finding is reported with limited context, not a line number.** A report that says
`foo.md:412` makes the fixer reopen the file, read around the line, and decide what to delete —
per finding. A report that quotes ~48 characters either side means the decision is made from
the report itself. This is the same argument `verification.md` §Reporting makes for logs: an
artifact that does not contain the decisive information sends its reader back to the source.
## What must be kept — the do-not-anonymize list
Every entry here has been observed in this repository's own documents, and every one of them
is a wrong answer from a shape-only scanner. The list is materialized in
`scripts/scan_leaks.py` as `BENIGN_PACKAGE_PREFIXES`, `PUBLIC_TARGET_PREFIXES`,
`BENIGN_IP_PREFIXES` and the `CODE_IDENTIFIER_HINTS` table, so it can be extended with a
one-line edit and audited with `--show-exempt`.
| Category | Examples | Why it is exempt |
|---|---|---|
| Tool and library names | `frida`, `apktool`, `jadx`, `libart.so`, `libfoo.so` | naming a tool is the content; the name is public and identical for every reader |
| Function and symbol names | `RegisterNatives`, `findExportByName`, `pthread_mutex_lock`, `JNI_OnLoad` | API vocabulary, not identity — and a large fraction of the reverse-engineering technique is exactly these names |
| DEX/ELF constant identifiers | `findExportByName`, `ImmutableDexFile`, `installPackageLI`, `LoadPackageParam` | 16-character camelCase runs that a serial-shaped regex matches; they are code constants |
| Protocol field names | `proto3`, `varint`, `packed repeated`, `length-delimited` | wire-format vocabulary |
| CVE identifiers | `CVE-2024-31317`, `CVE-2021-44228` | public advisory ids; redacting them destroys the reference |
| Hardening product names | `com.stub.StubApp`, `libjiagu.so`, `libDexHelper.so`, `com.secneo` | a *product* signature. It is what lets a reader recognise a shell, and it names no target |
| Public crackme / benchmark targets | `UnCrackable-Level1.apk`, `sg.vantagepoint.uncrackable3`, MASTG and its URLs | deliberately published fixtures; their package names are part of the published exercise |
| Platform and SDK packages | `com.android.settings`, `com.google.android.gms.ads`, `com.qq.e.ads.PortraitADActivity`, `androidx.work.WorkManager` | inventory. "Which SDK does this app carry" is a finding; the SDK's own name is not a secret |
| This repository's own fixtures | `com.example.*`, `probe.synthetic.*`, a probe module's package | synthetic by construction |
| Placeholders | `<PKG>`, `<DEVICE>`, `<serial>`, `<hash>`, `<work>`, `<user>`, `C:\Users\<user>\...` | the redaction mechanism itself; a scanner that reports these is unusable |
| Loopback / unspecified / emulator addresses | `127.0.0.1:27042`, `0.0.0.0:8080`, `10.0.2.2:8080`, `169.254.169.254` (as a documented endpoint) | the frida and ADB idioms are these addresses; they identify nothing |
**Deliberately *not* exempt, and observed as a finding here**: the RFC 5737 documentation
ranges (`192.0.2.0/24`, `198.51.100.0/24`, `203.0.113.0/24`). They are the correct way to write
a synthetic address, so a hit there is a signal rather than noise. Suppressing the range outright
would also hide the case where someone pastes a real address that happens to look synthetic.
This repository pays for that decision with the *only* findings it produces, and the price is
recorded rather than argued away: **four `endpoint/weak` hits and nothing else**, measured on the
tracked surface with file count 129. Three of the four are the three blocks named in the sentence
above — this file stating the decision is itself the hit — and the fourth is
`scripts/tls_check.py`'s own usage example, a benign context the report makes obvious in one line.
Every other category is at zero. That ratio is why the gate's pass condition lives in
`--fail-on` and not in an exemption table: the documentation of the decision *is* the residue, so
a gate that failed on it would be failing on its own rationale. The transcript is in
`references/evidence-summary.md` §The capability matrix.
**What is a leak here, in contrast** — the shapes worth reporting: `package="…"` in a manifest
line, a bundle id after `pm path` / `pidof` / `ps -A` / `component=`, a 16-character
`[A-Z0-9]` run standing alone or beside a device word, `github_pat_…`/`ghp_…`, an inline
`APPKEY`/`appSecretKey`/`SECRET_KEY` assignment with a literal value, a non-loopback `IP:port`,
and `/home/<name>/`, `/Users/<name>/`, `C:\Users\<name>\`.
## What a scan can and cannot decide
Decide in code: whether *this shape* is present, and whether a known-benign pattern explains
it. Never decide in code: whether the surrounding document is identity-bearing.
Three consequences, each observed by writing the scanner for this repository:
- **A range of four digits separated by dots is an IP address only if nothing nearby says
"version".** `JDK 17.0.4.1`, `build-tools 34.0.0` and `frida 16.7.19` are the shapes the first
version of the scanner reported twenty times across the evidence record condensed in `references/evidence-summary.md` §Where the full record lives. Context
(`jdk`, `python`, `frida`, … immediately before the match) separates them.
- **A 16-character uppercase token is a device serial only if it carries a letter.** Stack
addresses from a tombstone (`0000000040001000`) and hex protocol fixtures
(`0807060504030201`) are all-digit and were reported until that rule was added.
- **A reverse-domain string preceded by `args.` is a property lookup.** `args.package.split(".")`
in a script — observed in `scripts/spawn_patch_detach.py` — is not a bundle id, and the
exemption needs the *line*, which is why the exemption function takes it.
Each of those is a **false positive that a reader would have had to investigate**. Three of them
were found by running the scanner against this repository, not by reasoning about it — which is
the argument for making the scan a gate rather than a document.
## The scanner, and where it belongs in the pipeline
`scripts/scan_leaks.py` — standard library only, `--root` explicit or defaulting to its own
repository, categories `package` / `device` / `token` / `appkey` / `endpoint` / `path`, text and
JSON output, and a fixed exit-code contract:
| Exit | Token | Meaning | What a script does |
|---|---|---|---|
| 0 | `RESULT=clean` | nothing outside the exemption list | continue |
| 0 | `RESULT=leaks_found_strong_only` | findings exist, but every one of them is `weak` (an address from the documented range) | continue, and read the report — it is the publish-time checklist |
| 1 | `RESULT=leaks_found` | at least one `strong`/`certain` finding, or any finding under `--fail-on any` | fail the job and print the report |
| 2 | `RESULT=error` | bad usage, unreadable root, or a read error with no findings | fail the job as an infrastructure fault, not a content fault |
The token is the machine-readable line; the exit code is the interface. Keeping them separate
matters because a wrapper that only greps stdout cannot distinguish "clean" from "the scanner
never ran", and a wrapper that only checks `$?` cannot explain itself in a log.
Which findings fail the gate is chosen by `--fail-on`, and the two settings are two different
questions:
- `--fail-on strong` (default) — only `strong`/`certain` findings fail. This is the maintenance
habit's setting, and it exists because of the RFC 5737 decision above: if the documented ranges
are reported on purpose, this repository is never `clean`, and a gate that is red on its own
rationale is a gate people learn to disable.
- `--fail-on any` — every finding fails, `weak` ones included. This is the publish-time setting,
and the one to run against the file you are about to publish rather than the tree you are
working in.
Neither setting changes what is *reported*: weak findings are printed under both, and only the
exit code differs. The strict pass is therefore a decision recorded in a command, not a surprise.
Placement, and the reasoning is about *when* the wrong answer is cheap:
- **Not in the pre-commit hook alone.** A commit-time gate is the cheapest place to catch a
leak, but this repository's evidence files are written by long passes that commit once at the
end; a hook would fire on a tree nobody has finished writing.
- **In the maintenance gate, next to the other two checkers.** The habit that already exists is
`python check_repo.py && python check_refs.py` before a commit; a third line costs nothing and
runs at the moment the material is about to become permanent. Its exit 1 is a *content*
finding to read, not an error to retry — unlike the two checkers, whose failures are
mechanical.
- **Before publishing an evidence file**, once, by hand, with `--show-exempt`. The exemption
audit is where the interesting mistakes are: a value suppressed for the wrong reason looks
exactly like a value that was never there.
- **On the report itself, after it is written and before it is published.** A report that quotes
the hits in order to be concrete is a *copy* of the leak, in a file that is tracked and read.
This is not hypothetical here: the first version of the desensitization extension record did
exactly that, and the scanner flagged its own evidence file on the next run. Run the scan on
the artifact you are about to ship, which for an evidence file means the file itself.
- **In CI, on the published tree only.** A scanner run over a git-ignored work area will report
the work area — which is where the target identity legitimately lives — and a gate that cries
wolf on every run is a gate that gets disabled. `tools/` is excluded by default for exactly
this reason; a caller who points `--root` at it means it, and gets the findings.
**What the scan does not replace.** It cannot see identity that is *described* rather than
*quoted* ("a mid-size Flutter app with a 360 shell"), and it cannot see a value that has been
partially masked on purpose. Neither is a defect to fix by adding patterns: the first is why
this repository's evidence files open with a statement that target identity is absent, and the
second is a judgement call a human makes once and writes down.
## Failure modes
| Symptom | Mechanism | What to do |
|---|---|---|
| The scanner reports a hundred hits, none of them a leak | shape-only rules against a document full of legitimate identifiers (versions, hashes, symbol names) | Do not widen the exemption list one hit at a time. Run `--show-exempt`, group the reasons, and fix the *rule* — a context-free regex is the bug |
| The scanner reports nothing and you believe it | a root that excluded the directory the leak was in; a text file with an extension the walker skips; a value inside a fenced block the reader never scrolled to | Read the header line — it prints the file count and the root. Then point `--root` at one directory and confirm the count is plausible |
| A real leak was suppressed | an exemption matched a substring (`<pkg>` inside something else), or the benign-prefix list is too broad (`com.`-prefixed platform list catching a target that starts with the same two segments) | `--show-exempt` prints the *reason* per suppressed hit. An empty or vague reason is the defect |
| Same hit reported twice on one line | two rules whose scopes overlap (`SECRET_KEY=` is both a token and an appkey) | Expected, and left in deliberately: two independent rules naming one value is corroboration. Exact duplicates (same rule, same value, same line) are collapsed |
| The gate is green in CI and the leak is in the tarball | CI scans the tree; the artifact was built from a work area, or from a branch that was never scanned | Scan the directory the artifact is built from, with the same exclusions the build uses |
| A `<PKG>`-style placeholder is reported | the placeholder uses a different bracket, or no brackets at all | Add the token to `PLACEHOLDER_TOKENS`. This is the one exemption class that should grow freely — placeholders cannot leak |
| The published report contains the leak it was written about | the report quotes each hit value in order to be concrete, and the report is the published artifact — a report is a copy of what it found | Never quote a hit value in a published document. Keep rule, category, strength, position and count, and describe the value by shape (`<reverse-domain>.<n>`, `16×[A-Z0-9]`, `24×[0-9a-f]`) or with a placeholder. Scan the file you are about to publish, not the file you started from |
## The entry-point surface
A published skill is not only read by an agent; its **entry files are read as instructions**.
`SKILL.md`, a `README.md`, and the front-matter of any reference are the parts a host process
loads *before* it decides anything, and a repository assembled from third-party material — a
borrowed reference file, a contributed script — carries whatever those files say. This
repository's convention, observed in its own history: content reproduced from elsewhere is
summarised as prose with the source named, never quoted verbatim in a way that would land in an
instruction position, and a contributed file is read before it is linked.
The practical consequence for redaction is the same as for the rest of this file: **the
redaction must not be reversible by asking the file.** A placeholder that is defined next to
its own value, a "sample transcript" that still carries the serial in a comment, or a finding
report pasted into a document that later becomes an entry point all reintroduce the identity
that a scanner would have caught — because the scanner ran before the report was written. Run
the scan on the file you are about to publish, not on the file you started from.
## Cross-references
- `references/long-task-discipline.md` §Keep a live record, not a log — the record is written
during the work, so it is where identity lands first, and the file most likely to be published
by accident.
- `references/verification.md` §Reporting — the evidence/report split this file mirrors: what
ships is neutral and mechanical, what identifies the target stays in the analysis.
- `references/precedents/README.md` — the case library; its "write back to the repository"
checklist is where a desensitization finding becomes an index line.
- `references/third-party-builds.md` — the same do-not-anonymize reasoning applied to a sample
you did not produce, where the identity is someone else's.

349
vendor-new/apk-reverse/skills/apk-reverse/references/detection-and-anti-analysis.md Normal file
View File

@@ -0,0 +1,349 @@
# Detection, anti-analysis, and when to stop fighting it
Load this when the app fights back: it dies or misbehaves after you attach, refuses to run on your
device, detects root/emulator/hook/debugger, or when your dynamic tool simply will not work on the
environment you have.
This file is a **decision** file, not a bypass catalogue. The expensive mistake it exists to prevent is
spending hours escalating against a detection layer when a cheaper route — usually static — was
available the whole time.
## Step 0: the cheapest decision in this file
**Dynamic analysis is a convenience, not a prerequisite.** Everything that ends in an installable
artifact is decided statically: you edit a file and repackage. Dynamic work is how you *find* things,
and it is often the fastest way — but when it is blocked or unavailable, the correct move is usually to
switch to static, not to escalate.
So before doing anything clever, ask:
1. Is the thing I still need to learn actually discoverable statically? (Strings, code structure,
call sites, comparisons — usually yes.)
2. Do I need the process to run *to verify*, or only to *find*? Verification can often be done by
observing the shipped artifact's behaviour, which a hostile process still exposes.
If either answer allows a static path, take it and stop reading this file.
## Step 1: is it detection, or is it your environment?
Detection has a distinctive shape: **it is reproducible, tied to your instrumentation, and absent in the
same app run without it.** Everything else that looks like detection is usually one of these:
| Looks like detection | Usually is | How to tell |
|---|---|---|
| app dies right after attach | hook timing, a bad script, or a genuine RASP layer | run the same app with **no** hooks: alive? then it is your instrumentation or a hooking detector |
| app refuses to start at all | ABI mismatch, missing native lib for this device, install problem | check the live mapping (`scripts/lib_map.py`) and that a clean unmodified build starts |
| app exits on this device only | emulator/root detection, **or** a device-state problem | run preflight; try the same build on a different device class |
| app hangs forever | a frozen thread from a bad patch, or a real ANR | `pitfalls.md` P7 and the "never make it not return" rule |
| dynamic tool "cannot attach" | **the tool cannot work on this environment at all** | see Step 4 — this is the one people waste days on |
**Rule: reproduce with the control first.** Instrumentation-free run, then instrumented run, same
build. That single comparison separates "the app detects me" from "my tooling is broken", and it takes
one round.
## Step 2: if it *is* detection, decide by cost, not by pride
Three legitimate outcomes. Pick one deliberately and say which you picked.
**A. Work around it — only when the workaround is small and stable.**
Worth it when the detector is a simple, localised check you can neutralise in one place, and the
workaround lives in *your* environment rather than in the shipped artifact. Examples of the class:
running on a device the app does not object to, using a build that satisfies the check, or a single
one-line gate in a helper. Also legitimate: naming the artefact differently, using a different device
profile, or simply doing the work on a device that the app accepts.
**B. Change route.** When the workaround is a moving target, switch to static analysis and stay there.
This is the right answer more often than it feels like. Hooking-detection especially tends to escalate:
each hide provokes a stronger check, and you end up maintaining a cat-and-mouse setup that is worth
nothing at delivery time.
**C. Accept and report.** If the detection blocks the *deliverable* rather than your analysis — for
example, the app verifies something the user's device will not have — then the honest output is a
statement of what is blocked, with the evidence. Do not ship a build whose only purpose is to defeat a
detector you were not asked about.
**Signal to stop escalating:** you have spent more effort on the analysis environment than on the
change the user asked for. That is the drift this skill treats as most expensive, in its runtime form
(`long-task-discipline.md` §the most expensive drift).
## Step 3: locating the check — the order of search
Step 2 decides *whether* to fight. This step is the part the two steps around it do not provide:
**once you know a check fired, in what order do you look for it?** The order matters because the
cheap probes answer most cases and the expensive ones only pay off after a cheap probe has already
narrowed the field. Every stage below names its input signal, the smallest action that answers it,
the evidence that makes it true rather than plausible, and where to go when it does not.
Two shaping notes before the stages.
**Order is the whole content here.** Reading this as a menu and starting at the middle is how a
session becomes an arms race: each probe you add changes the process further, so the later probes are
measuring a target you already modified. Do them in order and stop as soon as one answers.
- **Stage order is a funnel, not a checklist.** Stage 1 is five minutes and rules out whole classes of
problem; Stage 5 requires the offsets Stage 2 produced. Skipping forward costs more than it saves.
- **Every stage ends in `observed` or in a stated negative.** "No hook fired" is only a fact once you
have shown the hook can fire on this target (`pitfalls.md` P25/P26 in tool form).
**Strength note.** The stages are a restatement, for judgement purposes, of a six-phase
anti-instrumentation pipeline published in `index-login/MobileRE-Skill`
(`https://github.com/index-login/MobileRE-Skill`, retrieved 2026-09). That project's phase *modules*
are not vendored here and none of its scripts are used; what is taken is the **order** and the
branch conditions, which are the parts that transfer. Stages 2, 3, 5 and 6 are **observed** as
mechanisms on this repository's own device (evidence and exact commands:
`references/evidence-summary.md` §The capability matrix). Stage 4's kill path was reached once and
is unstable across runs, so treat its detail there as `observed`-once, not reproducible.
Stage 1 is `observed` and is the one that most often ends the investigation without an escalation.
### Stage 0: rule out the two non-detection causes first
Before hooking anything: is the process alive at all, in what state, and does it die the same way
with nothing attached?
```bash
adb shell 'cat /proc/<pid>/status | head -4' # state matters: D is uninterruptible sleep
adb shell 'cat /proc/<pid>/stat | cut -d" " -f3'
```
**Observed failure that looks exactly like detection:** a target in **`D` (uninterruptible disk
sleep)** makes an attach hang and then fail, and makes a second attach report *"process not found"* —
while the process is still in `ps`. No detector is involved; a userspace attach needs the process to
run. On the measured run this was the actual cause, and it was read as "the app refuses
instrumentation" for one round. Check state before you blame the target, and check **whether attach
works on any other process on the same device** — that one-line control separates a broken toolchain
from a hostile target in a single command.
### Stage 1: a benign probe, to establish that your pipeline can fire at all
**Input signal:** nothing yet. **Action:** load an *observer-only* probe
(`scripts/anti_detect_probe.js`) that patches nothing. **Evidence it worked:** the probe's own
`armed` line and its environment self-report — `TracerPid`, which of your artefacts are visible in
the process's own maps, whether it can read `/proc/net/tcp` at all. **If nothing arrives:** the
problem is plumbing or liveness, not detection. Re-read Stage 0; do not escalate.
**The self-report is the cheapest half of this file's whole subject, and it is usually skipped.**
Before asking "does it detect frida", ask "is frida visible to this process". Measured on the test
device, a probe spawned into a target read 4 frida-named mappings in the target's own
`/proc/self/maps` before any check ran — so "was it detectable" was never in doubt, and the
remaining question was only which check reads it.
**Stream, do not batch, on a target that self-destructs.** A probe that reports only at the end of a
window reports *nothing* when the target dies inside the window — precisely the run that mattered.
Measured: three spawn-and-probe arms of one target; one delivered a full event sequence ending in a
self-destruct, the next two lost the script before its first timer fired. Emit a heartbeat and stream
each first hit. The heartbeat's absence at time *T* is itself the datum: it separates "the target was
quiet" from "the script was gone before *T*".
### Stage 2: name the detector, by caller offset
**Input signal:** Stage 1's environment events. **Action:** hook the path-access and process-control
surface (`open`/`openat`/`fopen`/`access`/`stat`/`readlink`, `dlopen`/`dlsym`, `pthread_create`,
`kill`/`tgkill`/`exit*`) and report, for each call, the **caller module + offset**. **Evidence:**
`libX.so+0x1cef8` — a named module and an address. **If no caller resolves:** you are on a runtime
that cannot unwind here; record the module and move to Stage 6's static path instead of adding a
third unwinding strategy.
This stage's product is a **name**, and everything after it is cheaper. Two branch conditions decide
which Stage 4 branch you take, so they are worth reading off explicitly:
| What Stage 2 shows | Branch | Why |
|---|---|---|
| the check runs on a thread the detector created itself (`pthread_create`/`clone` caller is the detector module) | **block at creation** | the polling loop never exists to fire |
| the check runs inline inside a library constructor, and the calls you can see are only libc wrappers | **Stage 3** | under that ordering a libc hook is the wrong layer |
| the detector is a *shell* library that also decrypts | **do not patch broadly** | the same module carries load-bearing work; scope to named offsets |
### Stage 3: if it fires before your hooks exist, take the constructor order
**Input signal:** the death happens before or during first library load, or Stage 2's caller offsets
all sit in a constructor path. **Action:** find the constructor dispatcher and hook it *before*
constructors run. **Evidence:** the module's `init_array` entries are visible and their addresses
map to the checks. **If the device's linker does not expose the symbol:** switch to the static route
(`scripts/svc_scan.py` on that library) rather than guessing an address.
```bash
# the symbol name is mangled; grep for the substring, not the bare name
readelf -sW <linker64> | grep call_constructors
```
**Measured:** on this repository's device the symbol exists as
`__dl__ZN6soinfo17call_constructorsEv` at `0x4a258` — a **mangled** name. A search for a bare
`call_constructors` export reports "absent" and sends the analyst to an offset-guessing path that
was never necessary. Confirm presence/absence before building either route.
**The measured cost of getting this stage wrong:** do not clear a module's whole `init_array`. A
shell library's constructor does payload decryption as well as detection; emptying it converts a
detectable app into a non-running one, which is a worse result than the detection was. Scope to
functions you have a reason to suspect — a per-function static scan for the syscall form in Stage 6
is how you get that reason.
### Stage 4: keep the process alive — deliberately, and with a stop condition
**Input signal:** every attempt shows events up to a point and then the process is gone. **Action:**
make the termination observable (record path and caller) and, if necessary, prevent it. **Evidence:**
the death is attributed to a module and an offset, not to "the platform". **If attributing it costs
more than three arms, stop attributing and switch to static.**
Two measured shapes, and they need opposite treatment:
- **A libc-visible kill.** Measured on a clean MASTI-style target: a probe saw
`strstr("frida")` at ~300 ms and then the process was gone, with **no** tombstone, no crash and no
ANR record — a clean self-exit. The abort path itself surfaced once as a libc `raise`/signal call;
it did not surface on the later arms that died sooner. That instability is the finding: **on a
fast detector the observable window is sub-second and not reproducible**, so a single successful
attribution is not a pipeline.
- **A libc-invisible kill.** If a module issues `svc #0` itself, no libc-level hook can see it, and
the absence of an event beside a death is the signal to go static (Stage 6).
**`exit_blocker`-style keepalive has a failure mode that looks like success.** Blocking the exit
turns a clean death into a spinner: the process stays alive, stops responding, and you can no longer
tell "the check is neutralised" from "the app is wedged". Keep the blocked-call count, a timeout, and
one behavioural check — does the UI still reach the state you care about — or the keepalive is
measuring itself.
### Stage 5: shellcode and runtime-generated code
**Input signal:** Stage 2's loader stream shows executable mappings appear that were never loaded
from a file. **Action:** watch allocation with execute permission and read-only-executable
transitions; disassemble around the allocation's caller. **Evidence:** an `mmap`/`mprotect` call with
execute permission whose caller offset is in the detector module. **If the region is unreadable:**
stop; the offsets from Stage 2 are all a static session needs.
A target can make itself harder to look at without doing anything exotic: on one measured run the
recovered caller chain sat inside `jit-cache`, i.e. **Java code the runtime had JIT-compiled**, not a
shipped library. A crash frame in JIT output cannot be mapped to a file offset. Expect it on any
API where the framework compiles reflection at runtime, and treat "the frame has no file" as a fact
about compilation, not as a hidden module.
### Stage 6: the precise stop, and the static alternative to it
**Input signal:** a named module + offset from Stage 2, Stage 3 or Stage 5. **Action:** neutralise
that site — or decide not to. **Evidence:** a fresh control run with the site left alone dies where
it died before, and the patched run does not. **If the offset came from a trace you could not
reproduce:** do not patch it; go to the static route below.
The static route is often strictly better than the dynamic one and needs no live process:
```bash
python scripts/svc_scan.py /data/local/tmp/libDetect.so
```
**Measured discriminator.** A module that contains **no** inline `svc` site for a termination syscall
cannot be bypassing libc for its kill — so a missing exit event is then a finding about your hook,
not about the target. Measured on this device: the ROM's `libc.so` carries exactly 4 termination
sites (`exit`, `exit_group`, `kill`, `tgkill`), and those are libc's *own* exported implementations —
so hooking `exit`/`kill` really does see callers that go through libc. A shell library on the other
hand showed 21 byte-scan "svc sites" that were all **data**, with no call-number load anywhere near
them: a byte scan for `svc` matches inside data, so read the neighbours before believing a count.
That distinction — structural absence vs. observed absence — is what Stage 6 is for.
### The routing chain, on one page
```
Stage 0 process state + does attach work on ANY process? -> D-state / broken toolchain, not detection
Stage 1 observer-only probe + environment self-report -> what the target can see
Stage 2 path/loader/thread/kill hooks with caller offsets -> a module+offset name
|-- caller is a created thread -> block at creation (Stage 4, branch A)
|-- death precedes hooks, or the checks are in a loader ctor -> Stage 3, or static (Stage 6)
Stage 3 constructor order / mangled symbol check -> the check is reachable at all
Stage 4 attribute the kill, then decide whether to block it -> attributed death, or "unreproducible"
Stage 5 executable mappings not backed by a file -> shellcode / JIT, often a dead end
Stage 6 NOP one site, or prove statically that libc is enough -> observed before/after
```
## Step 2B: two boundaries this file adopts
**(a) Observers and interceptors are separate modules and stay separate.** An observation module must
not change behaviour, and an intervention module must not double as a monitor. This is the discipline
the pipeline above is built to survive: Stage 1's value is that its output describes the *original*
process, and the moment the same script also blocks an exit or NOPs a function, every later
observation belongs to a target that no longer exists. The practical version: a probe that watches
and a patch that neutralises are two files, loaded in two runs, and the runs are the experiment.
`scripts/anti_detect_probe.js` is the observer half and patches nothing by contract.
Be honest about the leak in that contract: **hooking is itself observable.** Attaching changes
timing, and on a hardened target the first event the probe recorded was a `dlopen` of the target's
own code followed within ~50 ms by a check — i.e. the act of being watched is part of the
environment the target is reacting to. Separation makes runs comparable to each other; it does not
make them a description of the untouched process.
**(b) Name the cost before you add another module, and know when to leave.** Each avoidance module
is another thing the target can notice and another reason a later result is unattributable, and the
detection is not standing still: a check that was a string compare becomes an inline syscall becomes
a native check in a module you have not looked at. Track three numbers, and stop when two of them
are true: how many bypass modules are loaded, whether the run needs a keepalive to stay up at all,
and whether the target only behaves under the full stack of modules. **When they are true, switch to
static** — that is the same conclusion as Step 2B, reached with a threshold instead of a judgement
call, and it is the exit this whole step needs so that Stage 6 does not become a permanent home.
## Step 4: when the tool cannot work here at all
Some environments cannot run a given dynamic tool, and no amount of trying will change that. Recognise
it and move on — this is a *finding about the environment*, and recording it prevents the next person
from repeating the attempt.
**The instructive class: an emulator that executes a different architecture than it reports.** A device
may advertise one ABI while the process runs libraries of another through a translation layer. This
shows up as a tool that reports the device is fine, then fails to attach, attach-and-die, or attach and
see nothing. Symptoms worth treating as a hard signal:
- the tool's own attach path fails with an error that mentions tracer, ptrace, or a debugger — on an
environment where translation is in play;
- attaching kills the process immediately, repeatedly, with no Java stack;
- the tool attaches but **no** hook ever fires, including hooks on code you know runs.
**Do not spend the session proving it is impossible.** Establish it once, with one clean reproduction,
write the environment fact down, and switch to static.
**What to do instead:** the entire static toolchain is architecture-agnostic — string tables, code
structure, call-site counts, byte-level patching. None of it needs the app to run. If your verification
also needed the app to run, use the device-level observables instead: does the UI change, do the logs
fall silent, does a file appear or stop appearing. Those are properties of the shipped artifact, and they
do not care that you could not attach.
## Step 5: what a detection layer means for the deliverable
Two distinct questions — keep them separate, because conflating them produces either a broken artifact
or an unnecessary retreat.
1. **Does it block your analysis?** → handle with Step 2/3/4. This is your problem, and it is temporary.
2. **Will it block the patched build on the user's device?** → this is a property of the artifact. A
root check that merely requires an unrooted device is usually irrelevant to a repackaged APK. A
**self-integrity or signature check** is a different matter entirely, and it belongs to
`references/native-tamper-and-suicide.md` and `references/signature-derived-keys.md`.
Ask explicitly: *does this check fire because of how the app is built, or because of where I am running
it?* Only the first kind follows your build into delivery.
**Root is an environment fact, not a defeat.** "Runs only on a rooted device" is a legitimate analysis
environment and an illegitimate deliverable when the request was an installable build for a normal phone.
Say which one you actually have.
## Step 6: keep a one-line environment fact
When you conclude that something cannot be done in this environment, write one line in the task record:
```
cannot: <tool/technique> in <environment> because: <observed failure> route taken: <alternative>
```
That single line is what saves a future round. It is also the difference between "we could not attach,
so we did X" and an unqualified "dynamic analysis is impossible", which is a claim you have not earned
and which will mislead the next reader.
## Checklist
- [ ] **Located before escalating:** Stage 0 state check, an attach control on another process, then the
stages in order — and stopped at the first stage that answered
- [ ] **Observer and interceptor are not in the same script**, and the observation run was kept free of
any patch
- [ ] Detector named as **module+offset**, or the negative stated ("no caller resolvable on this runtime")
- [ ] **Cost measured, not felt:** bypass-module count, whether a keepalive is required, whether the
target only runs under the full stack — two of three true means switch to static
- [ ] Controlled the comparison: same build with and without instrumentation
- [ ] Preflight run, so device state is excluded before blaming a detector
- [ ] Classified: detection vs environment vs my own tooling
- [ ] Chose A/B/C deliberately and can say which, and why
- [ ] If the tool cannot work here: established it **once**, recorded the fact, switched route
- [ ] Separated "blocks my analysis" from "blocks the deliverable"
- [ ] Nothing in the shipped artifact exists solely to fool a detector
- [ ] Recorded the one-line environment fact if a route was closed

204
vendor-new/apk-reverse/skills/apk-reverse/references/dex-patching.md Normal file
View File

@@ -0,0 +1,204 @@
# Dex patching — choosing and executing the surgical edit
The single most important decision in a patch task. Get this wrong and you produce an APK that assembles perfectly and dies at runtime.
**Load this when:** you have located the class and method to change and must choose the edit technique. It gives the least-destructive-first order and the signal that says a byte patch is not enough.
## Pick the least destructive technique that can express your change
Ordered from safest to most dangerous.
| # | Technique | Expresses | Structure risk | Use when |
|---|---|---|---|---|
| 1 | **dexlib2 method-level rewrite** | Replace a method's implementation (return a constant, no-op, emit a fixed value) | **Lowest** — only the target `code_item` changes | Default choice. Any behavior change that can be expressed as "this method now does X instead" |
| 2 | **Byte-level string constant patch** | Change a string literal to another of **equal length** | Medium — must preserve `string_ids` ordering (see P2) | Renaming a path/key/protocol token |
| 3 | **Resource / asset edit** | Change a config file, JSON, image, or bundled jar | Low (no dex change) | The behavior is data-driven |
| 4 | **Manifest edit** | Disable a component, drop a permission, flip a flag | Low, but re-signing required | Turning off a service/receiver/activity |
| 5 | **Whole-tree smali round-trip** | Arbitrary code edits | **High** — breaks R8 output (see P3) | Last resort, and verify by running |
**Do not reach for #5 because it is familiar.** It is the most likely to produce a broken build.
## Where to patch: pick the layer, not the symptom
Ads and gates can be suppressed at several layers. Higher in this list = smaller blast radius.
| Layer | What it looks like | Example | Risk |
|---|---|---|---|
| **SDK initialization** | A single helper that calls `Sdk.init()` | `AdHelper.k(Application)` | Low — SDK never starts, nothing else depends on it |
| **Feature entry point** | The app method that shows the thing | `AdHelper.showSplash()`, `showReward()` | Low–medium — must preserve callbacks the caller awaits |
| **Data consumption** | Where a server list is filtered/rendered | filter out a `position` before it reaches UI state | Medium — must not break sibling data |
| **Renderer** | The composable/view for a **specific** element | a dedicated ad card composable | **High** — generic components are shared (see P6) |
| **Transport** | Blocking/repointing an endpoint | `/app/adverts` → dead path | **Very high** — shared endpoints take screens down (see P5) |
**Rule:** prefer the highest layer (closest to the SDK/feature) that produces the required effect. Never go to the transport layer to hide a UI element.
## Before you patch a method: blast-radius check
```bash
python scripts/find_refs.py <smali_tree|dex|dir|apk> 'Lcom/pkg/Helper;->methodName(args)RetType'
```
Read the `[scanned]` line before the count. It reports how many inputs were actually opened, and
`[scanned] 0 file(s)` exits 2 with the reason: an unsupported path, a typo, or a directory of archives
is **not** the same answer as "read 2 dex files and found nothing", and the two once printed
identically — which put a false zero on exactly the decision this check exists to protect. A dex input
is decoded directly (`dexutil.py`), so baksmali is not required.
- **1–3 callers, all in the same feature area** → safe to patch.
- **Many callers, or callers across unrelated packages** → it is a general utility. Do not patch it. Go one level up and patch the specific caller instead.
- **Look at the parameters too.** If the signature mentions `Modifier`, `ContentScale`, `ColorFormat`, `Shape`, `View`, or a content-generic model type, it is not specific to your target.
Real example: a composable `(AdModel, ColorScheme, Modifier, Function0, ContentScale, Shape, Composer, II)V` looked ad-specific because of its first parameter. It was actually the shared image-card renderer; neutering it removed all cover art and broke playback.
## Technique 1: dexlib2 method-level rewrite (preferred)
Concept: load the dex, find the target class, replace **only** the target method's implementation, write a new dex. Nothing else is touched.
See `scripts/dexpatch/` for a working Java implementation and build instructions.
Shape of the code:
```java
DexFile dex = DexFileFactory.loadDexFile(new File(in), Opcodes.forApi(API));
List<ClassDef> out = new ArrayList<>();
for (ClassDef cd : dex.getClasses()) {
List<Method> direct = new ArrayList<>(); for (Method m : cd.getDirectMethods()) direct.add(m);
List<Method> virtual = new ArrayList<>(); for (Method m : cd.getVirtualMethods()) virtual.add(m);
boolean touched = false;
// ... for the target class: find the method, build new instructions, list.set(i, newMethod)
out.add(touched
? new ImmutableClassDef(cd.getType(), cd.getAccessFlags(), cd.getSuperclass(), cd.getInterfaces(),
cd.getSourceFile(), cd.getAnnotations(), cd.getStaticFields(),
cd.getInstanceFields(), direct, virtual)
: cd);
}
DexFileFactory.writeDexFile(out, new ImmutableDexFile(Opcodes.forApi(API), out));
```
### Practical details that decide success
- **`direct` vs `virtual`.** `static`, `private`, and constructors live in `directMethods`; everything else (including `public final`) lives in `virtualMethods`. **Scan both** when searching. A "method not found" that is actually "in the other list" wastes a lot of time.
- **Preserve the register count.** Keep `m.getImplementation().getRegisterCount()`. `registers` must be `>= parameterRegisters`, and parameter registers are the **highest-numbered** ones: with `registers = N` and `k` parameters (including `this` for non-static), `p0 = N - k`.
- **Method parameters** come from the prototype. For `(Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;` on an instance method, `k = 3` → `p0=3, p1=4, p2=5` when `registers = 6`.
- **`const-wide` needs the 64-bit instruction form.** `ImmutableInstruction51l`, not `31i`.
- **Instruction register operands in `35c` form** (invoke) are `(opcode, registerCount, regC, regD, regE, regF, regG, ref)`. `invoke-static {v1, v2}, ...` → `registerCount=2, regC=1, regD=2`.
- **Every method body must end with a return of the right type.**
- **One dex, one write pass.** If you need two edits to the same dex, do them in the same program (see P4).
- **Null `tryBlocks` / `debugItems` on the rewritten method is fine** — you are replacing the whole body.
### Common rewrites
**Return a boolean constant** (`Z`):
```
const/4 v0, 0x1
return v0
```
`registers` = original count is fine; `const/4` needs a 4-bit register (`v0`–`v15`).
**No-op** (`V`):
```
return-void
```
**Return an object/empty collection**: build the constant, then `return-object`. For `java.lang.Long`, `const-wide` + `Long->valueOf(J)` + `move-result-object`.
**Coerce a suspend/Flow lambda to always yield a fixed value** — this is what "make a local flag permanently true" usually looks like in practice. See the worked example below.
### Worked example: permanently "not expired" without touching a shared helper
**Situation.** A local preference (`<key>_expires_at`) gates a promo popup. Writing a large value into the datastore works on the current device but is **runtime data**, so a fresh install loses it (see P11). The read path is a Flow built at construction time, so the getter cannot be patched meaningfully, and the generic `Long.valueOf` wrapper has 30+ callers (see P6).
**Target.** The dedicated map lambda, e.g. `feature/data/SomeStore$currentX$$inlined$map$1$2`:
```
.method public final emit(Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
iget-object v0, p0, L<same class>;->b:Lkw2; # downstream collector
const-wide v1, <big value>
invoke-static {v1, v2}, Ljava/lang/Long;->valueOf(J)Ljava/lang/Long;
move-result-object v1
invoke-interface {v0, v1, p2}, Lkw2;->emit(Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
move-result-object v0
return-object v0
.end method
```
Note the shape: the lambda class holds the downstream collector in a field (here `b:Lkw2;`). Always read that field name from the actual smali — do not assume.
This class has exactly one purpose, so patching it is safe.
## Technique 2: byte-level string patch
Only for **equal-length** replacements. Must preserve `string_ids` ordering.
```bash
python scripts/dex_strpatch.py <in.dex> <out.dex> "<old>" "<new>"
```
The script:
1. requires `len(new) == len(old)`,
2. requires exactly one occurrence,
3. looks up the entry's neighbours in `string_ids` and **rejects** the replacement unless `prev < new < next`,
4. recomputes `signature` (SHA-1 over bytes from offset 32) and `checksum` (adler32 over bytes from offset 12).
If a candidate is rejected, try another string that sorts inside the same interval. `/app/noadver` was rejected (`n > c` against neighbour `/app/configs/`); `/app/blocked` was accepted.
**Do not** use this technique to change behavior. Strings are renamed, not logic.
## Technique 3: smali tree editing (only when you must)
Use when the change cannot be expressed as a single-method rewrite (adding a field, restructuring control flow).
Tools: `scripts/smtool.py` (baksmali/smali with a bundled classpath) and `scripts/patch_smali.py` (method-body replacement by signature).
```bash
python scripts/smtool.py d <in.dex> <out_tree>
python scripts/patch_smali.py <tree> <patch.json> [--dry-run]
python scripts/smtool.py a <tree> <out.dex>
```
Then **must** verify (P3 will bite otherwise):
```bash
python scripts/dex_classdiff.py <original.dex> <rebuilt.dex>
```
Expect: same class count, `only_in_A=0`, `only_in_B=0`, `ACC_INTERFACE mismatch=0`.
If the class table is clean but the app crashes with `IncompatibleClassChangeError` / `VerifyError`, the round-trip damaged code items that table comparison cannot see. Fall back to technique 1.
Patch spec format for `patch_smali.py`:
```json
[
{
"file": "com/pkg/Helper.smali",
"method": ".method public final showAd(Landroid/app/Activity;)V",
"registers": 3,
"body": ["return-void"],
"note": "why this is safe"
}
]
```
## Finding the call site
Reverse-lookup from a smali tree or a set of disassembled dex:
```bash
python scripts/find_refs.py <tree> 'Lcom/pkg/AdHelper;->showSplash(...)V'
```
Search strategies that work:
- Search the **string table** first (`scripts/dex_strings.py`). API paths, keys, and SDK class names are usually string literals, and they are unique.
- Search for the **SDK's entry class** (`Sdk.init`, `Sdk.show`). The app's wrapper is almost always the only caller.
- When the decompiler is confusing, trust the **byte offsets** — `grep` the dex for the literal and note where it sits relative to other strings of the same family.
## Order of operations for a real patch task
1. Identify every place the behavior is produced (`recon` + `find_refs`).
2. Pick the highest safe layer per the table above.
3. Write all edits for a given dex into **one** dexlib2 program.
4. Patch each dex **once**.
5. `dex_classdiff` every patched dex against the original → expect zero structural drift.
6. Repack, sign, install, launch, exercise, read logcat.
7. If anything is off, revert that dex to the original and re-verify the control build.

502
vendor-new/apk-reverse/skills/apk-reverse/references/dynamic-frida.md Normal file
View File

@@ -0,0 +1,502 @@
# Dynamic analysis with Frida
Static analysis tells you what code exists. Frida tells you what code **runs**. When they disagree, runtime wins (`references/pitfalls.md` P10).
Use Frida when you need to answer:
- Which method actually renders this element? (static decompilation routinely misleads)
- What is the real call chain at the moment of interest?
- What does the app actually do on startup, in order?
- Which domains does it resolve, and when?
- Is the app detecting my instrumentation?
**Load this when:** static analysis cannot answer the question (which method renders this, what value reaches that call), or Frida will not install, attach, or stay alive. It gives setup, the four-layer probe, hook strategy, and the ROMs that hunt instrumentation.
## Setup — version alignment is a hard gate
**Do this first.** Nearly every "Frida is broken on this device" report is a version mismatch, and the error text rarely says so. The host `frida` package and the device-side `frida-server` must be the **same version**, and that version must actually support the device's Android release. Align, then debug everything else.
| Symptom | Cause | Fix |
|---|---|---|
| `unable to locate Android dynamic linker` | host/server too new for this Android release | drop to an older frida line (16.x is a safe baseline for older ROMs) |
| `Java is not defined` / `Java.perform is not a function` | that build ships no bundled Java bridge | align to a build that has it, or inline the bridge (see *17+ gotchas*) |
| `Failed to connect to remote frida-server` / `unexpected message` / `invalid message` | host package and device server differ | download both from the same release tag |
| `Java.choose` / `Java.use` throws immediately | bridge present but the VM is not ready | wrap everything in `Java.perform` |
| Attached, but hooks fire in the wrong process | USB auto-selection grabbed the emulator | use an explicit remote device (below) |
| `HOOK-OK` prints and nothing ever fires | not a version problem | go to *Hook never fires* |
### Prefer a remote device over USB
With a physical device *and* an emulator attached, `frida.get_usb_device()` can silently pick the emulator — you then attach to the wrong process and chase phantom failures for a long time. Address the device explicitly:
```bash
adb devices -l # get <serial>
adb -s <serial> forward tcp:27042 tcp:27042
frida -H 127.0.0.1:27042 -f <app.package> -l probe.js --no-pause
```
```python
device = frida.get_device_manager().add_remote_device('127.0.0.1:27042')
```
Keep `-s <serial>` on every other adb call too, or you will drift between targets mid-session.
### Attach by PID when the process list is incomplete
`device.enumerate_processes()` can return a list that simply does not contain the app you are
targeting, while `adb shell ps -A | grep <pkg>` shows it running. This is an enumeration gap,
not a permission problem, and no amount of retrying fixes it. Resolve the pid through adb and
attach to it directly:
```python
import subprocess
def find_pid(serial, pkg):
out = subprocess.run(['adb', '-s', serial, 'shell', 'ps -A -o PID,NAME'],
capture_output=True, text=True).stdout
for line in out.splitlines():
parts = line.split()
if len(parts) >= 2 and parts[1] == pkg:
return int(parts[0]) # last match wins; use the main process
return None
pid = find_pid(serial, pkg)
session = device.attach(pid) # works where attach(pkg) raises ProcessNotFound
```
Two related behaviours worth knowing:
- `attach(<package name>)` raises `ProcessNotFoundError` in exactly this situation, which is
easy to misread as "the app is not running".
- A pid captured earlier goes stale the moment the process is recycled. Re-resolve it
immediately before each injection rather than reusing a value from an earlier step.
### Verify the runtime before blaming the script
A script that reports nothing at all is usually not seeing the Java layer. Two failure modes
look identical from the host — test for both in one go:
```javascript
// send({ runtime: Script.runtime, java: typeof Java, objc: typeof ObjC });
```
- `Java` is `undefined` ⇒ the script runtime has no Java bridge. Pass `runtime='v8'`
explicitly when creating the script; some builds default to a runtime without it.
- The bridge exists but every hook misses ⇒ you attached to the wrong process, or the layer
below is native (go to the four-layer probe / `references/native-and-so.md`).
### Start the device server so it survives
Run it as root, give it a non-obvious name out of obvious paths, and **detach it from the shell's session**. A process started with `nohup ... &` from an `adb shell` that then exits gets reaped mid-run — the hooks work for a minute and then stop.
```bash
adb push frida-server-<ver>-android-<abi> /data/local/tmp/
adb shell "su -c 'mkdir -p /data/local/tmp/.svc'"
adb shell "su -c 'cp -f /data/local/tmp/frida-server-<ver>-android-<abi> /data/local/tmp/.svc/kwork'"
adb shell "su -c 'chmod 755 /data/local/tmp/.svc/kwork'"
adb shell "su -c 'setsid /data/local/tmp/.svc/kwork >/dev/null 2>&1 </dev/null &'"
adb shell "su -c 'pgrep -f kwork'" # must still be alive after the shell returned
```
If `setsid` is unavailable, hold the adb connection open instead: run `adb shell "su -c '/data/local/tmp/.svc/kwork'"` as a host-side background job and leave it running for the whole session.
### Spawn, don't attach, when timing matters
ABI must match the device (`getprop ro.product.cpu.abi`). Attaching usually misses startup — init, the first network calls, and splash logic all happen before you get a session.
```python
device = frida.get_device_manager().add_remote_device('127.0.0.1:27042')
pid = device.spawn([pkg])
session = device.attach(pid)
# ... load the script, wait until it reports that hooks are installed ...
device.resume(pid) # resume ONLY after hooks are ready
```
**Common bug:** resuming before the script has finished loading loses the first ~100–300 ms, which is exactly where init happens. Have the script `send()` a ready signal (the probe template below sends `PROBE-READY`) and resume only after receiving it.
### Spawn keeps the Activity stack down — patch, detach, then launch
Spawn mode is the right way to catch startup, and it carries a trap that costs a whole
observation round when it is not expected: **on some targets the Activity stack never comes
up while you are attached.** `dumpsys window | grep mCurrentFocus` stays `null`, screenshots
come back blank or a few kilobytes, and the process is alive with no UI — which reads as
"the app is broken" when it is only unrendered.
The app is not broken: launch it normally, with no session attached, and it renders. So the
ordering is the fix, not a different hook.
1. spawn and attach, as above;
2. let the probe write what it needs into memory (`Memory.patchCode`);
3. **detach** — a memory write is a plain write and survives, while every `Interceptor`
hook goes away with the session. For an observation run that is usually what you want,
because it removes your instrumentation from the picture entirely;
4. start the Activity normally and capture.
`scripts/spawn_patch_detach.py` implements exactly this, with
`scripts/hook_patch_only.js` as the minimal "neutralise one death site and report `PATCHED`"
probe.
Two consequences worth stating:
- **A memory patch is enough to observe a build that cannot run on its own.** If a build dies
at startup you do not have to ship a patched file to look at its UI — write the fix in
memory, detach, launch. This is also the cleanest way to run the **unmodified original** as
a control while still getting past its death site (`verification.md` §the control build rule).
- **Do not conclude "no UI" from a blank capture taken while attached.** Check
`mCurrentFocus`: if it is `null` under an attached session and non-null without one, you
have this problem, not a finding about the target.
## Frida 17+ gotchas
The preferred fix for every symptom in the table above is **version alignment**. Use these only when you genuinely must run a 17+ build.
- The **built-in Java bridge was removed**. `Java.perform(...)` is not available unless you inline the bridge yourself:
```python
bridge = open(<site-packages>/frida_tools/bridges/java.js, encoding='utf-8').read()
script = bridge + "\nObject.defineProperty(globalThis,'Java',{value:bridge,configurable:true});\n" + my_js
```
Pass the whole thing through a small loader script (below).
- **Export lookup renamed.** Use a compatibility helper rather than a single API:
```javascript
function resolveExport(name) {
try { if (Module.getGlobalExportByName) return Module.getGlobalExportByName(name); } catch (e) {}
try { if (Module.findExportByName) return Module.findExportByName(null, name); } catch (e) {}
try { const l = Process.findModuleByName('libc.so'); if (l) return l.getExportByName(name); } catch (e) {}
return null;
}
```
- **Large script injection times out.** A multi-hundred-KB bridge + script can exceed the transport timeout. Use a tiny loader and post the real payload:
```python
loader = "recv('go', m => { eval(m.payload.code); }); send('loader-ready');"
sc = session.create_script(loader)
sc.load()
sc.on('message', on_message) # wait for 'loader-ready'
sc.post({'type': 'go', 'payload': {'code': BIG_JS}})
```
## Hook strategy
### Java layer (Kotlin/Java app)
```javascript
Java.perform(() => {
const Cls = Java.use('com.example.Helper');
Cls.showAd.overload('android.app.Activity').implementation = function (a) {
send({ tag: 'Helper.showAd', stack: Java.use('android.util.Log').getStackTraceString(
Java.use('java.lang.Throwable').$new()) });
return this.showAd(a); // observe, then delegate
};
});
```
- **Use `overload(...)`** — obfuscated classes often have several methods with the same name.
- To find obfuscated names, enumerate at runtime rather than guessing from decompiled short names:
```javascript
Java.enumerateLoadedClasses({
onMatch: n => { if (n.indexOf('example') >= 0) send(n); },
onComplete: () => send('done')
});
```
- **Call stacks are the highest-value signal.** `Log.getStackTraceString(new Throwable())` or `Java.use('android.util.Log').getStackTraceString(...)` gives you the real chain from the framework down to the method — far more reliable than reading smali.
### Native layer
```javascript
const f = resolveExport('open');
Interceptor.attach(f, {
onEnter(args) { this.p = args[0].readCString(); },
onLeave(ret) { if (this.p) send({ tag: 'open', path: this.p }); }
});
```
### Network / domain observation (cheap and very informative)
```javascript
const Inet = Java.use('java.net.InetAddress');
Inet.getAllByName.overload('java.lang.String').implementation = function (h) {
send({ tag: 'dns', host: h });
return this.getAllByName(h);
};
```
This is often the **decisive evidence** for an ad-removal claim: if the ad SDK's domains are *never resolved*, the subsystem never started — a much stronger statement than "logcat was quiet". It also works for SDKs that bypass the system HTTP proxy.
## The four-layer probe template
When the question is "why did this request fail / where did it go", do not hook one class and hope. Ship **one long-lived script that hooks four layers at once**; whichever layer fires first localizes the problem immediately. This is the single most productive artifact of a runtime investigation — reuse it verbatim.
The layers, and why each one is there:
1. **The app's own network wrapper** — enumerate `getDeclaredMethods` and wrap *every* overload, so you do not have to guess the entry point.
2. **OkHttp end to end** — `newCall`, `Request$Builder.build`, `RealCall.execute`, `AsyncCall.run`, `RealInterceptorChain.proceed`. The last two are the ones that fire for asynchronous calls.
3. **`java.net.URL.openConnection`** — plenty of login/register paths never touch OkHttp; they use `HttpsURLConnection` and a completely different trust configuration.
4. **`Throwable.getMessage`** — pulls out the original text of exceptions an upper layer caught and swallowed. Without it, a caught failure looks like "nothing happened".
Rules that make the difference between a usable probe and a wasted session:
- **Write to a file as well as to `send()`.** Stdout is lossy and the CLI drops messages; the file is the evidence.
- **Let it stay resident.** Hook before the app touches the network, then leave it running while you drive the UI. A one-shot script misses everything that happens after its first second.
- **Wrap every hook in its own `try/catch`.** One missing class must not take down the other three layers.
- **Never capture an overload in a `var`.** Inside a loop, `var ov = ...` leaves every hook pointing at the *last* overload: they install cleanly and then mis-report forever. Use `let` or `.forEach()`.
`scripts/frida_probe.js` is the same probe in a fuller form (per-arity dispatch, DNS layer, dedup and hard caps). The script below is the minimal portable one — paste it into any loader and it adapts to the target.
```javascript
// probe.js — four-layer network + swallowed-exception probe.
const APP_PKG = '<app.package>';
const APP_NET_CLASS = 'com.example.app.net.HttpHelper'; // the app's own wrapper, once you have found it
const LOGFILE = '/data/user/0/' + APP_PKG + '/files/probe.log'; // app-private: always writable by the app
const MAX_THROW = 400; // Throwable.getMessage is hot — cap it
const INTERESTING = /(network|http|ssl|cert|fail|timeout|refused|unable|error|exception)/i; // widen for the app's UI language
let out = null;
try { out = new File(LOGFILE, 'a'); } catch (e) {}
function log(ev, data) {
const line = JSON.stringify(Object.assign({ t: Date.now(), ev: ev }, data));
try { send(line); } catch (e) {}
try { if (out !== null) { out.write(line + '\n'); out.flush(); } } catch (e) {}
}
function stack(n) {
try {
const t = Java.use('java.lang.Throwable').$new();
return Java.use('android.util.Log').getStackTraceString(t).split('\n').slice(1, (n || 6) + 1).join(' | ');
} catch (e) { return '<no stack>'; }
}
function sa(args) { // a hook must never die inside its own logging
try {
const o = {};
for (let i = 0; i < args.length; i++) o['a' + i] = args[i] === null ? 'null' : '' + args[i];
return o;
} catch (e) { return { err: '' + e }; }
}
// 1) the app's own wrapper: every declared method, every overload, separately wrapped
function hookAppNet(cls) {
try {
const C = Java.use(cls);
const ms = C.class.getDeclaredMethods();
for (let i = 0; i < ms.length; i++) {
try {
const name = ms[i].getName();
const ps = ms[i].getParameterTypes();
const sig = [];
for (let j = 0; j < ps.length; j++) sig.push(ps[j].getName());
const ov = C[name].overload.apply(C[name], sig);
ov.implementation = function () {
try { log('APP-NET', { cls: cls, m: name, args: sa(arguments), stack: stack(5) }); }
catch (e) { log('PROBE-ERR', { at: 'app-net', msg: '' + e }); }
return ov.apply(this, arguments);
};
} catch (e) { log('HOOK-SKIP', { cls: cls, m: ms[i].getName(), msg: '' + e }); }
}
log('HOOK-OK', { layer: 'app-net', cls: cls, n: ms.length });
} catch (e) { log('HOOK-FAIL', { layer: 'app-net', cls: cls, msg: '' + e }); }
}
// 2) OkHttp end to end — internal class names moved between major versions, so try every location
function hookOkHttp() {
const targets = [
['okhttp3.OkHttpClient', 'newCall'],
['okhttp3.Request$Builder', 'build'],
['okhttp3.RealCall', 'execute'],
['okhttp3.RealCall$AsyncCall', 'run'],
['okhttp3.internal.http.RealInterceptorChain', 'proceed'], // okhttp 3.x
['okhttp3.internal.connection.RealInterceptorChain', 'proceed'] // okhttp 4.x / 5.x
];
for (let i = 0; i < targets.length; i++) {
const cn = targets[i][0], mn = targets[i][1];
try {
const ovs = Java.use(cn)[mn].overloads; // every overload of that name; use .overload('<sig>') if this is undefined
for (let j = 0; j < ovs.length; j++) {
const ov = ovs[j]; // block-scoped: every hook keeps its own overload
ov.implementation = function () {
try { log('OKHTTP', { cls: cn, m: mn, args: sa(arguments), stack: stack(6) }); }
catch (e) { log('PROBE-ERR', { at: cn, msg: '' + e }); }
return ov.apply(this, arguments);
};
}
log('HOOK-OK', { layer: 'okhttp', cls: cn, m: mn, overloads: ovs.length });
} catch (e) { log('HOOK-SKIP', { layer: 'okhttp', cls: cn, m: mn, msg: '' + e }); }
}
}
// 3) java.net.URL — login/register frequently bypasses OkHttp entirely
function hookRawUrl() {
try {
const U = Java.use('java.net.URL');
const o0 = U.openConnection.overload();
o0.implementation = function () {
try { log('RAW-URL', { url: '' + this.toString(), stack: stack(6) }); } catch (e) {}
return o0.apply(this, arguments);
};
try {
const o1 = U.openConnection.overload('java.net.Proxy');
o1.implementation = function () {
try { log('RAW-URL', { url: '' + this.toString(), proxy: true, stack: stack(6) }); } catch (e) {}
return o1.apply(this, arguments);
};
} catch (e) {}
log('HOOK-OK', { layer: 'url' });
} catch (e) { log('HOOK-FAIL', { layer: 'url', msg: '' + e }); }
}
// 4) Throwable.getMessage — the original text of an exception an upper layer caught and swallowed
function hookThrowable() {
try {
const T = Java.use('java.lang.Throwable');
const seen = {};
let n = 0;
T.getMessage.implementation = function () {
const msg = this.getMessage(); // self-call inside the replacement is fine (Frida guards it)
try {
if (msg !== null && msg !== undefined && n < MAX_THROW) {
const cls = '' + this.getClass().getName();
const interesting = cls.indexOf('Exception') >= 0 || cls.indexOf('Error') >= 0 || INTERESTING.test('' + msg);
const key = cls + '|' + msg;
if (interesting && seen[key] === undefined) { // dedup: this method is on a very hot path
seen[key] = 1; n++;
log('THROW', { cls: cls, msg: '' + msg, stack: stack(4) });
}
}
} catch (e) {}
return msg;
};
log('HOOK-OK', { layer: 'throwable' });
} catch (e) { log('HOOK-FAIL', { layer: 'throwable', msg: '' + e }); }
}
Java.perform(function () {
hookAppNet(APP_NET_CLASS);
hookOkHttp();
hookRawUrl();
hookThrowable();
log('PROBE-READY', { pkg: APP_PKG, logfile: LOGFILE }); // the host waits for this before resuming
});
```
Read the log after driving the UI. `RAW-URL` (with no `OKHTTP` events) is how you learn the request went over `HttpsURLConnection` — a different trust path entirely (`references/tls-and-cert.md` §If OkHttp is the failing path). `THROW` is how you learn an upper layer swallowed the real error.
```bash
# pull the record (app-private path needs root to read)
adb -s <serial> shell "su -c 'cat /data/user/0/<app.package>/files/probe.log'" > probe.log
```
## Hook never fires — debug in this order
`HOOK-SKIP` / `HOOK-FAIL` lines answer this before you start guessing. When every layer reports `HOOK-OK` and still no event arrives:
1. **Does the class / method / overload actually exist?** A misspelled obfuscated name, a renamed okhttp internal class, or an overload taking `Object` instead of `String` all produce a wrapper that is simply never invoked.
2. **Is the ClassLoader the right one?** Multi-dex, plugin-loaded and packed apps can hold several loaders; `Java.use` uses the app loader by default and throws `ClassNotFoundException` for a class that is plainly in the APK. Enumerate and switch:
```javascript
Java.enumerateClassLoaders({
onMatch: function (l) { try { if (l.findClass('<app.net.Class>')) Java.classFactory.loader = l; } catch (e) {} },
onComplete: function () {}
});
```
3. **Is that code path reached at all?** Layer 4 settles it: if `THROW` events show the app failing earlier, your hook's call site is never executed and no amount of hooking will help.
4. **Did your UI action trigger business logic?** A tap that looks fine but fails a local form check returns before any network call. Verify the input actually reached the field (`references/environment.md` §Driving the UI from adb), not just that the button animated.
## `Java.choose` also matches dead instances
`Java.choose('com.example.app.MainActivity', ...)` returns **every** instance the VM still tracks, including finished and destroyed ones. Walking their view tree then yields an empty list — that is normal, not a bug in your script. Filter first, and prefer a direct reflective call over simulating a tap:
```javascript
Java.choose('com.example.app.MainActivity', {
onMatch: function (a) {
try {
if (a.isFinishing() || a.isDestroyed()) return; // dead instance: skip it
send({ tag: 'live', view: '' + a.findViewById(<viewId>) });
} catch (e) {}
},
onComplete: function () {}
});
```
## Reading obfuscated code at runtime
When names are meaningless (`a7`, `x6`, `uf0`), do not patch from names. Instead:
1. Find a **unique anchor**: an API path, a preference key, or an SDK class name from the string table (`scripts/dex_strings.py`).
2. Locate the small class/method that references it (byte search in the dex + a targeted disassembly).
3. Hook that anchor, capture the **stack**, and read off the real chain.
4. Hook the classes the stack reveals — that is where semantics become visible.
## Anti-instrumentation
Signs: the app exits or freezes shortly after attach; `logcat` shows a generic process death with no Java stack; strings like `frida`, `xposed`, `magisk`, `substrate` are referenced by **app code** (not just by an SDK's string list).
**Rule this out first — it is not always a detector.** A process that dies right after you
attach is frequently the ROM rather than the app: on aggressive OEM ROMs, merely backgrounding
the app (pressing HOME, switching away) triggers a freeze, and the logcat signature is a state
transition such as `state: R -> F` while hooks stop with no Java stack. That looks identical to
a detector killing you. Check `references/environment.md` for that signature, and keep the app
in the foreground, before building an anti-anti plan.
Countermeasures, in order of least disruption:
1. Spawn + late resume, so hooks are installed before the check runs.
2. Rename `frida-server` and move it out of obvious paths.
3. Hook the detector itself and neuter it — find it by hooking the suspicious API (e.g. `/proc/self/maps` reads, `File.exists`, `Runtime.exec`) and inspecting the stack.
4. Only then consider a gadget-based approach.
**Remember:** most `frida`/`root` strings in a decompiled APK belong to third-party SDKs' own detection lists, not to the app. Verify that **app code** references them before doing anti-anti work.
### When the ROM hunts your instrumentation
Some vendor ROMs treat a device-side `frida-server` as hostile and kill it on a timer. The
symptom is not an error message — it is a working session that disappears. A later
`attach`/`spawn` fails with `unable to connect to remote frida-server`, or with
`TransportError: connection closed` **during `spawn`, before `resume`**. That second shape is
easy to misread as the target crashing during startup. It is not the target.
Measured on one vendor build: the server was killed repeatedly across a work session, and on
one occasion the whole device restarted, briefly taking `system_server` with it
(`Can't find service: activity`, `No service published for: input`). Both symptoms are
environment, not evidence about the app.
- **Check the server is alive immediately before every experiment**, not once per session:
`adb shell "su -c 'pidof <server-name>'"`. Restart it if it is gone.
- **Distinguish the two failure shapes.** `connection closed` at `spawn` = server died or was
killed (environment). `connection closed` just after `resume` = the target exited — *that*
one is about the app.
- **Rename the server binary and move it off the obvious path.** It defeats simple filename
sweeps. It does not hide from a target that lists the directory, so do not treat the rename
as a fix for detection — only as protection from cleanup.
- **Budget a restart, not a re-analysis.** A killed server costs seconds; a round spent
re-deriving the previous conclusion costs far more.
- **If a device reboot interrupts the run, re-establish state before measuring**: server up,
`adb forward` re-created, screen awake and unlocked, installed-build hash re-read. A
measurement taken across a reboot is a measurement of two states
(`long-task-discipline.md` §keep the observation window clean).
### When live attach cannot work at all
Sometimes nothing short of "do not inject at runtime" is viable — the check runs
before your hooks can, it watches its own process, or the environment forbids
`ptrace`. Do not keep escalating on the same axis. There are non-live ways to get
runtime-grade information, and they are often enough to finish the job:
| Alternative | What it gives you | Cost |
|---|---|---|
| **Static instrumentation** — load a gadget/small agent by modifying the APK (e.g. an injected `Application` wrapper, a gadget `.so` added to `lib/<abi>/` and wired from the manifest) | Hooks run from process start with no external attach, no `ptrace`, no server process on the device to be found | Must repack and re-sign; the build you observe is no longer byte-identical to the original, so **it cannot serve as the unmodified control** (`verification.md` §the control build rule) |
| **Static read of the answer** — the check's inputs are usually visible: the field it reads, the string it compares, the response field it trusts | Often enough to plan a patch without ever observing | You do not learn the live values; combine with `runtime-data.md` for local state |
| **Self-recorded evidence** — the app's own cached responses, logs, or on-disk state | Real runtime data with no injection at all (a HTTP cache, a pref, a JSON snapshot left behind by a normal launch) | Only what the app happened to persist; needs one clean launch and then a filesystem read |
The last one is worth remembering because it is the cheapest and it is easy to
forget: a normally-launched app often leaves the exact runtime data you wanted —
server responses in its HTTP cache, config in a preferences file, feature state in
a snapshot. Read those **before** concluding that dynamic analysis is blocked.
**Two cautions that apply to every alternative above:**
- **Changed bytes change behaviour.** Any injected build is a different artifact,
and on a target that fingerprints its own layout that difference is the finding,
not the app's real behaviour. Always keep the unmodified run as the control.
- **Evidence from a different environment is a different claim.** A result obtained
with the app patched-for-injection, or on a device with different privilege state,
supports "verified under condition X". Say so; do not present it as the app's
unmodified behaviour (`long-task-discipline.md` §the most expensive drift).
## What to capture for the record
- spawn time, hook-ready time, resume time
- every hook hit with a stack (bounded — cap output)
- DNS hosts in order, with timestamps
- the exact build under test (hash) and the control build's results

340
vendor-new/apk-reverse/skills/apk-reverse/references/emulation-and-rpc.md Normal file
View File

@@ -0,0 +1,340 @@
# Emulation and RPC — running the target's code on your terms
There is a class of target where neither reading nor patching is the bottleneck: the code you need
is *inside* a hardened `.so`, statically unreadable (OLLVM, string encryption, VMP) or only
decrypted in memory, and what you actually want is its **output** — the signature it computes, the
blob it decrypts, the token it mints — not its source. Two routes give you that without winning the
unpacking war first:
- **Unidbg** — emulate the `.so` on your PC, feed it inputs, read outputs. Offline, batch, no
device, no detector. You pay in *environment*: everything the library touches must be faked.
- **Frida RPC** — let the real process run its own hardened code, and call it from Python through a
resident session. High fidelity (it is the real environment), no faking required. You pay in
*stability*: the process must tolerate injection, and the session must survive.
This file is about choosing between them and driving both. The unpacking itself is
`references/packers.md` and `references/advanced-unpacking.md`; the hooking mechanics live in
`references/dynamic-frida.md`. What is new here is treating the target's own crypto as a **callable
function** instead of a thing to understand.
**Load this when:** you need the *output* of a routine rather than a change to the app -- a signature, a token, a cipher. It gives emulation with its environment-filling cost, against service-ifying the live function over Frida RPC.
## When emulation or RPC is the right move
Reach for this file when any of these holds:
- The algorithm you need lives in a hardened `.so` and reverse-engineering it (weeks against OLLVM
or VMP) costs more than *calling* it (hours of environment work).
- You need **many** calls — fuzzing an input format, brute-forcing a key schedule, generating
signed request batches — where hand-driving a device per call is hopeless.
- The function's output depends on environment state (files, system properties, other libraries)
that a static read cannot evaluate.
- The gate you care about is a *computed value* (sign header, encrypted body) rather than a branch
— `references/server-api.md` told you the server checks it, and the client computes it in native
code (`references/signature-derived-keys.md`).
Do **not** reach for it when a two-byte dex patch or an LSPosed module solves the problem —
`references/dex-patching.md` and `references/lsposed-and-modules.md` are cheaper and more durable.
Emulation and RPC are for when the code that matters must keep *running*.
## The decision table
| | Unidbg (offline emulation) | Frida RPC (live process) |
|---|---|---|
| Environment fidelity | fake — you build it | real — the device provides it |
| Works without a device | yes | no (device or emulator required) |
| Works against anti-injection | **yes** — the code never knows | **no** — injection is the entry ticket |
| Batch throughput | thousands of calls/min | tens of calls/sec, process-bound |
| Setup cost | Java + maven + per-library env work | frida already installed, minutes |
| Maintenance cost | every missing JNI/syscall is a new stub | reconnect logic, process lifecycle |
| Reproducibility | deterministic (you control time/random) | real time, real randomness |
| Best for | signing algorithms, crypto, parsers | anything entangled with live app state |
| **Priority when either could work** | second — the environment bill below is real and routinely underestimated | **first** — the device already owns the environment |
The two-strike rule applies across the table: if a library refuses to run under unidbg after two
focused rounds of `DalvikVM` patching, stop emulating and move to RPC (or the reverse) instead of a
third round — a library that checks its own loading path or decrypts itself against device state
may simply not be worth emulating, and that is a finding, not a failure (`SKILL.md`
§Stop conditions).
### The environment bill — budget days, not hours
"Filling the environment" reads like a checklist and behaves like a project. The mistake this
section prevents is starting an emulation because the table says *works against anti-injection*, and
then discovering that this library's environment includes everything emulation cannot fake: an
Android `Context` backed by a real package manager, Binder round-trips into another process, a
`KeyStore` attestation that only succeeds on real hardware, or a self-check against device state
captured at install time. For a commercial native algorithm of that shape, "stub it until it runs"
is measured in **days, not hours** — each missing piece is discovered one fault at a time, and the
rounds do not get shorter.
Weigh it before starting:
| Question | If yes |
|---|---|
| Does it call through `Context` (package name, files dir, signature, `PackageManager`)? | a stub is often enough — cheap, keep going |
| Does it talk to another process (Binder service, bound SDK, remote provider)? | the stub surface grows fast; **prefer RPC** |
| Does it verify hardware (KeyStore, TEE, an attestation chain)? | emulation is likely a dead end — the value it wants cannot be produced here |
| Does it check its own loading environment (paths, maps, root, debugger, installer)? | readable and patchable *inside* the emulator, but budget one round per check |
| Must it be called thousands of times a minute? | the RPC throughput ceiling is real — this is the case that justifies the bill |
Default to **RPC first, emulation second**: the device already provides the environment, and
per-call cost only starts to matter once you need volume. The two claims in this subsection
(the day-scale cost of a Context/Binder/KeyStore-shaped library, and the RPC throughput ceiling)
are **inferred** from the mechanism and from community practice — this repository has not emulated a
commercial sample end to end, as `references/evidence-summary.md` §The capability matrix states.
## Part A — Unidbg
### What it is
Unidbg ([zhkl0228/unidbg](https://github.com/zhkl0228/unidbg), Java, Apache-2.0) emulates an
Android process on your PC:
- **CPU** — a `backend` layer over Unicorn (default) or Dynarmic/Hypervisor, executing the real
arm/arm64 code of your `.so`.
- **Memory & linker** — it maps the ELF, resolves `DT_NEEDED` dependencies from a bundled Android
system-library set (`AndroidResolver(apiLevel)`), and runs relocations.
- **JNI environment** — a Dalvik VM emulation (`emulator.createDalvikVM(apk)`) that answers the
library's `FindClass`/`GetMethodInfo`/`CallObjectMethod` calls against either real classes from
the APK's dex or **stubs you write in Java**.
- **Syscalls** — file IO, `mmap`, `pthread` primitives, system properties, `/proc` reads are
emulated or redirected.
You write a Java main that wires those together, calls `JNI_OnLoad` (many hardened libraries do
their initialization there), then calls the target function with your inputs. The library's real
arm code executes; its output comes back as a Java value.
**This repository does not bundle unidbg** — it is a Java/maven project, not a script. To use it:
```bash
git clone --depth 1 https://github.com/zhkl0228/unidbg.git
cd unidbg
# Windows + JDK 17 (measured): the shipped pom pins -source/-target 8, and on JDK 9+
# the compiler resolves against the current API where `Module` is ambiguous
# (java.lang.Module vs com.github.unidbg.Module) — unidbg-api fails to compile.
# <release>8</release> selects the JDK 8 API and fixes it; the bundled Maven wrapper
# is 3.5.4, which rejects maven-compiler-plugin 3.13, so 3.8.1 is the version that
# both supports <release> and runs on that wrapper.
set JAVA_HOME=C:\Program Files\Java\jdk-17.0.4.1
mvnw.cmd -B -pl unidbg-android -am -DskipTests compile # BUILD SUCCESS, ~48 s
# tests are skipped by <maven.test.skip>true</maven.test.skip> in the pom — override it:
mvnw.cmd -B -pl unidbg-android -am -Dmaven.test.skip=false \
-Dtest=MemoryTrackerTest -DfailIfNoTests=false test # Tests run: 3, Failures: 0
# first run downloads maven itself plus dependencies — minutes
```
Then pick the demo closest to your target under `unidbg-android/src/test/java` and adapt it; the
bundled demos (`TTEncrypt`, `SignUtil`, `QDReaderJni`, …) each need the `.so`/APK they reference
placed where the test expects it.
A library project that already uses maven/gradle can instead depend on the published artifacts
(group `com.github.zhkl0228`, artifacts `unidbg-android` + `unidbg-api`, on Maven Central) — the
clone-above route avoids version drift and gives you the test-suite demos to copy from.
### The minimal call template
Adapt this to your target — it is the shape every unidbg driver has (measured against the
0.9.x API; class names move rarely but do move, check a current test under
`unidbg-android/src/test/java` when something fails to resolve):
```java
import com.github.unidbg.AndroidEmulator;
import com.github.unidbg.LibraryLoader;
import com.github.unidbg.linux.android.AndroidEmulatorBuilder;
import com.github.unidbg.linux.android.AndroidResolver;
import com.github.unidbg.linux.android.dvm.*;
import com.github.unidbg.memory.Memory;
import java.io.File;
public class CallTarget {
public static void main(String[] args) {
// 64-bit or 32-bit must match the .so, not the PC.
AndroidEmulator emulator = AndroidEmulatorBuilder.for64Bit()
.setProcessName("com.example.app") // some libraries check this
.build();
Memory memory = emulator.getMemory();
memory.setLibraryResolver(new AndroidResolver(23)); // bundled system .so set
// The APK gives the VM real classes for JNI calls; null is allowed if
// you intend to stub everything yourself.
VM vm = emulator.createDalvikVM(new File("target.apk"));
vm.setVerbose(true);
vm.setJni(new AbstractJni() { // your JNI callbacks — see below
// override methods as the target demands them
});
DalvikModule dm = vm.loadLibrary(new File("libtarget.so"), false);
dm.callJNI_OnLoad(emulator); // init: decrypts, registers natives
// Calling a *statically exported* symbol:
// DvmObject<?> ret = dm.callFunction(...)
// Calling a *JNI-registered* method by its Java name (most common for hardened SDKs):
DvmClass<?> c = vm.resolveClass("com/example/NativeApi");
DvmObject<?> result = c.callStaticJniMethodObject(emulator,
"sign(Ljava/lang/String;)Ljava/lang/String;", "input");
System.out.println(result.getValue());
}
}
```
### Filling the environment — "supply what is missing"
A hardened library does not just compute; it *checks*. Every unidbg session converges on the same
loop: run, read the first unsupported callback or mapping fault, implement exactly that, repeat.
In order of frequency:
| The library wants | You supply |
|---|---|
| `JNI_OnLoad` → `FindClass`/`GetMethodID` on app classes | `AbstractJni` overrides (`callStaticObjectMethodV`, `getStaticObjectField`, …) returning canned values; or real classes from the APK via `createDalvikVM(apk)` |
| `Context` methods (`getPackageName`, `getFilesDir`, `getPackageManager` → signature!) | stubs — `vm.resolveClass("android/content/Context")` + your `AbstractJni` answers; a signature check wants the *original* APK's signature bytes, which `createDalvikVM(apk)` serves from that very file |
| file reads (`/proc/self/maps`, `/data/data/...`, config files) | `emulator.getSyscallHandler().addIOResolver(...)` redirecting to host files, or `virtualFileSystem` mounts; a `/proc/self/maps` check that looks for frida/xposed is satisfied by a clean canned maps file |
| other `.so` (cocos, tls, crypto) | `vm.loadLibrary` them first, in dependency order, or drop them beside the target so the resolver finds them |
| time / randomness | `emulator.getSyscallHandler` clock override; unidbg's deterministic RNG unless the target seeds from `/dev/urandom` (redirect it) |
Two structural rules that save hours:
- **Load order is init order.** `JNI_OnLoad` of the shell often registers natives and decrypts;
calling a function before the init that populates it yields a null deref that looks like a bug
in unidbg. Always `callJNI_OnLoad` first, and if the library has an explicit init/exported
setup entry, call that next.
- **32 vs 64 must match the `.so`,** not your PC and not the device's preference
(`references/native-and-so.md` §Cross-architecture applies here too: `for32Bit()`/`for64Bit()`
mirror the ABI you extracted from `lib/`).
### Failure modes you will meet
| Symptom | Meaning | Move |
|---|---|---|
| `unsupported syscall` / `signal 11 (SIGSEGV)` at a stable address | the library touched something unidbg does not fake yet | read the unidbg log line above the fault — it names the syscall/JNI call; stub it |
| `JNI_OnLoad` returns non-zero or throws | self-check failed inside init | enable `vm.setVerbose(true)`, read which check it ran (often signature, path, or a system property), satisfy it via stub |
| works, but output differs from device | environment input differs (time, random, files, another lib's state) | diff the inputs: redirect and pin them; on-device RPC (Part B) is the ground truth to compare against |
| crashes deep in the library, no unsupported call logged | the code path is genuinely wrong (wrong calling convention, wrong args) — or the library decrypted itself against device state it cannot have here | recheck the prototype (arg types/order) against the dex `native` declaration; then consider RPC instead |
| library refuses to load: dependency not found | a `DT_NEEDED` `.so` you did not provide | `readelf -d` the target, load missing libs first |
Know when to stop: a library that validates its own decrypt key against server data, or that
refuses any environment it did not boot in, is **not worth emulating** — that is exactly the shape
where the decision table sends you to Part B. (Which anti-emulation checks exist and how common
each is: inferred from community experience, not measured here.)
### Measured here
See `references/evidence-summary.md` §The capability matrix for the exact commands and outputs behind
every label in this section. Measured in that pass: the repository builds on Windows/JDK 17 once
the compiler plugin is pointed at the JDK 8 API, the emulator boots, and one bundled test suite
passes on the Dynarmic backend. **No target `.so` was emulated yet** — every per-library claim above
(which JNI callbacks a hardened library asks for, which syscalls it hits, how many rounds of stubbing
it costs) is **inferred** from the library's shape and from community practice, not from a run
against this sample.
## Part B — Frida RPC
### The mechanism
A Frida script can export functions to its host: assign to `rpc.exports`, and every property
becomes callable from Python (`script.exports_sync.name(...)` after frida 16) with JSON-serializable
arguments and return values, synchronously, from outside the process. That is the whole trick:
```javascript
rpc.exports = {
add: function (a, b) { return a + b; },
};
```
```python
script = session.create_script(code); script.load()
print(script.exports_sync.add(40, 2)) # -> 42
```
Everything else is packaging: keeping the session alive, finding the function address, converting
types across the NativeFunction boundary. The pattern for the case that matters — a hardened
library's signing function, exported or `RegisterNatives`-bound:
```javascript
// 1) exported symbol:
const addr = Module.findExportByName('libtarget.so', 'Java_..._sign');
// 2) dynamically registered (no export — the usual shape for hardened SDKs):
// hook RegisterNatives once at startup, note the fnPtr for the name you care about
const sign = new NativeFunction(addr, 'pointer', ['pointer', 'pointer']);
rpc.exports = { sign: function (s) { return sign(env, jstring(s)); } };
```
Java-side functions are equally callable (`Java.perform` wraps the call), which is how the
harmless connectivity probe — `getPackageName` over RPC — works in the template below.
### The kit's tooling
`scripts/frida_rpc_serve.py` + `scripts/rpc_template.js` implement the full loop:
- three front ends: one-shot `--mode call`, interactive `--mode repl`, and a resident
`--mode http` endpoint (`POST /call {"export": "...", "args": [...]}`, `GET /exports`,
`POST /reload`);
- spawn or attach (by name or by pid — attach-by-name fails on incomplete process enumeration,
`references/dynamic-frida.md` §Attach by PID);
- automatic reconnect: a detached session (app restart, killed server, ROM reaping the server)
re-attaches, reloads the script and retries the call, with per-request retry cycles and
exponential backoff; a killed-and-restarted device server is recovered by dropping the cached
remote device before reconnecting;
- the template exports `add` (transport sanity), `getpackagename` (Java probe),
`callnative(module, export, retType, argTypes, args)` and `callnativeaddr(module, offset, ...)`
for wrapping any native symbol, with a frida-16/17-compatible export resolver.
Typical session (device server already running and forwarded, `references/dynamic-frida.md`
§Setup):
```bash
adb -s <serial> forward tcp:27043 tcp:27043
python scripts/frida_rpc_serve.py --remote 127.0.0.1:27043 --package com.example.app \
--script scripts/rpc_template.js --mode repl
rpc> list
rpc> getpackagename
rpc> callnative ["libtarget.so","Java_com_example_Native_sign","pointer",["pointer","pointer"],[...]]
```
Then feed the recovered function into request generation the way `references/server-api.md`
describes — the RPC service turns "the server checks a signed header" into "I can compute that
header on demand", which is the whole reason this route exists.
### What the live route costs — measured failure modes
Every row below was hit against a real device during the verification pass behind this file
(exact outputs in `references/evidence-summary.md` §The capability matrix):
| Failure | What it looks like | Handling |
|---|---|---|
| **Hardened target kills injection** | attach: `process either refused to load frida-agent, or terminated during injection`, logcat `CRASH ... IterateRegisters found fp`; spawn: script loads, then every call dies with `script has been destroyed` | the shell's anti-instrumentation won; do not retry the same attach — switch route (unidbg for the crypto, or `references/lsposed-and-modules.md` / static patching), or fight the detector deliberately (`references/detection-and-anti-analysis.md`) |
| **ROM reaps the server** | `connection closed` mid-session, sometimes repeatedly | budget restarts, not re-analysis: the bridge reconnects once the server is back; check `pidof <server>` before every experiment block (`references/dynamic-frida.md`) |
| **Attach by name fails for processes the device can see** | `unable to find process with name 'com.example.app'` while `frida-ps -Uai` and `ps -A` both list it alive; reproduced for a system app and for a Magisk-manager process, for every package name tried | resolve the pid yourself and attach with `--pid`: measured on the same ROM, `device.attach(pid)` loaded the script and returned RPC results for the very processes whose *name* lookup failed (`--pid 19938` → `com.android.settings`, `--pid 21493` → `com.android.email`). Take the pid from `frida-ps -U`, `ps -A`, or `adb shell pidof <pkg>` |
| **The server comes back on a non-default port** | `frida.ServerNotRunningError: unable to connect to remote frida-server: closed` on attach even though `frida-ps -U` still works | the ROM or a watchdog reaped the visible server and a disguised copy returned elsewhere (measured: only `*.svc16` left alive, listening on 27043, nothing on 27042). Check the listening port, `adb forward tcp:27043 tcp:27043`, then `--remote 127.0.0.1:27043` — do not assume the default port |
| **Attach by pid times out on a busy process** | `frida.TimedOutError: unexpectedly timed out while waiting for signal from process with PID X` during attach (raised by an ordinary third-party app); it is not a `TransportError` subclass and escaped every handler | nothing is wrong with the tooling — the process did not answer the agent handshake; use another process for the connectivity probe, or retry once the app settles (the bridge now treats `TimedOutError` as retryable and fails with one line instead of a traceback) |
| **Short-lived processes** | the settings main activity process exits seconds after `am start` | pick a persistent process for connectivity tests; the business target only when it actually stays up |
The strategic point those rows make together: **on a hardened target, RPC is not guaranteed to be
available at all** — the injection ticket can be revoked by the target itself. That is the single
strongest argument for keeping the unidbg route warm, despite its environment cost: it is the one
route that works when the process refuses to be instrumented.
### Combining the routes
The routes compose rather than compete:
1. **RPC as oracle, unidbg as student.** When an emulated function's output must be trusted, diff
it against the live process's output for identical inputs; disagreement localizes the missing
environment piece far faster than reading the library (`Part A` §failure modes, "output
differs").
2. **Dump via RPC, emulate after.** A live call that returns the decrypted key/table the library
uses (`references/runtime-data.md`) can be replayed into the emulated run as a fixed stub.
3. **Batch offload.** Use the device session to discover the calling convention and to capture
sample inputs/outputs (few calls), then run the bulk of the work offline under unidbg.
## What to record
- which route was chosen and why (one line, against the decision table)
- for unidbg: the API level resolver, the list of JNI stubs you had to write, what the library
checked before it would compute (that list *is* the target's defense inventory)
- for RPC: server version vs host version, attach vs spawn, reconnect behaviour observed, and —
decisive for a hardened target — whether injection survived at all
- inputs/outputs for at least one call verified against an independent source (device-side log,
server response, or a known-answer test)

306
vendor-new/apk-reverse/skills/apk-reverse/references/environment.md Normal file
View File

@@ -0,0 +1,306 @@
# Environment — device, emulator, tooling, networking
**Load this when:** before the first experiment on a device, and again whenever a failure surprises you. It gives device/emulator selection, root, ADB and networking, the preflight check, and how to look at the screen instead of driving blind.
## Pick your target
| Option | Pros | Cons |
|---|---|---|
| **Rooted physical device, native ABI** | Truest behavior; native libs load natively; Frida works well | One at a time; USB flakiness |
| **Emulator with root** | Disposable; snapshots; easy reset | Different ABI; some apps detect it or refuse to run; native ARM libs may need translation |
| **Static only** | No device needed | Cannot verify anything. Never claim success from static analysis alone. |
**Rule:** the artifact must be verified where the user will run it. An emulator that runs the app is **not** evidence about a physical ARM device, and vice versa.
**Important:** if the app ships only `arm64-v8a` native libraries and your emulator is x86_64, it may still run via ARM translation — but translation changes timing, and some native checks misbehave. Prefer a real ARM device for the final verification pass.
## ADB basics worth pinning down
```bash
adb devices -l
adb -s <serial> shell getprop ro.product.cpu.abi
adb -s <serial> shell getprop ro.build.version.release
adb -s <serial> shell getprop ro.build.version.sdk
```
Root:
```bash
adb -s <serial> shell "su -c id"
```
`pm grant` and `pm install` frequently require root on OEM builds; the `shell` user gets `SecurityException`.
**Always specify `-s <serial>` when more than one device is attached** — otherwise adb errors out or picks the wrong one.
## Shell quoting (a real time sink)
The host shell expands `$`, `|`, `>`, and quotes **before** adb sees them. Windows PowerShell additionally mangles `$var:`, `[^"...`, and `$(...)`.
Symptom: "Could not find a part of the path", "Missing type name after '['", "no closing quote".
**Fix:** never inline device commands in the host shell. Use a small helper that calls adb from a script file and passes the command as a single argument:
```python
subprocess.run([ADB, '-s', SERIAL, 'shell', 'su -c "%s"' % cmd])
```
See `scripts/devsh.py`.
Same class of bug: `javac` reads sources using the platform default encoding. Pass `-encoding UTF-8` or non-ASCII comments break the build.
## Giving an offline device network over USB
Use when the device has no usable network (broken Wi-Fi, no SIM, restricted network) but your host does. **`adb reverse` runs on the device's loopback, so it needs no device-side network interface at all.**
```bash
# 1) host HTTP/HTTPS proxy (CONNECT-capable)
python scripts/usb_net_proxy.py 8080 proxy.log
# 2) forward device-localhost:8080 to host:8080
adb -s <serial> reverse tcp:8080 tcp:8080
# 3) point the device at it
adb -s <serial> shell "su -c 'settings put global http_proxy 127.0.0.1:8080'"
```
Notes:
- `adb reverse` **does not survive a device reboot** — recreate it after any restart.
- Keep the host proxy process alive; if it dies, the device goes offline again.
- Some SDKs bypass the system proxy entirely (many ad SDKs do). Proxy logs therefore **undercount** traffic — do not conclude "no ad traffic" from proxy logs alone; confirm with a runtime DNS hook.
- Clean up when done: `settings put global http_proxy :0`.
## Preflight — run this before every experiment block
`scripts/preflight.py` checks, read-only, everything that silently fakes a failure: device
reachability, whether more than one device is attached without an explicit serial, root, the
runtime translation layer, device/host clock drift, a leftover device-wide proxy setting, stale
port forwards, a dead device server, the ABI the package manager actually chose, and the free
space left for installs.
```bash
python scripts/preflight.py --pkg <app.package> --expect-root
python scripts/preflight.py --cleanup # also clears a leftover proxy and stale forwards
```
**The rule it exists to enforce:** *do not attribute a failure to your patch while preflight is
dirty.* A surprising fraction of "my change broke it" is a device that was already in a bad state.
Costing thirty seconds here is cheaper than costing several rounds to a wrong conclusion, and the
wrong conclusion is the one that gets written down and trusted later.
## Emulator notes
Emulators are the right place to iterate and the wrong place to conclude.
- **Verify the app can install at all.** INSTALL failures on emulators are common (ABI, min SDK,
vendor checks, a stale copy with a different signature) and are usually **not** related to your
patch. Get a clean install of the *unmodified* build working before you change anything.
- **Snapshot/rollback is the main advantage** — use it to A/B two builds quickly, and to get back
to a known-good state after a destructive experiment.
- **Never quote emulator behavior as proof for a device-only question** (and vice versa). An
emulator that runs the app is not evidence about a physical ARM device.
- **Most vendors ship a console binary that is far more reliable than the GUI.** Learn yours early;
it is what you will need when the GUI is unresponsive and adb is already down.
| Vendor | Typical console binary | Useful verbs |
|---|---|---|
| LDPlayer | `dnconsole.exe` (next to the player exe) | `list2`, `launch --index N`, `reboot --index N`, `quitall` |
| MuMu | `MuMuManager.exe` | `info -v all`, `control -v N launch`, `control -v N shutdown` |
| AVD / Android Emulator | `emulator.exe` | `-list-avds`, `@<avd>`, `-no-window` |
| Genymotion | `gmtool` | `admin list`, `admin start` |
- **Recovery order when `adb devices` goes empty.** Do these in order; the first two are the ones
people skip:
1. `adb kill-server && adb start-server` — clears a wedged host daemon.
2. Re-run `adb devices`. If still empty, check whether the **emulator process is actually alive**
(`tasklist` / `ps`). A running process with **no listening port** means the VM never finished
booting its adb bridge — restarting the *device* is the only fix.
3. Restart the instance through the vendor console (`reboot --index N`). Restarting via the
player binary's "launch" verb frequently leaves you with a process and still no adb.
4. Only then consider that something is wrong with the host.
Symptom worth memorising: **the emulator process exists but nothing is listening on the adb
port.** Distinguishing "not running" from "running but not booted" is what makes step 2 worth
doing — they have different fixes and only one of them needs the GUI.
- **Root is a per-vendor setting, not a given.** Most emulators expose it as a toggle in their
settings ("root permission" / "ROOT 权限"); some ship a separate rooted image. Always verify
rather than assume:
```bash
adb -s <serial> shell "su -c id" # want: uid=0(root)
```
A rooted emulator is usually the fastest environment for everything in this skill, which makes it
easy to forget that it is also the *least representative* one.
- **Running two instances.** Different serials make two emulators genuinely disjoint, which is
useful for holding a clean control (original build, untouched state) alongside your workbench.
Keep them labelled, always pass `--serial`, and **do not run the same experiment on both** — the
value of a second device is independence, and duplicating work destroys it. Operate one at a
time; the other is a reference point, not extra throughput.
- **Apps can detect the emulator** and change behaviour or refuse to run. If the app behaves
differently here than on hardware, that is a finding about the emulator, not about the app. Note
it and move the question to a real device.
## Which architecture is actually executing
Device selection and architecture are the same question. `getprop ro.product.cpu.abi` reports what
the device claims; it does **not** report what is executing. Read `native-and-so.md` §Cross-architecture
before you choose which library to patch, and `scripts/lib_map.py` to see the live truth.
## Install / reinstall
```bash
adb -s <serial> shell "su -c 'pm uninstall <pkg>'"
adb push out.apk /data/local/tmp/x.apk
adb -s <serial> shell "su -c 'pm install -r -t -d /data/local/tmp/x.apk'"
```
- `-r` reinstall (keep data, same signing key) · `-t` allow test-only · `-d` allow downgrade
- Different signing key than the installed app → must uninstall first (data is lost)
- Vendor installers may reject `adb install`; pushing + `su -c pm install` usually works
**After reinstalling:** if you restored a data directory, fix ownership, or the app crashes in a DB-init path:
```bash
su -c "chown -R <uid>:<uid> /data/user/0/<pkg>"
su -c "restorecon -R /data/user/0/<pkg>"
```
`<uid>` from `dumpsys package <pkg> | grep userId=`.
## Driving the UI from adb
Automated UI interaction is where verification loops usually break, and most of the breakage looks like a bug in your script. Most of it is not.
### `input tap` may simply not work on a given control
On some OEM ROMs `input tap <x> <y>` is silently ignored for certain controls while working fine for others at neighboring coordinates. **Do not conclude that your automation is wrong.** Before doubting the script:
```bash
adb -s <serial> shell "su -c 'uiautomator dump /sdcard/ui.xml'"
adb -s <serial> pull /sdcard/ui.xml
# bounds="[x1,y1][x2,y2]" -> tap the center: ((x1+x2)/2, (y1+y2)/2)
adb -s <serial> shell "su -c 'input tap <cx> <cy>'"
```
If the control still does not react, bypass the UI path entirely — `am start -n <app.package>/<activity>`, or invoke the logic from Frida. A finished task needs the *code path*, not the tap.
`input` requires `INJECT_EVENTS`, which the plain `shell` user does not have. An unprivileged `input` **fails silently** — no error, no effect. Always wrap it: `su -c 'input tap ...'`.
`input text` additionally mangles or drops non-ASCII input. For CJK text, either install an ADB-driven helper IME or set the field from a runtime call rather than typing.
### Look at the screen — do not drive and wait blind
The most expensive habit in device work is: tap a coordinate, sleep, tap again, sleep, conclude
something about the app. A screen that is *looked at* answers in one step what coordinate-guessing
cannot answer in five — the layout shifted, a different dialog came up, a countdown is frozen, the
button is disabled, the text on screen says exactly why.
**Treat a look as a routine step, not a debugging last resort.** Capture:
- **immediately before** anything time-dependent, so you know the starting state;
- **during** a wait, at intervals — state changes are the information, and a single sample at the end
cannot distinguish "it progressed" from "it never moved";
- **at every decision point**, before choosing the next action;
- **on any surprise**, before forming a theory about it.
`scripts/snap.py` does this with sane bounds, and tells you which kind of evidence you actually got:
```bash
python scripts/snap.py --out shots --tag before
python scripts/snap.py --out shots --tag waiting --count 6 --interval 3
```
**Use the stall detector.** If consecutive samples are byte-identical, nothing is happening and more
waiting cannot help. Stop, and go find out why — that is a different investigation from waiting
longer.
### Two kinds of screen evidence, and which one is trustworthy here
| Evidence | What it gives you | When it is the right one |
|---|---|---|
| **the image** | exactly what is rendered: layout, which dialog, disabled states, drawn text | always available; the **only** evidence for runtime-drawn UI |
| **the control tree** (`uiautomator dump`) | precise `bounds`, exact text, diffable | only when it actually has content |
The tree is more convenient *when it exists* — it gives you tap coordinates and text you can diff. But
**verify it has content before planning around it**:
```bash
adb -s <serial> shell "su -c 'uiautomator dump /sdcard/ui.xml'"
adb -s <serial> pull /sdcard/ui.xml
grep -c '<node' ui.xml # 0 nodes -> the tree is useless for this screen
```
**A runtime-rendered UI frequently exposes no real controls at all.** Cross-platform runtimes, web
views and canvas-drawn surfaces often produce an empty (or text-free) tree, which is why a plan built
on "read the bounds out of the XML" stalls on exactly those apps — and also why tapping a control that
"should be there" silently does nothing. When the tree is empty, the image is the **primary** evidence,
not a fallback, and you read it directly rather than trying to derive coordinates from a tree that does
not exist.
`screencap` itself returns a 0-byte file on some ROMs. Write on-device and pull instead:
```bash
adb -s <serial> shell "su -c 'screencap -p /sdcard/x.png'" && adb -s <serial> pull /sdcard/x.png
```
If that is also empty, do not fight it — but note that **a 0-byte capture is not evidence the screen is
blank** (`pitfalls.md` P20). And if `uiautomator dump` fails with `could not get idle state`, retry once;
a paused animation is usually the cause.
### Verify form input by reading it back
Filling a form and tapping submit is **not** a verified interaction. Read the actual field contents and lengths back first:
```bash
adb -s <serial> shell "su -c 'input text <value>'" # no literal spaces; %s encodes one
adb -s <serial> shell "su -c 'uiautomator dump /sdcard/ui.xml'"
adb -s <serial> pull /sdcard/ui.xml
grep -o 'text="[^"]*"' ui.xml # every populated field
```
A whole class of "the button does nothing" is really a local validation rejecting the input — two password fields of different length, a required field empty, a format check. The app returns **before** issuing any request, so a network probe stays silent and the tap looks broken. Compare the lengths you read back, fix the input, retry.
### ROM background freezing kills your hooks
Aggressive ROMs freeze backgrounded apps; the logcat signature is a process state transition `state: R -> F` (running to frozen). Once frozen, hooks stop firing and network calls stop, which looks exactly like a broken probe.
```bash
adb -s <serial> shell "su -c 'dumpsys deviceidle whitelist +<app.package>'"
adb -s <serial> shell "su -c 'cmd appops set <app.package> RUN_IN_BACKGROUND allow'"
```
While debugging, **do not press HOME** — backgrounding the app is what triggers the freeze. Return to it with `am start -n <app.package>/<activity>` and keep it in the foreground for the whole session.
## Signal extraction (what to actually read)
```bash
adb -s <serial> logcat -c # clear before the run
adb -s <serial> shell "am start -n <pkg>/<activity>"
adb -s <serial> logcat -d -v brief | grep -E '<pkg>|FATAL|VerifyError|IncompatibleClassChange|uncaughtException'
```
Failure signatures worth memorizing:
| Log line | Likely cause |
|---|---|
| `FATAL EXCEPTION` + Java stack | App-level crash — read the stack |
| `VerifyError` / `IncompatibleClassChangeError` | Damaged dex (round-trip or bad rewrite) → `pitfalls.md` P3 |
| `Failure starting process` (no stack, by `ActivityManager`) | ART refused the dex, or device state is broken → `pitfalls.md` P4, P9 |
| `ClassNotFoundException: <App.Application>` | Dex rejected wholesale (ordering / structure) → `pitfalls.md` P2 |
| `uncaughtException` with **no stack**, right after launch | A crash-reporter SDK swallowed it. Check whether the process is gone, and look for the SDK's own log file under the app's data dir |
| App alive but nothing rendered | A swallowed exception in the UI path; hunt the crash-reporter's log file |
**Crash-reporter SDKs hide your stack traces.** When an app installs a global uncaught-exception handler (友盟/UCrash/Bugly etc.), the Java stack never reaches `logcat` — only a line like `uncaughtException time: ...`. Three ways to get the stack:
1. **Frida**, hooking `Thread.setDefaultUncaughtExceptionHandler` or the handler class (most reliable).
2. **Race the reporter's log file**: it writes a file under `<app data>/<sdk>/...` then uploads and deletes it. Poll it every ~0.2 s from the device shell and copy on sight:
```bash
while [ $i -lt 400 ]; do
for f in <dir>/*.log; do [ -f "$f" ] && cp -f "$f" /data/local/tmp/capture.log; done
i=$((i+1)); sleep 0.2
done
```
3. Run a build with the reporter disabled (not always possible).
## Determinism
Before concluding anything from a failure: **reboot the device and retry**, and **run a control build with zero patches**. A surprising share of "my patch broke it" turns out to be device state (`pitfalls.md` P9).

102
vendor-new/apk-reverse/skills/apk-reverse/references/evidence-summary.md Normal file
View File

@@ -0,0 +1,102 @@
# Evidence summary — what is proven, how strongly, and where to look
Load this when a claim's strength decides whether you trust it, and the command that produced it is
not in front of you. It is the condensation that travels with the skill: `npx skills add` installs
`skills/apk-reverse/` only, so the run records, the tool verdicts and the public-target benchmark
matrix stay behind at the repository root. Every row below therefore answers four questions inside an
installed copy — **is it proven, how strongly, which capability is blocked, and which files to open
next** — and points at the one section that says "the full record is elsewhere, and here is its name".
## How to read this file
- **Status** is one of `ok` (route carried out on a real target, result recorded), `partial` (a
decisive step — usually install, launch, or a second independent producer — was not run), or
`blocked` (the route depends on something this skill does not ship, or nothing here is evidence
either way).
- **Strength** is the repository's three-tier label: `observed` (a command was run and its output
exists behind the claim), `inferred` (follows from a documented mechanism or a neighbouring
measurement), `unverified` (assumed or reported elsewhere, not reproduced here). `observed` is
reserved strictly.
- **Evidence** names files that exist in your install. When a claim's full record does not ship, this
file says so once, in one place, instead of scattering dead paths through the table.
- Machine-readable companions: `evidence/capability-matrix.json` (the same rows, with more fields),
`evidence/tested-tool-versions.json` (versions and the probe behind each), and
`evidence/known-limitations.md` (the installer-facing limit list).
## The capability matrix
| Capability | What is proven, in one line | Status | Strength | Evidence in this install |
|---|---|---|---|---|
| Equal-length dex surgical patch | 2 bytes rewritten in a 5,528-byte dex; whole-file diff is those 4 bytes plus header bytes 8..32; the written file reproduces both header integrity fields | ok | observed | `references/byte-level-patching.md`, `references/patch-audit.md`, `scripts/dex_patch_bytes.py` |
| Dex method-level rewrite (dexlib2) | Documented, not measured: the smali round-trip blind spot is real and a tree round trip can pass every table check and still fail at load | partial | unverified | `references/dex-patching.md`, `references/patch-audit.md` |
| Dex header integrity and verifier legality | Recompute order proven on one written file; `dex_classdiff` passing is necessary, not sufficient — it cannot see code-item damage | ok | observed | `references/patch-audit.md`, `scripts/dexutil.py`, `scripts/dex_check_verifier.py` |
| Dex string-constant patch | Equal-length only; the shipped script refuses unequal lengths, so a different-length edit is a different, unmeasured route | partial | unverified | `references/byte-level-patching.md`, `scripts/dex_strpatch.py` |
| Repack, sign, install (single APK) | End to end: STORED and 4-byte-aligned `resources.arsc`, v1+v2+v3 true, install succeeded, on-device hash matched the local build, control build still fails the old way | ok | observed | `references/repack-and-sign.md`, `references/verification.md`, `scripts/repack.py` |
| Split APK / App Bundle sets | Analyze, unified re-sign and merge measured on two real sets; merge correctly refuses a member carrying its own `resources.arsc` | partial | observed | `references/split-apk.md`, `scripts/repack.py` |
| Third-party build audit | Not measured: the APK differ is on the list of scripts no pass has run | partial | unverified | `references/third-party-builds.md`, `scripts/apk_diff.py` |
| Extraction-shell detection (trivial-body ratio) | The ratio is bimodal, not thresholded; the old "tens of percent" rule was wrong and was deleted | ok | observed | `references/advanced-unpacking.md`, `scripts/dex_dump_validate.py` |
| Dex-VMP declaration boundary | Static criteria can rule a VMP out, never in; a hand-written opcode table produced a VMP verdict on ordinary dalvik with zero structural errors | partial | observed | `references/advanced-unpacking.md`, `references/code-virtualization-and-custom-linkers.md` |
| VMP differential opcode map | Closed loop re-derived 218 of 218 emitted opcodes with zero fabrications; no hardening platform was ever contacted | partial | observed | `references/vmp-differential-analysis.md`, `scripts/vmp_diff_harness.py` |
| Java2C versus JNI sinking | Native-declaration density separates the shapes by about 2000x and `Java_*` symbols matched dex counts 1:1; no Dex-to-C compiler output was ever built here | ok | observed | `references/java2c-and-jni-sinking.md`, `scripts/java2c_probe.py` |
| Neutralising a native terminate path | Tooling works, patch does not: writes land and survive detach, and the target still dies at the same site | partial | observed | `references/native-tamper-and-suicide.md`, `scripts/spawn_patch_detach.py`, `scripts/hook_patch_only.js` |
| Instruction-level tracing (Stalker) | Exclusion keeps the target alive; it does not restore event delivery, which stayed at zero | partial | observed | `references/native-dbi-and-deobfuscation.md`, `scripts/stalker_trace.js`, `scripts/stalker_report.py` |
| Library mapping and PLT resolution | Mapping verified on a real target; the PLT script was broken and fixed after a false negative on a symbol that exists and is called | ok | observed | `references/native-and-so.md`, `scripts/lib_map.py`, `scripts/elf_plt.py` |
| Native crash triage and swallowed stacks | Unmeasured, including on the one target whose crash reporter hid exactly the stack the tool claims to recover | partial | unverified | `references/native-tamper-and-suicide.md`, `scripts/native_crash.py`, `scripts/grab_crash.py` |
| Runtime analysis with Frida | Instrumentation is a variable: attaching is what kills some targets, and exclusion is what keeps others alive | ok | observed | `references/dynamic-frida.md`, `references/environment.md`, `scripts/run_probe.py` |
| Local runtime data (DataStore and friends) | One real container round-tripped byte-exact and an edited value was read back; SharedPreferences and SQLite edits have no equivalent measurement | partial | observed | `references/runtime-data.md`, `scripts/datastore_inject.py` |
| Anti-instrumentation triage | The check was named and timed, and the `TracerPid=0` versus four frida-named mappings asymmetry was recorded; reproducibility is bounded to a sub-second window | ok | observed | `references/detection-and-anti-analysis.md`, `scripts/anti_detect_probe.js` |
| Emulation and Frida-RPC | The RPC bridge ran end to end on a live device, but the route itself is classified inferred: no target library was emulated | partial | inferred | `references/emulation-and-rpc.md`, `scripts/frida_rpc_serve.py` |
| Schema-free protobuf decode | 26/26 built-in fixtures, 21/21 against the official runtime, 8/8 framing checks, one real container byte-exact | ok | observed | `references/protocol-reverse.md`, `scripts/protobuf_decode_raw.py` |
| Server API probing and TLS scope | Determines who owns a gate, not how to break it; the native-pinning row was never run because it needs a toolchain this host lacks | partial | inferred | `references/server-api.md`, `references/tls-and-cert.md`, `scripts/probe_api.py` |
| Dart AOT analysis (given a dump) | Decoded identically to an independent disassembler (32/32, 96/96) and re-derived a caller index with a symmetric difference of 0 | partial | observed | `references/dart-aot.md`, `scripts/dart_disasm.py`, `scripts/dart_pprefs.py` |
| Dart AOT string-table format | arm64 packed form confirmed at the byte level; the documented armv7 UTF-16 form is refuted, and the extractor's zero is a format mismatch | partial | observed | `references/dart-aot.md`, `scripts/dart_pool_strings.py` |
| Producing a Dart AOT snapshot dump | Blocked: no resolver ships with this skill and none can be synthesized here | blocked | unverified | `references/dart-aot.md`, `references/coverage-and-limits.md` |
| Packer and custom-loader identification | Never exercised end to end; the target that was measured has no packer, so the tooling was recorded as not applicable | partial | unverified | `references/packers.md`, `references/code-virtualization-and-custom-linkers.md` |
| Module delivery instead of a repack | The scaffold builds end to end, but the delivery route has no public-target measurement and needs a device with the framework active | partial | inferred | `references/lsposed-and-modules.md`, `scripts/lsposed_scaffold.py` |
| Kernel-side syscall answer forging | Blocked on the kernel side: the generator is measured, no kernel artefact was compiled or loaded, and a userspace module cannot change a return value | blocked | unverified | `references/kernel-and-environment-hardening.md`, `scripts/kernelsu_syscall_mask.py` |
| `svc` site scanning | Two independent decoders agreed on an identical 214-site set; a byte scan also matches data, so neighbour context decides | ok | observed | `references/kernel-and-environment-hardening.md`, `scripts/svc_scan.py` |
| On-device tooling (MT Manager MCP) | The service-down path is measured; the connected path needs the service started by hand | partial | observed | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
| Client-side ads and server-issued UI config | Measured on a real mid-size target's ad chain, with a repacked build's on-screen change confirmed | ok | observed | `references/ad-removal.md`, `references/server-config-and-updates.md` |
| Is a membership or paywall gate client-enforceable? | The decision framework is documented; no gate-shaped public target was put through it here | ok | inferred | `references/membership-and-limits.md`, `references/account-gates.md` |
| Update and forced-upgrade neutralisation | Documented, no measurement; the failure shape is a build that installs, runs and fails every signed request | partial | inferred | `references/updates-and-forced-upgrade.md`, `references/signature-derived-keys.md` |
| Signature-derived keys | Documented only: neither the offline candidate path nor the live read has a public-target measurement here | partial | unverified | `references/signature-derived-keys.md`, `scripts/sig_probe.py` |
| Publishing sanitisation and leak scanning | Every rule fired on a planted corpus with zero false positives on the do-not-anonymize list, and the scan found 26 strong hits in its own evidence file | ok | observed | `references/desensitization-and-leak-scans.md`, `scripts/scan_leaks.py` |
| Unity / IL2CPP, React Native / Hermes, iOS | Outside this skill: the runtime can be identified, the logic recovery is not covered | blocked | unverified | `references/framework-runtimes.md`, `references/coverage-and-limits.md` |
| Defeating a server-side authority | Out of scope by design; report it as a residual rather than patching harder | blocked | unverified | `references/server-api.md`, `references/handoff-boundaries.md` |
## Where the full record lives — repository root, not shipped
Six entries, deliberately: everything the table needs beyond your install is here, and nothing else
in this skill should point outside itself. **All six sit at the repository root and are not installed
by `npx skills add`; their paths are names, not openable files, in your copy.**
- `docs/tool-verification/README.md` — the index of the evidence record (repository root, not shipped)
- `docs/tool-verification/TOOL-VERDICTS.md` — one verdict per script and external toolchain, with the independent cross-check behind it (repository root, not shipped)
- `docs/tool-verification/FINDINGS.md` — defects, contradictions and boundary evidence about the skill itself (repository root, not shipped)
- `docs/tool-verification/REPO-DECISIONS.md` — what changed as a result, and what deliberately did not (repository root, not shipped)
- `docs/tool-verification/EXTENSION-*.md` — one file per topic, each with its own strength note and the exact commands (repository root, not shipped)
- `tests/benchmark.md` — the B1–B13 public-target regression matrix: what happened, including the negative results (repository root, not shipped)
The reference documents still cite those paths where the detail matters, because a repository reader
can open them. The labels, the one-line verdicts and the blocked list are all here, so an installed
copy never has to fail silently at a path it cannot open.
## When the evidence does not reach your case
| Your situation | Do this |
|---|---|
| The row you need says `observed`, but for a different target shape | Treat it as a reason to try the route first, not as proof it will work. Keep your own control build. |
| The row says `inferred` | The mechanism is documented or borrowed from a neighbouring measurement. Budget one cheap experiment that would make it `observed` before building on it. |
| The row says `unverified` | Nobody here has paid for the counters. Do not cite this skill as support; run the step and label your own result. |
| The row says `blocked` | Do not improvise a route. State the missing dependency, and name the cheapest experiment that would identify whether it is the real blocker. |
| The step needs install or launch, and the row stops before it | The route is not end-to-end. Install it, launch it, and exercise the feature before calling it done. |
## Failure modes
- Reading "the tool ran" as "the tool is right". One script here produced a false negative on a symbol
that exists and is called; another answers "no references" for input it cannot read at all.
- Reading a confident paragraph as `observed`. The label decides, not the prose.
- Reading `unverified` as a refutation. It is an absence of evidence here, not evidence of absence.
- Treating silence in `evidence/known-limitations.md`'s never-exercised list as support for a route.
- Presenting a route as verified when the artifact that was verified is not the artifact being handed
over — the claim ladder in `references/verification.md` exists for exactly this gap.

120
vendor-new/apk-reverse/skills/apk-reverse/references/framework-runtimes.md Normal file
View File

@@ -0,0 +1,120 @@
# Cross-Platform Runtimes
Load this when the app's UI is not native, or when Java-layer hooks produce no hits at all.
Many apps ship a native shell plus a cross-platform runtime. Their business logic, their UI, and their
feature gates live **inside the runtime**, not in the dex. Patching the dex then does nothing — the Java
code you can see is mostly glue. Determine the runtime before choosing where to patch.
## Identify the runtime early
Signals in the APK:
| Runtime | Giveaways |
|---|---|
| Flutter | `libflutter.so`, `libapp.so`, `assets/flutter_assets/`, Dart snapshot |
| React Native | `libhermes.so` or `libjsc.so`, `index.android.bundle`, `assets/index.android.bundle` |
| Unity | `libunity.so`, `libil2cpp.so`, `assets/bin/Data/`, `globalgamemanagers` |
| Cordova / Ionic | `assets/www/`, `cordova.js` |
| Xamarin / .NET | `libmonodroid.so`, `assemblies/` in the APK |
A `classes.dex` that is small relative to the total payload is another strong hint that the real code
lives elsewhere.
## The layer trap
The expensive mistake: seeing a UI element (dialog, paywall, gate) and assuming it is a native control,
then spending a long time looking for it in the dex.
If it is drawn by the runtime, the dex contains only the bridge. Symptoms of being on the wrong layer:
- hooking the obvious Java dialog/Activity classes produces **zero** hits while the UI clearly appears;
- the classes that do appear in stacks are the runtime's own, with obfuscated names;
- hooking a Java method changes nothing about the visible behaviour.
Establish layer ownership **before** investing in a patch direction.
### How to tell which layer drew a given UI
1. Enumerate the Java dialog and presentation classes and hook their show/creation paths.
2. Reproduce the UI. If any hook fires, the caller stack names the detection and you are on the Java
layer.
3. If nothing fires while the UI is on screen, the UI is runtime-drawn or native-drawn. Switch axis.
4. Expect runtime-owned windows to appear as ordinary system `Dialog`/`Presentation` objects used as
**containers** for the runtime's surface. A hit on such a container is not evidence that the visible
element is a native dialog — read the class name carefully before drawing conclusions.
## Flutter specifics
- Business logic is compiled ahead-of-time into **`libapp.so`**; UI is rendered by `libflutter.so` with
no per-widget Java objects.
- **Widgets, dialogs, and paywalls are not Java views.** They will never appear in Java stack traces,
and no Java hook can intercept them.
- The Java side is thin: one Activity, plus plugin classes. Calls cross the boundary through the
platform-channel mechanism, whose class names are frequently obfuscated.
- Because channel classes may be renamed, locate them **by method signature and call shape**, not by
class name.
### Where to intervene in a Flutter app
| Goal | Layer | Notes |
|---|---|---|
| Stop the app being killed / refused at startup | host native library | earliest point, see `native-and-so.md` |
| Observe what the runtime is told | platform channel | shows the messages that drive the UI |
| Change a decision permanently | `libapp.so` | most direct, hardest to locate |
| Change startup behaviour only | host native library constructor | survives runtime restarts |
Editing `libapp.so` is the most durable but locating Dart AOT code is genuinely hard. This subject is
large enough to have its own reference: **`references/dart-aot.md`** covers pinning the Dart version,
building a decompiler for exactly that version, the object-pool model and how to index it, the register
and boolean conventions, the three assembly signatures that identify most business logic, the locating
workflow, and how to patch this layer safely.
## Locating logic without symbols
- **Strings first.** User-visible text (dialog bodies, feature labels) is the cheapest anchor in any
runtime. Find it, then find what references it.
- **Search them in the encoding the runtime actually uses, or you will conclude they do not exist.**
This is the single most common false negative in cross-platform work. A runtime snapshot does not
necessarily store text as UTF-8; UTF-16 (little-endian on these targets) is common and entirely
valid. A UTF-8 search of such a snapshot returns **zero hits**, which reads as "the strings were
stripped / encrypted, this route is dead" — and that conclusion is wrong.
```python
blob = open('libapp.so', 'rb').read() # or whichever artifact holds the snapshot
for phrase in ('<feature label>', '<dialog title>'):
print(phrase, 'utf-8:', blob.find(phrase.encode('utf-8')),
'utf-16le:', blob.find(phrase.encode('utf-16-le')))
```
Try both, and try a short distinctive substring rather than a long phrase. Which encoding applies
is predictable rather than random: in a Dart AOT snapshot ASCII literals are stored one-byte (a
UTF-8 search finds them) while CJK / non-Latin literals are UTF-16LE (a UTF-8 search returns zero).
`dart-aot.md` §7 gives the exact framing, and `scripts/dart_pool_strings.py --find` locates either
encoding by file offset.
- **A phrase that is absent may simply never exist as one literal.** UI text is often assembled from
fragments or templates, so "the whole sentence" can be missing while both halves are present.
Search the shortest distinctive token, and expect the interesting anchor to be a *label* rather
than a sentence.
- **Landing on a string tells you where the text lives, not which code decided to show it.** Treat the
hit as an anchor for reference-hunting, not as the answer.
- **Compare two builds.** The same feature in a slightly different version often reveals the code path.
- **Watch the boundary, not the interior.** For cross-platform apps it is usually far cheaper to observe
what crosses between layers than to reverse the interior of the runtime.
- **Do not assume the obfuscated names are stable or meaningful.** They are not, and treating them as
identities leads to conclusions that break on the next build.
- **Obfuscated identifiers can be non-ASCII and invisible.** Renaming passes can replace class and
member names with characters outside ASCII (combining marks, variant selectors and similar). If a
class exists at runtime but "cannot be found" by the name you read from a decompiler, suspect
encoding or normalisation in your tooling rather than a missing class. Resolve such members by
**shape** — parameter counts, types, interface implementation — instead of by name, and prefer
runtime enumeration of loaded classes over guessing identifiers.
## What this changes about your plan
- Budget for the runtime layer **up front**. If the app is Flutter/RN/Unity, the dex is not the main
battlefield and a dex-only plan will stall.
- Any behavioural claim must be validated **through the runtime's own UI**, because that is what the
user sees. A patch that changes internal state without changing the UI is not a fix.
- Keep the layer you are working in explicit in your notes. "Patched the paywall" is meaningless without
saying which layer owned it.

70
vendor-new/apk-reverse/skills/apk-reverse/references/handoff-boundaries.md Normal file
View File

@@ -0,0 +1,70 @@
# Hand-off boundaries — where this skill ends and another view begins
Four boundaries that are easy to walk into without noticing. Each names what the other side owns
rather than restating it, because two copies of the same advice drift apart. `SKILL.md` keeps the
one-line version of each so a routing decision can be made without loading this file; this file
carries the detail.
## 1. JNI — a Java `native` declaration and its implementation are two different views of one function
This skill reads the Java side (dex) and the native side (`.so`) with different tools, so the *join*
is where analyses go wrong.
| Form | What you see | How to find it |
|---|---|---|
| Static linkage | symbol `Java_<pkg>_<Class>_<method>` in `.dynsym` | search the dynamic symbol table. Under R8 the class name is a short name, so the symbol deforms with it and a search for the readable original finds nothing |
| Dynamic registration | **nothing** in the symbol table — binding happens at runtime | find `RegisterNatives` call sites, or hook it to read the binding table. Obfuscated targets prefer this, and a symbol search fails **silently** on it |
| Native → Java callbacks | native code pulling data back through Java | follow `FindClass` / `GetMethodID` / `CallObjectMethod` |
`FindClass`/`RegisterNatives` in a `.so` tell you a JNI boundary exists even when no `Java_*` symbol
does. **Strength note:** the three rows above are documented behaviour, not results from the first
verification pass, which did not trace a JNI boundary end to end; `FlutterJNI.loadLibrary` appearing
in a dex is the closest it came. Treat them as a map, not as a measurement.
**The benchmark pass then measured the failure the middle row predicts:** a real dynamically
registering library contains **zero `RegisterNatives` symbols**, because the C++ `jni.h` inlines the
call — so a symbol-based search fails structurally, not by bad luck. The combination that did fire
on a real library was *"exports `JNI_OnLoad` and zero `Java_*`"*. The authoritative treatment is
`references/java2c-and-jni-sinking.md` §The JNI boundary — why a symbol search fails silently; this
section is the pointer, not the source.
## 2. Hardening — a dex-side packer observation is a native-side implementation question
If the dex turns out to be a shell, the logic is behind a loader and the analysis moves to the `.so`
that performs the unpacking. `packers.md` owns the dex-side identification; the native deep dive
belongs on the other side of this boundary. **Not exercised by the first verification pass** — that
target had no packer, so this pointer carries no measurement from it. The benchmark pass measured the
*shape discrimination* on public samples but still did not run a live extraction shell
(the benchmark matrix (`references/evidence-summary.md` §The capability matrix) row B3).
## 3. The existing native boundaries — read native anomalies from the APK side, not from inside
`native-and-so.md` and `native-tamper-and-suicide.md` are deliberately scoped to what you can
conclude *from the APK side*: a repacked build that dies instantly with a null-looking fault, a
Java-layer check that reports success while the process dies, a terminate path you made not-return.
That judgement belongs here, because it is about deciding whether your *patch* caused the death. Deep
native work — restoring a symbol, rebuilding a call graph, reversing an OLLVM function — is a
different activity with a different toolchain. Point across rather than duplicating: if you need the
latter, say so instead of extending those two files into it.
## 4. When the deliverable stops being an APK, the verification question changes with it
`SKILL.md` §What "done" means is written for a rebuilt, installable artifact, and every word of it
assumes one. The other three forms in G1 each move the evidence somewhere else, and the failure mode
is quiet: **a privileged result gets reported in the language of a finished build.**
| Form | What "verified" now means | What is *not* evidence |
|---|---|---|
| **LSPosed / Xposed module** | The module was loaded into the target and its hook produced an observable effect **in the target's own log**, on a named build of the target | The module installed; `pm path` returned a path; the package was enabled in the manager. All three are true of a module whose entry class does not exist in its own dex |
| **Local RPC / emulation service** | A call returned the value the app itself would produce, from a named target build and a named device or emulated environment — and the harness survives a reconnect | "The script loaded"; a call that returned *something* without a reference value to compare against |
| **Analysis report with a stated boundary** | The evidence chain (commands, outputs, and the layer each conclusion belongs to) plus the boundary — what was **not** determined and why | Any implication that a route was exhausted when it was only abandoned |
The privileged-form drift R1 warns about lives here. `references/lsposed-and-modules.md`,
`references/emulation-and-rpc.md` and `references/verification.md` each carry the specific check; this
table is the reminder that changing the deliverable's form is a decision that must be re-stated out
loud, not a quiet downgrade of what counts as done.
**A fifth boundary, added by the benchmark pass:** when the deliverable is a **tool or a document in
this repository**, "verified" means an independent check exists — a second implementation, an
official disassembler, a byte-exact round trip. A tool's own self-test passing is not that check.
the benchmark matrix (`references/evidence-summary.md` §The capability matrix) records which rows have one and which do not.

149
vendor-new/apk-reverse/skills/apk-reverse/references/java2c-and-jni-sinking.md Normal file
View File

@@ -0,0 +1,149 @@
# Java2C and JNI Sinking — the dex is not where the code is
Load this when the dex shows method bodies missing or replaced by `native` declarations, and
you are about to go looking for a decrypted DEX in memory. **Decide which of the two shapes you
have before you spend anything**, because one of them has no DEX to find at any point in the
process lifetime, and the search for it is unbounded.
Two neighbouring routes have historically folded this shape into one: `advanced-unpacking.md`'s
dump-shape table and the SKILL.md symptom index both point "whole classes are bare `native`
declarations" at `code-virtualization-and-custom-linkers.md`. That file is about protection which
leaves the code **reachable** — a private container, a loader, a private opcode interpreter, all of
which you can dump and measure. **Java2C is not reachable, because it was never bytecode.** This
file splits that row and owns both halves of it; the virtualization file stays what it is.
**Strength note, read this first.** The measurements below are **observed** — every number was
produced by `scripts/java2c_probe.py` against a real sample during this pass, recorded in
`references/evidence-summary.md` §The capability matrix. But no Java2C library exists in this repository and
none could be built here (no NDK, no clang, WSL unavailable — see that record), so **the
Java2C-specific identification criteria are inferred, not observed end to end**. What *is*
observed is the discriminating measurement that separates Java2C from JNI sinking, and the JNI
boundary behaviour. Treat the type table as a map; label your own conclusions the same way.
## The five shapes — one table, because the wrong row costs days
| Shape | Static shape in dex | What is in memory at runtime | Route | Cost of the wrong row |
|---|---|---|---|---|
| **Landing shell** | `Application` is a third-party class; dex is a small stub | A full, decrypted DEX | Dump memory, filter, patch | Cheap if you notice the stub `Application` |
| **Extraction shell** | Reads as a normal dex until you measure it; bodies are stubs | A DEX whose bodies are filled in **on invocation** | Dump, measure `stub%`, FART-style active invocation | Medium — you keep dumping, and every dump is a skeleton |
| **VMP** | Dex parses; bodies are *present* but decode as nonsense | A DEX containing **private opcodes** interpreted by a native VM | Measure, then usually stop and report | Medium — recovery cost usually exceeds task value |
| **Java2C** | Whole classes are `native`; **no code_item at all** for them | **Never a DEX.** The translated methods exist only as compiled C in a `.so` | Read the `.so`; rebuild the JNI call chain | **Highest in this table** — you hunt a decrypted DEX that is never produced, then blame your dump tooling, then your anti-detection assumptions |
| **JNI sinking** | A handful of `native` declarations; the rest of the dex is ordinary Java | A normal DEX | Locate the Java call site; reverse one or a few native functions | Low — but you may widen the job to the whole app if you read it as Java2C |
The one question that decides the table: **does the artifact ever hold dex bytecode for the
methods you care about?**
- Landing shell, extraction shell, VMP: **yes** — the bytecode is there, in some state, and the
work is recovering or interpreting it.
- Java2C: **no** — a translated method is a `native` declaration plus an entry in a registration
table. Nothing is decrypted, so nothing can be dumped. `[inferred]` as a runtime claim: it
follows from the translation mechanism, and this pass did not observe a live Java2C app.
- JNI sinking: **yes** — the dex was never damaged.
## Identification
### The discriminating measurement is native density
This is the single most useful number and it separates the two shapes by roughly three orders of
magnitude. Measured on real samples with `scripts/java2c_probe.py` `[observed]`:
| Sample | dex methods | `native` methods | native ratio |
|---|---|---|---|
| JNI-sinking target A (MASTG L2) | 5081 | 2 | **0.04 %** |
| JNI-sinking target B (MASTG L3) | 11182 | 3 | **0.03 %** |
| App carrying a real JNI library | 23603 | 7 | **0.03 %** |
| Java2C-shaped fixture | 29 | 24 | **82.76 %** |
A JNI *sink* is by definition surgical: a few hot methods are moved out and the rest of the app
stays in Java. **A JNI sink is not "the app is native".** When the density is in the tens of
percent, and whole classes have zero Java methods left, you are looking at a translation pass.
`[observed]` in the same runs: two of the fixture's three classes were native-dominated (>= 70 %
of >= 3 named methods), against **zero** such classes in any of the three real targets. Note that
constructors are excluded from that test — a translation pass leaves `<init>`/`<clinit>` in Java
in practice, so "every method native" is an essentially unreachable test and should not be used.
### Native-layer signatures
| Evidence | Strength | Why |
|---|---|---|
| `.so` exports `Java_*` symbols whose count is roughly 1:1 with the dex native-method count | **strong** | Measured 2:2 (sample A) and 3:3 (sample B) `[observed]`. It is the signature of a *static-linkage* translation or sink. Note it does **not** distinguish Java2C from JNI sinking — only the density does |
| The `.so` contains one C function per Java method, each beginning `JNIEnv *env, jobject thiz, ...` | **strong** | Observed in generated, uncompiled output: every translated method became exactly one function with those leading parameters |
| Toolchain strings: `Dex2C`, `dynamic_register_compile_methods`, `ScopedLocalRef`, `well_known_classes` | **strong** | These are the runtime header and entry-point names a Dex-to-C toolchain emits. `[observed]` in generated sources; `[inferred]` that they survive compilation into the shipped `.so` |
| `JNI_OnLoad` present | **weak** | Present in nearly every JNI library. Measured true on a system library with zero `Java_*` symbols `[observed]` — it told us registration was dynamic, not that anything was hardened |
| `RegisterNatives` **string** anywhere in the `.so` | **weak** | See the boundary section: NDK's C++ headers do not emit this symbol at all |
| `libc++_shared.so` / `c++_static` | **weak** | An ordinary NDK C++ setting. The Dex-to-C toolchain read here does ship `APP_STL := c++_static`, but so do countless unrelated libraries `[observed]` |
| `classes.dex` byte-count per class far below the app's own pre-hardening build | **weak, uncalibrated** | Bodies leaving the dex should shrink it. Measured 584 B/class on the fixture versus 1152–1386 B/class on the real Java targets `[observed]` — but there is no population baseline for "normal", so this only compares an app against **itself** before and after |
`java2c_probe.py` prints every one of these with its strength attached, and prints the weak ones
as an explicit warning block. A weak hit is not a verdict and must not be reported as one.
## The route: read the .so, not memory
For Java2C the analyst's instinct — dump memory and find the dex — is not merely expensive, it is
**unbounded**, because the success condition can never occur.
1. **Do not dump.** There is no target artifact. If you dump anyway, you will recover a dex that
still shows the same `native` declarations you started with, and the natural (wrong) reading is
"the dump failed".
2. **List the translated functions.** With static linkage, `Java_<PKG>_<Class>_<method>__<proto>`
names map the `.so` back onto the dex one for one. Use the dex as the index and the `.so` as
the library.
3. **Rebuild the chain backwards.** A translated function calls *out* to Java constantly. The
strings `FindClass` / `GetMethodID` / `GetStaticMethodID` / `Call*Method` / `NewObject` /
`GetFieldID` inside it are the reverse edges. Following them reconstructs what the original
Java method did, which is the only way the logic comes back.
4. **Expect a register-machine shape.** Generated code declares a slot per dex register up front
and assigns them in sequence. Reading it as if a human wrote it wastes hours; read it as a
linearizer output, matching each call site against the JNI signature it passes.
The decompiler choice matters less here than for obfuscated code: the generated functions are
long but flat and unoptimized, which is precisely the shape a decompiler handles well.
## The JNI boundary — why a symbol search fails silently
Two independent mechanisms erase the symbol that a reader would search for. Both are real; the
second was measured this pass.
**1. Dynamic registration.** Nothing named `Java_*` is exported. Binding is performed at runtime
by a registration table built in a JNI entry point. A grep over the symbol table returns nothing,
exits zero, and looks like a legitimate answer.
**2. `-fvisibility=hidden`, and an inline `RegisterNatives`.** The Dex-to-C toolchain read here
ships `APP_CPPFLAGS += -fvisibility=hidden` in its `Application.mk` `[observed]`, which hides the
generated functions from the dynamic symbol table — so even a statically-named translation can
present as "no `Java_*` symbols". Compounding it: NDK's C++ `jni.h` implements `RegisterNatives`
as an **inline member that calls through the function table**, so calling it emits *no*
`RegisterNatives` symbol either `[observed]`: a real system library exporting `JNI_OnLoad` and
zero `Java_*` symbols contains zero literal `RegisterNatives` symbols. A rule that greps for that
name is structurally incapable of firing.
**The check that does work** `[observed]`: a library that **exports a JNI entry point and exports
zero `Java_*` symbols** is registering dynamically. That combination is affirmative evidence, and
it fired on a real library in this pass.
**Confirming it at runtime.** Hook the registration entry point (`JNI_OnLoad`) and read the table
it builds, or hook the class-load path and enumerate the class's methods' native backing. Do not
conclude "no JNI boundary" from an empty symbol table; conclude "the binding is not visible in the
symbol table" and go find where it happens.
## Failure modes
| What you see | What it is not | What it is |
|---|---|---|
| Whole classes are `native`, you dump memory and find the same `native` declarations | "The dump failed" / "the tool is broken" | The translation moved the code out of dex permanently. Re-read the dex shape and open the `.so` |
| `grep Java_ .dynsym` returns nothing | "There is no native implementation" | Dynamic registration, or hidden visibility — see the boundary section |
| A library exports `JNI_OnLoad` and nothing else you recognise | "It is a loader" | It may be an ordinary dynamically-registering JNI library. A loader normally also has an `init_array` entry and a payload to map |
| Native density is high but every class still has Java methods | Java2C | Could be a JNI sink applied widely, or an R8-stripped app. Check whether the `native` methods have matching `Java_*` symbols before concluding |
| The dex looks completely unremarkable on a sample you were told is hardened | "Nothing is hardened" | A VMP whose payload is decrypted at runtime has an ordinary static dex. Measured this pass: a VMP-labelled sample had 7 native methods out of 23603 and a 4 % stub ratio `[observed]`. **Static dex metrics cannot see runtime-only hardening** — that needs a runtime check, not more reading |
## Where this file stops
- **It does not decompile the generated C.** That is ordinary native reversing; `native-and-so.md`
and `native-dbi-and-deobfuscation.md` own it.
- **It does not recover a VMP.** Private-opcode interpretation is a different mechanism with a
different cost model (`advanced-unpacking.md`).
- **The Java2C runtime claim is inferred.** That no dex bytecode exists in memory for a translated
method follows from the mechanism, not from a measurement made here. If you get a live Java2C
sample, that is the first thing to verify, and it is cheap: attach, force the class to load, and
ask whether the method resolves to Java bytecode or to a registered native entry.

299
vendor-new/apk-reverse/skills/apk-reverse/references/kernel-and-environment-hardening.md Normal file
View File

@@ -0,0 +1,299 @@
# Kernel-level and environment hardening — where to go when userspace hooking is not enough
Load this when you have already concluded, via `detection-and-anti-analysis.md`, that the target
genuinely detects your instrumentation — and that the check cannot be neutralised from userspace
because it runs before your hook, or bypasses the layers a userspace hook can reach. This file is
the **escalation map**: what the next layer down actually is, what it costs, and when it is the
wrong answer.
Division of labour, so the two files never duplicate:
- `detection-and-anti-analysis.md` owns **identifying** detection and the **cost decision**
(work around / change route / accept and report). Read it first; its Step 0 still applies.
- This file owns **the routes below userspace** — root-implementation hiding properties, kernel
interception options, and the syscall-level realities that decide whether "just hook it" was
ever going to work.
**This repository does not do kernel development.** Everything in the kernel section is a map of
externally documented mechanisms with sources, labelled `inferred` — none of it was executed here.
That is deliberate: the purpose is to stop you from spending a day rediscovering the version
gate () or the conflict (), not to teach kernel hacking.
**Extension pass, read the next paragraph before relying on it.** This file now has a companion
generator — `scripts/kernelsu_syscall_mask.py` — which emits a loadable userspace module skeleton
plus kernel-side **templates** for the three routes in. The generator, its userspace output and
its own consistency check are `measured`; **the kernel-side code has never been compiled or loaded
anywhere**, and every generated kernel file says so in its own header. A template is a starting
point that removes the blank page, not a weapon that has been fired.
## 1. The attack-surface timeline — why "just hook it" fails
An app process initializes in a fixed order, and every anti-analysis check you will meet lives at
one of these stations:
```
linker (ld-android/inker) -> relocations resolved, DT_INIT run
.init_array of each loaded .so -> constructors: earliest native code that runs
JNI_OnLoad -> packers and RASP SDKs hook here
Application.onCreate -> first Java-level check
first frame -> UI-visible checks (and the end of "early")
```
The later your tool attaches, the more of this timeline has already executed and taken its
decision. Frida in attach mode arrives after all of it; spawn mode arrives at zygote fork, which
is why `dynamic-frida.md` prefers it — but a constructor in `.init_array` can still run before
the Frida agent finishes injecting, and a check placed there fires once and stores its verdict.
What those checks actually look for, and why each one is out of reach of a naive userspace hook
(the identification side belongs to `detection-and-anti-analysis.md`; this is the *mechanics* of
why the hook does not land):
| Detection mechanism | Why a userspace hook misses it |
|---|---|
| Inline `svc #0` syscalls (bypassing libc) | PLT/GOT hooks only intercept calls that go through the PLT. Hand-written `svc` instructions in the target's own `.so` never touch libc, so there is no PLT entry to redirect. |
| `/proc/self/maps` scan for foreign mappings | The read can be done with raw `openat`+`read` syscalls (same svc trick), and the thing being detected — your injected agent's mapping — exists in the process regardless of what you hook. |
| Named pipes / thread-name probes for known agents | Frida's runtime creates identifiable threads (`gum-js-loop` etc.) and (older versions) a linjector pipe. Renaming helps only if you control the runtime's identity, not the probe. |
| Code-page checksums of the target's own `.so` | An inline hook *is* a modification of the pages being checksummed. The check catches the hook by construction; the only userspace answer is to also neutralise the checksum, which recursively has the same problem if *that* code is checksummed too. |
| `/proc/self/status` `TracerPid` read | The field is written by the kernel. A userspace hook on the reader works only until the reader uses raw syscalls (row 1). See. |
The structural lesson: **userspace hooking and userspace anti-hooking are the same layer.** A
detection routine that lives in the same process as your hook, and that is willing to use raw
syscalls and self-verification, can always in principle race you. The routes below win by moving
to a layer the check cannot see from userspace — or by not being in the process at all.
## 2. Root implementations and what each one exposes
The root method you run decides which artefacts a detector can find *outside* the target process
(manager app, daemon, mount layout, SELinux contexts). Facts below are from the projects' own
documentation (sources at the end of the section); none were re-measured here — `inferred`.
| | **Magisk** | **KernelSU** | **APatch** |
|---|---|---|---|
| How su works | Patches the ramdisk/init; a `magiskd` runs in userspace; the manager app requests su through it | `su` handled **inside the kernel**; a manager app talks to the kernel via a driver/interface. Kernel modification. | Kernel modification via **KernelPatch**: patches the existing `boot.img` kernel — no kernel source needed (the difference from KernelSU) |
| Kernel version gate | None (works on old kernels — this is why the 4.14 test device runs it) | Official support = GKI 2.0, kernel **5.10+** (in practice: shipped with Android 12). Backported to 4.14 but you must build the kernel yourself | Same GKI-era expectation; needs only the stock `boot.img` to patch |
| Access control | Manager grants per-app; denylist for hiding | Manager + **App Profile** (constrain what a rooted app can do) | **SuperKey**: KPatch installs a new syscall (SuperCall); callers must present the SuperKey credential |
| SELinux | Modifies contexts (magiskpolicy) | Kernel-side handling | Hooks/bypasses SELinux rather than rewriting contexts — threads can be rooted in app context without `libsu`/IPC |
| Module format | Magisk modules (magic mount) | Magisk-compatible modules; `/system` modification needs a metamodule (overlayfs) | APM (Magisk-compatible) plus **KPM** — kernel-space modules with inline-hook and syscall-table-hook capability |
| Zygisk | Built-in | None built-in; add ZygiskNext | None built-in; add ZygiskNext / ReZygisk / NeoZygisk |
**What this means for hiding** (`inferred`):
- **Manager identity is the shared weak point.** All three ship a manager app the detector can
enumerate by package or by scanning for its UI. All three support "randomise/repackage the
manager" — use it, and do not assume the default package name is safe.
- **KernelSU/APatch remove the userspace su daemon** the detector could otherwise find as a
process or a socket. But they add a kernel-level surface of their own; a detector cannot see it
from an unprivileged app — which is the point.
- **Magisk on an old kernel is not a disadvantage for hiding**; it is a constraint of the device.
Do not switch root schemes to win a detection race — that is §7's stop signal.
- **Module ecosystems interoperate but do not merge**: KernelSU's module system conflicts with
Magisk magic mount (per KernelSU FAQ, running both means one stops working); su-only coexistence
is possible because KernelSU patches the kernel while Magisk patches the ramdisk.
Sources: KernelSU README + FAQ (github.com/tiann/KernelSU), APatch FAQ (apatch.dev, project
originally at github.com/bmax121/APatch), fetched 2026-09. Device facts (Magisk alpha on kernel
4.14.186) are `observed` on the reference device used for this pass.
## 3. Environment-hiding modules: Zygisk-Assistant vs Shamiko
Both are Zygisk modules that hide root + Zygisk from target apps. **Do not install both** — they
overlap in function (mount-namespace handling, trace wiping) and interfere; this is documented in
the module ecosystems and was the deciding fact on the reference device (`observed`: Shamiko kept as a backup zip and uninstalled there).
The practical difference is their **denylist dependency** (from each project's own README,
`inferred`):
| | **Shamiko** | **Zygisk-Assistant** |
|---|---|---|
| Reads the Magisk denylist | Yes — but **requires DenyList enforcement to be OFF** (it takes over the handling itself) | Magisk: add the target to the denylist, **Enforce DenyList OFF**. KernelSU/APatch: enable the manager's "Umount modules/Exclude modifications" for the target app |
| Extra modes | Whitelist mode via `/data/adb/shamiko/whitelist` (empty file); docs warn of performance/memory cost, test-only | — |
| On APatch | Officially unsupported ("Shamiko is proprietary software, we cannot adapt it" — APatch FAQ) | Works via ZygiskNext |
| Version gates | Newer Shamiko requires recent Magisk/KernelSU bases | Current |
Selection rule (`inferred`): on Magisk, either works alone — pick one, configure its denylist
dependency correctly, and verify on the actual target. On KernelSU/APatch, Zygisk-Assistant (+
ZygiskNext) is the supported path. The denylist/enforcement settings are the part people get
wrong: **enforcement off, list populated** is the working combination for both.
Verification status: neither module's hiding effect was tested against a real detector on the
reference device — treat every
"this hides X" claim above as documented-but-unproven here.
## 4. Kernel-level interception — the map, and the version gate
When you need to observe or alter what a target does *at syscall level* — e.g. it reads
`/proc/self/status` via raw `svc`, and you want that read to return a spoofed page — userspace is
structurally the wrong layer. The options below exist; **none of them were executed in this
repository** (`unverified`), and each has a hard prerequisite worth checking before you read any
further:
```
uname -r # on the device. The gate for most of this section is 5.10+
```
- **Why 5.10**: Android's GKI (Generic Kernel Image) programme starts at android12-5.10; the
kernel-BPF ecosystem for tracing (bpftrace-class tools, kprobe/uprobe attachment as the tooling
expects it) targets GKI kernels. Source: the Android kernel architecture documentation on
source.android.com (GKI and eBPF pages). On the reference
device (kernel **4.14.186**) this entire row is **closed** — `observed` (the version), with the
consequence that eBPF-based tooling
(stackplz, mcp-termux's tracing half) is unavailable there. Write the one-line environment fact
per `detection-and-anti-analysis.md` Step 5 and do not revisit it.
| Route | What it gives you | Prereqs | Strength |
|---|---|---|---|
| **eBPF kprobes/uprobes** | Fire a BPF program on kernel or userspace function entry; observe syscall arguments/results system-wide, invisible to the target process | GKI 5.10+ kernel with the BTF/tracing config; root; toolchain (bpftrace or a custom loader) | `inferred` — documented upstream, not run here |
| **seccomp-BPF filter** | Per-process syscall allow/deny — can make a syscall **fail**, and seccomp can be installed by the app on itself (no root needed for self-filtering) | Any modern kernel; but a filter you install into the target requires ptrace/zygisk injection first | `inferred`. **Key limitation**: seccomp can reject or error a syscall (SECCOMP_RET_ERRNO/TRAP); it cannot rewrite the *content* of what a successful read returns. It closes doors; it does not paint them. |
| **Kernel module hooking** (KPM on APatch; out-of-tree LKM elsewhere) | Inline hooks and syscall-table hooks in kernel space — the layer that *can* rewrite what a `/proc` read returns | APatch (KPM) or a self-built kernel/LKM load path; kernel-dev skills | `inferred`. This is genuine kernel development — out of scope for this skill () |
| **Zygisk injection** (not kernel, but below the target's defences) | Run code in the target's process from zygote fork, before `.init_array` of the app's own libs | Magisk/KernelSU+ZygiskNext + a Zygisk module; no ptrace involved | `measured` as a framework (LSPosed runs this way on the reference device); a purpose-built module for a given target is `inferred` here |
The row that matters most in practice: **Zygisk is the cheap "below userspace" route** — it is in
the process earlier than any userspace tool can be, needs no ptrace (so it does not trip
`TracerPid`), and its ecosystem is maintained by other people. A custom kernel module that
rewrites one `/proc` read is a research project; a Zygisk module that hooks the target's
constructor is a build task.
### 4a. From "no weapon" to "template + gate": what the extension pass added
The gap this closes is specific. §4 above was a map with no artefact behind it: a reader told
"Kernel module hooking — inline hooks and syscall-table hooks in kernel space — the layer that *can*
rewrite what a `/proc` read returns" had nowhere to go next. `scripts/kernelsu_syscall_mask.py`
produces the artifact; this subsection states its boundary, because the boundary is the part that
matters.
**The correction worth internalising first: a KernelSU module cannot do any of this.** A KernelSU
(or Magisk, or APatch-userspace) module is a *userspace* module whose scripts run as root in the
normal world. `module.prop`, `post-fs-data.sh` and `service.sh` cannot change what `openat`
returns — not because of a version gate, but because nothing in that format is ever in the kernel's
return path. Detectors of the kind §2 and §3 discuss are not defeated by a module of that shape.
The generator emits that skeleton anyway, because it is the right carrier for the configuration and
the metadata, and it says in its own README what the skeleton cannot do.
What *can* reach the return path is one of three artifacts, and each carries a gate that has to be
checked on the actual device:
| Artifact | What it is | Gate | Status of the shipped template |
|---|---|---|---|
| **KPM** (KernelPatch / APatch) | A relocatable `.kpm` loaded by kpimg injected into the kernel image; replaces syscall-table pointers (`fp_hook_syscalln`) or rewrites prologues (`hook_wrapN`) | A KernelPatch-patched boot image, plus a **bare-metal** ARM64 toolchain (`aarch64-none-elf-gcc`) — not the NDK | `unverified` — template only, never compiled |
| **Out-of-tree LKM** | An ordinary kernel module that reaches `sys_call_table` and swaps a pointer | Kernel source matching the device's exact vermagic, and on ≥5.7 a way around `kallsyms_lookup_name` no longer being exported | `unverified` — template only; rated last of the three on cost |
| **eBPF probe** | A BPF program on a syscall tracepoint or kprobe | GKI 5.10+ with the tracing/BTF machinery | `unverified` — template only; the version gate alone closes it on most older devices |
The eBPF row carries a capability ceiling that is easy to miss and is stated in the generated file
as well: **a tracepoint can observe a syscall, not rewrite its result.** `bpf_override_return()`
only applies to functions flagged `ALLOW_ERROR_INJECTION`, which raw syscall entries are not —
so "eBPF to spoof a `/proc` read" is, on most kernels, an observation plan wearing a rewrite
plan's clothes. Observation is genuinely useful (it tells you which syscall the check uses, which
`§5` requires you to know anyway); it is not spoofing.
**Field notes on the KPM route, from a public KPM development write-up** (`inferred` here — the
author's measurements, not this repository's; source: blackr0ck, *APatch KPM 开发*,
[bbs.kanxue.com/thread-291665.htm](https://bbs.kanxue.com/thread-291665.htm), 2026-06). These are
recorded because each one is a day of somebody's time:
- **An inline hook can install cleanly and never fire.** On a GKI kernel with LTO, the exported
symbol is frequently not the call site — the callee was inlined into its only caller, so
replacing instructions at the symbol's address intercepts nothing. The reported symptom is
"module loaded, callback never ran". Pointer replacement in the syscall table does not have this
failure mode, because the syscall entry path must go through the table.
- **The instruction-sequence length in a module header is a trap of its own.** The reported cause
of a hard-to-diagnose load failure: declaring a resolved kernel function with `extern` makes the
compiler emit an undefined reference (`*UND*`) instead of allocating the slot, and the loader
refuses the module with `unknown symbol`. Letting it be a tentative definition allocates `.bss`
and the loader fills it in. The generated template states this rule in the code, and
`verify` checks for it.
- **Per-syscall callbacks run on every syscall of that number, system-wide.** The reported
incident: a `write` hook that did not first reject `fd <= 2` intercepted the framework's own log
output tens of thousands of times and the device rebooted. **Performance is correctness here** —
the first statement in every callback is a filter that rejects the common case.
- **Which hook point survives is kernel-specific, and the cheap way to find out is to start at the
syscall layer.** The same write-up reports that modifying the SELinux internal function they
first targeted crashed the kernel under every combination tried (before/after, argument,
return value, skip-origin), while a syscall-table hook on `write` proved stable and sufficient.
Treat the syscall layer as the default and move inward only with evidence.
**Measured on the reference device, and why every template here is labelled the way it is.**
`<DEVICE>` runs kernel **4.14.186+** (`adb shell 'uname -r; cat /proc/version'` →
`4.14.186+`, `Linux version 4.14.186+ (nobody@android-build) (Android (6443078 based on r383902)
clang version 11.0.1 ...) #1 SMP PREEMPT Wed Mar 30 23:32:42 CST 2022`). On that device the two
remaining gates answer themselves:
- **eBPF**: 4.14 is far below the 5.10 gate. Closed, as §4 already recorded.
- **KPM**: no KernelPatch-patched image is present, and the host has no
`aarch64-none-elf-gcc` (`kernelsu_syscall_mask.py gates` reports `NOT FOUND`, along with no
`ndk-build` and no `make`). Installing APatch means patching the boot image — a bricking risk
that this repository does not take on an APK task.
- **LKM**: no kernel source for the device, so no matching vermagic is possible.
So on this machine the honest answer to "do you have a kernel-level weapon" is: **you have a
template, a gate table and a measured statement that the gate is closed here.** That is a route
decision, not a failure — and it is the same shape of answer §6 gives for the whole escalation
ladder. Do not let the existence of the template change what you claim about it: `unverified` is
still `unverified`.
## 5. The syscall-level realities that survive every hook
Two facts decide a lot of "unexplainable" behaviour, and both belong to the kernel's design
rather than to any tool:
1. **`TracerPid` is written by the kernel, not by a library.** Anything that ptraces the target
(classic Frida attach, debuggers) makes `/proc/self/status` show it, and the only readers that
miss it are the ones you broke. Ways out, in escalating cost:
- attach-free operation: **Zygisk** (injection at fork; no ptrace) or Frida **spawn** (still
ptrace-based but only during injection — verify for your version) or a memory patch applied
before detach (`scripts/spawn_patch_detach.py`, `measured`);
- kernel-level hiding of the field (KPM/eBPF rewrite of the proc read) —, `unverified`;
- **self-ptrace**: the target ptraces itself so nothing else can (classic anti-anti-debug);
works because only one tracer is allowed — but it constrains your tooling to non-ptrace
routes anyway. `inferred`.
2. **A raw `svc` cannot be intercepted from userspace at all.** No PLT, no libc, no hookable
symbol. Every plan that says "hook open/read and spoof the maps" silently assumes the target
calls libc; a hardened target does not. Verify which one you have before building the spoof:
disassemble the check's `.so` and look for `svc #0` (AArch64) / `int 0x80`-era equivalents in
the check path (`inferred`; `scripts/native_crash.py`-style capstone disassembly applies).
## 6. What the reference environment chose (and why it generalises)
On the 4.14-kernel reference device, the escalation ladder terminates early — and that is the
honest outcome (`observed` on that device):
| Layer | Status on a 4.14 Magisk device |
|---|---|
| LSPosed (Zygisk) module hooking | available — framework activation verified by log (`welcome to LSPosed!` lines, `lspd` daemon process) |
| MT Manager on-device editing/repack/sign | available (`on-device-tooling.md`) |
| eBPF tracing | closed (kernel gate) |
| KPM / kernel module route | closed on this device — no KernelPatch image, no kernel source, no bare-metal ARM64 toolchain; the **template and gate table** are shipped (§4a) |
| Frida with disguised server | available (a renamed server binary was present on the device from prior work) |
The generalisable rule: **enumerate the ladder for *your* device once, write the one-line
environment facts, and stop re-deriving them mid-task.** A closed rung is a route decision, not a
failure.
## 7. When to stop escalating — the decision table
This extends `detection-and-anti-analysis.md` Step 2 with the kernel rung. The repo's stance:
R3 (never ship or claim an unverified artifact) outranks winning the arms race; a detector you
defeated with a custom kernel module is not a deliverable anyone can install.
| Situation | Do this | Not this |
|---|---|---|
| Check is userspace, hookable, single-site | Neutralise at the site (`native-tamper-and-suicide.md` rules) | Building a hide stack |
| Check runs before your userspace tool (init_array) | Zygisk-route module, or static patch of the check itself | Earlier userspace attach racing the constructor |
| Check uses raw syscalls | Static patch of the check; or LSPosed/Zygisk module hooking the consuming code | "Hook libc open/read" (does nothing) |
| Check detects the *environment* (root/emulator), not your patch | `detection-and-anti-analysis.md` A/B/C — usually a different device or route | Kernel work |
| You are about to write a kernel module / patch a kernel | Stop. State what is blocked and the evidence; propose the static/module route | Kernel development (out of scope for this skill) |
| You have the kernel template and want to load it on a 4.14/Magisk device | Read `§4a`'s gate table, confirm the gate is closed, and say so | Patching a boot image to obtain a kernel route, on an APK task |
| A kernel-side template exists, so it feels like a weapon | It is `unverified` until a device with the matching gate compiles and loads it | Presenting a template as a working capability |
| Escalation effort exceeds the user's actual ask | `detection-and-anti-analysis.md` stop signal — switch to static | One more layer |
## Checklist
- [ ] `detection-and-anti-analysis.md` read first; A/B/C chosen deliberately
- [ ] Check located on the timeline () before choosing a counter-layer
- [ ] Root scheme's artefacts enumerated (manager/daemon/mounts) before blaming the target
- [ ] Hiding module: exactly one of Shamiko / Zygisk-Assistant; denylist configured to its spec
- [ ] Kernel rung checked against `uname -r` — one line recorded if closed
- [ ] Kernel-side template treated as `unverified` until a device with the matching gate
actually compiles and loads it (`§4a`); the userspace module skeleton is a carrier for
configuration, not a kernel capability
- [ ] Raw-syscall vs libc-call distinction verified by disassembly before building any spoof
- [ ] No kernel development undertaken as part of an APK deliverable

370
vendor-new/apk-reverse/skills/apk-reverse/references/long-task-discipline.md Normal file
View File

@@ -0,0 +1,370 @@
# Long-Task Discipline
Load this when a task is likely to run long: many rounds, many experiments, or a conversation that will
exceed what can be held in context. Also load it before resuming a task someone else (or a past you)
started.
The failure mode this file prevents is not tooling — it is **judgement drift**. Over a long task, three
things reliably go wrong, and each is expensive in a way that is invisible while it happens:
1. **You re-derive what you already proved.** Rounds get spent rediscovering a boundary or a mechanism
that was established earlier and then lost.
2. **You re-walk a route that was already excluded.** The exclusion is real, but the reason has been
forgotten, so the route looks promising again.
3. **A wrong conclusion keeps steering.** An early mis-attribution is written into your mental model and
silently removes good options, or props up bad ones, for a long time afterwards.
All three come from the same cause: **conclusions living in context instead of on disk**.
## Keep a live record, not a log
Maintain one artifact — a single file — that is the task's authoritative state. Update it as things are
learned, not at the end. What matters is not size; it is that every entry is **actionable later**.
Keep these sections:
| Section | Contents | Why |
|---|---|---|
| **Operating rules** | things that must be true every run (how to install, how to launch, how to capture) | prevents repeating a mechanical mistake |
| **Confirmed facts** | each with the run that proved it | the basis everything else builds on |
| **Refuted conclusions** | what you believed, why it was wrong, what replaced it | stops a dead idea from coming back |
| **Dead routes** | ruled out, with the evidence | the single biggest time saver |
| **Open questions** | what is still unknown | keeps the next step honest |
| **Environment** | device, ports, tool paths, credentials locations | avoids re-discovery |
Two rules make this file worth having:
- **Every claim carries its evidence.** "The shell rejects this" must include the run that showed it,
including the exact failure text. An unattributed claim is how a false conclusion gets adopted later.
- **Refutations are first-class entries.** When you find that an earlier conclusion was wrong, record
*both* the wrong conclusion and why it was wrong. Deleting the mistake loses the most valuable thing
you learned.
## Grade your own conclusions
Label every non-obvious statement with its strength, and never let a weaker label inherit the authority
of a stronger one:
| Grade | Meaning | May it justify a decision? |
|---|---|---|
| **Observed** | reproduced it, with the exact command and output | yes |
| **Inferred** | follows from an observation, but the step is reasoned | provisionally |
| **Hypothesis** | plausible, untested | only as something to test |
| **Refuted** | tested and found false | never — keep it only as a warning |
Most long-task damage comes from hypotheses drifting upward into "facts" simply by being repeated. If
you catch yourself referring to something as established, check the grade in the record. If it is not
observed, go observe it or label it again.
## Every script answers with a token, not with prose
A long task is a chain of decisions, and each decision reads the *result* of the step before it. When
that result is human-readable prose, the reading step becomes an interpretation — and the
interpretation is where a wrong conclusion enters with no error message attached. This repository has
already paid for it once: a `logcat`-only verdict concluded that a hooking module never ran, and the
module had run nine times.
So a script in this kit ends with a machine-decidable answer, and the exit code carries the same
meaning:
- **A final `RESULT=<token>` line**, from a small closed vocabulary (`clean`, `patched`, `success`,
`crash`, `timeout`, `unavailable`, …). The token is the answer; everything before it is evidence for
a human.
- **Exit codes with fixed meaning**: `0` the token names success, `1` a real negative finding, `2` the
script could not do its job (usage, unreadable input, missing dependency). **A `2` is never a
finding about the target** — that distinction is the whole point, and conflating it turns a broken
harness into a property of the app.
- **Counts as tokens too** where a count is the measurement (`PROBE_COUNT=`, `VULN_COUNT=`,
`events=0`), so "nothing happened" is a value you can branch on rather than an absence you have to
notice.
The rule for the reader is the other half: **decide from the token and the exit code, and only then
read the prose.** When the two disagree, the token is what the script's author defined and the prose
is what they were thinking that day.
## Single-variable discipline across the whole task
The most common source of a wrong *and durable* conclusion is a compound experiment: two changes, one
failure, one invented explanation. Because the failure is real, the explanation feels earned.
Consequences to enforce:
- When a route is about to be abandoned, **re-read why it was abandoned**. If the evidence was compound,
the abandonment is not yet justified — re-run it single-variable before writing it off.
- Keep a semantic control in your pipeline: a build with **no** changes, run through the same
install-and-launch path. If the control fails, nothing else you measure is meaningful.
## Guard against drift at natural checkpoints
Do these cheaply, and only when they can change a decision — not as ceremony.
- **Before starting a new experiment**: read the refuted-conclusions and dead-routes sections. If your
plan appears there, stop and read why.
- **Before declaring progress**: ask what the *user-visible* outcome is right now. An internal signal
improving is not progress (see P19). Re-state the actual target.
- **After any surprise**: write it down immediately with evidence, before you have a theory. Theories
written after the fact are hard to distinguish from observations.
- **When context feels long**: prefer writing to the record over re-reading conversation. The record is
what survives; the conversation is what gets truncated.
- **When resuming**: read the record first, and treat anything not in it as unknown, even if it feels
familiar.
## The most expensive drift: solving it in an environment the deliverable will never see
There is one drift in this domain that costs more than all the others, because it produces a
result that looks like success and is not one.
**A runtime-only result is easy to obtain and easy to mistake for a finished artifact.**
Editing a data file, hooking a live process, blocking a hostname, holding a proxy open — each of
these can make the app behave correctly *on the machine where you did it*, with no repackaging
required. It is often the fastest path to a visible win. It is also frequently **not the
deliverable at all**, because the requirement was never "make it work here".
The constraint axes that decide this — check every one against the original request, and write
down the answer as a testable sentence before you start:
| Axis | The question | Why it silently invalidates work |
|---|---|---|
| **Privilege** | must it run **unrooted**? | a rooted-only result cannot be given to a normal user at all |
| **Modification form** | must it be a **rebuilt/installable artifact**, or is a live-instrumentation result acceptable? | hooks and data edits do not ship inside an APK |
| **ABI / device class** | which ABI, which device family? | an emulator-only or x86-only result is not evidence for a physical ARM device |
| **Network** | must it work **online**? | a fix that depends on being offline or on a host proxy fails the moment it is used normally |
| **Persistence** | must it survive restart, upgrade, and a fresh install? | in-memory and session-scoped state evaporates |
| **Distribution** | does the *shipped file* have to be self-contained? | anything that needs a helper on the machine is not a shippable artifact |
**Checkpoint question, asked at the same moments as the drift checks above:**
> *If I handed over exactly what exists right now, would it satisfy the constraint sentence I wrote
> at the start?*
If the answer is no, you have made real progress on a **component**, and that is worth stating as
such — but it is not the task. Do not let "it works" stand in for "it works under the constraint".
**What to do instead of over-claiming.** Split the result into two explicitly labelled parts:
1. **What is achievable inside the constraint**, and how far along it is.
2. **The unconstrained workaround** (needs root / needs a host / needs a proxy), stated as a
deliberate fallback, with its requirements made obvious to the reader.
A privileged workaround is genuinely useful — it can be the difference between using the app and
not. It becomes a problem only when it is presented as the deliverable. Report it as a fallback,
keep the constrained goal open, and say plainly which one you have.
**Also watch the inverse**: if the constraint is "must be a rebuilt artifact", do not let a working
runtime result quietly close the investigation. Use it for what it is good for — it proves the
mechanism and identifies the exact code or data to change — then port that finding into the
artifact. The runtime result is the map, not the destination.
## Bound every wait
A long task stalls in ways that produce no information and consume the most valuable resource you
have: wall-clock time and the next person's patience. Three habits prevent almost all of it.
**1. Every command has a timeout, and its absence is the bug.** A helper that shells out without a
timeout can hang forever, and the failure presents as "the task stopped making progress" rather
than "this call blocked". Pass an explicit timeout on every subprocess, every HTTP request, every
device call. When one expires, the result is **unknown**, not failed — record it that way and
re-check state before retrying.
**2. Calibrate the expected duration from measurement, not from a guess.** A timeout only means
something if it is set relative to how long the operation *should* take, because that is what makes
"slow" and "hung" distinguishable. Guessed budgets are either so short that healthy work gets killed,
or so long that a stall looks like patience.
So: **measure once, write it down, then derive the bound.**
| Operation class | How to bound it |
|---|---|
| host-side tool (`baksmali`, `apksigner`, a compiler) | seconds to low minutes; time it once, then set roughly 3–5× observed |
| device shell call (`adb shell …`) | seconds — **except** the first `su` after a reboot, which can prompt and block |
| install / push of a large artifact | minutes; scales with size, not with the app's complexity |
| app cold start to first frame | tens of seconds — **longer with a packer**, which does real work before your code runs |
| a step needing a human or an external service | **unbounded in principle** — do not wait at all (see P22) |
| waiting for an on-screen state change | bound it *and* sample it (point 3) |
Record the observed durations in the live record. They are exactly the kind of mechanical fact that
gets re-derived painfully after a context loss, and having them turns a later timeout into an
interpretable signal instead of a mystery.
**Exceeding the expected window is itself the finding.** It tells you something about state — the
device is wedged, the app never reached that phase, the action never happened — and that is a
different investigation from waiting longer. Re-check state instead of extending the deadline.
**3. Every wait has a deadline, an observable to sample, and a look.** "Wait for the operation to
finish" is not a plan. Name the observable, how often you will sample it, how long you will sample
before giving up, and what you conclude on expiry:
```
waiting for: <observable> e.g. the overlay is gone / the counter incremented
sample: <interval + how> e.g. every 15 s: screenshot + current window focus
give up at: <deadline> e.g. 150 s
on expiry: <what I conclude> e.g. "did not complete within the window" -- not "cannot work"
```
**Sample with your eyes rather than sleeping blind.** When the step is gated on something visible — a
screen, a dialog, a progress indicator — capture it and **look at it** instead of sleeping through the
interval. A fixed sleep either wastes time or measures the wrong instant; a look tells you what
actually happened, and byte-identical consecutive samples tell you nothing is going to change
(`environment.md` §look at the screen). This is the cheapest way to avoid spending rounds on a state
that was never going to arrive.
Three rules that follow:
- **A deadline that passes is a measurement, not a verdict.** "Still not finished after N seconds"
tells you about latency and reliability. It does not tell you the approach is impossible, and
writing it down as impossible is how a viable route gets discarded.
- **Never busy-poll a long job while you have independent work.** Start it, do the other thing, and
collect it when it settles. Polling wastes the same resource the task is already spending.
- **Prefer a bounded observable over a fixed sleep.** Poll the state you actually care about, with a
cap — and prefer a sample you can inspect over a duration you hope is right.
## Keep the observation window clean
Every conclusion in this skill rests on "the build I am looking at is the build I made". That
assumption is silently false more often than any other, and a contaminated window does not look
contaminated — it looks like a result.
Three ways it happens, all of which have produced confidently wrong findings:
- **Something else touched the target while you were observing.** Another process, another agent,
another window of your own work installed, reverted, restored or cleared the app. Your screenshots
and logs are then a mixture of two states and describe neither.
- **You are looking at a stale process.** The app was never actually restarted, so the "before" and
"after" captures come from the same run.
- **The artifact on the device is not the artifact on disk.** An install that reported success, or was
skipped because the version matched, leaves the previous build running.
Guards, in order of cost:
1. **Pin the identity, not the filename.** Hash the artifact you built, hash the artifact you intend
to test, and hash what is actually installed (pull the installed APK or read its digest) — the
filename proves nothing. Do this immediately before the observation window, not hours earlier.
2. **Take the window deliberately.** Before a capture sequence that a conclusion will rest on, state
(to yourself or in the record) that the next N seconds are for this observation only, and do not
run another install/uninstall/clear inside it.
3. **Timestamp the window.** Record the wall-clock start and end. It costs one line and it is the only
way to later notice that a teammate, a background job or your own earlier command landed inside it.
4. **Prefer one continuous capture over several short ones.** A restarted capture invites a restarted
state; a single sequence cannot straddle an install it did not perform.
When you discover a window was contaminated, **the correct action is to discard it and re-capture**,
not to salvage it. A re-run costs minutes; a wrong conclusion costs the rest of the task, and it will
be re-derived from the same bad evidence because the record says it was observed.
### Captures you never looked at are not evidence
This is a distinct failure from a contaminated window, and it survives every other guard here. The
frames are real, the timestamps are honest, the hashes are right — and nobody looked at them.
Two shapes it takes, both of which produce a confident wrong answer:
- **You took the screenshots and moved on.** The images exist; the conclusion was drawn from logcat
or from the patch itself. A burst of uninspected frames reads in a report exactly like a burst of
inspected ones.
- **You did look, but at the wrong thing.** Every frame shows a window that is not your app — a
vendor installer confirmation left over from an earlier install, a system dialog, the launcher —
and because the frames are consistent with each other, conviction grows. Consistency is not
corroboration when all the frames share the same blind spot.
Guards:
1. **Check what is on screen before trusting any capture.** One command settles it:
`dumpsys activity activities | grep -m1 ResumedActivity`. If the component is not your package,
the frames describe something else. Do this at the start of a capture sequence and again at the
end — the foreground can change mid-sequence.
2. **Inspect immediately, not "later".** Look at the first frame, the middle one, and any frame at a
moment you care about (the second the splash would have shown, the moment a dialog would have
appeared). If you cannot describe in one sentence what a frame shows, you have not looked at it.
3. **Treat identical consecutive frames as a signal.** Byte-identical frames mean the screen is
static. That is a finding — a hang, a dialog waiting for input, an activity that never changed —
not a capture artefact to be skipped over.
4. **Name what you saw, per frame, in the record.** One line each. "f03 at 2.1 s: target main screen,
list populated" is evidence; "captured 12 frames" is a file listing.
5. **A screenshot of a clean-looking screen is not proof that a dialog is absent** unless you captured
continuously across the moment it would have shown. Sampling is not observation: a modal that
appears and is then occluded can fall entirely between samples, and every frame you happened to
take shows something else.
`scripts/coldstart.py` exists to make points 1 and 5 automatic: it takes a timed burst, prints the
foreground component, and refuses to present the run as meaningful if the foreground is not your
app. It still cannot look at the frames for you — that part is on you.
## Long-context decay: the same mistake, twice
A long task has a failure mode that has nothing to do with the target: **as the
working context fills, settled conclusions lose their force.** Something that was
established and verified two hours ago becomes, by the end, just another plausible
belief — and the cheapest way to get from a stuck point to a feeling of progress is
to re-try the thing that already failed.
The symptoms, which are recognisable if you watch for them:
- You are about to run a command whose result you already recorded earlier.
- You are about to re-derive a fact (package name, an offset, a version, which
patch landed) that is in your own notes.
- You are about to re-attempt an approach that failed — and the failure is not
obviously connected in your mind to this attempt.
- A conclusion you are treating as solid was actually never verified, only
assumed early on and repeated since.
- You are rewriting a summary of the task from memory rather than reading the
record.
**The antidote is a record written for the second half of the task, not for a
human at the end.** Keep it structured so a weakened-context read still gets the
decision-relevant content:
```markdown
## Settled (verified — do not re-derive)
- <fact> [how it was verified, one clause]
## Refuted (do NOT retry — each cost time)
- <approach> — fails because <mechanism>
## Unverified assumptions currently in play
- <assumption> — becomes a problem if <condition>
## Next action, and how it will be judged
- <one step> — success looks like <observable>
```
Three rules that make the difference:
1. **Record the mechanism, not just the outcome.** "Traversal via that endpoint
broke the home screen because the child request shares the parent load" survives
context decay; "tried the endpoint thing, did not work" does not — and the
second version invites a third attempt.
2. **Re-read the record at every checkpoint, and before starting any new
experiment.** Not at the end. The cost is seconds and it is the only thing
standing between you and a loop.
3. **Two failures of the same shape means the model is wrong, not the
parameters.** Do not run a third variation. That is the same rule as the
two-strike rule in `SKILL.md`, and long-context decay is exactly what makes
people violate it.
A useful asymmetry: **re-verifying is cheap, re-deciding is not.** Reading a value
back out of the artifact takes a second and is fine. Re-running a whole experiment
because you forgot its conclusion is what costs the round.
## Handover
A handover is the record plus three things, written for someone with **no** memory of the task:
1. **Operating rules first** — anything mechanical that silently wastes time if unknown.
2. **Honest current state** — including what is *not* achieved. An optimistic summary costs the next
person far more than an accurate one, because they will build on it.
3. **The next best step, and why** — plus the dead routes, so they do not start there.
Two things belong in every handover, because they are the hardest to recover:
- **What you got wrong**, with the evidence that corrected it.
- **What you never actually verified**, stated plainly. "Never confirmed" is more useful than silence.
## Avoiding the opposite failure
Discipline can itself become a burden. Guard against that too:
- **Do not log everything.** Record only what would change a future decision or prevent a repeated
mistake. A record nobody reads is worse than no record, because it looks like coverage.
- **Do not treat the record as authority.** It is a summary; the environment is ground truth. If the
record and a fresh observation disagree, re-test — do not defend the record.
- **Do not let grades calcify.** A hypothesis that becomes testable should be tested, not archived.
- **Do not let the record's structure dictate the work.** If a section is empty because it is not
relevant to this task, leave it empty.

353
vendor-new/apk-reverse/skills/apk-reverse/references/lsposed-and-modules.md Normal file
View File

@@ -0,0 +1,353 @@
# LSPosed modules — when to deliver a hook instead of a patched APK
Load this when the honest answer to "patch the APK" is *no*: the target checks its own bytes, derives
keys from its signature, re-downloads what you removed, or is simply cheaper to hook than to rebuild.
A module is a different **deliverable**, not a different way to analyse — it changes what you ship, how
it survives upgrades, and who has to install it.
This file covers the module route end to end: choosing it, the smallest project that works, the
Gradle-free build chain, deployment and verification, what hides it from the target, and what it
cannot reach at all.
**Route honesty first.** A module does not modify the APK, so it cannot be "the patched build". If the
user asked for an installable standalone APK, a module is not that artifact, and this file will not
make it one.
## Step 0 — pick the route deliberately
Three routes ship three different things. Choose on **who runs it and for how long**, not on which one
you enjoy more.
| Route | Deliverable | Runs when | Survives target update | Cost |
|---|---|---|---|---|
| **Frida** (attach/spawn) | a script, driven by you | while a server process is alive on the device | it does not; you re-run it | lowest to start, highest per-session |
| **LSPosed module** | an installable APK the user keeps | every process start, automatically, after the module is enabled and scoped | usually yes — the hook targets classes/methods, not offsets | mid: one project, no repackaging of the target |
| **Repackaged APK** | a modified, re-signed APK | whenever the app runs | no — every update needs rework | highest: integrity, signature, packing, repack |
Reach for a module when **any** of these is true:
- the target verifies its own signature or APK digest, so a repackaged build dies on arrival
(`references/signature-derived-keys.md`, `references/native-tamper-and-suicide.md`);
- the change must persist across restarts without your laptop being attached;
- attaching a debugger/injector is itself detected, but a startup-time hook is not
(`references/detection-and-anti-analysis.md`);
- you need a Java-level behaviour change only, and the target's own code is reachable from Java.
Do **not** reach for a module when the logic you need lives in native code or in an AOT-compiled
runtime: see *What this route cannot reach* below.
## What the module can and cannot change
| Layer | Module can reach it? | Where to go instead |
|---|---|---|
| Java/Kotlin app code (incl. classes loaded late by a packer's ClassLoader) | yes, once you have the right `ClassLoader` | — |
| `Application` / `Activity` / framework lifecycle | yes | — |
| Java SDK internals (OkHttp, ad SDKs, HTTP clients) | yes | — |
| Pure native logic inside a `.so` | no — Java hooks see only the JNI boundary | `references/native-and-so.md`, `references/native-dbi-and-deobfuscation.md` |
| Extracted / virtualised dex (instruction-抽取, dex VMP) | no — the original method bodies may not exist as dex at all | `references/advanced-unpacking.md` |
| Flutter / Dart AOT business logic | no — Dart is not Java; the Java layer only hosts the shell | `references/dart-aot.md` |
| Behaviour that is *decided* server-side | irrelevant — a hook cannot remove a server rule | `references/server-api.md`, `references/server-config-and-updates.md` |
The Flutter row is the one that most often surprises: hooking every Java entry point in a Flutter app
still leaves all business logic behind `libapp.so`. Confirm which layer owns your target before
building a module for it (`references/dart-aot.md`).
## How injection actually works
The mechanism matters because every failure mode below is a consequence of it.
- LSPosed ships as a **Zygisk module** (`zygisk_lsposed`). Zygisk loads into `zygote`, so every app
process forked from it is a candidate for injection. [inferred: standard Magisk/Zygisk design]
- A module is **not** injected everywhere. For each forked process LSPosed asks its daemon (`lspd`)
whether this package is in the module's **scope**, and only then loads the module. Scope is therefore
the real gate; anything inside the module that filters on package name is a second, weaker gate.
- Module code is loaded **from memory**, not from a file mapping — the module's dex is read out of the
installed APK and handed to an in-memory ClassLoader. This is why a target scanning `/proc/self/maps`
for path names finds nothing (`references` below, *What detects you anyway*).
- The entry point is the class named in `assets/xposed_init`, and it receives
`handleLoadPackage(XC_LoadPackage.LoadPackageParam)` **before the application's own `Application`
object is attached** — the earliest Java-visible moment in the process. [measured: the injection
point is why `Application.attach` and `Activity.onCreate` hooks installed from here fire even under a
packer]
- `LoadPackageParam.classLoader` is the **application's** loader (the packer's stub loader when the APK
is packed). It is the handle you use to find classes the app loads later, and it is also the object
anti-Xposed code counts — see *What detects you anyway*.
## Minimal module anatomy
Four files. Nothing else is required — no `res/`, no launcher Activity, no permissions.
| File | Why it is required |
|---|---|
| `AndroidManifest.xml` with `<meta-data android:name="xposedmodule" android:value="true"/>` | without this key the APK installs as an ordinary app and never appears in the module list |
| same manifest: `xposeddescription`, `xposedminversion` (`82`) | shown in the manager UI; `82` is the classic XposedBridge API level and is accepted by LSPosed |
| `assets/xposed_init` | one line: the fully qualified entry class. A typo here produces a module that loads and silently does nothing |
| the entry class implementing `IXposedHookLoadPackage` | the only class LSPosed instantiates for you |
`scripts/lsposed_scaffold.py` writes exactly this set, plus a README containing the build chain below,
so the project is reproducible without Gradle:
```sh
lsposed_scaffold.py --package com.example.probe --name "Example probe" \
--hook-target com.example.target --out ./module
```
The generated entry class logs on **every** scoped process load before anything else, and hooks
`Application.attach` / `Activity.onCreate`. Both are diagnostics first: the first line proves injection,
the second names the real Activity classes (under a packer, these are the *unpacked* classes, not the
manifest's stub).
## Building without Gradle
There is no resource to compile and no dependency resolution to do, so the whole toolchain is five
commands. [measured on a Windows bench, JDK 17, build-tools 34.0.0, Android API 28 android.jar,
Xposed api-82 stub jar; end-to-end 5.3 s for a one-class module]
```sh
BT=/path/to/build-tools/34.0.0
AJ=/path/to/android.jar # platforms/android-28/android.jar
API=/path/to/api-82.jar # de.robv.android.xposed:api:82
OUT=build
javac -encoding UTF-8 -source 8 -target 8 -nowarn -cp "$AJ:$API" \
-d "$OUT/classes" src/com/example/probe/MainHook.java # 850 ms
jar cf "$OUT/classes.jar" -C "$OUT/classes" . # 259 ms
"$BT/d8" --min-api 24 --lib "$AJ" --output "$OUT" "$OUT/classes.jar" # 1143 ms -> classes.dex
"$BT/aapt2" link -o "$OUT/base.apk" -I "$AJ" --manifest AndroidManifest.xml \
--min-sdk-version 24 --target-sdk-version 28 -A assets # 126 ms
cp "$OUT/base.apk" "$OUT/unsigned.apk"
cp "$OUT/classes.dex" . # must sit next to the apk
(cd "$OUT" && "$BT/aapt" add unsigned.apk classes.dex) # 97 ms
"$BT/zipalign" -f -p 4 "$OUT/unsigned.apk" "$OUT/aligned.apk" # 76 ms
"$BT/apksigner" sign --ks mod.keystore --ks-pass pass:android --key-pass pass:android \
--v2-signing-enabled true --out "$OUT/module.apk" "$OUT/aligned.apk" # 1794 ms
"$BT/apksigner" verify --print-certs "$OUT/module.apk" # 725 ms -> v2/v3 OK
```
Four traps, all measured rather than guessed:
- **`d8` rejects a directory.** Pointing it at `build/classes/` fails with *Unsupported source file
type*. Jar the classes first and pass the jar.
- **`aapt add` resolves its argument relative to the current directory and stores the same name.** Run
it from the output directory with `classes.dex` present there; a path like `dex/classes.dex` lands
inside the APK as `dex/classes.dex` and the module never loads.
- **`jar` and `keytool` are frequently not on `PATH` even when `javac` is.** Oracle's `javapath` shim
exposes only `java`/`javac`; call the JDK's `bin` directory explicitly.
- **`-bootclasspath` no longer exists on JDK 9+**, so the Android API comes from `-cp` together with the
Xposed stub jar. `-source 8 -target 8` keeps the class-file version low enough for `d8`.
The generated README carries the same chain in copy-paste form, including the Windows variants
(`.bat` wrappers, `;` classpath separator).
## Deploy, enable, and verify
**There are two independent switches.** Getting one right and not the other is the single most common
"I built a module and nothing happens" report.
| Switch | Where | What it actually controls |
|---|---|---|
| package enabled state | `pm enable <module>` / `pm disable <module>` | whether Android allows the package's components to run at all |
| module enabled | LSPosed Manager (module list) | whether LSPosed loads the module into scoped processes |
| scope | LSPosed Manager (module → scope) | *which* packages the module is injected into |
[measured: a freshly installed module can be in the module database yet disabled — the package
manager reported `enabled=0` for the module and `pm enable` flipped it to `enabled=1`, while the
LSPosed-side enable flag was still 0 in its own database]
A third, device-level gate sits on vendor ROMs: the *install* itself can be intercepted by the ROM's
security centre. The shape is a shell-identity `pm install` returning a bare `Failure [-99]` while the
vendor security app takes the foreground, and the **same APK installing cleanly as root**
(`su -c 'pm install -r <apk>'`). If a module APK "will not install" on a phone that is otherwise
healthy, try root before suspecting the APK. [measured; the extension record has the transcript]
Verification has **two channels**, and on a device with a broken platform log only one of them works:
```sh
# 1. the platform log — where the module's line normally appears
adb logcat -s <TAG> # then start the target app; do NOT clear the buffer first
# 2. LSPosed's own module log — where it appears when logd is broken
adb shell "su -c 'cat /data/adb/lspd/log/modules_<timestamp>.log'"
```
The expected first line in either channel is the one the scaffold writes on entry — it fires before
any app code:
```
<TAG> injected: <target.package>/<process> cl=<loader>@<hash>
```
[measured] On a device whose `logd` route is broken, channel 1 is **completely empty** — not merely
`logcat -s <TAG>`, but `logcat -s LSPosed-Bridge` and a full-buffer search for the tag as well — while
channel 2 shows a complete, successful injection for every process start. A ROM whose LSPosed log
opens with `Logd maybe crashed (err=Socket operation on non-socket)` is exactly this case, and it had
nine successful injections in it while logcat showed nothing at all. Debugging with `logcat` alone
there produces the confident and wrong conclusion "the module never ran".
Decision order when that line is missing — stop at the first hit:
1. module enabled in LSPosed Manager?
2. target package present in the module's **scope**?
3. module APK still installed (`pm path <module>`)?
4. **was the target process started after the scope change?** Scope is read for newly forked
processes; a process that was already running keeps the decision it got at fork time.
[external, matches LSPosed behaviour reported independently: a scope change takes effect on the
target's next start, no device reboot required — 看雪 thread-291750]
The LSPosed-side state is inspectable from a root shell:
| Path | Contents |
|---|---|
| `/data/adb/lspd/config/modules_config.db` | SQLite: `modules(mid, module_pkg_name, apk_path, enabled)`, `scope(mid, app_pkg_name, user_id)`, `configs(...)` [measured] |
| `/data/adb/lspd/log/` | `modules_<timestamp>.log`, `verbose_<timestamp>.log`, `kmsg.log`, `props.txt` [measured] |
| process `lspd` | the daemon that serves scope decisions; it is a separate process from the target [measured] |
Two measured caveats about that log directory, because both waste time:
- `modules_*.log` **is** the module log: it is the file LSPosed appends your `XposedBridge.log()` lines
to. It may *also* contain `Logd maybe crashed (err=Socket operation on non-socket), retrying in 1s...`,
which means the platform log channel is unavailable — not that your hook failed. Check this file
**and** `logcat -s <TAG>`; a device can deliver one and not the other. [measured: nine successful
injections were in this file while logcat showed nothing]
- `verbose_*.log` is a *logcat snapshot* that includes the whole system, so it is useful for
correlating a system-side event with your run, and useless as a module log.
**Do not kill `lspd` to force a configuration reload.** [measured] On the bench it is started by
`/data/adb/modules/zygisk_lsposed/service.sh`, which runs `unshare -m sh -c "$MODDIR/daemon --from-service &"`;
the `daemon` script ends in `exec /system/bin/app_process ... org.lsposed.lspd.Main`. Magisk runs
`service.sh` once per boot, so an externally killed `lspd` is not restarted by anything you can see
from a root shell. The consequence is worse than a dead daemon: newly forked processes stop getting
scope decisions. Configure scope through the manager, which talks to the running daemon.
The same reasoning applies to editing `modules_config.db` by hand: the file is the daemon's
**persisted** state, while the decisions actually served to `zygote` come from the daemon's live copy.
A hand-edited row is not wrong — it is simply not read until the daemon reloads it, and there is no
supported way to make it reload. [inferred from the storage layout and the daemon lifecycle; the
manager path is the one that updates both]
## Hiding the framework from the target
Injecting a Java hook is trivial to detect **if the target is allowed to see the root/Zygisk
environment at all**. On a rooted device the hiding layer is a separate module, and the two common
choices differ in exactly one operational detail: how they learn which apps to hide from.
| | Zygisk Assistant | Shamiko |
|---|---|---|
| source | open | closed-source, LSPosed project |
| target selection | Magisk **DenyList** entries (or KernelSU profiles); no UI of its own | also reads Magisk's DenyList, **but requires DenyList enforcement to be OFF** |
| extra mode | — | creating `/data/adb/shamiko/whitelist` switches to whitelist mode without a reboot; documented as costly in memory/performance and test-only |
| mechanism | unmounts root-related paths and masks Zygisk presence for the selected process | same family of process-level hiding, plus its own mount/`maps` handling |
[external: Zygisk Assistant module description and Shamiko README]
**The DenyList/enforcement distinction is where people lose afternoons.** Enforcement is Magisk's own
feature that unmounts Magisk-provided files for listed processes; the *list* is shared, the *enforcement
toggle* is not the same thing. Shamiko wants the list without enforcement; Zygisk Assistant is
documented to work with the list as configured.
[measured on the bench: the DenyList was **enforced but empty** — `magisk --denylist status` reported
`Denylist is enforced`, `magisk --denylist ls` printed nothing, `/data/adb/shamiko` did not exist, and
Zygisk Assistant v2.1.4 was installed. So on that device nothing was being hidden from any app, while
LSPosed and `lspd` were running normally. If a target seems to know it is on a rooted device, check
this before blaming the module.]
Practical rules:
- **Keep the scope minimal.** Every extra scoped package is both an extra place your hook can break
something and an extra ClassLoader in that process (next section).
- Hiding and hooking are separate problems. A module that is perfectly hidden can still be detected by
the ClassLoader count; a target that never checks cannot see either.
- If the target is the *only* app you care about, scoping to it alone is both the smallest blast radius
and the least observable configuration.
## What detects you anyway
Traditional Java-level anti-Xposed checks — `Class.forName("de.robv.android.xposed.XposedBridge")`,
exception-stack scanning for Xposed frames, reflection on `findAndHookMethod`, enumerating
in-memory-ClassLoader instances — are all **query interfaces**, and hiding layers exist precisely to
falsify their answers. They are no longer a reliable signal that a module is present or absent.
The check that does not depend on a query interface is the **ClassLoader count**. [external: 看雪
thread-291750, which reports a clean baseline of 3 versus 12 with a module injected, threshold ≥9; the
original idea is traced there to thread-289567]
The reason it works, in one paragraph: ART's `ClassLinker` keeps a `class_loaders_` list of every live
ClassLoader, and that list is part of GC reachability. A module's loader must stay on that list or the
GC will collect the module's code and the hook dies. So a module cannot both remain alive and be absent
from a structure the runtime itself walks. Hiding layers can fake lookups; they cannot remove a node
from a GC-traversal list without unloading the module.
The same source reports two further native-side traces of the same injection, which are worth knowing
because they are independent of ClassLoader counting: the module's dex shows up as **anonymous
executable memory** (it is loaded from memory, so no file mapping exists), and Zygisk's patching of
`libart` leaves **private-dirty pages** in the library's executable segments.
Engineering consequences for anyone choosing this route:
- **Scope discipline is detection hygiene.** Fewer scoped packages means fewer injected processes; a
multi-module stack inflates the same count. The number is per-process, so a clean device elsewhere
does not help this process.
- **A count check needs a calibrated baseline.** On complex hosts (WebView, split APKs, plugin
frameworks, other modules) the clean count is already high, and the threshold has to be measured per
host, never copied. [external: same source]
- **Detection of the framework is not the same as detection of your change.** If the target reacts to
the count by refusing to run, that is an environment finding to report
(`references/detection-and-anti-analysis.md`), not something to escalate against indefinitely.
- If the target's check is native and inline-hooks *your* detector's counterpart, you are in DBI
territory: `references/native-dbi-and-deobfuscation.md`.
## LSPosed vs Frida — the division of labour
They are not competitors; they are the two answers to "how long must this hook exist".
| Need | Frida | LSPosed module |
|---|---|---|
| Understand a flow once, now | **yes** — interactive, no build step | no: build + install + restart per iteration |
| Ship something that works every launch, unattended | no: needs a server process | **yes** |
| Hook a `.so` function directly | **yes** (`Interceptor`, Stalker) | no: Java hooks stop at JNI |
| Survive a target that kills external injectors | weak: `ptrace`-based attach is the thing being detected | **stronger: no external process, no attach, no port** |
| Replace a method's return value in app code | yes | yes |
| Be re-runnable after a target update | yes (re-attach) | usually yes (class/method names auto-update far better than byte offsets) |
| Need no root on the target device | no | no |
The productive pattern is to use **both in sequence**: Frida to discover which class and method matter
and what the real call chain is (`references/dynamic-frida.md`), then a module to make that finding
permanent without touching the APK. Rebuilding a module for every question is the expensive way to
explore; exploring with an interactive script and only then writing the module is the cheap way.
A third option exists when the goal is *calling* target code rather than changing it — exporting a
native function over RPC: `references/emulation-and-rpc.md`.
## Failure modes
| Symptom | Most likely cause | Check |
|---|---|---|
| module never appears in the manager | `xposedmodule` meta-data missing/mistyped after manifest merge | `aapt2 dump xmltree` the built APK, or re-read the manifest |
| module listed, no `injected:` line in either channel | module disabled in manager, package not in scope, **target process not restarted** after the scope change, or you looked at only one channel | the four-step order above, then the two-channel check |
| the platform log is empty but the module log is not | the ROM's `logd` route is broken; the module is working | the tell is `Logd maybe crashed (err=Socket operation on non-socket)` at the top of the module log |
| `injected:` line present, but `TARGETS` gate skips the process | in-module gate, not the framework | read the module's own log line, then its gate |
| hook installed, method never fires | wrong class/loader: packers and plugin frameworks hold several ClassLoaders | resolve the loader at runtime; enumerate `/proc/<pid>/maps`-independent evidence via `references/dex-patching.md` |
| target dies right after injection | target side: ClassLoader counting or anonymous-memory heuristics (above); device side: ROM behaviour | instrument-free control run first (`references/detection-and-anti-analysis.md`) |
| `pm enable` had no effect on injection | `pm` state and LSPosed state are different switches | the two-switch table above |
| `d8` fails on a directory / `aapt add` cannot find the dex / `keytool` not found | build-chain traps, all three measured | the four traps above |
## Verification for the record
- which module APK, its hash, and what it was built from (scaffold output or hand-written);
- the build commands and whether `apksigner verify` passed;
- `pm path`/`dumpsys package` output for the module (installed, enabled state);
- the scope list actually configured;
- the target process start time relative to the scope change;
- the module's logcat lines, in order, showing injection and any hook installs;
- whether the target misbehaved, and the instrument-free control result for the same build.
## See also
- `references/dynamic-frida.md` — the exploration half of the workflow above
- `references/detection-and-anti-analysis.md` — when the target fights back and when to stop
- `references/native-and-so.md`, `references/native-dbi-and-deobfuscation.md` — the layer a Java hook stops at
- `references/dart-aot.md` — the layer that owns a Flutter app's logic
- `references/advanced-unpacking.md` — extracted dex and the VMP boundary
- `references/repack-and-sign.md`, `references/signature-derived-keys.md` — what you are avoiding, and why it is sometimes unavoidable
- `references/on-device-tooling.md` — MT Manager and the rest of the on-device kit
- `references/verification.md` — what "done" means, including the control-build rule
- `scripts/lsposed_scaffold.py` — generates the project skeleton described here

90
vendor-new/apk-reverse/skills/apk-reverse/references/membership-and-limits.md Normal file
View File

@@ -0,0 +1,90 @@
# Membership, paywalls, and the honest limits of client-side patching
Read this **before** investing hours. Most "unlock VIP / remove the paywall" requests cannot be satisfied by patching the client, and knowing that early is the most valuable output you can produce.
## First question: who decides?
| Signal | Owner | Client patch can... |
|---|---|---|
| A local boolean/int/string field in an app model, computed locally | Client | Fully determine the outcome |
| A field that comes from an API response and is merely *read* by the UI | Server | Change display only — not access |
| Download/stream/play URLs issued by an API | Server | Nothing. You cannot mint a valid URL. |
| Entitlement checked on every protected action with a server round-trip | Server | Nothing |
| A local cache of a server value | Server (cached) | Temporarily fool the UI until it refreshes |
**Decision rule:** if the thing you want (a media URL, a decrypted payload, premium data) is *delivered by the server*, the client is not the gate. The gate is server-side and is not patchable.
## How to prove it in minutes
Do not guess. Test the API directly (`references/server-api.md`).
```
1. Call the protected endpoint with no credentials. -> 401/403 means auth-gated
2. Call it with an obviously forged token. -> another 401 confirms it is validated
3. Call the metadata endpoint. -> often 200, giving you the shape,
but deliberately omitting the asset URL
4. Compare: is the valuable field simply absent? -> then no client change can invent it
```
A concrete, real result pattern worth remembering:
- `GET /videos/{id}` → **200**, full metadata, cast list, source list — **but no playback URL at all**
- `GET /v2/sections/{id}/play-url` → **401** with no token
- same, with `Authorization: Bearer <forged>` → **401**
- `GET /user/me` with forged token → **401**
That combination is conclusive: metadata is public, the asset URL is server-minted, and the token is verified. **Client-side patching cannot unlock it.**
## What *is* legitimately achievable client-side
Don't overclaim, and don't underclaim either. Genuinely client-side wins:
- **Remove ads / SDKs.** Ads are a client behavior (`references/ad-removal.md`).
- **Remove a UI gate** where the underlying data is already delivered and only a local check hides it (a "locked" tab whose content is already fetched, a disabled button).
- **Skip promotional interstitials and self-promo popups** driven by local flags.
- **Unlock client-only features**: debug menus, extra player settings, higher local bitrate selection, gesture options.
- **Grant a client-side reward** only if the reward is neither validated nor authoritative server-side (check first — a reward that later fails server validation is worse than no reward).
- **Bypass a client-side trial timer** whose enforcement is purely local (but expect the server to re-assert on next sync).
## What is NOT achievable, and how to say so
- **Paid content whose URL comes from the server.** State it plainly and show the 401 evidence.
- **Server-validated rewards / purchases / credits.**
- **Anything the server re-checks on the next request.** The patch may look like it works for one screen and then silently revert.
Wording that is both honest and useful:
> The client cannot unlock this. Metadata is public, but the asset URL is issued by the server, and the endpoint returns 401 for both anonymous and forged credentials — so no local change can produce a valid URL. What *is* removable client-side is X, Y, Z. To actually get access you need a real account with the required entitlement.
## The fake-VIP trap
Forcing a local `isVip()`-style method to return `true` is the classic mistake. Two failure modes:
1. **Cosmetic only** — UI shows VIP styling, but protected actions still fail server-side.
2. **Actively harmful** — the client now believes it holds entitlements it does not, so it enters code paths that expect server data it never receives. Observed result: **blank screen** (process alive, nothing rendered, only a swallowed uncaught exception in the crash buffer).
If you patch such a method, patch it in a **conservatively correct** way, and verify all screens, not just the one you cared about:
- Do not fabricate entitlements that gate data fetching.
- Prefer patching *presentation* consumers of the flag over the flag's computation, if the flag is also used for access decisions.
## Local state vs code — the deliverable question
A value can be "fixed" in two places:
| Where | Survives fresh install? | Fits an APK deliverable? |
|---|---|---|
| Code (patched dex) | Yes | Yes |
| App data (datastore/prefs/db) | **No** | No — unless you also patch the read path |
If a fix works on your device but a user reports it gone after reinstall, this is why (`references/pitfalls.md` P11). Re-implement the fix by patching the **read path** so it always yields the desired value.
## Reporting template
```
Goal: <what the user asked for>
Gate owner: client | server | mixed (evidence: ...)
Achievable: <list, with evidence each>
Not achievable: <list, with the 401/absent-field evidence>
Residual: <what remains and the exact coupling that prevents removing it>
Verified on: <device, build hash, what was exercised>
```

195
vendor-new/apk-reverse/skills/apk-reverse/references/native-and-so.md Normal file
View File

@@ -0,0 +1,195 @@
# Native & SO Layer
Load this when the Java/dex layer is blocked or unsuitable, when you need code to run **before** the app's
own code, or when a native library is the only editable place left.
If instead the problem is that the app **dies on its own** — hangs, restarts, or crashes natively at a
roughly constant time after launch — that is a terminate mechanism, not a hooking problem, and the
methods differ: read `native-tamper-and-suicide.md` before this file.
The Java layer is usually the right choice. Go native when you need one of these:
- execution **earlier than any app code** (before `Application.onCreate`, before static initialisers),
- a change that survives the app re-loading or re-initialising its Java state,
- a host the integrity checks do not cover while the dex is covered.
## Picking a host library
A library is usable only if all three hold. Verify each; do not assume.
1. **Loaded in your scenario** — confirm in `/proc/<pid>/maps`, not by inspecting the APK.
2. **Editable** — per the boundary map you built in `packers.md`.
3. **Has a call the runtime makes for you** — an exported `JNI_OnLoad` is ideal, because the runtime
invokes it automatically when the library is loaded by the app, with no trigger of your own.
Prefer a host whose exported JNI entry point exists *and* which appears in `maps`.
## The auto-load trap: `DT_NEEDED` does not call `JNI_OnLoad`
Adding a `DT_NEEDED` entry so one library drags in another is a natural idea. It maps the second library,
but **`JNI_OnLoad` is not invoked for it** — that callback only fires when the runtime loads a library
through the normal library-load API. A dependency loaded purely by the dynamic linker is mapped, not
initialised.
If you rely on `DT_NEEDED`, your entry point must be an **ELF constructor** or a `DT_INIT` entry, not
`JNI_OnLoad`. Conversely, if your host exports `JNI_OnLoad` and you control when the app loads it, that is
the simplest reliable trigger.
Related: a constructor/`DT_INIT` runs so early that the VM may not exist yet. Retry, and treat failure as
non-fatal — a library that logs nothing must never take the process down.
## Relocations and page permissions
This is where most hand-built native payloads fail, and the failure looks like a crash **inside the
dynamic linker**, not inside your code.
Rules that follow from how Android maps libraries:
- **A relocation target must be writable at load time.** The linker writes the computed address into the
target slot. If the slot lives in a section that is mapped read-only, the write faults
(`SEGV_ACCERR` in `plain_relocate_impl`).
- **Modern Android refuses W+X segments.** Adding a read/write/execute segment to get both properties is
rejected; the loader will not give you one.
- Together these mean: relative relocations cannot heal pointer slots that sit in an executable-only page.
### Bootstrapping when relocations cannot work
If your payload's internal pointers all live in the same page as the code that reads them, and that page
must be executable, you cannot use relocations. Do the fixup at runtime instead:
1. derive the runtime page base (a PC-relative instruction gives you this without any relocation),
2. make the page writable/executable via a direct `mprotect` **syscall** (do not depend on the libc
symbol being imported; a syscall needs no linkage),
3. write the pointer slots yourself, computed from the runtime base,
4. restore the page to read+execute,
5. then branch into your entry point.
Every pointer you need is `runtime_base + fixed_offset`. No relocation is added, so the segment
properties never have to change.
Do **not** try to make a page writable by re-mapping it over itself with an anonymous fixed mapping:
that severs the file mapping, and any other thread executing code on that page faults immediately. Use a
permission change, not a re-map.
## Replacing a Java method from native code
You can redirect a Java method's implementation without touching dex:
- Look up the method id, walk to the `ArtMethod` structure, and rewrite its **quick entry point**.
- Emit a short stub: load a target address from a literal pool and branch to it; return values follow the
normal calling convention (arguments are already in place, return in the usual register).
- **Always preserve the original code you overwrote.** If the method can be entered through its own
entry point, a naive redirect makes it jump into your stub, which jumps back into the same entry —
infinite recursion. Copy the overwritten instructions into a trampoline and branch back after them.
- **Do not patch a method whose entry still points at a shared interpreter bridge.** Methods that have
not been compiled by JIT/AOT share an entry; rewriting it breaks every method that shares it.
A practical guard: if two unrelated target methods report the identical entry address, both are still
on the bridge — refuse to patch rather than corrupting the process.
- Also refuse when the first instruction at the entry is PC-relative (address computed from the current
program counter): copying it elsewhere silently changes what it points to.
Make installation **fail-safe**: if any precondition is not met, skip the patch and let the app run
unmodified. A hook that silently does not apply is survivable; a hook that corrupts a shared entry is not.
## Finding call sites without symbols
Stripped and obfuscated libraries still expose structure:
- Scan the code section for the pattern that reaches your target (a PC-relative address computation
followed by an indirect branch through a table slot). A method reached through a **cached table slot**
will ignore a rewrite of its own entry point, because callers never read that field — another reason
the entry-point rewrite must be validated per call site.
- Dump the executable range and disassemble around candidate offsets instead of trusting symbol names.
- When a payload was produced by a just-in-time compiler on the target, the displacement between its
pages is often zero, which is exactly why relocation-free bootstrapping is required (above).
## Cross-architecture notes
Architecture is not background information here — it decides **which artifact you must edit** and
**whether your hook can observe anything at all**. Get it from the runtime, not from the manifest.
### What the device claims vs what is executing
Three different answers, and they disagree more often than people expect:
| Source | Answers | Command |
|---|---|---|
| The device | which ABIs the system supports | `getprop ro.product.cpu.abi` and `ro.product.cpu.abilist` |
| The package manager | **which ABI it chose for this app on this device** | `dumpsys package <pkg> \| grep primaryCpuAbi` |
| The live process | **what is actually mapped right now** | `scripts/lib_map.py --pkg <pkg>` |
Only the third is ground truth. The first two are predictions, and they are wrong exactly when it
matters: an emulator whose primary ABI is `x86_64` can be running an `arm64-v8a`-only app through a
translator, and a fat APK can have the package manager pick an ABI you did not assume.
**Consequence: if the library you patched does not appear in the live mapping, your change cannot
matter.** That is a plan problem, and no amount of re-patching will fix it.
### Translation layers change what you are observing
When an ARM-only app runs on an x86 host, a translator (Intel Houdini, `libndk_translation`, a
`native_bridge` in general) is executing the guest code. Detect it from the maps — look for
translator marker libraries and for app libraries whose architecture differs from the host's
primary ABI (`lib_map.py` reports both).
What it changes:
- **Timing.** Translation is slower and less predictable. Anything you measured about latency is
about the translator, not the code.
- **Native integrity and anti-tamper checks can behave differently** under translation, in either
direction — a check that fails on hardware may pass here, and vice versa. A pass under translation
is not evidence of a pass on hardware.
- **Hook behaviour.** Intercepting translated code is not the same as intercepting native code:
address spaces, calling conventions, and what the host sees at a syscall boundary all differ. If a
native hook reports nothing while the feature plainly runs, suspect translation before suspecting
your script.
- **Syscall-level observation shows the host's view.** A file or network call made from the guest
may look different on the host side than it would natively.
**Rule: verify the artifact on the ABI the user will actually run.** An emulator-only result is a
mid-task checkpoint, never the final claim (`verification.md`).
### Fat APKs: patching the ABI that never loads
An APK can ship `lib/arm64-v8a/`, `lib/armeabi-v7a/`, `lib/x86_64/`… The package manager extracts
**one** of them into the install-time native library directory, and only that one is loaded.
- Check `primaryCpuAbi` and the live maps before editing a `.so`. Editing `arm64-v8a` while the
device loads `armeabi-v7a` produces a build that is byte-different and behaviorally identical.
- If the deliverable must work for an unknown user, remember their device may select a different
ABI than yours. Either patch every ABI present, or state which ABI your artifact targets.
- If the app's own libraries are written at *runtime* rather than extracted from the APK, the live
paths will not be inside the APK at all (`lib_map.py` marks these `materialized`). Those belong to
whatever produced them, and a repacked APK will not carry them in that location.
### Payloads and tooling are per-ABI too
- A native payload, an injected library, or a shellcode blob built for one ABI **will not load** on
another. The failure usually surfaces as a *missing library* or a generic linker error rather than
a clean "wrong architecture" message — do not read it as "my payload is broken".
- **Pointer size differs** between 32- and 64-bit targets. A script or struct layout that assumes
8-byte words misbehaves silently on a 32-bit target.
- Keep build artifacts per-ABI and label them with the ABI in the filename. Mixing them up is a
reliable source of "it crashed for no reason" when the same APK behaves differently on an emulator
and on a device.
### Quick decision list
1. `scripts/lib_map.py --pkg <pkg> --app-only` → which app libraries are loaded, from where, and at
what architecture.
2. If a translator is present and the task needs reliable native behaviour, move to a matching
device.
3. If the library you meant to patch is absent, stop — pick the right library first.
4. Only then start editing.
## Verification
A native patch earns no credit until the behaviour changes **and** the app stays healthy:
- log a marker from your entry point so you know it ran at all;
- confirm the library is in `maps`;
- confirm the app reaches its normal UI afterwards;
- keep the original library so you can produce an unmodified control build.
Never conclude "the hook works" from the absence of a crash. Absence of effect is the normal outcome of a
hook that never installed.

318
vendor-new/apk-reverse/skills/apk-reverse/references/native-dbi-and-deobfuscation.md Normal file
View File

@@ -0,0 +1,318 @@
# Native DBI and deobfuscation
**Load this when static disassembly of a `.so` has stopped producing information** — an exported
function decompiles into a dispatcher loop and arithmetic soup, a symbol you need has no name, or you
must answer "what actually ran" instead of "what could run". This file covers instruction-level
dynamic tracing: what a trace can and cannot decide, how to record one with the scripts in
`scripts/`, and where it hands off to emulation and symbolic execution.
**Hand-off.** `native-and-so.md` owns the APK-side conclusions about a library — which `.so` is
loaded, which ABI is executing, whether a hook can observe anything, how to neutralise a terminate
path by returning. This file owns execution-level evidence. If the question is "did my patch cause
the death", that is `native-and-so.md` and `native-tamper-and-suicide.md`; if it is "which blocks did
this function really execute, and how often", it is here.
**Claim strength.** `measured` = an exact command and its output exist in
`references/evidence-summary.md` §The capability matrix. `inferred` = it follows from measured behaviour or
from documented behaviour of the tool. `unverified` = reported by someone else or reasoned from
first principles without a run. The distinction matters more here than usual, because the honest
result of this pass is that **the trace pipeline is unverified on the test device** — see.
## 1. Which obfuscation are you looking at
Obfuscator-LLVM ships three transforms, and one target usually carries all three. Identify by
**which one made static reading fail**, because each is undone by a different observation.
| Transform | Static shape | Dynamic signature | What removes it |
|---|---|---|---|
| **Control-flow flattening** | One dominator block at the top of the function, reached from everywhere; a state variable (register or stack slot) assigned a constant before each jump back; a `switch`/jump table dispatch; the original block order is gone | The dispatcher block dominates the execution histogram by a wide margin; real blocks appear between dispatcher entries; execution sequence looks like `D R1 D R2 D R3 D` | Trace histogram + the state variable's constants: each dispatcher entry is preceded by a store of the next real block's id |
| **Bogus control flow (opaque predicate)** | A branch whose condition is algebraically constant (`(x*x) >= 0`, `x*x % 2 == 0`), with a junk block on the impossible side; junk blocks reference unreachable names | The junk branch **never appears** in the trace at all: not once, at any input | Trace absence, and then dead-code deletion of the never-taken side. Absence in a trace is only evidence if the trace covers the input that would take it |
| **Instruction substitution** | Arithmetic that is correct but unrecognisable: add chains with negated constants, `xor`/`and`/`or` rewrites of a comparison, MBA identities | The function's **result** is still correct at its boundary, and the single-step value at a known point (e.g. the return register) matches the un-obfuscated expectation | Symbolic execution or a brute-force input sweep; a trace alone does not simplify arithmetic |
Worked example of the distinction: a flattened function with substituted instructions produces a
trace whose histogram has an obvious top, but the arithmetic between dispatcher entries still tells
you nothing. Flattening is a **control**-flow problem and the trace answers it; substitution is a
**data**-flow problem and needs.
**Reported, not verified here:** a community write-up of this exact pipeline — trace, aggregate,
solve with a symbolic executor, patch the binary — is
[基于动态指令追踪与符号执行对抗OLLVM控制流平坦化的工程化实践](https://cloud.tencent.com/developer/article/2721228)
(cloud.tencent.com/developer/article/2721228, published 2026-08-05). It reports, on a 2,800-block
x64 fixture: 15 s of tracing ≈ 220 MB of raw log, aggregation to 48 KB of block records, ~12 min of
parallel symbolic execution resolving 67 dispatcher targets, and 1.2 s of binary patching over 147
sites. **Those numbers are the author's, on a Windows x64 DLL, and were not reproduced here** —
treat them as an order-of-magnitude sketch for the data-reduction ratio (raw log to block table is
roughly 4000:1 there), not as a budget for an arm64 Android target. The article's code snippets are
illustrative rather than runnable: its `onReceive` handler indexes parsed events by field name
(`item.kind`, `item.start`), while `Stalker.parse(..., {stringify: false})` returns **positional
arrays**. Take the method from that article; take the API shape from this repository and from the
script header in `scripts/stalker_trace.js`.
## 2. Pick the observation before you pick the tool
| Question | Observation | Tool |
|---|---|---|
| Which blocks executed, and how often? | Block-level trace with execution counts | `scripts/stalker_trace.js` + `scripts/stalker_report.py` |
| Which functions inside the module call each other? | Call edges from the same trace | same pair (`call` events) |
| Which imported function does this stub reach? | Relocation table, not trace | `scripts/elf_plt.py` |
| Where did the process die, and did it look arranged? | Crash record, signal, fault address, backtrace split by owner | `scripts/native_crash.py` |
| What value must the state variable have to reach block X? | Constraint solving over the dispatcher | angr / Triton () |
| What is the value at a specific point for a specific input? | Deterministic per-instruction replay with memory operands | QBDI () |
| What does this code do with no device at all? | Emulation of the `.so` with a synthetic environment | unidbg / Unicorn / QEMU |
The first two rows are one run of one script — take both. `elf_plt.py` and `native_crash.py` are not
competitors here: they answer a symbol and a death respectively, and both work when no trace can be
taken at all.
## 3. Recording a trace with `scripts/stalker_trace.js`
Frida Stalker recompiles every basic block a followed thread executes and hands the events back to
JavaScript. It needs no symbols, no source and no disassembler on the host, which is exactly why it
works on stripped OLLVM output.
**Three rules, each one a failure mode if broken**
1. **Follow one trigger, not the whole process.** An unrestricted follow of a busy thread produces
gigabytes and slows the target down until it dies.
2. **Filter to one module.** `targetModule` is the only module whose blocks are reported; call edges
report when either end is inside it. Filtering happens on the device, not in post-processing,
which is what keeps the log usable.
3. **Cap the volume.** `maxBlocks` stops the trace instead of letting the log eat the disk, and the
`DONE` line records whether truncation happened.
**Configuration** (edit `CONFIG` in the script, or override at runtime through `rpc.exports.config`
/ `script.post({type:'cfg', payload:{...}})`):
| Key | Meaning |
|---|---|
| `targetModule` | the one `.so` whose blocks are reported |
| `trigger` | `{kind:'export', module, export}` — start on a named export (a JNI function is the classic); `{kind:'offset', module, offset}` — start at module-relative offset; `{kind:'java', cls, method}` — start at the Java→native boundary; `{kind:'main'}` — follow the thread right after load |
| `followMs` | hard stop, so a forgotten trace cannot kill the process |
| `maxBlocks` | event cap; the `DONE` line reports truncation |
| `events.compile` | first translation of each block — deduplicated block set in first-visit order |
| `events.block` | every executed block — the execution histogram |
| `events.call` | call edges |
| `events.exec` | **per instruction; enormous.** Enable only around a known point |
| `events.ret` | return edges |
**Customising what gets translated (`transform`).** Event flags shape *what you are told*; the
`transform` callback shapes *what runs*. It fires before a block is translated and receives the
instruction iterator, so it can rewrite the code Stalker is about to execute — inline a counter at one
call site, neutralise a logging call inside a hot loop, or substitute a value at the one address you
care about. Reach for it when the question is "is this site reached, and how often" rather than "which
blocks ran"; do not reach for it when the answer must be an observation, because a transform changes
the process's behaviour and stops being passive measurement. Two properties to respect: it can run
once per block per thread, so any rewrite must be idempotent, and the rewritten bytes are the DBI's
copy, so a crash inside them will not point at a file offset. `inferred`: `scripts/stalker_trace.js`
uses events and `onReceive` only; the transform route is documented here, not exercised by this
repository's pass.
**Running it.** With the bundled injector, which writes every event to a log file *and* stdout:
```
python scripts/run_probe.py scripts/stalker_trace.js 1 --pkg <pkg> --via usb --log trace.log
```
With any driver that speaks Frida's rpc, so no file edit is needed:
```js
script.exports.config({ targetModule: 'libapp.so',
trigger: { kind: 'offset', module: 'libapp.so', offset: 0x9f6c0 },
followMs: 3000, maxBlocks: 50000 });
script.exports.start(<tid>); // follow this thread
script.exports.status();
script.exports.stop();
```
Pass `start(tid)` explicitly rather than relying on the default. **`Process.getMainThreadId()` is
not the Android main thread in practice**: on the measured run below, a process with pid `19938` was
followed on tid `25606`, and an idle-looking thread produced zero events. On Android the main thread
tid equals the pid, which is the value to start from.
**Output grammar** (one logical line per event; `scripts/stalker_report.py` parses it, including
logs where each line is wrapped by `run_probe.py` as `[host ts] [dev ts] TRACE ...`):
```
READY ... config echo and module status; nothing works before it
MOD name base=0x.. size=.. path=..
BB <seq> <mod>+0x<offset> block first translated (dedup'd skeleton)
BLK <seq> <mod>+0x<offset> [size=n] block executed (histogram input)
CALL <depth> <from> -> <to> call edge
DONE reason=timeout|trigger-leave|maxBlocks|stop-rpc blocks=n blk=n calls=n truncated=0|1
FATAL / TRIG-FAIL what did not install and why; the script keeps running
```
**Reducing the log**
```
python scripts/stalker_report.py trace.log --top 30 --skeleton 80 --edges 15 --json report.json
```
Four reductions, each answering a different question: the **execution histogram** (top of the list =
dispatcher candidate), the **first-visit order** (deduplicated block set ≈ CFG skeleton with the
flattening state machine stripped out), the **call-edge table** (module-internal communication with
no symbols), and the **collapsed execution sequence** (`block*N -> block -> ...`, which is where the
`D -> R -> D -> R` flattening rhythm becomes visible).
Diagnostics the report prints, and what they mean:
- `DONE ... truncated=1` — the trace hit `maxBlocks`. Ratios remain useful; **absolute counts do
not**, and a claim about a dispatcher's multiplicity must be re-measured with a higher cap.
- `zero BLK with nonzero BB` — the thread was translated but executed nothing in-window: the window
closed, or the trigger returned immediately.
- `zero BB and zero BLK with DONE present` — the module never ran on the followed thread. Change the
thread or the trigger; do not tune the follow time.
- Everything zero **and no `DONE`** — the trace never started. Read the `READY` / `TRIG-FAIL` lines
first; that is a plumbing problem, not an analysis result.
## 4. Stalker, QBDI, or emulation
| Situation | Choice | Why |
|---|---|---|
| Need block-level coverage and execution counts on a real device, no per-instruction semantics | **Stalker** | Cheapest to start, no host toolchain, works on code with no symbols |
| Need per-instruction values, memory operands, or a deterministic replay that can be re-run at a recorded PC | **QBDI** (QuarkslaB Dynamic Binary Instrumentation), via its Frida binding | An independent DBI engine with its own register/memory API; the trace is a program, not a log. It instruments at a different layer than Stalker, which matters on ART: Stalker translates the code it sees, QBDI exposes the instruction stream and its operands directly |
| Need to hurt the target as little as possible while instrumenting a hot path | **QBDI** | Deterministic, no reliance on the app's own JIT/AOT output |
| Need a result with **no device and no app process** — a hardened sample refuses to run, or the algorithm must be called thousands of times | **Emulation** (unidbg, Unicorn, QEMU) | Runs the `.so` in a synthetic environment with JNI stubs; completely different trade-off (environment fidelity for scale) |
| Need "what input reaches block X" rather than "what happened for input I" | **Symbolic execution** () | A trace is one path; a solver enumerates the condition |
**Stalker and ART do not compose for free.** The followed thread's code can be recompiled by ART's
JIT while Stalker has already translated it, and a block's address is only meaningful inside one
process lifetime. Practical consequences: keys in a log are `module+offset` and stay comparable
across runs only for code that is mapped from file; do not compare a `libart`-materialized block
address between two runs; and never place a follow/unfollow pair on a hot function (measured failure
in).
### The cost is not linear, and on arm64 it decides the tool choice
Two costs decide whether Stalker is the right tool, and neither is visible in the API:
- **Translation multiplier.** A followed thread pays a large constant factor on every translated
block. Community reporting for arm64 puts it at **20-50x**; the mechanism is not controversial
(every block is copied into the DBI's code cache and routed through a dispatcher), while the exact
number is workload-dependent. This one is `inferred` — this repository has not measured the
multiplier itself, only its consequences below.
- **What you let into the trace.** Every block of every library the thread touches is a candidate.
Following into `libc` / `libart` / `libhwui` is how a four-second window becomes a device-wide
event: measured here, one such run took the test phone's `load average` to `34.11` on 8 cores and
destroyed a co-running job's attach ().
`Stalker.exclude()` is the control for the second cost, and it is not optional.
`scripts/stalker_trace.js` applies it before `follow()` from its `excludeModules` list and reports
what it actually excluded on one `EXCL` line — read that line before believing any trace, because a
module that was not loaded at follow time was **not** excluded. Exclude every system library you are
not studying; the target module is never excluded even if its name appears in the list.
Three failure shapes follow from ignoring this: deadlock, a watchdog `SIGABRT`, and the zero-event
trace. The deadlock and the watchdog shape remain community reports (`inferred`); the other two were
measured, **and the measurement splits the claim in two** — the split matters more than the summary:
| Arm — one device, one package, one module, 6 s follow | Outcome |
|---|---|
| attach + resume, **no follow** | process survives |
| follow, `excludeModules: []` | **process dies**, script destroyed |
| follow, 20 system modules excluded | process survives, **but `blocks=0 blk=0 calls=0`** |
Exclusion is what keeps the target alive — now `observed`, with the no-follow arm ruling out "it
would have died under frida anyway". But exclusion does **not** restore event delivery: the
zero-event trace survived the treatment arm unchanged. Treat those as two problems, and do not offer
`exclude` as the fix for a follow that delivers nothing. Commands and outputs:
`references/evidence-summary.md` §The capability matrix.
**arm64 also raises the floor.** PAC/BTI-bearing code gives a translator more ways to mis-handle a
block than armv7 did, which is one more reason a trace that works on an emulator is not evidence
about a device. Treat any arm64 result as needing a control run (a followed thread executing a known
loop, nonzero `BLK` lines) before it is interpreted.
**Emulation is a legitimate alternative, not a consolation prize.** When the target is a pure
computation inside a `.so` and its environment can be faked, a Unidbg/Unicorn trace can be both
faster and more stable than Stalker on a real arm64 device: no device load, no watchdog, no PAC, and
a deterministic replay. That inversion of the usual intuition is half the reason
`emulation-and-rpc.md` exists — decide with its decision table rather than by habit.
## 5. From a trace to a deobfuscated function (inferred)
This is the part the community write-up above also describes, and it is **inferred** here — the
trace end was not reproducible on the test device, so the steps below are a method, not a record.
1. **Find the dispatcher.** Rank by execution count. In a flattened function the dispatcher runs
once per real block; loops multiply it further. A block that is (a) at the top by a wide margin,
(b) entered from many distinct predecessors, and (c) followed by many distinct successors, is the
dispatcher. `stalker_report.py` prints exactly the three items needed to check that.
2. **Find the state variable.** Disassemble the dispatcher entry window statically (the offset is in
the trace, so no symbol is needed). Look for the register or stack slot written immediately
before the indirect branch; in flattened output this is the value the dispatch table is indexed
by. If more than one candidate exists, correlate: the state variable is the one whose written
constants match the distinct block ids seen in the trace.
3. **Recover the real CFG.** Convert the flat sequence into edges: `dispatcher → real block →
dispatcher`. The real blocks' order and repetition come from the trace; the state constants come
from step 2; the mapping `state value → real block` turns the dispatcher into a jump table you
can then remove in the binary (the LIEF step in the write-up).
4. **Solve what the trace did not cover.** A trace is one input's path. For a state value you never
observed, symbolically execute the dispatcher **only** (a few tens of bytes: load the state,
bounds-check it, index the table) rather than the whole function. This is where angr or Triton
earns its cost: the entry point is the dispatcher offset the trace gave you, and the goal is the
table index, not the program's semantics. Reported pitfall from the same write-up, worth repeating
because it is structural: OLLVM inserts transitions that land outside the table, so prune states
whose index exceeds the table bound instead of letting the solver chase them.
5. **Only then patch the binary.** Direct-branch rewriting must respect instruction length: an
absolute jump needs more bytes than the `mov; jmp` pair it replaces, so pad rather than move
anything. `scripts/elf_plt.py` is the tool that tells you what a rewritten site actually reaches
(by relocation, not by position), and a byte-diff of two builds is how you prove your patch set is
what you think it is.
## 6. Failure modes
**Zero events, follow installed (measured, cause unverified).** With host frida 16.7.19 and a
matching on-device frida-server 16.7.19 on Android 11 / arm64, `Stalker.follow` on a main thread for
4 s produced `blocks=0 blk=0 calls=0` on two different system processes, and an `export`-triggered
follow produced `reason=trigger-leave blocks=0` on every one of hundreds of firings. Every line
before that was healthy: `READY`, `TRIG following tid=…`, `MOD libc.so base=0x…`. The conclusion to
carry forward is narrow and should not be widened: **on that combination, follow installs but events
do not arrive in `onReceive`.** It is not a licence to assume Stalker is broken everywhere, and not
evidence about any other ROM, frida version or target. Validate the pipeline before trusting a zero
result: follow a thread you control executing a known loop, and confirm nonzero `BLK` lines.
**High-frequency follow/unfollow crashes the target (measured).** Using a hot libc export as the
trigger — every call starts a follow and every return ends it — a system UI process died with
`SIGSEGV` after hundreds of cycles; the crash record's frame `#00` sat in an anonymous region (the
DBI's code cache) and the return address led into `libart`. The process restarted under a new pid.
The lesson is not "Stalker is unsafe"; it is **do not use a hot function as a follow trigger**, and
do not build a follow/unfollow cycle per-call in a hot path. A single follow of a chosen thread for a
bounded window is the shape that works.
**The device becomes the experiment (measured).** While this pipeline was running, the test device's
`load average` reached `34.11` on an 8-core mid-range phone, and a co-running dynamic job on the same
phone (a dex-dumping attach belonging to another task) was destroyed mid-flight. On a shared device
these are not independent experiments: one heavy DBI run makes every other dynamic result
unattributable. Announce the window, keep `followMs` in the low seconds, cap `maxBlocks`, never spawn
when attach is enough, and treat a `script has been destroyed` in a *different* job as a signal that
your load — not that job's code — is the variable. This is `long-task-discipline.md` §single-variable
discipline applied to hardware.
**A trace of a VM is a trace of the interpreter.** If the target is a virtualized function (private
bytecode executed by a handler loop), Stalker will happily report the interpreter's blocks: a huge,
flat, repetitive histogram around one dispatch loop. That is a valid finding — it tells you the code
is virtualized — but it is **not** the guest program's control flow. Recovering that needs handler
identification and a private opcode table; see
`code-virtualization-and-custom-linkers.md`, and do not present an interpreter trace as the
deobfuscated function.
**Instrumentation was detected before you started.** If the process dies when you attach, refuses to
run under root, or the module you want is never loaded while attached, the problem is upstream of
this file: read the first section of `detection-and-anti-analysis.md` before escalating the trace,
because the cheap answer is usually to switch to static analysis with `elf_plt.py` and
`native_crash.py`, not to fight the detector.
## 7. Reporting what a trace proves
- A trace proves **what executed on the followed thread inside the followed window**. It does not
prove absence: `BB` never appearing means it did not run *for that input, in that window* — which
is meaningful for an opaque-predicate junk block only when the rest of the analysis says the input
would have taken it.
- Counts are evidence only when `truncated=0`. State the cap.
- A dispatcher identification is a hypothesis about a block's role. Confirm it by disassembling the
block and finding the table it indexes; a high count alone is also what a spin loop or a memory
allocator's fast path looks like.
- Write the product's before/after into the evidence record condensed in `references/evidence-summary.md` §Where the full record lives, with the exact command, the
module, the offsets, and the strength label. The next person's first question is "was the pipeline
known to work when you got that zero", and the answer belongs in that record, not in a footnote.

293
vendor-new/apk-reverse/skills/apk-reverse/references/native-tamper-and-suicide.md Normal file
View File

@@ -0,0 +1,293 @@
# Native Tamper Response — When the Library Kills Its Own Process
Load this when the process dies **without a Java stack trace**, when a build that passed every
static check crashes seconds after launch, or when you are about to neutralise a native check.
This file exists because of one specific, expensive mistake. Read §The rule before you edit a
single byte of a shell's native library.
## The rule
> **Never neutralise a terminate path by making it not return. Make it return.**
A terminate routine is reached from ordinary code paths. If you replace it with something that
**never returns** — an infinite loop, a self-branch, a stub that spins — the caller never resumes,
locks are never released, and every unrelated thread that touches those locks wedges. You have not
suppressed the check; you have frozen the process.
The distinction decides the outcome:
| Neutralisation | Effect on the caller | Result |
|---|---|---|
| `ret` / return a benign value | continues normally | check suppressed, process healthy |
| spin / self-branch / `while(1)` | never resumes | process freezes, watchdog or user kills it, symptom looks unrelated |
**Two failure shapes this produces, both of which have been misread as "the patch did not work":**
- The app hangs with no crash record at all, then disappears. Logs show nothing, because nothing
crashed — it stopped.
- The watchdog or a system supervisor kills the frozen process, producing
`Force stopping … from uid 0` or an app-restart loop. The uid-0 killer is a strong tell: an app
cannot spawn a root-owned killer, so the executioner is **outside** the app, which means the app
froze rather than failed.
Both are self-inflicted and both are avoided by returning instead of spinning.
### The corollary: do not touch normal-path symbols
A hardening library imports a lot of libc. Some of those symbols are terminate paths; most are not.
**Symbols that must stay untouched** — they are on ordinary code paths and freezing them breaks
everything:
`pthread_exit` · `exit` · `abort` · `android_set_abort_message` · `snprintf` · `closedir` ·
`__cxa_atexit` · `__stack_chk_fail`
A concrete case: a set of hand-made patches rewired five PLT stubs to self-branches on one
architecture and a different five on another. On the first architecture two of the five were
`snprintf` and `closedir` — a string formatter and a directory-close — so the app froze on the
first log line. On the second the five included `pthread_exit`, so every thread that finished
wedged. The patches were meant to suppress a suicide check and instead destroyed the process.
**Before touching a stub, resolve what symbol it belongs to.** Do not infer it from the stub's
position or from a comment. See §Resolving a stub to its symbol.
## How a hardened library terminates the process
Enumerate these before you patch anything. Each has a different signature and a different remedy.
| Mechanism | Log signature | Has tombstone? | How to neutralise |
|---|---|---|---|
| `kill(getpid(), SIGKILL)` | `Process N exited due to signal 9 (Killed)`, **no** exit code, **no** tombstone | no | patch the call site or make the `kill` stub return 0 |
| `exit()` / `_exit()` | process ends with an exit code, no signal | no | find the caller; usually one branch of a check |
| `abort()` | `signal 6 (SIGABRT)` + tombstone | yes | usually a genuine assertion — find what tripped it |
| **Deliberate crash** (see §Deliberate-crash stubs) | `signal 11 (SIGSEGV)`, tiny `fault addr` such as `0x4`, `Cause: null pointer dereference` | yes | nop the faulting store |
| `tgkill` / `raise` | same as kill/abort | varies | often not imported at all — check the import table before assuming |
**Distinguish these before patching.** "The app crashed" is not a diagnosis. The signal number and
the presence or absence of a tombstone split the space in half; getting this wrong sends you to the
wrong layer for hours.
### Deliberate-crash stubs
The most deceptive mechanism: the library does not call anything. It **arranges a fault** so the
death looks like an ordinary bug.
```asm
; arm64 shape observed in the wild
mov x0, #4 ; load a small constant…
mov w1, #1
str w1, [x0] ; …and use it as a pointer. fault addr = 0x4
```
Runtime shape: `SIGSEGV`, `SEGV_MAPERR`, `fault addr 0x4` (or `0x0`/`0x8`), `Cause: null pointer
dereference`, and — critically — **the register holding that small constant value**.
How to recognise it rather than treating it as a real bug:
1. The faulting address is a **small integer**, not a plausible pointer.
2. Disassembling backwards from the faulting `pc` shows the constant loaded **a few instructions
earlier, in the same basic block**, with nothing in between that could produce a real pointer.
3. The block sits immediately before a function epilogue (canary check + `ret`), i.e. the normal
exit is right there and this store is bolted on in front of it.
4. There is often a **delay loop just above it** — `sleep`/`usleep` repeated N times — because this
is a *watchdog*: wait a while, then die if the condition still holds. That delay is why the crash
is always "a little while after launch" rather than immediate.
**Neutralise it by nop-ing the faulting store**, leaving the surrounding arithmetic alone. The
thread then falls through into the epilogue and returns normally. Do not remove the whole block:
the two `mov`s are harmless and keeping them makes the diff minimal and reviewable.
**This is why patching `kill` alone is not enough.** A deliberate crash uses no imported symbol at
all, so every PLT-level fix — including a correctly implemented one — leaves it untouched.
### Find every instance, not the first one
Search the binary for the **byte pattern**, not with a linear disassembler. The exact encoding is
architecture-specific and must be derived from the crash you already have, then searched as bytes:
- Take the faulting `pc`, read the instruction bytes backwards until you have the full constant-then-
store sequence.
- Search the whole file for that byte sequence.
- If the search returns more than one hit, patch each; if it returns exactly one, you have the
complete set — record that fact, it is a real result.
A linear disassembler is the wrong tool here for a reason covered in §Scanner traps.
## Resolving a stub to its symbol
When a stub's target is a PLT entry, the symbol comes from the relocation table, **not** from the
stub's bytes or its neighbours. Two architectures encode this differently:
- **x86_64** — the stub is `jmp qword ptr [rip+disp32]` (6 bytes). The target address is
`stub_addr + 6 + disp32`; look that address up in the PLT relocation map.
- **aarch64** — the stub is **four instructions / 16 bytes** (`adrp` → `ldr` → `add` → `br`). The
GOT address is `page(adrp) + add_imm`. Do not assume a 4-byte stub: a 16-byte stub leaves room
for a **two-instruction replacement** (`mov x0, #0` + `ret`), which is exactly what you need, and
assuming 4 bytes pushes you into borrowing the neighbouring slot — which belongs to a different
symbol and breaks it.
`scripts/elf_plt.py` does both architectures, including the relocation lookup.
**Never splice in a neighbouring stub's bytes.** Stubs are laid out back to back; the next one
along is a different function, and corrupting it produces a second, unrelated failure that you will
then spend hours attributing.
## Patching a terminate path correctly
Work in this order.
1. **Confirm the mechanism first.** Get the tombstone or the signal number. Do not patch on a
guess about *how* it dies.
2. **Decide between call-site and stub.** Call sites are precise but you must find all of them.
A stub covers every caller at once, including ones your scan missed — but it changes behaviour
for every symbol user, so it is only safe when that symbol means "terminate" and nothing else.
3. **For a stub, return success, not failure.** Callers commonly inspect the return value. Returning
**0** (success) lets the caller take its "already handled" branch; returning **-1** pushes it
into an error path that may try a different termination method. Make the stub *succeed*.
4. **Preserve every byte you did not intend to change.** Record the original bytes so the patch is
reversible and auditable.
5. **Re-verify the file after patching.** Length unchanged, only the intended offsets differ, and
the normal-path symbols are byte-identical to the original. `scripts/elf_plt.py --diff` prints
exactly that.
### What "faithful suppression" looks like
A library that decides "tampered" and then cannot terminate has an inconsistent state — it believes
it has killed the process and continues. That is **acceptable and usually harmless**, because the
terminate path is the end of the check; nothing downstream re-reads "did I die". But verify it
rather than assuming: after a successful suppression the app must reach its **normal UI and stay
there**, not loop through half-initialised states.
## ELF hardening you will meet in these targets
Hardening libraries are not normal ELF files. Assume the following until measured otherwise.
### Forged section headers
A shell may ship a library whose section header table is deliberately wrong — `.text` sized to a
handful of bytes, `.dynsym` truncated, sections overlapping. Tools that walk sections therefore
produce **confidently wrong** answers (a symbol list with one entry, an import table that is empty).
**Work from the program headers instead.** `PT_LOAD` gives you the real mapped segments and their
permissions; `PT_DYNAMIC` gives you the pointers you need, walked by hand:
```
DT_STRTAB(5) DT_SYMTAB(6) DT_STRSZ(10) DT_SYMENT(11)
DT_JMPREL(23) DT_PLTRELSZ(2) DT_PLTREL(20) DT_PLTGOT(3)
```
Then translate any virtual address to a file offset through the `PT_LOAD` that contains it. A
library with forged sections that yields a full import list this way is a good sign; one that
still yields almost nothing means the hardening is deeper and you should say so rather than
reporting the truncated result as the truth.
### Function boundaries come from `PT_GNU_EH_FRAME`, not from prologue guessing
You will need "where does this function start". Two bad answers and one good one:
- **Reverse-decoding backwards from a call site does not work on aarch64.** Almost any 4-byte
window decodes as *some* instruction, so a naive scanner reports hundreds of "function starts"
that are all fiction.
- **Guessing at prologues** (`sub sp, sp, #imm` + `stp`) produces a plausible set with no guarantee
of completeness.
- **`PT_GNU_EH_FRAME` is authoritative.** If the segment survives (it usually does, because
unwinding is needed at runtime), parsing it yields the real function entry table — every entry,
exactly. It is several lines of code and it removes an entire class of wrong conclusions.
**Check for it first.** If it is present, use it; if it is absent, say your boundary list is
heuristic.
### A function's entry is not where its logic is
Callers reach a routine through several routes: a direct branch, a cached table slot, or a function
pointer in writable data. Rewriting the entry point only covers the first. If a call resolves
through a cached pointer, your entry-point change is invisible to it — so before patching an entry,
scan for the other routes (`scripts/elf_plt.py`, or a byte-pattern scan for the address).
## Scanner traps
**A linear disassembler can stop silently.** Given a buffer whose start is not code, a decoder may
return a handful of instructions and then nothing, with no error. Concluding "there are no such
call sites in this library" from that output is a false negative with real consequences.
Guards, in order of preference:
1. **Byte-pattern search** for a sequence you already know (from a crash or a known call site).
Fast, exhaustive, no decoder involved.
2. **Fixed-bit-pattern scans** for instruction classes whose encoding is regular — the branch
instructions are the useful ones here (`BL`/`B` on aarch64, `call rel32` on x86_64). This finds
every call site without depending on linear decoding.
3. **Resynchronising scan** — decode, and on failure advance by one instruction unit (4 bytes on
fixed-width aarch64, 1 byte on x86_64) and retry. Slower, but it does not stop early.
Whichever you use, **state which one you used and what it can miss**. "No hits" from a scan whose
coverage you cannot describe is not evidence.
## Deciding whether a dynamic-resolution path matters
Hardening libraries frequently build a table of function pointers with `dlopen` + `dlsym` — often
40+ symbols including terminate calls. This raises an obvious worry: *a call through that table
bypasses the PLT stub, so my stub patch does nothing.*
**Measure it instead of assuming either way.** For each symbol of interest, find whether its table
slot is ever **read**:
- Locate the slot's address (the store from the `dlsym` return, or the table base + index).
- Search the code for instructions that load from that address (`adrp`+`add`+`ldr` on aarch64).
- **Zero readers means the entry is written and never used.** The table is a redundant fast path,
not a bypass, and PLT-level patching is complete.
Real outcome from a hardened target: the `kill` slot had **zero** readers while the `exit` slot had
exactly one, inside a routine that was already suppressed — so no dynamic-resolution bypass existed
at all, and the expensive "patch `dlsym` itself" plan would have broken 40 unrelated symbols for
nothing.
**Do not blanket-patch a symbol resolver.** It feeds `open`, `read`, `mmap`, `dlopen` and more.
Changing its behaviour to reach one terminate symbol is a large blast radius for a problem you may
not have.
## What these checks actually detect — and why your environment changes the answer
A terminate path is not reading your patch. It is reading **the world around the process**, so what
trips it is environmental, and the same build can die in one setup and live in another.
| Signal | Typical probe |
|---|---|
| a debugger is attached | `ptrace`, `TracerPid` in `/proc/self/status`, `prctl` |
| the parent process was replaced | a cached `getppid()` compared against a fresh one |
| the process was suspended | wait status showing `SIGSTOP` / `SIGTRAP` |
| instrumentation is present | hook-framework artefacts: listening ports, thread names, unusual mappings |
| a "non-standard" environment | root binaries, emulator fingerprints, writable system paths, build props |
Two consequences decide whether your experiments mean anything at all:
1. **A pass under translation is not a pass on hardware.** Under an ARM-on-x86 translator the
environment probes return different answers — in *both* directions. A check that fires natively
may stay quiet here, and one that is quiet natively may fire. Emulator or translated results are
a mid-task checkpoint, never the final verdict.
2. **Your own tooling can be the thing that trips it.** If the app only dies while you are attached,
you are looking at a probe aimed at *you*, not at the app's ordinary startup path. Reproduce with
nothing attached before you patch anything.
**Run the unmodified original through the identical conditions first.** If it dies the same way, the
check is firing on the environment rather than on your change, and every conclusion drawn from the
patched build's death is void. This is the same control-build rule as everywhere else in this skill,
and it is the single cheapest way to avoid chasing a detection that is not about you.
## Verification
A native suppression earns no credit until:
1. The process **survives the window in which it used to die** — and that window must be measured,
not guessed. If the observed death was ~40 s after launch, a 30-second test proves nothing.
2. It then **keeps surviving** through real interaction, not just idling.
3. `crash`/`tombstone` buffers are clean, **and** the process id never changed (a restart with a new
pid is a death even when nothing logged a crash).
4. The normal-path symbols are byte-identical to the original build.
5. A **control build** — same pipeline, no native patch — still dies the same way. Without the
control you cannot tell suppression from an unrelated change in timing.
**Record the observed time-to-death before patching.** It is the only thing that later tells you
whether your patch worked or merely moved the window.

145
vendor-new/apk-reverse/skills/apk-reverse/references/on-device-tooling.md Normal file
View File

@@ -0,0 +1,145 @@
# On-device tooling — MT Manager, APK MCP, LSPosed, Termux
Sometimes the PC is the wrong place to work: the device already has the APK installed, the edit is
one string, the target is a 38 MB repack you would otherwise push back and forth, or there simply
is no laptop in the loop. This file covers the **on-device toolchain** and — more importantly —
when each tool beats its PC counterpart and when it does not.
Evidence basis: the facts below come from a real configured environment (Android 11, Magisk +
Zygisk, LSPosed v1.9.2, MT Manager 2.26.9). Framework
activation and the module set are `observed` there; per-workflow usability notes are `measured`
where marked, otherwise `inferred`. The MT MCP tool surface is from MT's official documentation
(`observed` as a list; live invocation requires the service to be running — see).
**Load this when:** the work is better done on the phone than on the PC -- one-string edits, an already-installed target, or no laptop in the loop. It gives MT Manager edit/repack/sign and its APK MCP surface, LSPosed Manager, and Termux.
## 1. MT Manager as a reverse-engineering workbench
MT Manager (`bin.mt.plus`) is a file manager plus APK editor. For this skill's purposes it is a
**complete repack pipeline that runs on the phone**: view, edit, rebuild, sign.
What it covers, mapped to this skill's workflow stages:
| Skill stage | MT capability | Notes |
|---|---|---|
| Recon (manifest, SDKs) | APK viewer: manifest, entry list, string search across dex/resources | String search is the fast path; no full Java decompile in the editor workflow |
| Dex reading/editing | Dex viewer + smali-level editing | Smali is the editing granularity — same discipline as `dex-patching.md` |
| Removing signature verification | Built-in "去除签名校验" (signature-check killer) feature | Convenience feature; audit what it actually patched — a Java-layer signature killer can be a decoy while a native check still kills you (`code-virtualization-and-custom-linkers.md` §decoy) |
| Extracting artifacts | Pull `classes.dex`, `.so`, `resources.arsc`, axml out of any APK/zip | Useful to move a single `.so` to the PC toolchain without a full unzip |
| Repack + sign | Rebuild APK and sign it on device | Output lands in MT's working directory; verify with the same discipline as `repack-and-sign.md` (alignment, `resources.arsc` rules still apply — the platform enforces them regardless of which machine built the file) |
**When MT beats the PC pipeline:** quick single-file edits, on-the-spot verification on the same
device the app is installed on, and sessions where shuffling 40 MB APKs over USB dominates the
work. **When the PC wins:** anything scriptable/repeatable, dex-wide cross-referencing
(`find_refs.py`), byte-precision patching with header recomputation, and anything needing a real
decompiler. The honest split: MT for **breadth on device**, PC for **depth and repeatability**.
`inferred`.
## 2. MT's built-in APK MCP server
MT 2.26.9+ ships an "APK MCP" service: it exposes MT's analysis and modification capabilities to
an MCP (Model Context Protocol) client — which in practice means an AI agent drives MT's tooling
over HTTP while the human sees the results on the phone. The official framing: AI understands the
request and calls the tools; MT does the local read/modify/repack/sign.
**Transport and endpoint** (`measured` on the reference environment):
- Streamable HTTP MCP at `http://127.0.0.1:8787/mcp` from the PC, via `adb forward tcp:8787
tcp:8787`; or directly `http://<phone-ip>:8787/mcp` on the LAN.
- **The service must be started by hand in the MT UI** (side drawer → Tools → APK MCP → Start).
adb cannot start it. Before it is started, the port is simply not listening — probe with
`scripts/mt_mcp_probe.py`, which prints waiting instructions in that state.
**Tool surface** (official documentation; names observed from the published list):
| Category | Tools |
|---|---|
| Open/selection | `mt_apk_list_available_apks`, `mt_apk_open` |
| Reading/search | `mt_apk_list`, `mt_apk_search`, `mt_apk_read_text`, `mt_apk_read_bytes`, `mt_apk_continue` |
| Dex analysis | `mt_apk_dex_outline_class`, `mt_apk_dex_xref` |
| Resources | `mt_apk_resource_read`, `mt_apk_resource_xref` |
| Native static | `mt_apk_native_inspect`, `mt_apk_native_read_items`, `mt_apk_native_map_address`, `mt_apk_native_xref`, `mt_apk_native_disassemble`, `mt_apk_native_function_cfg` |
| Modification | `mt_apk_edit_open`, `mt_apk_edit_text`, `mt_apk_edit_resource`, `mt_apk_patch_bytes`, `mt_apk_native_patch_instructions`, `mt_apk_native_patch_string`, `mt_apk_edit_check`, `mt_apk_build` |
| Cleanup | `mt_apk_close` |
**Official constraints — budget your plans around these, not around hopes:**
1. **No Java decompilation.** The stated position: AI reads smali directly; decompiled Java of
hardened/R8'd code misleads more than it helps. Cross-references (`dex_xref`) and class
outlines are the structural view.
2. **`resources.arsc`: existing entries only.** No new locales or new entries — edit what ships,
do not plan feature-additions through arsc.
3. **`.so` support is static analysis only.** No dynamic execution, no emulation, no C/C++
pseudocode. Xrefs and CFG are "best-effort within a limited budget" — direct static results,
**not** proof of a complete call chain or of runtime behaviour. For depth, the PC side (IDA/
Ghidra-class, see `toolchain.md`) remains the answer.
**Handing an APK to the server** (three supported ways): open an APK in MT's file list and stay
on its info dialog ("the current APK file"); open an installed app's info dialog the same way; or
drop the APK into MT's configured MCP working directory and refer to it by filename/package. The
working directory is also where rebuild output lands.
**Probing from the PC:** `python scripts/mt_mcp_probe.py` does the full JSON-RPC handshake
(initialize → initialized → tools/list) with no MCP SDK dependency and prints a grouped tool
inventory. If MT's MCP has not been started in the UI, it exits with waiting instructions and
code 2 — usable as a loop check ("is it up yet") while you start it by hand. `measured`.
## 3. LSPosed Manager — the module route on device
LSPosed (via Zygisk) gives you a **Java-layer hook platform that needs no PC at runtime**: write/
install a module once, and it applies to target apps at every launch.
Facts from the reference environment (`observed`):
- Activation is verifiable from logs: the `ZygiskCompanion: welcome to LSPosed!` banner, the
version line, the `lspd` daemon process running as system.
- Install discipline: `magisk --install-module` unpacks to `/data/adb/modules_update/` and only
**merges after a reboot** — a module that "did not install" is usually one that has not merged
yet. `ls /data/adb/modules/` is the truth; `modules_update/` non-empty means pending.
- Zygisk itself needs one reboot after being enabled before its injection layer exists at all.
Limits that matter for this skill: LSPosed hooks **Java/ART** — it cannot hook inside a native
`.so` (that is Frida/IDA territory, and the reason `framework-runtimes.md`'s runtime check
exists), and it does nothing for Flutter/Dart AOT logic below the platform channel. Use it for
the layer it owns: Java API interception, intent/telemetry rewriting, feature gates that live in
Java. A hook module is also **not a deliverable** for a request that wants an installable APK —
it is an environment change on your device (`detection-and-anti-analysis.md` Step 4 keeps these
separate).
## 4. Termux + frida-server — device-side dynamic analysis
The phone can run the whole dynamic stack alone: Termux (or a plain root shell) hosts
`frida-server`, and scripts run through Termux's own Python.
- **frida-server version must match the PC-side frida tooling** when the PC is also in the loop
(`dynamic-frida.md` owns this). Device-only operation still needs a matching `frida` client in
Termux.
- **Disguise the server when the target hunts it**: a renamed binary and a non-default port are
the minimum; a detection-savvy target scans process names and default ports. The reference
environment carried renamed server copies from prior work — the technique is standard, and the
detection mechanics are `kernel-and-environment-hardening.md`.
- **32/64-bit**: a `zygote64_32` device runs both ABIs; make sure the frida-server architecture
matches the process you are attaching to (`environment.md` §which architecture is actually
executing).
- Keep sessions simple: attach-mode on a hardened target inherits every §1 problem of
`kernel-and-environment-hardening.md`; prefer spawn, and prefer `spawn_patch_detach.py`-style
patch-then-detach when you only need a memory fix to stick.
## 5. On-device forensics — where the data is
Rooted-phone data inspection does not need a PC round-trip. The full layout, formats and
ownership rules live in `runtime-data.md` (DataStore/MMKV/SQLite paths, `chown`/`restorecon`
after restores, the "app rewrites your edit" decision tree) — on device, MT Manager's file
browser with root covers the same reads, and Termux covers scripted pulls. The two on-device
cautions that most often bite: edit files **only with the app force-stopped** (the in-memory
copy wins otherwise), and re-check ownership after any reinstall (the uid increments).
## Checklist
- [ ] Chose the workbench deliberately: MT (on-device, fast) vs PC (scriptable, deep) — not by habit
- [ ] MCP started in MT's UI and probed (`mt_mcp_probe.py` exits 0) before planning agent-driven edits
- [ ] MCP plans respect the three official constraints (no Java decompile, arsc entries-only, `.so` static-only)
- [ ] LSPosed modules verified merged (`/data/adb/modules/`, not `modules_update/`) before testing
- [ ] frida-server architecture matches the target process; disguised if the target scans
- [ ] Data edits done with the app stopped; ownership fixed after reinstalls

155
vendor-new/apk-reverse/skills/apk-reverse/references/packers.md Normal file
View File

@@ -0,0 +1,155 @@
# Hardened / Packed Targets
Load this when recon says the app's `Application` class is not the app's own, or when edits make the
app die before your code ever runs. Packing is the single most common reason a correct patch appears to
"do nothing", so treat it as its own phase with its own evidence standard.
**Not packed, and still dying?** That is a different layer and a different file:
`code-virtualization-and-custom-linkers.md`. There `Application` is the app's own, the dex is fully
readable, and yet whole classes are `native` declarations and a private loader carries an embedded
validation payload. The single most expensive recon error in this domain is reading "no packer" as
"the code is editable" — check for that shape explicitly before planning an edit.
## What a packer actually does
A shell (jiagu / weapon / leggu / 360 / ijiami / bangcle class) replaces the manifest's
`application android:name` with its own stub. At process start the stub:
1. loads its own native library from a directory it controls (often extracted at first launch),
2. decrypts and loads the real dex, usually from `assets/` or a same-named container,
3. optionally verifies that nothing about the APK, its own components, or the environment changed,
4. only then forwards to the app's real `Application`.
Consequences that shape every later decision:
- **Your code in the app's own dex may never run** if the shell aborts first.
- **The shell's native library is the gatekeeper**, and it is usually the only thing that decides
"tampered vs clean".
- **The shell's own entry is native**, so Java-level reflection into it is usually a dead end.
## Reading the rejection signal
The most informative failure looks like this:
```
UnsatisfiedLinkError: JNI_ERR returned from JNI_OnLoad in "<app>/.jiagu/lib<shell>.so"
```
Read it precisely: the shell's **own** `JNI_OnLoad` returned `JNI_ERR`. That means the shell
**decided** to refuse — it is not a crash, not a missing library, not an ABI problem. Distinguish it from
the look-alikes:
| Message | Meaning |
|---|---|
| `JNI_ERR returned from JNI_OnLoad in <shell>.so` | shell actively refused (integrity or environment) |
| `dlopen failed: library "X" not found` | a dependency you added does not exist — **your** bug |
| `CANNOT LINK EXECUTABLE` / relocation errors | ELF you produced is malformed |
| `ClassNotFoundException` during `Application` init | a manifest component you renamed no longer resolves |
This distinction matters more than it looks: several classes of "the shell rejected me" are actually
self-inflicted. Always grep the crash for `linker`, `CANNOT LINK`, `dlopen` and `not found` **before**
concluding that the shell detected anything.
## Map the validation boundary with single-variable tests
Do not guess what the shell checks. Measure it. The only reliable method is one change at a time, each
with its own install-and-launch run:
1. **Baseline control** — repack with no changes at all, re-sign, install, launch. If this fails, stop:
your pipeline is the problem, not the shell.
2. **Existing file, semantically inert change** — flip one byte in a section that nothing reads
(a `.comment` string, a duplicated `DT_NEEDED` entry). Survives ⇒ edits to existing files of that
class are allowed.
3. **New file added** — add one benign ELF under the native library directory. Very commonly this alone
is enough to trip a check, which is a distinct policy from (2).
4. **Component rename in the manifest** — equal-length rename of one lazy component
(an `activity`). Survives ⇒ you have a cheap way to neutralise features.
5. **Structural change** — move a program header table, add a segment, add a relocation.
Record every result. The boundary is usually **not uniform**: typical real outcomes are
"existing native libraries editable, new ones not", "activities renamable, providers not",
"this dex fingerprinted, that container ignored".
### Why one variable at a time is not optional
The most expensive mistake in this phase is testing two hypotheses in one build. If you both edit a
shell component **and** add a library, and the app dies, you have learned nothing — and you will likely
blame the wrong one, discard a viable route, and lose hours (or in this skill's history, dozens of
rounds) re-deriving what a cleaner experiment would have shown in one run.
### Component-type rules worth knowing up front
- **Providers are instantiated by the system during process start.** Renaming one so its class no
longer resolves fails inside `Application.onCreate` and kills the process before anything else runs.
Providers are therefore a poor rename target; prefer disabling the feature that consumes them, or
hooking it.
- **Activities and services are resolved lazily**, only when actually used. Renaming them is a cheap,
low-risk way to make a feature unable to display, and it survives most integrity checks because it
changes nothing about the shell.
- **Equal-length renames** keep binary XML chunk structure valid, so no manifest re-encoding is needed:
replace the name bytes in `AndroidManifest.xml` in place, size unchanged.
## Native libraries as a place to live
When the shell blocks dex-level changes, native libraries often remain open — but only some of them are
useful. A library is a valid host for your code **only if all three hold**:
1. **It is actually loaded** in the scenario you care about.
2. **It is editable** (per your boundary map above).
3. **Something in it is called early enough** to matter.
Condition 1 is the one people skip, and it is the one that wastes the most time. A library shipped in
the APK is not necessarily loaded. **Check `/proc/<pid>/maps`** after a real launch rather than assuming:
```
grep -o '/[^ ]*\.so' /proc/<pid>/maps | sort -u
```
Libraries belonging to optional SDKs are frequently lazy: they exist, they may even be unpacked to disk,
and they are never mapped during the startup path you are trying to influence. Unpacked-but-unmapped is
the signature of a lazy SDK, not evidence that your file was rejected.
Pick a host that is (a) present in `maps`, (b) exports a JNI entry point the runtime must call, and
(c) editable. A library that exports `JNI_OnLoad` and appears in `maps` is the strongest candidate,
because the runtime calls `JNI_OnLoad` on its own, without any trigger from you.
## Where integrity evidence actually comes from
Shells commonly check a mix of: APK signature, `AndroidManifest.xml`, their own native libraries, the
dex they load, and environment signals (root, emulator, hooking frameworks, debugging).
Practical consequences:
- **Your own re-signing is itself a detected change** in most signature-based schemes. Assume any
signature check will see your build as tampered and plan around it rather than hoping otherwise.
- **The environment matters.** Hooking frameworks, debugging, and non-standard filesystems are commonly
probed. A patch that works on a clean device may be refused on a rooted one, and vice versa.
- **Absence of a log line is not absence of a check.** A shell that fails silently looks identical to a
shell that never checked.
## Locating what the dialog/detection is attached to
Blocking dialogs ("tampered", "third-party environment", "abnormal") are the usual user-visible symptom.
Establish **which layer draws it** before trying to suppress it:
- If it is a Java dialog, hooking the dialog class shows a caller stack that names the detection.
- If hooking the dialog classes produces **no** hits, the UI is likely drawn by a cross-platform runtime
(see `framework-runtimes.md`) or by native code, and Java-level hunting is the wrong axis entirely.
The cheapest high-value experiment, and one that is routinely skipped: run the **unmodified official
build** and the **repackaged build** through the *same* probe and diff what differs. The difference is
your detection signal. Doing this early is far cheaper than enumerating candidate checks one by one.
**When the symptom is a death rather than a dialog** — the process disappears, hangs, or dies at a
roughly constant time after launch — the check is not drawing anything, it is terminating. The
question then becomes *which mechanism*, and the answer decides the whole approach:
`native-tamper-and-suicide.md` covers how to tell an imported terminate call from an **arranged
fault** (which calls nothing and therefore defeats every PLT-level fix), how to find the site, and
why making a routine "not return" freezes the process instead of suppressing the check.
## Reporting
State the boundary you measured, the evidence for each entry, and what remains untested. "The shell
rejects X" must always carry the run that proves it, including the exact failure string. An unverified
claim about integrity behaviour is worse than no claim, because it silently removes routes from
consideration.

154
vendor-new/apk-reverse/skills/apk-reverse/references/patch-audit.md Normal file
View File

@@ -0,0 +1,154 @@
# Auditing your own patch — proving it landed, and proving it is legal
Two independent questions, and they fail independently:
1. **Did the edit land?** Did the bytes you intended actually reach the artifact?
2. **Is the result legal?** Will the runtime's verifier accept it?
A patch can land and be illegal (the class fails to load). It can also fail to land while every
build step reports success. Both are silent, and each needs its own check.
**Load this when:** you must prove the edit landed **and** that it is legal. It gives the length-versus-bytes comparison that catches equal-length blind spots, verifier-level legality, and how to report a patch that did not apply.
## 1. Did it land? Compare *length*, not bytes
The naive check is a byte-level diff of the target method against the original.
**It produces a flood of false positives.** A disassemble→reassemble round-trip rebuilds the
string pool — the assembler **de-duplicates identical strings** — which shifts every later
string index by a small delta. Hundreds of methods then look "changed" while their behaviour is
identical.
Observed shape of that noise: **~800 methods reported as differing, 84% of them by only 1–4
bytes**, all of them constant-pool index shifts, with the method byte-length unchanged.
**Use instruction-stream length as the first-order predicate.** A real patch that changes control
flow or constant structure changes the method's `code_item.insns_size`. Pure index shifts and
debug-info reordering do not.
### The blind spot, and how to close it
**An equal-length replacement changes nothing measurable by length.**
Examples that bite: `move-result vX` → `const/4 vX, 0x0` (both one code unit); swapping one
`const` for another of the same width; any edit that keeps the instruction count identical.
Length-only auditing therefore **silently under-reports**: an audit reported 20 changed methods
on a build whose patch set actually touched 24 — the four missing ones were all equal-length
replacements.
**Two-tier rule:**
1. **Tier 1 (length):** scan every method. This yields candidates with essentially zero false
positives.
2. **Tier 2 (opcode-level):** compare the two versions opcode by opcode **for every method the
patch set declares as a target** — not merely the Tier-1 candidates.
**Tier 2's input cannot come from Tier 1's output.** Equal-length replacements produce no Tier-1
candidate at all, so the only reliable index of "what to check at Tier 2" is the patch set's own
declarations.
⇒ **Corollary, which is really a documentation requirement: every patch must declare its target
method signature.** A patch that is both equal-length and undeclared is invisible to both tiers;
finding it requires opcode-comparing every class, which is possible but expensive.
### A sharper Tier-2 shape for equal-length replacements
Compare the **position set of one opcode** inside the method:
```
target method, `move-result` (0x0a) position sets
original : [25, 43, 66, 72, 216, ... , 899] (16 entries)
patched : [25, 43, 72, 216, ... , 899] (15 entries) <- exactly one removed, pc=66
```
This proves two things at once: the intended site was hit, **and** the other same-shaped call
sites in that method were not. Asking "did anything differ" cannot distinguish those.
## 2. Is it legal? The verifier is a layer static checks cannot see
An assembler accepting your smali proves the file is **syntactically** valid dex. It does not run
the bytecode verifier. A build can pass assembly, pass every class-table check, and still be
rejected at class load:
```
java.lang.VerifyError: Verifier rejected class X: void X.<clinit>() failed to verify:
[0x1C8] copyRes1 v0<- result0 type=Undefined
```
**Root cause in the observed case:** padding instructions were inserted **between** a producer and
its `move-result`. DEX requires `move-result*` to **immediately follow** the instruction that
produces the result (`invoke-*`, `filled-new-array*`). The padding broke adjacency, the result
type became `Undefined`, and the whole method — and therefore the class — failed to verify.
The length arithmetic ("the shorter form plus two padding units preserves the total") was
**correct** and still produced an illegal method.
**Four layers, cheapest first:**
| Layer | Check | Catches |
|---|---|---|
| 1 | assembler exit code | syntax, register overflow, bad labels |
| 2 | class-table diff (`scripts/dex_classdiff.py`) | class-set / access-flag drift |
| 3 | **predecessor legality of `move-result*`** | result consumers detached from their producer |
| 4 | device: first execution of that method | everything above, plus real semantics |
**Layer 3 is a set-difference, not an absolute count.**
- For both builds, collect every `move-result*` whose immediately preceding instruction is **not**
a producer → the "offenders" set.
- Compare the sets. **New offenders must be 0.**
- **Never use the absolute size as a signal.** A perfectly healthy build can show hundreds of
offenders under this definition — payload pseudo-instructions (`packed-switch-payload`,
`fill-array-data-payload`, …) are normally read as ordinary instructions by a naive linear
scan — so the absolute number is dominated by scanner artifacts. Only the delta means anything.
Worked consequence: a build whose sole new offender was one `move-result` following padding
crashed at class load on device. After moving the padding to **after** the `move-result`, the
delta returned to 0 and the same build ran. **The delta predicted the device result before the
device was touched.**
## 3. When patches are applied by text matching
Some toolchains apply patches by literal match against a text form (smali) rather than through a
dex API. The failure modes are textual and unforgiving:
- **Whitespace is content.** A fragment indented with 5 spaces will not match a file using 4.
There is no fuzzy matching; one misplaced space is a zero-hit.
- **Line endings.** If the reader normalises CRLF→LF and the writer does not restore it, the
patched file becomes LF-only. A later byte-level comparison against the CRLF source then shows
the *entire file* as changed. Diff with an ignore-CR-at-EOL option, or compare structure.
- **Instruction names must be complete.** `move-result vX` and `move-result-object vX` are
different instructions; an anchor missing the suffix matches the wrong thing or nothing.
- **Operand punctuation matters.** `instance-of p3, p0, LType` is not the source text when the
source reads `LType;`.
- **Uniqueness is a hard requirement.** A matcher that finds several hits must fail loudly rather
than patch the first. When a one-line anchor is not unique, extend it with a second unique line;
**when the *combination* is still not unique, add a third anchor that discriminates the
homographs** — usually the branch target label, since identical call shapes often differ in
where they jump.
- **A re-emitted old fragment is not a failed write.** A patch may legitimately re-emit the old
text inside the new one (inserting a guard *before* the lines it matched). A read-back check
asserting "the old text is gone" then reports failure on that patch forever. Assert instead
that the **new, unique** part of the replacement is present.
- **One patch document, one patch block.** A parser that treats every fenced block as a candidate
will apply an *example* diff as a real patch — silently cancelling the real one, or editing
unrelated code. Keep examples in prose, never in the same fence format.
## 4. Audit the patch set, not a patch
For a multi-patch build:
```
expected = every method declared as a target by the patch set
tier1 = methods whose insns_size changed
tier2 = methods whose opcode stream differs (run over `expected`)
missing = expected - (tier1 U tier2) # must be empty
```
**`missing` is the number that matters.** A build can be green on every build step and still be
missing a patch. Report `missing` explicitly; do not report the total as if the total were the
goal.
**And re-verify the artifact, not the intent.** If the tool says "applied" but the build on the
device is unchanged, you have a stale build, a second copy, or the wrong file
(`pitfalls.md` P24). A patch tool's success message describes the tool, not the artifact.

1289
vendor-new/apk-reverse/skills/apk-reverse/references/pitfalls.md Normal file
View File

File diff suppressed because it is too large Load Diff

97
vendor-new/apk-reverse/skills/apk-reverse/references/precedents/README.md Normal file
View File

@@ -0,0 +1,97 @@
# Precedent library — what a pass got right, and how it got there
`pitfalls.md` is the negative catalogue: what went wrong. This directory is the other half —
**cases where the work actually converged**, recorded so the route can be walked again instead
of re-derived. The two do not overlap on purpose: a case here points at `pitfalls.md` by entry
number rather than restating the failure, and a pitfall that already has a case does not need a
second copy in prose.
Read a case when you are **about to do this kind of work again** — not when you are stuck. A case
answers "what sequence produced a defensible result, and what did it cost"; when you are stuck
the symptom index in `SKILL.md` is the faster route.
## Why this exists at all
This repository's most expensive recurring loss is not ignorance. It is **a conclusion that was
correctly retracted and then re-adopted**, because the retraction lived in a conversation while
only the original claim was ever written to a file. Three of the cases below are exactly that
shape: an attribution that was wrong, corrected by measurement, and corrected again. In each one
the *wrong* version is the one that reads as more coherent, which is why it is the one that would
be believed by a reader who only saw the summary.
A case therefore records the execution chain **including the dead ends** — the wrong turn, and
the measurement that killed it. Without that part, the case is a success story, and success
stories teach nothing that survives the next surprise.
## The cases
| Case | Route it establishes | Strongest claim in it | Grade |
|---|---|---|---|
| `flutter-plus-shell-case-1.md` | classify a hardened Flutter sample before trusting any dynamic result: a pid that keeps moving has to be attributed before it is explained | the same drift was attributed to frida, then to memory pressure, then to the sample's own root-environment check — only the third survived | observed (drift and its cause), unverified (that this generalises) |
| `frida-spawn-hangs-zero-events-case-2.md` | a harness can manufacture the very negative result it is looking for: `spawn` leaves the process suspended, and `CONFIG.autoStart` follows before your configuration lands | a zero-event trace was produced by two independent harness defects, not by the target | observed (the mechanism, and the warning that now fires) |
| `logd-broken-module-never-ran-case-3.md` | decide whether a hooking module ran at all, on a device whose log daemon is broken | nine successful injections produced **zero** `logcat` lines; the only evidence was a file under `/data/adb/lspd/log/` | observed (both channels), inferred (the framework-bridge explanation) |
| `l1-equal-length-patch-case-4.md` | the full chain from a two-byte dex edit to a visible change on screen, with a control build | the dialog disappeared **and** the control build failed the same way first, which is what makes the four bytes responsible | observed end to end |
| `manual-grep-finds-what-nobody-grepped-case-5.md` | why a leak scan is a gate and not a document, told from this repository's own near-miss | a shipped pass had to be corrected after the fact for identity a human grep found | observed (the omission and the correction), inferred (that a scanner would have caught it) |
Each case ends with an action checklist naming the **repository files that should be written
back to** if you hit the same thing. A case that does not produce a file edit has not finished.
## Template
Copy this skeleton. Keep the headings — the read-back check depends on them, and the grade
column is the part that stops a hypothesis from drifting upward into a fact.
```markdown
# <Title> — one line naming the conclusion, not the activity
## Metadata
| Field | Value |
|---|---|
| Context | what was being done, in one line |
| Cost | what it cost before it converged, measured |
| Outcome | what artifact or conclusion exists now |
| Evidence | the evidence files, by path |
| Related | `pitfalls.md` entries, and the neighbouring reference files |
## Assertions and grade
| # | Assertion | Grade | Evidence |
|---|---|---|---|
| 1 | ... | observed / inferred / unverified | exact command, file, or line |
Reserve `observed` for a claim with a command and its output behind it. An assertion whose
evidence is "the mechanism implies it" is `inferred`, and saying so is not hedging.
## Execution chain (including the dead ends)
The wrong turn, the measurement that killed it, and only then the route that held. Numbered,
because the order is the information: a reader who starts from the conclusion will pick the
wrong sequence.
## Pits
| Pit | Cost | What it was mistaken for |
|---|---|---|
| ... | measured number, or a count of re-runs | ... |
Use a cost you measured. "Two round trips" is a measurement; "a while" is not.
## Reusable pattern
The part that transfers to a different target with no editing. If it needs this target's names,
it belongs in the chain above instead.
## Write back to the repository
- [ ] `<file>` — <what to add to it, so the next reader does not pay this again>
- [ ] `<file>` — <index line, if a new file was created>
```
## Rules
- **Grade every assertion, and grade it down when in doubt.** `observed` means a command was run
and its output is in the evidence record condensed in `references/evidence-summary.md` §Where the full record lives. "I reasoned it out" is `inferred`. This is the
same standard `references/long-task-discipline.md` §Grade your own conclusions sets for a live
record, and it is the reason a case is worth reading a year later.
- **Record the retraction with the claim.** If an earlier conclusion was wrong, both versions
belong in the case: the wrong one, why it was wrong, and what replaced it. Deleting the mistake
loses the most transferable thing here.
- **Do not restate `pitfalls.md`.** Point at the entry. A duplicated failure mode drifts out of
sync with its original and then contradicts it.
- **One case, one route.** A case that establishes three unrelated things establishes none of
them; split it.
- **A case with no write-back checklist is a diary entry.** The last section is what makes it
knowledge the repository holds rather than knowledge the author had.

74
vendor-new/apk-reverse/skills/apk-reverse/references/precedents/flutter-plus-shell-case-1.md Normal file
View File

@@ -0,0 +1,74 @@
# A pid that keeps moving must be attributed before it is explained
## Metadata
| Field | Value |
|---|---|
| Context | dynamic analysis of a hardened Android sample (Flutter AOT payload behind a third-party application-stub) on a rooted physical device |
| Cost | three attribution corrections; two of them were written down as findings before being refuted |
| Outcome | the drift is attributed to the sample's own environment check, and the operational rule that falls out of it ("configure root hiding, then re-baseline") |
| Evidence | `references/evidence-summary.md` §The capability matrix (the pid-drift section and the frida-dexdump section); `references/evidence-summary.md` §The capability matrix (the root-hiding layer) |
| Related | `SKILL.md` §Stop conditions (last bullet); `pitfalls.md` P9, P18; `references/detection-and-anti-analysis.md` |
## Assertions and grade
| # | Assertion | Grade | Evidence |
|---|---|---|---|
| 1 | The sample starts and stays resident with no instrumentation attached | observed | `ps -A \| grep <PKG>` → `u0_a623 22691 … 642488 SyS_epoll_wait 0 S <PKG>`, 12 s after `am start` |
| 2 | During a session its pid changed repeatedly: `22691 → 23452 → 24267 → 27274 → 31284 → 32259 → 1368 → 8161` | observed | the same evidence file's pid-drift transcript |
| 3 | The drift reproduces with `frida-server` **stopped** and no attach | observed | a 90-second observation loop, four pid changes, with the frida process absent |
| 4 | The process dies as the **foreground top activity**, with no crash, no tombstone and no ANR record, and the platform relaunches it on a 7–18 s cycle | observed | `adb logcat \| grep <PKG>` → `Process <PKG> (pid N) has died: fg TOP` followed by `Start proc <pid> for top-activity <PKG>` |
| 5 | No application on the device was hidden from root at the time | observed | `magisk --denylist status` = enforced, `--denylist ls` printed nothing, `/data/adb/shamiko` absent (the root-hiding section of `references/evidence-summary.md` §The capability matrix) |
| 6 | The cause is the sample detecting its rooted environment and exiting on purpose | **inferred** — the mechanism fits every observation, but it was not proven by disabling the check |
| 7 | Memory pressure was **not** the cause | observed (refuted) | the drift continued at 300–750 MB free of 11.5 GB, and a reclaim kill leaves an `lmkd` line and a low-memory record — neither present |
| 8 | Instrumentation was **not** the cause | observed (refuted) | the `frida-server`-stopped control run |
| 9 | The device's own hooking framework was already damaged during the window (`lspd` alive but not updating its configuration, `zygote crashed too many times, rolling-back`) | observed | the framework-damage section of `references/evidence-summary.md` §The capability matrix |
| 10 | A `start timeout` kill line discriminates between these causes | **observed negative** — it reads identically for a slow init, a reclaiming device and a deliberate delay, so it is not evidence either way | the `ActivityManager` transcript quoted in the same evidence file |
## Execution chain (including the dead ends)
1. The control run looked healthy: start the sample with no instrumentation, wait 12 s, `ps` shows
it resident at 642 MB.
2. Later the pid had changed eight times. The natural reading — *the sample fights instrumentation*
— was written down as the finding. **Dead end 1.**
3. The control that killed it: stop `frida-server` entirely, restart the sample, observe for 90
seconds. The pid still moved, roughly every 10–20 s. Instrumentation is out.
4. The replacement attribution: memory pressure while a Flutter app saturates the CPU during its
startup window (439 % CPU, `kswapd0` at two thirds of a core were both measured). **Dead end 2.**
5. The measurement that killed it, and the reason it is worth recording: **a different instrument
was pointed at the same question.** `logcat` — not pid sampling — showed
`has died: fg TOP` with no crash record and an immediate relaunch. A reclaiming device does not
do that to a foreground process, and a memory-pressure kill leaves different traces.
6. That left "the sample checks its own environment and quits", and the environment check was then
findable: nothing on the device was hidden from root. A hardened sample that refuses to run
rooted produces exactly this loop, forever.
7. Only after the attribution did the *practical* conclusion exist: re-run the baseline with root
hiding configured, and treat every earlier dynamic result on that device as measured against a
process that was trying to die.
## Pits
| Pit | Cost | What it was mistaken for |
|---|---|---|
| Sampling the pid instead of reading the platform's lifecycle log | two full attribution rounds | a target-resistance hypothesis, then a host-resource hypothesis — the log settled it in one command |
| A 12-second control window | it looked like a stable baseline | the sample's lifetime was 7–18 s, so a single sample can land inside one life and read as stability |
| A third candidate mechanism discovered mid-pass (a zygote in a crash-and-rollback cycle restarts app processes by itself) | it invalidated single-cause attribution for the whole window | it did not create the drift, but it means "the pid changed" had at least three independent causes on this device at that moment |
| Two refuted attributions already written into an evidence file | deletion cost, not re-measurement cost | each reads as more coherent than the truth, which is why they would have been believed |
## Reusable pattern
- **Attribute before explaining.** A target that dies, drifts, or restarts gets a *mechanism* from
the platform's own record first (`logcat`, tombstones, `dumpsys`) — sampling tells you that it
happened, never why.
- **Refute the cheapest alternative first, with a control that removes it entirely.** Stopping the
instrumentation is one command; it eliminated the leading hypothesis in a single run.
- **When a measurement disagrees with a recorded conclusion, reopen the conclusion.** That rule
(`SKILL.md` §Stop conditions) was applied twice to this pass's own record, and both times the
record was the thing in the wrong.
- **Record the device's framework state next to any dynamic result.** This device could restart
the target without the target being involved.
## Write back to the repository
- [ ] `references/detection-and-anti-analysis.md` — a "the target exits on a rooted device, and the
environment is why" subsection: the `fg TOP` + immediate relaunch shape, the two kills that
look like it and are not, and the re-baseline step.
- [ ] `references/precedents/README.md` — already indexed as case 1; update the grade column if a
future pass proves assertion 6 by disabling the check.
- [ ] `references/pitfalls.md` — only if the *sampling-instead-of-logcat* mistake is not already
covered there; if it is, extend that entry with this measured cycle length rather than adding
a second one.

76
vendor-new/apk-reverse/skills/apk-reverse/references/precedents/frida-spawn-hangs-zero-events-case-2.md Normal file
View File

@@ -0,0 +1,76 @@
# A zero-event trace was manufactured by the harness, not by the target
## Metadata
| Field | Value |
|---|---|
| Context | instruction-level tracing with Frida Stalker on Android 11 / arm64-v8a, on system processes, to test whether `Stalker.exclude()` fixes the community-reported zero-event and crash behaviour |
| Cost | three arms plus one harness correction; the two defects below both produce a *believable* zero-event trace, so they would have been read as a target finding |
| Outcome | the crash half has a clean cause (`exclude` fixes it); the zero-event half does not, and is now an explicit warning in the script rather than a silent empty log |
| Evidence | `references/evidence-summary.md` §The capability matrix; `references/evidence-summary.md` §The capability matrix (device attempts A and B) |
| Related | `references/native-dbi-and-deobfuscation.md`; `scripts/stalker_trace.js`, `scripts/stalker_report.py`; `pitfalls.md` P26 |
## Assertions and grade
| # | Assertion | Grade | Evidence |
|---|---|---|---|
| 1 | A follow with an **empty** exclusion list killed the target process; the same process survived an attach + resume with no follow at all | observed | three-arm run: `baseline alive=True`, `control alive=False err=script has been destroyed`, `treatment alive=True` |
| 2 | Excluding 20 system modules kept the process alive through the follow window | observed | `treatment … DONE\|reason=timeout` with the process still running, `EXCL\|excluded=20/20 […] not-loaded=3` |
| 3 | Exclusion did **not** restore event delivery | observed | `treatment` still reported `blocks=0 blk=0 calls=0 truncated=0` after the full 6 s window |
| 4 | The zero-event warning added to the script fires correctly | observed | `treatment` emitted `WARN\|zero events 1500ms after follow … -- the pipeline is NOT proven`; `control` could not, because its process was already gone |
| 5 | `CONFIG.autoStart` defaults to `true`, so loading the script starts a follow with default options before any runtime configuration can land | observed | the design section of `references/evidence-summary.md` §The capability matrix; the harness patches it to `false` and asserts the patch count |
| 6 | `frida` spawns a process **suspended**: without an explicit `device.resume(pid)` the followed thread executes nothing and the run reports zero events | observed | the same design section, and the `DONE … blocks=0` transcript in `references/evidence-summary.md` §The capability matrix |
| 7 | `Process.getMainThreadId()` returned a tid that was **not** the pid on Android (26261 for a pid of 19938), so the script's default thread selection watched the wrong thread | observed | `TRIG following tid=26261 (main-thread follow (pid=19938))` |
| 8 | Re-running with the real main thread `start(19938)` produced the same zero | observed | `DONE reason=timeout blocks=0 blk=0 calls=0` |
| 9 | A hot-function trigger (`malloc` in `libc.so`) produced 201 complete follow cycles with zero blocks in any of them | observed | 609 log lines, 201 follows, 201 `DONE`s, `nonzero_blk = 0` |
| 10 | A per-call follow/unfollow cycle on that hot function crashed a system process | observed once | `systemui` pid `18030 → 26932`, `SIGSEGV` with frame `#00` in an anonymous executable region and the return path into `libart` |
| 11 | The crash is the explanation for the zero-event results | **observed negative** — those runs produced their zeros before any death, on two other processes, and one never touched an export trigger | the separation stated in the same native-DBI evidence file |
| 12 | Why events do not arrive | **unverified** | the measurement says it is not the exclusion list; it does not identify the cause |
## Execution chain (including the dead ends)
1. The question was framed as a two-claim question, which is why it resolved: does `exclude` stop
the crash, and does it restore events? Three arms, one device, one package, one module, one
follow window, exactly one variable moving between them.
2. The `baseline` arm exists purely to make a death attributable: without it, "following killed it"
and "the app dies under frida anyway" are the same observation.
3. Two harness defects had to be removed **before** the measurement meant anything, and both were
found by noticing that an arm produced the *expected* result for the wrong reason:
- `CONFIG.autoStart: true` followed once at `load()` time with default options, which is neither
the control nor the treatment, and therefore contaminated both.
- spawn leaves the process suspended, so with no explicit `device.resume(pid)` the followed
thread ran nothing at all.
4. Result: the crash half is answered with a cause, and the zero-event half is answered as a
**negative** — exclusion is not its fix, and the community framing that lists "no events" among
the symptoms `exclude` cures is contradicted by this measurement.
5. Independent corroboration that the pipeline itself was healthy: the export-trigger run produced
201 complete follow cycles. A trigger that never fired would look different; a filter that
dropped everything would look the same, which is exactly why the claim is limited to "events do
not arrive" rather than "Stalker is broken".
## Pits
| Pit | Cost | What it was mistaken for |
|---|---|---|
| `autoStart` following at `load()` | it invalidated two arms at once | a legitimate treatment result |
| No `device.resume()` after spawn | a whole arm's worth of zero events | the target refusing instrumentation |
| Reading a zero-event log as "the code did not run" | the reason the `WARN\|zero events` line was added to the script this pass | a finding about the target |
| `Process.getMainThreadId()` trusted as "the main thread" | one wasted run | a mis-selected thread |
| A system process died mid-run from the trigger pattern | one attribution round | an unrelated crash that could have been folded into the wrong conclusion |
## Reusable pattern
- **A negative result needs its own control arm.** `baseline` (attach, resume, never follow) is
what converts "it died" into "the follow killed it".
- **Engineer the harness's own failure modes out before measuring.** Both defects here produced the
exact shape under investigation; a run that reports the expected symptom is not yet a run that
measured anything.
- **Make a dead pipeline announce itself.** A warning line the moment the follow window ends with
zero events turns an ambiguous empty log into a labelled observation — and it is testable, because
the arm whose process died cannot print it.
- **Separate "the follow is unsafe" from "the follow observes nothing".** Two arms, two conclusions,
never one.
## Write back to the repository
- [ ] `references/native-dbi-and-deobfuscation.md` §6 — keep the two measured boundaries; add the
harness-defect paragraph if absent, because a reader who reproduces a zero event will
otherwise attribute it to the device.
- [ ] `scripts/stalker_trace.js` — already carries the zero-event warning; ensure the `autoStart`
comment says *why* the default is dangerous (it is a trap, not a convenience).
- [ ] `references/precedents/README.md` — indexed as case 2; move assertion 12 to `observed` only if
a future pass identifies the cause.

80
vendor-new/apk-reverse/skills/apk-reverse/references/precedents/l1-equal-length-patch-case-4.md Normal file
View File

@@ -0,0 +1,80 @@
# Four bytes of dex, and a control build that still failed the old way
## Metadata
| Field | Value |
|---|---|
| Context | an equal-length dex patch against a public crackme (`UnCrackable-Level1`), taken all the way to an installed, launched build whose blocking dialog is gone |
| Cost | one full pipeline re-run after the repository changed mid-pass; one false verification caused by a reused remote filename |
| Outcome | the end-to-end chain is measured, with a zero-change control through the **same** pipeline — which is what makes the four bytes, rather than the pipeline, responsible |
| Evidence | `references/evidence-summary.md` §The capability matrix (the B1 sections); the benchmark matrix (`references/evidence-summary.md` §The capability matrix) row B1 |
| Related | `references/byte-level-patching.md`, `references/patch-audit.md`, `references/repack-and-sign.md`, `references/verification.md` (the control-build rule) |
## Assertions and grade
| # | Assertion | Grade | Evidence |
|---|---|---|---|
| 1 | The shipped dex's `signature` field is stale while its `checksum` is correct | observed | `checksum field 7fb7d9fa` / `adler32(d[12:]) 7fb7d9fa` matches, while `sha1(d[32:])` does **not** match the stored signature |
| 2 | The edit site is the first arm of a three-way root `OR`, and the *fall-through* is the "continue" side | observed | `0x9d0 39000e00 if-nez v0 -> 0x9ec` preceded by `invoke-static …a()Z` / `move-result`, and `0x9ec` is `const-string "Root detected!"` |
| 3 | Branch polarity was pinned by the following instruction, not by the method name | observed | `expect_next` required `invoke-static …b()` (the *next* root check) after the branch — `polarity: expect_next satisfied` |
| 4 | The patch owns exactly 4 bytes and the header fields own 24; nothing else in the file moved | observed | whole-file diff offsets `['0x8','0xa','0xb','0xc'…'0x1f', '0x9d0','0x9d2']`, and both header fields recomputed from the written file |
| 5 | The signature was computed **before** the checksum | observed | `zlib.adler32(d[12:])` and `hashlib.sha1(d[32:])` recomputed on the output reproduce both stored fields, which is only possible in that order |
| 6 | `repack.py` reported `resources.arsc STORED and 4-byte aligned (OK)` for control and patched alike | observed | the alignment gate output, identical for both builds |
| 7 | Both builds signed with v1+v2+v3 and the same certificate digest | observed | `apksigner verify --print-certs --verbose --min-sdk-version 21` |
| 8 | The installed artifact is the built artifact | observed | device `base.apk` sha256 equals the local sha256, for control (`e5a9f335…`) and patched (`ad6a51ce…`) |
| 9 | The control build — same pipeline, byte-identical dex — **still shows** the blocking dialog | observed | four window-focus samples, 2 s apart: `Window{… Root detected!}` for original and control, `Window{… <PKG>/…MainActivity}` for patched |
| 10 | The patched build reaches its own UI without a dialog | observed | the screenshots: control/greyed-out input with the modal over it, patched with the input live, caret present, `VERIFY` enabled |
| 11 | A second, independently built patched APK from the one-command pipeline produced the **same** sha256 as the manual route | observed | `ad6a51cef4699b73ff052d4e049ca9efd64e9570900d19509f0b7a7c20f87e56` |
| 12 | The app's functional check compares the user's input against a decrypted plaintext, and the 16-byte AES key is a `const-string` in the dex | observed | `a.b` hook output: key `8d12…73cc`, data decrypts to `I want to believe`, `[a.a] REAL input="" -> false` |
| 13 | The input field cannot be driven by `adb shell input text` on this ROM | observed | 7 of 17 characters survived and every space was dropped, across three independent quoting routes; `uiautomator dump` after each attempt is what distinguished "the app said Nope" from "the input never arrived" |
| 14 | The success branch was reached by forcing the boolean, not by typing the secret | observed, and labelled as such in the evidence file | `evidence/09_success_branch/*.png` shows the app's own `Success! …` dialog |
## Execution chain (including the dead ends)
1. **Read the branch structure before editing.** The edit target was chosen because its fall-through
continues into the next root check — not because of its name. That is the difference between a
patch that removes one arm of an `OR` and a patch that removes the check.
2. Dry run first: the tool prints the site with the instructions before and after, and the polarity
evidence. Nothing was written until `polarity: expect_next satisfied`.
3. Applied, then proved with a byte-level diff of the whole file rather than the tool's own report:
four payload bytes and the two header fields, and nothing else.
4. Repack. **Dead end:** the documented default signing route failed with
`error: signer jar not found: uber-apk-signer.jar`, and its own hint — "any zipalign+apksigner
based signer works here" — pointed at a code path that did not exist yet. The working route was
the supported `--no-sign` seam plus explicit `zipalign` → `apksigner`, and the repository changed
mid-pass (another writer landed an `apksigner` route), so the pass recorded **both** states and
re-ran the one-command form, which produced byte-identical output. The lesson kept with it:
*a "this tool cannot do X" finding on a shared worktree has an expiry date.*
5. Install through the **root** path (`pm install -r -d`), because this ROM intercepts the streaming
install (see case 5's sibling record for the `[-99]` shape).
6. Verify, in the order that matters: hash the installed `base.apk` against the local build; `am
start -W`; then window focus sampled four times; then **look at the screenshots**.
7. Build a zero-change control through the identical pipeline and repeat steps 5–6. It still fails
the old way, which is what closes the attribution.
## Pits
| Pit | Cost | What it was mistaken for |
|---|---|---|
| Scanning the method for a plausible name instead of reading the branch structure | would have patched an `OR` arm that changes nothing | a successful patch |
| Relying on the tool's own "self-verify: checksum_ok=True signature_ok=True" without an independent diff | the stale-signature starting state could have hidden a wrong header order | a correct header |
| Reusing the `--tag` value as the remote filename across runs | the final verification showed the **control** build's dialog, from the control run's leftover remote path | a patch that did not work |
| `adb shell input text` on a field that silently drops characters | two attempts read as "wrong secret" | a wrong secret |
| Trusting a screenshot alone | it cannot tell which build is installed | a verified patch |
## Reusable pattern
- **Equal-length byte patches are the cheap route; choose the site by branch semantics.** Nothing
moves, so no offset, `try` block or debug pointer can be invalidated, and the diff is auditable
down to the byte.
- **A patch needs a same-pipeline control that fails the old way.** "It installs and runs" and "the
behaviour changed" are different claims, and only the second one is the deliverable.
- **Hash the artifact on the device, at the moment you observe it.** This is the guard that caught
the reused-tag mistake, and it is one command.
- **Look at the screen, and look at more than one sample.** A modal that appears and is gone between
samples is invisible to a single capture.
- **Re-derive the header yourself once.** Recomputing both fields from the written bytes is what
proves the *order*, and the tool's own success message cannot.
## Write back to the repository
- [ ] `references/repack-and-sign.md` — if the `signer jar not found` message and its apksigner
route are not already reconciled, that is a documented contradiction worth closing.
- [ ] `references/verification.md` — the control-build rule is the load-bearing part of this case;
add the "hash the installed APK before observing" step if it is only implicit there.
- [ ] `references/precedents/README.md` — indexed as case 4; the row's grade stays `observed` end to
end.

70
vendor-new/apk-reverse/skills/apk-reverse/references/precedents/logd-broken-module-never-ran-case-3.md Normal file
View File

@@ -0,0 +1,70 @@
# "The module never ran" was an artefact of a broken log channel
## Metadata
| Field | Value |
|---|---|
| Context | verifying that an LSPosed hook module was actually injected into a hardened target, on a rooted Android 11 device |
| Cost | the verification criterion itself had to be corrected; `logcat` was empty across nine successful injections |
| Outcome | the injection is proven from `/data/adb/lspd/log/modules_<timestamp>.log`, and the read rule is now "both channels, always" |
| Evidence | `references/evidence-summary.md` §The capability matrix (the injection-line, log-surfaces and scope sections) |
| Related | `references/evidence-summary.md` §The capability matrix (the framework-damage section); `references/lsposed-and-modules.md`; `references/environment.md` |
## Assertions and grade
| # | Assertion | Grade | Evidence |
|---|---|---|---|
| 1 | The module was injected nine consecutive times, each run reaching the target's `MainActivity.onCreate`, the WebView process and an ad SDK activity | observed | nine full injection blocks; nine pids in ~3 minutes (`8111 → 9392 → 10557 → 11453 → 12345 → 13236 → 14155 → 15061 → 15934`) |
| 2 | `logcat -s <TAG>` was **empty**, as was `logcat -s LSPosed-Bridge` and a full-buffer `grep -i <TAG>` | observed | the injection-line section of the LSPosed evidence file, stated as the correction to the pass's own earlier draft |
| 3 | The module's output existed only in `/data/adb/lspd/log/modules_<timestamp>.log` | observed | the evidence file for the pass is 201 lines, of which 126 are `<TAG>` |
| 4 | The same file earlier appeared to contain nothing but `Logd maybe crashed (err=Socket operation on non-socket), retrying in 1s...` | observed | the log-surfaces section of the same evidence file quotes the entire content of an older `modules_*.log` |
| 5 | The ROM's `logd` route is broken, so LSPosed's logcat channel delivers nothing while its file channel works normally | **inferred** — the two observations above are consistent with exactly this, and the mechanism was not probed further |
| 6 | A reader debugging a module with `logcat` alone would conclude the module never ran, while looking at nine successful injections | observed (as a statement about the evidence), inferred (as a statement about other readers) |
| 7 | The target's double-`Application` packer swap is visible in the hook order, and hooks installed at `handleLoadPackage` time survive it | observed | `Application.attach` fires twice — the real `BaseApplication`, then the stub — and by `Activity.onCreate` the reported classes are the **unpacked** ones |
| 8 | Scope is per app, not per process | observed | the module also injected into the target's `com.google.android.webview` process and reported an ad SDK activity |
| 9 | The pre-reboot self-restart cycle (10–20 s) can be compared with the post-reboot cycle (22–23 s) | **inferred at best, and not a controlled result** — a reboot moves several variables at once | the injection-line section of the same evidence file |
| 10 | `verbose_*.log` is a full logcat snapshot, useful for correlating a run with system state and useless as a module log | observed | the log-surfaces section of the same evidence file |
## Execution chain (including the dead ends)
1. The module was built, installed, PM-enabled, and scoped through LSPosed Manager's UI — read from
`uiautomator dump` coordinates, never tapped by guess. The scope change was confirmed in the
framework's own SQLite database (`scope` gained `(2, '<PKG>', 0)`, `modules.enabled` flipped to
`1`, and the database file grew from 156,376 to 201,696 bytes), which is what proves the
framework — not merely its daemon process — came back.
2. The pass then read `logcat -s <TAG>` and found nothing. **Dead end:** the criterion in the
reference file, in the pass's own earlier draft, and in the scaffold's generated README all said
to watch `logcat`.
3. Rather than conclude "injection failed", the other channel was read: LSPosed's file log. Nine
full injection blocks.
4. The correct criterion was written down, and it is the transferable part: read **both** channels.
`logcat -s <TAG>` where the platform is healthy; `/data/adb/lspd/log/modules_*.log` always.
5. The reason the two disagree on this device is a broken `logd` (`Socket operation on non-socket`),
which is also visible in unrelated incident lines from the same window (`system server died`,
`am is dead`, `no response from bridge, retry in 1s`, `Magisk: zygote crashed too many times,
rolling-back`).
## Pits
| Pit | Cost | What it was mistaken for |
|---|---|---|
| Trusting one log channel on a ROM whose log daemon is damaged | it would have produced a false negative on a working feature | "the module never loaded" |
| `logcat` silence read as absence of evidence *for* the target's behaviour | the verification criterion itself was wrong, not just the reading | a statement about the module |
| The same silent-failure shape appears twice in the pass | once for the module channel, once for the framework's own state | two different "not installed"-looking states, one of which was real |
## Reusable pattern
- **Before accepting a silence as a finding, ask whether the instrument could have spoken.** This is
`pitfalls.md` P26's rule pointed at a *log channel* rather than at a script: check that the channel
has ever delivered for a healthy case.
- **Prefer a channel that is a file over a channel that is a daemon.** `logcat` depends on `logd`;
a file in the module's own directory depends on the module.
- **Verify the verifier.** The scope change was confirmed in the framework's own database rather
than in the manager's UI reflecting it back — a UI can render a cached state, and the file is the
state.
- **On a packed target, a Java-layer hook is reachable without touching the APK**: the double
`Application` swap is observable, and hooks installed before it survive it. That is a *capability*
result, separate from the log-channel lesson, and it is the reason this route is worth the setup.
## Write back to the repository
- [ ] `references/lsposed-and-modules.md` — the verification section must name both channels and the
exact file path pattern; a single-channel instruction is what this case disproves.
- [ ] `scripts/lsposed_scaffold.py` — the generated README's verification line should carry the same
two-channel instruction, because it is the artifact a user reads first.
- [ ] `references/precedents/README.md` — indexed as case 3; assertion 5 stays `inferred` until the
logd mechanism is probed directly.

83
vendor-new/apk-reverse/skills/apk-reverse/references/precedents/manual-grep-finds-what-nobody-grepped-case-5.md Normal file
View File

@@ -0,0 +1,83 @@
# The leak nobody grepped for, found by hand and only after the fact
## Metadata
| Field | Value |
|---|---|
| Context | this repository's own history: a complete verification pass was recorded, reviewed and merged, and the target identity in its evidence files was found afterwards |
| Cost | a manual sweep, and the omission survived at least one pass that produced evidence files |
| Outcome | a desensitization convention (`<PKG>` / `<DEVICE>` placeholders, target identity absent by policy) **and** an automated scan (`scripts/scan_leaks.py`), added in the same pass that recorded this case |
| Evidence | `references/evidence-summary.md` §The capability matrix; `references/evidence-summary.md` §The capability matrix (its identifier-convention and injection-line sections); `references/evidence-summary.md` §The capability matrix (its environment section) |
| Related | `references/desensitization-and-leak-scans.md`; `references/long-task-discipline.md` §Keep a live record, not a log; `references/verification.md` §Reporting |
## Assertions and grade
| # | Assertion | Grade | Evidence |
|---|---|---|---|
| 1 | Identity was found by a human grep after the material had already been written, and the files had to be corrected | observed as this repository's stated history; the correction itself is visible in the convention every evidence file now opens with | the environment section of `references/evidence-summary.md` §The capability matrix: *"Target identity is deliberately absent. The sample is referred to as the sample, and its package identifier appears as `<PKG>`"* |
| 2 | The convention was enforced per file, by hand, at write time | observed | the identifier-convention section of `references/evidence-summary.md` §The capability matrix normalises every occurrence of the package identifier and its sub-package prefixes, and states that the transcripts are otherwise verbatim |
| 3 | Verbatim transcripts were kept, and that is deliberate | observed | the same section keeps the hook-reported class names, "those are what make the evidence checkable" |
| 4 | No structural gate could have caught it | observed | `check_repo.py` validates layout/frontmatter/`--help`; `check_refs.py` validates section anchors; neither reads content for identity. Both passed on the material that had to be corrected |
| 5 | An automated scan would have caught it | **inferred** — the scanner was built and measured against a planted corpus, but it was never run against the historical, pre-correction tree | `references/evidence-summary.md` §The capability matrix, the planted-corpus and repository-scan sections |
| 6 | The scan finds the four classes that actually occur here — bundle ids in `pm`/`ps`/`manifest` contexts, 16-character serial-shaped tokens, inline key assignments, host user paths | observed | planted corpus: 30 findings across 6 categories, every rule firing |
| 7 | The scan's first run against the real tree was overwhelmingly false positives, and each one was a shape that legitimately appears in this documentation | observed | the repository-scan section of `references/evidence-summary.md` §The capability matrix: 40 findings when the file above was still being written, of which 26 were `strong`/`certain` — and every one of those 26 was inside that pass's own evidence file. After the correction: 0 strong, 4 weak (RFC 5737 usage examples) across 129 files |
| 8 | Over-redaction is the failure on the other side, and it is not hypothetical here | observed | the do-not-anonymize list in the reference file is derived from identifiers that a shape-only scanner flagged in this tree: tool names, SDK packages, dex constant identifiers, public crackme names |
| 9 | The ratio, not the count, decides whether a gate survives | observed from this case | the pass that produced 27-of-28 false positives also had to fix 26 real ones in its own report on the same day. Both numbers moved the design: exemptions for the shapes that legitimately appear, and a split exit code so the remaining documentation-range hits do not make the gate permanently red |
## Execution chain (including the dead ends)
1. The convention existed and was written down per evidence file: identity absent, placeholders for
the package, the device serial and the host paths, verbatim transcripts otherwise.
2. It was enforced by hand, at write time, by whoever wrote the file. **This is the dead end**: a
convention enforced by attention has no failure signal. The omission was invisible until someone
went looking, and by then the material was already in the tree.
3. The correction happened after the fact, which is why this case exists at all. A search for the
shape — not for the value — is the only route that scales, because the value is exactly what you
do not yet know.
4. The scan was then built, and the interesting result was **not** "it finds leaks" (that was the
easy part). It was that a first run over the real tree produced 28 findings of which 27 were
legitimate documentation: a JDK version that looks like an IPv4 literal, tombstone hex tokens
that look like serials, a public crackme's package name, a `probe.synthetic.*` fixture, a
property lookup. A scanner that shipped in that state would have been ignored within a week, and
the leak it exists to catch would have gone with it.
5. The final state separates the two decisions: the scanner reports shapes and admits known-benign
explanations **with a printed reason** (`--show-exempt`), and a human makes the identity call
from the context window the report already contains.
## Pits
| Pit | Cost | What it was mistaken for |
|---|---|---|
| A convention with no automated check | one manual sweep after the fact | adequate coverage |
| Searching for the *shape* after the fact, rather than during the write | it only works if you already suspect | a finished pass |
| A shape-only scanner | 27 false positives in 28 findings on this tree alone | a working gate |
| Suppressing the false positives by blocking whole address ranges | it would also have hidden the one real weak hit (`scripts/tls_check.py`'s RFC 5737 example) | noise reduction |
## Reusable pattern
- **Desensitize at write time, and verify by shape, not by value.** The value you have to remove is
the one you cannot enumerate in advance; the shape is the thing you can write a rule for.
- **Decide in code what is exempt, and let the code print why.** An exemption list that cannot be
audited is indistinguishable from an exemption list that is too broad, and only one of those is
visible in review.
- **Keep the transcripts verbatim and replace the identity with a placeholder.** Redacting the
*evidence* destroys the thing that makes the record checkable; redacting the *identifier* costs
nothing.
- **Measure the scanner against your own tree before trusting it.** The false-positive ratio, not
the true-positive count, decides whether anyone will keep running it.
## Write back to the repository
- [ ] `references/desensitization-and-leak-scans.md` — the reasoning, the split rule, and the
failure-mode table; this case is its motivating evidence.
- [ ] The evidence record's own index — `docs/tool-verification/README.md` at the **repository root,
not shipped** — carries the topic entry. The in-skill condensation is
`references/evidence-summary.md`.
- [ ] `references/precedents/README.md` — indexed as case 5.
- [ ] **Closed in this pass**: `scripts/scan_leaks.py` is wired into the repository workflow (run it
over the tree before a commit) and the `RESULT=` convention is documented in
`references/long-task-discipline.md` §Every script answers with a token, not with prose.
- [ ] **The scanner caught its own first pass.** The tool written to prevent this class of leak found
26 `strong`/`certain` hits on the day it was written, all of them inside its own evidence file —
the fixture's literal match values, quoted to prove the rules fire. The fix was to keep the
location, rule, category and grade for all 30 findings and replace every literal with a shape
description. Boundary that makes it a rule rather than a scare: **the corpus may contain values;
the published surface may not.** Evidence and transcripts: the "self-leak this file carries" and
"inherited versus post-fix" sections of `references/evidence-summary.md` §The capability matrix.
This is the strongest available argument for the gate — the author of the rule needed it on the
same day, and did not see it by hand.

338
vendor-new/apk-reverse/skills/apk-reverse/references/protocol-reverse.md Normal file
View File

@@ -0,0 +1,338 @@
# Protocol reverse engineering
**Load this when the app's traffic is not readable by the usual means**: the body is protobuf with no
schema, the transport is gRPC, the connection is QUIC rather than TCP, or the client refuses to trust
your proxy because it validates the certificate inside a native library.
Everything in this file assumes you have already decided the traffic matters and that a proxy is
allowed to see it. If a *feature-scoped* TLS failure has appeared while the rest of the app works,
that is a different problem with its own file: read `tls-and-cert.md` first — it owns the trust-chain
attribution and the permissive-defaults template, and this file deliberately does not repeat them.
**Claim strength.** `measured` = an exact command and output exist in
`references/evidence-summary.md` §The capability matrix. `inferred` = documented behaviour or a step that
follows directly from a measured one. `unverified` = reported by others, not reproduced here. The
wire-format section is `measured` on a self-built fixture; every tool named below is `unverified`
unless stated otherwise.
## 1. Protobuf on the wire (measured)
Protobuf has no field names on the wire and no header. A payload is a flat sequence of
`tag, value` pairs, and a nested message is just a value of type "bytes whose content is another
sequence". That is the whole format; the schema supplies meaning and nothing else.
| Element | Encoding |
|---|---|
| Varint | base-128, least-significant group first, high bit set on every byte but the last |
| Tag | `(field_number << 3) \| wire_type`, itself a varint |
| Wire type 0 | varint value (int32/int64/uint/bool/enum) |
| Wire type 1 | fixed 8 bytes (fixed64/sfixed64/double) |
| Wire type 2 | length-delimited: length varint, then that many bytes (string, bytes, nested message, packed repeated) |
| Wire type 5 | fixed 4 bytes (fixed32/sfixed32/float) |
| Wire types 3/4 | group start/end, legacy; treat as unsupported when walking a modern payload |
A host-side check of the rules, including the canonical `150 -> 96 01`, was run independently of the
shipped decoder — `references/evidence-summary.md` §The capability matrix records that cross-check against
the official runtime. Measured output on the fixture built for it:
```
message (29 B): 089601120774657374696e671a0508011201782206038e029ea7052800
field 1 varint 150
field 2 len-delimited(7) 'testing'
field 3 len-delimited(5) nested message:
field 1 varint 1
field 2 len-delimited(1) 'x'
field 4 len-delimited(6) hex '038e029ea705'
field 5 varint 0
rebuild from the walked rows equals original: True
```
Three consequences that decide how you interpret any decoded body:
1. **Field numbers are local to their message.** In the walk above, field 3 of the outer message is a
nested message whose own field 1 exists independently. Nothing on the wire says which message a
field number belongs to; only a schema does.
2. **A packed repeated field is invisible without a schema.** Field 4 above is three varints
(`3, 270, 86942`) packed into one length-delimited value. A schema-free decode can only report it
as opaque bytes — the length is known, the element boundaries are not.
3. **A decoded `0` and a field that was never set are two different questions.** An implicit
(no-presence) proto3 field whose value is the default writes **nothing at all**, so `field = 0`
and "never touched" produce identical bytes; a field **with** presence (`proto2 optional`,
`proto3 optional`) set to `0` does put its tag and a zero byte on the wire. Both halves were
measured against the official runtime; a schema-free decoder can only flag the case, never
resolve it. So a decoded `0` was emitted on purpose by some writer, and a field *missing* from a
decode is not evidence that 0 was the value in use.
### Decoding without a schema
- `protoc --decode_raw < body.bin` — the reference implementation's own raw decode; prints field
numbers and values in exactly the shape above. Note it prints strings and nested messages without
distinguishing them, which is why the fallback order matters: try a nested walk first, then
printable UTF-8, then hex.
- `protobuf-inspector` and `blackboxprotobuf` are the pip-installable equivalents, and both add
heuristics: guessing whether a length-delimited value is a string or a submessage, and (for
`blackboxprotobuf`) inferring a candidate type set per field across many samples. Useful when you
have traffic but no `protoc`; still unverified here.
- Decode **many samples of the same message**, not one. The number of distinct field numbers in a
large sample set is what tells you which fields are optional, which repeat, and which are enums
whose value set you can then enumerate.
- Anything the app stores locally is a sample source that needs no proxy at all: an OkHttp disk cache
(the `journal` file plus its entries), a DataStore file, or a Room blob column. `runtime-data.md`
covers reading these, and `datastore_inject.py` can re-encode an edited DataStore value once you
know the message.
### The decoder that ships here (measured)
`scripts/protobuf_decode_raw.py` walks a payload with no schema and prints a tree. Input is a hex
string, a binary file, or stdin (hex text and raw bytes are told apart automatically); `0x` prefixes,
spaces, commas, newlines and `\xNN` escapes are all accepted, and `|` splits independent frames.
`--json` emits the same tree as data, `--max-depth` bounds the nesting, and `--reencode` writes a
decoded (possibly hand-edited) tree back to bytes and can diff the result against the original.
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --hex "08 96 01 12 07 74657374696e67"
frame 0 bytes 0..12 (12 B) status=end_of_buffer
f1 varint 150
f2 len-delimited(7) view=utf8_string
candidate utf8_string 0.50 valid UTF-8 with no control characters
candidate packed_varint 0.50 the payload is exactly a sequence of 7 varint(s) ...
candidate bytes 0.20 opaque bytes: always consistent with the payload ...
tie: packed_varint and utf8_string are equally consistent with these bytes ...
string: 'testing'
```
The point of the tool is the candidate list. Fed a nested message that `google.protobuf` 6.33.6
itself produced (inner payload `0801120178`), it answers:
```
nested_message 0.50 | packed_varint 0.50 | utf8_string 0.35 | bytes 0.20
tie: nested_message and packed_varint are equally consistent ... the view is a display default
```
Both readings really are legal protobuf for those six bytes, and no heuristic closes the gap. The
confidence numbers are a documented ordering, not probabilities, and the `view` exists only so the
tree can be expanded and re-encoded -- it is not a claim about what the field is.
**Round trip, which is the strongest evidence available without a schema.** Decode, re-encode, and
compare bytes: the message serialized by the official runtime re-encodes byte-identically to
`SerializeToString()`; a real AndroidX DataStore file (81 B, written by `scripts/datastore_inject.py`)
round-trips byte-identically; and editing one value in the JSON tree and re-encoding produced a file
that the *other* tool read back as the new value. Two of the built-in fixture families are exceptions
by design and are reported as such rather than hidden: a non-canonical varint re-encodes to the
minimal form, and a `--split varint-length` run compares frame bodies only, since the length prefixes
are framing and not message data.
**What a schema-free decode cannot conclude -- write none of these down:**
- that a length-delimited field *is* a string, a submessage or a packed array, rather than that it
could be any of them;
- the element boundaries of a packed array, or its element type;
- that a field's value was 0 because it is absent from the decode, or that a sender "set" a field
because a 0 appears (see the two traps above);
- which message a field number belongs to, or that the same field number in two payloads means the
same thing;
- that a payload which parses cleanly is a real message at all -- §6 has that failure mode, and the
fixes are a second sample and a round trip, not a longer look at the first one.
The measured record for this tool -- every command, both input forms, the round trip, and the
framing mistake it does *not* protect you from -- is in
`references/evidence-summary.md` §The capability matrix.
## 2. Recovering the schema from the APK
### What generated protobuf code looks like
| Marker in the decompiled output | What it tells you |
|---|---|
| `extends GeneratedMessageLite` / `GeneratedMessageV3` | protobuf class; `Lite` means the reflection/schema runtime, not the descriptor-based one |
| `static final int FIELD_NUMBER = 3;` (or `int <name>_ = 3`) | the field number, usually right next to the field's getter |
| `switch (tag >>> 3)` with `case` constants | the parse loop's field dispatch — each `case` is a field number, in declaration order |
| `input.readInt32()`, `readInt64()`, `readBool()`, `readStringRequireUtf8()`, `readBytes()`, `readMessage(...)`, `readEnum()` | the field's wire type and semantic type, read *after* the tag |
| `input.readTag()` returning 0 as the loop exit | the standard `parsePartialFrom` skeleton; the loop body is where all field knowledge lives |
| `writeTo(CodedOutputStream)` / `getSerializedSize()` / `dynamicMethod(...)` | the class really is a generated message, not a hand-written DTO |
| `newMessageInfo(instance, "<escaped string>", ...)` | protobuf-lite's compact schema blob (see below) |
The generator keeps those framework calls even after R8, so an obfuscated app usually still carries
readable protobuf structure: class and field *names* are gone, field *numbers* and wire types are
not.
### Two routes to a `.proto`
**Route A — read the field table out of the bytecode (inferred).** Decompile the class, walk
`parsePartialFrom`, and record `case <n>` → the `read*` call that follows. Field name unknown? Name
it `field_<n>`; a valid `.proto` does not need the original identifiers, and the app's own getter
names often survive anyway. Then verify immediately against captured bytes with
`protoc --decode=<message> <schema.proto>`: a decode that produces a coherent structure for **every**
sample is the proof that your field table is right, and it costs seconds. A field that decodes into
nonsense means a wrong wire type, not a wrong field number — check the `read*` call before moving on.
**Route B — read protobuf-lite's own schema blob (inferred).** protobuf-java-lite 3.x embeds a
compact schema string (`rawMessageInfo`) per message class, consumed by `MessageSchema` at runtime.
Recovering it means locating that blob and feeding it to a matching runtime build. It is a strictly
larger win when it works — it carries field numbers, types, and labels in one place — and it is
version-sensitive, so treat Route A as the default and Route B as the acceleration. `unverified`
here: no target in this repository was analysed this way.
**Tooling that automates this (unverified):** `pbtk`/`protodump`-style scanners walk dex/JAR inputs
looking for the generated-code shape above and emit `.proto` files, and a protobuf-aware decompiler
plugin can do the same from a loaded APK. Use them to produce a first draft, then verify it with
`protoc --decode` against real captured bytes — that verification is the step that turns a guess into
a schema, and it is the step worth spending time on.
## 3. gRPC
gRPC is HTTP/2 with a message framing convention, so the first decision is whether you can see
HTTP/2 at all.
- **Recognise it.** A request carries `:method: POST`, `content-type: application/grpc[+proto]` (also
`+json`, `+thrift`), and `:path: /<package>.<Service>/<Method>` — that path shape is the reliable
fingerprint; the package in it usually matches a package name visible in the APK, which is how you
connect traffic to a client class.
- **Message framing.** Each message is: one compression-flag byte, then a 4-byte **big-endian**
length, then that many bytes of (possibly compressed) protobuf. Several messages can share one
DATA frame and one message can span frames, so reassemble by length, not by frame. The call's
outcome is in **trailers** (`grpc-status`, `grpc-message`), which many proxies display separately
from the body — a `200` with no body and a nonzero `grpc-status` is a normal gRPC error.
- **Read it after capture.** Extract the reassembled message bytes and hand them to
`protoc --decode_raw` or your recovered schema. `grpcurl -plaintext -protoset file.protoset`
replays against an endpoint once you have a schema, which is also the fastest way to confirm that
a recovered `.proto` has the right shape.
- **Capture it.** A transparent TCP proxy sees HTTP/2 fine; a tool that only understands HTTP/1.1 may
show the tunnel without the frames, which looks like "the app is not making requests". Interception
tooling for this comes in two shapes: a proxy add-on that understands the gRPC framing and prints
messages (`grpc-dump` and similar mitmproxy add-ons, or a gRPC-Web decoder when the client is a
browser-style stack), and a command-line client (`grpcurl`) for replay against a live endpoint. If
the app uses TLS (gRPC-Java does by default), the certificate question comes first — see
`tls-and-cert.md`, and the native-material trust chain below.
## 4. QUIC / HTTP/3
The failure to expect: a working HTTP proxy, a working CA, and **an empty capture**.
- **Why.** An HTTP proxy is a TCP endpoint, and a client reaches it with `CONNECT`. QUIC is a
UDP-based transport with TLS 1.3 built into the handshake; there is no TCP connection for a
`CONNECT` to carry and no separate TLS layer for a proxy to terminate.
- **Recognise it before debugging anything else.** `Alt-Svc: h3=":443"` in an earlier HTTP response
announced the upgrade; on the wire you see UDP traffic to port 443 whose first packets are QUIC
Initials (long header, version field, and a TLS ClientHello inside), and the client's ALPN is `h3`.
Check for UDP activity when a TCP proxy stays idle while the app plainly works.
- **Options, in order of how little they change:**
1. **Force a fallback to TCP.** Block or blackhole UDP 443 for the device. A client that cannot
reach an origin over QUIC falls back to TCP/HTTP2 if the server also offers it, and then your
existing proxy works unchanged. This is the cheapest option and the one to try first, but it is
a **network-condition change**: state it in the report, because "the app failed to load over
QUIC" and "the app failed" look identical in a screenshot.
2. **Decrypt with keys, not with a proxy.** TLS 1.3 key logging (`SSLKEYLOGFILE`-style) works for
QUIC exactly as it does for TLS-over-TCP, and a packet capture plus the key log can be
decrypted in Wireshark. It requires the client to support a key-log callback — Chromium-based
stacks and BoringSSL generally do, Java/OkHttp-based stacks generally do not — so this is a
per-client property, not a general solution.
3. **Turn HTTP/3 off in the client** by hooking whatever enables it (a library version check, a
remote-config flag, or a build-time constant). This is a client-side patch and therefore
belongs to the repack decision, not to the capture step.
- Everything in this section is `inferred`/`unverified` here: no HTTP/3 target was exercised in this
repository. Treat the ordering as a plan whose first step (block UDP 443) needs no tool support to
test.
## 5. SSL pinning that lives inside a native library
Three different mechanisms are routinely called "pinning", and only the last one survives a system
certificate module.
| Mechanism | Where it runs | What defeats it |
|---|---|---|
| OkHttp `CertificatePinner` (or a custom `TrustManager`) in Java/Kotlin | app's own client | Client-level, **not** the system path: `tls-and-cert.md` §Step 4 and §"If OkHttp is the failing path" own this case — do not apply the permissive-defaults template to it |
| System trust decisions via `HttpsURLConnection` / platform stack | platform | System store policy — see `tls-and-cert.md` §Step 5 |
| Verification performed by a **library the app ships**, e.g. a Flutter app's own BoringSSL, or a bundled `libcurl`/`mbedtls` | `libflutter.so`, `libapp.so`, another `.so` | Only an in-memory change to that library's verification path (below), because the library does not consult the platform trust manager you can influence from Java |
**Flutter, specifically.** A Flutter app's `dart:io` HTTP client runs on the engine's BoringSSL
inside `libflutter.so`; the app's own Dart code can additionally supply a custom trusted-certificate
context or a bad-certificate callback. Two consequences that decide which direction to work in:
- If the app is **not** pinning and simply fails to trust your CA, getting your CA into the **system**
store is the normal fix. `MoveCertificate`-style Magisk modules exist precisely because Android 7+
ignores user-store CAs for app traffic; moving the CA into the system store is a **device change**,
not an artifact, and it must be labelled as a fallback rather than the deliverable. Its boundary is
exactly this: it changes what the platform trust store contains, so it can only help a client that
reads that store.
- If the app **is** pinning, or supplies its own verification context, the system store is irrelevant
and the change has to land on the library's verification function.
**Locating the verifier inside `libflutter.so` (unverified — reported method).** The published
approach in
[universal-flutter-ssl-pinning](https://github.com/vichhka-git/universal-flutter-ssl-pinning)
(`github.com/vichhka-git/universal-flutter-ssl-pinning`) automates the anchor search with PyGhidra:
1. load `libflutter.so` headlessly in Ghidra and scan its defined strings for `ssl_client`;
2. resolve cross-references from that string to the **containing functions**;
3. decompile each candidate to count its parameters;
4. take the **3-parameter** function — that is BoringSSL's
`ssl_crypto_x509_session_verify_cert_chain`, the certificate-chain verification entry point;
5. emit the discovered RVA into two ready-to-run artifacts: a Frida script that attaches to that
address and replaces the return value with `1` (success), and a Renef script that patches the
function entry with the ARM64 sequence `MOV X0, #1 ; RET`.
The repository reports it working on Google Flutter arm64-v8a and Shorebird-patched builds with
Frida 16.x/17.x, and notes the RVA is identical across build types for the same engine version. The
same project also ships an HTTP monitor that auto-locates BoringSSL's `SSL_write`/`SSL_read` in
`libflutter.so` and prints reassembled, de-chunked plaintext — which is the route to take when the
traffic cannot be proxied at all, because it never needs a certificate the app will accept.
**Two things to verify before trusting that recipe on your target.** First, the string anchor is a
build artifact: a stripped or repacked engine can lose `ssl_client` while keeping the function, in
which case fall back to the structural search — find the `X509_verify_cert`-shaped call sequence in
the handshake code, or identify the function called with the handshake structure during
`certificate_verify`. Second, the failure mode of a wrong anchor is a **silent** success: replacing a
return value in a function that merely *looks* like the verifier makes the certificate check pass
in a process that then fails somewhere else, which is easy to misread as "the app detected my hook".
Always confirm the hook fires on a connection that would otherwise fail, and confirm it is
`SSL_VERIFY`-shaped semantics (the same value the library itself uses for success), rather than
assuming `1` is right.
**When the string anchor is gone (the `session_creator` route).** A second family of anchors does not
depend on the `ssl_client` literal at all: instead of finding the verifier through a string it
references, find it through the call chain that reaches it. BoringSSL's handshake creates the session
and then verifies the peer certificate chain, so the target is the function invoked with the
handshake/session structure on that path — this is the shape that Flutter-unpinning write-ups call the
session-creator route. It survives string and symbol stripping, because the call graph survives, and
it costs more: you must identify the handshake structure first, which is why the string anchor is the
fast path and this is the fallback when an engine build has dropped the string. `unverified` here — no
engine binary was analysed in this repository.
**When the system store is genuinely enough.** If the client is Java-side and non-pinning, moving the
CA into the system store is the least invasive change and needs no per-library work. The decision is
therefore: identify the client first (`tls-and-cert.md` §Step 4 attribution), then choose the
mechanism. Patching a library that was never consulting the store is work with no effect; installing
a system CA when a native verifier ignores it produces the same empty capture you started with.
## 6. Failure modes
**A decode that looks valid but is not.** protobuf is self-synchronising enough that random bytes
often decode into a plausible-looking field list. Guard against it by (a) decoding several samples of
the same message and expecting the same field set, (b) checking that nested lengths exactly consume
their parents, and (c) when you have a schema, re-encoding the decoded structure and comparing bytes
with the input — the same round-trip check used in, which caught a decoder bug in this
repository's own fixture.
**A framing choice that is wrong without being refused.** `--split varint-length` means protobuf's own
`writeDelimitedTo` framing: a varint length, then the message. gRPC frames a message differently — one
compression-flag byte, then a **four-byte big-endian** length (). Handed a real gRPC frame, the
varint splitter reported **five** frames rather than failing, while the same bytes decoded with no
splitting reported `stray_end_group`. Strip the 5-byte gRPC prefix yourself, and check the frame count
either way: a wrong split still produces a decode, which is what makes it expensive.
**Assuming the proxy sees everything.** The proxy sees what the client sends through it. An app that
uses QUIC (), a raw socket, a native client bypassing Java's HTTP stack, or a certificate-pinned
connection shows up as silence. Silence is evidence about the transport, not about the app: check
UDP activity, check whether a native library is doing the I/O (`lib_map.py`, `dynamic-frida.md`),
and only then conclude that a feature is client-side.
**Patching the wrong verifier.** Covered in §5: the failure is silent, and the resulting state is
worse than a clean failure because it looks like progress. Verify the hook against a connection
known to fail.
**Treating a recovered schema as ground truth for the server contract.** A schema recovered from the
client tells you what the client expects to send and parse — it does not prove the server honours
those fields, and it says nothing about authorization. `server-api.md` owns that question. The
recurring expensive mistake is to assume that because a field exists and is settable client-side, the
server will act on it.

157
vendor-new/apk-reverse/skills/apk-reverse/references/rasc-and-droidsaw.md Normal file
View File

@@ -0,0 +1,157 @@
# rasc and droidsaw — the Rust ASC, and what it changes about the indexer-first workflow
Load this when: you are about to reach for `droidasc` (ASC) to locate classes or references, or when
the Python indexer is the bottleneck on a real archive. It states what the Rust re-implementation is,
how it was measured here, and **the one class shape where it silently loses code** — which is the
part that decides whether you use it alone or keep a second tool next to it.
**When to prefer it, in one line:** `rasc` is the same interface with the same answers, several times
faster, and one measured blind spot; use it as the indexer, keep `droidasc` and JADX as the
cross-checks the rest of this skill already requires.
## What it is
`MG1937/ASC` has two implementations on different branches. `main` is the Python tool this skill
already documents (`pip install droidasc`). The **`rust` branch is `rasc`**, a re-implementation that
keeps the CLI shape and drops the Python runtime:
| | Python `droidasc` main | `rasc` (rust branch) |
|---|---|---|
| Runtime | CPython + androguard | none — one static-ish binary, **2.13 MB** |
| Decompiler behind `getclass` | androguard's | **`droidsaw-dex 2.0.0`** (pure Rust; its project advertises byte-identical DEX re-emission on F-Droid) |
| Install | `pip install droidasc` | **must be built** — no release asset, no crate (`cargo install rasc` fetches an unrelated maths parser) |
| Commands | `listclass` `getmanifest` `getclass` `findrefs` | `classes` `manifest` `getclass` `findrefs` |
| Flags | `--prefix` on `listclass` | `--filter` on `classes` (case-insensitive substring, not a prefix) |
| Shared flags | `--threads/--thread`, `-o/--output`, `--debug` | same, plus `--debug` prints phase timings on stderr |
`scripts/rasc_build.py` wraps the build and the verification, because **a build only one machine has
performed is a build nobody else can reproduce**: `--check` reports what is present, `--build`
clones and compiles, `--verify <apk>` compares the class-definition sets against `droidasc` and fails
if they differ.
## Measured here
Environment: Windows 11, rustup `stable-x86_64-pc-windows-gnu` (rustc 1.98.1), MinGW-w64 gcc 16.1.0
(UCRT, 64-bit) as the linker. Build from a clean checkout: **117 s**, `cargo build --release` with
FatLTO. Binary `rasc` 0.1.0, 2,236,416 bytes, sha256 `1becdb86b444fe7910589f8ade5bccf13794de9f2b466a1c07748b5669c0264b`,
from `MG1937/ASC@e809c7b4b454ebc8eb4467f56d44a82e6139472b` (rust branch).
Two archives, fastest of N runs, same machine, `PYTHONUTF8=1` for the Python side:
| Archive | Scenario | rasc | droidasc | ratio | answers |
|---|---|---|---|---|---|
| MASTG `UnCrackable-Level3.apk` (1.4 MB) | `classes` | 0.025 s | 0.116 s | 4.7× | **identical set** (1396 / 1396) |
| | `findrefs string http` | 0.021 s | 0.299 s | 14.0× | identical (3 / 3) |
| | `findrefs type String` | 0.021 s | 0.307 s | 14.7× | rasc superset (390 vs 384) |
| | `getclass` ×2 real classes | 0.086 / 0.082 s | 0.297 / 0.267 s | 3.3–3.5× | both emitted Java |
| a real 34.8 MB app (30,768 classes) | `classes` | 0.048 s | 0.190 s | **4.0×** | **identical set**, 0 differences both ways |
The speedup is smaller than the upstream README's geometric mean of 8.0× because that figure was
measured on a 343 MiB archive with 567k classes, where startup cost disappears; on a mid-size app the
Python runtime's import time keeps the ratio near 4×. Both are real, and the small-archive numbers are
the ones a normal task hits.
`python skills/apk-reverse/scripts/rasc_build.py --verify <apk>` reproduces the class-set comparison
and is the check to run after any rebuild.
## The blind spot: `getclass` on the outer enum does not inline the constant bodies
`bench/jadx_parity.py` from the `rasc` repository compares **string literals** between JADX and `rasc`
per class, counting a literal JADX has and `rasc` does not as a lost-code signal. Run here on a fair
sample — app-like classes, no obfuscated identifiers, since the harness itself refuses to judge a
class JADX cannot decompile — **20 classes picked, 17 judged, 1 with a difference, 0 reorderings**.
The one difference is `org.bouncycastle.crypto.PasswordConverter`, an enum whose constants each
override an abstract method. JADX inlines each constant's anonymous subclass; `rasc` prints the
constant list:
```java
// JADX
public enum PasswordConverter implements CharToByteConverter {
ASCII { // from class: ...PasswordConverter.1
@Override public byte[] convert(char[] cArr) { ... }
@Override public String getType() { return "ASCII"; }
},
...
```
```java
// rasc -- getclass on the OUTER class
public enum PasswordConverter implements org.bouncycastle.crypto.CharToByteConverter {
ASCII, UTF8, PKCS12;
public PasswordConverter() { }
public PasswordConverter(PasswordConverter$1 v3) { }
}
```
**What is actually lost, measured directly on the constants.** The bodies live in their own classes,
so the question is whether they survive as their own classes. They do:
| class | `rasc getclass` | `droidasc getclass` |
|---|---|---|
| `PasswordConverter$1` | 404 B, contains `convert(`/`getType(` | 460 B, contains both |
| `PasswordConverter$2` | 407 B, contains both | 463 B, contains both |
| `PasswordConverter$3` | 406 B, contains both | 462 B, contains both |
So the correct severity is **a missing inlining step in the outer-class view, not unrecoverable code**:
the source is one `getclass` away, on the anonymous subclass JADX names in its own comment
(`// from class: ...PasswordConverter.1`). This reference said "the method bodies are gone" when this
row was first written, and that was wrong — the subclasses were never queried. The two Python-visible
symptoms are the same in both tools (neither inlines), so the difference here is JADX's convenience,
not a `rasc` regression against the Python implementation.
`droidasc`'s outer output is nevertheless richer than `rasc`'s for the same class — 2,170 B against
314 B, carrying the synthetic `$VALUES`, `$values()` and the constructors explicitly — which matters
when the enum's *shape* is the thing you are reading.
The rule, now proportional to what was measured:
- **Use `rasc getclass` for locating and reading ordinary classes.** That is what it was measured on.
- **When the class is an `enum` with constant-specific bodies, read the constants.** A collapsed enum
is visible on sight (bare constant list, no bodies). Either query the anonymous subclasses directly
(`Lpkg/Enum$1;`) or use `droidasc`, whose outer-class listing is more complete.
- **Never treat a single decompiler's output as the definition of what exists.** The same rule the VMP
section states about opcode tables, applied to a class body.
Frequency for calibration: 1 of 17 judged classes here, and the other 16 agreed literal for literal.
Adopt it, and keep the cross-check — but do not describe this as lost code.
## What it does not change
- **The workflow.** The indexer is the same *stage* it always was: `rasc` answers *location*
questions, and reading a class you have already located is what a decompiler is for. It does not
decompile a whole APK into a project, and neither does the Python tool.
- **The verification discipline.** Every claim above is a measurement with a command behind it;
where `rasc` and `droidasc` disagree in the tables, the disagreement is recorded rather than
averaged away.
- **Packer behaviour.** On this repository's hardened sample both tools correctly return 4 classes —
the payload is encrypted inside the shell and is not in any DEX either tool can read. A small class
count on a hardened target is the target, not the tool.
## Building it yourself
```bash
python skills/apk-reverse/scripts/rasc_build.py --check # what is present
python skills/apk-reverse/scripts/rasc_build.py --build # clone + cargo build --release
python skills/apk-reverse/scripts/rasc_build.py --verify app.apk # compare against droidasc
```
Requirements and the traps measured here:
- **Rust 1.93+** (the crate is `edition = "2024"`; this build used 1.98.1) and `git`.
- On Windows, the **GNU host toolchain needs a 64-bit MinGW-w64 gcc**. A 32-bit MinGW fails at link
with `sorry, unimplemented: 64-bit mode not compiled in` — the fix is a 64-bit distribution (this
build used winlibs gcc 16.1.0), not a flag.
- The build fetches crates from crates.io; `vendor/` in the repository only carries the two patched
crates (`axmldecoder`, `droidsaw-dex`), so `--offline` fails with `no matching package named anyhow`.
- `cargo`/`rustc` installed to a non-default `CARGO_HOME` are invisible to a non-interactive PATH;
`rasc_build.py` prepends the known locations rather than reporting the toolchain as absent.
## Provenance
`https://github.com/MG1937/ASC` (branch `rust`, commit `e809c7b`), retrieved 2026-09-22.
`https://github.com/droidsaw/droidsaw` and `https://crates.io/crates/droidsaw-dex` (2.0.0),
consulted 2026-09-22. The Python implementation is `pip install droidasc` (0.1.1.post1 seen here;
0.1.1.post2 published 2026-09-21). Nothing from either project is vendored into this repository —
`rasc_build.py` clones it into the gitignored `tools/_work/` and the measurements above name the
commit they came from.

204
vendor-new/apk-reverse/skills/apk-reverse/references/recon.md Normal file
View File

@@ -0,0 +1,204 @@
# Recon — build the picture before touching anything
Ten minutes here prevents hours of wrong work. Answer the five questions from `SKILL.md` with concrete evidence.
**Load this when:** starting any new sample. It gives identity, packer detection, embedded SDKs, ABI, and where the app's own code lives -- the ten minutes that prevent hours of wrong work.
## 1. Identity
```bash
# Without a full decompiler (fast, works everywhere):
aapt dump badging app.apk | head -20
# or, from the extracted manifest:
# package name, versionName, versionCode, sdkVersion, targetSdkVersion
# or, if you have ddc (no JVM, single binary, also gives the launcher):
ddc info app.apk
```
Record: package name, version name/code, min/target SDK, all requested permissions, all declared components.
**Do not hand-parse the package name out of the binary manifest as your primary
source.** Real AXML string tables are full of class-name fragments that look
exactly like package names — in one sample, the strings visible without a real
parser suggested `com.<app>.offline` (a service class prefix) while the actual
package was `com.<vendor>.app`. Every later `pm list packages`,
`dumpsys package`, `/data/data/<pkg>` and `pm install` command depends on this
value, so getting it wrong sends the whole recon down a path where nothing is
found and the natural conclusion is "the tool is broken".
Cross-check two independent sources and require agreement:
- `ddc info app.apk` → `package`, `label`, `launcher`
- `aapt2 dump badging app.apk | head -1` (or `aapt dump badging`)
If they disagree, resolve it before proceeding. A single `label` value is also
worth having: it is what appears on the launcher, which makes device-side
identification unambiguous later.
**Note:** some `aapt` builds choke on non-ASCII paths. Normalize sample paths to ASCII before running tooling.
## 1b. Snapshot the config surface early
If the app has a remote-config endpoint, its DTOs are the cheapest thing in the
package to find, because data classes serialise their own field names into strings
that survive R8:
```bash
ddc findrefs app.apk string "SplashConfig" # or Config, Popup, Banner, Tabbar...
ddc strings app.apk -f "enabled" --with-locations
```
Do this during recon rather than later: it tells you immediately whether the
launch screen, popups and tab set are **client decisions** or **server data**,
which is the difference between a five-minute patch and a dead end. See
`server-config-and-updates.md`.
## 2. Is it packed?
Read `AndroidManifest.xml` → `application android:name`.
| What you see | Meaning |
|---|---|
| The app's own class (e.g. `com.example.app.App`) | **No packer.** Dex is directly editable. |
| A vendor class (`com.stub.StubApp`, `com.secneo...`, `com.tencent.StubShell`, `s.h.e.l.l.*`, `com.nagain.*`, …) | **Packed.** Unpack first. |
| A short meaningless name that does not match the app's package (e.g. `com.bc41`), plus `android:appComponentFactory` pointing at another short class | **Dex-level whole-APK packer.** The shell registers its own Application/proxy under names unrelated to the app. |
| `classes.dex` is only tens of KB and holds nothing but shell classes | The real code is not on disk. |
| A shell library in `lib/<abi>/` (`libDexHelper.so`, `libjiagu*.so`, `libshell*.so`, …) | Native half of the shell: decrypts and loads the payload. |
| One or more large high-entropy `assets/` blobs, possibly **disguised as JPEG** (valid magic bytes, no decodable image) | Encrypted payload. |
| `assets/` contains a second `.apk` or `.jar` | Wrapped/loader build. |
**A clean shape is also a result: record it and move on.** An app whose
`application android:name` is its own class, with a single plain `classes.dex` and
no native libraries, is directly editable — and that finding is what lets you skip
the entire packer/unpack branch of this skill. It is worth one explicit line in the
record ("no packer: application is the app's own class; dex readable; no lib/") so
that later steps cannot re-open a question that was already answered. Most real
targets are this shape; the packer material in this repository exists for the rest.
Decide from sizes and structure, not from names:
```bash
unzip -l app.apk | sort -k1 -n | tail -20 # biggest entries: assets? libs? dex?
```
**Loaded-shell check (highest fidelity):** if the app runs and its own classes are reachable, it is unpacked *at runtime* regardless of what the manifest says. If the app's own code is not among the dex classes, it is packed on disk.
### Unpacking a dex-level packer
**1. Map payload → dumped dex by length.** If the shell uses **length-preserving** encryption, each encrypted `assets/` payload has **exactly the same byte count as the plaintext dex** it decrypts to. That equality is the strongest single discriminator for "which dump is a real original dex" — far more reliable than sorting dumps by size. Measure it per file before trusting any dump.
**2. Check how the real dex is loaded.** A log line like `InMemoryDexFile[cookie=[0, ...]]` means the dex is built in memory from the payload and never written to disk, so there is no dropped file to find. Use a **memory-scanning** dumper (`frida-dexdump`, or your own script walking the class loaders) instead of hunting for artifacts under the app's data directory. See `references/dynamic-frida.md`.
**3. Two dumper caveats that cost real time.**
- `-D <device-id>` does **not** accept a remote device (`Device '<id>' not found`). Register a remote device with `-H <host>:<port>`; keep `-D` for local/USB ids.
- The `-o <dir>` output directory **must already exist** — the tool does not create it. Otherwise every dex write fails and the run still ends with `All done`, which reads exactly like a successful dump of nothing. `mkdir -p out` first, and judge the result by the file count, never by the final message.
**4. Filter the dumps before using any of them.** A memory dump contains three kinds of garbage:
- **Duplicate copies of one dex.** Identical byte count does **not** imply identical content — files of the same size have been measured to differ in ~73% of their bytes, carry a different dex version, and fail to parse. Dedupe by **content hash**, never by size.
- **SDK plugin dexes downloaded at runtime**, which were never part of the APK.
- **Structurally broken dexes** whose header and body disagree.
Cheap header check that catches the third class, plus the hash used for the first:
```python
import struct, hashlib
d = open(p, 'rb').read()
assert d[:4] == b'dex\n' # magic
assert struct.unpack_from('<I', d, 32)[0] == len(d) # file_size == actual size
assert struct.unpack_from('<I', d, 36)[0] == 112 # header_size
sha = hashlib.sha256(d).hexdigest()
```
Then confirm survivors actually parse (`baksmali`/dexlib2) or load on device. Header consistency alone is not proof of a usable dex.
**5. Which dump is the app's own business dex.** Count string-feature hits per dump instead of trusting file order:
```bash
python scripts/dex_strings.py <dump_dir> --find 'Lcom/example/app/' --per-file
python scripts/dex_strings.py <dump_dir> --find 'Lcom/example/adsdk/' --per-file
```
Heuristic: the business dex mixes the app's package prefix (hundreds of hits) with an ad-SDK prefix (thousands of hits). A pure SDK plugin dex has one vendor prefix at its maximum count and **zero** app-package hits. Zero app-package hits means it is not the app.
## 3. Where does the app's own code live?
| Indicator | Code location |
|---|---|
| Plain `classes*.dex`, app classes visible in them | **Dex** — patchable with dexlib2/smali |
| `libflutter.so` + `libapp.so` | **Flutter (Dart AOT)** — dex contains only a thin shell |
| `libil2cpp.so` + `assets/bin/Data/` | **Unity (IL2CPP)** — native |
| `libmono*.so` + `assets/bin/Data/Managed/` | **Unity (Mono)** — managed DLLs under `assets/` |
| `libreactnativejni.so` + `assets/index.android.bundle` | **React Native** — JS bundle under `assets/` |
| `libhermes.so` | **Hermes bytecode** |
| Almost everything in `.so` | **Native** — different toolchain entirely |
For a **Kotlin/Java app**, note the module layout: app code often lives in a small subset of dex files while large SDKs occupy the rest. Finding which dex holds the app's package is a huge time-saver:
```bash
python scripts/dex_strings.py <dex_dir> --find 'Lcom/example/app/' --per-file
```
Consequence for editing: you usually only need to replace **one or two dex files**, which keeps the repack minimal. When the app is packed, run this check against the **dumped** dexes, not the ones on disk ().
## 4. SDK and library inventory
Extract strings across all dex and group by vendor markers. This reveals ads, analytics, crash reporting, attribution, and any anti-tamper SDK, in one pass.
```bash
python scripts/dex_strings.py <dex_dir> --urls
python scripts/dex_strings.py <dex_dir> --find 'anythink|openadsdk|com.qq.e|umeng|bugly|crashsdk'
```
What to look for:
- **Ad networks / aggregators**: `openadsdk` `TTAdSdk` `Pangle` `com.qq.e` `GDTAd` `anythink` `ATSDK` `ksad` `mobads` `sigmob` `bdxadsdk`
- **Analytics / crash / attribution**: `umeng` `bugly` `crashsdk` `appsflyer` `adjust` `UMCrash`
- **Identity / device id**: `oaid` `msa` `SupplementaryDID` `freemme`
- **Anti-tamper / root / hook detection**: `libsgcore` `libInno` `libqmcheat`, strings like `frida` `xposed` `magisk` `substrate`, `/system/xbin/su`
- **Media stack**: `libmpv` `libavcodec` `libplayer` `libgdx`
Important distinction: **a vendor marker in the string table does not mean the app uses that feature**. Presence of `xposed`/`frida` strings is often just an SDK's own detection list. Decide based on *behavior*, not on strings (`references/verification.md`).
## 5. Signature and tamper checks
Search the app's **own** packages only (ignore third-party SDKs) for:
- `getPackageInfo`, `PackageInfo`, `signatures`, `GET_SIGNATURES`, `Signature` → app-side signature verification
- hardcoded SHA-1/SHA-256 hex constants, base64 license blobs → certificate pinning to the original signer
- `checkSignature`, `verifySignature`, `signCheck` → a named check
**Beware false positives.** `SignatureCheck` and `verifySignature` exist inside `okhttp3` (`SuppressSignatureCheck`, `BasicCertificateChainCleaner.verifySignature`) — TLS plumbing, not app tamper checks. Likewise a native `lib*.so` named like a security library may be an ad SDK's payload decryptor, not a shell.
Confirm by checking whether **app code** references it. If the only callers are library-internal, it is not your problem.
## 6. API surface
```bash
python scripts/dex_strings.py <dex_dir> --urls
```
Collect: base URLs, DoH/DoT lookups, path constants, CDN hosts, custom headers.
Treat **unique** strings as navigation aids — they can often be searched byte-wise in the dex to find the owning class (see `references/dex-patching.md` §finding-the-call-site).
Two patterns worth recognizing early:
- **Dynamic gateway**: the real API host is fetched at runtime (e.g. via a DNS TXT record over DoH). The hardcoded host in dex may be only a fallback. This affects server-side analysis, not patching.
- **Minimal headers / no request signing**: if auth is just `Authorization: Bearer` plus a static app-name header, then a repackaged client is not distinguishable to the server by request signature — which matters for `references/server-api.md`.
## Recon output
Write a short profile before patching. Minimum:
```
package / version / ABI
packed? (evidence)
if packed: packer family, unpack method, kept dumps + sha256
code location (dex / flutter / native)
app code in which dex files
ad SDK(s) and the app's wrapper class
analytics/crash SDKs
anti-tamper: present? where? app-side or library-internal?
API base + notable endpoints
device plan (rooted real device / emulator / static only)
```

279
vendor-new/apk-reverse/skills/apk-reverse/references/repack-and-sign.md Normal file
View File

@@ -0,0 +1,279 @@
# Repack and sign
Rebuilding is the step where otherwise-correct patches die. Two mistakes account for nearly all of it: **stripping `META-INF/`** and **recompressing entries that must stay stored**.
**Load this when:** rebuilding, signing, installing, or a repacked build misbehaves. It gives the two mistakes that account for nearly all failures here, the STORED-and-aligned rule, and every install refusal that looks like a broken build.
## The rules
**1. Strip only signature artifacts, never the whole directory.**
`META-INF/` contains ServiceLoader registrations the app needs at runtime. Details and the exact failure in `references/pitfalls.md` P1.
```python
SIG_EXT = ('.SF', '.RSA', '.DSA', '.EC')
def is_signature_entry(name):
if not name.upper().startswith('META-INF/'):
return False
rest = name[len('META-INF/'):]
if '/' in rest: # keep services/, androidx/, native-image/ ...
return False
up = rest.upper()
return up == 'MANIFEST.MF' or up.endswith(SIG_EXT)
```
**2. `AndroidManifest.xml` and `resources.arsc` must stay uncompressed (STORED).**
Recompressing them produces builds that fail to install or misbehave.
**2a. `resources.arsc` must ALSO be 4-byte aligned, and the failure is an install refusal.**
On Android 11+ (targetSdk 30+) the package manager rejects the install outright:
```
Failure [-124: Failed parse during installPackageLI: Targeting R+ (version 30 and
above) requires the resources.arsc of installed APKs to be stored uncompressed and
aligned on a 4-byte boundary]
```
Two independent requirements hide in that one sentence: STORED, and the entry's
**data offset** divisible by 4. Uncompressed `lib/*.so` follows the same rule.
A naive `zipfile.ZipFile(...).writestr(...)` loop satisfies neither reliably:
Python's zip writer gives you no control over entry offsets. `scripts/repack.py`
therefore emits local headers itself and pads the **local extra field** to reach
the boundary. The padding arithmetic has one trap worth knowing:
> A zip extra area is a sequence of `(id, size, payload)` records, so its minimum
> useful length is 4 bytes. A required pad of **1-3 bytes cannot be expressed**.
> When the natural pad falls in that range, insert a stored filler entry of
> exactly that size instead -- the filler has a computable size, so the following
> entry still lands on the boundary.
Verify by reading the archive, not by trusting the writer: for each entry Android
cares about, parse its local header, compute `header_offset + 30 + namelen +
extralen`, and check `compress_type == 0` and `offset % 4 == 0`.
`repack.py` prints exactly this ("alignment gate") and `check_alignment()`
returns the complaint list programmatically.
**Do not plan on repairing alignment with `zipalign` after the fact when you can
produce a correct archive directly.** A post-hoc pass rewrites the file; on a
target that fingerprints its own byte layout, that is a much larger change surface
than writing it correctly the first time. Keep `zipalign -c -v 4` as an
independent check when the tool is available.
**3. Keep the APK's zip entry metadata.** Preserve `compress_type`, `external_attr`, `date_time` for entries you copy through.
**4. Changing any dex changes nothing about resources.** If you only edited dex, do not touch `res/`, `assets/`, or `lib/`.
**4a. A full `apktool b` rebuild rewrites resource paths even when you edited none.** Decoding with
`apktool d` (without `-s`) and rebuilding re-encodes resources, and obfuscated short names are
expanded back to readable ones — an entry originally shipped as `res/-B.png` comes back as
`res/drawable-hdpi/<real-name>.png`. Expect an entry-level diff against the original to show on the
order of a thousand "removed + added" pairs that are **pure renames**, not content changes.
This is expected and usually harmless, but two consequences matter:
- **Never read that diff as "I broke something".** Compare by identity (rename-aware, or by content
hash grouped by size) before drawing a conclusion. A toy diff that reports 1100 changes when you
edited one method is a **tool** artifact and will send you hunting a bug that does not exist.
- **It changes the byte layout of the whole archive.** On a target that fingerprints its own file
against a stored hash, or whose protection binds offsets into a container, a full rebuild is a much
larger change surface than a dex-only swap. If your only edit is dex, prefer replacing the
`classes*.dex` entries inside the original zip (`compress_type` preserved) over a full rebuild.
Keep that dex-only path as a fallback for exactly this reason.
If you must avoid resource churn entirely, decode with `-s` (do not decode resources) and only
rebuild what you changed.
**5. Signing creates new `MANIFEST.MF`/`*.SF`/`*.RSA`.** That is expected — the check is that **no signature artifact from the ORIGINAL** survives, and **no non-signature entry was lost**.
**6. Any dex you edited must have its header integrity fields recomputed.**
Every dex carries two fields that cover the rest of the file:
```
bytes 12..32 = sha1(data[32:]) # signature — must be computed FIRST
bytes 8..12 = adler32(data[12:]) # checksum — covers the signature, so it is LAST
```
Skipping this does not always stop the app from starting, which is what makes it
dangerous. Android logs
`Failure to verify dex file ...: Bad checksum (computed, expected)` — the real
value appears as "expected", which reads backwards — and then falls back to
interpreting the dex instead of using a verified image. The visible symptom is an
unrelated startup failure such as `ClassNotFoundException` for an ordinary class
(the Application class, or a small AndroidX component), so the trail points at the
APK structure rather than at two stale header words.
`scripts/dexutil.py` exposes `fix_dex_header()` (correct order, plus
`verify_dex_header()` so you can assert the result), and
`scripts/dex_patch_bytes.py` runs both automatically before writing.
**7. `jarsigner` rewrites the archive; `apksigner` does not.**
Adding a v1 JAR signature with `jarsigner` recompresses entries as a side effect,
which **destroys the alignment from rule 2a**. The signature still verifies, so the
build looks correct until the install is refused with `[-124]`. Sign with
`apksigner` (v1+v2+v3) unless you have a specific reason not to.
## Repacking an unpacked (de-shelled) app
When the sample was packed, the dex you are about to patch came out of a memory dump while the APK still carries the shell. Build a **de-shelled base APK** first, then treat it as an ordinary APK for the rest of this document.
1. **Assemble the base.** Start from the original APK: replace `classes*.dex` with the dumped real dexes, and delete the shell library under `lib/<abi>/` plus the encrypted `assets/` payloads. Strip **only** signature artifacts — `META-INF/*.SF|*.RSA|*.DSA|*.EC` and the top-level `MANIFEST.MF`. Never delete the whole `META-INF/` (`references/pitfalls.md` P1).
2. **Decode with `apktool`.** `apktool d base.apk` gives the smali tree and a **readable text manifest**.
3. **Fix the manifest.** Point `application android:name` at the app's real Application class and **remove** `android:appComponentFactory`. Grep the decoded tree for leftover shell class references before building.
4. **Patch, then rebuild.** `apktool b` → `zipalign -p -f 4` → `apksigner` (v1+v2+v3) → install and verify against the checklist at the end of this file.
Why `apktool` rather than editing binary AXML in place: changing `android:name` in binary AXML means hand-editing the string pool and the attribute/chunk sizes around it, and a mis-sized chunk produces an APK that installs but throws far from the edit, typically in component lookup at startup. The text round-trip moves the risk to "did the rebuild preserve everything else", which item 5 of the checklist verifies directly.
## Pipeline
`scripts/repack.py` implements all of the above. Conceptually:
```
open original (zip)
for each entry:
skip entries that are signature artifacts (per rule 1)
skip entries being replaced
copy through, preserving compression + attributes
write replacement entries
close
sign (v1 + v2 + v3)
verify
```
Usage:
```bash
python scripts/repack.py --apk original.apk --dexdir extracted_dex_dir --out out.apk
# or replace specific dex files:
python scripts/repack.py --apk original.apk --dexdir extracted_dex_dir \
--dex "classes7.dex=patched/classes7.dex" \
--dex "classes8.dex=patched/classes8.dex" \
--out out.apk
```
`--dexdir` supplies the untouched dex files; `--dex name=path` overrides specific ones.
## Signing
Two viable routes.
**Route A — `uber-apk-signer` (handy, wraps zipalign + apksigner):**
```bash
java -jar uber-apk-signer.jar --apks in.apk --ks ks.jks \
--ksAlias <alias> --ksPass <pass> --ksKeyPass <pass> -o out_dir
```
Known quirk: `-o` and `--overwrite` are mutually exclusive; passing both errors out. Also, it **silently skips already-signed APKs** (`0 processed`) — so de-sign first.
**Route B — `zipalign` + `apksigner` directly (most explicit):**
```bash
zipalign -p -f 4 unsigned.apk aligned.apk
apksigner sign --ks ks.jks --ks-pass pass:<pass> --key-pass pass:<pass> \
--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true \
--out signed.apk aligned.apk
apksigner verify --print-certs --verbose \
--min-sdk-version 21 --max-sdk-version 34 signed.apk
```
Enable **v1 + v2 + v3**. v1 is needed for older Android; v2/v3 for modern verification.
Generate a keystore once:
```bash
keytool -genkeypair -v -keystore ks.jks -alias <alias> \
-keyalg RSA -keysize 2048 -validity 36500 \
-storepass <pass> -keypass <pass> \
-dname "CN=<name>, OU=dev, O=dev, L=NA, ST=NA, C=NA"
```
## `apksigner verify` picks schemes from the APK's own minSdk
`apksigner verify` decides which signature schemes to check from the APK's own `minSdkVersion`. With `minSdk >= 24` it prints v1/v2 as `false` **by design**, even though `META-INF/*.SF` is present and the signature is valid. That output is indistinguishable from "signing did not apply" and sends you into a re-signing loop over a file that was never wrong.
Always pass the explicit range (Route B above) and read the scheme list from that run only. Never judge a build from a default-argument `apksigner verify`.
## Installing over an existing app
| Situation | Command |
|---|---|
| Same signing key as installed version | `pm install -r` — keeps app data (login state, caches) |
| Different signing key | `pm uninstall` first, then install. **App data is lost** — say so in the delivery notes |
| OEM installer refuses (`INSTALL_FAILED_*`, vendor restrictions, a bare negative code) | push the APK and install via root: `su -c 'pm install -r -d /data/local/tmp/app.apk'` |
| Downgrade needed | add `-d` |
| Test-only flag needed | add `-t` |
Useful detail: keeping the same keystore across builds lets you iterate with `-r` and **preserve a logged-in session**, which matters a lot when the feature you are testing needs auth.
### Vendor install interception (OEM shells)
Some ROMs route `adb install` through their own "security centre", which can return
a bare failure code while the APK itself is fine:
```
Performing Streamed Install
adb.exe: failed to install app.apk: Failure [-99]
```
The device log shows the real actor and reason, and it is not your build:
```
ColorPackageInstallInterceptManager: OPPO_ADB_INSTALL_CANCEL ... packageName=<pkg>
```
**The root path bypasses the interception**, which is why the table above lists it:
```bash
adb -s <serial> push app.apk /data/local/tmp/app.apk
adb -s <serial> shell "su -c 'pm install -r -g -d /data/local/tmp/app.apk'"
```
Recognise the shape: a numeric-only failure with no `INSTALL_FAILED_*` constant, an
APK that installs fine through the device's own UI, and a device process (package
installer / security centre) in the log. Attributing that to the patch is a
classic wasted hour.
### The installer may still own the screen afterwards
After an intercepted or UI-driven install, a vendor **installer confirmation window
can remain the foreground activity indefinitely** — sometimes showing a stale
package name from an unrelated earlier attempt. Consequences:
- `am start` on your app "does nothing", because another window owns the display.
- Screenshots of your supposed launch actually show the installer.
- `am start -W` can block past any reasonable timeout waiting for a first frame
that will never come.
Clear it (`am force-stop <installer pkg>`, or a HOME key event) and re-launch. Before
trusting any capture, check the foreground component:
```bash
adb -s <serial> shell "dumpsys activity activities | grep -m1 ResumedActivity"
```
If it is not your package, the frames are evidence about something else.
`scripts/coldstart.py --expect-activity` performs this check automatically.
## Post-install / post-upgrade hazards
- **Data directory uid mismatch.** After reinstall the app uid increments; a restored `/data/user/0/<pkg>` directory owned by the old uid is unreadable. Symptom: crash in a DB-init path (`Cannot open database`). Fix:
```
su -c "chown -R <uid>:<uid> /data/user/0/<pkg>"
su -c "restorecon -R /data/user/0/<pkg>"
```
Get `<uid>` from `dumpsys package <pkg> | grep userId=`.
- **Restoring a data backup can itself cause this.** Prefer `cp -f` over an existing file (preserves owner) rather than deleting and re-extracting.
## Verify the build, not just the signature
Signature verification proves the file is well-formed. It does **not** prove the app works. Always:
1. `apksigner verify` passes with the explicit SDK range above — a default-args run can report v1/v2 `false` on a perfectly valid signature.
2. Install succeeds.
3. Launch succeeds; process stays alive.
4. `logcat` shows no `FATAL EXCEPTION` / `VerifyError` / `IncompatibleClassChangeError` / `uncaughtException`.
5. `META-INF/services/*` count matches the original.
6. Unchanged dex files are byte-identical to the original (compare hashes); in a de-shelled build, compare against the de-shelled base instead.
Build a **control** at least once: same pipeline, zero patches. If the control fails, your pipeline or environment is at fault, not your patch (`references/pitfalls.md` P9).

194
vendor-new/apk-reverse/skills/apk-reverse/references/routing.md Normal file
View File

@@ -0,0 +1,194 @@
# Routing tables — the on-demand inventory behind `SKILL.md`
This file is the **lookup layer**. `SKILL.md` loads in full when the skill activates, so it carries
only what has to be one hop there: the symptom index (recognising a failure must not become a two-hop
lookup) and the four gates with their pass criteria.
Everything you consult *after* deciding to work lives here, and one load gets all of it:
| You want | Table |
|---|---|
| every reference file, and when to load it | **Reference index** |
| every script this skill ships, and what it does | **Script index** |
| the symptom table as part of a whole-inventory read | **Symptom index** (mirror — the live copy is in `SKILL.md`; edit it there) |
Coherence is enforced: `python check_routing.py` fails when this file and `SKILL.md` disagree, when a
reference file is not named here, or when a script is not named here.
## Symptom index
| What you observe | Load first |
|---|---|
| A repackaged/re-signed build **dies before your code runs**; `SIGSEGV`, all registers zero, `pc=0`, `fault addr` near `0x0` | `native-tamper-and-suicide.md` (deliberate crash), then `code-virtualization-and-custom-linkers.md` |
| **No packer** (Application is the app's own, dex readable) **and it still dies** | `code-virtualization-and-custom-linkers.md` §a loader is still a possibility; but if the same build also dies on a *second, unrelated* device you are looking at an ordinary startup fault, not a hardened one |
| The app dies at startup on **every** device, packed or not, with **no tombstone** while `crash_dump` says `already traced` and logcat says `exited cleanly (0)` | a bundled crash reporter has taken the signal handlers, so the platform's own trail is gone. Frida spawn-gating is the recovery route |
| A `FORTIFY: pthread_mutex_lock called on a destroyed mutex` abort in a Flutter app, on the **main** thread, before the first frame completes | `dart-aot.md` — check `libapp.so` is actually being loaded; Flutter's engine bootstrap is the usual place a native lifecycle fault surfaces |
| Log says a **Java-layer** signature/integrity check **passed**, yet the process dies | `code-virtualization-and-custom-linkers.md` §a Java-layer "signature killer" is a decoy |
| Deleting a library fixes validation but yields `UnsatisfiedLinkError: dlopen failed: library "X" not found` | `code-virtualization-and-custom-linkers.md` §the deadlock that eats hours |
| Whole classes appear as bare `native` declarations with no body | `java2c-and-jni-sinking.md` — read it **before** dumping memory: if this is Java2C there is no DEX to find, at any point in the process lifetime. A handful of `native` methods in an otherwise ordinary dex is JNI sinking, not this |
| A `Java_*` search over a hardened library returns nothing at all | `java2c-and-jni-sinking.md` §The JNI boundary — why a symbol search fails silently — dynamic registration, or `-fvisibility=hidden`. The check that works is "exports `JNI_OnLoad` and zero `Java_*`" |
| You are about to publish an evidence file, a transcript or a README that quotes real work | `references/desensitization-and-leak-scans.md` — run `scripts/scan_leaks.py` **before** it is committed; the hit list is a set of lines to look at, and `--show-exempt` is where the wrong suppressions show |
| A hooking module appears to have run but its log tag is silent, and you are about to record "it never loaded" | `references/precedents/logd-broken-module-never-ran-case-3.md` — a broken `logd` delivers nothing on `logcat` while the module's whole run sits in LSPosed's file log; read both channels |
| A library's **SONAME does not match its filename** | `code-virtualization-and-custom-linkers.md`, `native-and-so.md` |
| Your edit had **no effect at all**, with no error | `server-config-and-updates.md` §3 (the value may be server-sent), then `packers.md` §map the validation boundary |
| Process **hangs** with no crash record, or dies to a `uid 0` killer | `native-tamper-and-suicide.md` §the rule (you probably made a terminate path *not return*) |
| Death looks like an ordinary null dereference in a hardened library | `native-tamper-and-suicide.md` §deliberate-crash stubs |
| The app dies **only while you are attached/rooted** | `detection-and-anti-analysis.md`; run the unmodified original under identical conditions first |
| **Install fails with `[-124]` and mentions `resources.arsc` / alignment** | `repack-and-sign.md` §2a — STORED **and** 4-byte aligned, both required |
| **Install fails with a bare numeric code (e.g. `[-99]`) and no `INSTALL_FAILED_*`** | `repack-and-sign.md` §vendor install interception — a device-side interceptor, not your build. Use the root `pm install` path |
| **After an install, `am start` does nothing / screenshots show another app / `am start -W` hangs** | `repack-and-sign.md` §the installer may still own the screen |
| Log shows `Failure to verify dex file ...: Bad checksum` and a startup `ClassNotFoundException` for an ordinary class | `byte-level-patching.md` §the dex header has two integrity fields — order matters |
| An install "succeeded" but nothing changed, or the version did not move | `long-task-discipline.md` §keep the observation window clean |
| Evidence contradicts itself, or a capture looks like two states mixed | `long-task-discipline.md` §keep the observation window clean |
| You took screenshots but drew the conclusion from logs or from the patch itself | `long-task-discipline.md` §captures you never looked at are not evidence |
| You are about to re-run an experiment whose result you already recorded | `long-task-discipline.md` §long-context decay |
| A script will not start, or a tool "is missing" | `scripts/doctor.py`, then `toolchain.md` §"not on PATH" is not "not installed" |
| A hook or probe reports **no events at all**, and you are about to call it detection | `scripts/anti_detect_probe.js` for the environment self-report first, then `detection-and-anti-analysis.md` §Step 3: locating the check — the order of search from Stage 0 |
| `attach` hangs and then fails **while the process is still in `ps`** | `detection-and-anti-analysis.md` §Step 3 Stage 0 — check for `D` in `/proc/<pid>/stat`, and attach a *different* pid as a one-line control before blaming the target |
| The app exits with no tombstone, no crash and no ANR record | `detection-and-anti-analysis.md` §Step 3 — a clean self-exit means the check ran before your hooks existed; the branch conditions there say which Stage |
| A dump region validates as the wrong thing, or an `r--s` view of `base.apk` looks like a dex | `advanced-unpacking.md` §What this route cannot do, and how to tell before you spend the window |
| Feature-scoped network failure (login/register/pay) while the rest works | `tls-and-cert.md` — do not assume your patch caused it |
| Everything works but **every signed request fails** after repack | `signature-derived-keys.md` |
| A re-signed build **runs fine, renders its whole UI and logs no error — but one feature silently never loads**, and `dumpsys`/DNS/logcat show **no request for it at all** (not a rejected request: *no request*) | `code-virtualization-and-custom-linkers.md` §what the native check actually reads — refusing **before** the request is built. Not the row above: "sent and rejected" and "never sent" have different owners |
| You cannot tell whether a missing feature is **your patch's fault or the target's own behaviour** | `long-task-discipline.md` §single-variable discipline. Run the **zero-change control through the same pipeline**, and the decisive variant: the unmodified original with the patch applied **in memory only**, same device, same network |
| Under Frida `spawn`, the UI never appears — `mCurrentFocus` stays `null`, screenshots come back blank, the Activity stack never builds | `dynamic-frida.md` §spawn keeps the Activity stack down: write the patch into memory, **detach**, then start the Activity normally |
| `frida-server` keeps disappearing mid-experiment, or the device reboots itself while you are working | `dynamic-frida.md` §when the ROM hunts your instrumentation |
| Ads still appear after a patch that should have killed them | `server-config-and-updates.md` §6 (cached config / remote re-enable), then `ad-removal.md` §step 4 (count the SDK's own log lines; n -> 0, not "I did not see it") |
| A forced-update or "must update" gate blocks the build | `updates-and-forced-upgrade.md` §step 6 |
| The dialog is gone but the feature is still locked | `membership-and-limits.md` / `account-gates.md` — decide server vs client authority before patching again |
| You are about to discard a route as "blocked" | `packers.md` — re-read it before writing any route off; mis-attributed failures have removed viable routes for hours |
| The task has run long and you are unsure what is already proven | `long-task-discipline.md` §keep a live record |
| A dumped dex parses in full, the classes are all there, and most method bodies are `return-void` stubs or nop fills | `references/advanced-unpacking.md` — an extraction shell: measure the `stub%` with `scripts/dex_dump_validate.py` before trusting any of it, and know that recovering the bodies is a different route |
| Your `frida` dump dies mid-write (`script has been destroyed`), or the process you are dumping keeps changing pid | `advanced-unpacking.md` §dumping when frida is refused — rule out memory pressure first; a reclaim-and-relaunch needs no instrumentation |
| A repack is refused by several independent checks, or the build has to keep working through store updates | `references/lsposed-and-modules.md` — deliver a module instead of an APK; G1's form table says when |
| A hook module is installed, enabled and scoped, yet its log tag never appears — and you are about to conclude it never ran | `lsposed-and-modules.md` §Deploy, enable, and verify — **a `logcat`-only verdict has already been wrong here**: on one ROM `logd` is broken and output reaches only `/data/adb/lspd/log/modules_<ts>.log` |
| A module's entry class is missing from its own dex (so it can never load), yet the package installs, enables and looks healthy | `references/lsposed-and-modules.md` — check `assets/xposed_init` against the dex's actual classes; installation is not evidence of anything |
| You only need to **call** the target's own routine (sign, token, encrypt) rather than change the app | `references/emulation-and-rpc.md` — emulate it, or service-ify the live function over Frida RPC |
| A native function is a many-thousand-line `switch` state machine, or the decompiler's output is meaningless | `references/native-dbi-and-deobfuscation.md` — OLLVM shapes, a Stalker trace, and how far a trace actually gets you |
| `Stalker.follow` installs but no events arrive, or following a hot libc export crashes the process | `references/native-dbi-and-deobfuscation.md` §6 failure modes — this repository measured both |
| The traffic is protobuf/gRPC/QUIC, or a proxy sees TLS but requests still fail on a Flutter app | `references/protocol-reverse.md` — schema-less protobuf, frame capture, and native-side pinning |
| Userspace hooks land and the app still dies: the check reads `/proc/self/status` through a raw `svc`, or runs before `JNI_OnLoad` | `references/kernel-and-environment-hardening.md` — what the next layer up and down can actually do, and when to stop |
| You must edit, repack, sign or inspect the APK **from the phone itself** | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
| A captured body decodes to nothing readable, or you cannot tell whether a length-delimited field is a string, a nested message or a packed array | `protocol-reverse.md`. Protobuf on the wire (measured) — run `scripts/protobuf_decode_raw.py`; the candidate list and its `tie:` lines are the answer |
| Method bodies are present but decode as **private opcodes**, and you need the mapping rather than an explanation of why VMP is hard | `vmp-differential-analysis.md`, then `advanced-unpacking.md` for the shape diagnosis |
| A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK |
_54 row(s) below the header._
## Reference index (no longer inline in `SKILL.md`)
| File | Load when |
|---|---|
| `references/recon.md` | Starting any new sample; identifying packer, SDKs, code location, ABI |
| `references/server-config-and-updates.md` | **The launch screen, a popup or the tab set is server-sent**; no ad SDK was found; a removed promo came back; anything controlled by a `*Config`/`*Popup` DTO with an `enabled` flag |
| `references/byte-level-patching.md` | You want to change behaviour by editing a few bytes rather than rebuilding a method — equal-length patches, locating an instruction's exact offset, dex header integrity fields, branch polarity, verifier legality |
| `references/packers.md` | The app is packed/hardened, or an edit makes it die before your code runs. Also load before discarding any route as "blocked by the shell" |
| `references/code-virtualization-and-custom-linkers.md` | **No packer, dex is readable, and a re-signed build still dies** — whole classes turned `native`, a private loader with a mismatched SONAME, a self-decrypting payload, a Java-layer "signature killer", and the keep-it/drop-it deadlock |
| `references/java2c-and-jni-sinking.md` | **Whole classes are bare `native` declarations and you are about to hunt for a decrypted DEX** — Java2C has none, ever; the code is in a `.so`. Also the JNI boundary: why a `Java_*` symbol search fails silently |
| `references/vmp-differential-analysis.md` | Method bodies are present but decode as **private opcodes** — a real Dex VMP: the known-plaintext differential, which links can and cannot be automated, how to *prove* a derived opcode table, smali generation, and when the route is closed |
| `references/framework-runtimes.md` | The UI is not native (Flutter / React Native / Unity / Cordova), or Java-layer hooks fire zero times while the UI clearly works |
| `references/dart-aot.md` | The logic lives in a Dart AOT snapshot (`libapp.so`): pinning the Dart version, building a matching decompiler, the object pool and reference indexing, register/boolean conventions, locating and patching Dart code |
| `references/native-and-so.md` | Patching in a `.so`, needing code to run before the app's own code, hand-built native payloads that crash inside the linker, or **deciding which library/ABI is actually loaded and executing** |
| `references/native-tamper-and-suicide.md` | The process dies on its own (no Java stack, or a native crash that looks like a bug); you are about to neutralise a `kill`/`exit`/`abort` path; or a hardened library's sections/function boundaries look wrong |
| `references/detection-and-anti-analysis.md` | The app fights back: it dies after you attach, refuses to run, detects root/hook/debugger. **Cost-first (A/B/C), plus the order of search** (see its Step 3 stage funnel) and when to go static |
| `references/toolchain.md` | Choosing or invoking tools, something is not installed (including "not on PATH but present on disk"), a tool's output smells wrong, or you need to know which tools exist only as a GUI |
| `references/ad-removal.md` | Task involves ads, trackers, sponsored cards, splash/interstitial/reward |
| `references/updates-and-forced-upgrade.md` | The patched build must **keep working over time**; the app has any version check, forced-upgrade dialog, self-update installer, or hot-update/resource channel. Load this for essentially every build you intend to ship. |
| `references/account-gates.md` | Task mentions "no login required", "don't force sign-in", "skip phone binding", "guest mode"; or a screen/feature is unreachable signed-out. Also load before promising that an account-scoped screen will show anything |
| `references/membership-and-limits.md` | Task involves VIP, subscription, paid content, unlock, "fully cracked" |
| `references/server-api.md` | The behavior is decided by a response, or you need to know if a patch can even matter |
| `references/dex-patching.md` | Any actual editing of dex/smali, choosing a patch layer, choosing a tool |
| `references/patch-audit.md` | Proving a patch **landed**, or that it is **legal**: length-vs-bytes comparison, the equal-length blind spot, verifier-level legality (`move-result*` adjacency), text-matching patch traps, and how to report a missing patch |
| `references/repack-and-sign.md` | Rebuilding, signing, installing, or a repacked app misbehaves |
| `references/signature-derived-keys.md` | The app reads `signatures[0]`/`toCharsString()`, or a rebuilt APK installs and runs but every signed request fails (`sign`/`_p`/`uth` empty or `-1`) |
| `references/runtime-data.md` | Local state matters: DataStore, SharedPreferences, SQLite, protobuf caches, tokens — **or your data edit keeps being reverted, or a stored value looks encrypted** |
| `references/dynamic-frida.md` | Frida setup, hooking strategy, tracing caller chains, finding the real call site |
| `references/environment.md` | Device/emulator setup, root, ADB, networking, offline devices, emulator console control and recovery, **the preflight check to run before every experiment block** |
| `references/verification.md` | Defining what "done" means; building the evidence chain |
| `references/tls-and-cert.md` | One feature fails at runtime (login, registration, payment, an API-backed screen) while the rest of the app works |
| `references/third-party-builds.md` | The input is a "cracked"/"modded" build you did not produce — audit it before trusting it |
| `references/long-task-discipline.md` | The task will run long, or you are resuming one. Live record, conclusion grading, drift checkpoints, **deliverable-form drift (rooted-only vs shippable)**, bound-your-waits, **captures-you-never-looked-at**, **long-context decay**, handover |
| `references/pitfalls.md` | Always worth a skim before building. This is the failure catalogue. |
| `references/rasc-and-droidsaw.md` | You are about to reach for the ASC indexer, or the Python one is the bottleneck: the Rust re-implementation, its measured speedup and **identical class sets**, the one class shape where it silently drops code, and how to build it |
| `references/coverage-and-limits.md` | You need to weigh a claim before trusting it: the evidence behind each covered item, the dependencies this skill does not ship, and the record of what was never exercised |
| `references/coverage/route-inventory.md` | You must answer "**can this skill actually do X**" in one pass: the per-route inventory — covered by verified mechanism, covered only by a documented route, the dependencies not shipped, and what is out of scope |
| `references/evidence-summary.md` | You need to know **whether a route was verified and how strong the evidence is**, from inside an installed copy: capability → one-line conclusion → strength → where the evidence lives |
| `references/routing.md` | You want the **whole inventory in one load**: every reference file with when to load it, every script with what it does, and a mirror of the symptom index |
| `references/desensitization-and-leak-scans.md` | You are about to publish anything derived from real work — evidence, a transcript, a README — or a leak scan reports a hit: what must be desensitised versus kept, the do-not-anonymize list, and how the scan gates a commit |
| `references/precedents/` | You are about to do a kind of work this repository has already converged on — a hardened Flutter target, a zero-event trace, a module whose log is silent, an equal-length patch taken to the device. Positive cases with graded evidence and dead ends |
| `references/handoff-boundaries.md` | A routing decision is about to cross into another discipline: the JNI form table, the packer-versus-loader split, and what "verified" means for each of G1's four deliverable forms |
| `references/advanced-unpacking.md` | **The dump landed but the bodies are empty** (an extraction shell), or decode as private opcodes: the stub-ratio measurement, why FART's hooks died on Android 12-16, the root-side dump, its **boundary**, the layered descent, and where recovery stops |
| `references/lsposed-and-modules.md` | The client-side logic is reachable but **a rebuilt APK is refused**: delivering a system-level hook module instead, its gradle-free build chain, scope configuration and verification, and the layer a Java module cannot reach |
| `references/emulation-and-rpc.md` | You need to **call** a routine rather than change the app — a signing routine, a token, a cipher: emulated execution (Unidbg/Unicorn) with its environment-filling cost, versus service-ifying the live function over Frida RPC |
| `references/native-dbi-and-deobfuscation.md` | A native function is an OLLVM state machine, or you need instruction-level execution evidence: Stalker traces, the trace-to-CFG route, the Stalker/QBDI/emulation decision, and the **measured** zero-event and crash boundaries |
| `references/protocol-reverse.md` | The traffic is protobuf without a schema, gRPC, or QUIC/HTTP3; or a proxy sees TLS while the app still fails — schema recovery, frame capture, and native-side certificate pinning (Flutter/BoringSSL) with its boundaries |
| `references/kernel-and-environment-hardening.md` | Userspace hooking provably cannot reach the check — raw `svc` syscalls, `init_array`-early detection: what each layer can still do, the kernel-route map with its version gate, and when escalating is wrong |
| `references/on-device-tooling.md` | Working **from the phone itself**: MT Manager edit/repack/sign and its built-in APK MCP, LSPosed Manager, Termux+frida, and on-device data inspection |
| `references/split-apk.md` | The target is a **split APK / App Bundle set** (`base.apk` + `split_config.*.apk`), or `pm path <PKG>` returned several files: reading a set, merge versus unified re-signing, and the install refusal each mistake produces |
_43 row(s) below the header._
## Script index (no longer inline in `SKILL.md`)
| Script | Purpose |
|---|---|
| `scripts/doctor.py` | **Run this first.** Script inventory is scanned from the directory (`registered/checked = N / N`, unjudgeable entries counted as `unknown`, never skipped) and capability is resolved as a real requires-closure — so it reports BLOCKED with the missing piece and an installable next step instead of inferring "can re-sign" from the presence of a JVM |
| `scripts/dexutil.py` | **Dependency-free dex reader**: structural walk, exact instruction decode, `fix_dex_header`/`verify_dex_header` in the correct order, branch/operand helpers. Shared by the dex scripts; also dumps one method with offsets standalone |
| `scripts/dex_find_insn.py` | Locate an instruction by **decoded semantics** and get its exact byte offset, with context and both sides of any branch. This is how you find a patch site without guessing offsets or scraping listings |
| `scripts/dex_patch_bytes.py` | Apply **equal-length byte patches** from a JSON spec: semantic match, polarity pin via `expect_next`, equal-length enforcement, verifier legality, dex header recompute, re-decode to prove the edit landed. `--dry-run` first |
| `scripts/dex_check_verifier.py` | Tier-3 check: does any **conditional branch target a `move-result*`** (bypassing its producer, so the class fails to load)? Compares two builds and distinguishes pre-existing findings from regressions your patch introduced |
| `scripts/coldstart.py` | Cold-launch and capture a **timed screenshot burst + logcat signals + installed-build facts + launch timing**, and warn when the foreground activity is not your app |
| `scripts/smtool.py` | baksmali/smali wrapper with a bundled classpath (assemble/disassemble dex) |
| `scripts/patch_smali.py` | Method-body replacement in a smali tree, matched by signature |
| `scripts/dex_strpatch.py` | Byte-level string constant patch with **string_ids ordering guard** |
| `scripts/dex_classdiff.py` | Compare two dex class tables (set + access flags) to prove a patch was surgical |
| `scripts/dex_strings.py` | Dump/extract strings and endpoints from dex without a decompiler |
| `scripts/dart_pprefs.py` | Build/query the object-pool offset -> code-site index for a Dart AOT snapshot (arithmetic decode; seconds, not minutes) |
| `scripts/dart_pool_strings.py` | Recover string literals from a Dart AOT snapshot: framed entries, the one-byte vs UTF-16 split, file offsets, and a run-length noise filter |
| `scripts/dart_disasm.py` | Annotated windowed disassembly of Dart AOT code (pool + boolean annotations) plus a B/BL caller index |
| `scripts/find_refs.py` | Count and list callers of a method/field (blast-radius check). Takes a smali tree, a `.dex`, a directory of either, or an `.apk`. Always prints what it scanned, so "the input was unreadable" cannot be mistaken for "nothing references it" |
| `scripts/repack.py` | Rebuild an APK: replace dex, strip only signatures, keep `META-INF/services/`, write a **4-byte-aligned** archive, sign, verify. Also **split sets**: `--split-dir`, `--split-mode resign`/`merge`, `--signer auto/jar/apksigner` |
| `scripts/dexpatch/` | dexlib2 method-level rewriter (for changes that genuinely need new instructions) + build notes |
| `scripts/devsh.py` | Quoting-safe ADB shell helper for rooted devices |
| `scripts/usb_net_proxy.py` | Give an offline device network over USB (adb reverse + local proxy) |
| `scripts/datastore_inject.py` | Encode/inject AndroidX DataStore preferences (protobuf) safely |
| `scripts/probe_api.py` | Probe an app's HTTP API with correct headers, report status/shape |
| `scripts/install_test.py` | Install a build and run a launch/health check with logcat signal extraction |
| `scripts/frida_probe.js` | Four-layer runtime probe: app network layer + OkHttp + java.net + swallowed exception messages |
| `scripts/run_probe.py` | Inject a probe, stream it to a timestamped log file, stay resident while you operate the app |
| `scripts/tls_check.py` | Strict certificate check for one or more hosts (expired / wrong host / untrusted CA) |
| `scripts/preflight.py` | Read-only environment check before every experiment block: device, root, ABI/translation, clock skew, leftover proxy/forwards, dead device server. Run this before blaming a patch. |
| `scripts/lib_map.py` | What is **actually mapped** into a live process: per-library path, base, architecture (`ELF e_machine`), and classification (system / from-APK / runtime-materialized). Answers "which library and which ABI is really executing". |
| `scripts/elf_plt.py` | Resolve a PLT stub to its imported symbol (x86_64 + aarch64) from the **relocation table**, list a symbol's callers, and **byte-diff two libraries naming the symbol each changed stub belongs to**. Run before patching any stub |
| `scripts/so_constpatch.py` | Same-length rewrite of an isolated string constant, for **redirecting a library load instead of defeating a check**. Enforces equal length, refuses substrings of longer identifiers, reports constant-pool hits, patches inside an APK or a bare `.so`. **APK rebuild preserves per-entry zip metadata** (STORE stays STORE, `extra`/`external_attr`/`create_system` untouched, `resources.arsc` STORED and 4-byte aligned) and **refuses by default** when `extractNativeLibs="false"` or the arsc is not STORED+aligned — pointing at `repack.py`; only `--unsafe-rebuild` forces it |
| `scripts/apk_diff.py` | Entry-level diff of two APKs by content hash: changed / **added** (injection candidates) / removed. Audits a third-party build and proves your own was surgical |
| `scripts/native_crash.py` | Locate a native death from a log or tombstone: signal, fault address, registers, frames split app vs system, the faulting instruction — plus a flag when the fault looks **arranged** rather than accidental |
| `scripts/grab_crash.py` | Recover a stack that a crash-reporter SDK swallowed, when the log shows the app died but prints no backtrace of its own. |
| `scripts/blob_decode.py` | Decode an opaque stored value by searching the parameter space (base64/base64url/hex × rotation × deflate/zlib/gzip) instead of guessing, then re-encode an edited payload with the same parameters. |
| `scripts/snap.py` | Bounded burst screenshot + control-tree capture, with a stall detector and an explicit verdict on whether the accessibility tree is usable at all. Use it so you *look* at the screen instead of driving blind. |
| `scripts/sig_probe.py` | Find the exact `signatures[0].toCharsString()` value: offline candidates from an APK (`--apk`), or the authoritative read from a live package (`--live`) |
| `scripts/spawn_patch_detach.py` | **Spawn under a probe, detach, then drive the UI.** Under spawn the Activity stack often never comes up; memory writes survive detach while hooks do not. Ordering matters: resume *before* waiting for `PATCHED` |
| `scripts/hook_patch_only.js` | The minimal probe for `spawn_patch_detach.py`: neutralise one native death site by offset and report `PATCHED`. `MODULE_NAME`, `FILE_OFFSET`, `PATCH_BYTES`; the replacement must be an equal-length "return" epilogue |
| `scripts/dex_dump_validate.py` | Dedupe, validate and rank dumped dex images: sha256 grouping, header integrity, extraction-shell discrimination via the **trivial-body ratio** (bimodal, not a threshold — `advanced-unpacking.md`), ranking, `--json`, `--trim` for page-aligned captures |
| `scripts/dex_mem_scan.py` | Find embedded dex images in memory captures and extract each at the size **its own header declares**: chunked scanning, magic validation, sha256 dedupe, `--dump DIR`, `--keep-partial` |
| `scripts/lsposed_scaffold.py` | Generate a minimal LSPosed/Xposed **module project skeleton**: manifest with the xposed meta-data, `assets/xposed_init`, the hook class, and build notes for a gradle-free toolchain (javac → d8 → aapt2 → zipalign → apksigner) |
| `scripts/frida_rpc_serve.py` | Bridge a Frida script's `rpc.exports` to a local caller with reconnect handling, so a live native function can be **called** instead of reversed |
| `scripts/rpc_template.js` | The editable companion to `frida_rpc_serve.py`: an `rpc.exports` skeleton plus a native-function call placeholder |
| `scripts/stalker_trace.js` | Instruction-level tracing with Frida Stalker: configurable module/offset targets, trigger selection, the event stream, output-size rules, and `transform` customisation |
| `scripts/stalker_report.py` | Reduce a `stalker_trace.js` log into block histograms and call sequences, and print an explicit diagnostic for the measured **zero-event** case |
| `scripts/mt_mcp_probe.py` | Probe MT Manager's on-device APK MCP (Streamable HTTP, `127.0.0.1:8787/mcp`): JSON-RPC handshake plus grouped `mt_apk_*` tool inventory; prints start-it-by-hand instructions and exits 2 while the service is down |
| `scripts/java2c_probe.py` | The evidence that separates **Java2C** from an extraction shell, a VMP and JNI sinking: per-class `native` density and stub ratio from the dex, plus `Java_*` / `JNI_OnLoad` / registration evidence from the `.so` |
| `scripts/protobuf_decode_raw.py` | **Schema-free protobuf decode**: hex / file / stdin to a JSON tree, every length-delimited field as a candidate set with ties labelled; `--reencode --check` for a byte-exact round trip |
| `scripts/vmp_diff_harness.py` | Differential hardening for Dex-VMP: build a labelled opcode-coverage fixture (**218 of 224 opcodes, measured**), derive a candidate private-opcode map from an original/hardened dex pair, verify it in a closed loop, render smali |
| `scripts/kernelsu_syscall_mask.py` | Generate a KernelSU/APatch syscall-masking scaffold: an installable userspace module plus KPM/LKM/eBPF kernel-side templates, each with its version gate and an explicit unverified label. **A userspace module cannot change a syscall return value** |
| `scripts/capabilities.py` | The capability registry `doctor.py` reads: per capability the requires-closure, what is present, what is missing, an install hint and an estimated cost — so "can this machine do X" is computed rather than guessed |
| `scripts/device_shell.py` | Shared device-side command construction: POSIX single-quote escaping for `su -c`, argv arrays rather than string concatenation, and Android identifier validation (`--pkg`, components) so a malformed value is rejected here instead of being parsed by the device shell |
| `scripts/rasc_build.py` | Build and verify **rasc**, the Rust re-implementation of ASC: `--check` what is present, `--build` clone + cargo (no release asset exists), `--verify <apk>` compares its class-definition set against the Python `droidasc` and fails on any difference |
| `scripts/scan_leaks.py` | Scan a repository for target identity before it is published: bundle ids in manifest/`pm`/`ps` contexts, serial-shaped tokens, PATs, inline appkey assignments, literal endpoints, host user paths. Built-in do-not-anonymize exemptions, context-bearing findings, `--show-exempt` prints why a hit was suppressed, `--fail-on strong\|any`, exit 0/1/2 with a `RESULT=` token |
| `scripts/svc_scan.py` | Name the syscall behind an inline `svc` and the segment it sits in — which decides whether a libc-level hook can observe the call at all. `--context` shows neighbours, because a byte scan also matches data |
| `scripts/anti_detect_probe.js` | Observer-only probe (patches nothing): path/loader/thread/kill hooks with **caller module + offset**, an environment self-report (`TracerPid`, frida-named mappings), and live streaming so a sub-second target still yields evidence |
_53 row(s) below the header._

255
vendor-new/apk-reverse/skills/apk-reverse/references/runtime-data.md Normal file
View File

@@ -0,0 +1,255 @@
# Runtime data — DataStore, SharedPreferences, SQLite, protobuf
Sometimes the cheapest fix is not code at all, but a value in app data. And sometimes that is exactly the wrong fix, because it does not survive a fresh install (`pitfalls.md` P11).
**Load this when:** the cheapest fix looks like a stored value rather than code -- DataStore, SharedPreferences, SQLite, a protobuf cache, a token. It gives how to edit each safely, and how to tell a value the app rewrites from one it keeps.
## Decide first: data or code?
| Question | Answer |
|---|---|
| Does the deliverable need to work after a clean install? | Then it must be **code**, not data |
| Are you iterating on your own device? | Data edits are a fast way to test a hypothesis **before** writing the patch |
| Is the value written by the app itself (a token, a cache, a counter)? | Data — and the server may re-assert it |
**Recommended workflow:** use a data edit to *prove the hypothesis* (cheap), then implement the same effect in code (durable), then verify with a clean install.
## Where app data lives
```
/data/data/<pkg>/ (== /data/user/0/<pkg>)
├── files/
│ ├── datastore/*.preferences_pb AndroidX DataStore (protobuf)
│ ├── mmkv/<name> MMKV (Tencent) — one file per named store
│ └── ...
├── shared_prefs/*.xml SharedPreferences
├── databases/*.db SQLite (Room, etc.)
└── <sdk caches>/ crash logs, download state, ad configs
```
Read with root:
```bash
adb shell "su -c 'ls -la /data/data/<pkg>/files/datastore/'"
adb shell "su -c 'xxd /data/data/<pkg>/files/datastore/<name>.preferences_pb | head -20'"
```
## AndroidX DataStore (protobuf)
Format:
```proto
PreferenceMap { map<string, Value> preferences = 1; }
Value {
oneof value {
bool boolean = 1; float float = 2; int32 integer = 3; int64 long = 4;
string string = 5; StringSet string_set = 6; double double = 7; bytes bytes = 8;
}
}
```
**Encoding an entry requires TWO tag levels:**
```
outer (map field 1, wire type 2) : 0A <len(inner)>
inner (one map entry) : 0A <len(key)> <key> 12 <len(value)> <value>
value payload (e.g. int64) : 20 <varint> # field 4, wire type 0
value payload (e.g. int32) : 18 <varint> # field 3, wire type 0
value payload (e.g. bool) : 08 <0x00|0x01> # field 1, wire type 0
value payload (e.g. string) : 2A <len> <utf8> # field 5, wire type 2
```
Omitting the **outer** tag produces a file the app cannot deserialize, and the failure is usually a **stack-less crash** (`pitfalls.md` P8). Use `scripts/datastore_inject.py` rather than hand-rolling it.
Procedure:
1. `am force-stop <pkg>` (DataStore caches in memory and writes back)
2. Write the file (as root), preserving ownership — `cp -f` over the existing file keeps its owner
3. `restorecon` if SELinux is enforcing
4. Start the app and observe
Verify by reading the file back with `xxd` before launching.
## SharedPreferences (XML)
Simple XML. Edit with the app stopped. Types are explicit (`<boolean>`, `<int>`, `<long>`, `<string>`). Same ownership rules.
## MMKV (Tencent)
Common in apps whose SDK stack is Chinese-market. One file per named store under `files/mmkv/`,
with a `.crc` companion. It is **not** a database and **not** XML — it is a small custom binary
format:
```
offset 0 uint32 actual_size size of the data region
offset 4 uint32 crc32 over the data region
offset 8 data region repeated: varint key_len, key, varint val_len, value
```
Files are preallocated, so trailing zeroes are normal; `actual_size` is what matters.
Parsing is mechanical, and no library is needed:
```python
import struct
def parse(blob):
size = struct.unpack_from('<I', blob, 0)[0]
pos, end, out = 8, 8 + size, []
def varint(p):
v = s = 0
while True:
b = blob[p]; p += 1
v |= (b & 0x7F) << s
if not b & 0x80:
return v, p
s += 7
while pos < end:
klen, pos = varint(pos); key = blob[pos:pos + klen]; pos += klen
vlen, pos = varint(pos); val = blob[pos:pos + vlen]; pos += vlen
out.append((key, val))
return out
```
Writing a change back means rebuilding the region and **recomputing both header fields**:
```python
import struct, zlib
body = new_region_bytes
blob = bytearray(capacity) # keep the original file length
struct.pack_into('<I', blob, 0, len(body))
struct.pack_into('<I', blob, 4, zlib.crc32(body) & 0xFFFFFFFF)
blob[8:8 + len(body)] = body
```
Cautions:
- **A wrong crc32 makes the library throw the whole store away.** The visible symptom is "my
edit did nothing" or "the app reset every setting", never an error.
- **Values are raw bytes**, not always text. JSON is common; a serialized protobuf or a small
encrypted payload is also common (`scripts/blob_decode.py` searches the framing).
- **The in-memory copy wins until the process restarts.** `am force-stop` before editing and
re-launch after, or the running app overwrites your change on exit.
- **Do not delete the `.crc` companion.** Leave it in place when hand-editing the store; it is
validated and regenerated by the library.
## SQLite
```bash
adb shell "su -c 'sqlite3 /data/data/<pkg>/databases/<db>.db \".tables\"'"
adb shell "su -c 'sqlite3 ... \"select * from <table> limit 5;\"'"
```
Useful for: auth sessions (tokens), user profile caches (often a raw JSON blob), and any server state the app persists. A JSON column often contains the exact server payload — the fastest way to learn field names.
## Ownership — the silent killer
The app runs as its own uid. A file written by root with the wrong owner is unreadable, and directory permissions are `drwx------`.
```bash
uid=$(adb shell "dumpsys package <pkg> | grep userId=" | tr -dc '0-9')
adb shell "su -c 'chown -R $uid:$uid /data/user/0/<pkg>'"
adb shell "su -c 'restorecon -R /data/user/0/<pkg>'"
```
Symptom of getting this wrong: crash in a database-init path right after launch, often `Cannot open database ... Directory ... doesn't exist`.
Note: after every reinstall the app's uid **increments**, so a restored data directory needs this again.
## Introducing a value that does not exist yet
If the key is absent, the app uses a default. Two options:
1. **Add the key** with the exact protobuf shape above.
2. **Confirm the default first** — sometimes removing a key already yields the behavior you want (e.g. clearing an expiry so it reads as "not set").
## Making the fix durable
If the effect must ship inside an APK, move it into code. Patterns, in order of preference:
- **Patch the read path** so the value is always the desired one. Find where the Flow/getter is built and make it emit a constant.
- **Patch the decision**, not the data: find the comparison that consumes the value and make it resolve the way you want.
- **Do not patch the generic encoder/boxer** used by the whole app (`pitfalls.md` P6).
Reference case: a promo popup was gated by `<key>_expires_at`. Writing a large value via DataStore suppressed it on the test device, but a fresh install lost it. The durable fix replaced the dedicated map-lambda that produces the value so it always yields a far-future timestamp — a single-purpose class, safe to patch, no effect on the shared serialization helpers.
## When the app rewrites your edit
A very common sequence: you change the value, relaunch, and the value is **back** — often byte for
byte identical to what it was before. That is not a failed write. It is the app re-establishing the
value from a source of truth you have not touched yet.
Two causes look identical and need different fixes:
| Cause | How to tell | Fix |
|---|---|---|
| The app **re-fetches from the server** and re-persists | take the app offline and relaunch: does your value survive? | lock the file, block the refresh request, or move the change into code |
| The app **rewrites the file on every start** from its own defaults | your value does not survive even with the network down | the value is not authoritative — patch the read path instead |
Do the offline test first. It costs one launch and separates the two cases.
### Making a data edit stick (in order of durability)
1. **Immutable file attribute** — the reliable, reversible way to stop a rewrite:
```bash
su -c "chattr +i <path/to/prefs.xml>"
su -c "lsattr <path/to/prefs.xml>" # expect the 'i' flag, e.g. -----i-------
# to undo:
su -c "chattr -i <path/to/prefs.xml>"
```
Verify with `lsattr`, not by assuming the command worked.
**Why permissions do not achieve this.** Tightening the mode or changing the owner looks like the
obvious move and does not work, because the app does not open-and-write the existing file — it
**deletes the file and creates a new one**. The new file is created by the app, with the app's own
mode and owner, so a restrictive `chmod` or a `chown root` is simply not inherited. The immutable
attribute is enforced by the filesystem against the delete itself, which is why it holds.
2. **Keep the app offline** so the refresh source is unreachable. Effective for a test, and
sometimes acceptable in use — but be explicit about it, because a fix that only works offline
fails the "works under normal conditions" constraint. See the deliverable-drift section in
`long-task-discipline.md` before presenting this as a result.
3. **Block the refresh request** at the runtime or network layer — durable while your instrumentation
is running, and requires you to have identified the endpoint (`references/server-api.md`).
4. **Move the change into code** — patch the read path or the decision. This is the only form that
ships inside an APK (`§Making the fix durable` above).
### Two cautions about locking
- **Verify the feature, not the value.** Blocking a write the app depends on can make it misbehave
or fail loudly. After locking, exercise the feature you care about — "the file still has my value"
is not the same as "the app still works".
- **A lock is device state; it does not travel with an APK.** If the recipe requires it, record it as
an environment requirement, so nobody later mistakes it for something baked into the artifact.
## Encoded values: do not assume "encrypted"
Preference values that carry server configuration are frequently stored as a single opaque string
rather than readable XML. Before concluding the value is encrypted, check whether it is merely
**framed**: a common shape is `base64( rotate( deflate( json ) ) )`, where the cyclic rotation exists
precisely so a naive decode fails.
`scripts/blob_decode.py` searches the parameter space instead of guessing it — outer encoding
(base64 / base64url / hex), rotation offset, and compression type — and reports every combination
that yields a structured document, plus the exact parameters to re-encode your edited payload:
```bash
# pull the value straight out of a preferences XML and decode it
python scripts/blob_decode.py --prefs-xml prefs.xml --name <key> # (see --help for exact name)
# after editing the decoded payload, rebuild the value with the winning parameters
python scripts/blob_decode.py --encode --file decoded.bin --outer base64 --inner raw --cut <N>
```
The rotation search is cheap — a few tens of thousands of candidates resolve in well under a second
— so there is no reason to guess. **Distinguish framing from real encryption before spending time
on a key:** a framed blob becomes structured the moment you remove the framing, whereas a keyed blob
stays random-looking. If it stays random, treat it as opaque and move to the code path that consumes
it.
## Cautions
- Editing data while the app is running is unreliable: in-memory caches win. Force-stop, edit, then launch.
- The server may overwrite your value on next sync — see *When the app rewrites your edit* above.
- Do not edit a token you do not own and expect it to be accepted; tokens are validated server-side (`references/server-api.md`).
- A restored data directory needs ownership fixed again after every reinstall, because the uid increments.

94
vendor-new/apk-reverse/skills/apk-reverse/references/server-api.md Normal file
View File

@@ -0,0 +1,94 @@
# Server-side analysis — deciding whether a patch can even matter
If the behavior you want to change is produced by a response, patch the client all you like: nothing real changes. Establishing server authority early is the cheapest high-value step in any "crack this" task.
## Harvest the API surface from the client
```bash
python scripts/dex_strings.py <dex_dir> --urls
python scripts/dex_strings.py <dex_dir> --find 'https?://'
```
Collect: base URL(s), path constants, auth headers, and any dynamic-gateway mechanism.
Two patterns to watch for:
- **Static base URL** — straightforward.
- **Dynamic gateway** — the real host is fetched at runtime (commonly a DNS TXT record over DoH, or a small config endpoint). The hardcoded host may be a fallback. Symptom: requests go to a host that differs from anything in the dex.
```bash
# TXT lookup, the usual shape:
# https://<doh-provider>/resolve?name=<domain>&type=txt
# returns something like "https://<gateway-host>"
```
This is an anti-blocking technique; it does not block repacking, but it means your static host guess may be stale.
## Reproduce requests faithfully
```bash
python scripts/probe_api.py --base <base_url> --path /some/path --header 'X-App-Name: <value>'
```
Rules that make the difference between 403 and 200:
- **Send the app's real headers.** A single custom header (e.g. an app-name header) frequently decides whether a WAF lets the request through. The header value is a string constant in the dex.
- **`User-Agent` matters** — often `okhttp/x.y.z` or a runtime-specific default (e.g. `ktor-client`).
- **Watch for a CDN/WAF in front.** A 403 with an HTML body naming a WAF means your request shape is wrong, not that the endpoint is gone. Compare with what the app sends (capture it from the device).
- **Beware your own egress.** If your host goes through a proxy/VPN, a WAF may block by geography while the device succeeds. Do not conclude "the server rejects the app" from a host-side 403.
- **`/health`-style endpoints are gold** — a 200 with a small JSON body proves the backend and its response envelope are alive before you debug anything else.
## Response envelope
Most apps wrap responses uniformly, e.g. `{"code":0,"msg":"","data":{...}}`. Learn it once, then read endpoints by shape rather than guessing:
- `code != 0` + a message → business error, and **the message often enumerates valid inputs** (this is how an ad `position` whitelist was discovered).
- `data:null` with HTTP 200 → soft failure.
- `401` with a null envelope → auth gate.
## Prove who owns the gate
Run this matrix and record it. It is the single most decisive artifact for a paywall or entitlement question.
| Request | Expected if client-owned | Expected if server-owned |
|---|---|---|
| protected endpoint, no credentials | would still succeed | **401 / 403** |
| same, with a forged token | would still succeed | **401 / 403** |
| metadata endpoint | — | 200, often **omitting** the protected field |
| list/config endpoints | — | 200 with data |
If the valuable field is simply **absent** from an otherwise-200 metadata response, no client change can invent it. Say so, with the evidence.
## Also test the *unknown input* case
When you plan to change a request parameter (a path, a `position`, an id), probe what the server does with an **invalid** value first:
- A `400` with a validation message tells you the field is whitelisted, and hands you the full list of valid values.
- A `404` tells you the path is exact.
**This matters before you patch**: redirecting a parameter to an invalid value may produce an error status that the client treats as fatal (`pitfalls.md` P5). Knowing the failure mode in advance prevents breaking the app.
## Behaviors that help you and behaviors that do not
**Repack-friendly (do not block you):**
- Minimal auth: `Authorization: Bearer <token>` and a static app-name header, with **no request signing, no nonce, no HMAC**.
- No client-certificate pinning to the app's own signature.
**Repack-hostile (will break a re-signed client):**
- Signature/certificate checked server-side, or pinned in native code.
- Request signing derived from the APK signature — the client computes with its own
certificate, so re-signing changes the key. Silent and easy to misdiagnose; detection,
extraction and the differential test are in `references/signature-derived-keys.md`.
- Device attestation (Play Integrity / SafetyNet, OEM attestation).
Determine which you face **before** shipping a repack. A client that authenticates fine on first launch but silently fails features later is the classic signature-validation symptom — and when computed signing parameters (`sign`, `_p`, `uth`) evaluate to `-1` or empty, it is the signature-derived-key case rather than a server decision.
## What to record
```
base url / gateway mechanism
required headers (exact values)
auth mechanism (token? signed? attested?)
response envelope shape
endpoints actually used, and their auth requirement
protected-asset endpoints, with status codes for: none / forged / (real if available)
invalid-input behavior for any parameter you plan to change
```

160
vendor-new/apk-reverse/skills/apk-reverse/references/server-config-and-updates.md Normal file
View File

@@ -0,0 +1,160 @@
# Server-driven UI config — promos, popups, tab bars, and remote re-enable
Many apps ship a **remote configuration endpoint** that decides what the client
renders: a launch screen image, a popup, an announcement, which tabs exist. This is
the single most common shape of "ad" in a modern app, and the one most often
mis-diagnosed, because there is no ad SDK anywhere in the package to find.
Two consequences that change the whole plan:
- **Searching for an ad SDK finds nothing, and that is arithmetically correct.** No
amount of further SDK hunting will help. Recognise the shape early and stop.
- **The behaviour can be re-enabled server-side without shipping a new client.**
Any patch that just leaves the config unread is a patch the operator can undo.
The durable fix is in the client's decision path, not in the data.
**Load this when:** a launch screen, popup, announcement or tab set must go, and there is no SDK to find. It gives the server-issued-config shape, how to tell it from a client-side flag, and the remote re-enable that undoes your patch.
## 1. Two-layer fetch: local default, then remote override
The most useful diagnostic — and it is a runtime observation, not a static one — is
that the config object **changes value once during startup**:
```
before the network round-trip: SplashConfig(enabled=true, duration=5, image=<built-in drawable>)
~1s later (config response): SplashConfig(enabled=true, duration=5, image=https://<cdn>/<promo>.png)
```
Seeing this pair proves three things at once: the screen is config-driven, the
built-in asset is only an offline fallback, and the content actually shown came
from the server. **Capture both states before designing any patch** — a
hook on the config type's constructor/factory plus its `toString()` is usually
enough (data-class `toString()` keeps the real field names even after R8, which is
why it is such a convenient anchor).
If you only ever see the built-in default, you are looking at an offline or
pre-config state and have not observed the real behaviour yet.
## 2. Locate the config surface
The DTOs are usually the easiest anchor in the entire app, because data classes
serialise their own field names.
```
ddc findrefs app.apk string "SplashConfig" # or any of the field names below
ddc strings app.apk -f "enabled" --with-locations
```
Field names worth grepping, in the plural because they recur:
```
splash, splashImage, splashSkip, splashLogo, launchScreen, bootAd
noticePopup, announcementPopup, updatePopup, forceUpdate, upgradeDialog
tabbar, tabs, items, enabled, showMode, startAt, endAt, showOnce
promo, promotion, banner, bannerList, featured, sponsored
```
A `*Config` / `*Popup` / `*Banner` suffix with an `enabled` boolean is the signal
you are in this layer. **The `enabled` flag's polarity is not guaranteed by its
name** — see.
Then find the **consumption point**, which is where you patch: the method that
reads the flag and decides. The DTO itself is usually immutable and shared, so
editing it is both fragile and unnecessary.
## 3. Patch the decision, not the data
Ordered from most to least durable:
| Approach | Durable? | Why |
|---|---|---|
| Neutralise the client-side branch that consumes `enabled` | **yes** | The server can send anything; the client no longer acts on it |
| Force the config field to its "off" value at parse time | partly | Works until a second parse path or a cache exists |
| Block or 404 the config endpoint | **no** | Usually breaks the screen that shares the request (see `pitfalls.md` P5) |
| Delete the local fallback asset | no | Only affects the offline case |
**How to neutralise a branch correctly** is in `byte-level-patching.md`: replace the
conditional branch with `nop`s so execution takes the fall-through path, rather
than redirecting the branch. Forcing a specific side by redirection adds a control
flow edge and can trip the verifier.
**Mandatory before editing: decode both sides of the branch and write down what
each one does.** Field names lie. In one real case the field was named `enabled`,
and the branch structure was:
```
iget-boolean v, obj -> Config.enabled
if-nez v, :countdown ; enabled == false jumps AWAY
invoke startMainScreen() ; fall-through: straight into the app
:countdown ... ; branch target: show the promo for N seconds
```
so `enabled == true` was the value that **skipped** the promo. Taking the name at
face value and "enabling the skip" produces a build that shows the ad on every
launch — and it starts cleanly, so nothing catches it until someone looks at the
screen.
## 4. Decide the scope of "remove" before you edit
Serve-side config usually controls several things at once, and they are not all
ads. Classify each field the same way you would a UI element:
| Field shape | Usually is | Usually not |
|---|---|---|
| Launch screen image + countdown with no dismiss | promotional | a functional splash |
| Popup with a channel/group link and `showOnce` | promotional | |
| Popup with changelog + `force` + an external URL | update nag → `updates-and-forced-upgrade.md` | |
| Tab bar item with `enabled=false` | absent by operator choice | |
| Announcement that is off server-side | nothing to remove | |
If the user asked for "no ads", removing a genuine brand splash is a judgement
call, not an automatic win — a self-owned logo screen with no third-party content
is arguably not an ad. State which one you removed; if the request was ambiguous,
the safer default is to remove only the server-sourced promotional content and
leave a purely local brand screen alone. See `long-task-discipline.md` on not
substituting your own goal for the stated one.
## 5. Verify at three levels, not one
A config-driven behaviour needs more evidence than a log line:
1. **Screen, before and after, frame by frame** — `scripts/coldstart.py`. The
observable is a visible change plus a change in time-to-first-meaningful-screen.
2. **The config object's own values at runtime** — if the app logs or you can hook
it, show the value it received. This separates "the server stopped sending it"
from "the client stopped obeying it", which are different claims.
3. **Correct negative**: if the flag arrives `enabled=true` and nothing happens,
that is the strong result. If it arrives `enabled=false`, you have verified
nothing about your patch.
## 6. Remote re-enable: the durability question
Two channels can undo client-side work without a version bump:
- **The config endpoint itself.** Your patch must be in the decision path so a
future `enabled=true` is still ignored (see). A patch that reads the flag and
happens to work today is not durable.
- **A hot-update / dynamic-resource channel** (a downloaded bundle, a patch dex, a
remotely loaded layout). Look for a cache directory that is not the config cache,
or a download-then-load path during startup. If one exists, the patch has to
survive it — which usually means the patched decision is the right place anyway,
because remote resources rarely replace compiled logic.
Also check whether the config is **cached to disk** (`SharedPreferences`, a JSON
snapshot, a DataStore file). A cached copy means the app can exhibit the old
behaviour offline for one launch after your change, which looks like a failed
patch. Clear the cache as part of the experiment, and say so when reporting.
## 7. Reporting template
```
Promo/popup surface: <field> (<DTO class>) | source: server config at <endpoint or unknown>
Observed before: <exact values, incl. whether enabled was true/false>
Observed after: <exact values / absence>
Patch: <class#method, byte offset, bytes before -> after>
Durability: <why a future server-side change cannot restore it>
Not removed: <fields deliberately left, with reason>
Side effects: <what shares the same config or code path>
Unverified: <anything not directly observed>
```

149
vendor-new/apk-reverse/skills/apk-reverse/references/signature-derived-keys.md Normal file
View File

@@ -0,0 +1,149 @@
# Signature-derived keys — when re-signing silently breaks the app
A class of app that repack-friendly guidance does not cover: the client feeds **its own APK
signing certificate** into a native routine and uses the result as the key for request
signing or for SDK payload encryption. Re-sign the APK and the key changes, so every signed
request fails — while the app itself launches perfectly.
This is distinct from "the server checks the signature". Here the *client* uses the signature
as key material, and the server was built against the original key. The failure looks like a
server problem and is actually a signing problem.
**Load this when:** the app reads `signatures[0]`/`toCharsString()`, or a rebuilt APK installs, launches and logs nothing wrong while every signed request fails. It gives the 15-minute check that prevents the most expensive silent repack failure.
## How to spot it (four greps, minutes)
```bash
# the usual shape: signature bytes handed to a crypto helper
grep -rn "toCharsString()" <decompiled_sources>
grep -rn "getPackageInfo(.*, *64)" <decompiled_sources> # 64 = GET_SIGNATURES
grep -rn "signatures\[0\]" <decompiled_sources>
# and the same pattern in SDK-internal helpers
grep -rn "signatures\[0\].toByteArray" <decompiled_sources>
```
Signals that confirm it matters:
- The value is passed into a `native` method (JNI) or a helper that also does HMAC/AES/DES.
- The string table of the native library contains algorithm primitives (`hmac_sha256`,
`hmac_md5`, `des_crypt`, `sha256_update`, `initKey`, `make_SubKey`, …) but **no hardcoded
certificate or key blob** — i.e. the key is supplied from Java.
- Request URLs carry computed parameters (`sign`, `_p`, `uth`, `sig`, `nonce`) that are not
derivable from the visible app data.
- The same certificate value is also fed to a third-party ad/OAID SDK helper.
If those hold, **any rebuilt APK needs the original certificate value hardcoded**, not the
new one. Patch that before you patch anything else, or you cannot distinguish "my patch broke
it" from "signing broke it".
## The trap: what that value actually is
Do **not** assume `signatures[0].toByteArray()` is the whole `META-INF/*.RSA` file. It is
platform-dependent, and the difference is invisible offline:
| Platform behaviour | What you get |
|---|---|
| Old JAR-signature path | the full PKCS#7 blob (`META-INF/CERT.RSA` content) |
| Modern (v2/v3 present) | a **single certificate from the chain**, DER-encoded — not the container, and not necessarily the first one |
Measured consequence on one sample: the `.RSA` file was 1199 bytes / 2398 hex chars, but the
runtime value was **777 bytes / 1554 hex chars** — the *last* certificate inside the PKCS#7.
Hardcoding the file bytes produced correct-looking smali, a clean build, a successful install,
and every request signing parameter evaluating to `-1`.
**Therefore: never derive this value offline and trust it.** Read it from the running app
(`scripts/sig_probe.py --live`), and only fall back to offline extraction to cross-check.
## Extract the real value
**Route A — from the device (authoritative):**
```bash
python scripts/sig_probe.py --live <pkg> # needs Frida + a rooted device
```
`--live` takes the package name as its own argument; there is no `--pkg` flag. Verify the call
against the script's own usage (`python scripts/sig_probe.py --help`) rather than against this page —
`check_commands.py` at the repository root validates documented commands for exactly this reason.
It prints `signatures[0].toCharsString()` and its length, read from the package manager on the
device. This is the only value that is guaranteed to match what the app computes.
**Route B — offline, to cross-check:**
```bash
python scripts/sig_probe.py --apk <original.apk>
```
This enumerates the candidate certificate blobs inside `META-INF/*.RSA` (each top-level
SEQUENCE inside the PKCS#7 `certificates` set) and prints each as hex with its length. Pick the
length that Route A reported; if Route A is unavailable, generate one build per candidate —
there are rarely more than two.
## Patch it
Replace every signature read with a string constant holding the original value, rather than
trying to make the new signature look like the old one.
In smali, the pattern to replace is an invoke/move-result pair:
```smali
invoke-virtual {vX}, Landroid/content/pm/Signature;->toCharsString()Ljava/lang/String;
move-result-object vX
```
Rewrite both instructions into a single load (a regex over the smali tree handles all sites at
once):
```smali
const-string vX, "<original-value-hex>"
```
Notes that save a build cycle:
- **Do not delete the surrounding `getPackageInfo` calls.** They are harmless and removing
them perturbs register allocation. Only the two instructions above need to change.
- **Miss one site and it still fails.** Count the sites before and after (`grep -c`), and assert
the post-patch count is zero. There is usually more than one — one per crypto helper.
- Keep the smali file's `.orig` backup so the patch is re-runnable; some build pipelines pick
`.orig` up as an unknown file and warn, which is harmless.
- The same treatment applies to copies of the pattern inside bundled SDK packages
(`…/secure/…Utils`, `…/oaid/…`), not just the app's own util class.
## Verify with a control, not with a feeling
A launch that "looks fine" proves nothing here: the app starts, then quietly fails every
network call. Use a differential test.
1. **Measure the signing parameter on both builds.** Hook the request builder and print the
computed parameter for a request the app always makes on startup.
- original APK → a value with the normal shape (encoded / base64-ish)
- rebuilt APK → the same shape ⇒ signing is consistent
- rebuilt APK → `-1`, empty, or null ⇒ the key is still wrong; go back to Route A
2. **Compare a full request URL**, parameter by parameter, between the two builds. Anything
that differs beyond a timestamp/device id is a candidate root cause.
3. **Exercise the feature**, not the launch: an API-backed screen must actually render data.
"Network error" on a screen that is otherwise intact is the signature of exactly this bug.
Before blaming your patch: run the **unmodified original** on the same device and network. If
it also fails, the problem is environmental (`references/tls-and-cert.md`,
`references/pitfalls.md` P9) and not the repack.
## Related variants of the same root cause
- **OAID / device-id SDKs** hash the signature (`MessageDigest("SHA1").digest(signatures[0].toByteArray())`)
and the result goes into a request header. Same fix, different call site.
- **Ad SDKs** use the signature to encrypt their own config payloads; leaving them unpatched
usually degrades ads rather than breaking the app, so patch them after the core path works.
- **Certificate pinning to the app's own signature** is a different problem: it is verified,
not used as a key. See `references/tls-and-cert.md`.
## Cost of getting this wrong
Without this step the task stalls in the worst possible way: the artifact is correct, the
patch is correct, the build is correct, and the app is unusable — which reads as "the client
patch is impossible" and sends you back to static analysis for hours. Budget 15 minutes for
this before the first repack of any signed-request app.

325
vendor-new/apk-reverse/skills/apk-reverse/references/split-apk.md Normal file
View File

@@ -0,0 +1,325 @@
# Split APKs and App Bundles
A store build is frequently **not one file**. Play and most OEM stores ship App Bundles, and the
device receives a set: `base.apk` plus one `split_config.*.apk` per ABI, screen density and language
(and, for a dynamic-feature bundle, the feature modules as well). Every member carries **the same
package name and its own signature**, and the package manager treats the set as one package.
Two failures follow from treating such a set as an APK, and both look like a broken build when they
are nothing of the kind:
- Signing only the base leaves the set disagreeing about its certificate; the installer rejects the
whole set and names the package, not the member you forgot.
- "Just merge them into one APK" is free *only* for members that carry code or native libraries. A
member that carries resources cannot be folded into the base without merging `resources.arsc`,
which is a resource-compiler job. A wrongly merged archive installs and then renders the wrong
thing, or dies on the first resource lookup.
`scripts/repack.py` handles both routes and **refuses to guess**: it inventories the set, decides
which route is legal from what the members actually carry, prints the reason, and only then acts.
**Strength labels** (same three this repository uses everywhere): **observed** = reproduced with an
exact command and output; **inferred** = follows from observed facts, step not executed;
**unverified** = reported or assumed, not independently confirmed. The measurements behind this
file are recorded in `references/evidence-summary.md` §The capability matrix.
**Load this when:** the target arrives as `base.apk` plus `split_config.*.apk`, or a rebuild is refused as a set although every member verifies. It gives reading the set, merge versus unified re-signing, and the refusal each mistake produces.
## 1. What the set is
| Member | Manifest `split` attribute | Carries | Foldable into the base? |
|---|---|---|---|
| base | *(absent)* | all `classes*.dex`, the main `resources.arsc`, every resource not moved out, the real `<application>` | — it *is* the base |
| ABI split | `config.arm64_v8a`, `config.armeabi_v7a`, `config.x86_64`, … | `lib/<abi>/*.so` only | **yes** — entry names do not collide |
| density split | `config.hdpi` … `config.xxxhdpi` | its own `resources.arsc` **and** `res/` entries | **no** — requires a resource merge |
| language split | `config.en`, `config.zh`, … | its own `resources.arsc` (strings) | **no** — same reason |
| feature split | `feature_x` (+ `isFeatureSplit="true"`, `configForSplit="base"`) | its own `classes*.dex`, possibly resources | **code only**; its resources hit the same wall |
**Observed** on a real 9-member set pulled from a stock Android 11 device (arm64-v8a target):
- the base carried `classes.dex` (872 KB) and `resources.arsc` (71 KB);
- the ABI member carried two `lib/arm64-v8a/*.so` files and **no `resources.arsc` at all**;
- each density member carried its own `resources.arsc` and its own `res/` entries;
- a second, 8-member set's density members carried 39-78 `res/` entries each and 9-15 KB tables.
So "which member owns the resources" is answerable **from the zip entry list alone** — no device, no
resource compiler, no decompiler. Copy a set to disk and ask.
**A member can own a nearly empty table.** A split is *required* to carry a `resources.arsc`, so a
placeholder one is normal: on the same real set, eight density members carried a **40-byte** table
and no `res/` entries at all — they are placeholders, not resources. Treating "has an arsc" as "owns
resources" would block a merge that is perfectly legal. `repack.py` uses *entries under `res/` or a
table bigger than 1 KB* as the test.
## 2. Getting the set off a device
`pm path` prints **one line per member**; a package with several lines is a split set. This is how
you find one to copy, and it is also how you tell, before anything else, that the thing you are about
to patch is not a single APK:
```bash
adb shell 'pm path <PKG>'
package:/system/priv-app/<DIR>/<APP>.apk
package:/system/priv-app/<DIR>/<APP>-arm64_v8a.apk
package:/system/priv-app/<DIR>/<APP>-xxhdpi.apk
```
To enumerate every split set on a device (toybox shell — no `--time-style`, no bash arrays):
```bash
adb shell 'for p in $(pm list packages | cut -d: -f2); do c=$(pm path $p | wc -l); \
if [ $c -gt 1 ]; then echo "$c $p"; fi; done'
```
Copy the whole directory out; the members live side by side:
```bash
adb shell 'su -c "ls -la /system/priv-app/<DIR>/"'
adb pull /system/priv-app/<DIR> <dest>/
```
Two traps, both **observed**:
- `adb pull <dir> <dest>` creates `<dest>/<dir>/` — the set arrives one level deeper than you expect.
`repack.py --split-dir` searches recursively for exactly this reason.
- A **third-party** split set is rare on a stock ROM. On the device measured here, `pm list packages -3`
returned **zero** split sets among ~40 third-party apps, while four preinstalled Google components
were split. If you need a real set and the third-party list is empty, that is the normal
distribution, not a mistake on your side.
**Do not test by reinstalling a pulled set under its own package name.** It is already installed on
that device, owned by the vendor certificate you do not have, so the install fails for a reason that
has nothing to do with your pipeline (). Either work on a set you own, or rename the pulled one
() to get an installable fixture.
## 3. The decision: merge or resign
Run this first — it writes nothing and answers with a reason:
```bash
python skills/apk-reverse/scripts/repack.py --split-dir <set>/ --split-mode analyze
```
| Situation | Route | Why |
|---|---|---|
| Every non-base member carries only code and/or `lib/` (a placeholder `resources.arsc` is fine) | **merge** | nothing that needs a resource table moves |
| Any member carries `res/` entries or a real table | **resign** | a resource merge rewrites the table; see §5 |
| You must keep the build installable **and** the app is distributed as a bundle | **resign** | this is what the store does; the set stays a set |
| You need one file to hand someone, sideload, or feed to a tool that takes an APK | **merge**, if legal | the only way to get a single artifact |
| The delivery must survive store updates, or the ROM's own installer refuses multi-APK installs | **resign** (`pm install-multiple`), or merge and accept a fixed build | — |
| A member declares `isFeatureSplit="true"` and ships `classes*.dex` **and** resources | **resign** | its code could fold, its resources cannot; a half-merged build is worse than either route |
Failure modes, one per route:
- **merge**, when a resource-carrying member is dropped to make it legal: the app starts and shows
the base's resources. Missing density variants show up as blurry or missing images; missing
language splits show as untranslated strings. This is a **downgraded** build and has to be
delivered as one — `repack.py` prints `DROPPED n entry/ies` and refuses unless you pass
`--drop-split-resources`, so the loss cannot be silent.
- **merge**, when the base still declares `isSplitRequired="true"`: install succeeds and start is
refused (`INSTALL_FAILED_MISSING_SPLIT`,). The attribute has to be cleared first, and that is a
binary-AXML edit.
- **resign**, when one member is missed: the set fails as a set, with a signature error naming the
package. Sign the directory, not a list you typed by hand.
- **resign**, when the target ROM's installer is not the platform installer: `adb install-multiple`
can be intercepted by an OEM security centre, exactly as single-APK installs are
(`references/repack-and-sign.md` §Vendor install interception (OEM shells)). Use the root
`pm install-multiple` path.
## 4. Unified signing — the route that always works
```bash
python skills/apk-reverse/scripts/repack.py \
--split-dir <set>/ --split-mode resign \
--split-out-dir <signed_set>/ \
--ks key.jks --apksigner <path/to/apksigner>
```
Each member is de-signed (only the signature entries), rewritten as a 4-byte-aligned archive, and
signed with **one** keystore and **v1+v2+v3**; the run then verifies every member and prints the
signer fingerprint per file. The acceptance criterion is not "each file verifies" — it is **one
certificate across the whole set**:
```
[result] OK: 8 members, one certificate (sha256 1eb31d9c…2c44)
```
Then install the whole set, base included:
```bash
adb install-multiple -r <signed_set>/*.apk
adb shell "su -c 'pm install-multiple -r /data/local/tmp/set/*.apk'" # when an OEM installer refuses the multi-APK form
```
**The signature must cover every member**, and `--abi`/density selection happens at install time, not
at signing time — sign them all, let the installer pick (see).
### Toolchain: `uber-apk-signer` is optional, `apksigner` is not
`repack.py`'s original signing route went through `uber-apk-signer` only, so a machine with a full
Android build-tools directory and no uber-apk-signer could not produce a signed build at all. The
script now picks the route itself:
- `--signer jar` — the uber-apk-signer path (used automatically when the jar exists);
- `--signer apksigner` — `zipalign -p -f 4` then `apksigner sign --v1 --v2 --v3`, which is the whole
of what the jar was doing for us;
- `--signer auto` (default) — jar if present, else apksigner.
`--apksigner` accepts all three shapes a build-tools install ships: `apksigner.bat`, the `apksigner`
shell wrapper, or the bare `lib/apksigner.jar`. **On Windows pass the `.jar` or the `.bat`** — the
extensionless wrapper is a shell script and will not execute. A `.jar` value is run through
`java -jar`, so `--java` (or java on PATH) is required for it.
Order is load-bearing: **align, then sign**. `apksigner` adds signature entries without reshuffling
the archive; the v1 (JAR) path emulates `jarsigner`, whose rewrite of the zip would destroy the
alignment Android R+ requires of `resources.arsc`.
## 5. Merging into a single APK
```bash
python skills/apk-reverse/scripts/repack.py \
--split-dir <set>/ --split-mode merge --out merged.apk \
--abi arm64-v8a --ks key.jks --apksigner <path/to/apksigner>
```
What folds: `classes*.dex` (renumbered so a member's `classes.dex` becomes `classes2.dex`, keeping
the base's own dex intact), `lib/**`, `assets/**`. What does not, and why: `res/**` and
`resources.arsc`.
**Why resource merging is not attempted.** `resources.arsc` is an indexed table: a global string
pool plus type-spec and type chunks, with every resource identified by a `(package, type, entry)`
triple and every string referenced by pool offset. Folding a member's table into the base's means
rewriting pool offsets, type/entry counts and the alignment of every string, in both directions —
that is `aapt2 link`, not a byte-level edit. The failure it produces is the worst kind: the archive
is *structurally* valid, installs, and then resolves resource ids to the wrong entries or to nothing.
An APK that renders garbage is harder to diagnose than one that refuses to install.
If you genuinely need one standalone file for a bundle whose resources are split, the honest routes are:
1. **`bundletool build-apks --mode=universal`** (the vendor tool, needs the original `.aab`) — this is
what merges resources correctly, and it is the only route that does.
2. **Keep the split set** and install it as a set ().
3. **Accept the downgrade** with `--drop-split-resources`, and say so in the delivery.
**Manifest constraints when you do merge — and when you patch any split at all.** These attributes
are the difference between "installs and runs" and an install or start refusal, and they are easy to
miss because they live in binary AXML:
| Attribute | Lives on | Effect if you get it wrong |
|---|---|---|
| `split="config.xxhdpi"` | `<manifest>` of a **split** | Identifies the member. A split whose `package` differs from the base's, or whose `split` name is duplicated, is refused as a set |
| `configForSplit="base"` | `<manifest>` of a split | Ties a configuration split to a feature module. Changing package/module names without updating it silently detaches the split |
| `isFeatureSplit="true"` | `<manifest>` of a feature split | Marks an install-time/on-demand module. A merge that folds code in but leaves the attribute declared describes a member that no longer exists |
| `isSplitRequired="true"` | `<manifest>` of the **base** | The platform refuses to start a base that believes it is incomplete. This is the one that breaks merged builds () |
Read them without a decompiler through `scripts/repack.py`'s own AXML reader (`--split-mode analyze`
prints the `split` name per member; `inspect_apk()` returns `package`, `split`, `configForSplit`,
`isSplitRequired`, `isFeatureSplit`). Changing any of them means editing the binary string pool and
the attribute chunks around it, which is what `apktool` is for — and note that a full
`apktool b` rebuild rewrites the whole archive
(`references/repack-and-sign.md` §Repacking an unpacked (de-shelled) app).
## 6. Install refusals, and what each one means
| Message | What it actually says | Attribution |
|---|---|---|
| `Failure [INSTALL_FAILED_INVALID_APK: ... signatures do not match previously installed version]` / `Package <PKG> signatures do not match` | The set you are installing is signed with a different certificate than the one already on the device | **Not** a malformed build. Either re-sign to the original key, or install to a package name that does not exist yet |
| `Failure [INSTALL_FAILED_ALREADY_EXISTS]` / `INSTALL_FAILED_UPDATE_INCOMPATIBLE` | Same package, different signer (or a downgrade without `-d`) | A pulled system component cannot be reinstalled over itself; see §7 |
| `Failure [INSTALL_FAILED_MISSING_SPLIT: Missing split for <PKG>]` | The base declares `isSplitRequired="true"` and the set is incomplete — or you merged it | The base is telling the truth: it expects members. Clear the attribute, or install the complete set |
| `Failure [INSTALL_FAILED_INVALID_APK: Failed to parse ... split ... has different signature]` | **One member** of the set is signed differently from the rest | This is the failure mode of signing only the base. Sign the whole directory |
| `Failure [-124: ... resources.arsc ... stored uncompressed and aligned on a 4-byte boundary]` | One member's `resources.arsc` is compressed or unaligned | Per-member, not per-set. `repack.py` reports it per member before signing (`references/repack-and-sign.md` §2a) |
| `Failure [-99]`, or a numeric-only code with no `INSTALL_FAILED_*` | An OEM installer interceptor rejected the request | Nothing to do with the set; use the root `pm install-multiple` path |
| `adb install-multiple` on a device that has no `install-multiple` support (very old `pm`) | Install each member with `pm install` in order, base last | Use `pm install-create` / `install-write` / `install-commit` explicitly |
`INSTALL_FAILED_INVALID_APK` in particular is a **family**, not one error: it covers stale
signatures, malformed base/split relationships, and alignment violations. Read the text after the
colon before attributing anything — the constant alone will send you looking at the wrong file.
## 7. Making an installable fixture from a pulled set
**Observed, and the reason this section exists:** a set pulled from the device is installed *on that
device*, under a name owned by the vendor certificate. Re-signed copies cannot be installed back
over it, so "does our signed set install?" cannot be answered with the pulled set as it stands.
The cheap, structure-preserving way out is an **equal-length package rename**: replace every
occurrence of the old package name in `AndroidManifest.xml`, `resources.arsc` and `classes*.dex`
with a new name of exactly the same length. Nothing moves — no length prefix, no string-pool offset,
no dex `string_ids` offset, no resource index — so the only field that must be recomputed is the dex
header's integrity pair (signature first, checksum last, see
`references/repack-and-sign.md` §Repacking an unpacked (de-shelled) app). Then run the ordinary
`--split-mode resign` over the renamed directory and install that.
A detail worth knowing before you touch a manifest: **a member's Java classes need not live under its
manifest `package` at all**. On the real set measured here, the manifest package was
`com.google.android.<name>` while all 677 class references in the dex were
`Lcom/android/<name>/...` — and every `<activity android:name>` was fully qualified. Renaming the
manifest package therefore left every component resolvable, with no dex edit needed. Assume the
opposite (that the two must match), and you will edit a dex for nothing.
## 8. ABI and density matching
The device tells you what it can run:
```bash
adb shell getprop ro.product.cpu.abilist # e.g. arm64-v8a,armeabi-v7a,armeabi
adb shell getprop ro.product.cpu.abi # the primary ABI
adb shell getprop ro.product.cpu.abilist32 # 32-bit list, on a 64-bit device
```
Match against the **split name**, not the file name, and mind the separator: split qualifiers use
**underscores** (`config.arm64_v8a`) while `lib/` directories use **hyphens** (`lib/arm64-v8a/`). The
mapping is a substitution, not a lookup — `arm64_v8a` → `arm64-v8a`, `armeabi_v7a` → `armeabi-v7a`;
`x86` and `x86_64` are unchanged. That is why `--abi arm64-v8a` matches a split named
`config.arm64_v8a`.
Four rules that decide ABI questions:
- **A 64-bit device still installs a 32-bit-only set**, and then runs it 32-bit. If the base carries
no native code and only the `armeabi_v7a` split does, the app is a 32-bit app on that device.
- **The ABI split is selected by the installer, not by `repack.py`.** `--abi` exists for *merging*
(where the choice is baked into one file); for `--split-mode resign`, sign every member and let
`pm` pick.
- **Merging without `--abi` folds in every ABI member**, which makes one APK with two or three copies
of the same library set. Legal, larger, and pointless on a single-device delivery.
- **`getprop` reports what the device claims**; what runs is a live mapping question
(`scripts/lib_map.py`). A device that reports `arm64-v8a` first can still execute a 32-bit process.
Density is the same shape without the separator trap: `config.xxhdpi` matches
`res/drawable-xxhdpi/`, and the installer picks the closest variant the device's `ro.sf.lcd_density`
can use. There is nothing to select on your side.
## 9. Traps worth paying for once
- **`resources.arsc` belongs to whoever ships it.** In a split set it exists *per member*, and the
base's table does not contain the density member's entries. Reading the base's table and concluding
"the resource exists" is wrong; it exists in one configuration only.
- **`android:extractNativeLibs`.** If the base declares `extractNativeLibs="false"` (the default for
modern targetSdk), the `.so` files are mmapped straight out of the archive, and they must be
**uncompressed and page-aligned** — this is what `zipalign -p` exists for. Merging native members
into a base and then writing the archive without that alignment produces an install that succeeds
and a start that dies in the linker. `repack.py` writes uncompressed `lib/**` 4-byte aligned, and
`zipalign -p -f 4` adds the page alignment the loader wants for them.
- **A placeholder `resources.arsc` is not a resource split** (). Blocking on "has an arsc" refuses
legal merges.
- **Do not `zipalign` after signing to "fix" a member.** It rewrites the archive and invalidates the
signature. Align, then sign.
- **`apktool` may not be usable even when a wrapper exists.** Measured on the pass machine: the
`apktool.bat` wrapper pointed at a jar that was not on disk
(`Error: Unable to access jarfile <missing>.jar`), while `apksigner.jar` and `zipalign.exe` in the
same layout were fine. Check the wrapper's target before planning a manifest edit around it.
- **The AXML `split` attribute is a string-pool entry**, which is why an equal-length rename of it is
safe and why a length-changing edit is not: the pool's own length prefixes and the attribute chunk
sizes around it would both need recomputing.
## 10. What this file does not claim
- **It does not merge resources.** No route here reproduces `bundletool build-apks --mode=universal`.
A bundle whose resources live in splits stays a set, or becomes a downgraded single APK by explicit
choice.
- **It does not split a monolithic APK back into modules**, and it cannot turn a `.aab` into APKs —
that is `bundletool`, and the `.aab` is the input it needs.
- **`--split-mode merge` covers code and native libraries only** (). Feature-split *code* folds;
feature-split *resources* do not.
- The install-failure texts in §6 are the shape the platform produces; **the exact wording varies by
Android version and by OEM installer**. Read the text after the constant, not the constant.

142
vendor-new/apk-reverse/skills/apk-reverse/references/third-party-builds.md Normal file
View File

@@ -0,0 +1,142 @@
# Third-party builds — auditing a "cracked" or "modded" APK before trusting it
**Load this when:** the input is a "cracked" or "modded" APK you did not produce. It gives the audit that tells you what was injected, and why such a build is never a patching workbench.
## Why this exists
A modded build of the exact app you are targeting is the most tempting shortcut available:
it appears to prove the goal is achievable, and it may already contain the patch you were
about to write. It is also the single most common source of wasted hours and of real risk,
for two reasons.
1. **It is usually not editable.** Authors who redistribute mods protect them, often with
more layers than the original app ever had. You cannot treat someone else's build as a
patching workbench.
2. **Its target-audience behaviour is unverified.** The binary may carry injected components,
extra endpoints, or embedded credentials. "It runs and the ads are gone" tells you nothing
about what else it does.
Treat a third-party build as an **untrusted sample to be audited**, never as a baseline.
The useful output of an audit is a decision plus evidence — not a binary you start from.
## Step 1: Can you even edit it?
Find the app's own package prefix (the one that appears in its own class definitions, e.g.
`com/<vendor>/<app>/`) and count how many times it appears as a **defined class** in each
plaintext dex.
```
business-prefix hits per dex:
classes.dex 0 <- shell
classes2.dex 0
classes3.dex 0
classes4.dex 0
```
Zero everywhere means the business code has been **moved out of the plaintext dexes** — the
build is protected at a level you cannot patch directly, even though it launches and works.
Stop here and treat it as a black box.
Also check the reported dex count against the file count: a manifest-declared app with only
library classes present is the same signal.
## Step 2: How many protection layers does it carry?
Mods frequently stack protection from multiple vendors, sometimes because the author
re-protected an already-protected app. Look for independent, coexisting families:
| Layer kind | Typical static evidence |
|---|---|
| Commercial packer | shell `Application`/`appComponentFactory` in the manifest, a `lib*protect*.so`, encrypted assets, `assets/*.jar` with near-maximal entropy |
| DEX-to-native (dex2c) | a "stub" `lib*.so` containing the tool's name, `EntryPoint`/`protect`-style symbols, and **business class names as strings** alongside a near-empty dex layer |
| Anti-debugging / RASP | strings such as `TracerPid`, `ptrace`, `NoNewPrivs`, `CheckPtraceSelf`, `AntiFrida`, plus a self-protection log path |
| Generic hardening marker files | small text files under `assets/` naming a protection vendor and a protection timestamp |
The practical consequence of an anti-debugging layer is that Frida-based dynamic work will hit
a self-check before it hits your hook. Note it, and prefer static conclusions on such a sample.
## Step 3: What did the author ADD?
This is where the audit earns its keep. Diff against the **original build of the same version**
(referenced as `<original>` below).
### New components
Build the set difference of `activity` / `service` / `receiver` / `provider` declarations,
then check whether each new class name actually exists in the plaintext dexes.
- Class exists and is readable → you can judge its purpose.
- Class name appears **nowhere** in plaintext but shows up inside a native library's strings →
the logic is in the native/dex2c layer, purpose unknown. Say so.
- Nothing registered with an implicit-launch intent-filter (`BOOT_COMPLETED`, `MAIN`) → it cannot
be started implicitly, which limits (but does not eliminate) the risk.
- A provider with an `authorities` value containing a package-name rewrite (a suffix/prefix that
is not the current package) is a strong hint the build was produced by **repackaging or app
cloning**, not by editing the original.
### New outbound endpoints
Extract every URL/domain/IP from the modded build's `lib/*.so`, `assets/*`, and dex strings,
and subtract the original's set. Any domain that exists **only** in the modded build is the most
important finding in the audit.
Rank what you find:
- A domain that also serves install attribution or update checks is explainable.
- A domain paired with an authorization-looking path (e.g. a `GET...`-style endpoint name
nearby in the same string neighbourhood) is *not* explainable by de-ad work.
### Embedded secrets
Search native libraries and assets for private-key markers:
```
-----BEGIN ... PRIVATE KEY----- , PRIVATE KEY , PKCS#8 , RSA PRIVATE
```
A **private key shipped inside a client artifact** means its "authorization" or "integrity"
path is not a security boundary — anyone can extract the key. Report it as a red flag on the
build's trust claims, regardless of intent.
## Step 4: Capability picture — read both sides
A native library's import table is only half the story.
- **What it does not have** is meaningful: no `socket`/`connect`, no `execve`/`fork`/`system`,
no `ptrace`, no installer-session strings → no direct evidence of command execution or
silent install.
- **What it references** can override that impression: string references to Java-layer
`java/net/Socket`, `InetSocketAddress`, reflection entry points, or executor services mean the
capability exists **through the Java layer**, off the native import table.
State findings as "capability present / no evidence of use" rather than "safe" — an import table
is evidence of shape, not of intent.
## Step 5: The verdict
Do not hand back a binary. Hand back a decision with evidence and explicit unknowns.
```
VERDICT: use / use with caution / do not use
EVIDENCE:
- business code plaintext? yes/no (+ counts)
- protection layers list
- injected components list, each with purpose known/unknown
- endpoints present only in this build list
- embedded credentials yes/no
UNCONFIRMED:
- <each item you could not establish, stated plainly>
```
"Use with caution" is the honest answer whenever a build is un-analysable **and** carries new
endpoints or unknown injected components. Do not upgrade it to "safe" because it launches
cleanly, and do not call it malicious without evidence — both are guesses.
## Using the audit result
Two legitimate outcomes:
1. The audit shows the mod is an honest de-ad of the same version → you may use it to **learn the
approach** (diff its patch points against the original), but still rebuild from the original
yourself so the artifact you ship has a known provenance.
2. The audit shows it is re-protected, injected, or un-analysable → discard it and do the work on
the original. That is usually faster than reverse-engineering someone else's protection *and*
your target's at the same time.

181
vendor-new/apk-reverse/skills/apk-reverse/references/tls-and-cert.md Normal file
View File

@@ -0,0 +1,181 @@
# TLS and certificate failures
**Symptom class:** after a repack, login or registration fails with a TLS error while the rest of the app behaves normally. This is the most common way a *correct* patch gets blamed for a *pre-existing, server-side* condition. Work the five steps below before changing a single byte of the app.
**Load this when:** one feature fails at runtime -- login, registration, payment, an API-backed screen -- while the rest of the app works. It gives the five steps that separate a real TLS/certificate problem from the patch you just built.
## Fast path
| # | Action | Evidence |
|---|---|---|
| 1 | Read the full exception chain, prove it is a date rule | `THROW` events from the probe, or logcat |
| 2 | Validate the certificate from a host you trust | `python scripts/tls_check.py <host>` |
| 3 | Compare another host from the same app, and the device clock | same script, `adb shell date` |
| 4 | Decide which trust chain owns the failing request | `RAW-URL` vs `OKHTTP` probe events |
| 5 | Only then patch the system-trust path | fix template below |
## Symptoms
- Login or registration reports a generic network failure: `Network request failed: Chain validation failed`.
- Everything else works: home, feed, images, playback, guest browsing.
- The full chain, visible only if something captures swallowed exceptions:
```
javax.net.ssl.SSLHandshakeException: Chain validation failed
caused by: java.security.cert.CertificateException: Chain validation failed
caused by: java.security.cert.CertPathValidatorException: timestamp check failed
```
- `timestamp check failed` is the decisive token. The validator rejected the chain on a **date rule**, not on an untrusted issuer or a hostname mismatch. Exactly two conditions produce it: the certificate is expired or not yet valid, or the **clock of whoever performs the check is wrong**.
## Step 1: get the real host out of the runtime
Never trust a host recovered from strings alone; a build config can hold dead or staging hosts. Get the host the app actually connects to:
- Run the four-layer probe: `scripts/frida_probe.js`, launched and streamed to a log by `scripts/run_probe.py` (setup, version alignment, and hooking strategy: `dynamic-frida.md`). Three event labels carry what you need.
- `RAW-URL`: the request went through `java.net.URL.openConnection` / `openStream`, the system trust path.
- `OKHTTP`: the request went through the app's own OkHttp client, a different trust path.
- `THROW`: the original text of an exception an upper layer caught and swallowed. This is where the real TLS message appears when the UI only shows a generic network error.
Output you need: the exact URL, its host, and which label logged it.
## Step 2: validate the chain out of band (no app involved)
This step is independent of the app, the patch, and the device. It runs on a host whose clock you trust:
```bash
python scripts/tls_check.py <host> # strict verification + certificate detail
python scripts/tls_check.py <host> <host2> # compare hosts in one run
python scripts/tls_check.py <host> --json # machine-readable; exit 2 if any host fails
```
The script validates strictly (system CAs, expiry, hostname) and, when verification fails, still decodes the peer certificate locally, so a failure arrives with a reason instead of a stack trace:
```
EXPIRED certificate has expired
NOT-YET-VALID certificate is not valid yet
HOSTNAME-MISMATCH chain is fine, the name does not match
UNTRUSTED-CA self-signed, or issuer not in the trust store
CHAIN-BROKEN bad signature / invalid CA / incomplete chain
```
Interpretation:
- `EXPIRED` reported from a trustworthy clock is proof that the chain is rejected by the certificate itself, not by the device, the network, or your patch.
- A clean pass means the certificate is fine and the failure is a client-side trust decision or a clock problem. Go to Step 3.
- Non-443 endpoint, or an address that needs a different SNI name (an IP connect, a CDN edge): `--port <port>` and `--sni <host>`. Getting this wrong produces a `HOSTNAME-MISMATCH` that has nothing to do with the real problem.
Minimal equivalent, if the script is not at hand:
```python
import socket, ssl
ctx = ssl.create_default_context() # strict: system CAs, expiry, hostname
with socket.create_connection(("<host>", 443), timeout=10) as raw:
with ctx.wrap_socket(raw, server_hostname="<host>") as tls:
print("OK", tls.version())
# on failure: ssl.SSLCertVerificationError: ... certificate has expired
```
Dates, subject, and issuer in one line, if you prefer no script at all:
```bash
openssl s_client -connect <host>:443 -servername <host> </dev/null 2>/dev/null \
| openssl x509 -noout -subject -issuer -dates
```
## Step 3: compare another host, and check the device clock
Run the same check against every host you extracted from the app, then compare:
- A very common shape: the business API host is expired while the config, image, or CDN host is valid. Same app, same network, same machine, different certificates. That pattern exonerates the network path and points at one certificate.
- Check the device clock; it costs one command:
```bash
adb shell date # compare against a known-correct time source
```
If Step 2 passes but the device fails, the cause is the device clock, not the certificate. Correcting the clock and relaxing the trust policy are different decisions; do not do both silently.
- Check the **unpatched original** in the same environment. See `verification.md` §the control build rule and `pitfalls.md` P9. If the original fails identically, the patch is not the cause, and no amount of dex work will fix it.
## Step 4: which trust chain owns the failing request
One app can carry two independent trust chains, which is exactly why "login is broken but everything else works".
| Path | Client | Trust configuration | Failure signature |
|---|---|---|---|
| OkHttp | OkHttp / Retrofit, via `newCall` | app-supplied `SSLSocketFactory` + `HostnameVerifier`, often pinning | pin-related messages (`CertificatePinner`, pin verification failed), or nothing at all if the app relaxed it |
| `java.net.URL.openConnection()` | `HttpsURLConnection`, also used by many image loaders, downloaders, and legacy HTTP wrappers | **platform default** trust store and default verifier (`HttpsURLConnection.setDefault*`) | `Chain validation failed` / `timestamp check failed`, the signature above |
Discriminating, in increasing order of certainty:
1. **Probe events.** The failing request appears under exactly one label from Step 1: `RAW-URL` (system path) or `OKHTTP` (app client). This is a fast screen, not proof.
2. **Trust-manager attribution (definitive).** Hook every `X509TrustManager.checkServerTrusted` implementation and print the instance class name plus a short `new Exception().getStackTrace()`. If the caller frames sit in `okhttp3.*`, or in a custom class the app installs on its own client, the OkHttp path owns the failure. If they sit in a platform trust manager reached through `HttpsURLConnection` / `SSLSocket` internals, the system path owns it. Wording alone is version-dependent, so trust the caller frames, not the message.
3. **Cross-check with the fix below.** If OkHttp owns the failing request, the template does nothing for it: that path never reads `HttpsURLConnection` defaults and needs a client-level change (`SSLSocketFactory` / `HostnameVerifier` / pinner; see `dex-patching.md` §where to patch).
Why the asymmetry exists: the app configured its own trust behavior for OkHttp (a relaxed verifier or a pin), so that path never consults the platform date rule, while the legacy `HttpsURLConnection` path still validates against the system trust store and rejects the expired chain.
## Step 5: fix template (system path only)
Install permissive TLS defaults for the `HttpsURLConnection` path at the earliest point of `Application.onCreate()`:
```java
// Call from Application.onCreate() before any other initialization.
private static void installPermissiveTlsDefaults() {
try {
javax.net.ssl.SSLContext ctx = javax.net.ssl.SSLContext.getInstance("TLS");
ctx.init(null, new javax.net.ssl.TrustManager[]{
new javax.net.ssl.X509TrustManager() {
public void checkClientTrusted(java.security.cert.X509Certificate[] c, String a) {}
public void checkServerTrusted(java.security.cert.X509Certificate[] c, String a) {}
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return new java.security.cert.X509Certificate[0];
}
}
}, new java.security.SecureRandom());
javax.net.ssl.HttpsURLConnection.setDefaultSSLSocketFactory(ctx.getSocketFactory());
javax.net.ssl.HttpsURLConnection.setDefaultHostnameVerifier(
new javax.net.ssl.HostnameVerifier() {
public boolean verify(String hostname, javax.net.ssl.SSLSession session) { return true; }
});
} catch (Exception ignored) {
// Leave platform defaults in place: pre-patch behavior, no new failure mode.
}
}
```
Land it as a small utility class plus its inner classes in smali, or as a dexlib2 rewrite that inserts the call at the head of `Application.onCreate` (`dex-patching.md`). Keep it inside the app's own package; do not edit third-party classes.
Why this shape is correct:
- It changes **one** trust decision: the default `HttpsURLConnection` factory and verifier. OkHttp derives its trust manager through `TrustManagerFactory` and its own socket factory, so app-side pinning and verifier behavior is untouched. You are not silently weakening a control the app deliberately added.
- Call it **before any other initialization**. Globals set after the first `HttpsURLConnection` use still apply, but a library that cached a factory reference earlier keeps the old one, which produces a confusing partial fix.
- The scope is still wider than "login". **Every request that goes through `URL.openConnection()`** inherits it, including image loaders, download managers, and older push or analytics SDKs. State that in the report instead of implying you touched one endpoint.
- The `catch` that leaves platform defaults alone is deliberate: the worst case is the behavior you started with.
- This is a **client-side workaround for an external condition**. The correct fix is renewing the server certificate. Say so, and do not present the patch as the fix.
Do not: change the device clock, install your own CA into the device trust store (device-side, harder to roll back, and not part of a shippable artifact), rewrite the request host to a different domain, or reach for `System.setProperty` tricks. Each of them either misses the actual cause or widens the change far beyond the failing flow.
## If OkHttp is the failing path
Then the failure is about the app's own client configuration, not the platform defaults. Typical causes: a pin that no longer matches, a hostname verifier rejecting a changed host, or a custom `SSLSocketFactory` built from a bundled keystore. Locate it the same way (Step 4, item 2) and patch at the client construction site instead of applying the template above. `dynamic-frida.md` covers finding the real construction site; `dex-patching.md` covers the patch layer.
## Method notes
- **Order of investigation is the lesson.** Every piece of evidence above comes from the server, the network, or the platform before anything is attributed to your patch: full exception chain, real host, out-of-band validation, second-host comparison, control build. Only when all of them point at a client trust decision is a patch justified. `pitfalls.md` P16 is this mistake written up as a failure.
- **This is a control-build case, not a patch case.** `verification.md` §the control build rule applies directly: same pipeline, zero patches. If the control fails the same way, stop debugging the patch.
- **Report the residual honestly.** "The server certificate is expired; the client patch relaxes validation on the `HttpsURLConnection` path only, OkHttp is unchanged, the proper fix is on the server" is the useful statement. "I fixed login" is not.
- **Claim strength.** This evidence chain reaches rung 6 on the `verification.md` claim ladder: the mechanism is proven, not merely observed. Say which rung you reached.
## Triage checklist
```
[ ] Full exception chain captured, contains "timestamp check failed"
[ ] Exact host and URL obtained from a runtime probe (not from strings)
[ ] Standalone strict validation run: EXPIRED / clean pass
[ ] A second host from the same app compared (isolates the certificate)
[ ] Device clock checked against a known-correct source
[ ] Original unpatched build reproduces the same failure (control build)
[ ] Failing request attributed to a trust manager implementation, not to a message
[ ] If system path: permissive defaults installed at the top of Application.onCreate
[ ] If OkHttp path: client-level config is the patch target instead
[ ] Delivery notes state the client-side scope and the server-side root cause
```

598
vendor-new/apk-reverse/skills/apk-reverse/references/toolchain.md Normal file
View File

@@ -0,0 +1,598 @@
# Toolchain — What to Use, How to Invoke It, and Where It Lies
Load this when you are choosing tools, when a tool produces an answer that smells wrong, or when
something is not installed. It is a map, not a tutorial: each entry says what the tool is *for*, how
to drive it non-interactively, and the failure mode that wastes time.
**Prefer tools you can drive from a command line.** An agent cannot click. A tool that ships only a
GUI is not automatically out of reach — check for a headless or MCP path first, which is an
install-and-set-up step rather than a reason to substitute the tool. When no such route exists, ask
for a human instead of stalling or silently downgrading to a weaker method.
## Tier 0 — present on almost any machine, no install
Use these before reaching for anything heavier. They answer structure questions in seconds and
cannot be broken by a missing dependency.
| Tool | Use it for |
|---|---|
| `file`, `readelf -h/-l/-d/-r`, `objdump -d`, `nm -D`, `strings` | ELF structure, program headers, dynamic section, relocations, dynamic symbols |
| `unzip -l`, `zipinfo` | what is actually inside an APK, entry sizes and compression |
| `sha256sum` / `Get-FileHash` | identity. **Always hash before and after; never trust "it should be the same file"** |
| `xxd` / `hexdump` | byte-level ground truth when a tool disagrees with another tool |
| `python3` | all the scripts here; the standard library alone covers most parsing |
`readelf`/`objdump` walk **section** headers. If a target's section table is forged (see
`native-tamper-and-suicide.md` §Forged section headers) they will print confidently wrong output —
cross-check with the program headers.
## Tier 1 — dex and Java
| Tool | Invoke | Why this one |
|---|---|---|
| `baksmali` / `smali` (jars) | `java -cp <jars> org.jf.baksmali.Main d <dex> -o <dir>` | round-tripping, reading a method precisely |
| `dexlib2` | small Java program | method-level rewrite, leaves everything else untouched (`dex-patching.md`) |
| **`rasc`** (Rust ASC) | `rasc findrefs app.apk string <S>` | **the same index, 4–15× faster**, no Python runtime, a Rust DEX decompiler behind `getclass`. Reach for this FIRST when it is built — its one blind spot is documented in `rasc-and-droidsaw.md` |
| **`droidasc`** (ASC) | `droidasc findrefs app.apk string <S>` | the original Python index, one `pip install` away, and the cross-check for `rasc`. Reach for it FIRST when `rasc` is not built |
| **`ddc`** | `ddc app.apk -c <Class>` | **fastest read of dex as Java, plus query subcommands** — reach for this SECOND, to *read*. See below |
| `apktool` | `java -jar apktool.jar d/b` | whole-app decode including resources |
| `jadx` | `jadx --no-res -d <out> <apk>` | readable Java for orientation; **not** a source of truth, and **not** a recon entry point (see below) |
| `aapt2` | `aapt2 dump badging <apk>` | manifest facts, package name, versions |
| `zipalign`, `apksigner` | from build-tools | alignment and signing |
### rasc — the same index in Rust, when you have built it
`rasc` is the `rust` branch of the same project (`MG1937/ASC`). Same CLI shape — `classes`,
`manifest`, `getclass`, `findrefs {string,type,method,field}` — no Python at all, and a 2.13 MB
binary. **Measured on this repository's archives: identical class-definition sets on both a 1.4 MB
MASTG challenge and a 34.8 MB app (30,768 classes, 0 differences either way), at 4.0–4.7× for
`classes` and up to 14.7× for `findrefs`.**
It must be built — there is no release asset and no crate — so the kit wraps that:
```bash
python scripts/rasc_build.py --check # what is present
python scripts/rasc_build.py --build # needs git + rustup; ~2 min
python scripts/rasc_build.py --verify app.apk # compares class sets against droidasc, fails on a difference
```
**It is a registered capability, so G2 does not have to guess:** `doctor.py` reports `dex_index_rust`
as `OK` when a binary is present (PATH, `APKREV_TOOLS`, or the work-area path `rasc_build.py` builds
into) and `BLOCKED` with that build command as the next action when it is not — alongside
`dex_index_python` for the `pip install` route. A machine with neither is not stuck: the Python
indexer is one command away, and the two are cross-checked against each other by
`rasc_build.py --verify`.
**The one thing to know before trusting its `getclass`:** on an `enum` whose constants override an
abstract method, the outer class is printed as a bare constant list and the per-constant bodies are
**not inlined** — no warning, and none in the Python tool either, because both leave those bodies in
their own subclasses. They are one query away (`Lpkg/Enum$1;`, and both tools decompile those fully);
`droidasc`'s outer-class listing is simply richer (2,170 B vs 314 B on the measured class, carrying
`$VALUES`, `$values()` and the constructors). Sampling 20 app-like classes from a real app, 17 were
judged by the JADX-parity harness and 16 agreed literal for literal; the one that differed was exactly
this shape. Use it to *locate* and to read ordinary classes, and read the constants when an enum shows
no bodies. The measurements and the failing class are in `references/rasc-and-droidsaw.md`.
### droidasc (ASC) — ask an APK "who references this?", in one query
Install it in one line. No JVM, no Android SDK, no GUI, no first-run indexing:
```bash
pip install droidasc # provides the `droidasc` CLI
```
It treats the artifact as a **read-only database** instead of exporting a source tree: it probes the
deflate stream in place and rebuilds only the minimal dex it needs, in memory, for the one class you
asked about. Nothing is inflated to disk; there is no index-building phase to wait through. (Design
and measurements are the author's; the BlackHat EU 2026 Arsenal abstract is the reference.)
```bash
droidasc findrefs app.apk string "/data/app/" # every site that mentions a literal
droidasc findrefs app.apk string com.example/sdk # a channel name, a URL fragment, a field name
droidasc findrefs app.apk type com/foo/Bar # who references a type
droidasc findrefs app.apk method notify --class MainActivity --fuzzy-class
droidasc listclass app.apk --prefix com/poc # what classes exist (28210 classes: seconds)
droidasc listclass app.apk -o classes.txt
droidasc getclass app.apk Lcom/poc/Main; -o Main.java
droidasc getmanifest app.apk -o AndroidManifest.xml
```
Each hit names the **class and method** it sits in. That is the whole value: it turns "which of 28,000
classes mentions this string" from a crawl over an exported tree into one sub-second query, and it works
straight off the APK you were handed.
**What it is for, and where it stops.** ASC answers *location* questions. It is not where you read a
method body, and it does not replace the kit's dex tooling for patching. Its place is the first ten
minutes of recon plus every later "where else is this used?" — the questions that otherwise send you
grepping a hand-exported smali tree.
**Three limits, and one of them is a trap.**
- **A non-ASCII needle is not trustworthy through a shell.** A `findrefs string` query for a non-ASCII
literal returned nothing under Windows/PowerShell, and **an empty result is indistinguishable from a
broken argument encoding.** Many non-ASCII literals genuinely live in a native library rather than in
dex, so "nothing came back" is a plausible answer — which is exactly what makes the unverified
negative dangerous. Confirm it another way before recording it: search the dex bytes directly with a
`\u`-escaped needle, or run a control query you know must hit (a path fragment, a package name you
already read in the manifest) in the same session.
- **A mangled class name may return only resources.** `listclass apk --prefix <the plugin's package>`
can come back with nothing but `R$anim` / `R$string` / `R$style` entries, which is **not** a dead end:
it is evidence the real class was renamed by R8. `findrefs string <the plugin's channel name>` still
hands you the obfuscated class implementing it. This is the single situation where ASC is not an
alternative to `ddc` but the only route — `ddc -c <FQCN>` needs the post-R8 name to start with.
- **A hit is a *reference*, not a *call*.** `findrefs` proves a literal or type is mentioned in a method;
whether that method runs on the path you care about is a different question, and the answer comes from
the disassembly or from runtime (`dynamic-frida.md`).
The division of labour is worth stating plainly: **ASC locates, `ddc` reads.** Running a full `ddc`
export first, then grepping it, is the slow path this tool exists to replace.
**Measured in the extension pass** (droidasc `0.1.1.post1`, installed with `python -m pip install
droidasc`; dependency is androguard only, no JVM, no SDK):
| Command | Result on a 38.7 MB hardened APK |
|---|---|
| `getmanifest` | 0.44 s → 32,345 B of complete XML |
| `findrefs … string jiagu` | 0.42 s → hits inside the shell class's `<clinit>` and `attachBaseContext` |
| `getclass com/stub/StubApp` | 0.33 s → 22,802 B of Java |
| `listclass` (whole APK) | 0.18 s → **4 classes** |
Two facts from that run are worth carrying into any recon: the CLI name is **`droidasc`**, not `asc`
(there is no `asc` binary, and `asc --version` is simply "command not found" — `python -m droidasc`
also works), and on a packed APK a full `listclass` returning four classes is not a broken tool: the
business dex is not in the file. Reaching for `getclass` on a class that is not there exits **1** with
`Class … not found in APK.` — treat ASC as a manifest-and-shell inspector on hardened input, and take
the business-logic question to a different layer.
### ddc — dex-to-Java with query subcommands (worth adopting)
A single-file Rust binary, no install, no JVM. Two things make it more useful than
"a decompiler":
```bash
ddc info app.apk # label, package, version, launcher, sdk, per-dex counts
ddc findrefs app.apk string "SomeToken" # every reference site, class#method
ddc strings app.apk -f "splash" --with-locations
ddc app.apk -c com.example.Foo # one class as Java
ddc app.apk -o out/ # full decompile, a few seconds for a normal APK
```
Measured on a 4.7 MB APK (4,070 classes / 30k methods): `info` ~0.13 s, `findrefs`
~0.11 s, one class ~0.25 s, full decompile ~2.8 s producing ~4,000 files.
**Measured in the extension pass** (ddc `0.1.8`, a single Rust binary — on Windows `bin/ddc.exe`,
1,028,096 B, **not on PATH** and carrying no PE version resource, so `ddc -V` is the only way to read
its version; source: the project's GitHub releases):
| Command | Result on a 38.7 MB hardened APK |
|---|---|
| `ddc info` | 0.24 s → label / package / version / application / launcher / sdk / size / md5, plus per-dex class, method, field and string counts |
| `ddc findrefs <apk> string jiagu` | 0.13 s → tabulated `dex | kind | class | method | refs` |
| `ddc strings -f jiagu --with-locations` | 0.14 s → strings mapped to the methods that hold them |
**The trap on this one is the exit code.** Asked for a class that is not present, `ddc` exits **2**
*and still prints its help text to stdout* — so a wrapper that tests "did anything come back?" reads a
usage error as an answer. Branch on the exit code, never on output emptiness. The same run also
confirms the layered picture from the other direction: `info`'s per-dex counters report 4 classes and
29 methods with code for the whole 8.9 MB shell dex.
Why the query subcommands matter more than the speed: **string cross-referencing
becomes a lookup instead of a crawl.** Asking "which class mentions this config
field name" is a sub-second command here and easily an hour of smali grepping by
hand. That single property is what turns "find the convergence point" from
exploration into a query.
`info` also **prevents a specific, costly mistake**: reading the package identity
out of the binary manifest by hand. Hand-extracted AXML strings are ambiguous —
class-name fragments look exactly like package names, and picking the wrong one
sends every later `pm`/`dumpsys`/data-dir query to a package that does not exist.
Let the tool report `package`, `label` and `launcher`, and cross-check with
`aapt2 dump badging` when available.
Limits to plan around: types are erased (no generics); R8 short names stay short,
so you still need string cross-references to infer meaning; output is a **reading
aid, not compilable source** (occasional declaration/use ordering is inverted, and
numeric resource ids appear as decimal integers); very large methods produce
thousands of lines and are better approached via `findrefs`/`getmethod` first.
Two more, both measured on a 28,210-class R8-flattened Flutter APK:
- **`-c <FQCN>` cannot reach a renamed class, and `listclasses <pattern>` cannot recover it either.**
Asking for the plugin's documented name returned `class not found`; asking for its package prefix
returned only `R$anim` / `R$string` / `R$style` resources. The implementing class existed under a
short R8 name, reachable only by string cross-reference. **Plan for `droidasc` to close this gap** —
it is not a nicety, it is the difference between locating the class and not.
- **A decompiled body can be semantically wrong, not merely ugly.** In one 9 KB helper class the
compiler emitted a `return 0;` inside a method declared to return `String`, reordered field
assignments, and produced empty `if/else` branches. None of that is safe to reason *about* control
flow without checking smali or bytecode. Treat a surprising construct as a decompiler artifact until
the bytecode agrees.
**Use `findrefs` to locate, `-c`/`getmethod` to read, and the bytecode to decide.** Reading order
matters: a full `ddc <apk> -o out/` export is a reasonable thing to *have*, but grepping that tree is
not a substitute for one indexed query, and it is the slower of the two by a wide margin on any APK
worth analyzing.
**Classpath gotcha.** `baksmali`/`smali` need their dependency jars on the classpath together
(`smali`, `antlr-runtime`, `stringtemplate`, `baksmali`, `dexlib2`, `util`, `jcommander`, `guava`).
Missing one produces `ClassNotFoundException` that reads like a broken target. `scripts/smtool.py`
carries the classpath so you pass it once.
**Entry-point gotcha.** The main class has moved between versions — `org.jf.*` in older builds,
`com.android.tools.smali.*` in newer ones. If `ClassNotFoundException` names the **main class**, it
is a version mismatch, not a missing jar.
**jadx is for reading, not for concluding.** Decompiled Java reorders and rewrites control flow;
line numbers and even which branch is which can differ from the bytecode. When a decision depends on
it, verify in smali or at runtime. `jadx` writing `(unknown)`/empty method bodies means it failed,
not that the method is empty.
**`smali` assembling can abort silently** — the tree produces no dex while the process still exits
0. Never treat exit code as proof of a build. Assert the artifact exists, is fresh, and is non-empty
(`patch-audit.md`).
## Tier 2 — native
| Tool | Invoke | Notes |
|---|---|---|
| `radare2` / `rizin` | `r2 -q -c '<cmds>' file` | scriptable disassembly/analysis; the practical choice when no GUI is available |
| `Ghidra` (headless) | `analyzeHeadless <proj> <name> -import <file> -postScript <s>` | decompiler without a GUI; slower to script but far better output than a raw disassembler |
| `capstone` (python) | library | decoding in your own scripts — see the silent-stop trap below |
| **`svc_scan.py`** | `python scripts/svc_scan.py libfoo.so [--context 3]` | **names the syscall behind an inline `svc`** and reports which PT_LOAD segment each hit is in. Use it *before* planning a libc-level hook: a module with its own `exit`/`kill` `svc` sites is not observable through libc, and one without them means a missing event is a finding about your hook. Read the neighbours (`--context`) — a byte scan for `svc` also matches inside data. Measured on the test device: device `libc.so` = 4 termination sites (its own exports), a shell library's 21 hits = all data |
| `keystone` (python) | library | assembling a short patch when you are editing bytes by hand |
| `pyelftools` (python) | library | **section-based — unreliable on hardened targets.** Prefer hand-walking `PT_LOAD`/`PT_DYNAMIC`, as `scripts/elf_plt.py` does |
| IDA Pro | GUI, **or headless via `idalib-mcp`** | the strongest decompiler output. Do not assume it needs a human in front of it — a headless MCP server exists (see *MCP tool servers* below). Ask for a human only when that setup is genuinely unavailable |
| `x64dbg` | GUI, Windows | human-driven live debugging of a native Windows target |
| `gdb` / `lldb` | CLI | live debugging where a device or emulator allows it |
**capstone can stop silently.** Decoding a buffer that does not begin at an instruction boundary may
return a few instructions and then nothing, with no error. A scan built on that reports "no matches"
for a library full of matches. Use byte-pattern search or a resynchronising scan for anything where
completeness matters (`native-tamper-and-suicide.md` §Scanner traps).
**Disassembler output is a hypothesis.** Fixed-width architectures (aarch64) decode almost any
4-byte window into *some* instruction, so a wrong start offset yields plausible-looking garbage.
Bound your window with a known entry point or a known call site.
**A decoder is a decoder, not an oracle.** Two independent decoders agreeing is the signal worth
having (measured: a hand-written word scan for the `svc` encoding and the `capstone`-based scan in
`svc_scan.py` returned the identical 214-site set on a device `linker64`, set difference empty); one
decoder's disagreement with itself is not. **`dexdump`** (from build-tools, e.g.
`E:\tools\android-14\dexdump.exe`) is the cheap independent reader for the dex half of this: use it
to count a dump's class and method records rather than trusting a self-written parser, the same way
`advanced-unpacking.md` §Establish "the bodies do not decode" with a decoder you did not write
requires.
### Ghidra — the decompiler for a target whose decisive layer is aarch64
The recurring shape: the app is fine, the dex is readable, and the layer that actually decides
behaviour is a stripped aarch64 `.so`. A disassembler gives you instructions; only a decompiler gives
you control flow. **Hand-walking an OLLVM control-flow-flattened function with `capstone` is the single
most expensive route available**, and it is what you fall into when no decompiler is installed.
Ghidra is free, runs headless, and decompiles aarch64. Setup is unzip plus a JDK (11+), which is why
"there is no decompiler on this box" is normally an install step rather than a finding about the target
(§Closing a capability gap).
```bash
# one-time: unzip the release; point Ghidra at a JDK (JAVA_HOME or the launch script)
# headless: import, auto-analyse, run a post-script, discard the project
analyzeHeadless <proj_dir> <proj_name> -import target.so \
-postScript DumpFunction.java <addr_or_symbol> -scriptPath <dir> -deleteProject
```
Three caveats learned the hard way:
- **`analyzeHeadless` is slow and its auto-analysis is not free.** Import plus analysis of a 6 MB
stripped library is minutes, so it is exactly the kind of run that needs an explicit timeout. Budget
it once and reuse the project; do not re-import per question.
- **Flattened control flow defeats the decompiler as well.** A `br x8` dispatcher with four constant
arithmetic operations in every function header is OLLVM flattening, and the output is a state machine
that resembles the program without being readable. **Removing the flattening is the work**
(`code-virtualization-and-custom-linkers.md`), not a prerequisite you can skip.
- **A decompiler cannot go where the data is not.** When a library's strings and action names are
**encrypted in the file and exist only once the module is mapped in memory**, no static tool recovers
them by reading harder — escalate to runtime (`dynamic-frida.md`) instead of escalating the static
toolchain. Telling "install a decompiler" apart from "the decompiler cannot help here" is worth real
time: the two look identical from the outside, and only the first has a static answer.
## Closing a capability gap — installing the tool IS the task
A missing tool for the layer you must work in is not a constraint to design around. It is the next
step. The measured cost of installation is almost always smaller than the cost of the workaround, and
the workaround is what produces "we could not determine X" reports on targets where X was decidable.
**The decision is not "can I do without it" — it is "how long does installing it take".** Three real
numbers, from one Windows box working an R8-flattened Flutter target:
| Gap | Install cost | What the workaround would have cost |
|---|---|---|
| No whole-APK cross-reference index | `pip install droidasc` — one command, seconds, no JVM, no SDK | hours of `grep` over a hand-exported smali tree, **per question asked** |
| No arm64 decompiler | Ghidra — unzip plus the JDK already present; minutes | a manual ELF/`capstone` walk over an OLLVM control-flow-flattened 6 MB library, per function |
| No Dart AOT snapshot front end | `aotopsy` — unzip and run (pure Go, no toolchain); or `blutter`, measured **≈78 s** end to end | "pool offsets can never be mapped", which turned out to be false |
The third row is the instructive one. The documented cost ("tens of minutes for the first build") was
about **30× pessimistic**, and the documented toolchain requirement was overstated. **A tool's stated
prerequisites are a claim, not a measurement**, and the cost of checking is one attempt.
Rules that follow:
1. **Write the capability down before spending against it.** "arm64 decompilation — absent — install
Ghidra" is a task item. "arm64 decompilation — absent" with nothing after it is how a route gets
written off for the wrong reason.
2. **Check off-PATH before declaring anything absent** (§"not on PATH" is not "not installed").
`APKREV_TOOLS` exists so a project-local tools directory or an SDK folder resolves.
3. **Install, do not substitute.** A weaker tool's output presented as equivalent to the stronger
tool's output is a wrong conclusion with a clean audit trail — worse than an admitted gap.
4. **Report the gap only when it is real.** "No arm64 decompiler exists here and it cannot be installed
because `<reason>`" is a valid finding. "I used `readelf` instead" is a different statement, and it
is not a finding about the target.
5. **"Paid" and "GUI-only" are questions, not walls.** IDA has a headless MCP path; Ghidra has
`analyzeHeadless`; both are set-up steps. Ask a human only after that route is genuinely
unavailable.
6. **One real disqualifier.** A decompiler cannot go where the data is not: if a target's strings and
action names are **encrypted in the file and only exist in memory**, then no static tool recovers
them by reading harder. Recognise that shape early and move the effort to runtime
(`dynamic-frida.md`) instead of escalating the static tools — that distinction is the difference
between "install a decompiler" and "the decompiler cannot help here", and they look identical from
the outside.
## Where to get a tool we do not ship — a sourced gap list, not a link dump
The list below exists because "install the tool" needs a *source*, and searching for one on a target's
clock is how a ten-minute install becomes an hour. **Two grades are used here and they are different
claims:** `URL verified` means the repository was resolved on the date given; `tool unverified` means
nobody has run it here. Most rows are `URL verified / tool unverified` — treat them as leads with a
date on them, not as recommended tools, and check the row's own prerequisites when you install it
(§Closing a capability gap — installing the tool IS the task). Rows marked `measured` have been used here.
| Gap | Source | Grade / what is actually known |
|---|---|---|
| Decompiler for dex/Java | `https://github.com/skylot/jadx` | URL verified 2026-09-21; tool unverified here (not installed). The kit's route is `droidasc`/`ddc` for queries and jadx only to *read* a class already located |
| Repacking beyond `scripts/repack.py` | `https://github.com/iBotPeaches/Apktool` | URL verified 2026-09-21; tool unverified. Not installed on this host, and the `apktool.bat` in `E:\tools\bin` is a dead link — do not trust a `.bat` to mean the tool works |
| smali round-trip outside the bundled dexlib2 kit | `https://github.com/JesusFreke/smali` | URL verified 2026-09-21, **last upstream push 2024-01-17** — maintenance status is a fact to check before adopting it for a new dex version |
| Anti-detection on the frida side (patched server plus a script surface aimed at RASP) | `https://github.com/CrackerCat/strongR-frida-android` | URL verified 2026-09-21; tool unverified. A patched `frida-server` is an **environment** change: record it in the task record, because results obtained under it are not comparable to a run without it |
| A dex dumper that does not use `ptrace` | `https://github.com/index-login/MobileRE-Skill` (`.kilo/skill/rev-dex-dumper/`) — `panda-dex-dumper`, `mem-dex-dumper` + C source | **Partly measured**: both ELF images were inspected here and neither imports a `ptrace` symbol (`panda` = aarch64 `ET_DYN`, 48 symbols; `mem` = aarch64 static, stripped), so the *claim* is consistent at the symbol layer; the **dump behaviour was not run here**. Our own `/proc/<pid>/mem` read is `advanced-unpacking.md`'s root-side route and is measured |
| eBPF-based dex extraction | `https://github.com/LLeavesG/eBPFDexDumper` | URL verified 2026-09-21; tool unverified, **and unusable on `<DEVICE>` anyway** — kernel 4.14 against a 5.10+ requirement (`kernel-and-environment-hardening.md`). Listed so the next reader does not re-derive the version gate from scratch |
| Decompilation/deobfuscation/unpacking without a JVM | `https://github.com/adam-040/Enigma`, `https://github.com/1-3-7/disrobe` | URLs verified 2026-09-21; tools unverified. Both are interesting precisely because they remove the JVM/IDE dependency a Ghidra or IDA route carries — evaluate prerequisites before adopting |
| Java2C generation to *build* a fixture | `https://github.com/amimo/dcc` | **Partly measured here**: `dcc` itself was run in a previous pass, but the C compile needs an NDK this host does not have, so no Java2C artifact was ever produced (`java2c-and-jni-sinking.md`) |
| Building an LSPosed module without gradle, or comparing against a maintained template | `https://github.com/Jordan231111/lsposed-universal-template` | URL verified 2026-09-21; tool unverified. Its layout is gradle-based, which this host cannot build — useful as a **manifest/scope reference** only. (`mabbcoll13/xposed-module-kit` no longer resolves: HTTP 404 on 2026-09-21, which is why this row's date is worth writing down) |
| Dart AOT snapshot front end | see the §Closing a capability gap table above | measured here: `aotopsy` runs with no toolchain; `blutter` measured ≈78 s end to end on this host |
Two habits that keep this list from rotting: **write the access date next to a URL** (a dead link with
no date reads as a current recommendation — one row above is already 404), and **promote a row only
when someone runs the tool**, so `URL verified` never silently becomes "we use this".
## MCP tool servers — an external dependency, not a tool on the shelf
An MCP server is neither a CLI tool nor one of the scripts here. It is a **separate installation plus
a running process**: something to install, a service to start, usually an extension to load, often an
authorization step. Listing one beside `readelf` would imply it is already present. Declare it as a
dependency with its prerequisites, and check whether it is actually available *before* planning work
that needs it — discovering this mid-task is the "missing tool" failure again.
### IDA Pro, headless
IDA does **not** require a human at a GUI. `mrexodia/ida-pro-mcp` ships `idalib-mcp`, a headless MCP
server that drives an IDA database with no GUI process at all:
```sh
uv run idalib-mcp --host 127.0.0.1 --port 8745 path/to/executable # open a binary up front
uv run idalib-mcp --host 127.0.0.1 --port 8745 # open databases on demand
uv run idalib-mcp --stdio # for stdio-based clients
```
Prerequisites, all of which must be arranged before the first call: IDA Pro 8.3+ (9 recommended;
**IDA Free is not supported**), a Python 3.11+ that `idapyswitch` can select, `uv`, and an `idalib`
activated globally via `py-activate-idalib.py`. Each open database lives in a worker process that
outlives the supervisor and is adopted transparently by a later supervisor on the same host, so
several sessions can share one analysis. Every tool call carries an explicit `database` argument —
there is no implicit "current database" — and `idb_open` returns the session id you must pass.
The GUI-plugin variant of the same project is deprecated upstream in favour of `idalib-mcp`; do not
set that up for agent work.
### Ghidra
`LaurieWired/GhidraMCP` is a Ghidra **extension** plus a separate Python bridge, not a headless
analysis server: Ghidra must be running with the plugin loaded (its HTTP server defaults to
`127.0.0.1:8080`) and the bridge is another process your client starts. Treat it as requiring an
interactive GUI session. For unattended work the shape that actually runs without a GUI is
`analyzeHeadless` with a post-script.
### The boundary: deobfuscate before you hand the binary to a model
Both are LLM-driven, and they inherit the model's weaknesses. The IDA MCP authors say it plainly in
their own README: **"LLMs will not perform well on obfuscated code"**, and they advise removing
string encryption, import hashing, control flow flattening, code encryption and anti-decompilation
tricks *before* asking a model to solve anything.
That matters more here than in most domains, because it is precisely the shape of the targets this
skill deals with. An OLLVM-style or virtualized binary is not a job for a model reading decompiler
output; the deobfuscated version is a *different binary*, and producing it is the work already
described in `code-virtualization-and-custom-linkers.md` and
`native-tamper-and-suicide.md`. The same advice has a second half: resolve library code first
(FLIRT/Lumina) so the model is not asked to reason about `memcpy` as if it were the program.
Two consequences worth carrying into the plan:
- Ask a model for **navigation and reading** — rename, comment, summarise a function, convert an
immediate — and verify anything that decides the patch against the disassembly yourself.
- If the server is not installed, that is a **missing dependency to report**, not a licence to present
a weaker tool's output as equivalent (`packers.md`).
## Tier 3 — runtime and network
| Tool | Invoke | Notes |
|---|---|---|
| `frida` (host) + `frida-server` (device) | `frida -U -f <pkg> -l script.js` | **host and device versions must match** — skew produces errors that look like a broken target (`pitfalls.md` P15) |
| `objection` | `objection -g <pkg> explore` | quick Java-layer poking on top of Frida |
| `mitmproxy` | `mitmproxy --mode regular` | request/response inspection; a device proxy is usually faster to set up than a transparent one |
| `HaE` (Burp extension) | load into Burp, then read its rule set | **traffic *triage*, not traffic capture.** See below |
| `adb` | everything | the primary device interface |
**HaE is a reader, not a capture tool, and it is not always the right next step.** HaE (Highlighter and
Extractor, `overspace-labs/HaENet`) is a Burp Suite extension that tags and extracts fields from HTTP
messages — tokens, IDs, secrets, endpoints, fingerprints — so that a large session becomes readable
without hand-scanning every request. It is genuinely useful when the bottleneck is *"there is traffic
and I cannot see the parts that matter."*
It is the wrong tool when the bottleneck is one of these three, and reaching for it there costs rounds:
- **The requests never leave the client.** No capture tool can display a request that was never built
(`code-virtualization-and-custom-linkers.md` §what the native check actually reads). **Establish
whether traffic exists before buying tooling to read it** — a DNS/`connect`-level probe answers that
in a single run, and on the measured target it was the observation that ended the investigation
branch, not a capture.
- **The body is encrypted by the app**, so the capture is ciphertext (`server-api.md`).
- **Certificate pinning defeats the proxy**, in which case the empty capture is itself the finding
(`tls-and-cert.md`).
Ordering that follows: **first prove there is readable traffic, then choose the reader.** `mitmproxy`
captures; HaE triages; neither substitutes for the DNS/connect check that decides whether there is
anything to triage. Treat "install a Burp plugin" as a step that must be justified by an existing
capture, not as recon.
Version alignment for Frida is a **hard gate**, not a nicety. Check both sides before writing a
script, and check the device architecture — the server binary is per-ABI.
## Tier 4 — cross-platform runtimes
| Runtime | Identifier | Tool |
|---|---|---|
| Flutter / Dart | `libflutter.so` + `libapp.so` | `blutter` (needs the **matching Dart version**) — `dart-aot.md` |
| Unity / IL2CPP | `libil2cpp.so` + `global-metadata.dat` | Il2CppDumper + a native decompiler |
| React Native | `libhermes.so` / `index.android.bundle` | Hermes bytecode tooling, or plain JS if the bundle is unminified |
| Cordova / hybrid | WebView + `assets/www` | read the JS directly; Java layer is usually thin |
**A version mismatch here wastes the most time of any tool in this file.** A Dart decompiler built
for a different engine version produces output that is subtly wrong rather than obviously broken.
Pin the version first (`dart-aot.md`) and do not "try it and see".
## Using the kit's scripts instead of writing your own
The scripts exist so that the *expensive, generic* parts of this work — decoding a dex instruction at an
exact offset, recomputing a dex header in the right order, writing a 4-byte-aligned archive, resolving a
PLT stub, bursting screenshots with evidence attached — are not re-derived per task.
Re-deriving them is not neutral. A hand-rolled instruction decode that silently loses sync, or a
hand-rolled repack that quietly drops alignment, produces **a confident wrong answer**, which is the
exact failure class this skill exists to prevent. The cost shows up later, as a patch that "had no
effect" or an install refused with a bare numeric code.
**The rule: before writing a script, check whether one here already does it.** The index is in
`SKILL.md`; the entry points that matter most are `doctor.py` (what can run here),
`dex_find_insn.py` (get an exact offset instead of guessing one), `dex_patch_bytes.py` (apply and prove
an equal-length patch), `apk_diff.py` (prove your own build was surgical), `repack.py` (aligned rebuild)
and `snap.py` / `coldstart.py` (look at the screen with the evidence attached).
**When a script here is genuinely wrong, that is a finding to fix and report — not a reason to quietly
route around it.** Two live examples of the difference:
- A decoder utility in this kit carried an **opcode name/width table misaligned by one entry across
`0x16`–`0x2C`** (23 rows). Everything decoding through that range **silently lost sync** and produced
offsets that were plausible and wrong — no exception, no warning. It was found by comparing against an
independent decoder and noticing a disagreement. **If two tools disagree, the disagreement is the
product**; it is the cheapest bug signal in this domain and the easiest to throw away by picking the
answer you preferred.
- Conversely, a *tool's* negative result is not a finding about the target (§the general trap below).
Establish that the tool could have found the thing before recording that the thing is absent —
especially for string searches, where an empty result is exactly what an unfound string looks like.
**Corollary for reporting.** "The kit's script X reported Y, and I cross-checked it against Z" is a
conclusion. "I wrote my own parser because the script was awkward" is a gap you introduced, and it
should be reported as one.
## "Not on PATH" is not "not installed"
`doctor.py` and `preflight.py` report whether each tool resolves on `PATH`. That
is a statement about `PATH`, not about the machine, and the difference has cost
real time: a full custom zip writer was built to work around a missing
`apksigner` that was already installed in an SDK directory nothing had added to
`PATH`.
**Before treating a tool as absent, search the filesystem once:**
```bash
# POSIX
find / -name apksigner -o -name zipalign -o -name baksmali.jar 2>/dev/null | head
# Windows (PowerShell) -- list every drive letter first, then search each
(Get-PSDrive -PSProvider FileSystem).Root |
ForEach-Object { Get-ChildItem -Path $_ -Recurse -Depth 5 -ErrorAction SilentlyContinue `
-Include apksigner.bat,zipalign.exe,keytool.exe }
```
Common places that are *not* on `PATH`:
| Artefact | Typical location |
|---|---|
| `apksigner`, `zipalign`, `aapt2` | any `build-tools/<ver>/` under an Android SDK, and portable tool bundles |
| `apksigner.jar` | inside those same directories (runnable as `java -jar`) |
| `keytool`, `jarsigner`, `javac` | a JDK install whose `bin` was never added to `PATH`; `java -XshowSettings:properties -version 2>&1 \| grep java.home` reveals the JDK root even when `keytool` does not resolve |
| `uber-apk-signer.jar`, `baksmali*.jar` | bundled with portable APK toolkits |
Record the resolved absolute paths once, in your running notes, and pass them to
the scripts that accept `--apksigner` / `--zipalign` / `--ks` style flags. Every
script in this repository takes an explicit path for this reason.
**A useful asymmetry:** missing `apksigner` is survivable (a v2-only signature
block can be appended without reordering the archive), but missing `zipalign` is
not a reason to hand-roll alignment either -- write the archive aligned in the
first place, as `scripts/repack.py` does. Prefer producing a correct archive over
repairing one afterwards: post-hoc alignment tools rewrite the file, which
changes every offset and can break a target that fingerprints its own layout.
## Signing: pick the signer deliberately
Two signers, and the choice changes the output bytes:
- **`apksigner`** (build-tools) appends the v2/v3 signature block after the zip
content. It does **not** reorder entries, so a carefully aligned archive stays
aligned. This is the default choice.
- **`jarsigner`** (JDK) rewrites the archive as a side effect of adding the v1 JAR
signature, which recompresses entries and **destroys the alignment** that
Android R+ requires. A build signed this way can fail to install with
`Failure [-124] ... resources.arsc ... aligned on a 4-byte boundary` even though
the same archive installed fine moments earlier, and `jarsigner -verify` reports
success. If you have run `jarsigner` on an archive and a previously-working
install starts failing, re-pack and sign with `apksigner` before investigating
anything else.
Enable **v1 + v2 + v3**. v1 keeps very old devices working; v2/v3 are what modern
platforms actually verify, and v1-only builds are rejected in more configurations
than people expect.
## The general trap: a tool's failure is not a finding about the target
A missing jar, a version skew, a forged section header, and a genuinely absent symbol all produce
**empty or partial output**. The output looks the same; the conclusions are completely different.
Before reporting "there is nothing there", confirm the tool was actually capable of finding it:
- Did it read the file at all? (entry count, segment count, non-zero output somewhere)
- Does a **known-present** item show up? (search for something you already know exists — a symbol
you saw in the dynamic table, a string you read in the hex dump)
- Would a different method see it? (byte pattern vs decoded scan)
**This is `pitfalls.md` P25 and P26 in tool form.** A self-built analyser that fails quietly reads
exactly like a clean target, and it removes routes from consideration for free.
## Working without a network
Assume nothing can be downloaded. What still works, in order of usefulness:
1. Tier 0 in full, plus the Python standard library — enough for structure, strings, hashing, and
most ELF/dex parsing.
2. Whatever jars and binaries are already present. **Locate them once and reuse the paths** rather
than re-searching on every command.
3. A local package cache or an offline mirror, if the environment has one.
If a tool genuinely cannot be obtained, say which step is blocked and **what a substitute would
give up** — do not silently substitute a weaker method and report its result as equivalent.
## Recording the toolchain actually used
When you report, name the tools and versions that produced each finding. Two agents with different
tool versions will reach different conclusions about the same binary, and without the version that
difference is unresolvable later.

243
vendor-new/apk-reverse/skills/apk-reverse/references/updates-and-forced-upgrade.md Normal file
View File

@@ -0,0 +1,243 @@
# Updates and forced upgrade
Load this whenever the deliverable is a patched build that must **keep working over time** — which is
almost always. A build that is correct on the day you ship it and dead a week later has failed, and the
cause is nearly always an update path you did not neutralise.
This is the file that decides whether a patch is *durable*, and it is cheap: the whole job is usually
one or two edits.
## Why this is not optional
Three independent failure modes, all of which present as "the mod stopped working" with no obvious link
to updates:
1. **Forced upgrade.** The app checks a version endpoint, sees a newer build, and blocks its UI behind a
non-dismissible dialog until the user installs the official (unpatched) package. Your patch is gone.
2. **Self-update install.** The app downloads an APK and hands it to the package installer. Because the
downloaded build is signed differently, the install either fails or — worse — succeeds and replaces
your work.
3. **Silent "compatibility" downgrade.** The server decides the client version is too old and starts
omitting fields or returning an error page. No local code changed, so nothing local looks broken.
A fourth case is a **resource / hot-update channel**: the app fetches a bundle (JS, a Dart/RN patch, a
config blob, a plugin APK) that can reintroduce the behaviour you removed. That is a separate code path
from version checking — handle it explicitly if the app has one (see §Step 5 below).
## Step 1: locate the check
There is usually exactly **one** update service. Find it before patching anything else.
**From the client's own vocabulary** (works on any runtime):
```
update / upgrade / new version / check version / force update
版本更新 / 检查更新 / 发现新版本 / 立即更新 / 强制更新
```
Search those in the app's string table and in class names; on heavy runtimes search the string pool,
remembering its encoding (see `framework-runtimes.md` §Locating logic without symbols — searching only one encoding
misses half your hits, and "nothing found" is usually the encoding, not an absence).
**Signals that a version check exists even if you cannot read the strings:**
- A network call early in startup whose response contains a version number, a download URL, and
sometimes a `force` / `must` / `minimum` flag.
- A stored "last update check time" preference (throttling a check implies one exists).
- The app ships the package installer path or requests `REQUEST_INSTALL_PACKAGES`.
- A permissions/manifest entry for an "unknown sources" install intent.
**Also look for the *coupling*.** Update checks are frequently run from the same startup routine as ad
loading and splash dismissal, and a blocking update dialog often **suspends those siblings** — you may
find logging that literally says so ("update dialog showing, skipping X"). That tells you both where the
check lives and that your patch must not leave the dialog state machine half-finished.
## Step 2: classify the shape
| Shape | How it behaves | Where to cut |
|---|---|---|
| **Optional update** | a dismissible prompt with a download button | suppressing the dialog is enough |
| **Forced update** | non-dismissible; blocks the app | must be cut at the decision, not the dialog |
| **Silent version gate** | no dialog; features degrade or the server 400s | client-side is *not* fixable — see §the honest answer |
| **Self-update installer** | downloads and installs an APK | cut the check, and ideally the install path too |
| **Hot-update / resource patch** | fetches a bundle that can restore behaviour | separate route; treat as a data channel |
Assume **forced** until you have evidence otherwise, because that is the case that costs the user the
most when missed.
## Step 3: patch the decision, not the dialog
**Prefer the earliest safe exit over every downstream surface.** Supressing the dialog only hides it;
the download may still start, and a later code path may still act on the "new version" verdict.
Two layers, in this order:
**Layer 1 — make the check itself a no-op.** Replace the entry of the update routine so it returns
immediately. Nothing is requested, nothing is compared, no dialog state is created. This is the root fix
and it removes the network call entirely, which also makes verification trivial (Step 4).
**Layer 2 — neutralise the comparison.** Somewhere the client compares the server version against its
own and branches on the result. Force that branch to the "already current" side. This catches any path
that reaches the comparison without going through the entry you patched (a second caller, a manual
"check for updates" button, a settings screen).
Layer 2 alone is a valid minimal fix if Layer 1 is awkward; Layer 1 alone is valid if you have confirmed
a single entry. **Doing both is cheap insurance and is what makes the claim "this build cannot be
upgraded into uselessness" defensible.**
```
target routine: <updateChecker>(...) -> return immediately (no request, no state)
target branch: <versionCompare> result -> force the "no update" side
```
**Do not** patch the dialog itself as your primary fix, and **do not** patch the comparison to lie in
the *other* direction (claiming an update when there is none) — that produces exactly the blocking
dialog you are trying to avoid.
### What not to touch
- **Do not rewrite the app's own version number** in the manifest or in a version string. It is the
cheapest-looking fix and it breaks things you are not looking at: server-side feature negotiation,
analytics, cached asset keys, and the app's own self-comparison. Change the *decision*, not the
identity.
- **Do not strip the installer permission** unless you have checked nothing else uses it.
- **Do not block the update hostname at the transport layer.** That is the same class of mistake as
blocking ad hosts (`pitfalls.md` P5): it fails offline/online transitions, needs a helper on the
machine, and is not a property of the artifact. It is also not required — the check is local code.
## Step 4: verify
Update paths are checked once at launch, so verification is a single clean cold start:
1. **Clear logs, cold start, and grep for the update vocabulary.** Expect **zero** hits. Not "quieter" —
absent. If the routine still runs, the string will still appear.
2. **Watch the screen through startup.** The failure mode is a modal, so a sample you did not look at is
not evidence (`environment.md` §look at the screen). A blocking dialog is unmistakable and it will be
on screen for a long time.
3. **Check the network.** With the check disabled there should be **no request** to the version/update
endpoint at startup. This is the strongest signal: it distinguishes "dialog suppressed" from "check
never happened".
4. **Exercise the manual path** if the app has a "check for updates" entry in settings — that is the
caller Layer 2 exists for.
5. **Confirm what you disabled is only updates.** The update routine sometimes shares a bootstrap with
other startup work. If you no-op'd a routine rather than its decision, verify that splash dismissal,
first-run setup, and the ad/consent flow still complete.
**Two-sample rule for the durability claim.** If you are asked whether the build *survives* a server-side
version bump, you cannot test that directly. What you can honestly claim is: *(a)* no version request is
issued at startup, and *(b)* the comparison no longer reaches its "newer" branch. State it in those terms
instead of asserting the future.
## Step 5: hot-update and resource channels
If the app can fetch a bundle at runtime, version checking is not the only way your change can be undone.
- **What to look for**: a startup fetch of a signed bundle/zip/config with a version or hash, a local
unpack directory, a dynamic plugin loader, or a JS/Dart/RN patch channel.
- **Why it matters**: even with updates disabled, a remotely fetched config can re-enable a feature, and
a fetched bundle can overwrite patched logic in the parts it owns.
- **Handle it like a data channel, not like an update**: identify what is fetched, whether the patched
behaviour is inside that payload, and cut the consumption point if it is. If the patched logic lives in
the shipped artifact and the fetched payload cannot override it, say so and move on — that is a
finding, not a gap.
Note the flip side: a **server-issued configuration** is the same category. If the thing you removed was
driven by remote config, the client patch holds only while the server keeps sending that config — see
`references/server-config-and-updates.md` §6 and `references/membership-and-limits.md` for the general
rule that *client-side enforcement can be patched, server-side authority cannot*.
## The honest answer for silent version gates
If the app degrades because the **server** decides the client is too old — no local dialog, just missing
data or errors on some screens — then there is no client-side fix. The only honest report is:
- the version the client reports to the server (with evidence of where it is read),
- the endpoint that makes the decision (with the observed response),
- and a clear statement that this is server-side authority.
Do not paper over it by forcing the version string at random read sites. If you do change a reported
version as an experiment, label it experimental, and remember that a version number is used in more
places than the one you are looking at.
## Delivery note: always state what was disabled
Update suppression changes behaviour a user can observe ("it never tells me about new versions"). Put it
in the delivery notes, in one line, together with anything else you neutralised. If the app also has a
legitimate reason to update (security fixes), say that the build will not pick them up automatically.
## Step 6: when the check is not where you expect
Three shapes that break the "find the Java comparison and flip it" plan. Recognise them early,
because each has a different cheapest remedy.
### The check is native
Symptom: you grep the whole dex tree for the version-comparison routine, the update endpoint, or the
dialog text and find **the method declared, and zero callers**. A `static native` update entry point
with no Java caller means the invocation lives in a `.so` (`registerNatives` from a static block is
the usual giveaway — see `code-virtualization-and-custom-linkers.md`).
Options, cheapest first:
1. **Attack the response, not the check.** Block or rewrite the update endpoint at the network layer,
or empty the version/config response. The native code still runs and still decides "no update".
This needs no native work at all and survives a rebuild — check this before reaching for a
disassembler.
2. **Send the check a version it likes.** If it compares against a value the app itself supplies
(a config key, a stored preference, a build field), setting that value is a one-line change.
3. **Patch the native comparison.** Last, because it is the most expensive and the most fragile.
Do not conclude "unpatchable" from "no Java caller". The gate usually reads a value you *can* control.
### The update hands off to a browser or a WebView
Very common in this category: the dialog's button fires an `ACTION_VIEW` intent at a download URL,
or loads it in an in-app WebView. Two consequences:
- **You will not find an installer call** in the app, so searching for `PackageInstaller`,
`REQUEST_INSTALL_PACKAGES` or a download service finds nothing. That is not evidence the dialog is
harmless; the browser does the install.
- **Logcat truncates the URL.** `ActivityTaskManager` prints `dat=https://host/...` and elides the
rest, so a naive grep gives you a host and no path. Do not build a plan on a truncated URL.
Ways to get the real URL, cheapest first:
| Approach | Notes |
|---|---|
| Hook `Intent` construction / `Context.startActivity` and log `intent.getDataString()` | Most reliable. A truncated log line is a *logging* limit, not a limit on what you can read |
| `dumpsys activity activities` / recents while the browser is still in the stack | Free; works only if the activity is still alive |
| The browser's own history database (root) | Survives the browser being closed; needs the right DB, and modern browsers may encrypt or prune it |
| Read the update *response* instead | Often easier: the URL came from a JSON field you can fetch or hook directly |
**Deliberately tapping the button is a legitimate probe** — but treat it as a measurement, not a
side effect: capture the intent (or the screen) in the same window, and know that on some devices a
second tap resumes an already-started download rather than re-issuing the intent.
### The gate is server-issued with a local fallback
`fetchEnabled()`-style helpers frequently **default to a local value on any failure** (non-200,
timeout, malformed body, signature mismatch). Two things follow:
- **Read the failure branch before you touch anything.** If failure ⇒ "no update" or "no ad", then
merely making the request fail is a complete fix, and it is far cheaper and more robust than
patching the comparison.
- **The fallback also tells you how to test offline.** Disconnect the device and relaunch: if the
gate disappears, the fallback is benign and you have a zero-code workaround as well as a
verification signal.
Conversely, if failure ⇒ "must update", failing the request makes things worse, and you must patch the
decision instead.
## Checklist
- [ ] Update vocabulary searched in **both** string encodings, in the right layer for the runtime
- [ ] The single update entry point identified (or its absence established, with evidence)
- [ ] Shape classified: optional / forced / silent / installer / hot-update
- [ ] Decision-level patch applied, not just the dialog
- [ ] Comparison branch neutralised as a second layer
- [ ] Manifest version and identity left alone
- [ ] Cold start: **zero** update log lines, no blocking modal, **no version request on the wire**
- [ ] Manual "check for updates" entry exercised
- [ ] Startup siblings (splash dismissal, ads, consent, first-run) still complete
- [ ] Hot-update / remote-config channel considered explicitly
- [ ] Delivery note states what is now disabled

131
vendor-new/apk-reverse/skills/apk-reverse/references/verification.md Normal file
View File

@@ -0,0 +1,131 @@
# Verification — what "done" means
The difference between a hobby patch and a usable deliverable is verification. "It assembles" and "the log is quiet" are not verification.
**Load this when:** you are about to claim the work is done, or you need to define done for a reviewer. It gives the claim ladder, the evidence chain, and what each deliverable form has to show before it counts.
## The claim ladder
Each rung is stronger evidence. Climb as high as the task requires, and **state clearly which rung you reached**.
| Rung | Claim | Evidence required |
|---|---|---|
| 0 | "The dex was edited" | Byte-level diff of the target method |
| 1 | "The structure is intact" | `dex_classdiff`: same classes, zero `ACC_INTERFACE` drift, unchanged classes byte-identical |
| 2 | "It builds and signs" | Signature verification passes |
| 3 | "It installs and launches" | Process alive, `logcat` free of fatal signatures |
| 4 | "The target behavior changed" | The specific feature observed changed (screen exercised, endpoint observed, UI confirmed) |
| 5 | "Nothing else regressed" | Adjacent features exercised: images, playback, lists, login, settings |
| 6 | "The mechanism is proven" | Independent evidence of *why* — e.g. the SDK's domains are never resolved because init never ran, or the SDK never created its working directory under the app's private storage |
| 7 | "The *distributed* artifact is the verified one" | Re-download the published file and compare its hash against the local build you tested |
**Rung 3 is the minimum for any deliverable. Rung 4 for a claim that the patch solved the problem. Rung 5 before handing it to a user. Rung 6 when the claim is "the subsystem is dead", not merely "the ad is hidden". Rung 7 whenever the deliverable leaves your machine — an upload can truncate, a build can be re-signed by a pipeline step, and the person downloading it has no way to know.**
## The control build rule
Before blaming a patch, build a control: **the same pipeline with zero patches**.
```bash
python scripts/repack.py --apk original.apk --dexdir <extracted> --out control.apk
```
- Control fails → your pipeline, the environment, or the device is at fault. Stop debugging the patch (`pitfalls.md` P9).
- Control passes, patched fails → the patch is implicated. Bisect: revert one dex at a time.
Do this **whenever something unexpectedly fails**, and at least once per project.
## Structural verification (automate it)
After every dex edit:
```bash
python scripts/dex_classdiff.py <original.dex> <patched.dex>
```
Expect exactly:
```
only_in_A=0 only_in_B=0
ACC_INTERFACE mismatch: 0
access_flags diff (same interface-ness): 0
```
Caveat, and it is an important one: **this check cannot detect code-item damage.** A whole-tree smali round-trip can pass every table check and still crash with `IncompatibleClassChangeError` (`pitfalls.md` P3). Passing this check is necessary, not sufficient.
The checks that *can* see further — instruction-length auditing, the equal-length-replacement blind spot, and verifier-level legality (`move-result*` adjacency) — are in `references/patch-audit.md`. Use them before claiming a patch landed.
Also confirm you changed what you intended and nothing more:
- Reverse the patched dex and inspect the target method.
- Compare unmodified classes byte-for-byte against the original where feasible.
## Runtime verification
```bash
adb -s <serial> logcat -c
adb -s <serial> shell "am start -n <pkg>/<activity>"
sleep 5; adb -s <serial> shell "pidof <pkg>" # empty == died
sleep 20; adb -s <serial> shell "pidof <pkg>" # same pid == stable
adb -s <serial> logcat -d -v brief | grep -E 'FATAL|VerifyError|IncompatibleClassChange|uncaughtException|Failure starting process'
```
**Sample the pid twice, spaced apart.** A single reading catches apps that die in a crash loop.
Failure signature → cause table lives in `references/environment.md`.
**Logcat is often not enough.** If the app installs a crash handler (友盟/UCrash/Bugly), the Java stack never reaches logcat. A process that dies with only `uncaughtException time: ...` and no stack **has crashed** — treat it as a failure, not as noise. Extraction techniques: `references/environment.md` §signal extraction.
## Behavioral verification (the part people skip)
Launching proves nothing about your change. Exercise it.
- **Ads removed** → open the screens that had ads (splash, home, detail, player, reward button). Confirm absence *and* confirm the screens still function. Use **three independent signals**, not one: the screens look right, the SDK's log tags disappeared, and the SDK never created its working directory under the app's private storage. An SDK that silently failed to fill an ad slot looks exactly like an SDK that was never initialised — only the last two signals tell them apart.
- **A gate patched** → walk the gated flow end to end.
- **A timeout/limit patched** → trigger it.
- **Data patched** → stop the app, clear/re-write the value, cold start, re-check (proves it is not an in-memory artifact).
**And then test the neighbours.** The most common real-world regression is a patch that fixed its target and quietly broke something unrelated:
- Images / cover art render?
- Video or audio playback works?
- Lists scroll and paginate?
- Login / session still valid?
- Settings screens open?
- Downloads still start?
This is exactly how a "generic image-card composable" patch removed all cover art while the ad still showed (`pitfalls.md` P6).
## Screenshot discipline
Capture before/after screenshots for each claim. Text descriptions are not evidence; a screenshot of the screen without the ad, alongside one from the control build with it, is.
## Reporting
```
Build: <path> <sha256>
Base: <original apk name/version>
Changes: <dex → what changed>, one line each
Structural: dex_classdiff result (per dex)
Signature: verified with --min-sdk-version 21 (v1/v2/v3); report the raw command, not just "ok"
Device: <model / android / abi / rooted?>
Runtime: pid stable over N s; no fatal signatures
Behavior: <screens exercised>, <before/after>
Regressions: <adjacent features checked>
Distribution: <where it was published> re-downloaded, sha256 matches the tested build
Rung reached: 0..7
Residual: <what is NOT fixed, and the exact reason>
```
**Be explicit about residuals.** "Ad X remains because it is delivered inside the screen's main data payload, and suppressing it at the transport layer takes the whole screen down" is a useful, honest result. Silently omitting it is not.
**Separate the two audiences.** The artifact goes to an end user; the analysis report goes to whoever asked for the work. User-facing notes stay neutral and practical — package name and version, install prerequisites, the signature-conflict warning (a different signer means the old install must be removed first), the checksum, and a scope/disclaimer line. Keep the mechanism, the patch points, and the reasoning in your own report, not in the file the end user opens.
## Anti-patterns
- Declaring success from a successful assembly.
- Claiming "no ads" from a quiet logcat when the SDK never printed anything either way.
- Concluding from a single launch without exercising the feature.
- Quoting emulator results as device results.
- Reporting a fixed feature without checking adjacent features.
- Hiding a known residual failure.
- Blaming your own patch for a failure the **unmodified original** also has — run the control build first.
- Accepting a UI automation result as proof when the interaction may never have fired: verify field contents and look for the actual request, not just the screen.
- Treating a feature-scoped network error as a patching problem before checking the server's certificate (`references/tls-and-cert.md`).

237
vendor-new/apk-reverse/skills/apk-reverse/references/vmp-differential-analysis.md Normal file
View File

@@ -0,0 +1,237 @@
# Differential hardening (the known-plaintext route to a private opcode table)
Load this when the dump is a **real Dex VMP** — bodies present, decoding as nonsense to a
dalvik disassembler, a native interpreter loop behind them — and you have decided the
private opcode space is worth attacking rather than walking away from. The neighbouring
file owns the diagnosis (`advanced-unpacking.md` §The honest boundary: real Dex VMP) and
the sentence that sends you here. This file owns what happens next, and — more of it than
is comfortable — what does not.
**Strength note, read before trusting any number here.** Every mechanism below was
exercised on locally built fixtures, and the closed loop is measured: `simulate` relabels
a fixture through a known bijection and `compare` re-derives that table **exactly, 218 of
218 emitted opcodes, zero wrong entries, zero fabricated entries** (`references/evidence-summary.md` §The capability matrix).
What was **not** exercised is the other half of the premise: **no third-party hardening
platform was contacted**. The assumption that a real engine performs a per-opcode
substitution at stable instruction length is **inferred** from public VMP write-ups and is
exactly the assumption this method can fail on. The cost table's platform column is
therefore reasoning from how those platforms are distributed, not from a measured upload.
## 1. The method, and the one thing it needs to be true
The hardened body cannot tell you where its instruction boundaries are: its opcodes are
private, so the format specification's length table does not apply to it. That is the
whole obstacle — a disassembler that cannot decide where instruction *n* ends cannot name
instruction *n+1*.
The differential dissolves it with an oracle you control. Compile a fixture whose every
instruction you know; get the same platform to harden it; then **take the original's
instruction boundaries and project them onto the hardened bytes**. Original offset
`0x14` was a `mul-int` occupying 2 code units, so hardened bytes `0x14..0x17` are the same
instruction, re-encoded. Read the hardened opcode byte at each projection and you have a
`(original opcode → private byte)` sample. Aggregate over thousands of instructions and
the substitution table falls out.
Three preconditions, and they are not negotiable:
| Precondition | Why it must hold | What it looks like when it fails |
|---|---|---|
| **The hardened body keeps the same length** | The projection is a stencil; a length change slides every boundary after the first edit | `compare` reports `resized` for the whole method and refuses the run |
| **The substitution is per-opcode, not per-method** | One private byte per original opcode is what makes the table a table | Every original opcode claims the same private byte (`compare` → `NOT-USABLE`, stub shape) |
| **The body is still in the dex** | Alignment needs two byte streams | Method has `code_off == 0` (`compare` reports `stripped`) |
The third is the one that kills the method most often in practice: a VMP that *extracts*
bodies to a native interpreter leaves nothing to align. The differential route then has no
input at all, and no amount of clever fixture design creates one. Check for it before
spending any time here.
## 2. Cost table — which links can be automated, and which cannot
This is the part the method's public descriptions leave out. The chain is six links; two
of them are unavoidably human, and pretending otherwise is how a "harness" ships that has
never once been run against a real target.
| # | Link | Automation | Where a human must stand | Time cost | Failure mode |
|---|---|---|---|---|---|
| 1 | Compile the coverage fixture into a dex + APK | **Full** — `scripts/vmp_diff_harness.py build`, measured | none | ~40 s on this machine (javac + d8 + aapt2), ~1 s of it the script | toolchain absent or version-drifted; `d8` rejects a non-existent `--output` directory, `aapt2 link` rejects `android:` attributes without `android.jar` |
| 2 | **Submit to the hardening platform** | **None. Not automatable.** | 100 % — a browser, an account, sometimes a queue and a review | minutes to **days**; the platform decides | **the binding constraint of the whole route.** No public API; per-account rate limits; uploads are frequently gated behind registration; the artifact often comes back as a download page rather than a file path. A synthetic fixture may simply be **refused** (no real app shape, no certificate, no package identity) |
| 3 | Retrieve the hardened output and extract its dex | Partial — if the output is an APK, `unzip` + `dexutil.load_dex` handles it | download and hand-off | seconds, once the file exists | output is an APK whose dex is *also* encrypted, or the platform returns only a repacked APK with a native loader and no readable dex |
| 4 | Derive the correspondence | **Full** — `compare`, measured end to end | none | **sub-second** on a 3,065-instruction fixture | alignment slips (see); engine is not a pure relabelling |
| 5 | Prove the derived table | Partial — the mechanical half is scripted and measured; the judgement is not | decide whether the held-out check was passed or merely not attempted | minutes | a table that is *self-consistent* but describes a stub () |
| 6 | Render Smali from the restored stream | Full — `emit-smali`, measured | none | sub-second | table has holes; restored stream decodes short of the method end |
**Conclusion, stated plainly: do not build a fully automatic pipeline.** Link 2 is
irreducibly manual — the platform has no interface a script can drive, and its output
cadence is not one a pipeline can wait on. The right shape is what this repository ships:
- links 1, 4 and 6 as **reliable local tools** (they are deterministic and were measured);
- link 5 as **a checklist plus one scripted test** () — the judgement stays human;
- link 2 as **a documented manual step**, with its cost stated up front, so nobody writes
an orchestrator that waits forever on a web form;
- and links 4–5 **self-verifiable without the platform at all**, which is what `simulate`
exists for. You can prove your comparison logic against a known table today; you cannot
prove the engine's behaviour until a human uploads a fixture.
The practical consequence: the deliverable of this route is an **analysis report with a
stated boundary**, not a pipeline. Say so before starting.
## 3. The coverage fixture — what a compiled probe can and cannot reach
A Java-compiled fixture reaches **218 of the 224 named opcodes in this repository's format
table (97.3 %)** — that number is measured, not estimated, and the six it misses are
structural rather than incidental. The reason to care is direct: **an opcode your fixture
never emits leaves no known plaintext, so its substitution is unreadable by construction.**
| Missed opcode | Why a javac + d8 fixture cannot reach it |
|---|---|
| `const-string/jumbo` (0x1b) | needs a `string_ids` index > 65535, i.e. a fixture shipping 65,536+ string constants |
| `goto/32` (0x2a) | needs a >32767-code-unit backward jump; javac refuses the method first with `error: code too large` — its own 64 KB per-method bytecode cap. **Measured** on a generated 33,000-statement loop |
| `const-method-handle` (0xfe), `const-method-type` (0xff) | no Java-language literal exists for these constants; reachable only via hand-written smali or direct dex construction |
| `move-object/16` (0x09) | needs **both** registers ≥ 256; the object frame pushes the source past 256 but d8 encodes that as `move-object/from16` |
| `invoke-custom/range` (0xfd) | d8 emitted the 35c form for every lambda shape tried, including a six-parameter one |
What the fixture **does** cover, by construction, and the design decisions that get it
there (each of these was a measured gap before it was closed):
- **Three-register forms.** javac emits the `/2addr` form whenever the destination is one
of the operands, so `sub-int`, `div-int`, `rem-int`, `and-int`, `or-int`, `xor-int`,
`shl-int`, `shr-int`, `ushr-int` and their long/float/double counterparts are **absent**
from ordinary code. Three distinct locals per operation is what forces them into the
stream; the `/2addr` forms then need the accumulator spelling. Both are wanted.
- **Literal widths.** `div-int/lit16` needs a divisor in 128..32767; `and-int/lit8` needs a
mask inside −128..127. Writing `& 255` produces `and-int/lit16` and silently skips the
lit8 form.
- **Per-type field access.** `iget-wide`, `iget-object`, `iget-boolean`, `iget-byte`,
`iget-char`, `iget-short` and the `sget` set all appear only when the load targets a
**fresh local**. Accumulating into an existing register is where d8 narrows the access.
- **`/range` invokes.** The 35c invoke forms carry at most five argument registers, so a
seven-register call (`this` plus six ints) is what makes the `/range` encoding the only
legal one.
- **Wide registers and long jumps** — `move/16`, `move-wide/16`, `move-object/from16`,
`goto/16` — have **no hand-written form at all**. They exist only past a register-number
(≥256) or branch-distance (>127 code units) threshold, so the generator emits bulk
source: 300 int locals, 140 long locals, 160 object locals, a 200-statement loop body.
Measured yield: `move/16` ×46, `move-wide/16` ×20, `move-object/from16` ×161, `goto/16` ×1.
- **Payloads and structure**: `packed-switch`/`sparse-switch` with their payload blocks,
`fill-array-data`, `filled-new-array/range`, try/catch ranges with `move-exception`,
`monitor-enter`/`monitor-exit`, `instance-of`/`check-cast`, and `invoke-custom` /
`invoke-polymorphic` (which is why `d8 --min-api 26 --no-desugaring` is the default —
desugaring replaces lambdas with synthetic classes and the two opcodes disappear).
**Report the coverage every time.** `build` prints it, `audit` recomputes it for any dex.
A differential run whose coverage is 60 % produces a table that is 40 % missing, and
nothing in the output will say so unless you look at this number.
## 4. Extracting the correspondence
`compare ORIGINAL HARDENED` does four things, and the first three are diagnostics that
decide whether the fourth is meaningful:
1. **Pair methods** by `(class, name, descriptor)` and classify each pair. `aligned` means
both sides have a `code_item` of **identical `insns_size`** — the only state in which
projection is legal. `resized` means the length moved. `stripped` means the hardened
method has `code_off == 0`. `missing`/`added` mean the method set itself moved.
2. **Project and sample.** For each aligned pair, walk the original's decoded instruction
list; for instruction at original offset *o*, read the hardened byte at
`hardened.insns_off + (o − original.insns_off)`. Record `(original opcode → hardened
byte)` with a count.
3. **Aggregate and audit the aggregate.** Per original opcode: one candidate byte and the
row is `high`; more than one and the row is a `conflict`. Then the **reverse
direction** — build `private byte → set of original opcodes` and flag every private byte
claimed by more than one original. This reverse pass is not a nicety; it is the only
thing that catches the stub shape ().
4. **Emit a run-level verdict**, because a table is dangerous without one: `usable`,
`partially-usable`, `not-usable`, or `not-applicable` with the reason. A `not-applicable`
run exits non-zero.
Unreadable opcodes are reported as `undetermined` with the reason — either the fixture
never emitted them () or the dex slot is unallocated. They are **not** guessed at.
## 5. Proving the table — the part that decides whether any of this is real
A candidate table is a hypothesis. Four checks, in increasing order of what they can
falsify:
**a. Closed-loop regression — measured, and the strongest available without a platform.**
`simulate` relabels a fixture's opcodes through a bijection whose ground truth it writes
out; `compare` must re-derive that table from the two dexes alone. Measured on this
repository's fixture: **218 rows derived, 218 of 218 emitted opcodes recovered, zero
entries wrong, zero entries fabricated, zero non-injective private bytes, verdict
`usable`.** Any change to the comparison logic that breaks this check is a regression, and
the check needs no platform, no device and no network.
**b. Injectivity — the check that catches a confident wrong answer.** Every private byte
must be claimed by exactly one original opcode. Measured against the stub shape (bodies
replaced by an equal-length `return-void` fill): per-opcode reading looks *perfect* — 218
rows, all `high`, zero conflicts — because every original opcode maps to the same byte.
Only the reverse direction exposes it: one private byte claimed by **218** originals →
`NOT-USABLE`. A table produced without this check would have been silently worthless.
**c. Held-out consistency.** Derive the table from one subset of methods, then use it to
disassemble a **different** subset that contributed nothing to the derivation. A correct
table restores the instructions such that the walk lands exactly on
`insns_off + insns_size × 2` and every branch target lands on an instruction boundary. A
wrong or incomplete table desynchronises and says so. `emit-smali`'s
`instructions restored` / `unmapped-opcode stops` counters are the readout: on the measured
fixture, 3,065 restored and 0 stops.
**d. Cross-check against a second, independent recovery.** The same private opcode space
observed through a *different* route — an in-memory dump of the original dex alongside the
hardened one, or the `/proc/<pid>/mem` route in
`advanced-unpacking.md` §Dumping when frida is refused — is the only evidence that the
platform's transformation is the one you assumed. **Not done here**; it requires a real
VMP sample and a real platform, and is the honest gap in this file.
Note what none of these checks establish: that the engine's substitution is *stable across
inputs*. Fixture-derived tables have been reported in public work to drift per
vendor-version and per hardening pass. Re-derive per target; never cache a table.
## 6. Generating Smali from a restored stream
`emit-smali HARDENED_DEX --table TABLE.json` restores opcode bytes through the table and
renders method bodies as an annotated smali **reading skeleton**:
- Boundaries come from the table itself — once a byte maps back to a standard opcode the
standard length table applies to that instruction, so the walk is self-bootstrapping.
A byte with no table entry stops that method's walk, and the count of such events is
printed as the honest coverage measure.
- `.class` / `.super` / `.source` / `.method` headers are real, with access flags decoded
from `access_flags`, so the skeleton is readable as smali rather than as a hex dump.
- Payload blocks (`packed-switch-payload` etc.) are stepped over by their own layout, not
treated as opcodes.
**It is explicitly not assembliable, and the output says so in its first ten lines.**
Recovered: instruction boundaries and opcode names. Not recovered: register numbers,
field/method/string indices as the engine rewrote them, and any register re-allocation —
**no map for those exists in this method at all.** Feeding the skeleton to a smali
assembler produces an empty method, which looks like success and is not.
## 7. When this route does not work — the honest retreat
| What `compare` reports | What it means | Do this |
|---|---|---|
| `verdict: not-applicable`, `stripped` > 0 | bodies left the dex for a native interpreter | the differential has no input. Stop. Return to `advanced-unpacking.md` §The honest boundary: real Dex VMP and price the island honestly |
| `verdict: not-applicable`, `resized` > 0 | the engine's stream has its own length table | the projection premise is false. Stop rather than guessing boundaries |
| `verdict: not-usable`, one private byte shared by many originals | bodies replaced by a common stub | the engine did not relabel per-opcode here. Stop |
| `verdict: partially-usable`, conflicts ≤ 30 % | partial relabelling, or alignment slipping on some methods | usable only for the `high` rows; say so and stop at the unreadable ones |
| `usable`, but coverage was low | a complete table over an incomplete opcode set | usable, with the uncovered opcodes named as unreadable |
And the retreat that outranks all of them: **the cheaper question is whether the behaviour
you need exists outside the virtualized island.** A hook at the method's Java boundary, an
emulated call to the routine rather than a reading of it
(`emulation-and-rpc.md`), or a server-side answer frequently reaches the goal without ever
paying for the island. Weeks of table recovery is the expensive way to ask a question a
boundary hook answers in an hour.
## 8. Decision summary
| Observation | Action | Where |
|---|---|---|
| Bodies present, decode as nonsense | Diagnose real VMP before anything else | `advanced-unpacking.md` §The honest boundary: real Dex VMP |
| Need the private opcode mapping | Build the coverage fixture, then submit it by hand | `scripts/vmp_diff_harness.py build`; §2 of this file |
| Asked to "automate the whole chain" | Say which link cannot be automated and what it costs | §2 — the platform step is manual, by construction |
| Fixture coverage below ~90 % | Add the missing opcode classes before submitting |, and the `audit` report |
| Two dexes in hand | Derive the candidate table, read the verdict first | `scripts/vmp_diff_harness.py compare` |
| About to trust a derived table | Run the closed loop, the injectivity check and a held-out subset | §5 — a/b/c are scripted |
| Table complete | Render the skeleton; expect operands, not registers | `scripts/vmp_diff_harness.py emit-smali` |
| `resized` / `stripped` / stub verdict | Stop and say so | §7 |

353
vendor-new/apk-reverse/skills/apk-reverse/scripts/anti_detect_probe.js Normal file
View File

@@ -0,0 +1,353 @@
/*
* anti_detect_probe.js -- an OBSERVER. It changes nothing.
*
* Purpose: answer "if a detector is running, what is it looking at?" without
* answering it with a patch. That division is deliberate and is the point of this
* file: an observation module must not also intervene (no exit_blocking, no
* Interceptor.replace of a detection function, no NOPing). A probe that patches
* while it watches produces a process whose behaviour you can no longer use as
* evidence about the original -- the death you prevented is exactly the signal you
* were there to date.
*
* What it reports, as one `RESULT=` block:
* - the mapped libraries, so a detection .so can be named rather than guessed
* - every open/openat/fopen/access/stat on a path, with the *caller module+offset*
* for the ones that look like environment probing (/proc, /sys, magisk, su,
* frida, xposed). The caller offset is the deliverable: it is what turns
* "something checks root" into "0x1cef8 in libX.so checks root".
* - kills: kill/tgkill/exit/exit_group, with caller, so a self-destruct is
* attributed to a module instead of to the platform
* - thread creation (pthread_create/clone) with the real entry point, because a
* polling detector thread is a different shape from an in-line check
* - dlopen/dlsym, so a runtime-resolved symbol is visible
* - the process's own view of "am I instrumented": TracerPid, frida-named maps,
* frida-named threads, and TCP listeners (a detector often reads these first)
*
* Load it with spawn (-f) to see the early phase, attach (-p) to see the steady
* state. On this repository's test ROM attach-by-name fails, so use --pid.
*
* Usage (from the kit's own driver):
* python scripts/run_probe.py scripts/anti_detect_probe.js <pid> --via usb --log probe.log
* Usage (raw frida, explicit pid because -n does not resolve on this ROM):
* frida -H 127.0.0.1:27099 -p <pid> -l scripts/anti_detect_probe.js
*
* Config comes from CONFIG below or from RPC:
* script.exports.config({ watchMs: 8000, pathFilter: '/proc' })
*/
var CONFIG = {
// How long to keep collecting before printing the RESULT block. 0 = print only
// on the stop() RPC, for an interactive session.
watchMs: 12000,
// Stream a heartbeat every N ms. This exists because of a measured failure mode:
// a detector that kills the process inside the watch window means a probe that
// only reports at the end reports nothing at all -- the one run that mattered
// produced no data. The heartbeat says "the script was still alive at T", which is
// what separates "the target died" from "my script never armed".
heartbeatMs: 1000,
// Stream a line the moment a category fires for the first time, so the last
// events before a death are on the wire already.
streamFirstHit: true,
// Substrings that mark a path access as environment probing. Everything else is
// still counted, but not individually reported (it would drown the interesting
// lines -- a busy app opens thousands of files).
pathFilter: ['/proc', '/sys', 'magisk', 'supersu', 'frida', 'xposed', 'gum-js',
'/data/local/tmp', 'busybox', 're.frida', 'linjector'],
// Report process-wide events for every thread (false) or only the main thread.
allThreads: true,
// Hard cap on reported lines per category, so a chatty target cannot flood the log.
maxLinesPerCategory: 60,
};
if (typeof CONFIG_OVERRIDE !== 'undefined' && CONFIG_OVERRIDE.anti_detect_probe) {
Object.assign(CONFIG, CONFIG_OVERRIDE.anti_detect_probe);
}
var counts = {};
var lines = {};
var started = Date.now();
function bump(cat, msg) {
counts[cat] = (counts[cat] || 0) + 1;
if (!lines[cat]) lines[cat] = [];
if (lines[cat].length < CONFIG.maxLinesPerCategory) lines[cat].push(msg);
if (CONFIG.streamFirstHit && counts[cat] === 1) {
// On the wire immediately: if the target dies a moment later, this is the only
// trace of the event that existed.
send({ type: 'LIVE', cat: cat, msg: msg, t: Date.now() - started });
}
}
function backtraceOwners(ctx) {
// Module+offset for every return address inside a mapped module. The offset is
// what a static tool needs; the module name is what tells a shell library from
// libc. The link register is read first because on arm64 it is where the caller
// of the hooked export lives, and it survives even when Thread.backtrace cannot
// unwind (Frida's unwinder needs the target's frames to be intact, which is not
// guaranteed in a constructor or a signal path).
try {
var owners = [];
var lr = null;
try { lr = ctx.lr; } catch (e) { lr = null; }
if (lr) { owners.push(describe(lr, 'lr')); }
var bt = Thread.backtrace(ctx, Backtracer.ACCURATE);
for (var i = 0; i < bt.length && owners.length < 5; i++) {
owners.push(describe(bt[i], 'bt' + i));
}
return owners.length ? owners : ['no-unwind'];
} catch (e) {
return ['bt-failed:' + e.message];
}
}
function describe(addr, tag) {
try {
var d = DebugSymbol.fromAddress(addr);
if (d && d.moduleName) {
var base = Module.findBaseAddress(d.moduleName);
var off = base ? addr.sub(base) : null;
return d.moduleName + (off ? '+0x' + off.toString(16) : '+?') + '(' + tag + ')';
}
if (d && d.name) return d.name + '(' + tag + ')';
return 'anon:' + addr + '(' + tag + ')';
} catch (e) {
return 'addr?' + addr;
}
}
function ownerString(ctx) {
// Takes the context explicitly. `this` inside a Frida Interceptor callback is not
// reliably set when the callback is an anonymous function expression, and the
// measured symptom of getting this wrong is a log full of
// "cannot read property 'context' of undefined" -- which reads like a hooking
// failure and is actually a JavaScript binding mistake. Callers pass `this.context`.
if (!ctx) return 'no-ctx';
return backtraceOwners(ctx).join(' | ');
}
function looksInteresting(p) {
for (var i = 0; i < CONFIG.pathFilter.length; i++) {
if (p.indexOf(CONFIG.pathFilter[i]) >= 0) return true;
}
return false;
}
function hookExport(modName, exp, cat, argIndex) {
var addr = Module.findExportByName(modName, exp);
if (addr === null) return false;
try {
Interceptor.attach(addr, {
onEnter: function (args) {
try {
var p = null;
if (argIndex >= 0) p = args[argIndex].readUtf8String();
var who = ownerString(this.context);
if (p === null) { bump(cat, exp + '() <- ' + who); return; }
if (looksInteresting(p)) {
bump(cat, exp + '("' + p + '") <- ' + who);
} else {
counts[cat + '_quiet'] = (counts[cat + '_quiet'] || 0) + 1;
}
} catch (e) { /* unreadable pointer: count it, never throw out of a hook */ }
}
});
return true;
} catch (e) {
bump('hook_errors', modName + '!' + exp + ': ' + e.message);
return false;
}
}
function hookTerminators() {
// A detector's self-destruct is often a direct libc call; the caller offset is
// what identifies it. Note: a module that issues `svc #0` itself will NOT show up
// here -- the absence of a report plus a death is itself the signal that a libc
// hook cannot see it (scripts/svc_scan.py decides that statically).
['exit', 'exit_group', '_exit', 'abort', 'kill', 'tgkill', 'raise', 'pthread_kill']
.forEach(function (fn) {
var a = Module.findExportByName('libc.so', fn);
if (a === null) return;
try {
Interceptor.attach(a, {
onEnter: function (args) {
bump('termination', fn + '(' + args.join(',') + ') <- ' +
ownerString(this.context));
}
});
} catch (e) { bump('hook_errors', fn + ': ' + e.message); }
});
}
function hookThreadCreation() {
var pc = Module.findExportByName('libc.so', 'pthread_create');
if (pc !== null) {
try {
Interceptor.attach(pc, {
onEnter: function (args) {
bump('thread_create', 'pthread_create entry=' + args[2] + ' <- ' +
ownerString(this.context));
}
});
} catch (e) { bump('hook_errors', 'pthread_create: ' + e.message); }
}
// clone() is how a detector bypasses the pthread wrapper; args[3] points at the
// argument block whose fifth word holds the real entry point on arm64.
var cl = Module.findExportByName('libc.so', 'clone');
if (cl !== null) {
try {
Interceptor.attach(cl, {
onEnter: function (args) {
var extra = '';
try { extra = ' realEntry=' + args[3].add(96).readPointer(); } catch (e) {}
bump('thread_create', 'clone()' + extra + ' <- ' + ownerString(this.context));
}
});
} catch (e) { bump('hook_errors', 'clone: ' + e.message); }
}
}
function hookLoader() {
['dlopen', 'android_dlopen_ext', 'dlsym'].forEach(function (fn) {
var a = Module.findExportByName('libc.so', fn);
if (a === null) return;
try {
Interceptor.attach(a, {
onEnter: function (args) {
var s = null;
try { s = args[0].readUtf8String(); } catch (e) {}
bump('loader', fn + '("' + s + '") <- ' + ownerString(this.context));
}
});
} catch (e) { bump('hook_errors', fn + ': ' + e.message); }
});
}
function selfView() {
// What the process can see about itself right now. This is evidence about the
// *environment*, not about the detector: it says what is available to be found,
// which is the other half of attributing a death.
var out = {};
var files = ['/proc/self/status', '/proc/self/maps', '/proc/self/task',
'/proc/net/tcp', '/proc/net/tcp6', '/proc/self/cmdline'];
out.files = {};
files.forEach(function (f) {
try {
var content = File.readAllText(f);
if (f === '/proc/self/status') {
var m = content.match(/TracerPid:\s*(\d+)/);
out.tracerpid = m ? m[1] : 'absent';
// A detector may match any of these names; report which are present so a
// "frida is visible here" claim is not made from memory.
out.status_flags = (content.match(/^(Name|State|Seccomp):.*$/gm) || []).join(' ; ');
} else if (f === '/proc/net/tcp' || f === '/proc/net/tcp6') {
var listening = (content.match(/^[^\n]*\s0A\s/gm) || []).length;
out[f] = 'lines=' + content.split('\n').length + ' listening=' + listening;
} else if (f === '/proc/self/maps') {
var hits = (content.match(/frida|gum|gadget|linjector/gi) || []);
out.frida_named_maps = hits.length;
out.map_sample = hits.slice(0, 5);
} else {
out[f] = 'readable (' + content.length + ' B)';
}
} catch (e) {
out[f] = 'unreadable: ' + e.message;
}
});
try {
out.current_thread = Process.getCurrentThreadId();
out.threads = Process.enumerateThreads().map(function (t) { return t.id; }).slice(0, 24);
} catch (e) { out.threads = 'enumerate failed: ' + e.message; }
try {
out.modules_named = Process.enumerateModules()
.filter(function (m) { return /frida|gum|gadget|linjector/i.test(m.name); })
.map(function (m) { return m.name + '@' + m.base; });
} catch (e) { out.modules_named = []; }
return out;
}
function report() {
var out = {
kind: 'anti_detect_probe',
mode: 'observer-only (this script never changes the target)',
ranMs: Date.now() - started,
counts: counts,
details: lines,
selfView: selfView(),
};
send({ type: 'RESULT', payload: out });
console.log('\n=== RESULT (probe) ===');
console.log('ranMs=' + out.ranMs);
Object.keys(counts).sort().forEach(function (k) {
console.log(' ' + k + ' = ' + counts[k]);
});
var sv = out.selfView;
console.log(' tracerpid=' + sv.tracerpid + ' frida_named_maps=' + sv.frida_named_maps +
' frida_modules=' + JSON.stringify(sv.modules_named));
var interesting = ['termination', 'thread_create', 'env_probe', 'loader'];
interesting.forEach(function (cat) {
if (!lines[cat]) return;
console.log('\n=== ' + cat + ' ===');
lines[cat].forEach(function (l) { console.log(' ' + l); });
});
console.log('\n=== how to read this ===');
console.log(' a hook that reports nothing is a finding about the hook (is the export');
console.log(' really there? did the code run?) -- establish the pipeline on a control');
console.log(' target before concluding "no detection".');
}
function start() {
hookExport('libc.so', 'open', 'env_probe', 0);
hookExport('libc.so', 'openat', 'env_probe', 1);
hookExport('libc.so', 'fopen', 'env_probe', 0);
hookExport('libc.so', 'fopen64', 'env_probe', 0);
hookExport('libc.so', 'access', 'env_probe', 0);
hookExport('libc.so', 'stat', 'env_probe', 0);
hookExport('libc.so', 'stat64', 'env_probe', 0);
hookExport('libc.so', 'lstat', 'env_probe', 0);
hookExport('libc.so', 'readlink', 'env_probe', 0);
hookExport('libc.so', 'opendir', 'env_probe', 0);
hookExport('libc.so', 'scandir', 'env_probe', 0);
hookExport('libc.so', 'strstr', 'strstr_probe', 1);
hookExport('libc.so', 'strcmp', 'strcmp_probe', 1);
hookTerminators();
hookThreadCreation();
hookLoader();
console.log('=== anti_detect_probe armed (observer only) ===');
var sv0 = selfView();
console.log('selfView at arm time: ' + JSON.stringify(sv0));
send({ type: 'LIVE', cat: 'armed', t: 0, msg: 'script armed', selfView: sv0 });
if (CONFIG.heartbeatMs > 0) {
// The heartbeat is the liveness proof. Its absence at time T means the script
// (or the whole process) was gone before T, which is a different finding from
// "the target was quiet".
setInterval(function () {
send({ type: 'LIVE', cat: 'heartbeat', t: Date.now() - started,
counts: JSON.parse(JSON.stringify(counts)) });
}, CONFIG.heartbeatMs);
}
if (CONFIG.watchMs > 0) {
var t = setTimeout(report, CONFIG.watchMs);
setTimeout(function () { clearTimeout(t); }, CONFIG.watchMs + 1);
}
}
rpc.exports = {
config: function (patch) { Object.assign(CONFIG, patch); return CONFIG; },
status: function () { return { counts: counts, ranMs: Date.now() - started }; },
report: function () { report(); return true; },
// Kept for interface parity with the kit's other probes; nothing is patched.
patch: function () { return 'observed-only: nothing patched'; },
};
if (Java.available) {
// Only to record what the Java side is, not to hook it: an observer that hooks
// Java would change the timing it is there to measure.
try {
Java.perform(function () {
console.log('java classloader ready, app=' +
(Java.androidVersion ? 'Android ' + Java.androidVersion : 'unknown'));
});
} catch (e) { /* Java optional */ }
}
setImmediate(start);

164
vendor-new/apk-reverse/skills/apk-reverse/scripts/apk_diff.py Normal file
View File

@@ -0,0 +1,164 @@
#!/usr/bin/env python3
"""Entry-level APK diff: what actually changed between two builds.
Two jobs, both recurring:
1. **Audit a build you did not produce.** Given an original and a
"cracked"/"modded" copy, this lists every entry that differs, every entry
the author ADDED, and every entry they REMOVED. Added entries under
`lib/`, `assets/`, or new dex files are the interesting ones -- that is
where injected payloads live.
2. **Verify your own build was surgical.** After repacking, the diff against
the original must contain exactly the entries you intended to touch and
nothing else. A surprise entry is a bug in your pipeline, not in the app.
It compares *content*, not just sizes: two builds can have identical sizes and
different bytes, and a same-size replacement is exactly the case a careless
comparison misses.
Usage
-----
python apk_diff.py original.apk modified.apk
python apk_diff.py original.apk modified.apk --hide-identical
python apk_diff.py original.apk modified.apk --only 'classes.*\\.dex|lib/.*'
python apk_diff.py original.apk modified.apk --summary
"""
import argparse
import hashlib
import re
import sys
import zipfile
# Directories worth naming explicitly -- this is where injected code lives.
INTERESTING = ("lib/", "assets/", "META-INF/services/")
def snapshot(path):
"""{entry_name: (size, sha256, compress_type, crc)}"""
out = {}
with zipfile.ZipFile(path) as z:
for it in z.infolist():
if it.is_dir():
continue
data = z.read(it.filename)
out[it.filename] = (
len(data),
hashlib.sha256(data).hexdigest(),
it.compress_type,
it.CRC,
)
return out
def classify(name):
if re.match(r"^classes\d*\.dex$", name):
return "dex"
if name.startswith("META-INF/") and name.endswith((".RSA", ".SF", ".MF", ".DSA", ".EC")):
return "signature"
if name.startswith(INTERESTING):
return "code/assets"
return "other"
def main():
ap = argparse.ArgumentParser(description="Entry-level APK diff.")
ap.add_argument("original")
ap.add_argument("modified")
ap.add_argument("--only", help="regex: restrict output to matching entry names")
ap.add_argument("--hide-identical", action="store_true",
help="omit the identical-count line")
ap.add_argument("--summary", action="store_true",
help="counts only, no per-entry listing")
ap.add_argument("--max", type=int, default=200,
help="max entries listed per section (default 200)")
args = ap.parse_args()
try:
a = snapshot(args.original)
b = snapshot(args.modified)
except Exception as e:
sys.exit("error reading APK: %s" % e)
only_a = sorted(set(a) - set(b))
only_b = sorted(set(b) - set(a))
common = sorted(set(a) & set(b))
changed = [n for n in common if a[n][1] != b[n][1]]
rx = re.compile(args.only) if args.only else None
def keep(n):
return rx.search(n) if rx else True
def sha(path, n):
return a[n][1][:16] if n in a else b[n][1][:16]
print("A (original) %s entries=%d" % (args.original, len(a)))
print("B (modified) %s entries=%d" % (args.modified, len(b)))
print("")
print("only in A (removed) : %d" % len(only_a))
print("only in B (added) : %d" % len(only_b))
print("content changed : %d" % len(changed))
if not args.hide_identical:
print("identical : %d" % (len(common) - len(changed)))
print("")
# Signature entries always differ; say so once instead of listing them.
sig_only = [n for n in (only_a + only_b) if classify(n) == "signature"]
if sig_only:
print("(note: %d signature entries account for most of the add/remove noise)"
% len(sig_only))
if args.summary:
return
if only_b:
print("\n=== ADDED in B (injection candidates) ===")
shown = 0
for n in only_b:
if not keep(n) or classify(n) == "signature":
continue
print(" + %-52s %9d B [%s]" % (n, b[n][0], classify(n)))
shown += 1
if shown >= args.max:
print(" ... (truncated)")
break
if only_a:
print("\n=== REMOVED from A ===")
shown = 0
for n in only_a:
if not keep(n) or classify(n) == "signature":
continue
print(" - %-52s %9d B [%s]" % (n, a[n][0], classify(n)))
shown += 1
if shown >= args.max:
print(" ... (truncated)")
break
if changed:
print("\n=== CONTENT CHANGED ===")
shown = 0
for n in changed:
if not keep(n):
continue
sa, sb = a[n], b[n]
cmark = "" if sa[2] == sb[2] else " ctype %d->%d" % (sa[2], sb[2])
same_size = " (SAME SIZE)" if sa[0] == sb[0] else ""
print(" ~ %-52s A %9d %s B %9d %s%s%s"
% (n, sa[0], sa[1][:16], sb[0], sb[1][:16], same_size, cmark))
shown += 1
if shown >= args.max:
print(" ... (truncated)")
break
if any(a[n][0] == b[n][0] for n in changed if keep(n)):
print("\n (SAME SIZE means the change is not a length difference --")
print(" a size-only comparison would have missed it entirely)")
if not (only_a or only_b or changed):
print("no differences at all")
if __name__ == "__main__":
main()

274
vendor-new/apk-reverse/skills/apk-reverse/scripts/blob_decode.py Normal file
View File

@@ -0,0 +1,274 @@
#!/usr/bin/env python3
"""Decode (and re-encode) an opaque single-value blob cached by an app.
Many apps cache server-issued configuration as ONE opaque string inside a
preferences file or a key-value store. The encoding is usually some stack of:
outer: base64, base64url, or hex
transform: a cyclic rotation of the byte stream (so the compressed stream
does not begin at its header), sometimes preceded by a short
header to skip
inner: raw deflate, zlib, or gzip
You do not have to guess the parameters. This script searches them and reports
every combination that decodes to something structured, then can re-encode your
edited payload with the same parameters so it can be written back.
Why rotation matters: a plain base64->inflate attempt fails on these blobs, and
the failure looks like "this value is encrypted". Exhaustively trying cyclic
shifts is cheap and frequently succeeds, which turns an opaque blob into an
editable JSON/XML document.
Usage:
# decode the value of a key inside a prefs XML
python blob_decode.py --prefs-xml prefs.xml --name flutter.remoteConfig
# decode a blob stored in its own file (or piped on stdin)
python blob_decode.py --file value.txt
cat value.txt | python blob_decode.py
# keep searching harder (slower) and write the best result out
python blob_decode.py --file value.txt --max-skip 64 --out decoded.bin
# re-encode an edited payload with the parameters that worked
python blob_decode.py --encode --file decoded.bin --outer base64 --inner raw --cut 15435 --out value.txt
Notes:
* Output is written to --out; nothing on device is touched.
* The winning parameters are printed as a copy-pasteable line for --encode.
"""
import argparse
import base64
import binascii
import re
import sys
import time
import zlib
INNERS = [("raw", -15), ("zlib", 15), ("gzip", 31)]
# --------------------------------------------------------------------------- #
# outer encodings
# --------------------------------------------------------------------------- #
def outer_decode(text: str, which: str) -> bytes | None:
t = text.strip().replace("\n", "").replace("\r", "").replace(" ", "")
try:
if which == "base64":
return base64.b64decode(t + "=" * (-len(t) % 4))
if which == "base64url":
return base64.urlsafe_b64decode(t + "=" * (-len(t) % 4))
if which == "hex":
return binascii.unhexlify(re.sub(r"[^0-9a-fA-F]", "", t))
except Exception:
return None
return None
def outer_encode(data: bytes, which: str) -> str:
if which == "base64":
return base64.b64encode(data).decode("ascii")
if which == "base64url":
return base64.urlsafe_b64encode(data).decode("ascii")
if which == "hex":
return binascii.hexlify(data).decode("ascii")
raise ValueError("unknown outer encoding: %s" % which)
def sniff_outer(text: str) -> list[str]:
"""Cheap guess first so the common case is instant."""
t = text.strip()
guesses = []
if re.fullmatch(r"[0-9a-fA-F\s]+", t) and len(t) >= 16:
guesses.append("hex")
if re.fullmatch(r"[A-Za-z0-9+/\s=]+", t):
guesses.append("base64")
if re.fullmatch(r"[A-Za-z0-9\-_\s=]+", t):
guesses.append("base64url")
# always keep base64 last-resort, then everything else
for w in ("base64", "base64url", "hex"):
if w not in guesses:
guesses.append(w)
return list(dict.fromkeys(guesses))
# --------------------------------------------------------------------------- #
# inner transforms
# --------------------------------------------------------------------------- #
def try_inflate(body: bytes, max_out: int | None = None) -> tuple[str, bytes] | None:
best = None
for name, wbits in INNERS:
try:
if max_out is None:
out = zlib.decompress(body, wbits)
else:
d = zlib.decompressobj(wbits)
out = d.decompress(body, max_out)
if out:
if best is None or len(out) > len(best[1]):
best = (name, out)
except Exception:
continue
return best
def inflate_raw(data: bytes) -> bytes:
co = zlib.compressobj(9, zlib.DEFLATED, -15)
return co.compress(data) + co.flush()
def score(out: bytes) -> float:
"""How much does this look like a real document we can edit?"""
if not out:
return -1.0
printable = sum(1 for b in out[:4096] if 32 <= b < 127 or b in (9, 10, 13))
ratio = printable / min(len(out), 4096)
s = ratio * 10.0
head = out[:64].lstrip()
if head[:1] in (b"{", b"[", b"<", b"#"):
s += 6.0
if b'"' in out[:512] or b"=" in out[:512]:
s += 2.0
if len(out) < 32:
s -= 3.0
return s
# --------------------------------------------------------------------------- #
def search(blob: bytes, max_skip: int, log) -> list[dict]:
hits: list[dict] = []
n = len(blob)
t0 = time.time()
for skip in range(0, min(max_skip, n) + 1):
base = blob[skip:]
m = len(base)
if m < 8:
break
for k in range(m):
cand = base[k:] + base[:k]
got = try_inflate(cand, max_out=8_000_000)
if got:
name, out = got
hits.append({
"skip": skip, "cut": k, "inner": name, "size": len(out),
"score": round(score(out), 2), "data": out,
})
if hits and log and skip == 0:
log(" first hit found with skip=0 after %.1fs" % (time.time() - t0))
if log and skip and skip % 16 == 0 and time.time() - t0 > 30:
log(" ... skip=%d, %.0fs elapsed" % (skip, time.time() - t0))
log(" search finished in %.1fs (%d candidate(s))" % (time.time() - t0, len(hits)))
return hits
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
src = ap.add_mutually_exclusive_group()
src.add_argument("--file", help="file containing the encoded value")
src.add_argument("--prefs-xml", help="a preferences XML file to pull the value out of")
ap.add_argument("--name", help="key name when using --prefs-xml")
ap.add_argument("--outer", choices=["base64", "base64url", "hex", "auto"], default="auto")
ap.add_argument("--max-skip", type=int, default=0,
help="also try dropping 0..N leading bytes before the rotation search (slower)")
ap.add_argument("--min-size", type=int, default=200, help="ignore results smaller than this")
ap.add_argument("--top", type=int, default=5, help="how many candidates to print")
ap.add_argument("--out", help="write the best decoded result here")
ap.add_argument("--encode", action="store_true", help="re-encode instead of decoding")
ap.add_argument("--inner", choices=[n for n, _ in INNERS], default="raw")
ap.add_argument("--cut", type=int, default=0, help="rotation used for write-back")
a = ap.parse_args()
def log(msg: str) -> None:
print(msg, file=sys.stderr)
# ------------------------------------------------------------------ encode
if a.encode:
if not a.file:
log("--encode requires --file (the payload to encode)")
return 2
payload = open(a.file, "rb").read()
body = inflate_raw(payload) if a.inner == "raw" else zlib.compress(payload, 9)
if a.inner == "zlib":
body = zlib.compress(payload, 9)
cut = a.cut % max(len(body), 1)
rotated = body[-cut:] + body[:-cut] if cut else body
enc = outer_encode(rotated, "base64" if a.outer == "auto" else a.outer)
if a.out:
open(a.out, "w", encoding="utf-8", newline="\n").write(enc)
log("wrote %d chars to %s" % (len(enc), a.out))
else:
print(enc)
log("write-back parameters: outer=%s inner=%s cut=%d"
% ("base64" if a.outer == "auto" else a.outer, a.inner, cut))
return 0
# ------------------------------------------------------------------ input
if a.prefs_xml:
if not a.name:
log("--prefs-xml requires --name <key>")
return 2
xml = open(a.prefs_xml, encoding="utf-8", errors="replace").read()
m = re.search(r'name="%s"[^>]*>([^<]*)<' % re.escape(a.name), xml)
if not m:
log("key not found in XML: %s" % a.name)
log("keys present: %s" % ", ".join(re.findall(r'name="([^"]+)"', xml)[:40]))
return 2
text = m.group(1)
log("value length: %d" % len(text))
elif a.file:
text = open(a.file, encoding="utf-8", errors="replace").read()
else:
text = sys.stdin.read()
if not text.strip():
log("empty input")
return 2
# ------------------------------------------------------------------ decode
candidates = [a.outer] if a.outer != "auto" else sniff_outer(text)
for which in candidates:
blob = outer_decode(text, which)
if blob is None or len(blob) < 8:
log("outer=%s -> not decodable, skipping" % which)
continue
log("outer=%s -> %d bytes; searching rotation (and skip<=%d)" % (which, len(blob), a.max_skip))
hits = [h for h in search(blob, a.max_skip, log) if h["size"] >= a.min_size]
if not hits:
continue
hits.sort(key=lambda h: (-h["score"], -h["size"]))
print("=" * 66)
print("outer=%s candidates=%d" % (which, len(hits)))
for h in hits[:a.top]:
print(" score=%-6s skip=%-4d cut=%-7d inner=%-5s size=%d"
% (h["score"], h["skip"], h["cut"], h["inner"], h["size"]))
best = hits[0]
print("-" * 66)
print("write-back line:")
print(" python blob_decode.py --encode --file <edited_payload> --outer %s --inner %s --cut %d"
% (which, best["inner"], best["cut"]))
print("-" * 66)
preview = best["data"][:1200]
try:
print(preview.decode("utf-8"))
except UnicodeDecodeError:
print(preview.decode("latin1"))
print("(non-UTF-8; first 64 bytes: %s)" % best["data"][:64].hex())
if a.out:
open(a.out, "wb").write(best["data"])
log("wrote %d bytes to %s" % (len(best["data"]), a.out))
return 0
print("no decodable candidate found.", file=sys.stderr)
print("Next steps:", file=sys.stderr)
print(" * raise --max-skip, or pass --outer explicitly", file=sys.stderr)
print(" * the blob may be genuinely encrypted (a keyed cipher) rather than merely", file=sys.stderr)
print(" encoded -- confirm before spending time on it: a keyed blob should look", file=sys.stderr)
print(" random, whereas an encoded one still has structure once the framing is", file=sys.stderr)
print(" removed. See references/runtime-data.md.", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())

1248
vendor-new/apk-reverse/skills/apk-reverse/scripts/capabilities.py Normal file
View File

File diff suppressed because it is too large Load Diff

307
vendor-new/apk-reverse/skills/apk-reverse/scripts/coldstart.py Normal file
View File

@@ -0,0 +1,307 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Cold-launch an app and capture a time line of screenshots plus logcat signals.
WHY THIS EXISTS
---------------
"Did the patch work?" for anything user-visible -- a splash/ad gate, a forced
update dialog, a login wall, a crash-on-start -- is answered by looking at the
screen over the first seconds of a launch, and by comparison against the
unmodified build. Doing that by hand produces two failures that look like
findings and are not:
* **Sampling is not observation.** A dialog that appears and is then occluded
can fall entirely between samples, and every frame you happened to take shows
something else, which feels like corroboration. This tool captures on a fixed
short cadence and, crucially, prints what it captured so a human or agent
actually LOOKS at the frames.
* **`am start -W` can block past a naive timeout.** It waits for the first
frame; on some ROMs that never arrives while another window owns the display
and the command simply hangs. This tool launches without `-W` and derives the
time line from captures plus logcat instead.
It also records the two facts that decide whether an install even happened:
the installed version, and the on-disk APK hash. "Install succeeded" describes
the request, not the app on disk.
IMPORTANT -- before trusting any screenshot, confirm the foreground activity is
your target. A vendor package installer left on screen (a pending confirmation
from an earlier `adb install`, for example) will be captured instead of your app
and looks completely plausible. `--expect-activity` makes that check automatic.
Usage
-----
python coldstart.py --serial <SERIAL> --pkg com.example \\
--activity com.example/.MainActivity --duration 14 --interval 0.8 \\
--out work/verify/orig --expect-activity com.example/.MainActivity
Requires: adb. `su` on the device is used when available for version/hash facts,
otherwise it degrades to plain `pm`/`dumpsys`.
"""
import argparse
import os
import re
import shutil
import subprocess
import sys
import time
def run(cmd, timeout=30, serial=None):
"""Run a command with a hard timeout; never let a call stall the run."""
try:
return subprocess.run(cmd, capture_output=True, timeout=timeout,
text=True, errors='replace')
except subprocess.TimeoutExpired:
return subprocess.CompletedProcess(cmd, 124, '', 'timeout after %ss' % timeout)
class Device(object):
def __init__(self, serial, use_su=True, timeout=30):
self.serial = serial
self.timeout = timeout
self.adb = shutil.which('adb') or 'adb'
self.su = use_su and self._su_works()
def _su_works(self):
if self.serial:
r = run(['adb', '-s', self.serial, 'shell', "su -c 'id'"], 15)
else:
r = run(['adb', 'shell', "su -c 'id'"], 15)
return r.returncode == 0 and 'uid=0' in (r.stdout or '')
def shell(self, cmd, timeout=None):
"""Run a device shell command, preferring root when available."""
t = timeout or self.timeout
if self.serial:
base = ['adb', '-s', self.serial, 'shell']
else:
base = ['adb', 'shell']
if self.su:
return run(base + ["su -c '%s'" % cmd.replace("'", "'\\''")], t)
return run(base + [cmd], t)
def screencap(self, path, timeout=30):
"""Capture one PNG. Returns bytes written, or -1 on failure."""
if self.serial:
base = ['adb', '-s', self.serial]
else:
base = ['adb']
try:
with open(path, 'wb') as fh:
# The PNG is validated from the file that comes back, so the exit status is
# deliberately discarded: a non-zero screencap that still wrote a valid PNG is
# a success, and the header check below is the authority.
r = subprocess.run(base + ['exec-out', 'screencap', '-p'], # noqa: F841
stdout=fh, timeout=timeout)
size = os.path.getsize(path)
if size < 8:
return -1
with open(path, 'rb') as fh:
if fh.read(8) != b'\x89PNG\r\n\x1a\n':
return -1
return size
except subprocess.TimeoutExpired:
return -1
def foreground_activity(dev):
r = dev.shell('dumpsys activity activities | grep -m1 ResumedActivity')
m = re.search(r'([A-Za-z0-9_.]+)/([A-Za-z0-9_.$]+)', r.stdout or '')
return '%s/%s' % (m.group(1), m.group(2)) if m else None
def installed_facts(dev, pkg):
facts = {}
r = dev.shell("dumpsys package %s | grep -E 'versionCode|versionName|firstInstallTime|lastUpdateTime'" % pkg)
for line in (r.stdout or '').splitlines():
m = re.match(r'\s*(versionCode|versionName|firstInstallTime|lastUpdateTime)=(.*)', line)
if m:
facts[m.group(1)] = m.group(2).strip()
r = dev.shell('ls /data/app/*/%s*/base.apk 2>/dev/null' % pkg)
apk_path = (r.stdout or '').strip().splitlines()[:1]
if apk_path:
facts['apk_path'] = apk_path[0]
r2 = dev.shell('sha256sum %s' % apk_path[0])
m = re.search(r'([0-9a-f]{64})', r2.stdout or '')
if m:
facts['apk_sha256'] = m.group(1)
r = dev.shell('dumpsys package %s | grep -m1 userId=' % pkg)
m = re.search(r'userId=(\d+)', r.stdout or '')
if m:
facts['uid'] = m.group(1)
return facts
def signal_counts(log_path, patterns):
"""Count occurrences of each pattern in a logcat dump."""
try:
with open(log_path, encoding='utf-8', errors='replace') as fh:
text = fh.read()
except OSError:
return {}
return {name: len(re.findall(rx, text)) for name, rx in patterns.items()}
SIGNAL_PATTERNS = {
'FATAL EXCEPTION': r'FATAL EXCEPTION',
'VerifyError': r'VerifyError',
'Bad checksum': r'Bad checksum',
'IncompatibleClassChangeError': r'IncompatibleClassChangeError',
'ClassNotFoundException': r'ClassNotFoundException',
'uncaughtException': r'uncaughtException',
'am_crash': r'am_crash',
'ANR': r'ANR in ',
'Displayed': r'Displayed .*\+([0-9]+)ms',
}
def main(argv):
ap = argparse.ArgumentParser(
description='Cold-launch an app and capture a screenshot time line plus '
'logcat signals, for before/after comparison.',
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__.split('Usage')[-1][:1200])
ap.add_argument('--serial', help='adb serial (REQUIRED when several devices are '
'online, so the wrong one is not picked at random)')
ap.add_argument('--pkg', required=True, help='package name')
ap.add_argument('--activity', help='component to start (default: monkey/launcher)')
ap.add_argument('--out', default='coldstart', help='output directory')
ap.add_argument('--duration', type=float, default=14.0, help='seconds to capture')
ap.add_argument('--interval', type=float, default=0.8, help='seconds between frames')
ap.add_argument('--expect-activity', help='warn if the foreground activity is not '
'this (and not the package at all)')
ap.add_argument('--no-su', action='store_true', help='never use device root')
ap.add_argument('--keep-old', action='store_true',
help='do not clear existing screenshots in the output dir')
args = ap.parse_args(argv[1:])
if not shutil.which('adb'):
print('error: adb not found on PATH')
return 2
if not args.serial:
r = run(['adb', 'devices'])
online = [row.split()[0] for row in (r.stdout or '').splitlines()[1:]
if len(row.split()) >= 2 and row.split()[1] == 'device']
if len(online) > 1:
print('error: %d devices online (%s) and no --serial given. adb would '
'pick one at random and you would debug the wrong target.'
% (len(online), ', '.join(online)))
return 2
if online:
args.serial = online[0]
print('[i] single device online; using %s' % args.serial)
dev = Device(args.serial, use_su=not args.no_su)
print('[i] serial=%s root=%s' % (args.serial, dev.su))
os.makedirs(args.out, exist_ok=True)
shots = os.path.join(args.out, 'shots')
os.makedirs(shots, exist_ok=True)
if not args.keep_old:
for f in os.listdir(shots):
if f.endswith('.png'):
os.remove(os.path.join(shots, f))
facts = installed_facts(dev, args.pkg)
print('\n== installed build facts')
for k in sorted(facts):
print(' %-18s %s' % (k, facts[k]))
if not facts.get('versionCode'):
print(' WARNING: no versionCode found -- the package may not be installed')
print(' (install success describes the request, not the app on disk; the '
'sha256 above is the on-disk truth)')
print('\n== clearing logcat')
dev.shell('logcat -c', timeout=20)
print('== force-stopping %s' % args.pkg)
dev.shell('am force-stop %s' % args.pkg, timeout=20)
print('== launching')
t0 = time.time()
if args.activity:
r = dev.shell('am start -n %s' % args.activity, timeout=25)
else:
r = dev.shell('monkey -p %s -c android.intent.category.LAUNCHER 1' % args.pkg,
timeout=25)
out = ((r.stdout or '') + (r.stderr or '')).strip()
print(' %s' % (out.splitlines()[0] if out else '(no output)'))
if 'Error' in out or 'Exception' in out:
print(' LAUNCH ERROR -- read the line above before blaming the patch')
# live logcat into a file while we capture
log_path = os.path.join(args.out, 'logcat.txt')
if args.serial:
log_cmd = ['adb', '-s', args.serial, 'logcat', '-v', 'threadtime']
else:
log_cmd = ['adb', 'logcat', '-v', 'threadtime']
log_fh = open(log_path, 'w', encoding='utf-8', errors='replace')
log_proc = subprocess.Popen(log_cmd, stdout=log_fh, stderr=subprocess.DEVNULL)
frames = []
idx = 0
try:
while True:
el = time.time() - t0
if el > args.duration:
break
name = 'f%02d_%05.2fs.png' % (idx, el)
path = os.path.join(shots, name)
n = dev.screencap(path)
frames.append((name, n, el))
print(' %-20s %s' % (name, ('%d bytes' % n) if n > 0 else 'CAPTURE FAILED'))
idx += 1
time.sleep(max(0.0, args.interval))
finally:
time.sleep(0.5)
log_proc.terminate()
try:
log_proc.wait(timeout=10)
except subprocess.TimeoutExpired:
log_proc.kill()
log_fh.close()
fg = foreground_activity(dev)
print('\n== foreground at end: %s' % fg)
if args.expect_activity and fg and args.pkg not in fg:
print(' *** FOREGROUND IS NOT YOUR APP (%s) ***' % fg)
print(' Everything captured above shows something else -- most often a '
'vendor package-installer confirmation left over from an install.')
print(' Clear it (e.g. close the installer, press HOME) and re-run; do not '
'read these frames as evidence about your build.')
counts = signal_counts(log_path, SIGNAL_PATTERNS)
print('\n== logcat signals (%s)' % log_path)
for name in SIGNAL_PATTERNS:
print(' %-32s %s' % (name, counts.get(name, 0)))
m = re.findall(r'Displayed ([^:]+): \+([0-9]+)ms',
open(log_path, encoding='utf-8', errors='replace').read())
if m:
print('\n== launch timing (logcat Displayed)')
for comp, ms in m:
print(' %s %s ms' % (comp.strip(), ms))
else:
print('\n no "Displayed" line: the launch may not have reached a first '
'frame (a hang or a blocking dialog looks exactly like this)')
zero = sum(1 for _n, n, _e in frames if n <= 0)
print('\n== %d frame(s) in %s, %d failed' % (len(frames), shots, zero))
# the point of the tool: make inspection happen
print('\n== INSPECT THE FRAMES. A capture nobody looked at is not evidence.')
nonzero = [f for f in sorted(os.listdir(shots)) if f.endswith('.png')]
if nonzero:
print(' start with: %s' % os.path.join(shots, nonzero[0]))
mid = nonzero[len(nonzero) // 2]
print(' and: %s' % os.path.join(shots, mid))
print(' Byte-identical consecutive frames mean the screen is static -- that '
'is a finding (a hang, or a dialog waiting), not a capture problem.')
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))

212
vendor-new/apk-reverse/skills/apk-reverse/scripts/dart_disasm.py Normal file
View File

@@ -0,0 +1,212 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Annotated, windowed disassembly of a Dart AOT snapshot, plus a caller index.
WHY THIS EXISTS
---------------
Two things make Dart AOT code readable at all:
1. **Pool annotations.** A raw `ldr x2, [x27, #0x9d0]` is meaningless; annotated with the
pool entry it becomes `"vipflag"` and the surrounding code explains itself. This tool
resolves those loads and prints what they point at, including how many other sites
reference the same entry.
2. **Windowed disassembly.** Decoding an entire multi-MB `.text` in one pass is how a host
gets pinned at 1-2 GB and the run looks like a hang. Disassemble only the ranges you
care about -- that is what this tool does, and it is not a limitation but the method.
Booleans are annotated too: Dart materialises `true`/`false` as small offsets from the null
register, so `add x0, x22, #0x20` is marked TRUE and `#0x30` is marked FALSE. Those are the
cheapest patch sites in the whole snapshot.
The caller index answers the other half of every question -- "what calls this function".
It is built from B/BL target arithmetic on the raw words, so it needs no disassembler at all
and finishes in seconds.
USAGE
-----
# a window around one address (default: 0x180 before, 0x220 after)
python dart_disasm.py libapp.so --pp pp.txt --refs pp_refs.json 0x26e390
# an explicit range
python dart_disasm.py libapp.so --pp pp.txt --refs pp_refs.json --range 0x64f310 0x64f3e0
# widen / narrow the window
python dart_disasm.py libapp.so --refs pp_refs.json 0x64f1bc --before 0x80 --after 0x40
# caller index
python dart_disasm.py libapp.so --build-index callers.json
python dart_disasm.py libapp.so --index callers.json 0x26e230 0x64f0f0
INPUTS
------
* `--pp` is the decompiler's pool listing (blutter `pp.txt`): lines shaped
`[pp+0x9d0] String: "vipflag"`. Without it, loads are annotated with the raw offset only.
* `--refs` is the pool-offset -> code-site index built by `dart_pprefs.py`.
Requires `capstone` for disassembly; the caller index works without it.
"""
import argparse
import json
import os
import re
import struct
import sys
POOL_REG = 27
BOOL_TRUE, BOOL_FALSE = 0x20, 0x30
PP_LINE = re.compile(r'^\[(pp\+0x[0-9a-f]+)\](.*)$')
def text_section(path):
blob = open(path, 'rb').read()
if blob[:4] != b'\x7fELF':
raise SystemExit('not an ELF file: %s' % path)
if blob[4] != 2:
raise SystemExit('not ELF64: %s' % path)
e_shoff = struct.unpack_from('<Q', blob, 0x28)[0]
e_shentsize = struct.unpack_from('<H', blob, 0x3A)[0]
e_shnum = struct.unpack_from('<H', blob, 0x3C)[0]
e_shstrndx = struct.unpack_from('<H', blob, 0x3E)[0]
def sh(i):
return struct.unpack_from('<IIQQQQ', blob, e_shoff + i * e_shentsize)
_, _, _, _, shstr_off, shstr_size = sh(e_shstrndx)
strs = blob[shstr_off:shstr_off + shstr_size]
for i in range(e_shnum):
name, _t, _f, addr, off, size = sh(i)
end = strs.find(b'\x00', name)
if strs[name:end].decode('ascii', 'replace') == '.text':
return addr, blob[off:off + size]
raise SystemExit('.text not found')
def load_pp(path):
pool = {}
if not path or not os.path.exists(path):
return pool
for line in open(path, encoding='utf-8', errors='replace'):
m = PP_LINE.match(line.rstrip('\n'))
if m:
pool[int(m.group(1)[5:], 16)] = m.group(2).strip()[:120]
return pool
def load_refs(path):
if not path or not os.path.exists(path):
return {}
return {int(k, 16): v for k, v in json.load(open(path, encoding='utf-8')).items()}
def disasm_window(sample, addr, before, after, pp, refs):
from capstone import Cs, CS_ARCH_ARM64, CS_MODE_LITTLE_ENDIAN
text_addr, blob = text_section(sample)
end_of_text = text_addr + len(blob)
start = max(text_addr, addr - before)
stop = min(end_of_text, addr + after)
md = Cs(CS_ARCH_ARM64, CS_MODE_LITTLE_ENDIAN)
md.skipdata = True # without this, one undecodable byte silently ends the range
md.detail = True # without this, operands are unavailable
print('range 0x%x..0x%x focus=0x%x' % (start, stop, addr))
pending = {}
for ins in md.disasm(blob[start - text_addr:stop - text_addr], start):
if ins.id == 0: # SKIPDATA pseudo-instruction: no operands
print('%x: %s' % (ins.address, ins.mnemonic))
continue
note = ''
ops = ins.operands
rn = ins.reg_name
if (ins.mnemonic == 'add' and len(ops) == 3 and ops[1].type == 1
and rn(ops[1].reg) == 'x27' and ops[2].type == 2
and ops[2].shift.value == 12):
pending[rn(ops[0].reg)] = ops[2].imm << 12
elif (ins.mnemonic == 'add' and len(ops) == 3 and ops[1].type == 1
and rn(ops[1].reg) == 'x22' and ops[2].type == 2):
note = {BOOL_TRUE: ' ; TRUE', BOOL_FALSE: ' ; FALSE'}.get(ops[2].imm, '')
elif ins.mnemonic in ('ldr', 'ldur') and len(ops) == 2 and ops[1].type == 3:
base = rn(ops[1].mem.base) if ops[1].mem.base else ''
off = None
if base == 'x27':
off = ops[1].mem.disp
elif base in pending:
off = pending.pop(base) + ops[1].mem.disp
if off is not None:
text = pp.get(off, '')
sites = refs.get(off, [])
note = ' ; pp+0x%x %s [%d refs]' % (off, text, len(sites))
mark = ' <<<' if abs(ins.address - addr) < 4 else ''
print('%x: %-8s %s%s%s' % (ins.address, ins.mnemonic, ins.op_str, note, mark))
def build_index(sample, out):
"""B/BL target -> call sites, decoded arithmetically (no disassembler needed)."""
text_addr, blob = text_section(sample)
index = {}
words = struct.unpack_from('<%dI' % (len(blob) // 4), blob, 0)
for i, w in enumerate(words):
if w & 0xFC000000 in (0x14000000, 0x94000000):
imm = w & 0x03FFFFFF
if imm & 0x02000000:
imm -= 0x04000000
site = text_addr + i * 4
index.setdefault(site + (imm << 2), []).append(site)
with open(out, 'w', encoding='utf-8') as fh:
json.dump({hex(k): v for k, v in index.items()}, fh)
print('distinct targets : %d' % len(index))
print('call sites : %d' % sum(len(v) for v in index.values()))
print('written %s' % out)
def lookup_index(sample, path, addrs):
index = {int(k, 16): v for k, v in json.load(open(path, encoding='utf-8')).items()}
for spec in addrs:
target = int(spec, 16)
sites = sorted(index.get(target, []))
print('0x%x: %d caller(s)' % (target, len(sites)))
for s in sites[:80]:
print(' 0x%x' % s)
if len(sites) > 80:
print(' ... %d more' % (len(sites) - 80))
def main():
ap = argparse.ArgumentParser(
description=__doc__.split('\n')[0],
formatter_class=argparse.RawDescriptionHelpFormatter, epilog=__doc__)
ap.add_argument('sample', help='libapp.so')
ap.add_argument('addr', nargs='*', help='hex address(es) to disassemble around')
ap.add_argument('--pp', metavar='PP_TXT', help="decompiler pool listing (blutter pp.txt)")
ap.add_argument('--refs', metavar='REFS_JSON', help='index from dart_pprefs.py')
ap.add_argument('--range', nargs=2, metavar=('START', 'END'))
ap.add_argument('--before', default='0x180')
ap.add_argument('--after', default='0x220')
ap.add_argument('--build-index', metavar='OUT_JSON')
ap.add_argument('--index', metavar='CALLERS_JSON')
a = ap.parse_args()
if a.build_index:
return build_index(a.sample, a.build_index)
if a.index:
if not a.addr:
raise SystemExit('give at least one hex address to look up')
return lookup_index(a.sample, a.index, a.addr)
pp = load_pp(a.pp)
refs = load_refs(a.refs)
if a.range:
start, stop = int(a.range[0], 16), int(a.range[1], 16)
disasm_window(a.sample, start, 0, stop - start, pp, refs)
return 0
if not a.addr:
raise SystemExit('give an address, --range, --build-index, or --index')
for spec in a.addr:
disasm_window(a.sample, int(spec, 16),
int(a.before, 0), int(a.after, 0), pp, refs)
return 0
if __name__ == '__main__':
sys.exit(main())

273
vendor-new/apk-reverse/skills/apk-reverse/scripts/dart_pool_strings.py Normal file
View File

@@ -0,0 +1,273 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Recover string literals from a Dart AOT snapshot, and locate them by file offset.
WHY THIS EXISTS
---------------
Strings are the cheapest anchor for locating logic in a stripped AOT snapshot: field
names, endpoint paths and labels survive compilation even when every symbol name is gone.
Two things this gives you that a decompiler's own listing does not:
* **file offsets**, where the constant physically sits in the binary (what you need to
patch bytes, as opposed to a pool offset);
* **identifiers the decompiler's listing omits** -- measured on a real sample, most of
what this recovers was absent from the pool listing entirely. Treat the two as
complementary, not competitive.
THE FRAMING
-----------
[ tag ][ payload ] tag = 0x80 | (len << 1) | <two-byte flag>
* tag bit 7 set -> a string entry
* (tag >> 1) & 0x3F -> payload length in characters (this framing tops out at 63)
* the low bit -> nominally "payload is UTF-16LE"
ENCODING, AND WHY ONE-BYTE IS THE DEFAULT
-----------------------------------------
The two payload kinds are searched differently, and this is where most people lose time:
* **one-byte** payloads are ASCII / Latin-1 -> a **UTF-8 byte search finds them**
* **two-byte** payloads are **UTF-16LE** -> a UTF-8 search returns *zero hits*
So search ASCII as UTF-8 and CJK/non-Latin as UTF-16LE; forcing one encoding on both is
the classic false negative (`pitfalls.md` P25).
**But automatic two-byte detection does not work reliably.** The low tag bit is not a
dependable discriminator against ordinary data, and UTF-16LE decoding essentially never
fails -- *any* byte pair is a legal code unit -- so decoding success filters nothing. On a
real sample, two-byte output was ~10k entries of near-pure garbage that happened to form
long runs, while one-byte output was almost entirely genuine. Hence:
* default: one-byte only (reliable; verified against a decompiler listing on a real sample)
* `--two-byte`: opt in, and **verify every hit yourself**
* to find a *known* non-Latin literal, use `--find` -- a direct search is far more
reliable than enumerating candidates
WHY THE RUN LENGTH IS REPORTED
------------------------------
A per-byte scan for tag-like bytes hits constantly inside ordinary data; a multi-MB
snapshot yields hundreds of thousands of candidates. Most real string tables are laid out
consecutively, so a genuine entry usually begins a run of back-to-back entries (each
entry's tag+payload ends exactly where the next begins). `--min-chain` uses that to shed
the scattered tail.
Note what the measurement showed: raising this threshold barely changes one-byte
precision while steadily destroying recall. It is a **noise filter, not a confidence
score** -- do not tune it expecting accuracy to improve.
USAGE
-----
python dart_pool_strings.py libapp.so strings.tsv
python dart_pool_strings.py libapp.so strings.tsv --pp pp.txt # annotate cross-hits
python dart_pool_strings.py libapp.so strings.tsv --min-chain 2 --two-byte
python dart_pool_strings.py libapp.so --find /activateVipCode --find vipflag
python dart_pool_strings.py libapp.so --stats
OUTPUT
------
offset_hex <TAB> text <TAB> run <TAB> form [<TAB> in_pp]
`in_pp` is present with `--pp`: `1` means the decompiler's own pool listing contains this
exact string. Absence is **not** evidence of a false positive -- the decompiler's listing
is incomplete by design.
NOTES
-----
* `--max` is capped at 63 by the framing; longer literals are serialized differently and
will not appear here (a few hundred per sample at most).
* Never edit a string without counting its references first: a shared entry changes every
consumer at once (`references/dart-aot.md` section 10).
"""
import argparse
import collections
import re
import sys
TAG_STR = 0x80
BAD = re.compile(r'[\x00-\x08\x0b\x0c\x0e-\x1f]')
def decode_payload(raw, two_byte):
if two_byte:
if len(raw) % 2:
return None
try:
text = raw.decode('utf-16-le')
except UnicodeDecodeError:
return None
else:
try:
text = raw.decode('utf-8')
except UnicodeDecodeError:
return None
if BAD.search(text):
return None
return text
def extract(blob, min_len, max_len, want_two_byte):
"""Return {start: (end, text, form)} for every framed candidate of the wanted kind."""
n = len(blob)
cands = {}
for i in range(n - 2):
tag = blob[i]
if not (tag & TAG_STR):
continue
ln = (tag >> 1) & 0x3F
if not (min_len <= ln <= max_len):
continue
two = tag & 1
if two and not want_two_byte:
continue
end = i + 1 + (ln * 2 if two else ln)
if end > n:
continue
text = decode_payload(blob[i + 1:end], two)
if text is None or len(text.strip()) < min_len:
continue
cands[i] = (end, text, 'utf16le' if two else 'one-byte')
return cands
def run_lengths(cands):
"""For each candidate, how many back-to-back entries follow it (inclusive)."""
nxt = {s: cands[s][0] for s in cands}
lengths = {}
closed = set()
for start in sorted(cands):
if start in closed:
continue
seq = []
cur = start
guard = 0
while cur in nxt and cur not in closed:
seq.append(cur)
closed.add(cur)
cur = nxt[cur]
guard += 1
if guard > 1_000_000:
break
for i, item in enumerate(seq):
lengths[item] = len(seq) - i
return lengths
def load_pp(path):
"""Strings present in a decompiler pool listing, for cross-annotation."""
if not path:
return set()
out = set()
pat = re.compile(r'^\[pp\+0x[0-9a-f]+\] String: (.*)$')
for line in open(path, encoding='utf-8', errors='replace'):
m = pat.match(line.strip())
if not m:
continue
s = m.group(1).strip()
if len(s) >= 2 and s[0] == '"' and s[-1] == '"':
s = s[1:-1]
if s:
out.add(s)
return out
def histogram(lengths):
counts = collections.Counter(lengths.values())
for run in sorted(counts)[:10]:
print(' run=%-4d %d' % (run, counts[run]))
tail = sum(v for k, v in counts.items() if k > 10)
if tail:
print(' run>%-3d %d' % (10, tail))
def search(blob, needles):
total = 0
for needle in needles:
for label, enc in (('utf-8', needle.encode('utf-8')),
('utf-16le', needle.encode('utf-16-le'))):
offsets = []
start = blob.find(enc)
while start != -1 and len(offsets) < 8:
offsets.append(start)
start = blob.find(enc, start + 1)
if offsets:
total += len(offsets)
print('%-26s %-9s file offset %s' % (
needle, label, ' '.join(hex(o) for o in offsets)))
if not total:
print('no hits in either encoding. Try the shortest distinctive fragment -- text is\n'
'often assembled from parts, and a literal may only exist as fragments\n'
'(pitfalls.md P25).')
return 0
def main():
ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])
ap.add_argument('sample', help='libapp.so or any artifact holding the snapshot')
ap.add_argument('out', nargs='?', help='output tsv')
ap.add_argument('--min', type=int, default=4, help='minimum payload length (default 4)')
ap.add_argument('--max', type=int, default=63,
help="maximum payload length; 63 is this framing's ceiling (default 63)")
ap.add_argument('--min-chain', type=int, default=3,
help='noise filter: minimum run of consecutive entries (default 3). '
'This does not improve accuracy -- see the module docstring.')
ap.add_argument('--two-byte', action='store_true',
help='ALSO emit UTF-16 candidates. Detection is unreliable; verify each.')
ap.add_argument('--pp', metavar='PP_TXT',
help='decompiler pool listing, to mark which strings it also contains')
ap.add_argument('--stats', action='store_true',
help='print the run-length histogram and exit')
ap.add_argument('--keep-isolated', action='store_true',
help='also write entries below --min-chain (noisy; triage only)')
ap.add_argument('--find', action='append', default=[], metavar='TEXT',
help='search a literal in BOTH encodings and print file offsets '
'(repeatable; the reliable way to locate a known literal)')
a = ap.parse_args()
blob = open(a.sample, 'rb').read()
if a.find:
return search(blob, a.find)
if not a.out and not a.stats:
raise SystemExit('give an output path, or --find / --stats')
cands = extract(blob, a.min, a.max, a.two_byte)
lengths = run_lengths(cands)
if a.stats:
print('candidates (two-byte included: %s): %d' % (a.two_byte, len(cands)))
print('run-length histogram:')
histogram(lengths)
return 0
pp = load_pp(a.pp)
kept = [s for s in cands if lengths[s] >= a.min_chain]
rows = kept + (sorted(s for s in cands if lengths[s] < a.min_chain)
if a.keep_isolated else [])
with open(a.out, 'w', encoding='utf-8') as fh:
fh.write('# offset_hex\ttext\trun\tform%s\n' % ('\tin_pp' if pp else ''))
for s in sorted(rows):
_end, text, form = cands[s]
flat = text.replace('\n', '\\n').replace('\t', ' ')
extra = ('\t%d' % (1 if text in pp else 0)) if pp else ''
fh.write('0x%06x\t%s\t%d\t%s%s\n' % (s, flat, lengths[s], form, extra))
one = sum(1 for s in kept if cands[s][2] == 'one-byte')
print('candidates : %d' % len(cands))
print('kept (run >= %d) : %d (one-byte %d / utf16le %d)'
% (a.min_chain, len(kept), one, len(kept) - one))
print('below threshold : %d%s' % (
len(cands) - len(kept), '' if a.keep_isolated else ' [--keep-isolated to keep them]'))
if pp:
hits = sum(1 for s in kept if cands[s][1] in pp)
print('also in --pp listing : %d / %d (absence is NOT a false positive: the'
% (hits, len(kept)))
print(' decompiler listing is incomplete by design)')
if a.two_byte:
print('WARNING: --two-byte detection is unreliable (see module docstring);')
print(' verify each UTF-16 hit before trusting it.')
print('written %s' % a.out)
return 0
if __name__ == '__main__':
sys.exit(main())

185
vendor-new/apk-reverse/skills/apk-reverse/scripts/dart_pprefs.py Normal file
View File

@@ -0,0 +1,185 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Dart AOT object-pool reference index: pool offset -> the instructions that load it.
WHY THIS EXISTS
---------------
blutter's `pp.txt` tells you what is *in* the object pool (strings, types, closures,
field metadata) but not which code reads it. "Who uses this string" is the most frequent
question in Dart AOT work, and it needs an index you build once and query constantly.
Building that index with a general-purpose disassembler is the trap here: decoding every
instruction of a multi-MB `.text` with detail enabled and asking for per-instruction
register writes costs minutes of CPU and 1-2 GB of peak memory, and on a 16 GB host it is
indistinguishable from a hang -- no output, no progress, nothing to interrupt. Only three
encodings can reach the pool, so this decodes them from the raw 32-bit words instead:
seconds, and a flat memory profile.
Recognised forms (AArch64, little endian), with x27 = pool base:
add xD, x27, #imm12, lsl #12 -> xD = x27 + (imm12 << 12)
ldr XD, [xN, #imm12 * size] -> GP register file, size from bits 31-30
ldr DD, [xN, #imm12 * size] -> SIMD&FP register file (bit 26 set)
ldur XD/DD, [xN, #simm9] -> unscaled variant, both files
Both register files must be covered. Restricting the LDR form to the GP words
(0xF94/0xB94) silently drops `ldr dN, [x27, #imm]` (0xFD400000), which is how every
*double* constant in the pool is read -- an entire class of references disappears from
the index. On a real Dart 3.6.0 libapp.so that was 302 pool offsets and 1,802 sites lost
(measured, see the verification record); one of them had 150 reference sites that would
all have been reported as zero.
A reference is either a direct load whose base is x27, or a load consuming the register an
ADD just derived from x27. The compiler emits that pair back to back, so a small window
keeps it exact without full liveness analysis.
USAGE
-----
python dart_pprefs.py libapp.so pp_refs.json # build the index
python dart_pprefs.py --lookup pp_refs.json 0x1d1a8 0xc9d0
python dart_pprefs.py libapp.so out.json --window 5 # widen the ADD->LDR window
NOTES
-----
* Offsets are **pool** offsets -- the `pp+0x...` space used by pp.txt -- not file offsets.
Keep the two spaces apart in your notes; conflating them costs hours.
* One offset commonly has many reference sites. Count them before editing the entry:
a shared constant will change behaviour everywhere at once (see references/dart-aot.md).
"""
import argparse
import json
import struct
import sys
POOL_REG = 27 # x27: object-pool base in Dart AOT on arm64
DEFAULT_WINDOW = 3 # instructions an ADD-derived address may survive
def text_section(path):
"""Return (vaddr, bytes) of .text, parsing ELF64 section headers directly."""
blob = open(path, 'rb').read()
if blob[:4] != b'\x7fELF':
raise SystemExit('not an ELF file: %s' % path)
if blob[4] != 2:
raise SystemExit('not ELF64: %s' % path)
e_shoff = struct.unpack_from('<Q', blob, 0x28)[0]
e_shentsize = struct.unpack_from('<H', blob, 0x3A)[0]
e_shnum = struct.unpack_from('<H', blob, 0x3C)[0]
e_shstrndx = struct.unpack_from('<H', blob, 0x3E)[0]
def sh(i):
return struct.unpack_from('<IIQQQQ', blob, e_shoff + i * e_shentsize)
_, _, _, _, shstr_off, shstr_size = sh(e_shstrndx)
strs = blob[shstr_off:shstr_off + shstr_size]
for i in range(e_shnum):
name, _typ, _flags, addr, off, size = sh(i)
end = strs.find(b'\x00', name)
if strs[name:end].decode('ascii', 'replace') == '.text':
return addr, blob[off:off + size]
raise SystemExit('.text not found; pass --text with its virtual address')
def sx(value, bits):
m = 1 << (bits - 1)
return (value ^ m) - m
def scan(text_addr, blob, window):
refs = {}
pending = {}
words = struct.unpack_from('<%dI' % (len(blob) // 4), blob, 0)
for i, w in enumerate(words):
addr = text_addr + i * 4
# add xD, x27, #hi, lsl #12
if w & 0xFFC00000 == 0x91400000 and (w >> 5) & 0x1F == POOL_REG:
# Keep the shift out of the mask expression: `a & m << n` binds as
# `a & (m << n)` in Python, which silently produces a wrong base.
imm = (w >> 10) & 0xFFF
pending[w & 0x1F] = [imm << 12, i]
continue
# LDR (immediate) / LDUR across BOTH register files. Bits 31-30 give the access
# size and therefore the immediate scale; bit 26 selects the SIMD&FP file.
# Matching only the GP words (0xF94/0xB94) drops `ldr dN, [x27, #imm]`
# (0xFD400000) and with it every double constant read from the pool.
ldr_scale = {0xF9400000: 3, 0xB9400000: 2, 0x79400000: 1, 0x39400000: 0,
0xFD400000: 3, 0xBD400000: 2, 0x7D400000: 1, 0x3DC00000: 4,
}.get(w & 0xFFC00000)
ldur = (w & 0xFFE00C00) in (0xF8400000, 0xB8400000, 0x78400000, 0x38400000,
0xFC400000, 0xBC400000, 0x7C400000, 0x3C400000)
if ldr_scale is not None or ldur:
rn = (w >> 5) & 0x1F
rd = w & 0x1F
if ldr_scale is not None:
disp = ((w >> 10) & 0xFFF) << ldr_scale
else:
disp = sx((w >> 12) & 0x1FF, 9)
if rn == POOL_REG:
refs.setdefault(disp, []).append(addr)
elif rn in pending:
base, seq = pending.pop(rn)
if i - seq <= window:
refs.setdefault(base + disp, []).append(addr)
pending.pop(rd, None) # this load overwrote rd
continue
if w & 0xFFC00000 in (0x91000000, 0x91400000):
pending.pop(w & 0x1F, None)
for r in [r for r, (_o, s) in pending.items() if i - s > window]:
del pending[r]
return refs
def build(so, out, window):
text_addr, blob = text_section(so)
refs = scan(text_addr, blob, window)
with open(out, 'w', encoding='utf-8') as fh:
json.dump({hex(k): v for k, v in refs.items()}, fh)
total = sum(len(v) for v in refs.values())
print('.text vaddr=0x%x bytes=%d' % (text_addr, len(blob)))
print('distinct pool offsets : %d' % len(refs))
print('reference sites : %d' % total)
for off in sorted(refs, key=lambda k: -len(refs[k]))[:8]:
print(' pp+0x%-8x %4d sites' % (off, len(refs[off])))
print('written %s' % out)
def lookup(path, offsets):
refs = {int(k, 16): v for k, v in json.load(open(path, encoding='utf-8')).items()}
for spec in offsets:
off = int(spec, 16)
sites = sorted(refs.get(off, []))
print('pp+0x%x: %d ref site(s)' % (off, len(sites)))
for s in sites:
print(' 0x%x' % s)
def main():
ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])
ap.add_argument('sample', nargs='?', help='libapp.so (build mode)')
ap.add_argument('out', nargs='?', help='output json (build mode)')
ap.add_argument('--lookup', metavar='REFS_JSON', help='query an existing index')
ap.add_argument('--window', type=int, default=DEFAULT_WINDOW,
help='instructions an ADD-derived address may survive (default %d)'
% DEFAULT_WINDOW)
a = ap.parse_args()
if a.lookup:
rest = sys.argv[sys.argv.index('--lookup') + 2:]
if not rest:
raise SystemExit('give at least one hex pool offset')
lookup(a.lookup, rest)
return 0
if not (a.sample and a.out):
raise SystemExit('usage: dart_pprefs.py <libapp.so> <out.json>')
build(a.sample, a.out, a.window)
return 0
if __name__ == '__main__':
sys.exit(main())

347
vendor-new/apk-reverse/skills/apk-reverse/scripts/datastore_inject.py Normal file
View File

@@ -0,0 +1,347 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Encode or inject AndroidX DataStore (Preferences) protobuf entries.
WHY THIS EXISTS
---------------
Some feature gates are not decided in code but in a preferences file that the app
reads at runtime, e.g. a "free until <timestamp>" value. Changing the code alone can
then be pointless, while writing the state directly proves the hypothesis in one
shot: if the behavior changes, the key really is the gate; if it does not, the gate
is elsewhere (usually server-side).
WHAT THE FILE IS
----------------
DataStore/Preferences serializes to a protobuf on disk. On a device it lives under
the app's private directory, typically:
<app_files_dir>/datastore/<name>.preferences_pb
and the container is:
message PreferenceMap { map<string, Value> preferences = 1; }
message Value { oneof { bool boolean = 1; float float = 2; int32 integer = 3;
int64 long = 4; string string = 5; double double = 6;
bytes bytes = 8; } }
The map field is the part people get wrong: every entry needs BOTH the outer
0x0A <len> (the map field itself) AND the inner 0x0A <len key> <key> +
0x12 <len value> <value>. Writing only the inner form produces a file that
deserializes into nothing, and some apps then die on the next read with an
exception that is swallowed by their crash reporter -- no stack, no clue.
Bytes writes here are the exact encoding, so this script doubles as a reference.
USAGE
-----
# write a fresh file with one big "never expires" timestamp
python datastore_inject.py --file example.preferences_pb \
--key example_expiry_epoch_ms --type long --value 4102444800000
# keep every other key from an existing file and replace/add just this one
python datastore_inject.py --in pulled.preferences_pb --file patched.preferences_pb \
--key example_expiry_epoch_ms --type long --value 4102444800000
# add more than one entry
python datastore_inject.py --file out.pb --key example_flag --type bool --value true \
--extra example_name=hello:string
# inspect a pulled file (what keys exist, and what they decode to)
python datastore_inject.py --in pulled.preferences_pb --list
Deploying the result: the app must not be running while you overwrite the file
(it will rewrite the file on exit), so push, fix ownership/selinux context if
needed, then start the app. Verify by reading the key back with --list.
Pure standard library, Python 3.9+.
"""
import argparse
import struct
import sys
DEFAULT_FILE = 'example.preferences_pb'
DEFAULT_KEY = 'example_expiry_epoch_ms'
DEFAULT_VALUE = '4102444800000' # ~2100-01-01 in ms: "far future" in either unit
TYPES = ('long', 'int', 'bool', 'float', 'double', 'string', 'bytes')
# tag = (field_number << 3) | wire_type, per the Value oneof above.
VALUE_TAG = {
'bool': (1, 0),
'float': (2, 5),
'integer': (3, 0),
'long': (4, 0),
'string': (5, 2),
'double': (6, 1),
'bytes': (8, 2),
}
def varint(n):
"""Unsigned LEB128. Negative int32/int64 are encoded as 64-bit twos complement."""
if n < 0:
n += 1 << 64
out = bytearray()
while True:
b = n & 0x7F
n >>= 7
if n:
out.append(b | 0x80)
else:
out.append(b)
return bytes(out)
def read_varint(buf, pos):
result = 0
shift = 0
while True:
if pos >= len(buf):
raise ValueError('truncated varint at offset %d' % pos)
b = buf[pos]
pos += 1
result |= (b & 0x7F) << shift
if not (b & 0x80):
return result, pos
shift += 7
if shift > 70:
raise ValueError('varint too long at offset %d' % pos)
def tag_bytes(field, wire):
return varint((field << 3) | wire)
def encode_value(kind, raw):
"""Encode the Value payload (the bytes after the inner 0x12 <len>)."""
if kind == 'bool':
field, wire = VALUE_TAG['bool']
truthy = str(raw).strip().lower() in ('1', 'true', 'yes', 'on')
return tag_bytes(field, wire) + varint(1 if truthy else 0)
if kind in ('long', 'int'):
field, wire = VALUE_TAG['long' if kind == 'long' else 'integer']
return tag_bytes(field, wire) + varint(int(str(raw), 0))
if kind == 'float':
field, wire = VALUE_TAG['float']
return tag_bytes(field, wire) + struct.pack('<f', float(raw))
if kind == 'double':
field, wire = VALUE_TAG['double']
return tag_bytes(field, wire) + struct.pack('<d', float(raw))
if kind == 'string':
field, wire = VALUE_TAG['string']
data = str(raw).encode('utf-8')
return tag_bytes(field, wire) + varint(len(data)) + data
if kind == 'bytes':
field, wire = VALUE_TAG['bytes']
data = bytes.fromhex(str(raw))
return tag_bytes(field, wire) + varint(len(data)) + data
raise ValueError('unknown type: %s' % kind)
def encode_entry(key, value_payload):
"""Encode one PreferenceMap entry.
Two levels of 0x0A on purpose:
outer: 0x0A <len(inner)> <- the map field (field 1)
inner: 0x0A <len(key)> <key> 0x12 <len(value)> <value>
Dropping the outer tag+length is the classic mistake: DataStore then
deserializes an empty map, and the failure surfaces far away from here.
"""
kb = key.encode('utf-8')
inner = (tag_bytes(1, 2) + varint(len(kb)) + kb
+ tag_bytes(2, 2) + varint(len(value_payload)) + value_payload)
return tag_bytes(1, 2) + varint(len(inner)) + inner
def split_entries(data):
"""Split a PreferenceMap into raw top-level field slices."""
pos = 0
entries = []
while pos < len(data):
start = pos
tag, pos = read_varint(data, pos)
wire = tag & 0x07
if wire == 2:
length, pos = read_varint(data, pos)
pos += length
elif wire == 0:
_, pos = read_varint(data, pos)
elif wire == 5:
pos += 4
elif wire == 1:
pos += 8
else:
raise ValueError('unsupported wire type %d at offset %d' % (wire, start))
if pos > len(data):
raise ValueError('truncated field at offset %d' % start)
entries.append((tag, data[start:pos]))
return entries
def decode_entry(raw):
"""Return (key, value_payload) of one entry, or (None, None) if malformed."""
try:
pos = 0
tag, pos = read_varint(raw, pos)
if (tag >> 3) != 1 or (tag & 7) != 2:
return None, None
length, pos = read_varint(raw, pos)
inner = raw[pos:pos + length]
p = 0
t1, p = read_varint(inner, p)
if (t1 >> 3) != 1:
return None, None
klen, p = read_varint(inner, p)
key = inner[p:p + klen].decode('utf-8', 'replace')
p += klen
t2, p = read_varint(inner, p)
if (t2 >> 3) != 2:
return None, None
vlen, p = read_varint(inner, p)
return key, inner[p:p + vlen]
except Exception:
return None, None
def describe_value(payload):
"""Best-effort decode of a Value payload for the --list output."""
try:
pos = 0
tag, pos = read_varint(payload, pos)
field, wire = tag >> 3, tag & 7
if field == 1 and wire == 0:
val, _ = read_varint(payload, pos)
return 'bool=%s' % bool(val)
if field == 3 and wire == 0:
val, _ = read_varint(payload, pos)
return 'integer=%d' % val
if field == 4 and wire == 0:
val, _ = read_varint(payload, pos)
return 'long=%d' % val
if field == 2 and wire == 5:
return 'float=%r' % struct.unpack('<f', payload[pos:pos + 4])[0]
if field == 6 and wire == 1:
return 'double=%r' % struct.unpack('<d', payload[pos:pos + 8])[0]
if field == 5 and wire == 2:
length, pos = read_varint(payload, pos)
return 'string=%r' % payload[pos:pos + length].decode('utf-8', 'replace')
if field == 8 and wire == 2:
length, pos = read_varint(payload, pos)
return 'bytes=%s' % payload[pos:pos + length].hex()
except Exception:
pass
return 'raw=%s' % payload.hex()
def parse_extra(items):
"""--extra key=value[:type] -> [(key, type, value)]"""
out = []
for item in items:
if '=' not in item:
raise SystemExit('--extra expects key=value[:type], got %r' % item)
key, rest = item.split('=', 1)
kind = 'string'
if ':' in rest:
rest, kind = rest.rsplit(':', 1)
if kind not in TYPES:
raise SystemExit('--extra %s: unknown type %r (choose from %s)'
% (key, kind, ', '.join(TYPES)))
out.append((key, kind, rest))
return out
def main():
ap = argparse.ArgumentParser(
description='Encode or inject AndroidX DataStore (Preferences) protobuf '
'entries, with the correct two-level map encoding.')
ap.add_argument('--file', default=DEFAULT_FILE,
help='output .pb path (default: %s)' % DEFAULT_FILE)
ap.add_argument('--key', default=DEFAULT_KEY,
help='preference key to write (default: %s)' % DEFAULT_KEY)
ap.add_argument('--value', default=DEFAULT_VALUE,
help='value for --key (default: %s)' % DEFAULT_VALUE)
ap.add_argument('--type', choices=TYPES, default='long',
help='value type (default: long)')
ap.add_argument('--in', dest='existing', default=None,
help='existing .pb to read: preserves all other keys')
ap.add_argument('--extra', action='append', default=[],
help='key=value[:type], repeatable, added alongside --key')
ap.add_argument('--list', action='store_true',
help='list the entries of --in and exit (no writing)')
ap.add_argument('--dry-run', action='store_true',
help='print what would be written, write nothing')
args = ap.parse_args()
if args.list:
if not args.existing:
raise SystemExit('--list needs --in <file.preferences_pb>')
with open(args.existing, 'rb') as fh:
data = fh.read()
print('[in] %s (%d bytes)' % (args.existing, len(data)))
shown = 0
for tag, raw in split_entries(data):
key, payload = decode_entry(raw)
if key is None:
print(' <unparsed field tag=0x%02x, %d bytes>' % (tag, len(raw)))
continue
print(' %-32s %s (value=%d bytes)' % (key, describe_value(payload), len(payload)))
shown += 1
print('[ok] %d entry(ies)' % shown)
return 0
# Build the entry list: preserved entries first, then the injected ones.
chunks = []
if args.existing:
with open(args.existing, 'rb') as fh:
original = fh.read()
replaced = {args.key}
replaced.update(k for k, _, _ in parse_extra(args.extra))
kept = 0
for _tag, raw in split_entries(original):
key, _payload = decode_entry(raw)
if key is None:
chunks.append(raw) # unknown field: keep it verbatim
continue
if key in replaced:
print('[keep-skip] %s is being replaced' % key)
continue
chunks.append(raw)
kept += 1
print('[in] preserved %d existing entry(ies) from %s' % (kept, args.existing))
target = encode_entry(args.key, encode_value(args.type, args.value))
chunks.append(target)
print('[inject] %s (%s) = %s -> %d bytes' % (args.key, args.type, args.value, len(target)))
for key, kind, value in parse_extra(args.extra):
entry = encode_entry(key, encode_value(kind, value))
chunks.append(entry)
print('[inject] %s (%s) = %s -> %d bytes' % (key, kind, value, len(entry)))
data = b''.join(chunks)
# Self-check: re-parse what we just built. A file that does not parse back is
# exactly the failure mode that kills the app with no usable stack.
found = {}
for _tag, raw in split_entries(data):
key, payload = decode_entry(raw)
if key is not None:
found[key] = describe_value(payload)
for key in [args.key] + [k for k, _t, _v in parse_extra(args.extra)]:
if key not in found:
raise SystemExit('internal error: %r does not parse back; refusing to write' % key)
print('[check] re-parsed %d entry(ies); %s = %s' % (len(found), args.key, found[args.key]))
print('[hex] %s' % data.hex(' '))
if args.dry_run:
print('[dry-run] nothing written')
return 0
with open(args.file, 'wb') as fh:
fh.write(data)
print('[ok] wrote %s (%d bytes)' % (args.file, len(data)))
print('[note] push it to <app_files_dir>/datastore/<name>.preferences_pb while the app '
'is NOT running, then start the app and read the key back with --list')
return 0
if __name__ == '__main__':
sys.exit(main())

301
vendor-new/apk-reverse/skills/apk-reverse/scripts/device_shell.py Normal file
View File

@@ -0,0 +1,301 @@
#!/usr/bin/env python3
"""Device-side command construction: quoting, argv, and identifier validation.
Why this module exists: `adb shell` hands your string to a **device shell** (toybox `sh`), and two of
this kit's scripts had different, partially-wrong ideas about that. One escaped only double quotes, so
a `;`, a `$(...)` or a backtick in a value still reached the shell as syntax; the other quoted
correctly but let a malformed value go through to fail later, far from the caller, as a confusing
`grep: no such file` or an empty result. Both are the same defect seen from two sides: **a value that
becomes shell syntax**.
None of this is exotic. The values are package names, component names, paths and sizes that the user
typed or that came out of a manifest, and the failure mode when one is malformed is a wasted round,
not an exploit. The rules are still worth enforcing in one place:
* **Validate what has a known shape.** An Android package name has a defined grammar; a value that
does not match it will never select a package, so rejecting it here is strictly better than
passing it to the device and reading an empty result as "not installed".
* **Quote what does not.** Paths and free-form text are escaped with POSIX single quotes -- the
only form that makes `$`, backticks, `;`, `|` and newlines literal to `sh`.
* **Prefer an argv array over a string** whenever the command is not being run through `su -c`.
`['adb', 'shell', 'cmd', arg]` sends `arg` as a separate argument; adb quotes it for the device
shell, and there is no string to get the quoting wrong in.
Usage as a module:
from device_shell import validate_package, validate_component, sh_quote, su_wrap, adb_shell
adb_shell(serial, ['dumpsys', 'package', validate_package(pkg)]) # argv, no string
su_wrap(['cat', validate_device_path(path)]) # -> one quoted string
Usage as a tool:
python device_shell.py --check com.example.app
python device_shell.py --quote "a b;c"
python device_shell.py --selftest
Exit codes: 0 success / 1 the value was refused / 2 usage error / 4 internal error.
"""
import argparse
import re
import sys
# Android's own grammar: one or more Java-package segments, each starting with a letter.
PACKAGE_RE = re.compile(r'^[A-Za-z][A-Za-z0-9_]*(\.[A-Za-z][A-Za-z0-9_]*)+$')
# A component is `pkg/Class`, `pkg/.Class` or `pkg/full.Class.Path`.
COMPONENT_RE = re.compile(
r'^(?P<pkg>[A-Za-z][A-Za-z0-9_]*(\.[A-Za-z][A-Za-z0-9_]*)+)'
r'/(?P<cls>\.?[A-Za-z][A-Za-z0-9_.$]*)$')
# Device paths we are willing to interpolate: absolute, no control characters, no shell metachars.
PATH_SAFE_RE = re.compile(r'^/[A-Za-z0-9_./@+:=~-]*$')
# A user id as `pm`/`dumpsys` reports it, or a process id.
PID_RE = re.compile(r'^[1-9][0-9]{0,6}$')
class Refused(ValueError):
"""A value failed validation. Callers translate this into exit code 1, never a traceback."""
def _reject(kind, value, why):
raise Refused('%s is not valid: %r -- %s' % (kind, value, why))
def validate_package(value):
"""A package name, or refuse. Raises `Refused`."""
if not isinstance(value, str) or not value:
_reject('package name', value, 'empty')
if len(value) > 255:
_reject('package name', value, 'longer than the platform maximum')
if not PACKAGE_RE.match(value):
_reject('package name', value,
'must be dot-separated segments each starting with a letter (Android grammar)')
return value
def validate_component(value, default_package=None):
"""`pkg/Class`, `pkg/.Class` or a bare `.Class` when `default_package` is given."""
if not isinstance(value, str) or not value:
_reject('component', value, 'empty')
if '/' not in value and default_package:
value = '%s/%s' % (validate_package(default_package), value)
m = COMPONENT_RE.match(value)
if not m:
_reject('component', value, 'expected package/Class with a valid package part')
validate_package(m.group('pkg'))
if m.group('cls').endswith('.'):
_reject('component', value, 'class part ends with a dot')
return value
def validate_device_path(value):
"""An absolute device path with no character that could be shell syntax."""
if not isinstance(value, str) or not value:
_reject('device path', value, 'empty')
if len(value) > 4096:
_reject('device path', value, 'unreasonably long')
if not PATH_SAFE_RE.match(value):
_reject('device path', value,
'must be absolute and free of whitespace, quotes and shell metacharacters')
if '..' in value.split('/'):
_reject('device path', value, 'contains a parent-directory segment')
return value
def validate_pid(value):
if isinstance(value, int):
value = str(value)
if not isinstance(value, str) or not PID_RE.match(value):
_reject('pid', value, 'expected a positive decimal pid')
return value
def sh_quote(value):
"""POSIX single-quote quoting: the one form that makes every metacharacter literal."""
if not isinstance(value, str):
value = str(value)
return "'" + value.replace("'", "'\\''") + "'"
def su_wrap(argv):
"""Wrap already-validated argv into a single `su -c '<cmd>'` string for the device shell."""
if isinstance(argv, str):
argv = [argv]
return 'su -c %s' % sh_quote(' '.join(argv))
def adb_shell(serial=None, argv=(), root=False, quote=True):
"""Build an adb argv list. `root=True` routes through `su -c`; `quote=False` sends argv directly."""
if isinstance(argv, str):
argv = [argv]
base = ['adb'] + (['-s', serial] if serial else []) + ['shell']
if root:
return base + [su_wrap(argv)]
if not quote:
return base + list(argv)
return base + [' '.join(sh_quote(a) for a in argv)]
def selftest():
"""Assert the properties this module claims. Returns a list of (name, ok, detail)."""
cases = []
payloads = ['a; rm -rf /sdcard/x', 'a$(id)b', 'a`id`b', 'a b', 'a\nb', 'a|b', "a'b"]
for p in payloads:
quoted = sh_quote(p)
# A single-quoted POSIX string must contain no unescaped quote and must round-trip.
ok = quoted.startswith("'") and quoted.endswith("'")
inner = quoted[1:-1]
ok = ok and re.fullmatch(r"[^']*('\\''[^']*)*", inner) is not None
cases.append(('sh_quote makes %r inert' % p, ok, quoted))
for good in ('com.example.app', 'a.b.c_d.e1'):
try:
validate_package(good)
cases.append(('accepts valid package %r' % good, True, ''))
except Refused as exc:
cases.append(('accepts valid package %r' % good, False, str(exc)))
for bad in ('', 'nodots', 'com..app', '1com.example', 'com.example.app;rm -rf /',
'com.example.app$(id)', 'com.example.app`id`'):
try:
validate_package(bad)
cases.append(('refuses invalid package %r' % bad, False, 'it was accepted'))
except Refused as exc:
cases.append(('refuses invalid package %r' % bad, True, str(exc)))
for good in ('/data/local/tmp/x', '/sdcard/Android/data/com.example.app/files'):
try:
validate_device_path(good)
cases.append(('accepts valid path %r' % good, True, ''))
except Refused as exc:
cases.append(('accepts valid path %r' % good, False, str(exc)))
for bad in ('relative/path', '/data/../etc/passwd', '/sdcard/x;id', '/sdcard/x$(id)',
'/sdcard/a b'):
try:
validate_device_path(bad)
cases.append(('refuses unsafe path %r' % bad, False, 'it was accepted'))
except Refused as exc:
cases.append(('refuses unsafe path %r' % bad, True, str(exc)))
try:
validate_component('com.example.app/.MainActivity')
validate_component('.MainActivity', default_package='com.example.app')
cases.append(('accepts component forms', True, ''))
except Refused as exc:
cases.append(('accepts component forms', False, str(exc)))
for bad in ('com.example.app', 'com.example.app/', 'nodots/Main'):
try:
validate_component(bad)
cases.append(('refuses bad component %r' % bad, False, 'it was accepted'))
except Refused as exc:
cases.append(('refuses bad component %r' % bad, True, str(exc)))
argv = adb_shell('SERIAL', ['dumpsys', 'package', validate_package('com.example.app')])
# With quote=True the adb argv is `adb -s SERIAL shell "<one string for the device shell>"`, so
# the package must appear as a *quoted* token inside that string; with quote=False it must be its
# own argv element. Both forms are asserted -- an assertion that accepts only one of them is how
# this selftest first failed, by hard-coding a length instead of checking the property.
quoted_form = argv[-1] if argv[-2] == 'shell' else ''
direct = adb_shell('SERIAL', ['dumpsys', 'package', 'com.example.app'], quote=False)
cases.append(('quoted form carries the package as one quoted token',
"'com.example.app'" in quoted_form, quoted_form))
cases.append(('direct form carries the package as one argv element',
direct[-1] == 'com.example.app' and direct[-2] == 'package', ' '.join(direct)))
cases.append(('root form routes through su -c as a single string',
adb_shell('SERIAL', ['id'], root=True)[-1].startswith('su -c '),
adb_shell('SERIAL', ['id'], root=True)[-1]))
wrapped = su_wrap(['cat', '/data/local/tmp/x'])
cases.append(('su wrapper is a single quoted string',
wrapped.startswith('su -c \'') and wrapped.endswith('\''), wrapped))
return cases
def exit_code_for(main_fn):
"""Run a script's `main()` and turn a refusal into exit 2 with one line, never a traceback.
Kept here so every script that validates device values reports the same way: a malformed
identifier is a *usage* error (2), distinguishable from "the device said no" (1) and from a
defect in the tool (4).
"""
try:
return main_fn()
except Refused as exc:
print('refused: %s' % exc, file=sys.stderr)
print('RESULT=refused')
return 2
def main(argv=None):
ap = argparse.ArgumentParser(
prog='device_shell.py',
description='Validate Android identifiers and quote values for the device shell. '
'Importable as a module; the CLI is for checking a value by hand.',
epilog='examples:\n'
' device_shell.py --check com.example.app\n'
' device_shell.py --check com.example.app/.MainActivity\n'
' device_shell.py --quote "a b;c"\n'
' device_shell.py --selftest\n')
ap.add_argument('--check', metavar='VALUE',
help='validate as a package name')
ap.add_argument('--check-component', metavar='VALUE',
help='validate as package/Class, or as .Class together with --package')
ap.add_argument('--package', metavar='PKG', help='default package for --check-component')
ap.add_argument('--check-path', metavar='PATH', help='validate a device path')
ap.add_argument('--quote', metavar='VALUE', help='print the POSIX-quoted form')
ap.add_argument('--su', metavar='ARGV', help='print a su -c wrapper for a space-separated argv')
ap.add_argument('--selftest', action='store_true', help='assert this module\'s own claims')
ap.add_argument('--json', action='store_true')
args = ap.parse_args(argv)
if args.selftest:
cases = selftest()
bad = [c for c in cases if not c[1]]
for name, ok, detail in cases:
print(' %-52s %s' % (name, 'ok' if ok else 'FAIL'))
print()
print('RESULT=%s' % ('selftest_ok' if not bad else 'selftest_failed'))
return 0 if not bad else 4
try:
if args.check:
print('accepted: %s' % validate_package(args.check))
print('RESULT=accepted')
return 0
if args.check_component:
print('accepted: %s' % validate_component(args.check_component, args.package))
print('RESULT=accepted')
return 0
if args.check_path:
print('accepted: %s' % validate_device_path(args.check_path))
print('RESULT=accepted')
return 0
if args.quote:
if args.su:
print(su_wrap([args.quote]))
else:
print(sh_quote(args.quote))
print('RESULT=quoted')
return 0
if args.su:
print(su_wrap(args.su.split()))
print('RESULT=quoted')
return 0
except Refused as exc:
print('refused: %s' % exc, file=sys.stderr)
print('RESULT=refused')
return 1
ap.print_help()
print('RESULT=usage_error')
return 2
if __name__ == '__main__':
if hasattr(sys.stdout, 'reconfigure'):
try:
sys.stdout.reconfigure(encoding='utf-8')
except (ValueError, OSError):
pass
sys.exit(main())

129
vendor-new/apk-reverse/skills/apk-reverse/scripts/devsh.py Normal file
View File

@@ -0,0 +1,129 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Quoting-safe ADB helper.
Why this exists: if you build device commands inside a host shell, the host expands
`$`, `|`, `>`, and quotes *before* adb sees them. Symptoms are host-side errors that
look like device failures -- "Could not find a part of the path", "no closing quote",
"Missing type name after '['", unexpanded variables. Windows PowerShell is the worst
offender (it also eats `$var:`, `[^"...`, `$(...)`).
Calling adb from Python with an argument list avoids all of it.
Configuration (first match wins):
1. CLI flags: --serial / --adb
2. env: ADB_SERIAL / ADB_PATH
3. defaults below
Usage
-----
python devsh.py sh "pm list packages | grep myapp" # as the shell user
python devsh.py su "pm grant <pkg> android.permission.X"
python devsh.py sh "dumpsys window | grep mCurrentFocus"
python devsh.py pull /data/local/tmp/x.apk ./x.apk
python devsh.py push ./x.apk /data/local/tmp/x.apk
python devsh.py dev # list devices
`su` wraps the command in `su -c "..."`; internal double quotes are escaped.
"""
import argparse
import os
import subprocess
import sys
try:
import device_shell as _shell # same directory as this script
except ImportError: # run from the repository root instead
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import device_shell as _shell
DEFAULT_ADB = 'adb'
DEFAULT_SERIAL = None
def resolve():
ap = argparse.ArgumentParser(add_help=False)
ap.add_argument('--adb')
ap.add_argument('--serial')
known, _ = ap.parse_known_args()
adb = known.adb or os.environ.get('ADB_PATH') or DEFAULT_ADB
serial = known.serial or os.environ.get('ADB_SERIAL') or DEFAULT_SERIAL
return adb, serial
def run(adb, serial, args, timeout=300):
cmd = [adb]
if serial:
cmd += ['-s', serial]
cmd += args
r = subprocess.run(cmd, capture_output=True, text=True, errors='replace', timeout=timeout)
out = r.stdout or ''
if (r.stderr or '').strip():
out += '\n[stderr]\n' + r.stderr
return r.returncode, out
def main():
if len(sys.argv) < 2:
print(__doc__)
return 2
adb, serial = resolve()
argv = [a for a in sys.argv[1:] if not a.startswith('--adb=') and not a.startswith('--serial=')]
# strip flag/value pairs
cleaned = []
skip = False
for a in argv:
if skip:
skip = False
continue
if a in ('--adb', '--serial'):
skip = True
continue
cleaned.append(a)
argv = cleaned
if len(argv) < 2 and argv and argv[0] not in ('dev',):
print(__doc__)
return 2
mode = argv[0]
try:
if mode == 'dev':
rc, out = run(adb, None, ['devices', '-l'])
elif mode in ('sh', 'shell'):
rc, out = run(adb, serial, ['shell', argv[1]])
elif mode == 'su':
# POSIX single-quote quoting, via the shared module. The previous form escaped only double
# quotes, so `;`, `$( )` and backticks still reached the device shell as syntax -- and this
# helper sits directly under an agent's hands. See scripts/device_shell.py.
rc, out = run(adb, serial, ['shell', _shell.su_wrap([argv[1]])])
elif mode == 'suf': # read a device text file
# `suf` interpolates a path and a byte count into a device command, so both are validated:
# a path with a space or a metacharacter is refused here rather than mangled on the device.
path = _shell.validate_device_path(argv[1])
size = argv[2] if len(argv) > 2 else '6000'
if not size.isdigit():
print('suf: size must be a decimal byte count, got %r' % size, file=sys.stderr)
return 2
rc, out = run(adb, serial, ['shell', _shell.su_wrap(['head', '-c', size, path])])
elif mode == 'pull':
rc, out = run(adb, serial, ['pull', argv[1], argv[2]])
elif mode == 'push':
rc, out = run(adb, serial, ['push', argv[1], argv[2]])
else:
print(__doc__)
return 2
except _shell.Refused as exc:
# A malformed device value is a usage error (2): distinguishable by a caller from "the device
# said no" (1) and from a defect in this tool (4). Never a traceback.
print('refused: %s' % exc, file=sys.stderr)
print('RESULT=refused')
return 2
sys.stdout.write(out)
return 0 if rc == 0 else 1
if __name__ == '__main__':
sys.exit(main())

170
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_check_verifier.py Normal file
View File

@@ -0,0 +1,170 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Tier-3 verifier check: does any branch target a `move-result*` instruction?
WHY THIS EXISTS
---------------
`move-result*` is not an ordinary instruction. The verifier requires it to be the
immediate successor of the invoke that produced the value it copies. A branch that
lands on a `move-result` therefore bypasses the producer, and the class fails to
load with:
VerifyError: ... copyRes1 v11 <- result0 type=Undefined
This matters specifically for byte-level patching. Redirecting a branch
(`if-*` -> `goto`) is the intuitive way to force one side of a condition, and it
is the way that silently creates this violation. Replacing the branch with a pair
of `nop`s cannot: it removes a control-flow edge instead of adding one.
The check must be done on the control-flow graph, not linearly. A "is the
previous instruction the producer?" scan over the instruction list reports clean
on a method that has a branch jumping straight onto a `move-result`, because the
producer does sit immediately before it in the stream. Only asking "does ANY
branch target this offset" sees the problem.
Usage
-----
python dex_check_verifier.py app.apk # whole image
python dex_check_verifier.py app.apk --class 'Lcom/ex/Foo;'
python dex_check_verifier.py before.dex after.dex # compare two builds
Exit codes: 0 = no findings, 1 = findings present, 2 = usage/parse error. The
non-zero-on-findings convention is deliberate so a pipeline can gate on it.
"""
import argparse
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from dexutil import IF_TEST, IF_TESTZ, MOVE_RESULT_OPS, load_dex # noqa: E402
def analyse_method(dex, code_off):
"""Return (findings, clean, detail) for one method body.
findings: list of (branch_offset, move_result_offset, is_conditional)
clean: whether the decode consumed exactly insns_size units
detail: human string when not clean
ONLY conditional branches are reported. An unconditional `goto` that lands on
a `move-result` is a normal compiler-generated merge point: the producer runs
on the incoming path and the goto is a *forward* jump to the shared epilogue,
so nothing is bypassed. Reporting those as violations makes the tool unusable
-- on a real sample they outnumber genuine findings by 20:1 and are pure
compiler output.
The real hazard is narrower: redirecting a CONDITIONAL branch so that it lands
directly on a `move-result`. That skips the producing invoke on the taken path.
"""
insns = list(dex.decode(code_off))
info = dex.code_info(code_off)
expected = info["insns_off"] + info["insns_size"] * 2
ended = (insns[-1]["off"] + insns[-1]["units"] * 2) if insns else info["insns_off"]
clean = (ended == expected)
detail = "" if clean else ("decode ended at 0x%x, method body ends at 0x%x"
% (ended, expected))
move_results = {i["off"] for i in insns if i["op"] in MOVE_RESULT_OPS}
findings = []
for i in insns:
if i["op"] not in IF_TEST and i["op"] not in IF_TESTZ:
continue
t = dex.branch_target(i)
if t is not None and t in move_results:
findings.append((i["off"], t, True))
return findings, clean, detail
def scan_image(path, entry, cls_filter):
dex, entry_name = load_dex(path, entry)
problems = dex.check()
if problems:
for p in problems:
print(" [structure] %s" % p)
raise RuntimeError("structurally broken: %s" % path)
total_methods = 0
total_findings = 0
unclean = 0
for fqcn in dex.class_names():
if cls_filter and cls_filter not in fqcn:
continue
try:
methods = list(dex.methods_of(fqcn))
except Exception:
continue
for _section, _midx, _c, name, desc, code_off in methods:
if code_off == 0:
continue
total_methods += 1
findings, clean, detail = analyse_method(dex, code_off)
if not clean:
unclean += 1
if findings:
total_findings += len(findings)
print(" VIOLATION %s.%s%s" % (fqcn, name, desc))
for b_off, mr_off, _cond in findings:
b_insn = dex.insn_at(code_off, b_off)
mr_insn = dex.insn_at(code_off, mr_off)
print(" conditional branch 0x%x (%s)"
% (b_off, dex.describe(b_insn)))
print(" -> move-result 0x%x (%s)"
% (mr_off, dex.describe(mr_insn)))
print(" the producing invoke is bypassed on the taken "
"path: the class will fail to load")
return {"methods": total_methods, "findings": total_findings,
"unclean": unclean, "entry": entry_name}
def main(argv):
ap = argparse.ArgumentParser(
description="Detect branch-into-move-result verifier violations.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__.split("Usage")[-1][:800])
ap.add_argument("targets", nargs="+", help="one .dex/.apk, or two to compare")
ap.add_argument("--entry", help="archive member (default: first classes*.dex)")
ap.add_argument("--class", dest="cls", help="substring filter on the class name")
args = ap.parse_args(argv[1:])
results = []
for path in args.targets:
print("== %s" % path)
try:
stats = scan_image(path, args.entry, args.cls)
except Exception as exc:
print(" error: %s" % exc)
return 2
results.append(stats)
print(" methods checked: %d violations: %d unclean decodes: %d"
% (stats["methods"], stats["findings"], stats["unclean"]))
print("")
if len(results) >= 2:
before, after = results[0], results[1]
print("== comparison")
print(" violations %d -> %d" % (before["findings"], after["findings"]))
if after["findings"] > before["findings"]:
print(" REGRESSION: your patch introduced %d new violation(s). A "
"branch now lands on a move-result." % (after["findings"] - before["findings"]))
print(" Fix: replace the branch with `nop`s (removes an edge) instead "
"of redirecting it (adds one).")
return 1
if before["findings"] and after["findings"] == before["findings"]:
print(" note: violations exist in BOTH builds, so they are pre-existing "
"and not caused by the patch. Do not 'fix' them as part of this task.")
if any(r["unclean"] for r in results):
print("== WARNING: unclean decodes reported above. Offsets from those "
"methods are unreliable -- widen the instruction format table or "
"cross-check against a disassembler before trusting a patch offset.")
if any(r["findings"] for r in results):
return 1
print("== no branch-into-move-result violations")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))

130
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_classdiff.py Normal file
View File

@@ -0,0 +1,130 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Diff two dex files at the class_defs level: class names + access_flags.
The interesting flag is ACC_INTERFACE (0x200).
Use it to prove that a disassemble -> reassemble round-trip did not damage the dex's
interface/class relationships. The classic symptom of that damage is
IncompatibleClassChangeError ("Found interface X, but class was expected") at runtime.
Important limitation: this check compares tables only. It cannot see code-item damage —
see references/patch-audit.md for the checks that can.
"""
import argparse
import struct
import sys
ACC_INTERFACE = 0x200
def uleb128(data, off):
result = 0
shift = 0
while True:
b = data[off]
off += 1
result |= (b & 0x7F) << shift
if not (b & 0x80):
break
shift += 7
return result, off
def read_strings(data):
strings_size = struct.unpack('<I', data[0x38:0x3C])[0]
strings_off = struct.unpack('<I', data[0x3C:0x40])[0]
out = []
for i in range(strings_size):
sdata_off = struct.unpack('<I', data[strings_off + i * 4: strings_off + i * 4 + 4])[0]
n, p = uleb128(data, sdata_off)
raw = data[p:p + n]
out.append(raw.decode('utf-8', 'replace'))
return out
def class_map(path):
data = open(path, 'rb').read()
strings = read_strings(data)
n_types = struct.unpack('<I', data[0x40:0x44])[0]
type_ids_off = struct.unpack('<I', data[0x44:0x48])[0]
types = []
for i in range(n_types):
idx = struct.unpack('<I', data[type_ids_off + i * 4: type_ids_off + i * 4 + 4])[0]
types.append(strings[idx] if idx < len(strings) else '')
class_defs_size = struct.unpack('<I', data[0x60:0x64])[0]
class_defs_off = struct.unpack('<I', data[0x64:0x68])[0]
out = {}
for i in range(class_defs_size):
base = class_defs_off + i * 32
class_idx, access_flags = struct.unpack('<II', data[base:base + 8])
name = types[class_idx] if class_idx < len(types) else '?'
out[name] = access_flags
return out
def main():
ap = argparse.ArgumentParser(
description='Compare two dex files at class_defs level (names + access_flags). '
'Proves a reassembly did not damage interface/class relationships.')
ap.add_argument('a', help='first dex (e.g. the original)')
ap.add_argument('b', help='second dex (e.g. the rebuilt one)')
ap.add_argument('filter', nargs='?', default=None,
help='only report names containing this substring')
ap.add_argument('--max-list', type=int, default=10,
help='how many A-only / B-only names to print (default 10)')
ap.add_argument('--max-flags', type=int, default=25,
help='how many access_flags differences to print (default 25)')
ap.add_argument('--quiet', action='store_true',
help='print only the summary counts')
args = ap.parse_args()
A = class_map(args.a)
B = class_map(args.b)
only_a = sorted(set(A) - set(B))
only_b = sorted(set(B) - set(A))
diff_iface = []
diff_flags = []
for name in sorted(set(A) & set(B)):
fa, fb = A[name], B[name]
if bool(fa & ACC_INTERFACE) != bool(fb & ACC_INTERFACE):
diff_iface.append((name, fa, fb))
elif fa != fb:
diff_flags.append((name, fa, fb))
print('A classes=%d B classes=%d' % (len(A), len(B)))
print('only_in_A=%d only_in_B=%d' % (len(only_a), len(only_b)))
if not args.quiet:
for n in only_a[:args.max_list]:
print(' A-only:', n)
for n in only_b[:args.max_list]:
print(' B-only:', n)
print('\n*** ACC_INTERFACE mismatch: %d ***' % len(diff_iface))
if not args.quiet:
for name, fa, fb in diff_iface:
if args.filter and args.filter not in name:
continue
print(' %-70s A=%s B=%s' % (name, hex(fa), hex(fb)))
print('\naccess_flags diff (same interface-ness): %d' % len(diff_flags))
if not args.quiet:
shown = 0
for name, fa, fb in diff_flags:
if args.filter and args.filter not in name:
continue
if shown >= args.max_flags:
print(' ... (%d more suppressed)' % (len(diff_flags) - shown))
break
print(' %-70s A=%s B=%s' % (name, hex(fa), hex(fb)))
shown += 1
# Exit non-zero when a structural difference exists, so a caller can gate on it.
verdict = (len(only_a) == 0 and len(only_b) == 0 and len(diff_iface) == 0)
print('\nverdict: %s' % ('class tables identical'
if verdict else 'STRUCTURAL DIFFERENCE — inspect above'))
return 0 if verdict else 1
if __name__ == '__main__':
sys.exit(main())

412
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_dump_validate.py Normal file
View File

@@ -0,0 +1,412 @@
#!/usr/bin/env python3
"""Validate a directory of dumped dex files: dedupe, structural checks, stub-body ratio.
A memory dump (frida-dexdump & co.) hands you a pile of images: duplicate copies of
one dex, SDK plugin dexes that were never in the APK, structurally broken fragments,
extraction-shell skeletons whose method bodies are stubbed, and -- if you are lucky --
the original. `references/recon.md` states the rule (dedupe by content hash, validate
structure before trusting); this script is the tool that rule was missing.
Per file it reports: size, sha256, dex version, checksum/signature verdicts
(`dexutil.verify_dex_header`), class count, method-body census (no-code / real /
trivial-stub / nop-erased / minimal-form / empty), the stub and emptied ratios that
flag an extraction-shell skeleton, and --find pattern hits across the string table.
Files are grouped by sha256 and the survivors are ranked: most likely original first.
**Read `emptied%`, not `stub%`, for the skeleton call.** The two differ, and the
difference is load-bearing: `stub%` counts only bodies left as a lone `return*`, while
`emptied%` also counts bodies wiped to nothing but nops. A shell that clears the slot
instead of writing a return scores `stub% = 0.0` on a fully emptied image — measured,
and it used to be ranked the *most likely original* because of it. `stub%` is also
bimodal in practice: it lands at the app's own baseline or at ~100 %, with no band in
between, and partial extraction (75 % of bodies removed) moves it by 0.2 points. Full
matrix and commands: `docs/tool-verification/EXTENSION-extraction-shell-bench.md`.
Exit code 0 unless nothing in the input parses as a dex at all (exit 1).
Examples:
python dex_dump_validate.py dumps/ --find 'Lcom/example/app/'
python dex_dump_validate.py dumps/ --find 'Lcom/stub/' --json > report.json
python dex_dump_validate.py one_dumped.dex
"""
import argparse
import hashlib
import json
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from dexutil import OFFSETS, read_uleb, u16, u32, verify_dex_header # noqa: E402
MAGIC = b"dex\n"
KNOWN_VERSIONS = {b"035", b"036", b"037", b"038", b"039"}
RETURN_OPS = {0x0E, 0x0F, 0x10, 0x11} # return-void / return / return-wide / return-object
def classify_body(data, code_off):
"""Classify one code_item's instruction stream.
Returns one of 'empty' | 'stub' | 'erased' | 'malformed' | 'real'.
'stub' -- skip plain nop units (0x0000) and exactly one return* remains.
That is what an extraction shell leaves behind when it defers
decryption to first invocation, and what a repair fixture writes
when mimicking one.
'erased' -- insns_size > 0 but every unit is 0x0000. Measured on a fixture:
a body emptied by nop fill lands here, and reporting it as 'real'
(the pre-2026-09 behaviour) inverted the ranking, because a fully
nop-filled skeleton scored stub% = 0.0 and was named the most
likely original. A real compiler never emits a whole body of
nops, so this is skeleton evidence, not code.
'minimal' -- a body truncated to `const/4 vR, #0; return vR` and nop-padded,
with a non-zero high byte on the return. This is a *legal* minimal
non-void body, not a malformed one -- and it is a skeleton shape
the stub detector is blind to by design. Counted separately so the
blind spot is visible in the report instead of hiding inside
'real'. Measured: a fixture with all 5,061 bodies cut to this form
reports stub% = 1.9%, the same as its untouched control.
'real' -- anything else, including a body of a single non-return unit.
A two-unit `const/4 v0, #0; return v0` is deliberately **not** a stub: it is what
a *legal* minimal body looks like, and treating it as skeleton evidence would
make every tiny accessor a false positive. That asymmetry is the detector's
designed boundary, and it is measured in
`docs/tool-verification/EXTENSION-extraction-shell-bench.md`.
"""
insns_size = u32(data, code_off + 12)
if insns_size == 0:
return "empty"
base = code_off + 16
meaningful = 0
last_unit = 0
for k in range(insns_size):
unit = u16(data, base + k * 2)
if unit == 0x0000: # plain nop / cleared slot
continue
meaningful += 1
last_unit = unit
if meaningful > 2:
return "real" # three units can never be a stub or a minimal
if meaningful == 0:
return "erased" # every unit was 0x0000
if meaningful == 1:
op = last_unit & 0xFF # first byte holds the opcode for 10x/11x
if op == 0x0E:
# 10x return-void has no operand: `0xNN0E` with NN != 0 is `return vNN`,
# i.e. a non-void return wearing a return-void opcode -- minimal form.
return "stub" if last_unit == 0x000E else "minimal"
if op in (0x0F, 0x10, 0x11): # 11x returns: high byte is the register
return "stub"
return "real"
# meaningful == 2: legal minimal body only when it is `const/4 vR,#0; return* vR`
first_unit = 0
for k in range(insns_size):
if u16(data, base + k * 2):
first_unit = u16(data, base + k * 2)
break
if (first_unit & 0xFF) == 0x12 and (last_unit & 0xFF) in (0x0E, 0x0F, 0x10, 0x11):
return "minimal"
return "real"
def walk_methods(data, header):
"""Yield code_off for every method defined in class_defs (0 = abstract/native)."""
for i in range(header["class_defs_size"]):
cd = header["class_defs_off"] + i * 32
p = u32(data, cd + 24) # class_data_off
if p == 0:
continue
static_f, p = read_uleb(data, p)
inst_f, p = read_uleb(data, p)
direct_m, p = read_uleb(data, p)
virtual_m, p = read_uleb(data, p)
for _ in range(static_f + inst_f):
_, p = read_uleb(data, p) # field_idx_diff
_, p = read_uleb(data, p) # access_flags
for _ in range(direct_m + virtual_m):
_, p = read_uleb(data, p) # method_idx_diff
_, p = read_uleb(data, p) # access_flags
code_off, p = read_uleb(data, p)
yield code_off
def string_find_hits(data, header, patterns):
"""Count string-table entries containing each pattern."""
hits = {}
for pat in patterns:
pat_bytes = pat.encode("utf-8", "surrogateescape")
n = 0
for idx in range(header["string_ids_size"]):
p = u32(data, header["string_ids_off"] + idx * 4)
try:
_, p = read_uleb(data, p)
end = data.index(b"\x00", p)
except Exception:
continue
if pat_bytes in data[p:end]:
n += 1
hits[pat] = n
return hits
def profile_dex(path, patterns, trim=False):
"""Build the report dict for one file. 'error' key marks a rejected image.
With trim=True, an image whose header file_size is *shorter* than the file is
accepted after cutting the tail: /proc/<pid>/mem dumps are page-aligned, so the
recorded VMA range is always longer than the dex it holds (measured on a real
root-side dump: 17/17 images overran by 68-3724 bytes). A file *shorter* than
its own file_size is still rejected -- that is a truncated read, not padding.
"""
name = os.path.basename(path)
with open(path, "rb") as fh:
data = fh.read()
prof = {
"path": os.path.abspath(path),
"name": name,
"size": len(data),
"sha256": hashlib.sha256(data).hexdigest(),
}
if len(data) < 112:
prof["error"] = "too small (%d B) to carry a dex header" % len(data)
return prof
if data[:4] != MAGIC:
prof["error"] = "magic=%r" % data[:4]
return prof
version = data[4:7]
prof["version"] = version.decode("ascii", "replace")
if version not in KNOWN_VERSIONS:
prof["error"] = "unknown version %r" % version
return prof
header = {k: u32(data, v) for k, v in OFFSETS.items()}
if header["file_size"] != len(data):
if trim and 0 < header["file_size"] < len(data):
prof["trimmed_from"] = len(data)
prof["trimmed_to"] = header["file_size"]
data = data[: header["file_size"]]
prof["size"] = len(data)
else:
prof["error"] = "file_size=%d actual=%d" % (header["file_size"], len(data))
return prof
if header["header_size"] != 112:
prof["error"] = "header_size=%d (expected 112)" % header["header_size"]
return prof
for key in ("string_ids_off", "type_ids_off", "proto_ids_off",
"field_ids_off", "method_ids_off", "class_defs_off"):
off = header[key]
if off and not (0 < off < len(data)):
prof["error"] = "%s=0x%x out of range" % (key, off)
return prof
cksum_ok, sig_ok = verify_dex_header(bytearray(data))
prof["checksum_ok"] = cksum_ok
prof["signature_ok"] = sig_ok
prof["classes"] = header["class_defs_size"]
no_code = with_code = stubs = empties = erased = minimal = 0
try:
for code_off in walk_methods(data, header):
if code_off == 0: # abstract / native declaration
no_code += 1
continue
if code_off + 16 > len(data):
continue # broken pointer: skip, don't die
with_code += 1
kind = classify_body(data, code_off)
if kind == "stub":
stubs += 1
elif kind == "empty":
empties += 1
elif kind == "erased":
erased += 1
elif kind == "minimal":
minimal += 1
except Exception as exc: # desynced class_data: keep counts
prof["walk_error"] = str(exc)
prof["methods_no_code"] = no_code
prof["methods_with_code"] = with_code
prof["bodies_real"] = with_code - stubs - empties - erased - minimal
prof["bodies_stub"] = stubs
prof["bodies_empty"] = empties
prof["bodies_erased"] = erased
prof["bodies_minimal"] = minimal
# trivial_ratio keeps its original definition (return*-stub share of bodies) so
# existing records stay comparable. The skeleton signal is the union of every
# measured emptied shape; a body wiped to nothing at all must never score lower
# than a body wiped to `return-void`, or the ranking inverts (see
# docs/tool-verification/EXTENSION-extraction-shell-bench.md).
prof["trivial_ratio"] = (stubs / with_code) if with_code else 0.0
prof["emptied_ratio"] = ((stubs + erased) / with_code) if with_code else 0.0
if patterns:
try:
prof["find_hits"] = string_find_hits(data, header, patterns)
except Exception as exc:
prof["find_hits_error"] = str(exc)
return prof
def collect_files(paths):
out = []
for p in paths:
if os.path.isdir(p):
out += [os.path.join(p, n) for n in sorted(os.listdir(p))
if n.endswith(".dex")]
elif os.path.isfile(p):
out.append(p)
else:
print("warning: %s is not a file or directory, skipped" % p,
file=sys.stderr)
return out
def rank_key(prof):
"""Sort key: most-likely-original first.
Both body signals are needed, in the right order, and each was measured to fail
alone:
* the `return*`-stub share alone let a **nop-filled** skeleton score 0.0 and be
named the most likely original, because a wiped body counts as neither a stub
nor real code;
* `emptied_ratio` alone still failed, because it is **bimodal** -- it sits at the
host app's own baseline or at ~100 %, with nothing in between. A dex with 25 %
of its bodies emptied measures *below* its own untouched control, 1.7 % against
1.9 %, so the sort pointed at a modified image and called it the original;
* `minimal + erased` alone failed too, and in the opposite direction: a skeleton
that writes a bare `return-void` into every body has minimal = erased = 0 and
would sort first, despite `emptied%` = 100.
So the key combines both, and the combination has to let *either* signal disqualify
an image rather than letting them tie:
* level 1 -- `emptied_ratio >= 0.5`. Nothing that is mostly emptied is a candidate,
whatever else it scores. This is the level that evicts the two skeletons the
plain counts cannot see: an image with a bare `return-void` in every body has
minimal = erased = 0, so a pure-count key ranked it first.
* level 2 -- `minimal + erased`, the skeleton-evidence count, low-first. This is
what orders the **bimodal band**, where `emptied_ratio` cannot: the untouched
control carries 82, and the count is monotone as bodies are removed (1299 /
2524 / 3736 / 4946 at 25 / 50 / 75 / 100 %).
* level 3 -- `emptied_ratio` low-first, to separate what is left.
"""
return (1 if prof.get("emptied_ratio", prof["trivial_ratio"]) >= 0.5 else 0,
prof.get("bodies_minimal", 0) + prof.get("bodies_erased", 0),
prof.get("emptied_ratio", prof["trivial_ratio"]),
0 if prof["checksum_ok"] else 1,
0 if prof["signature_ok"] else 1,
prof["name"])
def main(argv=None):
ap = argparse.ArgumentParser(
description="Dedupe and structurally validate a directory of dumped dex "
"files; rank which image is the most likely original.")
ap.add_argument("path", nargs="+",
help="a directory of *.dex dumps, or individual .dex files")
ap.add_argument("--find", action="append", default=[], metavar="PATTERN",
help="count string-table hits for PATTERN (repeatable), "
"e.g. --find 'Lcom/example/app/'")
ap.add_argument("--json", action="store_true",
help="emit the full report as JSON (deterministic; redirect "
"to a file to keep it)")
ap.add_argument("--trim", action="store_true",
help="accept page-aligned dumps: cut a tail longer than the header's "
"file_size (/proc/<pid>/mem exports always overrun; a file shorter "
"than its own file_size is still rejected)")
args = ap.parse_args(argv)
files = collect_files(args.path)
if not files:
print("no .dex files found in the given path(s)", file=sys.stderr)
return 1
profiles = [profile_dex(f, args.find, args.trim) for f in files]
valid = [p for p in profiles if "error" not in p]
ranking = sorted(valid, key=rank_key)
if args.json:
print(json.dumps({"profiles": profiles,
"ranking": [p["name"] for p in ranking]}, indent=2))
return 0 if valid else 1
# ---- human-readable report -------------------------------------------------
print("== dex dump validation: %d file(s), %d parse, %d rejected =="
% (len(profiles), len(valid), len(profiles) - len(valid)))
fmt = "%-28s %9s %-12s %3s %-7s %-7s %6s %6s %6s %7s %7s %7s"
print(fmt % ("name", "size", "sha256[:12]", "ver", "cksum", "sig",
"class", "noco", "code", "stub%", "erased%", "emptied%"))
for p in profiles:
if "error" in p:
print("%-28s %9s %-12s %3s REJECTED: %s"
% (p["name"], p["size"], p["sha256"][:12],
p.get("version", "?"), p["error"]))
continue
print(fmt % (p["name"], p["size"], p["sha256"][:12], p["version"],
"ok" if p["checksum_ok"] else "BAD",
"ok" if p["signature_ok"] else "BAD",
p["classes"], p["methods_no_code"], p["methods_with_code"],
"%.1f%%" % (100.0 * p["trivial_ratio"]),
"%.1f%%" % (100.0 * (p["bodies_erased"] / p["methods_with_code"]
if p["methods_with_code"] else 0.0)),
"%.1f%%" % (100.0 * p.get("emptied_ratio", 0.0))))
if p.get("trimmed_from"):
print("%-28s trimmed %d -> %d B (page-aligned dump)"
% ("", p["trimmed_from"], p["trimmed_to"]))
if p.get("walk_error"):
print("%-28s walk stopped early: %s" % ("", p["walk_error"]))
if p.get("bodies_minimal"):
print("%-28s minimal-form bodies: %d (const/4+return truncation -- a "
"skeleton shape the stub%% column cannot see)"
% ("", p["bodies_minimal"]))
for pat, n in sorted(p.get("find_hits", {}).items()):
print("%-28s find %-24s %d" % ("", pat, n))
groups = {}
for p in valid:
groups.setdefault(p["sha256"], []).append(p["name"])
dup_groups = {h: n for h, n in groups.items() if len(n) > 1}
print("\ndedupe: %d unique image(s) among %d valid file(s)"
% (len(groups), len(valid)))
for h, names in sorted(dup_groups.items()):
print(" %s %s" % (h[:16], ", ".join(names)))
if ranking:
print("\nranking (most likely original first):")
for i, p in enumerate(ranking, 1):
note = ""
if p.get("emptied_ratio", 0.0) >= 0.5:
note = (" <-- SKELETON: emptied%%=%.0f, extraction-shell shape"
% (100.0 * p["emptied_ratio"]))
elif p.get("bodies_erased", 0):
note = (" <-- %d body(ies) wiped to nops: skeleton evidence"
% p["bodies_erased"])
print(" %d. %-28s stub%%=%.1f erased%%=%.1f min/erased=%d cksum=%s "
"sig=%s classes=%d%s"
% (i, p["name"], 100.0 * p["trivial_ratio"],
100.0 * (p["bodies_erased"] / p["methods_with_code"]
if p["methods_with_code"] else 0.0),
p.get("bodies_minimal", 0) + p.get("bodies_erased", 0),
"ok" if p["checksum_ok"] else "BAD",
"ok" if p["signature_ok"] else "BAD",
p["classes"], note))
top = ranking[0]
copies = len(groups.get(top["sha256"], []))
print("\nverdict: %s is the best-supported candidate for the original "
"(%d copy(ies) in this set)" % (top["name"], copies))
print("ranked by: mostly-emptied images evicted first (emptied%% >= 50), then "
"the minimal+nop-erased body count low-first, then emptied%%.\n"
" Limits, stated rather than hidden: this cannot see a `throw`-stub "
"skeleton (it scores at the control's own baseline), and a\n"
" partially extracted image still outranks a heavily stubbed one. "
"Confirm the winner before using it as a patch baseline --\n"
" docs/tool-verification/EXTENSION-extraction-shell-bench.md")
else:
print("\nverdict: no image parsed as a dex -- nothing to rank")
return 0 if valid else 1
if __name__ == "__main__":
sys.exit(main())

276
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_find_insn.py Normal file
View File

@@ -0,0 +1,276 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Locate an instruction by decoded semantics and report its exact byte offset.
WHY THIS EXISTS
---------------
You know what the instruction *does* ("a branch on the config boolean whose target
starts the countdown") but you need the *byte offset* to patch it. Two shortcuts
fail:
* baksmali listings give smali line numbers, not code offsets. `.line` tracks
source lines, breaks 1:N, and cannot be converted reliably.
* a hard-coded offset dies the moment the sample is rebuilt, and a wrong offset
in an equal-length patch corrupts the neighbouring instruction silently.
So this tool decodes the method and filters on semantics. It always prints the
surrounding instructions so you can see which side of a branch you are about to
change -- the polarity mistake is the most common way a patch does the exact
opposite of the goal and still starts cleanly.
Filter language (all optional; combined with AND, comma = OR within a repeated
flag):
NAME substring match on the class, method or string operand
@Member field or method name operand equals/contains Member
=literal a const/4 literal equals this integer
#Class/member a branch whose TARGET reads this field of this class
>kind instruction kind: if-test if-testz invoke return
move-result const4 nop goto switch field sfield
$name the ENCLOSING method name must contain this
Examples
--------
# a boolean gate whose taken path reads Cfg.enabled
python dex_find_insn.py app.apk --class 'Lcom/example/SplashActivity;' \\
--method u --filter '#Lcom/example/Cfg;*enabled'
# every invoke of a method named like goHome, with 3 lines of context
python dex_find_insn.py app.apk --filter '@goHome' --kind invoke -C 3
# where a specific literal is compared
python dex_find_insn.py app.apk --class 'Lcom/example/PlayerActivity;' --filter '=1' --kind if-testz
Exit codes: 0 = at least one hit, 1 = no hits (a finding: say so, do not retry the
same filter), 2 = usage error.
"""
import argparse
import os
import re
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from dexutil import IF_TEST, IF_TESTZ, INVOKE_OPS, MOVE_RESULT_OPS, load_dex # noqa: E402
_KIND_MAP = {
"if-test": IF_TEST,
"if-testz": IF_TESTZ,
"invoke": INVOKE_OPS,
"return": (0x0F, 0x10, 0x11),
"move-result": MOVE_RESULT_OPS,
"const4": (0x12,),
"const16": (0x13,),
"goto": (0x28, 0x29, 0x2A),
"switch": (0x2B, 0x2C),
"nop": (0x00,),
}
_FIELD_READ = tuple(range(0x52, 0x59))
_FIELD_WRITE = tuple(range(0x59, 0x60))
_SFIELD = tuple(range(0x60, 0x6E))
_KIND_MAP["field"] = _FIELD_READ
_KIND_MAP["field-write"] = _FIELD_WRITE
_KIND_MAP["sfield"] = _SFIELD
class Filter(object):
"""One compiled filter expression."""
def __init__(self, text):
self.raw = text
self.kind = text[1:] if text.startswith(">") else None
self.member = text[1:] if text.startswith("@") else None
self.literal = int(text[1:]) if text.startswith("=") else None
self.target_field = None
self.target_class = None
self.enclosing = text[1:] if text.startswith("$") else None
self.text = None
if text.startswith("#"):
spec = text[1:]
if "/" in spec and ("*" in spec):
cls, mem = spec.split("*", 1)
self.target_class, self.target_field = cls, mem
elif "*" in spec:
self.target_field = spec.split("*", 1)[1]
else:
self.target_class = spec
elif not any(text.startswith(p) for p in ("@", "=", ">", "$")):
self.text = text
def describe(self):
return self.raw
def _operand_text(dex, insn):
"""All names this instruction mentions (class, method, field, string)."""
parts = []
op = insn["op"]
if op in INVOKE_OPS:
cls, nm, ds = dex.method(int.from_bytes(insn["raw"][2:4], "little"))
parts += [cls, nm, ds]
elif op in _FIELD_READ + _FIELD_WRITE + _SFIELD:
cls, nm, ty = dex.field(int.from_bytes(insn["raw"][2:4], "little"))
parts += [cls, nm, ty]
elif op in (0x19, 0x1A):
parts.append(dex.string_safe(int.from_bytes(insn["raw"][2:4], "little")))
return parts
def _matches(dex, insn, filters, code_off):
"""All filters must hold (AND)."""
for f in filters:
if f.kind:
ops = _KIND_MAP.get(f.kind)
if ops is None:
raise ValueError("unknown kind: %s" % f.kind)
if insn["op"] not in ops:
return False
if f.literal is not None:
if insn["op"] != 0x12:
return False
val = insn["raw"][1] & 0xF
if val > 7:
val -= 16
if val != f.literal:
return False
if f.member:
if insn["op"] not in INVOKE_OPS:
return False
_cls, nm, _ds = dex.method(int.from_bytes(insn["raw"][2:4], "little"))
if f.member not in nm:
return False
if f.target_class or f.target_field:
tgt = dex.branch_target(insn)
if tgt is None:
return False
t_insn = dex.insn_at(code_off, tgt)
if t_insn is None or not (0x52 <= t_insn["op"] <= 0x58):
return False
cls, nm, _ty = dex.field(int.from_bytes(t_insn["raw"][2:4], "little"))
if f.target_class and f.target_class != cls:
return False
if f.target_field and f.target_field not in nm:
return False
if f.text:
hay = " ".join(_operand_text(dex, insn))
if f.text.lower() not in hay.lower():
return False
return True
def scan(dex, filters, kind_only=None, class_pattern=None, method_pattern=None,
limit=None):
"""Yield (fqcn, method_name, descriptor, code_off, insn, all_insns, idx)."""
hits = 0
cls_re = re.compile(class_pattern) if class_pattern else None
mth_re = re.compile(method_pattern) if method_pattern else None
for fqcn in dex.class_names():
if cls_re and not cls_re.search(fqcn):
continue
try:
methods = list(dex.methods_of(fqcn))
except Exception:
continue
for section, midx, _c, name, desc, code_off in methods:
if code_off == 0:
continue
if mth_re and not mth_re.search(name):
continue
for f in filters:
if f.enclosing and f.enclosing not in name:
break
insns = list(dex.decode(code_off))
# a decode that does not land on the method end means offsets here
# cannot be trusted; report the method rather than silent misses
info = dex.code_info(code_off)
expected = info["insns_off"] + info["insns_size"] * 2
clean = (insns[-1]["off"] + insns[-1]["units"] * 2 == expected) if insns else False
for i, insn in enumerate(insns):
if kind_only and insn["op"] not in _KIND_MAP.get(kind_only, ()):
continue
if filters and not _matches(dex, insn, filters, code_off):
continue
yield (fqcn, name, desc, code_off, insn, insns, i, clean)
hits += 1
if limit and hits >= limit:
return
def main(argv):
ap = argparse.ArgumentParser(
description="Find an instruction by decoded semantics; print its exact "
"byte offset and surrounding context.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__.split("Examples")[-1][:1500])
ap.add_argument("target", help=".dex or .apk/.zip (use --entry to pick a dex)")
ap.add_argument("--entry", help="archive member (default: first classes*.dex)")
ap.add_argument("--class", dest="cls", help="regex over the class descriptor")
ap.add_argument("--method", help="regex over the method name")
ap.add_argument("--filter", action="append", default=[],
help="filter expression, repeatable (AND across, OR within a flag "
"list). See the module docstring for the language.")
ap.add_argument("--kind", help="restrict to one instruction kind (see docstring)")
ap.add_argument("-C", "--context", type=int, default=2,
help="instructions of context to print (default 2)")
ap.add_argument("--limit", type=int, default=40, help="max hits (default 40)")
ap.add_argument("--show-unclean", action="store_true",
help="also print hits from methods whose decode did not align")
args = ap.parse_args(argv[1:])
dex, entry = load_dex(args.target, args.entry)
problems = dex.check()
if problems:
print("structural problems in %s (%s):" % (args.target, entry))
for p in problems:
print(" - %s" % p)
return 2
filters = [Filter(a) for a in args.filter]
print("== %s (%s) %d classes" % (args.target, entry,
dex.header["class_defs_size"]))
if filters:
print(" filters: %s" % ", ".join(f.describe() for f in filters))
found = 0
for fqcn, name, desc, code_off, insn, insns, idx, clean in scan(
dex, filters, args.kind, args.cls, args.method, args.limit):
if not clean and not args.show_unclean:
print("\n!! %s.%s%s decode did not align; offsets unreliable, "
"skipping (use --show-unclean to see anyway)"
% (fqcn, name, desc))
continue
found += 1
print("\n-- %s.%s%s" % (fqcn, name, desc))
print(" code_off=0x%x insns_off=0x%x (decode clean: %s)"
% (code_off, code_off + 16, clean))
lo = max(0, idx - args.context)
hi = min(len(insns), idx + args.context + 1)
for j in range(lo, hi):
marker = " <== HIT" if j == idx else ""
print(" %s%s" % (dex.describe(insns[j]), marker))
print(" >>> file offset of the instruction: 0x%x" % insn["off"])
if insn["op"] in IF_TEST or insn["op"] in IF_TESTZ:
tgt = dex.branch_target(insn)
t_insn = dex.insn_at(code_off, tgt)
print(" >>> branch TAKEN target 0x%x: %s"
% (tgt, dex.describe(t_insn) if t_insn else "?"))
nxt = insn["off"] + insn["units"] * 2
n_insn = dex.insn_at(code_off, nxt)
print(" >>> branch FALL-THROUGH 0x%x: %s"
% (nxt, dex.describe(n_insn) if n_insn else "?"))
print(" >>> decide which side you want BEFORE patching; a wrong "
"polarity still starts cleanly")
print("\n== %d hit(s)" % found)
if not found:
print(" no match. Do not retry the same filter: either the decode is "
"unclean (see --show-unclean) or the behaviour lives in another "
"layer (native / another dex / another class).")
return 1
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))

245
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_mem_scan.py Normal file
View File

@@ -0,0 +1,245 @@
#!/usr/bin/env python3
"""Scan memory captures for embedded dex images and optionally extract them.
A packer can keep a decrypted dex in a buffer that never appears in
`/proc/<pid>/maps` under a dex-looking name -- it sits inside an anonymous rw-p
mapping. Neither `maps` analysis nor a per-mapping dump finds that on its own;
searching the bytes for the dex magic does. This script does the search and, with
`--dump`, writes each hit out at the size its own header declares.
Typical use, after taking raw regions from a live process (root required):
adb shell 'su -c "cat /proc/<pid>/mem"' ... # or per-region dd exports
python skills/apk-reverse/scripts/dex_mem_scan.py region_dir/ --dump out/
Feeding the extracted images to `dex_dump_validate.py` is the next step: this
script only *finds* and *cuts*; that one judges whether what came out is a real
body or an extraction-shell skeleton.
**Measured hit rate on the reference device: zero for anonymous regions.** The
root-dump pass recorded in `docs/tool-verification/EXTENSION-rootdump.md` scanned
142 anonymous mappings and returned **no hits**, while the *named* ART dex mappings
in the same process yielded 17 dex images without any search at all. A negative
result from this script is therefore weak evidence: it is compatible with "the
payload is not resident", "the payload is resident but not as a contiguous
dex-magic image", and "the payload was paged out between capture and scan". Treat a
zero-hit scan as a reason to check the named mappings first
(`[anon:dalvik-classes*.dex extracted in memory from <src>]`), not as a conclusion
about the packer. A positive hit inside a genuinely unnamed mapping has not been
observed here yet.
Scope note: the header's `file_size` field decides the cut. When the declared size
does not fit inside the capture, the hit is reported and skipped rather than
guessed at -- a dex whose tail is missing is not recoverable by truncating the
wrong end. Page-aligned captures from a VMA routinely overrun the image they hold,
so a run against raw VMA exports usually needs `--keep-partial` or a trim step
before `dex_dump_validate.py` will accept the output.
"""
import argparse
import hashlib
import json
import os
import struct
import sys
MAGIC = b"dex\n0"
HEADER_SIZE = 112
OVERLAP = 0x80 # enough to complete a header straddling two chunks
def valid_version(four_bytes):
"""`dex\\n0XY` -> 'XY' when both digits are decimal, else None."""
ver = four_bytes[4:7]
if len(ver) != 3 or not (ver[0:1].isdigit() and ver[1:2].isdigit()
and ver[2:3].isdigit()):
return None
return ver.decode("ascii")
def find_offsets(path, chunk_size):
"""Yield (absolute offset, buffer, index in buffer, buffer base) for each magic.
Chunks overlap by OVERLAP bytes so a header straddling a chunk boundary is
still seen whole; the caller dedupes repeated offsets.
"""
size = os.path.getsize(path)
carry = b""
pos = 0
with open(path, "rb") as fh:
while pos < size:
fh.seek(pos)
buf = carry + fh.read(chunk_size)
base = pos - len(carry)
start = 0
while True:
i = buf.find(MAGIC, start)
if i < 0:
break
start = i + 1
yield base + i, buf, i, base
if len(buf) <= OVERLAP:
break
carry = buf[-OVERLAP:]
pos += len(buf) - len(carry)
def read_header(fh, offset):
"""Return (file_size, class_defs_size, link_size) or None if the header is short."""
fh.seek(offset)
head = fh.read(HEADER_SIZE)
if len(head) < HEADER_SIZE:
return None
if head[:4] != b"dex\n"[:4]:
return None
file_size = struct.unpack_from("<I", head, 32)[0]
class_defs_size = struct.unpack_from("<I", head, 96)[0]
link_size = struct.unpack_from("<I", head, 104)[0]
return file_size, class_defs_size, link_size
def scan_file(path, chunk_size, min_size, do_dump, dump_dir, keep_partial):
"""Scan one capture; return (records, extracted_count, skipped_count)."""
records = []
extracted = 0
skipped = 0
seen = set()
size = os.path.getsize(path)
with open(path, "rb") as fh:
for offset, buf, i, base in find_offsets(path, chunk_size):
ver = valid_version(buf[i:i + 8]) if i + 8 <= len(buf) else None
if ver is None:
continue
if offset in seen:
continue
seen.add(offset)
head = read_header(fh, offset)
if head is None:
continue
file_size, class_defs_size, link_size = head
if file_size < 112 or file_size > (1 << 32):
continue
fits = file_size <= (size - offset)
rec = {
"capture": os.path.basename(path),
"offset": offset,
"version": ver,
"file_size": file_size,
"class_defs_size": class_defs_size,
"link_size": link_size,
"fits": fits,
}
if fits and file_size >= min_size:
takes = file_size
elif keep_partial and not fits:
takes = size - offset
rec["partial"] = takes
else:
rec["skipped"] = ("declared %d B, %d B available"
% (file_size, size - offset))
skipped += 1
records.append(rec)
continue
fh.seek(offset)
blob = fh.read(takes)
rec["sha256"] = hashlib.sha256(blob).hexdigest()
if do_dump:
stem = os.path.splitext(os.path.basename(path))[0]
name = "%s+0x%08x.dex" % (stem, offset)
out = os.path.join(dump_dir, name)
with open(out, "wb") as ofh:
ofh.write(blob)
rec["written"] = out
extracted += 1
records.append(rec)
return records, extracted, skipped
def collect_targets(paths):
targets = []
for p in paths:
if os.path.isdir(p):
for name in sorted(os.listdir(p)):
full = os.path.join(p, name)
if os.path.isfile(full) and not name.endswith((".txt", ".json", ".md")):
targets.append(full)
else:
targets.append(p)
return targets
def main(argv=None):
ap = argparse.ArgumentParser(
description="Scan memory captures (or any blob) for embedded dex images "
"'dex\\n03x', report each hit, and optionally extract them at "
"the size their own headers declare. Pair with "
"dex_dump_validate.py, which judges what was extracted.")
ap.add_argument("path", nargs="+",
help="a capture file, or a directory of region exports")
ap.add_argument("--dump", metavar="DIR",
help="write each hit into DIR as <capture>+0x<offset>.dex")
ap.add_argument("--json", action="store_true",
help="emit the record list as JSON")
ap.add_argument("--chunk-size", type=int, default=64 << 20, metavar="N",
help="read the capture in N-byte chunks (default 64 MiB); use a "
"smaller value for very large captures")
ap.add_argument("--min-size", type=int, default=4096, metavar="N",
help="ignore hits whose declared file_size is below N bytes "
"(default 4096) -- stray magics in unrelated data")
ap.add_argument("--keep-partial", action="store_true",
help="also write hits whose declared size runs past the end of "
"the capture (truncated region); the result is NOT a dex")
args = ap.parse_args(argv)
targets = collect_targets(args.path)
if not targets:
print("no capture files found in the given path(s)", file=sys.stderr)
return 1
dump_dir = None
if args.dump:
dump_dir = os.path.abspath(args.dump)
os.makedirs(dump_dir, exist_ok=True)
all_records = []
total_extracted = 0
total_skipped = 0
for t in targets:
records, extracted, skipped = scan_file(
t, args.chunk_size, args.min_size,
dump_dir is not None, dump_dir or ".", args.keep_partial)
all_records.extend(records)
total_extracted += extracted
total_skipped += skipped
if args.json:
print(json.dumps({"records": all_records,
"extracted": total_extracted,
"skipped": total_skipped}, indent=2))
return 0 if all_records else 1
print("== dex memory scan: %d capture(s), %d hit(s) =="
% (len(targets), len(all_records)))
fmt = "%-34s %12s %3s %12s %10s %-6s"
print(fmt % ("capture", "offset", "ver", "file_size", "classes", "state"))
for r in all_records:
state = "ok" if r.get("fits") and not r.get("partial") else (
"partial" if r.get("partial") else "skipped")
print(fmt % (r["capture"][:34], "0x%x" % r["offset"], r["version"],
r["file_size"], r["class_defs_size"], state))
if r.get("written"):
print("%-34s wrote %s" % ("", r["written"]))
if r.get("skipped"):
print("%-34s skipped: %s" % ("", r["skipped"]))
unique = {r["sha256"] for r in all_records if r.get("sha256")}
print("\n%d hit(s) with a complete declared size, %d unique by sha256, "
"%d skipped" % (total_extracted, len(unique), total_skipped))
if total_extracted:
print("next: python dex_dump_validate.py %s" % (dump_dir or "<dump dir>"))
return 0 if all_records else 1
if __name__ == "__main__":
sys.exit(main())

435
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_patch_bytes.py Normal file
View File

@@ -0,0 +1,435 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Apply equal-length byte patches to a dex, with structural and verifier checks.
WHY THIS EXISTS
---------------
The cheap way to change behaviour is to rewrite a few bytes in place rather than
rebuild a method body. It is cheap for a reason: nothing moves, so no try/catch
block, debug-info pointer or branch displacement can be invalidated by
construction. But it has four failure modes that all produce a build which looks
fine and misbehaves, and this tool exists to make each one impossible to skip:
1. **Same-length is not enough** -- the replacement must decode to a valid
instruction sequence of exactly the same length. A 4-byte branch replaced by
a 4-byte branch is fine; a 2-byte branch replaced by a 4-byte instruction
corrupts the next instruction.
2. **Polarity** -- the overwhelmingly common error is patching the wrong side of
a condition. So a spec can *require* the instruction that follows the edit,
and this tool aborts if it is not the one you named. Name the fall-through
you expect and the mistake cannot survive.
3. **Verifier legality** -- a branch whose target is a `move-result*` bypasses
the producing invoke and the class fails to load (`VerifyError`). Checked
here before writing.
4. **dex header staleness** -- the checksum and signature fields must be
recomputed, signature FIRST (the checksum covers it). Skipping this makes
Android fall back to interpreting the dex, which surfaces as an unrelated
ClassNotFoundException at startup.
Spec format (JSON; a list, or {"patches": [...]}):
[{
"name": "human label",
"reason": "why this edit is correct",
"class": "Lpkg/Name;",
"method": "someMethod",
"desc": "(Lpkg/Arg;)V", // optional: disambiguates overloads
"match": {"kind": "if-testz", "target_reads_field_of": "Lpkg/Cfg;",
"target_field": "enabled"},
"expect_next": {"kind": "invoke", "target_method": "goHome"},
"replace": {"kind": "nops"} // or {"bytes": "28060000"}
}]
`match` selects the instruction; `expect_next` (optional but strongly advised)
asserts what immediately follows, which is what pins the polarity; `replace`
gives the bytes. Run with --dry-run first: it prints the match, the neighbours,
the verifier verdict and the predicted byte diff without writing anything.
Exit codes: 0 ok, 1 failure (structure, match, verifier, or self-verify), 2 usage.
"""
import argparse
import json
import os
import struct
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from dexutil import ( # noqa: E402
Dex, IF_TEST, IF_TESTZ, INVOKE_OPS, MOVE_RESULT_OPS, RETURN_OPS,
fix_dex_header, load_dex, verify_dex_header,
)
def _kind_of(insn):
"""Coarse semantic kind used by spec matching."""
op = insn["op"]
if op in IF_TEST:
return "if-test"
if op in IF_TESTZ:
return "if-testz"
if op in INVOKE_OPS:
return "invoke"
if op in RETURN_OPS:
return "return"
if op in MOVE_RESULT_OPS:
return "move-result"
if op == 0x12:
return "const4"
return insn["name"]
def _describe_neighbours(dex, code_off, target_off, span=4):
"""Render up to `span` instructions before and after an offset."""
insns = list(dex.decode(code_off))
idx = None
for i, insn in enumerate(insns):
if insn["off"] == target_off:
idx = i
break
if idx is None:
return [], None, []
before = insns[max(0, idx - span):idx]
after = insns[idx + 1:idx + 1 + span]
return before, insns[idx], after
def _matches(dex, insn, crit, code_off=None):
"""Does one instruction satisfy a match/expect criterion?"""
if crit is None:
return True
want = crit.get("kind")
if want and _kind_of(insn) != want:
return False
if want in ("if-test", "if-testz"):
treg = crit.get("target_reg")
if treg is not None:
regs, _ = dex.branch_regs(insn)
if treg not in regs:
return False
# The usual way to pin "this is the gate for config X": require that the
# branch TARGET instruction reads a field of a named class. This is what
# distinguishes the gate you want from the other branches in the method.
fcls = crit.get("target_reads_field_of")
fname = crit.get("target_field")
if fcls or fname:
if code_off is None:
return False
tgt = dex.branch_target(insn)
t_insn = dex.insn_at(code_off, tgt)
if t_insn is None:
return False
if not (0x52 <= t_insn["op"] <= 0x58):
return False
fidx = struct.unpack_from("<H", t_insn["raw"], 2)[0]
cls, nm, _ty = dex.field(fidx)
if fcls and cls != fcls:
return False
if fname and nm != fname:
return False
if want == "const4":
lit = crit.get("literal")
if lit is not None:
raw = insn["raw"][1]
val = raw & 0xF
if val > 7:
val -= 16
if val != lit:
return False
if want == "invoke":
midx = struct.unpack_from("<H", insn["raw"], 2)[0]
cls, nm, _ds = dex.method(midx)
if crit.get("target_class") and crit["target_class"] not in cls:
return False
if crit.get("target_method") and crit["target_method"] != nm:
return False
if want in ("return", "move-result", "nop"):
pass
raw_eq = crit.get("bytes")
if raw_eq and insn["raw"].hex() != raw_eq.replace(" ", "").lower():
return False
return True
def _find_site(dex, code_off, spec):
"""Return the single instruction matching spec['match'].
Exactly one match is required. Ambiguity is treated as failure rather than
"take the first": picking the wrong site in a method with several similar
branches is a silent, expensive error.
"""
insns = list(dex.decode(code_off))
hits = [i for i in insns if _matches(dex, i, spec.get("match"), code_off)]
if len(hits) == 1:
return hits[0], insns
if not hits:
return None, insns
# allow an ordinal to disambiguate deliberately
ordv = spec.get("match", {}).get("ordinal")
if ordv is not None and 0 <= ordv < len(hits):
return hits[ordv], insns
return ("ambiguous", hits), insns
def _verifier_report(dex, code_off, target_off_for_new_edge=None):
"""Tier-3 check: does any branch target a move-result instruction?
`move-result*` must be immediately preceded by its producing invoke. A branch
into one bypasses the producer and the class fails to load with
`VerifyError ... copyResN v <- result0 type=Undefined`. A linear "is the
previous instruction the producer" scan misses this; only the CFG view sees
it.
"""
findings = []
for insn in dex.decode(code_off):
if insn["op"] not in MOVE_RESULT_OPS:
continue
tgt = insn["off"]
for other in dex.decode(code_off):
t = dex.branch_target(other)
if t == tgt:
findings.append((other["off"], tgt))
return findings
def _apply_replace(dex_data, insn, repl):
"""Return the new bytes for this instruction, enforcing equal length."""
kind = repl.get("kind")
if kind == "nops":
n = len(insn["raw"])
return b"\x00" * n
if kind == "bytes":
blob = bytes.fromhex(repl["bytes"].replace(" ", ""))
return blob
raise ValueError("replace must be {'kind':'nops'} or {'kind':'bytes','bytes':...}")
def process_one(dex, data, spec, index, dry_run):
"""Apply one spec entry to `data`; return a report dict, or raise."""
cls = spec["class"]
name = spec["method"]
desc = spec.get("desc")
label = spec.get("name") or ("patch%d" % index)
report = {"name": label, "ok": False}
if desc:
found = dex.find_method(cls, name, desc)
if not found:
raise RuntimeError("[%s] method not found: %s->%s%s"
% (label, cls, name, desc))
section, midx, code_off = found
else:
overloads = dex.find_methods_named(cls, name)
if len(overloads) != 1:
raise RuntimeError(
"[%s] %s->%s has %d overloads; give \"desc\" to disambiguate: %s"
% (label, cls, name, len(overloads),
[d for _s, _i, d, _c in overloads]))
section, midx, dsc, code_off = overloads[0]
desc = dsc
report.update({"class": cls, "method": name, "desc": desc,
"section": section, "method_idx": midx, "code_off": code_off})
site, _insns = _find_site(dex, code_off, spec)
if site is None:
raise RuntimeError("[%s] no instruction matched spec['match']" % label)
if isinstance(site, tuple) and site and site[0] == "ambiguous":
cands = ", ".join("0x%x" % h["off"] for h in site[1])
raise RuntimeError("[%s] match is ambiguous (%d hits: %s); tighten it or "
"set match.ordinal" % (label, len(site[1]), cands))
before, cur, after = _describe_neighbours(dex, code_off, site["off"])
report.update({
"site_off": cur["off"],
"old_bytes": cur["raw"].hex(),
"old_text": dex.describe(cur),
"before": [dex.describe(i) for i in before],
"after": [dex.describe(i) for i in after],
})
# polarity pin: what must immediately follow?
exp = spec.get("expect_next")
if exp is not None:
nxt = after[0] if after else None
if nxt is None or not _matches(dex, nxt, exp, code_off):
raise RuntimeError(
"[%s] expect_next not satisfied at 0x%x. Expected %r, found %r.\n"
" This is the polarity check: the instruction after the branch "
"is not the one your spec assumes, so the branch points at the "
"wrong side."
% (label, cur["off"], exp, dex.describe(nxt) if nxt else None))
report["expect_next_ok"] = True
new = _apply_replace(data, cur, spec.get("replace", {"kind": "nops"}))
if len(new) != len(cur["raw"]):
raise RuntimeError(
"[%s] replacement is %d bytes but the instruction is %d. Equal length "
"is required: a longer replacement overwrites the next instruction."
% (label, len(new), len(cur["raw"])))
report.update({"new_bytes": new.hex(), "size_delta": 0})
# verifier: simulate the new instruction to see if it adds a branch edge
fake = Dex(bytes(data), dex.name)
fake_probe = dict(cur)
fake_probe["raw"] = new
fake_probe["op"] = new[0]
report["new_text"] = fake.describe(fake_probe)
if dry_run:
report["ok"] = True
return report, data
data[cur["off"]:cur["off"] + len(new)] = new
# Re-read the patched dex and confirm the edit landed.
#
# Compare the BYTE RANGE, not a decoded instruction: replacing a 4-byte branch
# with a nop pair yields two 1-unit instructions, so "the instruction at this
# offset has length 4" is false even though the patch is perfectly correct.
# The byte range is what the patch actually owns.
span = slice(cur["off"], cur["off"] + len(new))
if bytes(data[span]) != new:
raise RuntimeError(
"[%s] bytes at 0x%x are %s, expected %s"
% (label, cur["off"], bytes(data[span]).hex(), new.hex()))
patched = Dex(bytes(data), dex.name)
insns_after = list(patched.decode(code_off))
info = patched.code_info(code_off)
expected_end = info["insns_off"] + info["insns_size"] * 2
ended = (insns_after[-1]["off"] + insns_after[-1]["units"] * 2
if insns_after else info["insns_off"])
report["post_instruction_count"] = len(insns_after)
report["post_decode_clean"] = (ended == expected_end)
landed = [i for i in insns_after
if cur["off"] <= i["off"] < cur["off"] + len(new)]
report["post_text"] = " | ".join(patched.describe(i) for i in landed)
# the next instruction boundary after the replaced span must still be the
# instruction that followed the original one
nxt_after = patched.insn_at(code_off, cur["off"] + len(new))
report["post_next_text"] = patched.describe(nxt_after) if nxt_after else None
if not report["post_decode_clean"]:
raise RuntimeError(
"[%s] the method no longer decodes to exactly insns_size units after "
"the edit -- the replacement changed the instruction stream length. "
"Use an equal-length replacement." % label)
edges = _verifier_report(patched, code_off)
report["verifier_edges"] = len(edges)
if edges:
raise RuntimeError(
"[%s] verifier: a branch targets a move-result at %s; the class will "
"fail to load (VerifyError). Use a nop pair instead of a branch."
% (label, ", ".join("0x%x->0x%x" % e for e in edges)))
report["ok"] = True
return report, data
def main(argv):
ap = argparse.ArgumentParser(
description="Apply equal-length byte patches to a dex, with structural, "
"polarity and verifier checks.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__.split("Spec format")[-1][:1200] if __doc__ else None)
ap.add_argument("dex", help="input .dex, or an APK/zip (use --entry to pick)")
ap.add_argument("--entry", help="archive member to read (default: classes.dex)")
ap.add_argument("--spec", required=True, help="JSON spec file")
ap.add_argument("-o", "--out", help="output dex path")
ap.add_argument("--dry-run", action="store_true",
help="report matches and diffs without writing")
ap.add_argument("--report", help="write a machine-readable JSON report here")
args = ap.parse_args(argv[1:])
dex, entry = load_dex(args.dex, args.entry) if os.path.exists(args.dex) else (None, None)
if dex is None:
# allow raw bytes piped through a path that does not exist yet: fail clearly
print("error: no such file: %s" % args.dex)
return 2
problems = dex.check()
if problems:
print("structural problems in %s (%s):" % (args.dex, entry))
for p in problems:
print(" - %s" % p)
print("refusing to patch a source that does not parse cleanly")
return 1
with open(args.spec, encoding="utf-8") as fh:
spec_doc = json.load(fh)
specs = spec_doc["patches"] if isinstance(spec_doc, dict) else spec_doc
ok_before = verify_dex_header(dex.d)
print("== source: %s (%s, %d bytes)" % (args.dex, entry, len(dex.d)))
print(" header before: checksum_ok=%s signature_ok=%s"
% ok_before)
if not all(ok_before):
print(" note: the SOURCE dex header is already inconsistent. Some")
print(" producers ship a zeroed signature field; the patch will fix it.")
data = bytearray(dex.d)
reports = []
for i, spec in enumerate(specs):
rep, data = process_one(dex, data, spec, i, args.dry_run)
reports.append(rep)
print("\n-- [%s] %s.%s%s" % (rep["name"], rep["class"], rep["method"], rep["desc"]))
print(" site 0x%x %s" % (rep["site_off"], rep["old_text"]))
for line in rep["before"][-2:]:
print(" (before) %s" % line)
print(" -> %s" % rep.get("new_text"))
for line in rep["after"][:2]:
print(" (after) %s" % line)
if rep.get("expect_next_ok"):
print(" polarity: expect_next satisfied")
if args.dry_run:
print("\n== dry run: nothing written")
if args.report:
with open(args.report, "w", encoding="utf-8") as fh:
json.dump(reports, fh, indent=2)
print(" report: %s" % args.report)
return 0
header = fix_dex_header(data)
ok_after = verify_dex_header(data)
print("\n== header checksum 0x%08x -> 0x%08x" % (header["before_checksum"],
header["after_checksum"]))
print(" signature %s -> %s"
% (header["before_signature"].hex()[:16],
header["after_signature"].hex()[:16]))
print(" self-verify: checksum_ok=%s signature_ok=%s" % ok_after)
if not all(ok_after):
print("FATAL: header does not self-verify; nothing written")
return 1
out = args.out
if not out:
root, ext = os.path.splitext(args.dex)
out = root + ".patched" + (ext or ".dex")
with open(out, "wb") as fh:
fh.write(bytes(data))
print("== wrote %s (%d bytes, delta %d)"
% (out, len(data), len(data) - len(dex.d)))
if args.report:
with open(args.report, "w", encoding="utf-8") as fh:
json.dump(reports, fh, indent=2)
print(" report: %s" % args.report)
print("\nNext: repack (scripts/repack.py) then sign with apksigner, then run "
"the device + screen checks in references/verification.md.")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))

167
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_strings.py Normal file
View File

@@ -0,0 +1,167 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Extract strings, URLs and vendor markers from dex files without a decompiler.
Fast recon: dex strings are stored as plain MUTF-8, so ASCII substrings can be found
by scanning the raw bytes. This answers most classification questions in seconds --
which dex holds the app's code, which ad SDKs ship with it, what endpoints it calls.
Usage
-----
# all URLs across a directory of dex files
python dex_strings.py <dex_dir> --urls
# find which dex files contain a marker (ASCII substring)
python dex_strings.py <dex_dir> --find 'openadsdk|com.qq.e|anythink' --per-file
# dump a class/package inventory per dex (type descriptors)
python dex_strings.py <dex_dir> --classes 'Lcom/example/app/'
# raw string-table dump, length-filtered
python dex_strings.py <dex_dir> --strings --min 8 --max 60
# compare two dex files' string tables (what changed after a patch)
python dex_strings.py --diff a.dex b.dex
Notes
-----
* A marker in the string table means the class/URL is *present*, not that the app
*uses* that feature. Confirm with runtime behavior before acting.
* `--urls` output is the fastest way to see an app's whole API surface and its
third-party endpoints at once.
* A dex may be a multi-dex set: run this per directory, not per file, to see which
split holds a given marker.
"""
import argparse
import glob
import os
import re
import struct
import sys
URL_RE = re.compile(rb'https?://[A-Za-z0-9\.\-_:/%\.\?=&~#]{4,200}')
def uleb128(data, off):
r = 0
s = 0
while True:
b = data[off]
off += 1
r |= (b & 0x7F) << s
if not (b & 0x80):
break
s += 7
return r, off
def dex_strings(path):
"""Yield the dex string table in table order, as bytes."""
data = open(path, 'rb').read()
if data[:4] != b'dex\n':
return
n = struct.unpack('<I', data[0x38:0x3C])[0]
off = struct.unpack('<I', data[0x3C:0x40])[0]
for i in range(n):
sdata = struct.unpack('<I', data[off + i * 4: off + i * 4 + 4])[0]
ln, p = uleb128(data, sdata)
yield bytes(data[p:p + ln])
def iter_dex(target):
if os.path.isdir(target):
return sorted(glob.glob(os.path.join(target, '*.dex')))
return [target]
def main():
ap = argparse.ArgumentParser(
description='Extract strings / URLs / class descriptors / vendor markers from dex files '
'without a decompiler.')
ap.add_argument('target', nargs='?', default=None,
help='a .dex file or a directory containing .dex files (omit when using --diff)')
ap.add_argument('--urls', action='store_true', help='print only http(s) URLs')
ap.add_argument('--strings', action='store_true', help='dump raw string table entries')
ap.add_argument('--classes', metavar='PREFIX',
help='print only entries starting with this prefix (e.g. Lcom/example/app/)')
ap.add_argument('--find', metavar='REGEX',
help='only keep entries matching this regex (applied to the decoded text)')
ap.add_argument('--per-file', action='store_true',
help='with --find: report one line per dex with a hit count, instead of each string')
ap.add_argument('--diff', nargs=2, metavar=('A', 'B'),
help='compare two dex files string tables')
ap.add_argument('--min', type=int, default=6, help='minimum string length (default 6)')
ap.add_argument('--max', type=int, default=200, help='maximum string length (default 200)')
ap.add_argument('--max-print', type=int, default=5000,
help='stop printing after this many lines (default 5000; use 0 for unlimited)')
a = ap.parse_args()
# --diff mode needs no target
if a.diff:
A = set(dex_strings(a.diff[0]))
B = set(dex_strings(a.diff[1]))
only_a = sorted(x for x in A - B if a.min <= len(x) <= a.max)
only_b = sorted(x for x in B - A if a.min <= len(x) <= a.max)
print('A strings=%d B strings=%d' % (len(A), len(B)))
print('only in A: %d' % len(only_a))
for x in only_a[:40]:
print(' -', x.decode('utf-8', 'replace'))
print('only in B: %d' % len(only_b))
for x in only_b[:40]:
print(' +', x.decode('utf-8', 'replace'))
return 0
if not a.target:
ap.error('target is required unless --diff is used')
# Two patterns: the per-file mode scans raw bytes (fast), the per-string mode
# matches decoded text. Keeping both avoids the bytes/str mismatch that silently
# breaks one of the two paths.
pat_text = re.compile(a.find) if a.find else None
pat_bytes = re.compile(a.find.encode('utf-8')) if a.find else None
files = iter_dex(a.target)
if a.find and a.per_file:
for fp in files:
data = open(fp, 'rb').read()
hits = len(pat_bytes.findall(data))
if hits:
print('%-22s hits=%d' % (os.path.basename(fp), hits))
return 0
printed = 0
seen = set()
for fp in files:
for raw in dex_strings(fp):
s = raw.decode('utf-8', 'replace')
if not (a.min <= len(s) <= a.max):
continue
if pat_text and not pat_text.search(s):
continue
if a.classes and not s.startswith(a.classes):
continue
if a.urls and not raw.startswith((b'http://', b'https://')):
continue
if s in seen:
continue
seen.add(s)
if a.max_print and printed >= a.max_print:
print('[note] output cap reached (%d). Use --max-print 0 to lift.' % a.max_print)
return 0
if a.classes:
print('%-18s %s' % (os.path.basename(fp), s))
else:
print(s)
printed += 1
if a.urls and not seen:
# fall back to a raw byte scan: covers URLs stored outside the string table
print('[note] no URLs in string tables; raw scan:')
for fp in files:
for m in sorted(set(URL_RE.findall(open(fp, 'rb').read()))):
print(os.path.basename(fp), m.decode('utf-8', 'replace'))
return 0
if __name__ == '__main__':
sys.exit(main())

133
vendor-new/apk-reverse/skills/apk-reverse/scripts/dex_strpatch.py Normal file
View File

@@ -0,0 +1,133 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Byte-level, in-place patch of a string constant inside a dex — no smali round-trip.
Why not a smali round-trip: a disassemble -> reassemble of a whole tree can damage
R8-optimized output in ways the class table does not show. Observed symptom: a synthetic
access bridge lost its interface/class relationship, producing
IncompatibleClassChangeError ("Found interface X, but class was expected") at runtime even
though class_defs and access_flags looked untouched. See references/pitfalls.md P3.
What this script does instead (zero structural change to the dex):
1. Replaces the target string with another of **exactly equal byte length**. Equal length
keeps the string_data_item uleb128 length prefix unchanged, so offsets, indices and
every table stay exactly where they were.
2. Recomputes the dex header signature (SHA-1, from offset 32) and checksum (adler32,
from offset 12).
Use only when the replacement can be equal-length. Otherwise use a method-level dex API
rewrite (references/dex-patching.md, technique 1).
Usage: python dex_strpatch.py <in.dex> <out.dex> <old_str> <new_str>
Requires len(new) == len(old) in UTF-8 bytes, and exactly one occurrence of old in the dex.
"""
import hashlib
import struct
import sys
import zlib
def _uleb128(data, off):
result = 0
shift = 0
while True:
b = data[off]
off += 1
result |= (b & 0x7F) << shift
if not (b & 0x80):
break
shift += 7
return result, off
def _read_strings(data):
off = struct.unpack('<I', data[0x3C:0x40])[0]
size = struct.unpack('<I', data[0x38:0x3C])[0]
out = []
for i in range(size):
sdata_off = struct.unpack('<I', data[off + i * 4: off + i * 4 + 4])[0]
n, p = _uleb128(data, sdata_off)
out.append((sdata_off, bytes(data[p:p + n])))
return out
def _order_ok(data, off, ob, nb):
"""Check whether replacing `ob` with `nb` still satisfies the string_ids ordering rule.
Note: string_ids stores the offset of each string_data_item — i.e. the position of the
**length prefix** (a few bytes ahead of the UTF-8 payload), not of the text itself — so
matching by string-data offset directly is not possible; this matches the entry by
content instead.
"""
entries = _read_strings(data)
idx = None
for i, (_sdata_off, raw) in enumerate(entries):
if raw == ob:
idx = i
break
if idx is None:
print('[order] target string not found in string_ids, skip check')
return True
prev = entries[idx - 1][1] if idx > 0 else None
nxt = entries[idx + 1][1] if idx + 1 < len(entries) else None
if prev is not None and nb <= prev:
print('[order] FAIL: new %r <= prev %r' % (nb, prev))
return False
if nxt is not None and nb >= nxt:
print('[order] FAIL: new %r >= next %r' % (nb, nxt))
return False
print('[order] OK: %r < %r < %r' % (prev, nb, nxt))
return True
def main():
if len(sys.argv) > 1 and sys.argv[1] in ('-h', '--help'):
print(__doc__)
return 0
if len(sys.argv) < 5:
print('error: needs 4 arguments, got %d\n' % (len(sys.argv) - 1))
print(__doc__)
return 2
src, dst, old, new = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]
ob, nb = old.encode('utf-8'), new.encode('utf-8')
if len(ob) != len(nb):
print('[FAIL] length mismatch: %d vs %d (must be equal)' % (len(ob), len(nb)))
return 1
data = bytearray(open(src, 'rb').read())
if data[:8] != b'dex\n035\x00' and data[:4] != b'dex\n':
print('[warn] unexpected magic: %r' % bytes(data[:8]))
n = data.count(ob)
print('[info] occurrences of %r = %d' % (old, n))
if n != 1:
print('[FAIL] expected exactly 1 occurrence')
return 1
off = data.find(ob)
# The dex spec requires string_ids to be ordered by string content. An equal-length
# replacement moves no offset, but it does change where this entry sits in that ordering
# — and once it lands out of order the whole dex is rejected by the ClassLoader
# (symptom: ClassNotFoundException naming the Application class). So validate the
# interval before writing.
if not _order_ok(data, off, ob, nb):
print('[FAIL] replacement would break string_ids ordering. '
'Pick a string that stays between its two neighbours.')
return 1
data[off:off + len(ob)] = nb
print('[ok] patched at offset 0x%x: %r -> %r' % (off, old, new))
# signature = SHA-1 over data[32:], stored at 12..32
sig = hashlib.sha1(bytes(data[32:])).digest()
data[12:32] = sig
# checksum = adler32 over data[12:], stored at 8..12 (little endian)
chk = zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF
data[8:12] = struct.pack('<I', chk)
print('[ok] signature=%s checksum=0x%08x' % (sig.hex(), chk))
open(dst, 'wb').write(bytes(data))
print('[ok] wrote %s (%d bytes)' % (dst, len(data)))
return 0
if __name__ == '__main__':
sys.exit(main())

97
vendor-new/apk-reverse/skills/apk-reverse/scripts/dexpatch/PatchMethod.java Normal file
View File

@@ -0,0 +1,97 @@
import org.jf.dexlib2.DexFileFactory;
import org.jf.dexlib2.Opcode;
import org.jf.dexlib2.Opcodes;
import org.jf.dexlib2.iface.ClassDef;
import org.jf.dexlib2.iface.Method;
import org.jf.dexlib2.iface.MethodImplementation;
import org.jf.dexlib2.builder.MutableMethodImplementation;
import org.jf.dexlib2.immutable.ImmutableMethod;
import org.jf.dexlib2.writer.pool.DexPool;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
/**
* dexlib2 定点 patch：把指定方法的实现整体替换为 return-void（或 return 常量）。
*
* 为什么不用 baksmali/smali 整树往返：实测在一个多 dex 应用上，整树往返重建后运行时报
* IncompatibleClassChangeError: Found interface io.ktor.client.engine.HttpClientEngine,
* but class was expected
* （R8 生成的 synthetic access bridge 被破坏）。本工具只改目标方法的 code_item，
* 其余类/方法/字符串表原样保留，因此不会有这类损伤。
*
* 用法: java PatchMethod <in.dex> <out.dex> <descriptor> <methodName> <returnType>
* 例: java PatchMethod classes7.dex out.dex Lx6; d V
*/
public class PatchMethod {
public static void main(String[] args) throws Exception {
if (args.length < 5) {
System.err.println("usage: PatchMethod <in.dex> <out.dex> <descriptor> <methodName> <returnType>");
System.exit(2);
}
String inPath = args[0], outPath = args[1], desc = args[2], mName = args[3], ret = args[4];
org.jf.dexlib2.iface.DexFile dex =
DexFileFactory.loadDexFile(new File(inPath), Opcodes.forApi(34));
List<ClassDef> outClasses = new ArrayList<ClassDef>();
int patched = 0;
for (ClassDef cd : dex.getClasses()) {
List<Method> direct = new ArrayList<Method>();
for (Method m : cd.getDirectMethods()) direct.add(m);
List<Method> virtual = new ArrayList<Method>();
for (Method m : cd.getVirtualMethods()) virtual.add(m);
boolean isTargetClass = cd.getType().equals(desc);
boolean changed = false;
if (isTargetClass) {
for (int pass = 0; pass < 2; pass++) {
List<Method> list = (pass == 0) ? direct : virtual;
for (int i = 0; i < list.size(); i++) {
Method m = list.get(i);
if (!m.getName().equals(mName) || !m.getReturnType().equals(ret)) continue;
// 用 return-void 替换整个方法体。
// 寄存器数必须保留原值：Compose/协程方法体里可能仍被其它指令引用同一寄存器窗口，
// 且 dex 校验要求 registers >= 参数寄存器数。
int regs = m.getImplementation() != null ? m.getImplementation().getRegisterCount() : 0;
MethodImplementation impl = new org.jf.dexlib2.immutable.ImmutableMethodImplementation(
regs,
java.util.Collections.singletonList(
new org.jf.dexlib2.immutable.instruction.ImmutableInstruction10x(Opcode.RETURN_VOID)),
null,
null);
Method nm = new ImmutableMethod(
m.getDefiningClass(), m.getName(), m.getParameters(), m.getReturnType(),
m.getAccessFlags(), m.getAnnotations(), m.getHiddenApiRestrictions(),
impl);
list.set(i, nm);
patched++;
changed = true;
System.out.println("[patch] " + desc + "->" + mName + m.getReturnType()
+ " registers=" + regs);
}
}
}
if (!changed) {
outClasses.add(cd);
} else {
outClasses.add(new org.jf.dexlib2.immutable.ImmutableClassDef(
cd.getType(), cd.getAccessFlags(), cd.getSuperclass(),
cd.getInterfaces(), cd.getSourceFile(), cd.getAnnotations(),
cd.getStaticFields(), cd.getInstanceFields(), direct, virtual));
}
}
DexFileFactory.writeDexFile(outPath,
new org.jf.dexlib2.immutable.ImmutableDexFile(Opcodes.forApi(34), outClasses));
System.out.println("[done] patched=" + patched + " -> " + outPath);
if (patched == 0) {
System.err.println("[warn] no method matched!");
System.exit(1);
}
}
}

202
vendor-new/apk-reverse/skills/apk-reverse/scripts/dexpatch/PatchMethodExample.java Normal file
View File

@@ -0,0 +1,202 @@
import org.jf.dexlib2.DexFileFactory;
import org.jf.dexlib2.Opcode;
import org.jf.dexlib2.Opcodes;
import org.jf.dexlib2.iface.ClassDef;
import org.jf.dexlib2.iface.Method;
import org.jf.dexlib2.iface.instruction.Instruction;
import org.jf.dexlib2.immutable.ImmutableClassDef;
import org.jf.dexlib2.immutable.ImmutableDexFile;
import org.jf.dexlib2.immutable.ImmutableMethod;
import org.jf.dexlib2.immutable.ImmutableMethodImplementation;
import org.jf.dexlib2.immutable.instruction.ImmutableInstruction11x;
import org.jf.dexlib2.immutable.instruction.ImmutableInstruction22c;
import org.jf.dexlib2.immutable.instruction.ImmutableInstruction35c;
import org.jf.dexlib2.immutable.instruction.ImmutableInstruction51l;
import org.jf.dexlib2.immutable.reference.ImmutableFieldReference;
import org.jf.dexlib2.immutable.reference.ImmutableMethodReference;
import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
/**
* dexlib2 surgical rewrites -- worked example: force one lambda's emit() to hand a
* constant downstream instead of the value it read from its source.
*
* WHAT IT PATCHES
* <target>$$inlined$map$1$2.emit(Object, Continuation)Object
* -> send a fixed wide (long) constant to the downstream collector, unconditionally
*
* WHY THIS SHAPE
* Kotlin's `map { ... }` on a Flow compiles to a synthetic class whose emit()
* reads the source value from a field and forwards it to the collector. Replacing
* that single forwarding body turns "value read from the data source" into
* "constant", without touching the data source itself. Doing it in dex keeps the
* change inside the APK, so a fresh install behaves the same way -- writing the
* value into the app's runtime state instead would not survive a reinstall.
*
* ONE READ, ONE WRITE. Never rewrite the same dex twice in the same run: dexlib2
* round-trips degrade an R8-optimized dex, and the second pass is where it shows.
*
* ADAPT TO YOUR TARGET
* java PatchMethodExample <in.dex> <out.dex> [targetClass] [fieldOwner:fieldName:fieldType]
* [collectorIface] [continuationType]
* Every optional argument defaults to an obviously-fake example value below, and
* the effective values are printed at startup, so a run is never ambiguous.
*
* HOW TO FIND THE REAL VALUES
* * target class: search the dex strings for the source's field/data key, then read
* the call site with baksmali -- see scripts/dex_strings.py and scripts/find_refs.py.
* * collector iface / continuation type: read them off the emit() signature in smali.
* After R8 they are usually single letters; do not guess, copy them.
*
* DO NOT PATCH SHARED CODE BY ACCIDENT
* A generic card/image composable is commonly shared between ordinary content and
* ad slots. Turning such a method into `return-void` breaks normal rendering too.
* Before patching anything, count its callers (scripts/find_refs.py).
*/
public class PatchMethodExample {
/** Synthetic class that owns the lambda: <Owner>$<member>$$inlined$map$1$2. */
private static String TARGET_CLASS =
"Lcom/example/app/data/ExampleStore$currentValue$$inlined$map$1$2;";
/** The field on the lambda that holds the downstream collector. */
private static String FIELD_OWNER = "Lcom/example/app/data/ExampleStore$currentValue$$inlined$map$1$2;";
private static String FIELD_NAME = "b";
private static String FIELD_TYPE = "Lcom/example/app/Collector;";
/** The collector type and the Continuation type used by emit(). */
private static String COLLECTOR_IFACE = "Lcom/example/app/Collector;";
private static String CONTINUATION_TYPE = "Lkotlin/coroutines/Continuation;";
private static final long CONSTANT = 4102444800000L; // ~2100-01-01 (ms)
private static void usage() {
System.out.println("usage: PatchMethodExample <in.dex> <out.dex> "
+ "[targetClass] [fieldOwner:fieldName:fieldType] "
+ "[collectorIface] [continuationType]");
System.out.println(" defaults (example values, replace for your target):");
System.out.println(" targetClass " + TARGET_CLASS);
System.out.println(" fieldRef " + FIELD_OWNER + ":" + FIELD_NAME + ":" + FIELD_TYPE);
System.out.println(" collectorIface " + COLLECTOR_IFACE);
System.out.println(" continuationType " + CONTINUATION_TYPE);
}
private static void applyArgs(String[] args) {
if (args.length > 2 && !args[2].isEmpty()) {
TARGET_CLASS = args[2];
}
if (args.length > 3 && !args[3].isEmpty()) {
String[] parts = args[3].split(":");
if (parts.length != 3) {
throw new IllegalArgumentException(
"fieldRef must be owner:name:type, got " + args[3]);
}
FIELD_OWNER = parts[0];
FIELD_NAME = parts[1];
FIELD_TYPE = parts[2];
}
if (args.length > 4 && !args[4].isEmpty()) {
COLLECTOR_IFACE = args[4];
}
if (args.length > 5 && !args[5].isEmpty()) {
CONTINUATION_TYPE = args[5];
}
}
public static void main(String[] args) throws Exception {
if (args.length < 2) {
usage();
System.exit(2);
}
applyArgs(args);
String in = args[0], out = args[1];
System.out.println("[in ] " + in);
System.out.println("[cfg] targetClass=" + TARGET_CLASS);
System.out.println("[cfg] fieldRef=" + FIELD_OWNER + ":" + FIELD_NAME + ":" + FIELD_TYPE);
System.out.println("[cfg] collectorIface=" + COLLECTOR_IFACE);
System.out.println("[cfg] continuationType=" + CONTINUATION_TYPE);
System.out.println("[cfg] constant=" + CONSTANT);
org.jf.dexlib2.iface.DexFile dex =
DexFileFactory.loadDexFile(new File(in), Opcodes.forApi(34));
List<ClassDef> outClasses = new ArrayList<ClassDef>();
int patched = 0;
for (ClassDef cd : dex.getClasses()) {
List<Method> direct = new ArrayList<Method>();
for (Method m : cd.getDirectMethods()) direct.add(m);
List<Method> virtual = new ArrayList<Method>();
for (Method m : cd.getVirtualMethods()) virtual.add(m);
boolean touched = false;
if (cd.getType().equals(TARGET_CLASS)) {
for (int pass = 0; pass < 2; pass++) {
List<Method> list = (pass == 0) ? direct : virtual;
for (int i = 0; i < list.size(); i++) {
Method m = list.get(i);
if (!m.getName().equals("emit")
|| !m.getReturnType().equals("Ljava/lang/Object;")
|| m.getImplementation() == null) {
continue;
}
// Register layout for emit(Object, Continuation):
// v0 scratch (return value)
// v1:v2 the wide constant we build
// v3 = p0 this (the lambda)
// v4 = p1 the value being forwarded (ignored)
// v5 = p2 the downstream Continuation
List<Instruction> body = new ArrayList<Instruction>();
body.add(new ImmutableInstruction22c(Opcode.IGET_OBJECT, 0, 3,
new ImmutableFieldReference(FIELD_OWNER, FIELD_NAME, FIELD_TYPE)));
body.add(new ImmutableInstruction51l(Opcode.CONST_WIDE, 1, CONSTANT));
body.add(new ImmutableInstruction35c(Opcode.INVOKE_STATIC, 2, 1, 2, 0, 0, 0,
new ImmutableMethodReference("Ljava/lang/Long;", "valueOf",
Collections.singletonList("J"), "Ljava/lang/Long;")));
body.add(new ImmutableInstruction11x(Opcode.MOVE_RESULT_OBJECT, 1));
body.add(new ImmutableInstruction35c(Opcode.INVOKE_INTERFACE, 3, 0, 1, 5, 0, 0,
new ImmutableMethodReference(COLLECTOR_IFACE, "emit",
Arrays.asList("Ljava/lang/Object;", CONTINUATION_TYPE),
"Ljava/lang/Object;")));
body.add(new ImmutableInstruction11x(Opcode.MOVE_RESULT_OBJECT, 0));
body.add(new ImmutableInstruction11x(Opcode.RETURN_OBJECT, 0));
list.set(i, new ImmutableMethod(m.getDefiningClass(), m.getName(),
m.getParameters(), m.getReturnType(), m.getAccessFlags(),
m.getAnnotations(), m.getHiddenApiRestrictions(),
new ImmutableMethodImplementation(6, body, null, null)));
patched++;
touched = true;
System.out.println("[patch] emit -> constant=" + CONSTANT
+ " (list=" + pass + ")");
}
}
}
if (!touched) {
outClasses.add(cd);
} else {
outClasses.add(new ImmutableClassDef(
cd.getType(), cd.getAccessFlags(), cd.getSuperclass(), cd.getInterfaces(),
cd.getSourceFile(), cd.getAnnotations(), cd.getStaticFields(),
cd.getInstanceFields(), direct, virtual));
}
}
DexFileFactory.writeDexFile(out, new ImmutableDexFile(Opcodes.forApi(34), outClasses));
System.out.println("[out ] " + out + " patched=" + patched);
if (patched == 0) {
// The usual causes: wrong target class (R8 renamed it), the method is not
// emit(Object,Continuation), or the class is not in THIS dex.
System.err.println("[fail] target not found in this dex. Check: the exact "
+ "TARGET_CLASS descriptor, that emit(Object,Continuation) exists there, "
+ "and that you loaded the dex that actually contains this class.");
System.exit(1);
}
}
}

78
vendor-new/apk-reverse/skills/apk-reverse/scripts/dexpatch/README.md Normal file
View File

@@ -0,0 +1,78 @@
# dexlib2 method-level patcher
The **preferred** way to change app behavior. It replaces only a target method's
implementation and writes a new dex, leaving every other class, method, string and
reference untouched.
Why not whole-tree smali round-trip: `baksmali` → `smali` rebuilds misrepresent R8's
synthetic access bridges. The class table still looks perfect — same class count, zero
`ACC_INTERFACE` mismatches — but at runtime you get
`IncompatibleClassChangeError: Found interface X, but class was expected`.
See `../../references/pitfalls.md` P3.
## Build
Needs the same eight jars `smtool.py` uses:
```
smali-2.5.2.jar antlr-runtime-3.5.2.jar stringtemplate-3.2.1.jar
baksmali-2.5.2.jar util-2.5.2.jar jcommander-1.64.jar
guava-27.1-android.jar dexlib2-2.5.2.jar
```
```bash
# point at your jars (or reuse ../smali_cp.txt)
CP=$(cat ../smali_cp.txt | grep -v '^#' | tr '\n' ';' | sed 's/;$//')
javac -encoding UTF-8 -cp "$CP" PatchMethod.java
```
**Always pass `-encoding UTF-8`** if the source has non-ASCII comments — `javac`
otherwise reads them with the platform default encoding and fails.
## Run
```bash
java -cp "$CP;." PatchMethod <in.dex> <out.dex> <descriptor> <methodName> <returnType>
# example: make Lx6;->d(...)V a no-op
java -cp "$CP;." PatchMethod classes7.dex out.dex 'Lx6;' d V
```
`PatchMethod` replaces the body with `return-void`. It scans **both** `directMethods`
and `virtualMethods` — a method you cannot find is often in the other list
(`static`/`private`/`<init>` are direct; everything else, including `public final`,
is virtual).
Use `PatchMethodExample.java` as the template for anything non-trivial (returning a
constant, emitting a fixed value from a Flow lambda, applying **several** edits to the
same dex in one pass).
## Rules that decide success
1. **One dex, one write pass.** Never chain two patchers over the same dex — the second
serialization drops metadata and ART then refuses to start the process
(`Failure starting process`, no Java stack). Combine all edits into one program.
2. **Preserve the register count.** `registers` must be `>=` the parameter register
count. With `registers = N` and `k` parameters (counting `this` for non-static),
`p0 = N - k`. For `(Ljava/lang/Object;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;`
on an instance method, `k = 3`, so with `registers = 6` you get `p0=3, p1=4, p2=5`.
3. **`const-wide` needs the 64-bit instruction form** (`ImmutableInstruction51l`), not
the 32-bit one.
4. **`invoke` operand order in `35c`** is `(opcode, registerCount, regC, regD, regE, regF, regG, ref)`.
`invoke-static {v1, v2}, ...` → `registerCount=2, regC=1, regD=2`.
5. **End every body with a return of the correct type.**
6. **Check the blast radius first** — `../find_refs.py`. A generic helper with dozens of
callers is not your patch point (`../../references/pitfalls.md` P6).
## Verify
```bash
python ../dex_classdiff.py <original.dex> <patched.dex>
python ../smtool.py d <patched.dex> <tmp_tree> # confirm it parses
# then read the target method in tmp_tree to confirm it says what you intended
```
Expect `only_in_A=0`, `only_in_B=0`, `ACC_INTERFACE mismatch: 0`. Then install and
launch — table checks cannot see code-item damage, so runtime verification is
mandatory (`../../references/verification.md`).

707
vendor-new/apk-reverse/skills/apk-reverse/scripts/dexutil.py Normal file
View File

@@ -0,0 +1,707 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Minimal, dependency-free dex reader: structure walk + exact instruction decode.
WHY THIS EXISTS
---------------
Locating the exact byte offset of one instruction is where byte-level patching
either works or wastes an afternoon. Two shortcuts do not work and are worth
naming so nobody re-derives them:
* Reconstructing offsets from a baksmali listing. `.line` directives track
SOURCE lines, and one source line can span several instructions, so the
values repeat and do not map 1:1 onto code offsets.
* Matching a guessed byte sequence. Encodings vary with register numbers and
operand widths, and a plausible-looking opcode can belong to a different
format than you assume (0x38 is `if-test`, 22t/4 bytes -- not `if-testz`).
So: walk class -> class_data_item -> method -> code_item, then decode forward with
a complete format table, and match on decoded semantics instead of bytes.
Three details in the walk that silently produce wrong answers:
* `class_data_item` member indices are DELTAS against the previous entry in the
same list. Reading a raw uleb as an absolute index yields real-looking wrong
members.
* dalvik encodes switch/array payloads as `00 <ident> <size>` with ident 1..3.
A plain `00 00` is an ordinary one-unit nop; treating every 00 as a payload
swallows the next instruction and desynchronises the rest of the method.
* The header field order is easy to mis-remember. See OFFSETS below, and always
sanity-check the parsed sizes against the file.
Pure standard library. Import it, or run it for a per-method dump:
python dexutil.py <dex-or-apk> <Class/Name;> <method> <descriptor>
"""
import hashlib
import sys
import zipfile
# Verified dex header offsets. After magic(8) + checksum(4) + signature(20):
OFFSETS = {
"file_size": 0x20, "header_size": 0x24, "endian_tag": 0x28,
"link_size": 0x2C, "link_off": 0x30, "map_off": 0x34,
"string_ids_size": 0x38, "string_ids_off": 0x3C,
"type_ids_size": 0x40, "type_ids_off": 0x44,
"proto_ids_size": 0x48, "proto_ids_off": 0x4C,
"field_ids_size": 0x50, "field_ids_off": 0x54,
"method_ids_size": 0x58, "method_ids_off": 0x5C,
"class_defs_size": 0x60, "class_defs_off": 0x64,
"data_size": 0x68, "data_off": 0x6C,
}
# NOTE — known defect, measured, deliberately not guessed at.
#
# This NAME table is misaligned with the opcode slots from roughly 0x1a onward. Measured
# against dexlib2 by joining both decoders on instruction offset: the slot this table
# calls `array-length` decodes as `instance-of`, the slot called `new-instance` decodes
# as `array-length`, the slot called `goto/16` decodes as `goto`. Exact or not, the point
# is that the names are NOT a safe key for looking up an authoritative width.
#
# The width decisions in `insn_units` are keyed on the opcode VALUE, not on these names,
# so a wrong name does not by itself corrupt a decode -- but it will mislead a reader,
# and it misled one here into "fixing" two widths that were already right.
#
# Fixed and verified in this pass (method: full-decode alignment on three real dex
# images, asserting each walk ends exactly on insns_off + insns_size*2):
# * payload widths. The field after the `00 <ident>` is a DIFFERENT quantity per
# payload kind: packed-switch has size(uint16)+first_key(int32), sparse-switch has
# size(uint16)+size*(key,target), fill-array-data has element_width(uint16)+
# size(uint32). All three were previously read as `1 + <one uint16>`, so a 30-element
# 4-byte-wide array payload counted as 5 units instead of 64.
# * 0x22 is 22c (2 units) and 0x23/0x24 are 35c/3rc (3 units); they sat in each other's
# groups.
# * 0x20 was grouped as 2 units.
# Result: methods whose decode fails to land on the exact end went from 137/113/70 to
# 44/51/30 across the three images.
#
# Still open: the name misalignment above. It needs the table rebuilt from an
# independent decoder in one pass, not edited slot by slot -- two slot-wise edits made
# during this investigation were reverted after measurement showed they made alignment
# worse. Treat a name from this table as a hint, and check any width you intend to rely
# on against a second decoder.
OP_NAMES = {
0x00: "nop", 0x01: "move", 0x02: "move/from16", 0x03: "move/16",
0x04: "move-wide", 0x05: "move-wide/from16", 0x06: "move-wide/16", 0x07: "move-object",
0x08: "move-object/from16", 0x09: "move-object/16", 0x0A: "move-result", 0x0B: "move-result-wide",
0x0C: "move-result-object", 0x0D: "move-exception", 0x0E: "return-void", 0x0F: "return",
0x10: "return-wide", 0x11: "return-object", 0x12: "const/4", 0x13: "const/16",
0x14: "const", 0x15: "const/high16", 0x16: "const-wide/16", 0x17: "const-wide/32",
0x18: "const-wide", 0x19: "const-wide/high16", 0x1A: "const-string", 0x1B: "const-string/jumbo",
0x1C: "const-class", 0x1D: "monitor-enter", 0x1E: "monitor-exit", 0x1F: "check-cast",
0x20: "instance-of", 0x21: "array-length", 0x22: "new-instance", 0x23: "new-array",
0x24: "filled-new-array", 0x25: "filled-new-array/range", 0x26: "fill-array-data", 0x27: "throw",
0x28: "goto", 0x29: "goto/16", 0x2A: "goto/32", 0x2B: "packed-switch",
0x2C: "sparse-switch", 0x2D: "cmpl-float", 0x2E: "cmpg-float", 0x2F: "cmpl-double",
0x30: "cmpg-double", 0x31: "cmp-long", 0x32: "if-eq", 0x33: "if-ne",
0x34: "if-lt", 0x35: "if-ge", 0x36: "if-gt", 0x37: "if-le",
0x38: "if-eqz", 0x39: "if-nez", 0x3A: "if-ltz", 0x3B: "if-gez",
0x3C: "if-gtz", 0x3D: "if-lez", 0x44: "aget", 0x45: "aget-wide",
0x46: "aget-object", 0x47: "aget-boolean", 0x48: "aget-byte", 0x49: "aget-char",
0x4A: "aget-short", 0x4B: "aput", 0x4C: "aput-wide", 0x4D: "aput-object",
0x4E: "aput-boolean", 0x4F: "aput-byte", 0x50: "aput-char", 0x51: "aput-short",
0x52: "iget", 0x53: "iget-wide", 0x54: "iget-object", 0x55: "iget-boolean",
0x56: "iget-byte", 0x57: "iget-char", 0x58: "iget-short", 0x59: "iput",
0x5A: "iput-wide", 0x5B: "iput-object", 0x5C: "iput-boolean", 0x5D: "iput-byte",
0x5E: "iput-char", 0x5F: "iput-short", 0x60: "sget", 0x61: "sget-wide",
0x62: "sget-object", 0x63: "sget-boolean", 0x64: "sget-byte", 0x65: "sget-char",
0x66: "sget-short", 0x67: "sput", 0x68: "sput-wide", 0x69: "sput-object",
0x6A: "sput-boolean", 0x6B: "sput-byte", 0x6C: "sput-char", 0x6D: "sput-short",
0x6E: "invoke-virtual", 0x6F: "invoke-super", 0x70: "invoke-direct", 0x71: "invoke-static",
0x72: "invoke-interface", 0x74: "invoke-virtual/range", 0x75: "invoke-super/range", 0x76: "invoke-direct/range",
0x77: "invoke-static/range", 0x78: "invoke-interface/range", 0x7B: "neg-int", 0x7C: "not-int",
0x7D: "neg-long", 0x7E: "not-long", 0x7F: "neg-float", 0x80: "neg-double",
0x81: "int-to-long", 0x82: "int-to-float", 0x83: "int-to-double", 0x84: "long-to-int",
0x85: "long-to-float", 0x86: "long-to-double", 0x87: "float-to-int", 0x88: "float-to-long",
0x89: "float-to-double", 0x8A: "double-to-int", 0x8B: "double-to-long", 0x8C: "double-to-float",
0x8D: "int-to-byte", 0x8E: "int-to-char", 0x8F: "int-to-short", 0x90: "add-int",
0x91: "sub-int", 0x92: "mul-int", 0x93: "div-int", 0x94: "rem-int",
0x95: "and-int", 0x96: "or-int", 0x97: "xor-int", 0x98: "shl-int",
0x99: "shr-int", 0x9A: "ushr-int", 0x9B: "add-long", 0x9C: "sub-long",
0x9D: "mul-long", 0x9E: "div-long", 0x9F: "rem-long", 0xA0: "and-long",
0xA1: "or-long", 0xA2: "xor-long", 0xA3: "shl-long", 0xA4: "shr-long",
0xA5: "ushr-long", 0xA6: "add-float", 0xA7: "sub-float", 0xA8: "mul-float",
0xA9: "div-float", 0xAA: "rem-float", 0xAB: "add-double", 0xAC: "sub-double",
0xAD: "mul-double", 0xAE: "div-double", 0xAF: "rem-double", 0xB0: "add-int/2addr",
0xB1: "sub-int/2addr", 0xB2: "mul-int/2addr", 0xB3: "div-int/2addr", 0xB4: "rem-int/2addr",
0xB5: "and-int/2addr", 0xB6: "or-int/2addr", 0xB7: "xor-int/2addr", 0xB8: "shl-int/2addr",
0xB9: "shr-int/2addr", 0xBA: "ushr-int/2addr", 0xBB: "add-long/2addr", 0xBC: "sub-long/2addr",
0xBD: "mul-long/2addr", 0xBE: "div-long/2addr", 0xBF: "rem-long/2addr", 0xC0: "and-long/2addr",
0xC1: "or-long/2addr", 0xC2: "xor-long/2addr", 0xC3: "shl-long/2addr", 0xC4: "shr-long/2addr",
0xC5: "ushr-long/2addr", 0xC6: "add-float/2addr", 0xC7: "sub-float/2addr", 0xC8: "mul-float/2addr",
0xC9: "div-float/2addr", 0xCA: "rem-float/2addr", 0xCB: "add-double/2addr", 0xCC: "sub-double/2addr",
0xCD: "mul-double/2addr", 0xCE: "div-double/2addr", 0xCF: "rem-double/2addr", 0xD0: "add-int/lit16",
0xD1: "rsub-int", 0xD2: "mul-int/lit16", 0xD3: "div-int/lit16", 0xD4: "rem-int/lit16",
0xD5: "and-int/lit16", 0xD6: "or-int/lit16", 0xD7: "xor-int/lit16", 0xD8: "add-int/lit8",
0xD9: "rsub-int/lit8", 0xDA: "mul-int/lit8", 0xDB: "div-int/lit8", 0xDC: "rem-int/lit8",
0xDD: "and-int/lit8", 0xDE: "or-int/lit8", 0xDF: "xor-int/lit8", 0xE0: "shl-int/lit8",
0xE1: "shr-int/lit8", 0xE2: "ushr-int/lit8", 0xFA: "invoke-polymorphic", 0xFB: "invoke-polymorphic/range",
0xFC: "invoke-custom", 0xFD: "invoke-custom/range", 0xFE: "const-method-handle", 0xFF: "const-method-type",
}
# Instruction length in 16-bit code units, one entry per valid opcode.
#
# Keyed by opcode from the format specification, NOT inferred from a mnemonic
# and NOT copied from a run of neighbours. 0x16-0x2C alternates
# 2,3,5,2,2,3,2,1,1,2,2,1,2,2,3,3,3,1,1,2,3,3,3 -- an entire run where the
# plausible-looking "same family, same width" assumption is wrong at nine
# opcodes. A single wrong entry shifts every instruction after it in the same
# method, and the decode keeps producing plausible instructions, so nothing
# looks broken until an operand index walks off its table.
OP_UNITS = {
0x00: 1, 0x01: 1, 0x02: 2, 0x03: 3,
0x04: 1, 0x05: 2, 0x06: 3, 0x07: 1,
0x08: 2, 0x09: 3, 0x0A: 1, 0x0B: 1,
0x0C: 1, 0x0D: 1, 0x0E: 1, 0x0F: 1,
0x10: 1, 0x11: 1, 0x12: 1, 0x13: 2,
0x14: 3, 0x15: 2, 0x16: 2, 0x17: 3,
0x18: 5, 0x19: 2, 0x1A: 2, 0x1B: 3,
0x1C: 2, 0x1D: 1, 0x1E: 1, 0x1F: 2,
0x20: 2, 0x21: 1, 0x22: 2, 0x23: 2,
0x24: 3, 0x25: 3, 0x26: 3, 0x27: 1,
0x28: 1, 0x29: 2, 0x2A: 3, 0x2B: 3,
0x2C: 3, 0x2D: 2, 0x2E: 2, 0x2F: 2,
0x30: 2, 0x31: 2, 0x32: 2, 0x33: 2,
0x34: 2, 0x35: 2, 0x36: 2, 0x37: 2,
0x38: 2, 0x39: 2, 0x3A: 2, 0x3B: 2,
0x3C: 2, 0x3D: 2, 0x44: 2, 0x45: 2,
0x46: 2, 0x47: 2, 0x48: 2, 0x49: 2,
0x4A: 2, 0x4B: 2, 0x4C: 2, 0x4D: 2,
0x4E: 2, 0x4F: 2, 0x50: 2, 0x51: 2,
0x52: 2, 0x53: 2, 0x54: 2, 0x55: 2,
0x56: 2, 0x57: 2, 0x58: 2, 0x59: 2,
0x5A: 2, 0x5B: 2, 0x5C: 2, 0x5D: 2,
0x5E: 2, 0x5F: 2, 0x60: 2, 0x61: 2,
0x62: 2, 0x63: 2, 0x64: 2, 0x65: 2,
0x66: 2, 0x67: 2, 0x68: 2, 0x69: 2,
0x6A: 2, 0x6B: 2, 0x6C: 2, 0x6D: 2,
0x6E: 3, 0x6F: 3, 0x70: 3, 0x71: 3,
0x72: 3, 0x74: 3, 0x75: 3, 0x76: 3,
0x77: 3, 0x78: 3, 0x7B: 1, 0x7C: 1,
0x7D: 1, 0x7E: 1, 0x7F: 1, 0x80: 1,
0x81: 1, 0x82: 1, 0x83: 1, 0x84: 1,
0x85: 1, 0x86: 1, 0x87: 1, 0x88: 1,
0x89: 1, 0x8A: 1, 0x8B: 1, 0x8C: 1,
0x8D: 1, 0x8E: 1, 0x8F: 1, 0x90: 2,
0x91: 2, 0x92: 2, 0x93: 2, 0x94: 2,
0x95: 2, 0x96: 2, 0x97: 2, 0x98: 2,
0x99: 2, 0x9A: 2, 0x9B: 2, 0x9C: 2,
0x9D: 2, 0x9E: 2, 0x9F: 2, 0xA0: 2,
0xA1: 2, 0xA2: 2, 0xA3: 2, 0xA4: 2,
0xA5: 2, 0xA6: 2, 0xA7: 2, 0xA8: 2,
0xA9: 2, 0xAA: 2, 0xAB: 2, 0xAC: 2,
0xAD: 2, 0xAE: 2, 0xAF: 2, 0xB0: 1,
0xB1: 1, 0xB2: 1, 0xB3: 1, 0xB4: 1,
0xB5: 1, 0xB6: 1, 0xB7: 1, 0xB8: 1,
0xB9: 1, 0xBA: 1, 0xBB: 1, 0xBC: 1,
0xBD: 1, 0xBE: 1, 0xBF: 1, 0xC0: 1,
0xC1: 1, 0xC2: 1, 0xC3: 1, 0xC4: 1,
0xC5: 1, 0xC6: 1, 0xC7: 1, 0xC8: 1,
0xC9: 1, 0xCA: 1, 0xCB: 1, 0xCC: 1,
0xCD: 1, 0xCE: 1, 0xCF: 1, 0xD0: 2,
0xD1: 2, 0xD2: 2, 0xD3: 2, 0xD4: 2,
0xD5: 2, 0xD6: 2, 0xD7: 2, 0xD8: 2,
0xD9: 2, 0xDA: 2, 0xDB: 2, 0xDC: 2,
0xDD: 2, 0xDE: 2, 0xDF: 2, 0xE0: 2,
0xE1: 2, 0xE2: 2, 0xFA: 4, 0xFB: 4,
0xFC: 3, 0xFD: 3, 0xFE: 2, 0xFF: 2,
}
# Opcode groups whose operands are a pair of registers plus an int16 offset.
IF_TEST = set(range(0x32, 0x38)) # 22t: 2 register nibbles + int16
IF_TESTZ = set(range(0x38, 0x3E)) # 21t: 1 register nibble + int16
RETURN_OPS = (0x0F, 0x10, 0x11)
MOVE_RESULT_OPS = (0x0A, 0x0B, 0x0C)
INVOKE_OPS = tuple(range(0x6E, 0x73)) + tuple(range(0x74, 0x79))
# opcodes that read an instance field: (dest_reg, object_reg, field_idx)
IFIELD_OPS = tuple(range(0x52, 0x59))
# opcodes that write an instance field: (value_reg, object_reg, field_idx)
PFIELD_OPS = tuple(range(0x59, 0x60))
SFIELD_OPS = tuple(range(0x60, 0x6E))
def u16(b, o):
return b[o] | (b[o + 1] << 8)
def u32(b, o):
return b[o] | (b[o + 1] << 8) | (b[o + 2] << 16) | (b[o + 3] << 24)
def s16(v):
return v - 0x10000 if v > 0x7FFF else v
def read_uleb(b, o):
"""ULEB128 -> (value, new_offset). Member indices in class_data are deltas."""
result = 0
shift = 0
while True:
x = b[o]
o += 1
result |= (x & 0x7F) << shift
if (x & 0x80) == 0:
break
shift += 7
return result, o
def insn_units(op, data, pos, end):
"""Instruction length in 16-bit code units.
Widths come from OP_UNITS, the format specification's table -- not from a
mnemonic, and not from a run of neighbours. The failure that replaces was
quiet in the worst way: a wrong width keeps producing plausible instructions,
just shifted, so every offset derived after that point is wrong and nothing in
the output says so. The traps worth naming:
* Width is not a family property. `const-wide/16` (0x16) is 2 units,
`const-wide/32` (0x17) is 3 and `const-wide` (0x18) is 5; `const-class`
(0x1C) is 2 while `monitor-exit` (0x1E) is 1; `instance-of` (0x20) is 2
while `array-length` (0x21) is 1. Nine opcodes in 0x16-0x2C break the
"same family, same width" reading.
* `0x32`-`0x3D` (if-test 22t / if-testz 21t) are **2** units, not 1.
Treating them as 1 invents a fake second instruction at every branch.
* `0x1A` (const-string, 21c) and `0x1B` (const-string/jumbo, 31c) are
different widths; only the jumbo form carries a 32-bit string index.
* The `goto` family sits at `0x28` (10t, **1** unit), `0x29` (20t, **2**)
and `0x2A` (30t, **3**) -- one slot later than the obvious reading, which
puts `throw`, `goto`, `goto/16` at 0x27/0x28/0x29.
Whenever a decode is used to derive a patch offset, assert that the walk ends
exactly on `insns_off + insns_size*2`. See `decode_all`.
"""
if op == 0x00:
# Pseudo-instructions. `00 <ident>` with ident 1..3 is a switch or
# fill-array payload; a plain `00 00` is a one-unit nop, so never treat
# every 00 as a payload.
#
# Each payload has its OWN layout, and the field after the ident is not
# the same quantity in all three:
#
# 0x0100 packed-switch-payload: ident, size(uint16), first_key(int32),
# then size * int32 targets
# -> 2 + 2 + size*4 bytes = 4 + size*2 units
# 0x0200 sparse-switch-payload: ident, size(uint16), then size pairs of
# (int32 key, int32 target)
# -> 2 + 2 + size*8 bytes = 2 + size*4 units
# 0x0300 fill-array-data-payload: ident, element_width(uint16),
# size(uint32), then the raw data padded to an even byte count
# -> 4 units + ceil(size * element_width / 2)
#
# Reading size out of the wrong slot desynchronises silently: the walk
# keeps producing plausible instructions, only shifted. Measured on a
# real AOT class initialiser, a fill-array payload of 30 four-byte
# elements (64 units) was counted as 5, which is the bug this fixes.
if pos + 8 <= end:
ident = data[pos + 1] & 0xFF
if ident == 0x01:
return 4 + u16(data, pos + 2) * 2
if ident == 0x02:
return 2 + u16(data, pos + 2) * 4
if ident == 0x03:
width = u16(data, pos + 2)
count = u32(data, pos + 4)
return 4 + (count * width + 1) // 2
return 1
# Unallocated opcodes (0x3E-0x43, 0x73, 0x79-0x7A, 0xE3-0xF9) never appear in
# a valid dex; anything unlisted is treated as one unit so a corrupt stream
# still terminates rather than running off the end.
return OP_UNITS.get(op, 1)
class Dex(object):
"""Structural reader for one dex image."""
def __init__(self, data, name="classes.dex"):
self.d = data
self.name = name
self.header = {k: u32(data, v) for k, v in OFFSETS.items()}
# ---- sanity -----------------------------------------------------------
def check(self):
"""Return a list of structural complaints; empty means it parses."""
problems = []
size = len(self.d)
if self.d[:4] != b"dex\n":
problems.append("not a dex: magic=%r" % self.d[:4])
for key in ("string_ids_off", "type_ids_off", "proto_ids_off",
"field_ids_off", "method_ids_off", "class_defs_off"):
off = self.header[key]
if not (0 < off < size):
problems.append("%s=0x%x out of range (file size %d)"
% (key, off, size))
declared = self.header["file_size"]
if declared and declared != size:
problems.append("header file_size=%d but actual=%d" % (declared, size))
return problems
# ---- index tables -----------------------------------------------------
def string(self, idx):
p = u32(self.d, self.header["string_ids_off"] + idx * 4)
_utf16_len, p = read_uleb(self.d, p)
end = self.d.index(b"\x00", p)
return self.d[p:end].decode("utf-8", "replace")
def string_safe(self, idx):
"""Like string() but never raises -- a desynced decode passes junk index.
A renderer that throws on a bad index hides the very symptom the caller is
looking for (a decode that has drifted), so return a marker instead.
"""
try:
if idx >= self.header["string_ids_size"]:
return "<string_idx %d out of range>" % idx
return self.string(idx)
except Exception:
return "<string_idx %d unreadable>" % idx
def type_(self, idx):
return self.string(u32(self.d, self.header["type_ids_off"] + idx * 4))
def proto(self, idx):
o = self.header["proto_ids_off"] + idx * 12
ret = self.type_(u32(self.d, o + 4))
poff = u32(self.d, o + 8)
params = []
if poff:
n = u32(self.d, poff)
p = poff + 4
for _ in range(n):
params.append(self.type_(u16(self.d, p)))
p += 2
return "(" + "".join(params) + ")" + ret
def field(self, idx):
o = self.header["field_ids_off"] + idx * 8
return (self.type_(u16(self.d, o)), self.string(u32(self.d, o + 4)),
self.type_(u16(self.d, o + 2)))
def method(self, idx):
o = self.header["method_ids_off"] + idx * 8
return (self.type_(u16(self.d, o)), self.string(u32(self.d, o + 4)),
self.proto(u16(self.d, o + 2)))
def find_class(self, fqcn):
"""Return the class_def_item offset for a 'Lpkg/Name;' FQCN, or None."""
for i in range(self.header["class_defs_size"]):
o = self.header["class_defs_off"] + i * 32
if self.type_(u32(self.d, o)) == fqcn:
return o
return None
def methods_of(self, fqcn):
"""Yield (section, method_idx, class, name, descriptor, code_off)."""
cd = self.find_class(fqcn)
if cd is None:
raise KeyError("class not found: %s" % fqcn)
for item in self.methods_at(cd):
yield item
def methods_at(self, class_def_off):
"""Same as methods_of(), from a class_def_item offset you already have.
methods_of() calls find_class(), which is a linear scan of class_defs, so
walking every class in a dex through it costs O(n^2). A caller that is
already iterating class_defs should come through here instead.
"""
p = u32(self.d, class_def_off + 24)
if p == 0:
return
sf, p = read_uleb(self.d, p)
inf, p = read_uleb(self.d, p)
dm, p = read_uleb(self.d, p)
vm, p = read_uleb(self.d, p)
for _ in range(sf + inf): # skip field lists
_i, p = read_uleb(self.d, p)
_a, p = read_uleb(self.d, p)
for section, count in (("direct", dm), ("virtual", vm)):
running = 0
for _ in range(count):
diff, p = read_uleb(self.d, p)
running += diff # indices are delta-encoded
_acc, p = read_uleb(self.d, p)
code_off, p = read_uleb(self.d, p)
cls, name, desc = self.method(running)
yield section, running, cls, name, desc, code_off
def find_method(self, fqcn, name, desc):
"""Return (section, method_idx, code_off) or None."""
for section, idx, _cls, nm, ds, code_off in self.methods_of(fqcn):
if nm == name and ds == desc:
return section, idx, code_off
return None
def find_methods_named(self, fqcn, name):
"""All overloads of a name -> list of (section, idx, desc, code_off)."""
out = []
for section, idx, _cls, nm, ds, code_off in self.methods_of(fqcn):
if nm == name:
out.append((section, idx, ds, code_off))
return out
def class_names(self):
for i in range(self.header["class_defs_size"]):
o = self.header["class_defs_off"] + i * 32
yield self.type_(u32(self.d, o))
# ---- code -------------------------------------------------------------
def code_info(self, code_off):
return {
"registers": u16(self.d, code_off),
"ins": u16(self.d, code_off + 2),
"outs": u16(self.d, code_off + 4),
"tries": u16(self.d, code_off + 6),
"debug_info_off": u32(self.d, code_off + 8),
"insns_size": u32(self.d, code_off + 12),
"insns_off": code_off + 16,
}
def decode(self, code_off):
"""Yield dicts describing each instruction of one method body.
Callers should assert the walk ends exactly on insns_off+insns_size*2;
landing short or long means the format table is wrong somewhere.
"""
info = self.code_info(code_off)
pos = info["insns_off"]
end = pos + info["insns_size"] * 2
while pos < end:
op = self.d[pos]
units = insn_units(op, self.d, pos, end)
if units < 1 or pos + units * 2 > end:
return
yield {
"off": pos, "op": op, "units": units,
"name": OP_NAMES.get(op, "op_%02x" % op),
"raw": bytes(self.d[pos:pos + units * 2]),
"registers": info["registers"],
}
pos += units * 2
def decode_all(self, code_off):
"""Full listing as a list, plus a verdict on whether it ended cleanly."""
insns = list(self.decode(code_off))
info = self.code_info(code_off)
expected = info["insns_off"] + info["insns_size"] * 2
ended = insns[-1]["off"] + insns[-1]["units"] * 2 if insns else info["insns_off"]
return insns, (ended == expected), expected
def insn_at(self, code_off, target_off):
for insn in self.decode(code_off):
if insn["off"] == target_off:
return insn
return None
# ---- operand helpers --------------------------------------------------
def branch_target(self, insn):
"""Absolute target of a branch, or None if the instruction is not one."""
op, pos, raw = insn["op"], insn["off"], insn["raw"]
if op in IF_TEST or op in IF_TESTZ:
return pos + s16(u16(self.d, pos + 2)) * 2
if op == 0x28: # goto (10t, signed byte)
off = raw[1]
if off > 127:
off -= 256
return pos + off * 2
if op == 0x29: # goto/16 (20t, signed int16)
return pos + s16(u16(self.d, pos + 2)) * 2
if op == 0x2A: # goto/32 (30t, signed int32)
off = u32(self.d, pos + 2)
if off > 0x7FFFFFFF:
off -= 0x100000000
return pos + off * 2
return None
def branch_regs(self, insn):
"""(regs_read, kind) for a conditional branch."""
if insn["op"] in IF_TEST:
return [insn["raw"][1] >> 4, insn["raw"][1] & 0xF], "if-test"
if insn["op"] in IF_TESTZ:
return [insn["raw"][1] & 0xF], "if-testz"
return [], None
def all_targets(self, code_off):
"""Set of every absolute branch target in one method."""
targets = set()
for insn in self.decode(code_off):
t = self.branch_target(insn)
if t is not None:
targets.add(t)
return targets
# ---- description ------------------------------------------------------
def describe(self, insn):
"""One-line human rendering with resolved field/method/string names."""
op, pos, raw = insn["op"], insn["off"], insn["raw"]
text = "0x%x: %-14s %s" % (pos, raw.hex(), insn["name"])
if op in IF_TEST:
text += " v%d,v%d -> 0x%x" % (raw[1] >> 4, raw[1] & 0xF,
self.branch_target(insn))
elif op in IF_TESTZ:
text += " v%d -> 0x%x" % (raw[1] & 0xF, self.branch_target(insn))
elif op in (0x28, 0x29, 0x2A):
text += " -> 0x%x" % self.branch_target(insn)
elif op in IFIELD_OPS:
cls, nm, ty = self.field(u16(self.d, pos + 2))
text += " v%d <- v%d.%s:%s" % (raw[1] & 0xF, raw[1] >> 4, nm, ty)
elif op in PFIELD_OPS:
cls, nm, ty = self.field(u16(self.d, pos + 2))
text += " v%d -> v%d.%s:%s" % (raw[1] & 0xF, raw[1] >> 4, nm, ty)
elif op in SFIELD_OPS:
cls, nm, ty = self.field(u16(self.d, pos + 2))
text += " %s.%s:%s" % (cls, nm, ty)
elif op in INVOKE_OPS:
cls, nm, ds = self.method(u16(self.d, pos + 2))
text += " %s.%s%s" % (cls, nm, ds)
elif op == 0x1A:
# const-string (21c): uint16 string index at byte offset 2.
text += ' "%s"' % self.string_safe(u16(self.d, pos + 2))
elif op == 0x1B:
# const-string/jumbo (31c): the index is a uint32, so reading it as a
# uint16 would silently name a different string.
text += ' "%s"' % self.string_safe(u32(self.d, pos + 2))
elif op == 0x12:
lit = raw[1] & 0xF
text += " v%d, %d" % (raw[1] >> 4, lit - 16 if lit > 7 else lit)
return text
# ---------------------------------------------------------------------------
# dex header integrity
# ---------------------------------------------------------------------------
def fix_dex_header(data):
"""Recompute a dex header's checksum and signature IN THE CORRECT ORDER.
Order is not cosmetic:
bytes 12..32 = sha1(data[32:]) -- signature first
bytes 8..12 = adler32(data[12:]) -- checksum last, covers the signature
Reversed, the adler32 is taken while the signature field is still zeroed, so
the header never verifies. Android logs
`Failure to verify dex file ...: Bad checksum (computed, expected)` -- the
real value shows up as "expected" -- and falls back to interpreting the dex,
which can surface as an unrelated ClassNotFoundException at startup.
Accepts and returns a bytearray; also returns the before/after values so a
caller can print them.
"""
before_checksum = int.from_bytes(data[8:12], "little")
before_signature = bytes(data[12:32])
data[12:32] = hashlib.sha1(bytes(data[32:])).digest()
after_signature = bytes(data[12:32])
import zlib
after_checksum = zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF
data[8:12] = after_checksum.to_bytes(4, "little")
return {
"before_checksum": before_checksum, "after_checksum": after_checksum,
"before_signature": before_signature, "after_signature": after_signature,
}
def verify_dex_header(data):
"""Return (checksum_ok, signature_ok) for a bytearray/bytes dex."""
import zlib
stored_c = int.from_bytes(data[8:12], "little")
calc_c = zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF
return stored_c == calc_c, bytes(data[12:32]) == hashlib.sha1(bytes(data[32:])).digest()
# ---------------------------------------------------------------------------
# loading
# ---------------------------------------------------------------------------
def load_dex(source, entry=None):
"""Load a Dex from a .dex path, an .apk/.zip path, or raw bytes.
For an archive, `entry` picks a member (default: the first classes*.dex).
Returns (Dex, entry_name).
"""
if isinstance(source, (bytes, bytearray)):
return Dex(bytes(source), entry or "<bytes>"), entry or "<bytes>"
path = str(source)
if path.lower().endswith((".apk", ".zip", ".jar", ".xapk", ".apks", ".apkm")):
with zipfile.ZipFile(path) as z:
if entry:
return Dex(z.read(entry), entry), entry
candidates = sorted(n for n in z.namelist()
if n.startswith("classes") and n.endswith(".dex"))
if not candidates:
raise ValueError("no classes*.dex in %s" % path)
name = candidates[0]
return Dex(z.read(name), name), name
with open(path, "rb") as fh:
return Dex(fh.read(), path), path
def main(argv):
if len(argv) != 5:
print(__doc__)
return 2
target, fqcn, name, desc = argv[1:5]
dex, entry = load_dex(target)
print("== %s (entry %s, %d bytes)" % (target, entry, len(dex.d)))
problems = dex.check()
for p in problems:
print(" [structure] %s" % p)
if problems:
print(" refusing to continue on a structurally broken read")
return 1
found = dex.find_method(fqcn, name, desc)
if not found:
print("method not found: %s->%s%s" % (fqcn, name, desc))
print("methods in class:")
for section, idx, _c, nm, ds, code_off in dex.methods_of(fqcn):
print(" [%s] %s%s code_off=0x%x" % (section, nm, ds, code_off))
return 1
section, idx, code_off = found
info = dex.code_info(code_off)
insns, clean, expected = dex.decode_all(code_off)
print("== %s.%s%s [%s] method_idx=%d" % (fqcn, name, desc, section, idx))
print(" code_off=0x%x insns_off=0x%x insns_size=%d registers=%d"
% (code_off, info["insns_off"], info["insns_size"], info["registers"]))
print(" decoded %d instructions; ended cleanly: %s" % (len(insns), clean))
if not clean:
print(" WARNING: decode did not land on 0x%x -- offsets below are suspect"
% expected)
regs = info["registers"]
over = [i for i in insns if _max_reg(i) >= regs]
if over:
print(" WARNING: %d instruction(s) reference a register >= %d; "
"desync likely" % (len(over), regs))
targets = dex.all_targets(code_off)
print(" branch targets: %s" % ", ".join(sorted("0x%x" % t for t in targets)))
print("-- listing --")
for insn in insns:
mark = " <== branch target" if insn["off"] in targets else ""
print(" " + dex.describe(insn) + mark)
return 0
def _max_reg(insn):
"""Highest register number named by an instruction (rough but sufficient)."""
raw, op = insn["raw"], insn["op"]
if op in IF_TEST or op in IFIELD_OPS or op in PFIELD_OPS:
return max(raw[1] >> 4, raw[1] & 0xF)
if op in IF_TESTZ:
return raw[1] & 0xF
if op == 0x12:
return raw[1] >> 4
if op in (0x01, 0x04, 0x07, 0x0F, 0x10, 0x11):
return raw[1] & 0xF # 12x: two 4-bit registers
if op in (0x0A, 0x0B, 0x0C, 0x0D, 0x1C, 0x1D, 0x1E, 0x1F, 0x22):
return raw[1] # 11x / 21c: one 8-bit register
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))

677
vendor-new/apk-reverse/skills/apk-reverse/scripts/doctor.py Normal file
View File

@@ -0,0 +1,677 @@
#!/usr/bin/env python3
"""Environment doctor: what can actually run here, and what is missing.
Why this exists
---------------
This skill ships a lot of scripts, and the most common failure mode reported by
users is not "the script is wrong" but "the script would not start and I could not
tell why" - a missing jar, a Python module that does not exist on this interpreter,
a device that is not connected. That wastes a whole round before any reverse
engineering happens.
Run this once before a work block. It answers three questions:
1. Which capabilities are available right now (static / native / dynamic / device)?
2. For every script, is it runnable as-is, or what exactly is missing?
3. Are there environment facts that will silently poison experiments
(clock skew, a leftover proxy, a dead device server, a wrong-ABI device)?
It never installs anything and never touches a target. Read the report, then go.
What changed, and why the report got worse
------------------------------------------
This script used to answer question 1 from a hand-written table of 28 scripts (the
directory held 51) and from a formula that read
caps['repack + sign'] = bool(tools['java']['path']) and py['ok']
caps['smali round-trip'] = bool(tools['java']['path']) and py['ok']
Neither ability follows from `java`: signing needs a signer and `zipalign`, a smali
round-trip needs eight jars. On a host with only a JDK both reported OK, and every gate
downstream inherited an optimism nobody had measured. A capability verdict that is
wrong in the permissive direction is worse than no verdict at all, because it is used
to decide whether a claim is supportable.
So now:
* the script list is **scanned from this directory**, so it cannot go stale, and the
printed `registered/checked = N / N` counts every `.py` and `.js` file here;
* each script's third-party dependencies come from a **static AST pass** over its
imports, checked against `sys.stdlib_module_names` -- no script is imported or run
to find out what it needs;
* a script this pass cannot classify is printed as `unknown` and counted, never
dropped from the denominator;
* capability verdicts come from `capabilities.py`, the single registry, which probes
every atom of a capability's real closure (a missing input artifact included) and
prints an executable next step with a cost estimate.
A report that says BLOCKED on a host which genuinely cannot sign is the point. Fix the
named atom, or route the work through a capability that is ok.
Usage
-----
python doctor.py # capability + script summary
python doctor.py --scripts # per-script runnability table
python doctor.py --capabilities # every capability, with next actions
python doctor.py --device <serial> # include device checks
python doctor.py --json # machine-readable
Exit codes: 0 = nothing blocked, 3 = at least one capability blocked, 2 = usage error,
4 = internal error. The last line is `RESULT=<token>`.
"""
import argparse
import ast
import json
import os
import platform
import re
import subprocess
import sys
import time
HERE = os.path.dirname(os.path.abspath(__file__))
SKILL_DIR = os.path.dirname(HERE)
REPO_ROOT = os.path.dirname(os.path.dirname(SKILL_DIR))
if HERE not in sys.path:
sys.path.insert(0, HERE)
import capabilities as caps # noqa: E402 (path is set above on purpose)
EXIT_OK = 0
EXIT_USE = 2
EXIT_ENV = 3
EXIT_INTERNAL = 4
TOKENS = {
'ok': 'env_ok',
'partial': 'env_partial',
'blocked': 'env_blocked',
'use': 'usage_error',
'internal': 'internal_error',
}
_PROBE = caps.Probe()
# Host executables worth recognising inside a script's source. A literal outside this set
# is treated as data (an adb subcommand, a shell builtin, a path), never as a host tool:
# guessing too eagerly would report a missing tool that was never needed.
KNOWN_HOST_TOOLS = {
'java', 'javac', 'keytool', 'jarsigner', 'zipalign', 'apksigner', 'adb', 'aapt',
'aapt2', 'd8', 'dx', 'dexdump', 'frida', 'frida-ps', 'objection',
'node', 'npm', 'unzip', 'zip', '7z', 'git', 'sqlite3', 'tshark', 'tcpdump',
'mitmdump', 'rizin', 'rabin2', 'gdb', 'readelf', 'objdump', 'nm', 'strings',
'clang', 'ndk-build', 'cmake', 'make', 'python', 'python3', 'blutter', 'jadx',
'apktool', 'dex2jar', 'd2j-dex2jar',
}
# ---------------------------------------------------------------- probing
def extra_tool_dirs():
"""Directories to search beyond PATH (delegated to the capability registry)."""
return caps.extra_tool_dirs()
def which(name):
return _PROBE.which(name)
def run(cmd, timeout=15):
"""Run a command, returning (rc, stdout+stderr). Never raises, always bounded."""
try:
p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
timeout=timeout)
return p.returncode, p.stdout.decode('utf-8', 'replace').strip()
except FileNotFoundError:
return 127, 'not found'
except subprocess.TimeoutExpired:
return 124, 'TIMEOUT after %ss' % timeout
except Exception as e: # pragma: no cover
return 1, '%s: %s' % (type(e).__name__, e)
def have_module(name):
"""Existence only. Importing a probe target can start threads and touch devices."""
return _PROBE.module(name)
# ---------------------------------------------------------------- tools
def tool_info():
"""name -> {path, version, note}. Version probes are bounded and failure is data."""
out = {}
probes = [
('java', ['java', '-version']),
('javac', ['javac', '-version']),
('keytool', ['keytool', '-help']),
('jarsigner', ['jarsigner', '-help']),
('adb', ['adb', 'version']),
('apksigner', ['apksigner', '--version']),
('zipalign', ['zipalign']),
('dexdump', ['dexdump']),
('frida', ['frida', '--version']),
('frida-ps', ['frida-ps', '--version']),
('objection', ['objection', '--version']),
('apktool', ['apktool', '--version']),
('jadx', ['jadx', '--version']),
('python', [sys.executable, '--version']),
('unzip', ['unzip', '-v']),
('git', ['git', '--version']),
('sqlite3', ['sqlite3', '--version']),
('node', ['node', '--version']),
('tshark', ['tshark', '-v']),
('mitmdump', ['mitmdump', '--version']),
('rizin', ['rizin', '-v']),
('rabin2', ['rabin2', '-v']),
('gdb', ['gdb', '--version']),
('readelf', ['readelf', '--version']),
('adb-devices', ['adb', 'devices']),
]
for name, cmd in probes:
if cmd[0] != sys.executable and which(cmd[0]) is None:
# A tool that ships as a runnable .jar is not on PATH but is still usable via
# `java -jar`. Reporting it missing would make the report lie downward.
jar_hit = _PROBE.find_jar(name)
if jar_hit:
out[name] = {'path': jar_hit,
'version': 'runnable as: java -jar %s'
% os.path.basename(jar_hit),
'note': 'jar-only (not a PATH command)'}
else:
out[name] = {'path': None, 'version': None, 'note': 'not on PATH'}
continue
rc, txt = run(cmd)
first = ''
for line in txt.splitlines():
line = line.strip()
if line:
first = line
break
out[name] = {
'path': which(cmd[0]) or cmd[0],
'version': first[:160],
'note': '' if rc == 0 else 'exit %d: %s' % (rc, txt[:120]),
}
return out
def java_toolchain():
"""Find the jars a smali round-trip or a signer needs, wherever they live."""
found = {}
for d in _PROBE.jar_dirs():
for dirpath, _dirs, files in os.walk(d):
if dirpath.count(os.sep) - d.count(os.sep) > 3:
continue
for f in files:
if f.lower().endswith('.jar'):
found.setdefault(f, os.path.join(dirpath, f))
return found
# ---------------------------------------------------------------- static script analysis
def _const_strings(tree):
"""Module- and function-level `NAME = 'literal'` bindings, best effort."""
consts = {}
for node in ast.walk(tree):
if isinstance(node, ast.Assign) and len(node.targets) == 1:
tgt, val = node.targets[0], node.value
if isinstance(tgt, ast.Name) and isinstance(val, ast.Constant) \
and isinstance(val.value, str):
consts.setdefault(tgt.id, val.value)
return consts
def _as_name(node, consts):
if isinstance(node, ast.Constant) and isinstance(node.value, str):
return node.value
if isinstance(node, ast.Name):
return consts.get(node.id)
return None
def _first_list_elem(call_node, consts):
"""`subprocess.run(['adb', ...])`: the first element of the first list argument."""
for arg in getattr(call_node, 'args', []):
if isinstance(arg, ast.List) and arg.elts:
return _as_name(arg.elts[0], consts)
return None
def static_python_deps(path):
"""Third-party modules and host tools a .py file needs, from its source alone.
Returns (modules, tools, error). `error` is set when the file could not be parsed,
which the caller must surface as `unknown` rather than as "no dependencies".
"""
try:
with open(path, encoding='utf-8', errors='replace') as fh:
src = fh.read()
tree = ast.parse(src, path)
except (OSError, SyntaxError) as exc:
return [], [], '%s: %s' % (type(exc).__name__, exc)
stdlib = set(sys.stdlib_module_names)
local = {f[:-3] for f in os.listdir(HERE) if f.endswith('.py')}
modules, tools = set(), set()
consts = _const_strings(tree)
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for a in node.names:
modules.add(a.name.split('.')[0])
elif isinstance(node, ast.ImportFrom):
if node.level == 0 and node.module:
modules.add(node.module.split('.')[0])
elif isinstance(node, ast.Call):
fn = node.func
attr = fn.attr if isinstance(fn, ast.Attribute) else (
fn.id if isinstance(fn, ast.Name) else '')
# A dependency is a tool the script *resolves in order to use*, or the
# program of a subprocess. A bare `shutil.which('x')` is only a probe -- the
# script usually handles the negative branch itself -- and counting it as a
# dependency reports doctor.py as needing frida-server, which is a device
# binary it merely looks for.
if attr in ('resolve_tool', 'find_executable'):
if node.args:
name = _as_name(node.args[0], consts)
if name in KNOWN_HOST_TOOLS:
tools.add(name)
elif attr in ('run', 'check_output', 'check_call', 'Popen', 'call', 'spawn',
'execvp', 'execv'):
name = _first_list_elem(node, consts)
if name in KNOWN_HOST_TOOLS:
tools.add(name)
third = sorted(m for m in modules
if m not in stdlib and m not in local and not m.startswith('_'))
return third, sorted(tools), None
def static_js_deps(path):
"""Modules a Frida JS script `require()`s. Frida's require is not a Python import,
so these are recorded as declarations, not resolved against the Python environment."""
try:
with open(path, encoding='utf-8', errors='replace') as fh:
src = fh.read()
except OSError as exc:
return [], 'OSError: %s' % exc
# strip line comments cheaply; a require() inside a string is not worth the parser
body = '\n'.join(row.split('//')[0] for row in src.splitlines())
return sorted(set(re.findall(r"""require\(\s*['"]([^'"]+)['"]\s*\)""", body))), None
def discover_scripts():
"""Every `.py` and `.js` in this directory. The list is the directory, not a table."""
names = []
for f in sorted(os.listdir(HERE)):
if f.endswith(('.py', '.js')) and os.path.isfile(os.path.join(HERE, f)):
names.append(f)
return names
def scan_scripts(cap_results):
"""Per-script facts: static deps, declared capability, and a verdict.
The verdict is the conjunction of the two independent facts about the script --
whether its own dependencies resolve, and whether the capability it serves is
available -- so a script is never reported runnable merely because it exists.
"""
rows = []
for name in discover_scripts():
path = os.path.join(HERE, name)
kind = 'js' if name.endswith('.js') else 'py'
cap_ids, cap_source = caps.capabilities_for_script(name, path)
row = {
'script': name, 'kind': kind, 'present': os.path.isfile(path),
'capabilities': cap_ids if cap_ids else [],
'capability_source': cap_source,
'modules': [], 'missing_modules': [], 'tools': [], 'missing_tools': [],
'js_requires': [], 'error': None, 'deps_source': '', 'verdict': 'ok',
}
if kind == 'py':
mods, tools, err = static_python_deps(path)
row['deps_source'] = 'ast-import-scan'
row['error'] = err
row['modules'] = mods
row['tools'] = tools
row['missing_modules'] = [m for m in mods if not have_module(m)]
row['missing_tools'] = [t for t in tools if which(t) is None]
else:
reqs, err = static_js_deps(path)
row['deps_source'] = 'js-require-scan'
row['error'] = err
row['js_requires'] = reqs
cap_states = [cap_results[c]['status'] for c in row['capabilities']
if c in cap_results]
missing_modules = row['missing_modules']
missing_tools = row['missing_tools']
cap_unknown = cap_ids is None
# A script's verdict is the conjunction of two independent facts: whether its own
# dependencies resolve, and whether the capability it serves is available.
#
# A missing *module* is fatal to the process (the import is at module level), so
# it blocks. A missing *tool* is not always fatal: repack.py resolves apksigner /
# keytool / zipalign only on the signing path, so on a host with no build-tools it
# is still fully usable with --no-sign, and calling that BLOCKED would be a
# pessimism of its own. When at least one capability the script serves is not
# blocked, a missing tool downgrades the script to PARTIAL and names the tool.
if cap_unknown or row['error']:
row['verdict'] = 'unknown'
elif cap_states and all(s == 'blocked' for s in cap_states):
row['verdict'] = 'blocked'
elif missing_modules:
row['verdict'] = 'blocked'
elif missing_tools and not cap_states:
row['verdict'] = 'blocked'
elif missing_tools or 'partial' in cap_states:
row['verdict'] = 'partial'
else:
row['verdict'] = 'ok'
row['runnable'] = row['verdict'] in ('ok', 'partial')
rows.append(row)
return rows
# ---------------------------------------------------------------- device
def device_report(serial=None):
if which('adb') is None:
return {'available': False, 'reason': 'adb not on PATH',
'devices': [], 'root': False, 'device_frida_processes': []}
cmd = ['adb'] + (['-s', serial] if serial else []) + ['devices', '-l']
rc, txt = run(cmd, timeout=20)
if rc != 0:
return {'available': False, 'reason': txt[:200], 'devices': [],
'root': False, 'device_frida_processes': []}
devices = []
for line in txt.splitlines()[1:]:
line = line.strip()
if not line or line.startswith('*'):
continue
parts = line.split()
if len(parts) >= 2:
devices.append({'serial': parts[0], 'state': parts[1],
'info': ' '.join(parts[2:])[:160]})
out = {'available': True, 'devices': devices, 'raw': txt[:400], 'root': False,
'device_frida_processes': []}
if not devices:
out['reason'] = 'no device/emulator attached'
return out
tgt = serial or devices[0]['serial']
props = {}
for key in ('ro.product.cpu.abi', 'ro.product.cpu.abilist', 'ro.build.version.release',
'ro.build.version.sdk', 'ro.product.model'):
rc, v = run(['adb', '-s', tgt, 'shell', 'getprop', key], timeout=15)
props[key] = v if rc == 0 else ''
out['target'] = tgt
out['props'] = props
# root?
rc, v = run(['adb', '-s', tgt, 'shell', 'su -c id'], timeout=15)
out['root'] = (rc == 0 and 'uid=0' in v)
out['root_raw'] = v[:120]
# clock skew: a classic silent experiment-poisoner
rc, dev_epoch = run(['adb', '-s', tgt, 'shell', 'date +%s'], timeout=15)
try:
dev_t = int(dev_epoch.strip())
out['clock_skew_s'] = abs(time.time() - dev_t)
except Exception:
out['clock_skew_s'] = None
# leftover forwards and proxies
rc, fwd = run(['adb', '-s', tgt, 'forward', '--list'], timeout=15)
out['forwards'] = [row for row in fwd.splitlines() if row.strip()][:10]
rc, prox = run(['adb', '-s', tgt, 'shell', 'settings get global http_proxy'], timeout=15)
out['http_proxy'] = prox.strip()[:120]
# is a frida server already running on device? (a common source of "the app
# suddenly detects instrumentation" while you believe nothing is attached)
rc, ps = run(['adb', '-s', tgt, 'shell', 'ps -A'], timeout=20)
hits = [row for row in ps.splitlines()
if 'frida' in row.lower() and 'grep' not in row.lower()]
out['device_frida_processes'] = [h.strip()[:140] for h in hits[:6]]
return out
# ---------------------------------------------------------------- reporting
def capability_section(results, verbose=True):
lines = caps.human_report(results, verbose=verbose)
blocked = [c for c, r in results.items() if r['status'] == 'blocked']
partial = [c for c, r in results.items() if r['status'] == 'partial']
lines.append('')
lines.append(' ok=%d partial=%d blocked=%d (of %d capabilities)'
% (len(results) - len(blocked) - len(partial), len(partial),
len(blocked), len(results)))
return lines
class _Parser(argparse.ArgumentParser):
"""argparse exits 2 on a usage error. Keep the RESULT= contract on that path too:
a caller that branches on the last line must not have to special-case bad usage."""
def error(self, message):
self.print_usage(sys.stderr)
sys.stderr.write('%s: error: %s\n' % (self.prog, message))
print('RESULT=%s' % TOKENS['use'])
raise SystemExit(EXIT_USE)
def main():
ap = _Parser(
description='Check what this environment can actually do. Exit 0 = nothing '
'blocked, 3 = a capability is blocked, 2 = usage, 4 = internal.')
ap.add_argument('--device', default=None, metavar='SERIAL', help='probe this device')
ap.add_argument('--scripts', action='store_true', help='per-script runnability table')
ap.add_argument('--capabilities', action='store_true',
help='every capability with its missing atoms and next actions '
'(the default prints the same list compactly)')
ap.add_argument('--json', action='store_true', dest='as_json', help='emit JSON')
args = ap.parse_args()
try:
py = {'version': '%d.%d.%d' % sys.version_info[:3],
'ok': sys.version_info[:2] >= (3, 9),
'note': '' if sys.version_info[:2] >= (3, 9)
else 'scripts target 3.9+; older interpreters may fail on syntax'}
dev = device_report(args.device)
# Hand the device facts to the registry so it does not re-probe the device.
_PROBE.set_device({
'available': bool(dev.get('devices')),
'devices': [d['serial'] for d in dev.get('devices', [])],
'root': bool(dev.get('root')),
'frida_server': bool(dev.get('device_frida_processes')),
'reason': dev.get('reason', ''),
})
cap_results = caps.resolve_all(_PROBE)
tools = tool_info()
jars = java_toolchain()
rows = scan_scripts(cap_results)
except Exception as exc: # pragma: no cover - defensive
sys.stderr.write('internal error: %s: %s\n' % (type(exc).__name__, exc))
print('RESULT=%s' % TOKENS['internal'])
return EXIT_INTERNAL
status, code, token = caps.verdict(cap_results)
counts = {'ok': 0, 'partial': 0, 'blocked': 0, 'unknown': 0}
for r in rows:
counts[r['verdict']] = counts.get(r['verdict'], 0) + 1
unknown = [r['script'] for r in rows if r['verdict'] == 'unknown']
total = len(rows)
py_files = sum(1 for r in rows if r['kind'] == 'py')
js_files = total - py_files
warnings = []
for cid, r in cap_results.items():
for m in r['partial']:
warnings.append('%s: reduced -- %s' % (cid, m['atom']))
if unknown:
warnings.append('dependency verdict unknown for %d script(s): %s'
% (len(unknown), ', '.join(unknown)))
for r in rows:
for m in r['missing_modules']:
warnings.append('%s: missing python module %s' % (r['script'], m))
for t in r['missing_tools']:
warnings.append('%s: missing tool %s' % (r['script'], t))
# The next action is the first *blocked* capability's, not the first partial one's:
# a reduced capability is a caveat, a blocked one is what stops the work.
hints = [cap_results[c]['next_action'] for c in caps.CAPABILITIES
if c in cap_results and cap_results[c]['status'] == 'blocked'
and cap_results[c]['next_action']]
if args.as_json:
payload = {
'status': status,
'exit_code': code,
'capability': None,
'next_action': hints[0] if hints else '',
'warnings': warnings,
'evidence': [
{'kind': 'capability', 'capability': cid, 'atom': e['atom'],
'ok': e['ok'], 'detail': e['detail'], 'source': e['source']}
for cid, r in cap_results.items() for e in r['evidence']
] + [
{'kind': 'script', 'script': r['script'],
'detail': 'verdict=%s deps=%s capabilities=%s'
% (r['verdict'], r['deps_source'],
','.join(r['capabilities']) or 'unknown')}
for r in rows
],
# the pre-existing keys are kept so an existing consumer keeps working
'python': py,
'tools': tools,
'jars': jars,
'device': dev,
'capabilities': cap_results,
'capabilities_legacy': {c: r['status'] for c, r in cap_results.items()},
'scripts': rows,
'script_counts': {'registered': total, 'checked': total,
'py': py_files, 'js': js_files,
'ok': counts['ok'], 'partial': counts['partial'],
'blocked': counts['blocked'], 'unknown': counts['unknown']},
}
print(json.dumps(payload, indent=2, ensure_ascii=False))
# No `RESULT=` line on this path: appending one makes the stream invalid JSON for a caller
# doing `--json | jq` or `json.loads(stdout)`, which is what `--json` is for. The status is a
# field in the document, and the exit code carries the same meaning. (Same defect that was
# found in check_commands.py, and it was found here by trying to parse this output.)
return code
print('=' * 74)
print('apk-reverse environment doctor')
print('=' * 74)
print('platform : %s %s / %s' % (platform.system(), platform.release(),
platform.machine()))
print('python : %s %s' % (py['version'], '' if py['ok'] else '<-- ' + py['note']))
print('\n--- capabilities (%d) ---' % len(cap_results))
for line in capability_section(cap_results, verbose=args.capabilities):
print(line)
print('\n--- tools ---')
for name, info in tools.items():
if name in ('adb-devices',):
continue
mark = 'OK ' if info['path'] else '-- '
ver = info['version'] or info['note']
print(' [%s] %-12s %s' % (mark, name, (ver or '')[:96]))
print('\n--- java jars found near this skill ---')
if jars:
for k in sorted(jars):
print(' %s' % k)
else:
print(' (none) - a smali round-trip needs the baksmali/smali/dexlib2 jar set.')
print(' smtool.py reads --cp, then APK_REVERSE_SMALI_CP, then scripts/smali_cp.txt.')
print(' doctor.py itself looks under APKREV_JARS and this skill directory.')
print('\n--- device ---')
if not dev.get('available'):
print(' unavailable: %s' % dev.get('reason'))
elif not dev.get('devices'):
print(' no device attached (adb works)')
else:
print(' target : %s' % dev.get('target'))
for k, v in (dev.get('props') or {}).items():
print(' %-9s: %s' % (k.replace('ro.', ''), v))
print(' root : %s' % ('yes' if dev.get('root') else 'no'))
skew = dev.get('clock_skew_s')
if skew is not None:
flag = ' <-- FIX THIS, clock drift poisons time-based checks' if skew > 30 else ''
print(' clock skew: %ss%s' % (skew, flag))
if dev.get('forwards'):
print(' leftovers : adb forward entries still present: %s' % dev['forwards'])
if dev.get('http_proxy', '').strip() not in ('', 'null', ':0'):
print(' leftovers : device http_proxy = %s' % dev['http_proxy'])
if dev.get('device_frida_processes'):
print(' WARNING : a frida process is already running on device:')
for p in dev['device_frida_processes']:
print(' %s' % p)
print(' If the target dies only while this is up, you are looking at')
print(' a probe aimed at YOU. Stop it before concluding anything.')
print('\n--- scripts ---')
print(' directory: %s' % HERE)
print(' registered/checked = %d / %d (.py=%d, .js=%d)'
% (total, total, py_files, js_files))
print(' ok=%d partial=%d blocked=%d unknown=%d'
% (counts['ok'], counts['partial'], counts['blocked'], counts['unknown']))
if args.scripts:
for r in rows:
mark = {'ok': 'OK ', 'partial': 'PARTIAL', 'blocked': 'BLOCKED',
'unknown': 'UNKNOWN'}[r['verdict']]
why = []
if not r['present']:
why.append('file missing')
if r['error']:
why.append('unparsed: %s' % r['error'][:60])
if r['missing_modules']:
why.append('missing modules: ' + ', '.join(r['missing_modules']))
if r['missing_tools']:
why.append('missing tools: ' + ', '.join(r['missing_tools']))
if r['kind'] == 'js':
why.append('frida JS (requires scanned: %s)'
% (', '.join(r['js_requires']) or 'none'))
if r['capability_source'] == 'mapped' and not r['capabilities']:
why.append('no capability mapped')
if r['capability_source'] == 'self':
why.append('reports on this environment rather than using it')
caps_txt = ','.join(r['capabilities']) if r['capabilities'] else (
'self' if r['capability_source'] == 'self' else 'unknown')
print(' [%s] %-30s %-28s %s'
% (mark, r['script'], caps_txt, '; '.join(why)))
if counts['unknown']:
print('\n UNKNOWN means this pass could not decide the dependency set. It is '
'counted above,\n not skipped: read the file before trusting it.')
else:
print(' use --scripts for the per-script table')
if counts['blocked']:
blocked_names = [r['script'] for r in rows if r['verdict'] == 'blocked']
print(' blocked scripts (%d): %s'
% (len(blocked_names), ', '.join(blocked_names[:8])
+ (' ...' if len(blocked_names) > 8 else '')))
if hints:
print('\n--- next action ---')
print(' %s' % hints[0])
print('\nRead references/long-task-discipline.md before a long block:')
print(' bound every wait, look at the screen while you wait, and record the '
'time-to-death before patching.')
print('RESULT=%s' % token)
return code
if __name__ == '__main__':
sys.exit(main())

456
vendor-new/apk-reverse/skills/apk-reverse/scripts/elf_plt.py Normal file
View File

@@ -0,0 +1,456 @@
#!/usr/bin/env python3
"""PLT stub -> imported symbol mapping, and byte-level .so diffing.
Built for hardened/rebuilt native libraries where the usual paths lie:
* section headers are often forged, so tools that walk sections (readelf -x,
objdump -d on named sections, pyelftools' section API) return confidently
wrong or empty results;
* a linear disassembler can stop silently on a buffer that does not start at
an instruction boundary, so "no matches" from it is not evidence.
Therefore this tool walks PT_LOAD / PT_DYNAMIC by hand and decodes the two
stub shapes with its own decoders (no capstone dependency, no silent stops).
Why you need the stub -> symbol map: when a hardening library terminates the
process, it does so through a PLT stub. To suppress it safely you must know
*which* symbol a given stub resolves to -- inferring from position or from a
comment is how you end up freezing `snprintf` or `pthread_exit` and reporting
it as "the patch did not work".
Usage
-----
# full stub table
python elf_plt.py libfoo.so
# what do these stubs call?
python elf_plt.py libfoo.so --query 0x9920 0x9950
# find the stub for one symbol (and every caller of it)
python elf_plt.py libfoo.so --symbol kill --callers
# what changed between the original and a rebuilt library?
python elf_plt.py original.so --diff rebuilt.so
# same, but also name the symbol each changed stub belongs to
python elf_plt.py original.so --diff rebuilt.so --name-regions
"""
import argparse
import hashlib
import struct
import sys
PT_LOAD = 1
PT_DYNAMIC = 2
DT_PLTRELSZ = 2
DT_PLTGOT = 3
DT_STRTAB = 5
DT_SYMTAB = 6
DT_RELA = 7
DT_RELASZ = 8
DT_STRSZ = 10
DT_SYMENT = 11
DT_PLTREL = 20
DT_JMPREL = 23
EM_X86_64 = 62
EM_AARCH64 = 183
ARCH_NAME = {EM_X86_64: "x86_64", EM_AARCH64: "aarch64"}
# --------------------------------------------------------------------------
# ELF parsing (program headers only -- never trust the section table here)
# --------------------------------------------------------------------------
class Elf:
def __init__(self, data, path=""):
self.data = data
self.path = path
if data[:4] != b"\x7fELF":
raise ValueError("not an ELF file")
if data[4] != 2:
raise ValueError("only 64-bit ELF is supported")
self.machine = struct.unpack_from("<H", data, 18)[0]
self.phoff = struct.unpack_from("<Q", data, 32)[0]
self.phentsize = struct.unpack_from("<H", data, 54)[0]
self.phnum = struct.unpack_from("<H", data, 56)[0]
self.loads = [] # (vaddr, offset, filesz, memsz, flags)
self.dynamic = None # (offset, size)
self.gnu_eh_frame = None
for i in range(self.phnum):
off = self.phoff + i * self.phentsize
p_type, p_flags = struct.unpack_from("<II", data, off)
p_offset, p_vaddr, _pa, p_filesz, p_memsz, _al = struct.unpack_from(
"<QQQQQQ", data, off + 8)
if p_type == PT_LOAD:
self.loads.append((p_vaddr, p_offset, p_filesz, p_memsz, p_flags))
elif p_type == PT_DYNAMIC:
self.dynamic = (p_offset, p_filesz)
elif p_type == 0x6474e550:
self.gnu_eh_frame = (p_vaddr, p_filesz)
@property
def arch(self):
return ARCH_NAME.get(self.machine, "machine=%d" % self.machine)
def exec_segment(self):
for seg in self.loads:
if seg[4] & 1:
return seg
raise ValueError("no executable PT_LOAD segment")
def vaddr_to_off(self, vaddr):
for va, fo, _fs, ms, _fl in self.loads:
if va <= vaddr < va + ms:
return fo + (vaddr - va)
return None
def dynamic_tags(self):
if self.dynamic is None:
raise ValueError("no PT_DYNAMIC segment")
tags = {}
off, size = self.dynamic
for j in range(size // 16):
t, v = struct.unpack_from("<qQ", self.data, off + j * 16)
if t == 0:
break
tags[t] = v
return tags
def reloc_map(elf):
"""{GOT vaddr: imported symbol name} from DT_JMPREL (+ DT_RELA when present)."""
tags = elf.dynamic_tags()
if DT_STRTAB not in tags or DT_SYMTAB not in tags:
return {}
strtab_o = elf.vaddr_to_off(tags[DT_STRTAB])
symtab_o = elf.vaddr_to_off(tags[DT_SYMTAB])
strsz = tags.get(DT_STRSZ, 0)
if strtab_o is None or symtab_o is None:
return {}
def symname(idx):
so = symtab_o + idx * 24
if so + 4 > len(elf.data):
return "?"
(st_name,) = struct.unpack_from("<I", elf.data, so)
if st_name >= strsz:
return "?"
start = strtab_o + st_name
end = elf.data.find(b"\x00", start)
if end < 0:
return "?"
return elf.data[start:end].decode("utf-8", "replace")
out = {}
for tag_off, tag_sz in ((DT_JMPREL, DT_PLTRELSZ), (DT_RELA, DT_RELASZ)):
if tag_off not in tags:
continue
base = elf.vaddr_to_off(tags[tag_off])
if base is None:
continue
n = tags.get(tag_sz, 0) // 24
for k in range(n):
if base + k * 24 + 16 > len(elf.data):
break
r_offset, r_info = struct.unpack_from("<QQ", elf.data, base + k * 24)
out[r_offset] = symname(r_info >> 32)
return out
# --------------------------------------------------------------------------
# Stub decoders -- written out rather than delegated, so nothing stops early
# --------------------------------------------------------------------------
def find_stubs(elf, relocs):
"""{stub_vaddr: (symbol, got_vaddr)} for the library's stub table."""
base, foff, fsz, _ms, _fl = elf.exec_segment()
if elf.machine == EM_X86_64:
return _stubs_x86(elf.data, base, foff, fsz, relocs)
if elf.machine == EM_AARCH64:
return _stubs_arm64(elf.data, base, foff, fsz, relocs)
return {}
def _stubs_x86(data, base, foff, fsz, relocs):
"""x86_64 PLT stub: `ff 25 <disp32>` == jmp qword ptr [rip+disp32]."""
out = {}
i = 0
while i < fsz - 6:
if data[foff + i] == 0xFF and data[foff + i + 1] == 0x25:
disp = struct.unpack_from("<i", data, foff + i + 2)[0]
va = base + i
target = va + 6 + disp
if target in relocs:
out[va] = (relocs[target], target)
i += 1
return out
def _stubs_arm64(data, base, foff, fsz, relocs):
"""aarch64 PLT stub: adrp x16 / ldr x17,[x16,#off] / add x16,x16,#off / br x17.
Note the stub is FOUR instructions (16 bytes), not four bytes. Assuming the
shorter form leads to "borrowing" the next slot, which belongs to a
different symbol.
"""
out = {}
off = 0
while off < fsz - 16:
w0, w1, w2, w3 = struct.unpack_from("<IIII", data, foff + off)
dec = _decode_adrp(w0, base + off)
if dec is not None:
page, rd0 = dec
ldr = _decode_ldr_unsigned(w1)
add = _decode_add_imm(w2)
br = _decode_br(w3)
if (ldr is not None and add is not None and br is not None
and rd0 == 16 and ldr[0] == 16 and ldr[1] == 17
and add[0] == 16 and add[1] == 16
and ldr[2] == add[2] and br == 17):
got = page + add[2]
if got in relocs:
out[base + off] = (relocs[got], got)
off += 4
return out
def _sign_extend(value, bits):
if value & (1 << (bits - 1)):
value -= (1 << bits)
return value
def _decode_adrp(word, pc):
"""adrp Rd, #imm -> (target_page, Rd). None if not an adrp."""
if (word >> 31) & 1 != 1:
return None
if ((word >> 24) & 0x1F) != 0x10:
return None
immlo = (word >> 29) & 0x3
immhi = (word >> 5) & 0x7FFFF
imm = (immhi << 2) | immlo
imm = _sign_extend(imm, 21)
return (pc & ~0xFFF) + (imm << 12), (word & 0x1F)
def _decode_ldr_unsigned(word):
"""ldr Rt, [Rn, #imm] (64-bit unsigned offset) -> (Rn, Rt, byte_offset)."""
if (word >> 30) != 0b11: # size == 64-bit
return None
if ((word >> 27) & 0x7) != 0b111:
return None
if ((word >> 24) & 0x3) != 0b01:
return None
imm12 = (word >> 10) & 0xFFF
rn = (word >> 5) & 0x1F
rt = word & 0x1F
return rn, rt, imm12 * 8
def _decode_add_imm(word):
"""add Rd, Rn, #imm -> (Rn, Rd, imm).
Encoding: sf op S 100010 sh imm12 Rn Rd
bits[31]=sf bits[30]=op bits[29]=S bits[28:23]=100010 (6 bits, not 9)
"""
if (word >> 31) & 1 != 1: # sf must be 1 (64-bit form)
return None
if ((word >> 23) & 0x3F) != 0b100010:
return None
if (word >> 22) & 1: # shifted form -- not what stubs use
return None
imm12 = (word >> 10) & 0xFFF
rn = (word >> 5) & 0x1F
rd = word & 0x1F
return rn, rd, imm12
def _decode_br(word):
"""br Rn -> Rn, or None."""
if (word & 0xFFFFFC1F) == 0xD61F0000:
return (word >> 5) & 0x1F
return None
def find_branch_callers(elf, target_vaddr, limit=500):
"""Addresses of BL/B instructions that branch to target_vaddr."""
base, foff, fsz, _ms, _fl = elf.exec_segment()
hits = []
if elf.machine == EM_AARCH64:
off = 0
while off < fsz - 4:
(w,) = struct.unpack_from("<I", data_slice(elf.data, foff + off, 4))
if (w & 0xFC000000) in (0x94000000, 0x14000000): # BL / B
imm = _sign_extend(w & 0x03FFFFFF, 26) << 2
if base + off + imm == target_vaddr:
hits.append(base + off)
if len(hits) >= limit:
break
off += 4
elif elf.machine == EM_X86_64:
i = 0
while i < fsz - 5:
if elf.data[foff + i] == 0xE8:
disp = struct.unpack_from("<i", elf.data, foff + i + 1)[0]
if base + i + 5 + disp == target_vaddr:
hits.append(base + i)
if len(hits) >= limit:
break
i += 1
return hits
def data_slice(data, off, n):
return data[off:off + n]
# --------------------------------------------------------------------------
# Commands
# --------------------------------------------------------------------------
def load(path):
with open(path, "rb") as fh:
return Elf(fh.read(), path)
def cmd_list(elf, args):
relocs = reloc_map(elf)
stubs = find_stubs(elf, relocs)
base, foff, fsz, _ms, _fl = elf.exec_segment()
print("== %s" % elf.path)
print(" arch=%s exec seg: vaddr=0x%x file=0x%x size=0x%x" % (elf.arch, base, foff, fsz))
print(" imported relocations=%d stubs resolved=%d" % (len(relocs), len(stubs)))
if not stubs:
print(" NOTE: zero stubs resolved. Either the library has no PLT, or the")
print(" hardening removed/forged the relocation tables. Do not read")
print(" this as 'the library imports nothing'.")
for va in sorted(stubs):
sym, got = stubs[va]
print(" 0x%08x -> %-34s (GOT 0x%x)" % (va, sym, got))
return stubs
def cmd_query(elf, args):
relocs = reloc_map(elf)
stubs = find_stubs(elf, relocs)
for a in args.query:
va = int(a, 0)
hit = stubs.get(va)
if hit:
print("0x%x -> %s (GOT 0x%x)" % (va, hit[0], hit[1]))
else:
print("0x%x -> not a resolved stub" % va)
def cmd_symbol(elf, args):
relocs = reloc_map(elf)
stubs = find_stubs(elf, relocs)
wanted = args.symbol
found = [(va, sym, got) for va, (sym, got) in stubs.items() if sym == wanted]
if not found:
near = sorted({s for _v, (s, _g) in stubs.items()
if wanted in s or s in wanted})[:10]
print("no stub resolves to %r" % wanted)
if near:
print(" similar: %s" % ", ".join(near))
return
for va, sym, got in sorted(found):
print("stub 0x%x -> %s (GOT 0x%x)" % (va, sym, got))
if args.callers:
callers = find_branch_callers(elf, va)
print(" callers of the stub: %d" % len(callers))
for c in callers:
print(" 0x%x" % c)
print(" NOTE: a stub-level patch covers all of these. A call-site-level")
print(" patch must cover every one of them, plus any caller reached")
print(" through a cached function pointer (not listed here).")
def cmd_diff(elf_a, args):
path_b = args.diff
elf_b = load(path_b)
a, b = elf_a.data, elf_b.data
print("== diff")
print(" A %s %d bytes sha256=%s" % (elf_a.path, len(a), hashlib.sha256(a).hexdigest()[:16]))
print(" B %s %d bytes sha256=%s" % (path_b, len(b), hashlib.sha256(b).hexdigest()[:16]))
if len(a) != len(b):
print(" SIZE DIFFERS: %d vs %d -- a length change is itself a finding"
% (len(a), len(b)))
n = min(len(a), len(b))
runs = []
i = 0
while i < n:
if a[i] != b[i]:
j = i
while j < n and a[j] != b[j]:
j += 1
runs.append((i, j - i))
i = j
else:
i += 1
total = sum(r[1] for r in runs)
print(" differing bytes=%d in %d run(s)" % (total, len(runs)))
regions = {}
if args.name_regions:
relocs_a = reloc_map(elf_a)
stubs_a = find_stubs(elf_a, relocs_a)
def describe(off):
for va, (sym, _got) in stubs_a.items():
size = 16 if elf_a.machine == EM_AARCH64 else 16
if va <= off < va + size:
return "PLT stub for %s" % sym
return ""
regions = {"describe": describe}
for off, ln in runs:
note = ""
if regions:
note = regions["describe"](off)
if note:
note = " <- " + note
print(" off=0x%06x len=%-4d A: %s" % (off, ln, a[off:off + ln].hex(" ")))
print(" %s B: %s%s" % (" " * 14, b[off:off + ln].hex(" "), note))
if args.name_regions and not runs:
print(" (identical)")
def main():
ap = argparse.ArgumentParser(
description="PLT stub -> symbol mapping and .so diffing for hardened libraries.",
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("so", help="path to the ELF (.so)")
ap.add_argument("--query", nargs="+", metavar="ADDR",
help="resolve specific stub addresses (hex ok)")
ap.add_argument("--symbol", metavar="NAME",
help="find the stub that resolves to this symbol")
ap.add_argument("--callers", action="store_true",
help="with --symbol: also list branch call sites")
ap.add_argument("--diff", metavar="OTHER_SO",
help="byte-compare against another library")
ap.add_argument("--name-regions", action="store_true",
help="with --diff: name the symbol each changed stub belongs to")
args = ap.parse_args()
try:
elf = load(args.so)
except Exception as e:
sys.exit("error: %s" % e)
if args.diff:
cmd_diff(elf, args)
elif args.query:
cmd_query(elf, args)
elif args.symbol:
cmd_symbol(elf, args)
else:
cmd_list(elf, args)
if __name__ == "__main__":
main()

286
vendor-new/apk-reverse/skills/apk-reverse/scripts/find_refs.py Normal file
View File

@@ -0,0 +1,286 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Find every reference to a method/field/class, so you can judge blast radius
BEFORE patching it.
This is the check that prevents the classic mistake: patching something that looked
ad-specific but is actually a general utility with dozens of callers.
Usage
-----
# a smali tree, a single .dex, a directory holding either, or an .apk
python find_refs.py <smali_tree|dex|dir|apk> 'Lcom/pkg/Helper;->showAd(Landroid/app/Activity;)V'
# a whole class (all of its members)
python find_refs.py <smali_tree|dex|dir|apk> 'Lcom/pkg/Helper;'
# a field
python find_refs.py <smali_tree|dex|dir|apk> 'Lcom/pkg/Helper;->count:I'
# just the counts, no listing
python find_refs.py <smali_tree|dex|dir|apk> 'Lcom/pkg/Helper;->showAd' --count-only
Reading the result
------------------
few callers, all inside one feature area -> safe to patch
many callers, or callers in unrelated pkgs -> general utility. DO NOT patch it.
Go one level up and patch the
specific caller instead.
The first line is part of the answer, not decoration. `[scanned] 0 file(s)` means
the input was never read -- an unsupported path, a typo, a directory with nothing
this script can open -- and it is a different result from "read N files and found
nothing". The two used to print identically, which put a false zero on exactly the
decision this script exists to protect. Input that cannot be read now exits 2 and
says why.
Input forms, precisely:
.smali file text scan (no decoder needed)
.dex file decoded with dexutil (no baksmali/jadx required)
directory every .smali and .dex found under it, recursively
.apk/.zip/... every classes*.dex inside the archive
anything else refused with an error, rather than reported as "no references"
Also inspect the signature. If it mentions Modifier / ContentScale / Shape / View /
ColorScheme or a content-generic model type, it is shared UI, not your target.
"""
import argparse
import os
import re
import sys
import zipfile
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from dexutil import ( # noqa: E402
IFIELD_OPS, INVOKE_OPS, PFIELD_OPS, SFIELD_OPS, insn_units, load_dex, u16,
)
TEXT_EXT = ('.smali',)
DEX_EXT = ('.dex',)
ARCHIVE_EXT = ('.apk', '.zip', '.jar', '.xapk', '.apks', '.apkm')
REF_RE_TMPL = r'invoke[^\n]*%s|sget[^\n]*%s|sput[^\n]*%s|new-instance[^\n]*%s|check-cast[^\n]*%s'
FIELD_OPS = tuple(IFIELD_OPS) + tuple(PFIELD_OPS) + tuple(SFIELD_OPS)
# new-instance (0x22) and check-cast (0x1F) both carry a type index at byte 2,
# which is where the 21c format puts it.
TYPE_OPS = (0x1F, 0x22)
# classes.dex, classes2.dex, ... in an archive.
ARCHIVE_DEX_RE = re.compile(r'(?:^|/)classes\d*\.dex$')
EXIT_FOUND = 0
EXIT_NO_MATCH = 1
EXIT_BAD_INPUT = 2
def gather(root):
"""Return (smali_paths, dex_paths, archive_paths).
Raises ValueError for input this script cannot read, so that "nothing was
scanned" can never be mistaken for "nothing references it".
"""
if os.path.isdir(root):
smali, dexs = [], []
for dp, _dirs, files in os.walk(root):
for fn in sorted(files):
path = os.path.join(dp, fn)
if fn.endswith(TEXT_EXT):
smali.append(path)
elif fn.endswith(DEX_EXT):
dexs.append(path)
if not smali and not dexs:
raise ValueError(
'no .smali and no .dex files under %s.\n'
' A directory of .apk archives is not scanned; pass the '
'.apk itself, or disassemble first (%s).'
% (root, 'smtool.py'))
return smali, dexs, []
if not os.path.exists(root):
raise ValueError('no such file or directory: %s' % root)
low = root.lower()
if low.endswith(TEXT_EXT):
return [root], [], []
if low.endswith(DEX_EXT):
return [], [root], []
if low.endswith(ARCHIVE_EXT):
return [], [], [root]
raise ValueError(
'%s is not a .smali file, a .dex, a directory, or an archive.\n'
' Accepted: .smali, .dex, a directory holding either, '
'%s' % (root, '/'.join(ARCHIVE_EXT)))
def method_owner(text, idx):
"""Map a character offset to the enclosing .method declaration."""
head = text.rfind('.method ', 0, idx)
if head < 0:
return '?'
end = text.find('\n', head)
return text[head:end].strip()
def scan_smali(path, pat):
"""Return [(offset, owner, matched_text)] for one smali file."""
try:
with open(path, encoding='utf-8', errors='replace') as fh:
text = fh.read()
except OSError:
return []
return [(m.start(), method_owner(text, m.start()), m.group(0).strip())
for m in pat.finditer(text)]
def walk_insns(dex, code_off):
"""Yield (offset, opcode) for one method body.
Deliberately not dex.decode(): this walks every instruction of every method
in the dex, and the per-instruction dict decode() builds is not needed to
read an operand index out of the raw bytes.
"""
info = dex.code_info(code_off)
pos = info['insns_off']
end = pos + info['insns_size'] * 2
while pos < end:
op = dex.d[pos]
units = insn_units(op, dex.d, pos, end)
if units < 1 or pos + units * 2 > end:
return
yield pos, op
pos += units * 2
def scan_dex(dex, needle):
"""Return [(insn_off, owner, rendered)] for references inside one dex.
The rendered form is the same string the smali side matches textually
(`Lcom/pkg/Helper;->showAd(Landroid/app/Activity;)V`), so both input forms
answer the same question the same way.
"""
out = []
for i in range(dex.header['class_defs_size']):
cd_off = dex.header['class_defs_off'] + i * 32
for _section, _midx, cls, name, desc, code_off in dex.methods_at(cd_off):
if not code_off:
continue
owner = '%s->%s%s' % (cls, name, desc)
for pos, op in walk_insns(dex, code_off):
if op in INVOKE_OPS:
cls2, nm, ds = dex.method(u16(dex.d, pos + 2))
rendered = '%s->%s%s' % (cls2, nm, ds)
elif op in FIELD_OPS:
cls2, nm, ty = dex.field(u16(dex.d, pos + 2))
rendered = '%s->%s:%s' % (cls2, nm, ty)
elif op in TYPE_OPS:
rendered = dex.type_(u16(dex.d, pos + 2))
else:
continue
if needle in rendered:
out.append((pos, owner, rendered))
return out
def dex_sources(path):
"""Return [(label, Dex)] for a .dex file or every classes*.dex in an archive."""
if path.lower().endswith(DEX_EXT):
dex, _entry = load_dex(path)
return [(path, dex)]
with zipfile.ZipFile(path) as z:
names = sorted(n for n in z.namelist() if ARCHIVE_DEX_RE.search(n))
if not names:
raise ValueError('no classes*.dex inside %s' % path)
out = []
for name in names:
dex, _entry = load_dex(path, entry=name)
out.append(('%s!%s' % (path, name), dex))
return out
def main():
ap = argparse.ArgumentParser(
description='Count and list references to a method/field/class, in a smali '
'tree, a .dex, a directory of either, or an .apk.')
ap.add_argument('root', help='.smali file, .dex file, directory, or archive')
ap.add_argument('needle', help='e.g. Lcom/pkg/Helper;->showAd(...)V, '
'Lcom/pkg/Helper;, or Lcom/pkg/Helper;->count:I')
ap.add_argument('--count-only', action='store_true')
ap.add_argument('--max-print', type=int, default=60)
a = ap.parse_args()
esc = re.escape(a.needle)
# match the needle plus one more char so we do not match a longer prefix
pat = re.compile(REF_RE_TMPL % (esc, esc, esc, esc, esc))
try:
smali_paths, dex_paths, archives = gather(a.root)
except ValueError as exc:
print('[error] %s' % exc)
return EXIT_BAD_INPUT
per_file = [] # (count, label, hits)
scanned = 0
unreadable = []
for path in smali_paths:
hits = scan_smali(path, pat)
scanned += 1
if hits:
per_file.append((len(hits), path, hits))
for path in dex_paths + archives:
try:
sources = dex_sources(path)
except (ValueError, OSError, zipfile.BadZipFile) as exc:
unreadable.append('%s (%s)' % (path, exc))
continue
for label, dex in sources:
problems = dex.check()
if problems:
unreadable.append('%s (%s)' % (label, problems[0]))
continue
hits = scan_dex(dex, a.needle)
scanned += 1
if hits:
per_file.append((len(hits), label, hits))
if scanned == 0:
print('[error] nothing readable was scanned under %s' % a.root)
for u in unreadable:
print(' unreadable: %s' % u)
print(' Input was refused rather than reported as zero references, '
'because the two are not the same answer.')
return EXIT_BAD_INPUT
total = sum(c for c, _l, _h in per_file)
per_file.sort(key=lambda row: row[0], reverse=True)
print('[scanned] %d file(s) (%d smali, %d dex)'
% (scanned, len(smali_paths), scanned - len(smali_paths)))
print('[total refs] %d across %d files' % (total, len(per_file)))
for u in unreadable:
print('[unreadable] %s' % u)
print()
printed = 0
for cnt, label, hits in per_file:
print('%-70s %d' % (label, cnt))
if a.count_only:
continue
for off, owner, rendered in hits:
if printed >= a.max_print:
print(' ... (truncated)')
return EXIT_FOUND
print(' in %s' % owner)
print(' %s' % rendered)
printed += 1
if total == 0:
print('[note] scanned %d file(s) and found no reference to %r. Check the '
'descriptor prefix "L...;", the exact obfuscated name, and whether the '
'behaviour lives in another dex or in native code.'
% (scanned, a.needle))
return EXIT_NO_MATCH
return EXIT_FOUND
if __name__ == '__main__':
sys.exit(main())

572
vendor-new/apk-reverse/skills/apk-reverse/scripts/frida_probe.js Normal file
View File

@@ -0,0 +1,572 @@
/*
* frida_probe.js -- four-layer full-chain network probe for a hardened Android app.
*
* WHY THIS SHAPE
* --------------
* When you cannot tell why a request never reaches the server (or why it fails
* silently), a single-layer hook always leaves you guessing. This script hooks
* FOUR independent layers at once, so *whichever* layer the app actually uses
* produces evidence:
*
* [APP] the app's own network wrapper class(es) -> what business method asked
* [OKHTTP] OkHttp full chain -> newCall/build/execute/async/proceed
* [RAW-URL] java.net.URL.openConnection -> HttpsURLConnection, which uses the
* SYSTEM trust store and therefore
* fails differently from OkHttp
* [THROW] java.lang.Throwable.getMessage -> recovers the exception text that an
* upper layer swallowed with catch {}
*
* The RAW-URL vs OKHTTP distinction is the single most valuable signal here: an app
* commonly has two unrelated trust chains, so "login fails but browsing works" is
* explained by which of the two fired. See LESSONS G3.
*
* Every hook is installed inside its own try/catch. One missing class (a different
* OkHttp major version, a renamed obfuscated class) never disables the other three
* layers. Failures are reported as HOOK-FAIL lines instead of killing the script.
*
* ADAPT TO YOUR TARGET (only this block matters)
* ---------------------------------------------
* 1) APP_NET_CLASSES -> replace with the app's own network wrapper class name(s)
* 2) APP_PKG_PREFIX -> the app's package prefix, used only to filter noise
* 3) ENABLE_* flags -> turn layers off if they are too noisy
* Everything else is target-independent.
*
* HOW TO RUN
* ----------
* Prefer the bundled injector, which also writes a log file and stays resident:
* python run_probe.py frida_probe.js 10 --pkg com.example.app
* Or by hand against a remote device:
* frida -H 127.0.0.1:27042 -p <pid> -l frida_probe.js
*
* OUTPUT FORMAT
* -------------
* Every line is <ISO-ish local timestamp> <TAG> <key=value ...>
* TAGS: READY | HOOK-FAIL | FATAL | APP | OKHTTP-NEWCALL | OKHTTP-BUILD |
* OKHTTP-EXEC | OKHTTP-ASYNC | OKHTTP-PROCEED | RAW-URL | DNS | THROW
* Add STACK to a line only where a call chain is genuinely useful (APP, RAW-URL),
* because it is the expensive part.
*/
/* ===================================================================
* 1. ADAPT THESE LINES
* =================================================================== */
// The app's own network wrapper / API helper class(es). Enumerate ALL declared
// methods at runtime, so you do not have to know the obfuscated method names.
// Example: 'com.example.app.net.HttpHelper', 'com.example.app.api.ApiClient'
var APP_NET_CLASSES = ['com.example.app.net.HttpHelper'];
// Used only to filter THROW noise: messages from classes whose name contains this
// prefix are always kept. Leave as-is if you do not care.
var APP_PKG_PREFIX = 'com.example.app';
// Layer switches. Turn a layer off if it produces more noise than signal.
var ENABLE_APP = true; // [APP] app's own wrapper classes (needs APP_NET_CLASSES)
var ENABLE_OKHTTP = true; // [OKHTTP] OkHttp newCall/build/execute/AsyncCall/proceed
var ENABLE_RAW_URL = true; // [RAW-URL] java.net.URL.openConnection (HttpsURLConnection)
var ENABLE_THROWABLE = true; // [THROW] Throwable.getMessage, deduplicated
var ENABLE_DNS = true; // [DNS] InetAddress.getAllByName, one line per unique host
var STACK_ON_APP = true; // attach a stack to [APP] hits
var STACK_ON_RAW_URL = true; // attach a stack to [RAW-URL] hits
var STACK_LINES = 12; // how many stack frames to keep
// Safety valves: this probe must never become the thing that kills the app.
var MAX_STRING = 3000; // truncate any single value
var THROWABLE_MAX_LINES = 400;// hard cap on THROW lines
var THROWABLE_STACK = false; // stacks on every Throwable line are usually too much
/* ===================================================================
* 2. Plumbing (target-independent)
* =================================================================== */
function pad(n, w) {
var s = '' + n;
while (s.length < w) s = '0' + s;
return s;
}
// Local wall-clock with milliseconds. Local, not UTC, because you will be
// comparing these lines against logcat and your own shell history.
function stamp() {
var d = new Date();
return d.getFullYear() + '-' + pad(d.getMonth() + 1, 2) + '-' + pad(d.getDate(), 2) + ' ' +
pad(d.getHours(), 2) + ':' + pad(d.getMinutes(), 2) + ':' + pad(d.getSeconds(), 2) + '.' +
pad(d.getMilliseconds(), 3);
}
function str(v, max) {
if (v === null) return 'null';
if (v === undefined) return 'undefined';
var s;
try { s = '' + v; } catch (e) { s = '<unprintable>'; }
max = max || MAX_STRING;
if (s.length > max) s = s.substring(0, max) + '...<+' + (s.length - max) + ' chars>';
return s;
}
function sendLine(tag, fields) {
var parts = [];
if (fields) {
for (var k in fields) {
if (!fields.hasOwnProperty(k)) continue;
if (fields[k] === null || fields[k] === undefined || fields[k] === '') continue;
parts.push(k + '=' + str(fields[k]));
}
}
var line = { t: stamp(), tag: tag, msg: parts.join(' ') };
try { send(line); } catch (e) { /* transport gone; nothing sane to do */ }
}
// Call-chain capture. Expensive, so it is opt-in per hook.
function stackText(lines) {
try {
var Log = Java.use('android.util.Log');
var Throwable = Java.use('java.lang.Throwable');
var full = Log.getStackTraceString(Throwable.$new());
var rows = ('' + full).split('\n');
var kept = [];
// Drop the probe's own frames at the top; they are never the interesting part.
for (var i = 0; i < rows.length && kept.length < (lines || STACK_LINES); i++) {
if (rows[i].indexOf('frida') >= 0) continue;
if (rows[i].indexOf('java.lang.Throwable') >= 0) continue;
if (rows[i].indexOf('android.util.Log') >= 0) continue;
kept.push(rows[i].replace(/^\s+/, ''));
}
return kept.join(' <- ');
} catch (e) {
return '<stack unavailable: ' + e + '>';
}
}
// The whole point: a failure to install one hook must not affect the others.
function guard(label, fn) {
try {
fn();
return true;
} catch (e) {
sendLine('HOOK-FAIL', { where: label, err: e });
return false;
}
}
// Run one whole layer and return how many hooks it really installed.
// > 0 that many hooks are live
// = 0 the layer installed nothing (its classes were not found); NOT a success
// =-1 the layer is switched off in the config header
// Returning a count instead of a boolean matters: a layer whose classes are absent
// still returns normally (each inner hook catches its own error), so a boolean
// "did not throw" would report a layer as installed when it hooked nothing at all.
function runLayer(label, fn) {
try {
return fn();
} catch (e) {
sendLine('HOOK-FAIL', { where: 'install ' + label, err: e });
return 0;
}
}
/* ===================================================================
* 3. Layer [APP]: the app's own network wrapper
* =================================================================== */
// Why getDeclaredMethods: obfuscated wrappers routinely expose the same name with
// several overloads, and a decompiler's short names (a, b, c) are not reliable
// enough to enumerate by hand. Enumerate at runtime, wrap each overload, and let
// the stack tell you which one the business code called.
// Build a replacement of a fixed arity that reports the hit and then delegates.
//
// Two deliberate choices here, both about not breaking the app:
// * The original is NOT cached in a variable. Frida does not hand out the pre-hook
// implementation through `overload.implementation` (it is undefined until you
// replace it), so `var orig = ov.implementation; ... orig.apply(...)` would
// throw a TypeError on the first real call and propagate straight into the app's
// network path. Calling the same name on `this` from inside the replacement is
// the documented way to reach the original, and it re-resolves the overload from
// the runtime argument types.
// * The delegate call is written out per arity instead of spread/apply, so no
// engine-specific behaviour is involved. Methods with more than 8 parameters are
// reported and left alone rather than bound incorrectly.
function appReplacement(className, methodName, arity) {
var label = className + '.' + methodName;
var note = function (argc) {
sendLine('APP', {
call: label,
argc: argc,
stack: STACK_ON_APP ? stackText() : null
});
};
switch (arity) {
case 0: return function () {
note(arguments.length); return this[methodName]();
};
case 1: return function (a0) {
note(arguments.length); return this[methodName](a0);
};
case 2: return function (a0, a1) {
note(arguments.length); return this[methodName](a0, a1);
};
case 3: return function (a0, a1, a2) {
note(arguments.length); return this[methodName](a0, a1, a2);
};
case 4: return function (a0, a1, a2, a3) {
note(arguments.length); return this[methodName](a0, a1, a2, a3);
};
case 5: return function (a0, a1, a2, a3, a4) {
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4);
};
case 6: return function (a0, a1, a2, a3, a4, a5) {
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4, a5);
};
case 7: return function (a0, a1, a2, a3, a4, a5, a6) {
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4, a5, a6);
};
case 8: return function (a0, a1, a2, a3, a4, a5, a6, a7) {
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4, a5, a6, a7);
};
default: return null;
}
}
function installAppLayer() {
if (!ENABLE_APP) return -1;
if (!APP_NET_CLASSES.length) {
sendLine('HOOK-FAIL', { where: 'APP', err: 'APP_NET_CLASSES is empty; edit the header' });
return 0;
}
var wrappedTotal = 0;
APP_NET_CLASSES.forEach(function (className) {
guard('APP ' + className, function () {
var Cls = Java.use(className);
var declared = Cls.class.getDeclaredMethods();
var total = declared.length;
var wrapped = 0;
var skipped = 0;
for (var i = 0; i < total; i++) {
(function (idx) {
var methodName;
try { methodName = '' + declared[idx].getName(); } catch (e) { return; }
guard('APP ' + className + '.' + methodName, function () {
var overloads = Cls[methodName];
if (!overloads || !overloads.overloads) return;
overloads.overloads.forEach(function (ov) {
var argTypes;
try { argTypes = ov.argumentTypes; } catch (e) { argTypes = null; }
var arity = (argTypes && typeof argTypes.length === 'number') ? argTypes.length : -1;
var label = className + '.' + methodName + '(' + (argTypes ? argTypes.join(',') : '?') + ')';
guard('APP bind ' + label, function () {
var replacement = appReplacement(className, methodName, arity);
if (replacement === null) {
skipped++;
sendLine('APP', {
call: label,
skipped: 'arity ' + arity + ' exceeds 8; left unwrapped on purpose'
});
return;
}
ov.implementation = replacement;
wrapped++;
});
});
});
})(i);
}
wrappedTotal += wrapped;
sendLine('APP', {
call: className,
declared_methods: total,
overloads_wrapped: wrapped,
overloads_skipped: skipped
});
});
});
return wrappedTotal;
}
/* ===================================================================
* 4. Layer [OKHTTP]: the whole OkHttp chain
* =================================================================== */
// Five independent anchors on purpose. Any single one can be missing depending on
// the OkHttp major version and on R8 inlining, and each still answers a different
// question:
// newCall -> a request object reached OkHttp (earliest reliable URL)
// build -> the URL existed even if the call was never executed
// execute -> synchronous call actually ran, with status code and duration
// AsyncCall.run-> asynchronous call actually ran (enqueue path)
// proceed -> every actual network step, including redirects and retries
function installOkHttpLayer() {
if (!ENABLE_OKHTTP) return -1;
var installed = 0;
var can = function (label, fn) { if (guard(label, fn)) installed++; };
can('OKHTTP newCall', function () {
var Client = Java.use('okhttp3.OkHttpClient');
Client.newCall.overload('okhttp3.Request').implementation = function (req) {
var url = '<unknown>';
try { url = '' + req.url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
sendLine('OKHTTP-NEWCALL', { url: url });
return this.newCall(req);
};
});
can('OKHTTP Request$Builder.build', function () {
var Builder = Java.use('okhttp3.Request$Builder');
Builder.build.implementation = function () {
var req = this.build();
var url = '<unknown>';
try { url = '' + req.url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
sendLine('OKHTTP-BUILD', { url: url });
return req;
};
});
can('OKHTTP RealCall.execute', function () {
var RealCall = Java.use('okhttp3.internal.connection.RealCall');
RealCall.execute.implementation = function () {
var url = '<unknown>';
try { url = '' + this.request().url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
var t0 = Date.now();
var resp = this.execute();
var code = '?';
try { code = '' + resp.code(); } catch (e) { code = '<code error>'; }
sendLine('OKHTTP-EXEC', { url: url, ms: Date.now() - t0, code: code });
return resp;
};
});
can('OKHTTP RealCall$AsyncCall.run', function () {
var AsyncCall = Java.use('okhttp3.internal.connection.RealCall$AsyncCall');
AsyncCall.run.implementation = function () {
// The URL is not reliably reachable from here across OkHttp versions; the
// newCall/build hooks above supply it. This hook proves *when* the async
// path actually started and finished, which is what matters for timing.
var t0 = Date.now();
this.run();
sendLine('OKHTTP-ASYNC', { phase: 'done', ms: Date.now() - t0 });
};
});
can('OKHTTP RealInterceptorChain.proceed', function () {
var Chain = Java.use('okhttp3.internal.http.RealInterceptorChain');
Chain.proceed.overload('okhttp3.Request').implementation = function (req) {
var url = '<unknown>';
try { url = '' + req.url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
var t0 = Date.now();
var resp = this.proceed(req);
var code = '?';
try { code = '' + resp.code(); } catch (e) { code = '<code error>'; }
sendLine('OKHTTP-PROCEED', { url: url, ms: Date.now() - t0, code: code });
return resp;
};
});
return installed;
}
/* ===================================================================
* 5. Layer [RAW-URL]: HttpsURLConnection -> the SYSTEM trust store
* =================================================================== */
// Many apps route login/registration through java.net.URL.openConnection instead
// of OkHttp. That path validates against the system trust store, so an expired or
// untrusted server certificate breaks it while the OkHttp path keeps working.
// If this layer fires but the OkHttp layer stays silent, you are on the other
// trust chain -- check the certificate out-of-band with tls_check.py.
function installRawUrlLayer() {
if (!ENABLE_RAW_URL) return -1;
var URL;
try {
URL = Java.use('java.net.URL');
} catch (e) {
sendLine('HOOK-FAIL', { where: 'RAW-URL java.net.URL', err: e });
return 0;
}
var installed = 0;
var can = function (label, fn) { if (guard(label, fn)) installed++; };
can('RAW-URL openConnection()', function () {
// Report AFTER the original returns, so we can also name the concrete connection
// class (HttpURLConnection vs HttpsURLConnection) -- that distinction is what
// tells you which trust chain the request is about to use.
// As in the APP layer, the original is reached via `this.openConnection(...)`
// rather than a cached function reference.
URL.openConnection.overload().implementation = function () {
var url = '<unknown>';
try { url = '' + this.toString(); } catch (e) { url = '<url error: ' + e + '>'; }
var conn = this.openConnection();
var connCls = '?';
try { connCls = '' + conn.getClass().getName(); } catch (e) { connCls = '<class error>'; }
sendLine('RAW-URL', {
url: url,
kind: 'no-arg',
conn: connCls,
stack: STACK_ON_RAW_URL ? stackText() : null
});
return conn;
};
});
can('RAW-URL openConnection(Proxy)', function () {
URL.openConnection.overload('java.net.Proxy').implementation = function (proxy) {
var url = '<unknown>';
try { url = '' + this.toString(); } catch (e) { url = '<url error: ' + e + '>'; }
var conn = this.openConnection(proxy);
var connCls = '?';
try { connCls = '' + conn.getClass().getName(); } catch (e) { connCls = '<class error>'; }
sendLine('RAW-URL', {
url: url,
kind: 'proxy',
conn: connCls,
stack: STACK_ON_RAW_URL ? stackText() : null
});
return conn;
};
});
return installed;
}
/* ===================================================================
* 6. Layer [THROW]: recover swallowed exception text
* =================================================================== */
// An upper-layer `catch (Exception e) { }` hides exactly the message you need.
// Hooking Throwable.getMessage is the cheapest way to get it back, but it is a
// very hot method, so: only concrete Exception/Error subclasses, deduplicated by
// class+message, hard-capped, and stacks off by default.
function installThrowableLayer() {
if (!ENABLE_THROWABLE) return -1;
// Returns 1 only when the hook is really bound to the runtime.
return guard('THROWABLE getMessage', function () {
var seen = {};
var printed = 0;
var suppressed = 0;
var Throwable = Java.use('java.lang.Throwable');
Throwable.getMessage.implementation = function () {
var msg;
try {
msg = this.getMessage();
} catch (e) {
return null;
}
try {
var cls = '' + this.getClass().getName();
var looksLikeError = (cls.indexOf('Exception') >= 0 || cls.indexOf('Error') >= 0);
var isAppClass = (cls.indexOf(APP_PKG_PREFIX) === 0);
if (looksLikeError || isAppClass) {
var key = cls + '|' + msg;
if (seen[key]) {
seen[key]++;
} else if (printed < THROWABLE_MAX_LINES) {
seen[key] = 1;
printed++;
sendLine('THROW', {
cls: cls,
msg: msg,
stack: THROWABLE_STACK ? stackText() : null
});
} else {
suppressed++;
if (suppressed === 1 || suppressed % 200 === 0) {
sendLine('THROW', { note: 'cap reached, further unique messages suppressed', printed: printed });
}
}
}
} catch (e) {
// Never let the reporting path throw: this hook is on a very hot path.
}
return msg;
};
}) ? 1 : 0;
}
/* ===================================================================
* 7. Optional layer [DNS]: which hosts the app really resolves
* =================================================================== */
// Cheap, high-signal: if an SDK's domains are never resolved, that subsystem never
// started -- much stronger evidence than "logcat was quiet". Deduplicated per host
// so a retry loop cannot flood the log.
function installDnsLayer() {
if (!ENABLE_DNS) return -1;
return guard('DNS getAllByName', function () {
var seen = {};
var Inet = Java.use('java.net.InetAddress');
Inet.getAllByName.overload('java.lang.String').implementation = function (host) {
var first = false;
try {
var key = '' + host;
if (!seen[key]) { seen[key] = 1; first = true; }
} catch (e) { /* ignore */ }
if (first) sendLine('DNS', { host: host });
return this.getAllByName(host);
};
}) ? 1 : 0;
}
/* ===================================================================
* 8. Boot
* =================================================================== */
var JAVA_BRIDGE_HINT =
'align the host frida package and the on-device frida-server to the same 16.x version; ' +
'17.x can fail to locate the Android dynamic linker and removes the built-in Java bridge';
function main() {
if (typeof Java === 'undefined' || Java === null) {
// frida 17+ removed the built-in Java bridge.
sendLine('FATAL', {
err: 'Java bridge is not available in this frida runtime',
fix: JAVA_BRIDGE_HINT
});
return;
}
// Java.perform can fail BEFORE the callback ever runs: on a non-Android host, and
// on frida 17.x, where it throws "Java API not available". Without this catch the
// whole script dies as an opaque `type: error` message with a stack pointing into
// frida-java-bridge, which tells you nothing about the version mismatch that
// actually caused it.
try {
Java.perform(function () {
var installed = [];
var failed = [];
var disabled = [];
[
['APP', installAppLayer],
['OKHTTP', installOkHttpLayer],
['RAW-URL', installRawUrlLayer],
['THROW', installThrowableLayer],
['DNS', installDnsLayer]
].forEach(function (pair) {
var count = runLayer(pair[0], pair[1]);
if (count === -1) disabled.push(pair[0]);
else if (count > 0) installed.push(pair[0] + ':' + count);
else failed.push(pair[0]);
});
sendLine('READY', {
installed: installed.join(',') || 'none',
failed: failed.join(',') || 'none',
disabled: disabled.join(',') || 'none',
note: 'hooks are live; now drive the app UI'
});
if (failed.length) {
sendLine('READY-HINT', {
failed: failed.join(','),
fix: 'a layer reporting 0 hooks means its classes were not found: the app may '
+ 'not use that library, R8 may have renamed the class, or the classloader '
+ 'differs. Layers that installed keep working -- debug the failed ones by '
+ 'their HOOK-FAIL line, not the working ones.'
});
}
});
} catch (e) {
sendLine('FATAL', {
err: 'Java.perform failed: ' + e,
fix: JAVA_BRIDGE_HINT
});
}
}
// Last-resort net: any unexpected top-level error still leaves a readable line in
// the log instead of only a raw frida script error.
try {
main();
} catch (e) {
sendLine('FATAL', { err: 'probe aborted: ' + e, note: 'no hooks were installed' });
}

424
vendor-new/apk-reverse/skills/apk-reverse/scripts/frida_rpc_serve.py Normal file
View File

@@ -0,0 +1,424 @@
#!/usr/bin/env python3
"""Bridge a Frida script's rpc.exports to local Python, a REPL, or a tiny HTTP endpoint.
Given a device, a target process and a Frida script that defines `rpc.exports`,
this tool keeps a long-lived session alive (with automatic reconnect) and lets
you invoke those exports three ways:
--mode call one shot: --export add --args '[1, 2]' -> prints the result, exits
--mode repl interactive: `list`, `name [json-args]`, `reload`, `quit`
--mode http POST /call {"export": "add", "args": [1, 2]} -> JSON result;
GET /exports lists them; POST /reload re-creates the script
The typical use is running a hardened target's own crypto (the function the
shell or SDK uses to sign/encrypt) as if it were a local function: spawn or
attach, keep the session resident, and call it from automation instead of from
a hand-driven `frida -l` session. See references/emulation-and-rpc.md for when
this beats emulating the .so offline (unidbg) and when it does not.
Reconnect policy: a detached session (app restart, server death, ROM killing
the server — see references/dynamic-frida.md) marks the bridge dead; the next
call re-attaches (re-spawning first if --spawn was given), reloads the script
and retries, with exponential backoff up to --retries times.
Device selection follows the skill's rule of explicitness: --device-serial
picks that exact USB device; --remote HOST:PORT attaches through an explicit
`adb forward` (preferred when several devices/emulators are attached); with
neither, frida's default USB device is used.
Examples:
python frida_rpc_serve.py --device-serial SSBY... --package com.example.app \
--script rpc_template.js --mode call --export add --args '[1, 2]'
python frida_rpc_serve.py --remote 127.0.0.1:27042 --package com.example.app \
--script rpc_template.js --mode repl
python frida_rpc_serve.py --device-serial SSBY... --spawn --package com.example.app \
--script rpc_template.js --mode http --port 8765
"""
import argparse
import http.server
import json
import sys
import threading
import time
try:
import frida
except ImportError: # pragma: no cover
sys.stderr.write("frida is required: pip install frida (host package must match the device server)\n")
sys.exit(2)
# Device-side failures worth retrying: a target that is still starting, a server
# that was reaped, a process that vanished. frida.TimedOutError is *not* a
# TransportError subclass (measured on frida 16.7.19: attaching to a busy app
# raised TimedOutError and escaped every handler), so collect them by name.
RETRYABLE = tuple(
getattr(frida, name)
for name in ("TransportError", "NotSupportedError", "ServerNotRunningError",
"ProcessNotFoundError", "InvalidOperationError", "TimedOutError")
if hasattr(frida, name)
)
class BridgeError(Exception):
pass
class RpcBridge:
"""Owns device -> session -> script and re-establishes them when detached."""
def __init__(self, opts):
self.opts = opts
self.device = None
self.session = None
self.script = None
self.exports = None
self.spawned_pid = None
self.alive = False
self.attempt = 0
self._lock = threading.Lock()
# -- connection lifecycle ------------------------------------------------
def _get_device(self):
if self.opts.remote:
return frida.get_device_manager().add_remote_device(self.opts.remote)
if self.opts.device_serial:
return frida.get_device(self.opts.device_serial)
return frida.get_usb_device()
def _attach_or_spawn(self, device):
if self.opts.spawn:
if not self.opts.package:
raise BridgeError("--spawn requires --package")
pid = device.spawn([self.opts.package])
self.spawned_pid = pid
return device.attach(pid)
if self.opts.pid is not None:
return device.attach(self.opts.pid)
if self.opts.package:
# attach-by-name can fail while the process list is incomplete
# (references/dynamic-frida.md); the caller may fall back to --pid.
return device.attach(self.opts.package)
raise BridgeError("need one of --package / --pid")
def connect(self):
"""Establish device, session and script. Raises on failure."""
with self._lock:
self._teardown()
if self.opts.remote:
# A cached remote device may hold the dead socket of a server
# that was killed and restarted; drop it so we reconnect fresh.
try:
frida.get_device_manager().remove_remote_device(self.opts.remote)
except Exception:
pass
device = self._get_device()
session = self._attach_or_spawn(device)
session.on("detached", self._on_detached)
with open(self.opts.script, "r", encoding="utf-8") as fh:
code = fh.read()
script = session.create_script(code, runtime=self.opts.runtime)
script.on("message", self._on_message)
script.load()
# Resume only after the script has loaded: resuming earlier loses
# the first hook window (references/dynamic-frida.md).
if self.spawned_pid is not None:
device.resume(self.spawned_pid)
self.device, self.session, self.script = device, session, script
# frida >= 16 prefers exports_sync; older builds expose exports.
self.exports = getattr(script, "exports_sync", None) or script.exports
self.alive = True
self.attempt = 0
def _teardown(self):
script, session = self.script, self.session
self.script = self.session = self.exports = None
self.spawned_pid = None
self.alive = False
# Unload the script *and* detach the session; without the detach every
# reload/retry cycle leaks one attached session on the device.
if script is not None:
try:
script.unload()
except Exception:
pass
if session is not None:
try:
session.off("detached", self._on_detached)
except Exception:
pass
try:
session.detach()
except Exception:
pass
def _on_detached(self, reason, *rest):
self.alive = False
sys.stderr.write("[bridge] session detached: %s %s\n" % (reason, rest or ""))
def _on_message(self, message, data):
if message.get("type") == "error":
sys.stderr.write("[script-error] %s\n" % message.get("description"))
if message.get("stack"):
sys.stderr.write(message["stack"])
else:
sys.stderr.write("[script] %s\n" % json.dumps(message, ensure_ascii=False, default=str))
def ensure(self):
"""Connect if needed; retry with backoff while the target is reachable.
Each ensure() call is its own retry cycle: a failed cycle does not
poison the next request (the device may come back seconds later).
"""
if self.alive:
return
self.attempt = 0
while True:
self.attempt += 1
if self.attempt > self.opts.retries:
raise BridgeError(
"could not (re)connect after %d attempts" % self.opts.retries)
try:
self.connect()
return
except RETRYABLE + (BridgeError, OSError) as exc:
wait = min(self.opts.backoff * (2 ** (self.attempt - 1)), 15.0)
sys.stderr.write("[bridge] connect failed (%s); retry %d in %.1fs\n"
% (exc, self.attempt, wait))
time.sleep(wait)
# -- the RPC surface -----------------------------------------------------
def call(self, name, args):
self.ensure()
# frida's export proxy fails from deep inside the transport with its own
# "unable to find method 'x'" wording; consult the advertised list first
# so the caller gets the actionable message.
try:
available = self.list_exports()
except Exception:
available = None
if available is not None and name not in available:
raise BridgeError("no such rpc export: %s (available: %s)"
% (name, " ".join(available)))
fn = getattr(self.exports, name, None)
if fn is None:
raise BridgeError("no such rpc export: %s (try `list`)" % name)
try:
return fn(*args)
except (frida.InvalidOperationError, frida.TransportError):
# Session died mid-call: one transparent reconnect + retry.
self.alive = False
self.ensure()
fn = getattr(self.exports, name, None)
if fn is None:
raise BridgeError("export disappeared after reconnect: %s" % name)
return fn(*args)
def list_exports(self):
self.ensure()
# frida 16.7 deprecates Script.list_exports() in favour of
# list_exports_sync(); try the new name first so the bridge stays
# quiet on current hosts and keeps working when the old one goes.
for attr in ("list_exports_sync", "list_exports"):
fn = getattr(self.script, attr, None)
if fn is None:
continue
try:
return fn()
except Exception:
pass
return sorted(k for k in dir(self.exports) if not k.startswith("_"))
def reload(self):
with self._lock:
self._teardown()
self.ensure()
# -- mode: one-shot call ------------------------------------------------------
def run_call_mode(bridge, opts):
# One-shot mode is meant to be scripted: report every failure as one JSON
# object plus a non-zero exit code instead of a Python traceback.
try:
args = json.loads(opts.args) if opts.args else []
except ValueError as exc:
print(json.dumps({"ok": False, "error": "--args is not valid JSON: %s" % exc},
ensure_ascii=False))
return 1
if not isinstance(args, list):
print(json.dumps({"ok": False, "error": "--args must be a JSON array"},
ensure_ascii=False))
return 1
try:
result = bridge.call(opts.export, args)
except BridgeError as exc:
print(json.dumps({"ok": False, "export": opts.export, "error": str(exc)},
ensure_ascii=False))
return 1
except Exception as exc: # a JS-side throw arrives as frida.core.RPCException
print(json.dumps({"ok": False, "export": opts.export,
"error": "%s: %s" % (type(exc).__name__, exc)},
ensure_ascii=False))
return 1
print(json.dumps({"ok": True, "export": opts.export, "result": result},
ensure_ascii=False, default=str))
return 0
# -- mode: REPL ---------------------------------------------------------------
def run_repl_mode(bridge, opts):
print("connected; exports are callable by name. commands: list | name [json-args] | reload | quit")
while True:
try:
line = input("rpc> ").strip()
except (EOFError, KeyboardInterrupt):
print()
return 0
if not line:
continue
if line in ("quit", "exit", "q"):
return 0
if line == "list":
try:
print(" ".join(bridge.list_exports()))
except BridgeError as exc:
print("error: %s" % exc)
continue
if line == "reload":
try:
bridge.reload()
print("reloaded")
except BridgeError as exc:
print("error: %s" % exc)
continue
parts = line.split(None, 1)
name = parts[0]
args = json.loads(parts[1]) if len(parts) > 1 and parts[1].strip() else []
if not isinstance(args, list):
print("args must be a JSON array")
continue
try:
result = bridge.call(name, args)
print(json.dumps(result, ensure_ascii=False, default=str))
except BridgeError as exc:
print("error: %s" % exc)
except Exception as exc: # JS-side throw arrives here as frida.core.RPCException
print("js-error: %s" % exc)
# -- mode: HTTP ---------------------------------------------------------------
def run_http_mode(bridge, opts):
class Handler(http.server.BaseHTTPRequestHandler):
def _send(self, code, payload):
body = json.dumps(payload, ensure_ascii=False, default=str).encode("utf-8")
self.send_response(code)
self.send_header("Content-Type", "application/json; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def do_GET(self):
if self.path == "/exports":
try:
self._send(200, {"ok": True, "exports": bridge.list_exports()})
except BridgeError as exc:
self._send(502, {"ok": False, "error": str(exc)})
else:
self._send(404, {"ok": False, "error": "GET /exports only"})
def do_POST(self):
if self.path == "/reload":
try:
bridge.reload()
self._send(200, {"ok": True})
except BridgeError as exc:
self._send(502, {"ok": False, "error": str(exc)})
return
if self.path != "/call":
self._send(404, {"ok": False, "error": "POST /call or /reload"})
return
try:
length = int(self.headers.get("Content-Length") or 0)
req = json.loads(self.rfile.read(length).decode("utf-8"))
name = req.get("export")
args = req.get("args", [])
if not name or not isinstance(args, list):
raise ValueError('body must be {"export": "name", "args": [...]}')
result = bridge.call(name, args)
self._send(200, {"ok": True, "result": result})
except BridgeError as exc:
self._send(502, {"ok": False, "error": str(exc)})
except Exception as exc:
self._send(400, {"ok": False, "error": str(exc)})
def log_message(self, fmt, *args):
sys.stderr.write("[http] " + (fmt % args) + "\n")
server = http.server.ThreadingHTTPServer((opts.host, opts.port), Handler)
print("listening on http://%s:%d (POST /call, GET /exports, POST /reload)"
% (opts.host, opts.port))
try:
server.serve_forever()
except KeyboardInterrupt:
pass
finally:
server.server_close()
return 0
def main(argv=None):
parser = argparse.ArgumentParser(
description="Bridge a Frida script's rpc.exports to Python/REPL/HTTP with auto-reconnect.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__.split("Examples:")[-1].strip() if __doc__ else None,
)
parser.add_argument("--script", required=True, help="path to the .js defining rpc.exports")
parser.add_argument("--package", help="target package (attach by name, or spawn with --spawn)")
parser.add_argument("--pid", type=int, help="target pid (use when attach-by-name fails)")
parser.add_argument("--device-serial", help="explicit USB device serial (see adb devices -l)")
parser.add_argument("--remote", metavar="HOST:PORT",
help="explicit remote frida via adb forward, e.g. 127.0.0.1:27042")
parser.add_argument("--spawn", action="store_true",
help="spawn the package instead of attaching (catches startup)")
parser.add_argument("--mode", choices=("call", "repl", "http"), default="repl")
parser.add_argument("--export", help="export name (call mode)")
parser.add_argument("--args", help="JSON array of arguments (call mode)")
parser.add_argument("--host", default="127.0.0.1", help="HTTP bind host (http mode)")
parser.add_argument("--port", type=int, default=8765, help="HTTP bind port (http mode)")
parser.add_argument("--runtime", default="v8", choices=("v8", "qjs"),
help="script runtime (v8 carries the Java bridge)")
parser.add_argument("--retries", type=int, default=5, help="reconnect attempts before giving up")
parser.add_argument("--backoff", type=float, default=1.0,
help="first reconnect delay (seconds); doubles per attempt")
opts = parser.parse_args(argv)
if opts.mode == "call" and not opts.export:
parser.error("--mode call requires --export")
bridge = RpcBridge(opts)
try:
bridge.ensure()
except RETRYABLE + (BridgeError, OSError) as exc:
sys.stderr.write("initial connect failed: %s\n" % exc)
return 2
if opts.mode == "call":
return run_call_mode(bridge, opts)
if opts.mode == "http":
return run_http_mode(bridge, opts)
return run_repl_mode(bridge, opts)
if __name__ == "__main__":
sys.exit(main())

104
vendor-new/apk-reverse/skills/apk-reverse/scripts/grab_crash.py Normal file
View File

@@ -0,0 +1,104 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Capture a crash stack when the app's crash-reporter SDK is hiding it.
Problem this solves
-------------------
Many apps install a global uncaught-exception handler (友盟/UCrash, Bugly, Firebase
Crashlytics in some configs). The Java stack then never reaches `logcat` -- you only
see something like:
W/UCrash.Java: uncaughtException time: <ts>
...and the process dies. Without a stack you are guessing.
Three ways to get it, in order of reliability:
1. Frida: hook Thread.setDefaultUncaughtExceptionHandler (use references/dynamic-frida.md)
2. Race the reporter's own log file (this script): it writes a file under the app's
data dir, then zips, uploads, and deletes it. Poll fast and copy on sight.
3. Use a build with the reporter disabled.
Usage
-----
python grab_crash.py --pkg com.example.app --activity .MainActivity
python grab_crash.py --pkg com.example.app --activity .MainActivity \
--scan-dir /data/data/com.example.app --wait 25
Notes
-----
* `--scan-dir` defaults to /data/data/<pkg>; the script searches a few levels deep for
newly appeared *.log / *.txt / *.stacktrace files, so you do not have to know which
SDK is installed.
* The watcher loop self-terminates after ~80 s, so nothing is left running on device.
* Requires root (the app's data dir is private).
"""
import argparse
import subprocess
import sys
import time
CAPTURE = '/data/local/tmp/_skill_crash_capture.log'
def adb(args, serial=None, timeout=180):
cmd = ['adb']
if serial:
cmd += ['-s', serial]
cmd += args
r = subprocess.run(cmd, capture_output=True, text=True, errors='replace', timeout=timeout)
return (r.stdout or '') + (('\n[stderr]\n' + r.stderr) if (r.stderr or '').strip() else '')
def su(cmd, serial=None):
return adb(['shell', 'su -c "%s"' % cmd.replace('"', '\\"')], serial)
def main():
ap = argparse.ArgumentParser(add_help=True)
ap.add_argument('--pkg', required=True)
ap.add_argument('--activity', required=True)
ap.add_argument('--serial')
ap.add_argument('--scan-dir')
ap.add_argument('--wait', type=int, default=25)
a = ap.parse_args()
scan = a.scan_dir or ('/data/data/%s' % a.pkg)
s = a.serial
# On-device watcher: copy any *.log/*.txt that appears under the app dir.
watch = (
"i=0; while [ $i -lt 400 ]; do "
" find {d} -maxdepth 5 -type f \\( -name '*.log' -o -name '*.txt' -o -name '*.stacktrace' "
" -o -name '*.crash' \\) -newermt '-3 minutes' -exec cp -f {{}} {c} \\; 2>/dev/null; "
" i=$((i+1)); sleep 0.2; "
"done"
).format(d=scan, c=CAPTURE)
su('am force-stop %s' % a.pkg, s)
su('rm -f %s' % CAPTURE, s)
print('[grab] arming on-device watcher over %s' % scan)
subprocess.Popen(['adb'] + (['-s', s] if s else []) +
['shell', "su -c '%s'" % watch],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
time.sleep(1.5)
print('[grab] starting app')
print(adb(['shell', 'am start -n %s/%s' % (a.pkg, a.activity)], s))
time.sleep(a.wait)
pid = adb(['shell', 'pidof %s' % a.pkg], s).strip()
print('[grab] pid: %s' % (pid or '(dead -> likely crashed)'))
print('[grab] ==== captured file ====')
out = su('head -c 6000 %s' % CAPTURE, s)
if 'No such file' in out or not out.strip():
print('(nothing captured)')
print('[hint] the reporter may have deleted it faster than 0.2 s, or it writes '
'outside --scan-dir. Use the Frida approach instead.')
else:
print(out)
return 0
if __name__ == '__main__':
sys.exit(main())

69
vendor-new/apk-reverse/skills/apk-reverse/scripts/hook_patch_only.js Normal file
View File

@@ -0,0 +1,69 @@
// Minimal Frida probe: neutralise ONE native death site in memory, then report PATCHED.
//
// Intended for use with spawn_patch_detach.py. The write lands while attached; the
// driver then detaches so the app runs with no instrumentation present and its UI
// actually renders. Memory.patchCode writes survive detach; Interceptor hooks do not.
//
// Configure the four constants below from your own analysis. There is nothing
// target-specific here on purpose: the shape to look for is a short "recover the frame
// and return" epilogue that replaces a terminate call. Never substitute a NOP for a
// call that has live code after it -- the fall-through is the bug you are creating.
// See references/native-tamper-and-suicide.md.
'use strict';
// --- configure -----------------------------------------------------------------
var MODULE_NAME = 'libtarget.so'; // the module that holds the site
var FILE_OFFSET = 0x0; // byte offset of the sequence to replace
var PATCH_BYTES = []; // equal-length replacement bytes
var PRE_EXISTING = ''; // hex of PATCH_BYTES, space separated, for the idempotence check
// -------------------------------------------------------------------------------
var T0 = Date.now();
function el() { return ((Date.now() - T0) / 1000).toFixed(2) + 's'; }
function hx(p, n) {
try {
return Array.from(new Uint8Array(p.readByteArray(n))).map(function (b) {
return ('0' + b.toString(16)).slice(-2);
}).join(' ');
} catch (e) { return '<unreadable>'; }
}
if (!PATCH_BYTES.length) {
console.log('[!] PATCH_BYTES is empty -- configure this probe before using it');
send('PATCHFAIL');
} else {
var timer = setInterval(function () {
var m = null;
try { m = Process.findModuleByName(MODULE_NAME); } catch (e) { m = null; }
if (!m) return;
clearInterval(timer);
var at = m.base.add(FILE_OFFSET);
var before = hx(at, PATCH_BYTES.length);
console.log('[' + el() + '][i] ' + MODULE_NAME + ' base=' + m.base +
' bytes@0x' + FILE_OFFSET.toString(16) + ' = ' + before);
if (PRE_EXISTING && before === PRE_EXISTING) {
console.log('[' + el() + '][i] already carries the patch');
send('PATCHED');
return;
}
try {
Memory.patchCode(at, PATCH_BYTES.length, function (code) {
code.writeByteArray(PATCH_BYTES);
});
console.log('[' + el() + '][PATCH] now = ' + hx(at, PATCH_BYTES.length));
send('PATCHED');
} catch (e) {
console.log('[' + el() + '][!] patch failed: ' + e);
send('PATCHFAIL');
}
}, 10);
}
// Keep the message channel alive so the driver can see the probe is loaded. Cheap, and
// it gives a heartbeat you can read while waiting for the site to appear.
setInterval(function () { console.log('[' + el() + '][alive]'); }, 10000);
console.log('=== patch-only probe armed ===');

130
vendor-new/apk-reverse/skills/apk-reverse/scripts/install_test.py Normal file
View File

@@ -0,0 +1,130 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Install an APK and run a launch health check, extracting the signals that matter.
Automates the verification loop from references/verification.md:
install -> clear logcat -> launch -> sample pid twice -> scan for fatal signatures
-> capture a screenshot.
Why the pid is sampled TWICE: a crash loop shows a live pid at any single instant.
The second sample catches it.
Usage
-----
python install_test.py --apk out/app.apk --pkg com.example.app --activity .MainActivity
python install_test.py --apk out/app.apk --pkg com.example.app --activity .MainActivity \
--shots out/shot.png --uninstall-first --grant READ_PHONE_STATE --grant ACCESS_FINE_LOCATION
Useful flags
------------
--uninstall-first wipe app data (needed when the signing key changed, and the only
way to test "does this survive a fresh install")
--keep-data reinstall over the existing app (requires the same signing key;
preserves login state -- valuable when the feature under test
needs auth)
--grant NAME runtime permission to grant (repeatable). Must be granted as
root: the shell user gets SecurityException.
--wait N seconds between the two pid samples (default 18)
Exit code is 0 only when the app launched AND both pid samples agree.
"""
import argparse
import re
import subprocess
import sys
import time
FATAL = re.compile(
r'FATAL EXCEPTION|VerifyError|IncompatibleClassChangeError|'
r'ClassNotFoundException|NoClassDefFoundError|LinkageError|'
r'Failure starting process|uncaughtException',
re.I)
def adb(args, serial=None, timeout=600):
cmd = ['adb']
if serial:
cmd += ['-s', serial]
cmd += args
r = subprocess.run(cmd, capture_output=True, text=True, errors='replace', timeout=timeout)
return (r.stdout or '') + (('\n[stderr]\n' + r.stderr) if (r.stderr or '').strip() else '')
def sh(cmd, serial=None):
return adb(['shell', cmd], serial)
def su(cmd, serial=None):
return adb(['shell', 'su -c "%s"' % cmd.replace('"', '\\"')], serial)
def main():
ap = argparse.ArgumentParser(add_help=True)
ap.add_argument('--apk', required=True)
ap.add_argument('--pkg', required=True)
ap.add_argument('--activity', required=True, help='e.g. .MainActivity or fully qualified')
ap.add_argument('--serial')
ap.add_argument('--uninstall-first', action='store_true')
ap.add_argument('--keep-data', action='store_true')
ap.add_argument('--grant', action='append', default=[])
ap.add_argument('--wait', type=int, default=18)
ap.add_argument('--shots')
a = ap.parse_args()
s = a.serial
print('[0] devices:')
print(adb(['devices', '-l']))
if a.uninstall_first:
print('[1] uninstall')
print(su('am force-stop %s; pm uninstall %s' % (a.pkg, a.pkg), s))
stage = '/data/local/tmp/_skill_stage.apk'
print('[2] push')
print(adb(['push', a.apk, stage], s))
print('[3] install')
print(su('pm install -r -t -d %s' % stage, s))
for perm in a.grant:
p = perm if perm.startswith('android.permission.') else 'android.permission.' + perm
su('pm grant %s %s' % (a.pkg, p), s)
print('[4] clear logcat')
adb(['logcat', '-c'], s)
comp = '%s/%s' % (a.pkg, a.activity if a.activity.startswith('.') or '.' in a.activity
else a.activity)
print('[5] launch')
print(sh('am start -n %s' % comp, s))
time.sleep(6)
p1 = sh('pidof %s' % a.pkg, s).strip()
print('[6] pid @+6s : %s' % (p1 or '(dead)'))
time.sleep(a.wait)
p2 = sh('pidof %s' % a.pkg, s).strip()
print('[7] pid @+%ds : %s' % (6 + a.wait, p2 or '(dead)'))
print('[8] focus : %s' % sh('dumpsys window | grep mCurrentFocus', s).strip())
if a.shots:
su('screencap -p /sdcard/_skill_shot.png', s)
print(adb(['pull', '/sdcard/_skill_shot.png', a.shots], s))
print('[9] fatal signatures:')
log = adb(['logcat', '-d', '-v', 'brief'], s)
hits = [ln for ln in log.splitlines() if FATAL.search(ln)]
if hits:
for ln in hits[-25:]:
print(' ', ln)
else:
print(' none')
ok = bool(p1) and p1 == p2 and not hits
print()
print('[result] %s' % ('OK (launched, stable, no fatal signatures)'
if ok else 'CHECK (see above)'))
print('[note] stability is NOT proof the patch worked. Exercise the changed '
'feature and its neighbours -- see references/verification.md.')
return 0 if ok else 1
if __name__ == '__main__':
sys.exit(main())

963
vendor-new/apk-reverse/skills/apk-reverse/scripts/java2c_probe.py Normal file
View File

@@ -0,0 +1,963 @@
#!/usr/bin/env python3
"""java2c_probe.py - collect the evidence that decides HOW a target was hardened.
The question this answers is narrow on purpose. Faced with "the Java methods have
no bodies", four different mechanisms produce that observation and they need four
different routes:
Java2C Java bytecode translated to C, compiled into a .so. No dex
bytecode exists at runtime, ever. Route: read the .so.
Extraction shell Real dex bytecode, encrypted at rest, decrypted at runtime.
Route: dump memory, then validate the dump.
VMP Real dex bytecode replaced by private opcodes fed to an
interpreter. Route: measure, then usually stop.
JNI sinking A few hot methods deliberately moved to native by hand.
Route: reverse the .so; the dex is still readable.
The expensive mistake is reading the first shape as the second: the analyst goes
looking for a decrypted DEX that is never produced, because nothing was ever
encrypted. This script prints the discriminating evidence and labels each item
strong / medium / weak so the reading is not mistaken for a verdict.
Usage (paths are always explicit; nothing is discovered implicitly):
python java2c_probe.py --apk sample.apk
python java2c_probe.py --apk sample.apk --json
python java2c_probe.py --dex classes.dex --so lib/arm64-v8a/libnc.so
python java2c_probe.py --dir extracted/ # every *.dex and *.so below
Exit codes: 0 = evidence collected, 1 = no usable input, 2 = argument error.
"""
import argparse
import json
import os
import re
import struct
import sys
import zipfile
# ---------------------------------------------------------------------------
# dex parsing (self-contained; deliberately does not import the repo's dexutil)
# ---------------------------------------------------------------------------
ACC_NATIVE = 0x0100
ACC_ABSTRACT = 0x0400
# Instruction length in 16-bit code units, for the subset that a stub or a
# decompiled-from-C body produces. Unknown opcodes return 1 so a counting error
# can never inflate the trivial-body ratio; see the note on that metric below.
_INSN_UNITS = {
0x00: 1, 0x01: 1, 0x02: 1, 0x03: 1, 0x04: 1, 0x05: 1, 0x06: 1, 0x07: 1,
0x08: 1, 0x09: 1, 0x0a: 1, 0x0b: 1, 0x0c: 1, 0x0d: 1, 0x0e: 1, 0x0f: 1,
0x10: 1, 0x11: 1, 0x12: 1, 0x13: 2, 0x14: 3, 0x15: 2, 0x16: 2, 0x17: 3,
0x18: 5, 0x19: 2, 0x1a: 2, 0x1b: 3, 0x1c: 2, 0x1d: 1, 0x1e: 1, 0x1f: 1,
0x20: 1, 0x21: 1, 0x22: 1, 0x23: 1, 0x24: 3, 0x25: 3, 0x26: 3, 0x27: 1,
0x28: 1, 0x29: 1, 0x2a: 1, 0x2b: 1, 0x2c: 1, 0x2d: 1, 0x2e: 1, 0x2f: 1,
0x30: 1, 0x31: 1, 0x32: 1, 0x33: 1, 0x34: 1, 0x35: 1, 0x36: 1, 0x37: 1,
0x38: 1, 0x39: 1, 0x3a: 1, 0x3b: 1, 0x3c: 1, 0x3d: 1, 0x3e: 1, 0x3f: 1,
0x40: 1, 0x41: 1, 0x42: 1, 0x43: 1, 0x44: 2, 0x45: 2, 0x46: 2, 0x47: 2,
0x48: 2, 0x49: 2, 0x4a: 2, 0x4b: 2, 0x4c: 2, 0x4d: 2, 0x4e: 2, 0x4f: 2,
0x50: 2, 0x51: 2, 0x52: 2, 0x53: 2, 0x54: 2, 0x55: 2, 0x56: 2, 0x57: 2,
0x58: 2, 0x59: 2, 0x5a: 2, 0x5b: 2, 0x5c: 2, 0x5d: 2, 0x5e: 2, 0x5f: 2,
0x60: 2, 0x61: 2, 0x62: 2, 0x63: 2, 0x64: 2, 0x65: 2, 0x66: 2, 0x67: 2,
0x68: 2, 0x69: 2, 0x6a: 2, 0x6b: 2, 0x6c: 2, 0x6d: 2, 0x6e: 3, 0x6f: 3,
0x70: 3, 0x71: 3, 0x72: 3, 0x73: 1, 0x74: 1, 0x75: 1, 0x76: 1, 0x77: 1,
0x78: 2, 0x79: 2, 0x7a: 2, 0x7b: 2, 0x7c: 2, 0x7d: 2, 0x7e: 2, 0x7f: 2,
0x80: 2, 0x81: 2, 0x82: 2, 0x83: 2, 0x84: 2, 0x85: 2, 0x86: 2, 0x87: 2,
0x88: 2, 0x89: 2, 0x8a: 2, 0x8b: 2, 0x8c: 2, 0x8d: 2, 0x8e: 2, 0x8f: 2,
0x90: 2, 0x91: 2, 0x92: 2, 0x93: 2, 0x94: 2, 0x95: 2, 0x96: 2, 0x97: 2,
0x98: 2, 0x99: 2, 0x9a: 2, 0x9b: 2, 0x9c: 2, 0x9d: 2, 0x9e: 2, 0x9f: 2,
0xa0: 2, 0xa1: 2, 0xa2: 2, 0xa3: 2, 0xa4: 2, 0xa5: 2, 0xa6: 2, 0xa7: 2,
0xa8: 2, 0xa9: 2, 0xaa: 2, 0xab: 2, 0xac: 2, 0xad: 2, 0xae: 2, 0xaf: 2,
0xb0: 1, 0xb1: 1, 0xb2: 2, 0xb3: 2, 0xb4: 2, 0xb5: 2, 0xb6: 2, 0xb7: 2,
0xb8: 2, 0xb9: 2, 0xba: 2, 0xbb: 2, 0xbc: 2, 0xbd: 2, 0xbe: 2, 0xbf: 2,
0xc0: 2, 0xc1: 2, 0xc2: 2, 0xc3: 2, 0xc4: 2, 0xc5: 2, 0xc6: 2, 0xc7: 2,
0xc8: 2, 0xc9: 2, 0xca: 2, 0xcb: 2, 0xcc: 2, 0xcd: 2, 0xce: 2, 0xcf: 2,
0xd0: 2, 0xd1: 2, 0xd2: 2, 0xd3: 2, 0xd4: 2, 0xd5: 2, 0xd6: 2, 0xd7: 2,
0xd8: 2, 0xd9: 2, 0xda: 2, 0xdb: 2, 0xdc: 2, 0xdd: 2, 0xde: 2, 0xdf: 2,
0xe0: 2, 0xe1: 2, 0xe2: 2, 0xe3: 0, 0xe4: 0, 0xe5: 0, 0xe6: 0, 0xe7: 0,
0xe8: 0, 0xe9: 0, 0xea: 0, 0xeb: 0, 0xec: 0, 0xed: 0, 0xee: 0, 0xef: 0,
0xf0: 0, 0xf1: 0, 0xf2: 0, 0xf3: 0, 0xf4: 0, 0xf5: 0, 0xf6: 0, 0xf7: 0,
0xf8: 0, 0xf9: 0, 0xfa: 0, 0xfb: 0, 0xfc: 0, 0xfd: 0, 0xfe: 0, 0xff: 0,
}
_RETURN_OPS = (0x0e, 0x0f, 0x10, 0x11)
def _uleb128(buf, pos):
result = 0
shift = 0
while True:
if pos >= len(buf):
raise ValueError("uleb128 truncated")
byte = buf[pos]
pos += 1
result |= (byte & 0x7F) << shift
if not (byte & 0x80):
return result, pos
shift += 7
if shift > 35:
raise ValueError("uleb128 too long")
class Dex(object):
"""Minimal structural reader: header, class_defs, class_data, code_item."""
def __init__(self, data, name):
self.data = data
self.name = name
self.header_ok = False
self.header_note = ""
self.methods = [] # dicts: class, name, flags, code_off, units
self.class_names = []
self.native_methods = 0
self.abstract_methods = 0
self.body_methods = 0
self.no_code_methods = 0 # neither native nor abstract, yet code_off == 0
self.trivial_bodies = 0
self.native_with_code = 0
self.fully_native_classes = 0
self.native_dominated_classes = 0
self.parse_error = None
self._parse()
def _u16(self, off):
return struct.unpack_from("<H", self.data, off)[0]
def _u32(self, off):
return struct.unpack_from("<I", self.data, off)[0]
def _parse(self):
d = self.data
if len(d) < 112 or d[:4] != b"dex\n":
self.parse_error = "not a dex image (bad magic or too short)"
return
try:
file_size = self._u32(0x20)
# Read for completeness against the dex header layout: this function walks the
# header field by field and a reader comparing it to the format spec should not
# have to wonder whether a field was skipped. Not every field feeds the verdict.
map_off = self._u32(0x34) # noqa: F841
string_ids_size = self._u32(0x38)
string_ids_off = self._u32(0x3C)
type_ids_size = self._u32(0x40)
type_ids_off = self._u32(0x44)
proto_ids_size = self._u32(0x48) # noqa: F841 (see map_off above)
proto_ids_off = self._u32(0x4C) # noqa: F841 (see map_off above)
method_ids_size = self._u32(0x58)
method_ids_off = self._u32(0x5C)
class_defs_size = self._u32(0x60)
class_defs_off = self._u32(0x64)
except Exception as exc: # pragma: no cover
self.parse_error = "header read failed: %s" % exc
return
# Header claims vs. reality. A truncated or padded capture is the normal
# way this probe is handed bad input, so say which one happened.
if file_size != len(d):
self.header_note = "header file_size=%d actual=%d" % (file_size, len(d))
else:
self.header_ok = True
if not (0 < string_ids_size < 1 << 22 and string_ids_off + string_ids_size * 4 <= len(d)):
self.parse_error = "string_ids table out of bounds"
return
string_offsets = [self._u32(string_ids_off + 4 * i) for i in range(string_ids_size)]
def string_at(idx):
if idx < 0 or idx >= len(string_offsets):
return "<bad-string-idx-%d>" % idx
off = string_offsets[idx]
_, pos = _uleb128(d, off) # utf16 length, ignored
end = d.find(b"\x00", pos)
if end < 0:
return "<unterminated>"
return d[pos:end].decode("utf-8", "replace")
try:
type_desc = [string_at(self._u32(type_ids_off + 4 * i))
for i in range(type_ids_size)]
except Exception as exc:
self.parse_error = "type_ids read failed: %s" % exc
return
try:
if method_ids_off + method_ids_size * 8 > len(d):
raise ValueError("method_ids out of bounds")
method_info = []
for i in range(method_ids_size):
base = method_ids_off + 8 * i
class_idx = self._u16(base)
name_idx = self._u32(base + 4)
desc = type_desc[class_idx] if class_idx < len(type_desc) else "?"
method_info.append((desc, string_at(name_idx)))
except Exception as exc:
self.parse_error = "method_ids read failed: %s" % exc
return
if class_defs_off + class_defs_size * 32 > len(d):
self.parse_error = "class_defs table out of bounds"
return
per_class_native = {}
per_class_total = {}
for ci in range(class_defs_size):
base = class_defs_off + 32 * ci
class_idx = self._u32(base)
class_data_off = self._u32(base + 24)
cname = type_desc[class_idx] if class_idx < len(type_desc) else "?%d" % class_idx
self.class_names.append(cname)
if class_data_off == 0:
continue
try:
pos = class_data_off
static_fields, pos = _uleb128(d, pos)
instance_fields, pos = _uleb128(d, pos)
direct_methods, pos = _uleb128(d, pos)
virtual_methods, pos = _uleb128(d, pos)
for _ in range(static_fields + instance_fields):
_, pos = _uleb128(d, pos) # field_idx_diff
_, pos = _uleb128(d, pos) # access_flags
# direct_methods and virtual_methods are two independent encoded
# lists: each one's first method_idx_diff is relative to 0. Walking
# them as a single run silently mis-attributes every virtual
# method, so they are decoded as two passes.
for list_size in (direct_methods, virtual_methods):
method_idx = 0
for _ in range(list_size):
diff, pos = _uleb128(d, pos)
flags, pos = _uleb128(d, pos)
code_off, pos = _uleb128(d, pos)
method_idx += diff
if method_idx < len(method_info):
mcls, mname = method_info[method_idx]
else:
mcls, mname = "?", "?%d" % method_idx
units = None
if code_off:
units = self._u16(code_off + 12) # insns_size
self.methods.append({
"class": mcls, "name": mname, "flags": flags,
"code_off": code_off, "units": units,
})
# Constructors are excluded from the per-class shape: a
# Java2C pass leaves <init>/<clinit> in Java in practice,
# and counting them would make "all methods native" an
# essentially unreachable test.
named = mname not in ("<init>", "<clinit>")
if named:
per_class_total[cname] = per_class_total.get(cname, 0) + 1
if flags & ACC_NATIVE:
self.native_methods += 1
if named:
per_class_native[cname] = per_class_native.get(cname, 0) + 1
if code_off:
self.native_with_code += 1
elif flags & ACC_ABSTRACT:
self.abstract_methods += 1
elif code_off == 0:
self.no_code_methods += 1
else:
self.body_methods += 1
if self._is_trivial(code_off, units):
self.trivial_bodies += 1
except Exception:
# One unreadable class_data must not void the whole measurement.
continue
for cname, total in per_class_total.items():
if total < 3:
continue
native_here = per_class_native.get(cname, 0)
if native_here == total:
self.fully_native_classes += 1
if native_here >= 0.7 * total:
self.native_dominated_classes += 1
def _is_trivial(self, code_off, units):
"""A body that only returns a constant: the classic stub shape.
Approximation, and labelled as such: it walks the instruction stream with
the length table above and accepts only nop/move/const/return. Anything
that touches a field, calls out, or branches is not a stub.
"""
if not units:
return False
insns_off = code_off + 16
if insns_off + units * 2 > len(self.data):
return False
try:
pos = 0
seen = 0
while pos < units and seen < 8:
word = struct.unpack_from("<H", self.data, insns_off + pos * 2)[0]
op = word & 0xFF
if op in _RETURN_OPS:
return True if seen <= 2 else False
if op == 0x00 or 0x01 <= op <= 0x09 or 0x12 <= op <= 0x19:
step = _INSN_UNITS.get(op, 1)
if step == 0:
return False
pos += step
seen += 1
continue
return False
return False
except Exception:
return False
# -- derived metrics ---------------------------------------------------
@property
def total_methods(self):
return len(self.methods)
@property
def native_ratio(self):
return (float(self.native_methods) / self.total_methods) if self.total_methods else 0.0
@property
def trivial_ratio(self):
return (float(self.trivial_bodies) / self.body_methods) if self.body_methods else 0.0
def summary(self):
return {
"name": self.name,
"size": len(self.data),
"header_size_matches": self.header_ok,
"header_note": self.header_note,
"classes": len(self.class_names),
"total_methods": self.total_methods,
"native_methods": self.native_methods,
"native_ratio": round(self.native_ratio, 4),
"abstract_methods": self.abstract_methods,
"body_methods": self.body_methods,
"no_code_methods": self.no_code_methods,
"native_with_code": self.native_with_code,
"trivial_bodies": self.trivial_bodies,
"trivial_ratio": round(self.trivial_ratio, 4),
"fully_native_classes": self.fully_native_classes,
"native_dominated_classes": self.native_dominated_classes,
"bytes_per_class": int(len(self.data) / len(self.class_names)) if self.class_names else 0,
"parse_error": self.parse_error,
}
# ---------------------------------------------------------------------------
# ELF parsing (self-contained; no lief / pyelftools dependency)
# ---------------------------------------------------------------------------
# Each entry: (regex, label, strength, meaning). Strength is what stops a weak
# hit from being read as a verdict - see the printed weak-criteria block.
_STRINGSIGS = [
(rb"Dex2C", "Dex2C", "strong", "dcc toolchain marker in code/rodata"),
(rb"dynamic_register_compile_methods", "dcc-register-fn", "strong",
"dcc's generated RegisterNatives entry point"),
(rb"dcc", "dcc", "weak", "3 letters: matches unrelated words (e.g. 'addcc')"),
(rb"JNI_OnLoad", "JNI_OnLoad", "weak", "present in almost every JNI library"),
(rb"RegisterNatives", "RegisterNatives", "weak",
"dynamic registration; ordinary JNI libraries use it too"),
(rb"libc\+\+_shared\.so", "libc++_shared", "weak", "NDK C++ runtime, not a hardening marker"),
(rb"c\+\+_static", "c++_static", "weak", "dcc's default APP_STL; also a common NDK setting"),
(rb"FindClass", "FindClass", "weak", "any JNI callback path"),
(rb"GetMethodID", "GetMethodID", "weak", "any JNI callback path"),
(rb"GetStaticMethodID", "GetStaticMethodID", "weak", "any JNI callback path"),
(rb"CallObjectMethod", "CallObjectMethod", "weak", "any JNI callback path"),
(rb"CallStaticObjectMethod", "CallStaticObjectMethod", "weak", "any JNI callback path"),
(rb"NewLocalRef", "NewLocalRef", "weak", "generated code that manages refs by hand"),
(rb"ScopedLocalRef", "ScopedLocalRef", "medium", "dcc runtime header name"),
(rb"well_known_classes", "well_known_classes", "medium", "dcc runtime header name"),
]
_JNI_CALLS = ("FindClass", "GetMethodID", "GetStaticMethodID", "CallObjectMethod",
"CallStaticObjectMethod", "CallIntMethod", "CallVoidMethod",
"CallStaticIntMethod", "NewObject", "GetFieldID")
class Elf(object):
"""Enough of ELF to answer: what is exported, what is imported, what is inside."""
def __init__(self, data, name):
self.data = data
self.name = name
self.arch = "unknown"
self.is_elf = False
self.dynsym = [] # (name, shndx)
self.sig_hits = {} # label -> (count, strength, meaning)
self.java_symbols = []
self.has_jni_onload = False
self.imports_register_natives = False
self.jni_call_refs = []
self.elf_note = ""
self._parse()
def _parse(self):
d = self.data
if len(d) < 64 or d[:4] != b"\x7fELF":
self.elf_note = "not an ELF image"
return
self.is_elf = True
ei_class = d[4]
ei_data = d[5]
if ei_class not in (1, 2):
self.elf_note = "bad EI_CLASS=%d" % ei_class
return
endian = "<" if ei_data == 1 else ">"
is64 = ei_class == 2
try:
e_type = struct.unpack_from(endian + "H", d, 16)[0]
e_machine = struct.unpack_from(endian + "H", d, 18)[0]
if is64:
e_shoff = struct.unpack_from(endian + "Q", d, 0x28)[0]
e_shentsize = struct.unpack_from(endian + "H", d, 0x3A)[0]
e_shnum = struct.unpack_from(endian + "H", d, 0x3C)[0]
e_shstrndx = struct.unpack_from(endian + "H", d, 0x3E)[0]
else:
e_shoff = struct.unpack_from(endian + "I", d, 0x20)[0]
e_shentsize = struct.unpack_from(endian + "H", d, 0x2E)[0]
e_shnum = struct.unpack_from(endian + "H", d, 0x30)[0]
e_shstrndx = struct.unpack_from(endian + "H", d, 0x32)[0]
except struct.error as exc:
self.elf_note = "header read failed: %s" % exc
return
self.arch = {
3: "x86", 8: "mips", 20: "ppc", 40: "arm", 62: "x86_64",
183: "aarch64", 243: "riscv",
}.get(e_machine, "machine-%d" % e_machine)
if e_type not in (2, 3):
self.elf_note = "e_type=%d (not EXEC/DYN)" % e_type
# Strings first: it is the check that works even when the section table
# has been stripped, which is the common case for a hardened library.
self._scan_strings()
if e_shoff and e_shnum and e_shoff + e_shnum * e_shentsize <= len(d):
self._read_sections(endian, is64, e_shoff, e_shentsize, e_shnum, e_shstrndx)
else:
self.elf_note = (self.elf_note + "; " if self.elf_note else "") + \
"no usable section table (stripped?) - symbol checks skipped"
def _scan_strings(self):
d = self.data
for pattern, label, strength, meaning in _STRINGSIGS:
count = len(re.findall(pattern, d))
if count:
self.sig_hits[label] = (count, strength, meaning)
for call in _JNI_CALLS:
if re.search(re.escape(call).encode(), d):
self.jni_call_refs.append(call)
def _read_sections(self, endian, is64, shoff, shentsize, shnum, shstrndx):
d = self.data
sections = []
for i in range(shnum):
base = shoff + i * shentsize
try:
if is64:
name_off = struct.unpack_from(endian + "I", d, base)[0]
sh_type = struct.unpack_from(endian + "I", d, base + 4)[0]
sh_offset = struct.unpack_from(endian + "Q", d, base + 0x18)[0]
sh_size = struct.unpack_from(endian + "Q", d, base + 0x20)[0]
sh_link = struct.unpack_from(endian + "I", d, base + 0x28)[0]
sh_entsize = struct.unpack_from(endian + "Q", d, base + 0x38)[0]
else:
name_off = struct.unpack_from(endian + "I", d, base)[0]
sh_type = struct.unpack_from(endian + "I", d, base + 4)[0]
sh_offset = struct.unpack_from(endian + "I", d, base + 0x10)[0]
sh_size = struct.unpack_from(endian + "I", d, base + 0x14)[0]
sh_link = struct.unpack_from(endian + "I", d, base + 0x18)[0]
sh_entsize = struct.unpack_from(endian + "I", d, base + 0x24)[0]
except struct.error:
return
sections.append((name_off, sh_type, sh_offset, sh_size, sh_link, sh_entsize))
if shstrndx >= len(sections):
return
_, _, str_off, str_size, _, _ = sections[shstrndx]
if str_off + str_size > len(d):
return
shstr = d[str_off:str_off + str_size]
def sec_name(off):
end = shstr.find(b"\x00", off)
return shstr[off:end if end >= 0 else len(shstr)].decode("utf-8", "replace")
for name_off, sh_type, sh_offset, sh_size, sh_link, sh_entsize in sections:
# SHT_DYNSYM == 11, SHT_SYMTAB == 2
if sh_type not in (2, 11):
continue
if sh_link >= len(sections):
continue
_, _, dstr_off, dstr_size, _, _ = sections[sh_link]
if dstr_off + dstr_size > len(d) or sh_offset + sh_size > len(d):
continue
dstr = d[dstr_off:dstr_off + dstr_size]
entsize = sh_entsize or (24 if is64 else 16)
if entsize == 0:
continue
for off in range(sh_offset, sh_offset + sh_size, entsize):
try:
if is64:
st_name = struct.unpack_from(endian + "I", d, off)[0]
st_shndx = struct.unpack_from(endian + "H", d, off + 6)[0]
else:
st_name = struct.unpack_from(endian + "I", d, off)[0]
st_shndx = struct.unpack_from(endian + "H", d, off + 14)[0]
except struct.error:
break
if st_name >= len(dstr):
continue
end = dstr.find(b"\x00", st_name)
sname = dstr[st_name:end if end >= 0 else len(dstr)].decode("utf-8", "replace")
if not sname:
continue
self.dynsym.append((sname, st_shndx))
if sname.startswith("Java_"):
self.java_symbols.append(sname)
if sname == "JNI_OnLoad":
self.has_jni_onload = True
# Substring match, so a C implementation (bare name) and a C++
# one (mangled form) are both caught. In practice NEITHER is
# present: see the note on dynamic-registration in classify().
if "RegisterNatives" in sname and st_shndx == 0:
self.imports_register_natives = True
def summary(self):
return {
"name": self.name,
"size": len(self.data),
"is_elf": self.is_elf,
"arch": self.arch,
"elf_note": self.elf_note,
"dynsym_entries": len(self.dynsym),
"java_symbols": len(self.java_symbols),
"java_symbol_sample": self.java_symbols[:5],
"exports_JNI_OnLoad": self.has_jni_onload,
"imports_RegisterNatives": self.imports_register_natives,
"jni_call_refs": sorted(self.jni_call_refs),
"signature_hits": {k: {"count": v[0], "strength": v[1], "meaning": v[2]}
for k, v in sorted(self.sig_hits.items())},
}
# ---------------------------------------------------------------------------
# verdict
# ---------------------------------------------------------------------------
def classify(dexes, elves, so_bytes_total):
"""Turn measurements into a type guess plus the reasoning behind it.
Every rule below is a shape test, not a proof, and the confidence is reported
so a low-confidence Java2C reading is not mistaken for a confirmed one.
"""
evidence = []
verdict = "no-hardening-signal"
confidence = "low"
dex_metrics = [d for d in dexes if not d.parse_error]
total_methods = sum(d.total_methods for d in dex_metrics)
total_native = sum(d.native_methods for d in dex_metrics)
native_ratio = (float(total_native) / total_methods) if total_methods else 0.0
fully_native = sum(d.fully_native_classes for d in dex_metrics)
native_dominated = sum(d.native_dominated_classes for d in dex_metrics)
no_code = sum(d.no_code_methods for d in dex_metrics)
trivial_bodies = sum(d.trivial_bodies for d in dex_metrics)
body_methods = sum(d.body_methods for d in dex_metrics)
trivial_ratio = (float(trivial_bodies) / body_methods) if body_methods else 0.0
java_syms = sum(len(e.java_symbols) for e in elves)
register_natives_ref = any(e.imports_register_natives for e in elves)
has_jni_onload = any(e.has_jni_onload for e in elves)
# The reliable static signature of dynamic registration is the ABSENCE of
# Java_* symbols from a library that does export a JNI entry point. Looking
# for a literal RegisterNatives symbol does not work: NDK's C++ jni.h
# implements it as an inline member that calls through the function table,
# so no such symbol is emitted (measured: a real library exports JNI_OnLoad
# with zero Java_* and zero RegisterNatives symbols).
# Judged per library, not across the set: one library registering
# dynamically says nothing about another that exports static symbols.
dynamic_register_shape = any(e.has_jni_onload and not e.java_symbols for e in elves)
tool_markers = []
for e in elves:
for label, (count, strength, meaning) in e.sig_hits.items():
if strength == "strong":
tool_markers.append("%s(x%d) in %s" % (label, count, os.path.basename(e.name)))
medium_markers = []
for e in elves:
for label, (count, strength, meaning) in e.sig_hits.items():
if strength == "medium":
medium_markers.append("%s in %s" % (label, os.path.basename(e.name)))
# --- dex-layer shape -------------------------------------------------
java2c_dex_shape = (native_dominated >= 2) or (native_ratio >= 0.35 and total_native >= 10)
jni_sink_dex_shape = 0 < total_native <= 20 and native_ratio < 0.15
stub_shape = no_code > 0 or (body_methods >= 8 and trivial_ratio >= 0.5)
if dex_metrics:
evidence.append({
"claim": "dex layer: native declaration density %.2f%% (%d/%d)"
% (native_ratio * 100, total_native, total_methods),
"strength": "medium" if java2c_dex_shape else "strong",
"supports": "java2c" if java2c_dex_shape else "baseline",
"note": "High density with whole classes turned native is the Java2C shape. "
"It is NOT sufficient on its own: a hand-written JNI class looks "
"identical in dex, so the .so layer decides.",
})
evidence.append({
"claim": "dex layer: %d classes are native-dominated (>=70%% native of >=3 "
"named methods), %d fully native" % (native_dominated, fully_native),
"strength": "medium",
"supports": "java2c" if native_dominated >= 2 else "baseline",
"note": "A hand-written JNI boundary class normally keeps its Java half, so "
"whole classes turned native are the Java2C shape. Constructors are "
"excluded: they stay in Java even after a translation pass.",
})
if no_code:
evidence.append({
"claim": "dex layer: %d non-native, non-abstract methods have no code_item"
% no_code,
"strength": "strong",
"supports": "extraction-shell/vmp",
"note": "Structurally invalid for plain Java; expect a loader that "
"fills bodies at runtime, or a private-opcode converter.",
})
if trivial_ratio:
evidence.append({
"claim": "dex layer: %.0f%% of method bodies are return-a-constant stubs "
"(%d/%d)" % (trivial_ratio * 100, trivial_bodies, body_methods),
"strength": "medium",
"supports": "extraction-shell/vmp" if stub_shape else "baseline",
"note": "Approximate metric (see --help). Empty bodies are the signature "
"of an extraction shell or a VMP, not of Java2C.",
})
# --- native-layer shape ----------------------------------------------
if elves:
evidence.append({
"claim": "native layer: %d exported Java_* symbols across %d library(ies)"
% (java_syms, len(elves)),
"strength": "strong" if java_syms else "medium",
"supports": "java2c" if java_syms else "none",
"note": "Static-linkage form. Compare the count against the dex native "
"method count: a rough 1:1 map is the Java2C signature.",
})
if dynamic_register_shape:
dyn_libs = sorted(os.path.basename(e.name)
for e in elves if e.has_jni_onload and not e.java_symbols)
evidence.append({
"claim": "native layer: dynamic registration in %s "
"(JNI_OnLoad exported, zero Java_* symbols)"
% ", ".join(dyn_libs),
"strength": "strong",
"supports": "dynamic-registration",
"note": "Binding happens at runtime, so a symbol search comes back empty "
"and fails SILENTLY. Do not expect a RegisterNatives symbol "
"either: NDK's C++ jni.h makes it an inline member that calls "
"through the function table, so none is emitted.",
})
if register_natives_ref:
evidence.append({
"claim": "native layer: a RegisterNatives symbol is imported",
"strength": "weak",
"supports": "dynamic-registration",
"note": "Only a C-side implementation leaves this name; its absence "
"proves nothing.",
})
if has_jni_onload:
evidence.append({
"claim": "native layer: JNI_OnLoad exported",
"strength": "weak",
"supports": "jni-boundary-exists",
"note": "Present in nearly every JNI library. Holds no information "
"about which hardening was applied.",
})
if tool_markers:
evidence.append({
"claim": "native layer: toolchain markers found: %s" % ", ".join(tool_markers),
"strength": "strong",
"supports": "java2c",
"note": "A Dex-to-C toolchain leaves its own runtime symbols behind.",
})
if medium_markers:
evidence.append({
"claim": "native layer: runtime markers found: %s" % ", ".join(medium_markers),
"strength": "medium",
"supports": "java2c",
"note": "dcc runtime header names; suggestive, and easy to rename.",
})
# --- verdict ----------------------------------------------------------
if tool_markers and (java2c_dex_shape or java_syms):
verdict, confidence = "java2c", "high"
elif java2c_dex_shape and java_syms and total_native and \
(float(java_syms) / total_native) >= 0.5:
verdict, confidence = "java2c", "high"
elif java2c_dex_shape and medium_markers:
verdict, confidence = "java2c", "medium"
elif java2c_dex_shape and dynamic_register_shape:
# A Dex-to-C toolchain that hides its symbols (dcc's Application.mk ships
# -fvisibility=hidden) leaves exactly this: a JNI entry point and a
# registration table, with no Java_* name to search for.
verdict, confidence = "java2c", "medium"
elif stub_shape:
verdict, confidence = "extraction-shell-or-vmp", "medium"
elif java2c_dex_shape:
verdict, confidence = "java2c-suspect-needs-so", "low"
elif jni_sink_dex_shape:
verdict, confidence = "jni-sinking", "medium"
elif dex_metrics or elves:
verdict, confidence = "no-hardening-signal", "medium"
routes = {
"java2c": [
"Do NOT look for a decrypted DEX: none exists, at any point, in memory.",
"Read the C in the .so: each translated method is one function with a "
"JNIEnv*/jobject prefix.",
"Rebuild the call graph through the JNI reverse calls "
"(FindClass/GetMethodID/CallXxxMethod).",
],
"java2c-suspect-needs-so": [
"Supply the matching .so (--so) before acting: the dex shape alone "
"cannot separate Java2C from a hand-written JNI class.",
"If no .so carries the logic, re-read the dex shape as ordinary Java.",
],
"extraction-shell-or-vmp": [
"Dump memory and measure the dump with scripts/dex_dump_validate.py.",
"If the dump is mostly stubs, the bodies are filled at invocation time: "
"references/advanced-unpacking.md.",
"If bodies decode as private opcodes, that is a VMP - recovery cost "
"usually exceeds the task value.",
],
"jni-sinking": [
"The dex is still readable: locate the Java call site, then reverse "
"the one or few native functions.",
"references/native-and-so.md, then the algorithm tooling.",
],
"dynamic-registration": [
"Hook or find RegisterNatives to read the binding table at runtime.",
],
"no-hardening-signal": [
"Proceed with ordinary dex-level analysis.",
],
}
return {
"verdict": verdict,
"confidence": confidence,
"metrics": {
"dex_total_methods": total_methods,
"dex_native_methods": total_native,
"dex_native_ratio": round(native_ratio, 4),
"dex_fully_native_classes": fully_native,
"dex_native_dominated_classes": native_dominated,
"dex_no_code_methods": no_code,
"dex_trivial_ratio": round(trivial_ratio, 4),
"so_java_symbols": java_syms,
"so_dynamic_registration": dynamic_register_shape,
"so_bytes_total": so_bytes_total,
},
"evidence": evidence,
"route": routes.get(verdict, []),
"weak_criteria_warning": [
"JNI_OnLoad / RegisterNatives / libc++_shared.so / FindClass appear in "
"ordinary JNI libraries: a hit is not a hardening verdict.",
"A high native ratio is also produced by hand-written JNI code and by R8 "
"stripping Java bodies; only the .so layer separates these.",
"The trivial-body ratio is approximate and flags extraction shells and "
"VMPs, never Java2C.",
],
}
# ---------------------------------------------------------------------------
# input collection
# ---------------------------------------------------------------------------
def _looks_like_dex(data):
return data[:4] == b"dex\n"
def _looks_like_elf(data):
return data[:4] == b"\x7fELF"
def collect(args):
dex_blobs = []
elf_blobs = []
sources = []
def add_file(path):
try:
with open(path, "rb") as fp:
data = fp.read()
except OSError as exc:
print("warning: cannot read %s (%s)" % (path, exc), file=sys.stderr)
return
if _looks_like_dex(data):
dex_blobs.append((os.path.basename(path), data))
sources.append(path)
elif _looks_like_elf(data):
elf_blobs.append((os.path.basename(path), data))
sources.append(path)
else:
print("warning: %s is neither a dex nor an ELF image, skipped" % path,
file=sys.stderr)
for path in args.dex or []:
add_file(path)
for path in args.so or []:
add_file(path)
apks = list(args.apk or [])
for directory in (args.dir or []):
for root, _dirs, files in os.walk(directory):
for fn in files:
full = os.path.join(root, fn)
low = fn.lower()
if low.endswith(".apk"):
apks.append(full)
elif low.endswith((".dex", ".so")):
add_file(full)
for apk in apks:
if not os.path.exists(apk):
print("warning: apk not found: %s" % apk, file=sys.stderr)
continue
sources.append(apk)
try:
with zipfile.ZipFile(apk) as zf:
for info in zf.infolist():
low = info.filename.lower()
if not (low.endswith(".dex") or low.endswith(".so")):
continue
if info.file_size > 256 * 1024 * 1024:
print("warning: skipping oversized entry %s (%d B)"
% (info.filename, info.file_size), file=sys.stderr)
continue
data = zf.read(info)
label = os.path.basename(info.filename)
if _looks_like_dex(data):
dex_blobs.append((label, data))
elif _looks_like_elf(data):
elf_blobs.append((label, data))
except zipfile.BadZipFile:
print("warning: %s is not a readable zip/apk" % apk, file=sys.stderr)
return dex_blobs, elf_blobs, sources
def main(argv=None):
parser = argparse.ArgumentParser(
description="Collect the evidence that separates Java2C from an extraction "
"shell, a VMP and ordinary JNI sinking.",
epilog="Weak criteria (hits that do NOT establish a verdict on their own): "
"JNI_OnLoad, RegisterNatives, libc++_shared.so, FindClass/GetMethodID, "
"and a bare 'dcc' substring. The trivial-body ratio is advisory: it "
"flags extraction shells and VMPs, never Java2C.",
)
parser.add_argument("--apk", action="append", help="APK to inspect (repeatable)")
parser.add_argument("--dex", action="append", help="dex image (repeatable)")
parser.add_argument("--so", action="append", help="ELF library (repeatable)")
parser.add_argument("--dir", action="append",
help="directory scanned recursively for *.apk/*.dex/*.so")
parser.add_argument("--json", action="store_true", help="emit machine-readable JSON")
args = parser.parse_args(argv)
if not any([args.apk, args.dex, args.so, args.dir]):
parser.error("at least one of --apk/--dex/--so/--dir is required")
dex_blobs, elf_blobs, sources = collect(args)
if not dex_blobs and not elf_blobs:
print("error: no usable dex or ELF input found", file=sys.stderr)
return 1
dexes = [Dex(data, name) for name, data in dex_blobs]
elves = [Elf(data, name) for name, data in elf_blobs]
result = classify(dexes, elves, sum(len(d) for _n, d in elf_blobs))
report = {
"inputs": sources,
"dex": [d.summary() for d in dexes],
"native": [e.summary() for e in elves],
"classification": result,
}
if args.json:
print(json.dumps(report, indent=2, sort_keys=False))
return 0
print("=" * 72)
print("java2c_probe: hardening classification evidence")
print("=" * 72)
print("inputs: %d file(s)" % len(sources))
for src in sources:
print(" - %s" % src)
print("\n[dex layer]")
if not dexes:
print(" (no dex supplied: the dex-layer metrics are unmeasured, so a")
print(" Java2C reading cannot be supported or excluded from this input)")
for d in dexes:
s = d.summary()
if s["parse_error"]:
print(" %s: PARSE ERROR: %s" % (s["name"], s["parse_error"]))
continue
print(" %s: %d B, %d classes, %d methods, native %d (%.2f%%), "
"native-dominated classes %d, fully native %d"
% (s["name"], s["size"], s["classes"], s["total_methods"],
s["native_methods"], s["native_ratio"] * 100,
s["native_dominated_classes"], s["fully_native_classes"]))
print(" no-code methods %d, trivial bodies %d/%d (%.0f%%), header %s"
% (s["no_code_methods"], s["trivial_bodies"], s["body_methods"],
s["trivial_ratio"] * 100,
"OK" if s["header_size_matches"] else "MISMATCH: " + s["header_note"]))
print("\n[native layer]")
if not elves:
print(" (no .so supplied)")
for e in elves:
s = e.summary()
if not s["is_elf"]:
print(" %s: not ELF (%s)" % (s["name"], s["elf_note"]))
continue
print(" %s: %d B, %s, dynsym %d entries, Java_* %d, JNI_OnLoad %s, "
"RegisterNatives-import %s"
% (s["name"], s["size"], s["arch"], s["dynsym_entries"],
s["java_symbols"], s["exports_JNI_OnLoad"],
s["imports_RegisterNatives"]))
if s["elf_note"]:
print(" note: %s" % s["elf_note"])
if s["signature_hits"]:
for label, info in s["signature_hits"].items():
print(" [%s] %s x%d - %s"
% (info["strength"], label, info["count"], info["meaning"]))
print("\n[classification]")
print(" verdict : %s" % result["verdict"])
print(" confidence : %s" % result["confidence"])
print("\n evidence:")
for item in result["evidence"]:
print(" (%s) %s" % (item["strength"], item["claim"]))
print(" note: %s" % item["note"])
print("\n route:")
for line in result["route"]:
print(" - %s" % line)
print("\n weak criteria - a hit here is NOT a verdict:")
for line in result["weak_criteria_warning"]:
print(" ! %s" % line)
print("=" * 72)
return 0
if __name__ == "__main__":
sys.exit(main())

936
vendor-new/apk-reverse/skills/apk-reverse/scripts/kernelsu_syscall_mask.py Normal file
View File

@@ -0,0 +1,936 @@
#!/usr/bin/env python3
"""kernelsu_syscall_mask.py -- generate a configurable kernel-side syscall-masking scaffold.
What this generates, and the correction it exists to make
---------------------------------------------------------
There is a widespread misreading worth killing early: **a KernelSU module is a
Magisk-compatible userspace module and cannot change what a syscall returns.**
Its `post-fs-data.sh` / `service.sh` run as root in the normal world; nothing in
that format reaches the kernel's return path. Everything a module of that shape
can do is userspace work -- bind mounts, setting props, starting a daemon,
writing config.
Actually rewriting what `openat`/`read`/`stat` return for `/proc/self/maps` or
`/proc/self/status` needs one of exactly three things:
kpm KernelPatch / APatch Kernel Patch Module -- pointer replacement in the
syscall table (`fp_hook_syscalln`) or inline hooks (`hook_wrapN`).
Built with a bare-metal ARM64 toolchain (aarch64-none-elf-gcc) into a
relocatable .kpm ELF. Needs a KernelPatch-patched boot image.
lkm An out-of-tree kernel module. Needs kernel source matching the device's
exact version and vermagic, and a loader path.
ebpf A BPF program attached to a syscall tracepoint or kprobe. Needs a GKI
kernel with the tracing/BTF machinery -- 5.10+ in Android practice.
So this script emits a **module directory that is honest about which half is
loadable**: the userspace skeleton (module.prop + scripts + config) is real and
installable, and each kernel-side target is a template with its toolchain gate
stated in its own header. None of the kernel-side code is compiled or loaded
anywhere by this script.
What is measured vs unverified
------------------------------
`generate`, `gates` and `verify` are exercised on the host (see
docs/tool-verification/EXTENSION-kernel-weapons.md). **Whether the kernel-side
code compiles or loads is unverified** -- that needs a device with the matching
kernel and the matching toolchain, and the reference device is a 4.14.186 Magisk
build where all three routes are closed.
Examples
python kernelsu_syscall_mask.py gates
python kernelsu_syscall_mask.py generate --out work/sysmask --package <PKG>
python kernelsu_syscall_mask.py verify --dir work/sysmask
"""
import argparse
import json
import os
import re
import sys
# ---------------------------------------------------------------------------
# Kernel gate table. Every number here is either a toolchain fact or a
# documented platform threshold; the "how to check" column is what turns it
# from trivia into a test you run on your own device.
# ---------------------------------------------------------------------------
GATES = [
{
"route": "eBPF tracepoint/kprobe",
"gate": "GKI kernel 5.10+ (Android 12+)",
"why": "Android's BPF/tracing support as bpftrace-class tooling and "
"stackplz expect it is a GKI-era feature; BTF is required for "
"CO-RE style probes.",
"check": "uname -r -> expect 5.10.x or newer; ls /sys/kernel/btf/vmlinux",
"ref": "https://source.android.com/docs/core/architecture/kernel/bpf",
},
{
"route": "KernelPatch / APatch KPM",
"gate": "KernelPatch-patched boot image + kpm toolchain",
"why": "KPMs load through kpimg injected into the kernel image's payload "
"segment; building one needs a bare-metal ARM64 compiler "
"(aarch64-none-elf-gcc), not the NDK.",
"check": "ls /data/adb/ | grep -i apatch ; which aarch64-none-elf-gcc",
"ref": "https://github.com/bmax121/KernelPatch",
},
{
"route": "Out-of-tree LKM",
"gate": "Kernel source matching the device's exact version + vermagic",
"why": "A module built against a different kernel revision is refused at "
"load time on a mismatched vermagic, and the device's kernel "
"source is frequently not published at all.",
"check": "uname -r ; look for a matching kernel source tree for the device",
"ref": "https://source.android.com/docs/core/architecture/kernel/android-common",
},
{
"route": "seccomp-BPF (for contrast)",
"gate": "any modern kernel, but it is not this route",
"why": "seccomp can make a syscall FAIL (SECCOMP_RET_ERRNO/TRAP); it "
"cannot rewrite the CONTENT a successful read returns. It closes "
"doors, it does not paint them -- which is the whole requirement "
"for a spoofed /proc read.",
"check": "n/a -- this is a capability ceiling, not a version check",
"ref": "references/kernel-and-environment-hardening.md",
},
]
DEFAULT_RULES = [
{
"syscall": "openat",
"path": "/proc/self/maps",
"action": "deny",
"errno": "ENOENT",
"note": "hides the mapping list a userspace instrumentation agent "
"advertises itself in",
},
{
"syscall": "newfstatat",
"path": "/proc/self/maps",
"action": "fake_size",
"size": 0,
"note": "a stat that reports 0 bytes is as good as absent for a reader "
"that sizes its buffer first",
},
{
"syscall": "openat",
"path": "/proc/self/status",
"action": "filter_line",
"prefix": "TracerPid",
"note": "the write side owns this field; see "
"references/kernel-and-environment-hardening.md section 5",
},
{
"syscall": "read",
"path": "/proc/self/status",
"action": "filter_line",
"prefix": "TracerPid",
"note": "line filter applied to the read once the fd is open",
},
]
MODULE_PROP = """id={id}
name={name}
version={version}
versionCode=1
author=apk-reverse
description={description}
"""
POST_FS_DATA = """#!/system/bin/sh
# Runs at post-fs-data, as root, in the normal world.
# The kernel side (if you built and loaded one) owns the syscall table; this
# script's whole job is to make the configuration visible to it and to leave a
# record that the module actually ran. Do not put a syscall hook here: this is
# userspace, and nothing here can change what openat/read/stat return.
MODDIR=${{0%/*}}
CONF=$MODDIR/config/syscall_mask.json
LOG=/data/adb/{id}.log
if [ ! -f "$CONF" ]; then
echo "$(date) missing $CONF" >> "$LOG"
exit 0
fi
echo "$(date) post-fs-data: config present, $(wc -c < "$CONF") bytes" >> "$LOG"
# The KPM/LKM loader path is intentionally NOT invoked from here. Loading a
# kernel module is a boot-image-level decision with a bricking risk; do it once,
# by hand, and read the evidence file first.
exit 0
"""
SERVICE = """#!/system/bin/sh
# Runs in late_start service mode, as root. Kept minimal and non-blocking on
# purpose: a module that spins here delays boot on every start, and this one
# only reports state.
MODDIR=${{0%/*}}
CONF=$MODDIR/config/syscall_mask.json
LOG=/data/adb/{id}.log
if [ -r /proc/sys/kernel/version ]; then
echo "$(date) service: kernel $(uname -r)" >> "$LOG"
fi
echo "$(date) service: rules=$(grep -c '"syscall"' "$CONF" 2>/dev/null)" >> "$LOG"
exit 0
"""
UNINSTALL = """#!/system/bin/sh
# Uninstall hook. Deliberately does not attempt to unload a kernel module: if one
# is loaded, unloading it is a separate deliberate act with its own risk, and a
# silent unload during package removal is how a device ends up in a boot loop.
rm -f /data/adb/{id}.log
exit 0
"""
CUSTOMIZE = """#!/system/bin/sh
# KernelSU/Magisk install-time script. Kept as a no-op so the module always
# installs; see the module README for why nothing is set up here.
exit 0
"""
KPM_C = r'''// SPDX-License-Identifier: GPL-2.0
/*
* syscall_mask.c -- config-driven syscall return masking, KernelPatch/APatch KPM.
*
* Generated by skills/apk-reverse/scripts/kernelsu_syscall_mask.py
*
* STATUS: TEMPLATE, NOT COMPILED AND NOT LOADED. Every hook below follows the
* KernelPatch KPM interface as documented in public KPM development write-ups
* (KPM_NAME/KPM_INIT/KPM_EXIT sections, kfunc_def symbol resolution without
* `extern`, fp_hook_syscalln pointer replacement, hook_fargs4_t callbacks with
* args->skip_origin / args->ret). VERIFY EVERY NAME AND SIGNATURE against the
* kpmodule.h of the KernelPatch revision you build against -- this file was not
* compiled anywhere, and an interface that has drifted produces a module that
* fails to load, or worse, loads and faults.
*
* Build (bare-metal ARM64, NOT the NDK):
* aarch64-none-elf-gcc -O2 -fno-stack-protector -c syscall_mask.c
* ... link into a relocatable ELF per the KernelPatch module docs
* Load: push the .kpm and add it through the APatch app's KPM manager.
*
* Design notes that are load-bearing rather than stylistic:
* - Callbacks run on EVERY syscall of their number, system-wide. The first
* thing each one does is reject the overwhelmingly common case, because a
* hook that does real work on every write is how a device locks up.
* - Inline hooks (hook_wrapN) rewrite the target's prologue. On an LTO kernel
* the exported symbol is frequently NOT the call site (the call got inlined),
* so an inline hook installs cleanly and never fires. Pointer replacement in
* the syscall table (fp_hook_syscalln) does not have that failure mode,
* because the syscall entry path must go through the table.
* - Never sleep in these callbacks except where the syscall itself runs in
* process context and the copy helper is the sleeping variant.
*/
#include <linux/types.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <kpmodule.h>
KPM_NAME("{id}");
KPM_VERSION("{version}");
KPM_LICENSE("GPL");
KPM_AUTHOR("apk-reverse");
KPM_DESCRIPTION("{description}");
/* kfunc declarations must NOT carry `extern`: an extern declaration compiles to
* an undefined reference (*UND*) and the loader rejects the module with
* "unknown symbol". Letting it be a tentative definition allocates the slot in
* .bss and the loader fills it in. Measured and documented in public KPM work. */
int kfunc_def(strncpy_from_user)(char *dst, const char __user *src, long count);
/* ------------------------------------------------------------------ rules --
* Generated from config/syscall_mask.json. Keep this table in sync by
* re-running the generator rather than by hand-editing it.
*/
#define MASK_DENY 1 /* return -err before the real syscall runs */
#define MASK_FAKE_SIZE 2 /* openat succeeds, stat reports a bogus size */
#define MASK_FILTER_LINE 3 /* strip lines with this prefix from read() */
struct mask_rule {
const char *path; /* userspace path to match, NULL = any */
const char *prefix; /* line prefix for MASK_FILTER_LINE, else NULL */
int action;
int err; /* -errno for MASK_DENY */
long value; /* size for MASK_FAKE_SIZE */
};
static const struct mask_rule rules[] = {
{RULE_TABLE}
};
#define NRULES (sizeof(rules) / sizeof(rules[0]))
/* A bounded, allocation-free matcher. The path is copied in first, so the
* comparison never touches userspace after the initial copy -- touching
* userspace memory twice is how a hook races with the caller's own unmapping. */
static int copy_and_match(const char __user *upath, char *kbuf, unsigned long n,
const char **matched)
{{
long copied;
unsigned long i;
if (!upath || n == 0)
return 0;
if (!kfunc(strncpy_from_user))
return 0; /* symbol unresolved: fail open */
copied = kfunc(strncpy_from_user)(kbuf, upath, (long)(n - 1));
if (copied <= 0)
return 0;
kbuf[copied] = '\0';
for (i = 0; i < NRULES; i++) {{
if (!rules[i].path)
continue;
if (strstr(kbuf, rules[i].path)) {{
*matched = rules[i].path;
return rules[i].action;
}}
}}
return 0;
}}
/* ------------------------------------------------------------------ openat --
* The path argument is arg2 on arm64 (dfd, path, flags, mode). If the path is
* unreadable, the rule is skipped rather than guessed at.
*/
static void before_openat(hook_fargs4_t *args, void *udata)
{{
const char __user *upath = (const char __user *)syscall_argn(args, 1);
char kbuf[256];
const char *matched = 0;
int action;
(void)udata;
action = copy_and_match(upath, kbuf, sizeof(kbuf), &matched);
if (action == MASK_DENY)
args->ret = MASK_DENY_ERRNO;
if (action == MASK_FAKE_SIZE)
args->skip_origin = 0; /* let it open; the stat is spoofed */
}}
/* --------------------------------------------------------------- newfstatat --
* arm64 has no plain `stat` syscall; readers go through newfstatat (or fstatat64
* on 32-bit). A zero size is what makes a reader that sizes its buffer first
* decide there is nothing to read.
*/
static void after_newfstatat(hook_fargs4_t *args, void *udata)
{{
const char __user *upath = (const char __user *)syscall_argn(args, 1);
char kbuf[256];
const char *matched = 0;
(void)udata;
if (copy_and_match(upath, kbuf, sizeof(kbuf), &matched) != MASK_FAKE_SIZE)
return;
/* The struct stat* is arg2; zeroing st_size is the whole point. Field
* offsets are ABI-specific -- resolve them against the target's headers
* rather than trusting an offset copied from a different architecture. */
/* TODO(verify-on-device): zero the size field of the struct at arg2. */
}}
/* -------------------------------------------------------------------- read --
* Line filtering happens on the read side, once the fd is already open. Doing
* it here rather than at openat is what lets a reader that opens the file
* before the filter is installed still be covered.
*/
static void before_read(hook_fargs4_t *args, void *udata)
{{
long count = (long)syscall_argn(args, 2);
(void)udata;
if (count <= 0 || count > 8192)
return; /* first filter: the common case */
/* TODO(verify-on-device): identify the fd's path, copy the buffer, rewrite
* it in place minus the filtered lines, and adjust the returned length. */
}}
/* ------------------------------------------------------------------ init -----
* Registration order matters: install the syscall hooks first, then enable the
* rules. If the module is unloaded between the two, the hooks must already be
* gone.
*/
static long mask_init(const char *args, const struct kernel_patch_funcs *kp)
{{
(void)args;
(void)kp;
if (!kfunc(strncpy_from_user)) {{
/* Failing loudly here is deliberate: a half-armed hook set that cannot
* read a path would pass every check through, which looks like success
* and is not. */
return -EINVAL;
}}
/* fp_hook_syscalln(__NR_openat, before_openat, 0, 0); */
/* fp_hook_syscalln(__NR_newfstatat, before_newfstatat, 0, 0); */
/* fp_hook_syscalln(__NR_read, before_read, 0, 0); */
/* TODO(verify-on-device): the argument lists above follow the public
* examples; confirm arity against your kpmodule.h before enabling them. */
return 0;
}}
static long mask_exit(void *udata)
{{
(void)udata;
/* Unhook in the exact reverse order of registration. */
return 0;
}}
KPM_INIT(mask_init);
KPM_EXIT(mask_exit);
'''
LKM_C = r'''// SPDX-License-Identifier: GPL-2.0
/*
* syscall_mask.c -- out-of-tree LKM variant of the same idea.
*
* STATUS: TEMPLATE, NOT COMPILED. Worse than the KPM variant in one specific
* way worth understanding before choosing it: to replace a syscall you must
* reach the syscall table, and `kallsyms_lookup_name` stopped being exported in
* Linux 5.7. On pre-5.7 kernels a direct call works; after that you need a
* kprobe to recover the address, or an explicit kernel export. The device's own
* kernel source must also match its exact vermagic, and an out-of-tree module
* built against a mismatched tree is refused at insmod.
*
* This is genuine kernel development with a bricking risk. It is the most
* expensive of the three routes and the one least likely to be worth it.
*/
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/syscalls.h>
#include <linux/uaccess.h>
#include <linux/version.h>
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("{description}");
MODULE_AUTHOR("apk-reverse");
/*
* Pre-5.7: kallsyms_lookup_name() is exported, so the table pointer is
* recoverable directly. 5.7+: NOT exported -- you must either carry a kprobe to
* find the symbol or patch the kernel's export list. There is no portable
* version of this, which is the honest reason this route is rated last.
*/
#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 7, 0)
static unsigned long *sys_call_table_addr(void)
{
return (unsigned long *)kallsyms_lookup_name("sys_call_table");
}
#else
static unsigned long *sys_call_table_addr(void)
{
/* TODO(verify-on-device): recover via kprobe on kallsyms_lookup_name, or
* enable an explicit export in your own kernel build. */
return NULL;
}
#endif
static int __init mask_init(void)
{
unsigned long *table = sys_call_table_addr();
if (!table) {
pr_err("syscall_mask: cannot locate sys_call_table on this kernel\n");
return -EINVAL;
}
pr_info("syscall_mask: sys_call_table at %px\n", table);
/* TODO(verify-on-device): save the original pointer, write CR0 (or use
* set_memory_rw) to make the table writable, swap in your handler. Getting
* the write-protect dance wrong takes the device down. */
return 0;
}
static void __exit mask_exit(void)
{
/* TODO(verify-on-device): restore the saved pointers. */
}
module_init(mask_init);
module_exit(mask_exit);
'''
EBPF_C = r'''// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
/*
* syscall_mask.bpf.c -- eBPF probe template: filter sys_enter_openat, then
* rewrite the result.
*
* STATUS: TEMPLATE, NOT COMPILED AND NOT ATTACHED. Not runnable on a 4.14
* kernel: see the gate table in the module README (GKI 5.10+ in Android
* practice, plus BTF for CO-RE style probes).
*
* READ THIS BEFORE COUNTING ON THE RESULT REWRITE. There are two different
* attachment points and they are not interchangeable:
*
* - A *tracepoint* (syscalls/sys_enter_openat) is an observation point. It can
* read the arguments and it can emit events, and it CANNOT change the
* syscall's return value. A tracepoint-only program gives you visibility,
* not spoofing.
* - A *kprobe* on the syscall's entry (or a kretprobe on its exit) can modify
* a register with bpf_override_return() -- but only for functions flagged
* ALLOW_ERROR_INJECTION, which kernel syscall entry points are not, in
* general. On many kernels the helper is refused for exactly this target.
*
* So the honest eBPF shape for this job is: observe with a tracepoint, and do
* the rewrite in a place that actually owns the return value. What follows is
* the observation half done properly, plus the rewrite half marked for the
* device-side verification it needs.
*/
#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
char LICENSE[] SEC("license") = "GPL";
#define MAX_PATH 256
#define TARGET_LEN 15 /* strlen("/proc/self/maps") */
struct event {
__u32 pid;
__u32 uid;
__s32 dfd;
__s64 ret;
char path[MAX_PATH];
char comm[16];
};
/* /proc/self/status is 17; kept as a separate constant so the two rules can be
* toggled independently. */
#define TARGET_STATUS_LEN 17
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 1 << 24);
} events SEC(".maps");
/* Filter the target package's processes. Populated from userspace with the
* uids the app can run as; an empty map means "all processes", which is a
* deliberate default because a filter that silently matches nothing looks
* exactly like a probe that is not firing. */
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 64);
__type(key, __u32); /* uid */
__type(value, __u8);
} target_uids SEC(".maps");
static __always_inline int want(void)
{
__u32 uid = bpf_get_current_uid_gid() & 0xffffffff;
__u8 *hit = bpf_map_lookup_elem(&target_uids, &uid);
/* Empty map -> observe everything. Populated map -> only listed uids. */
return hit || (bpf_map_lookup_elem(&target_uids, &uid) == 0);
}
SEC("tracepoint/syscalls/sys_enter_openat")
int tp_sys_enter_openat(struct trace_event_raw_sys_enter *ctx)
{
struct event *e;
const char *upath = (const char *)ctx->args[1];
if (!want())
return 0;
e = bpf_ringbuf_reserve(&events, sizeof(*e), 0);
if (!e)
return 0;
e->pid = bpf_get_current_pid_tgid() >> 32;
e->uid = bpf_get_current_uid_gid() & 0xffffffff;
e->dfd = (__s32)ctx->args[0];
e->ret = 0;
bpf_get_current_comm(&e->comm, sizeof(e->comm));
bpf_probe_read_user_str(&e->path, sizeof(e->path), upath);
bpf_ringbuf_submit(e, 0);
return 0;
}
/*
* The result-rewrite half. Two dead ends are recorded here on purpose so they
* are not re-derived:
*
* 1. bpf_override_return() only works on functions marked
* ALLOW_ERROR_INJECTION. A raw syscall entry is not one of them, so the
* helper is typically refused for this target -- check the return value
* and the verifier message rather than assuming the rewrite happened.
* 2. A tracepoint cannot rewrite at all. If the rewrite is essential, the
* route is a kprobe plus a kernel that permits the override, or one of the
* KPM/LKM routes; eBPF's strength here is observation.
*/
SEC("kprobe/do_sys_openat2")
int BPF_KPROBE(kp_do_sys_openat2, int dfd, const char *filename)
{
char buf[64];
if (bpf_probe_read_user_str(&buf, sizeof(buf), filename) <= 0)
return 0;
if (buf[0] == '/' && buf[1] == 'p' && buf[2] == 'r') {
/* TODO(verify-on-device): bpf_override_return(ctx, -ENOENT) here only
* after confirming ALLOW_ERROR_INJECTION on this target kernel. */
}
return 0;
}
'''
KPM_MAKEFILE = """# aarch64-none-elf-gcc is a BARE-METAL ARM64 compiler, not the Android NDK
# toolchain. The NDK targets Android userspace and will not produce a loadable
# KPM. STATUS: untested -- this toolchain is not present on a Magisk-only device.
CROSS ?= aarch64-none-elf-
CC := $(CROSS)gcc
OBJCOPY := $(CROSS)objcopy
TARGET := {id}
KPM_DIR ?= ../KernelPatch
CFLAGS := -O2 -fno-stack-protector -fno-pic -fno-builtin \\
-I$(KPM_DIR)/kernel/include -I$(KPM_DIR)/include \\
-Wall -Wno-unused-variable
all: $(TARGET).kpm
$(TARGET).kpm: src/{id}.c
\t$(CC) $(CFLAGS) -c $< -o $@.o
\t$(OBJCOPY) -O binary --only-section=.text $@.o /dev/null 2>/dev/null || true
\t@echo "NOTE: linking follows the KernelPatch module format; see the docs"
\t@echo " for the current link step before trusting this target."
clean:
\trm -f *.o *.kpm
.PHONY: all clean
"""
MODULE_README = """# {name} -- syscall-return masking scaffold
Generated by `skills/apk-reverse/scripts/kernelsu_syscall_mask.py`.
**Read this first: the userspace module in this directory cannot change what a
syscall returns.** It is a KernelSU/Magisk-compatible module: its scripts run as
root in the normal world. Its only jobs are to carry the configuration, to
record that it ran, and to be a place to put the module metadata. The kernel
side is a separate artifact with a separate toolchain and a separate risk.
## What is here
| Path | Status |
|---|---|
| `module.prop`, `customize.sh`, `post-fs-data.sh`, `service.sh` | real, installable module skeleton |
| `config/syscall_mask.json` | the rule set, read by the kernel side |
| `kpm/{id}.c`, `kpm/Makefile` | APatch/KernelPatch template -- **not compiled, not loaded** |
| `lkm/{id}.c` | out-of-tree module template -- **not compiled, not loaded** |
| `ebpf/syscall_mask.bpf.c` | eBPF probe template -- **not compiled, not attached** |
## Gate table -- check your device before choosing a route
| Route | Gate | How to check | Why |
|---|---|---|---|
{gates}
## Reference device, measured
```
$ adb shell 'uname -r; cat /proc/version'
4.14.186+
Linux version 4.14.186+ (nobody@android-build) (Android (6443078 based on r383902)
clang version 11.0.1 ...) #1 SMP PREEMPT Wed Mar 30 23:32:42 CST 2022
```
On that device **every kernel-side route above is closed**:
- eBPF: kernel 4.14 is well below the 5.10 gate.
- KPM: no KernelPatch-patched image, and no `aarch64-none-elf-gcc` on the host;
installing APatch means patching the boot image, which is a bricking risk.
- LKM: no kernel source for the device, so no matching vermagic is possible.
That is a **route decision, not a failure** -- and it is why every kernel-side
file here is labelled a template. Nothing in this directory was loaded on
anything.
## If you do take a kernel route
Do it once, deliberately, and read the evidence first:
1. Confirm the gate in the table above on the actual device.
2. Build the kernel side on a machine that has the right toolchain.
3. **Back up the boot image and know the recovery path before loading anything.**
A kernel module that faults takes the device down before you can read why.
4. Load it by hand, watch `dmesg`, and keep the previous boot image in hand.
5. Only then wire it into boot, and only if the load is repeatable.
`post-fs-data.sh` in this scaffold deliberately does **not** load anything. That
is not an omission.
"""
def _render(template, **kw):
"""Fill a template's {placeholders} without str.format().
Braces are the common case in C, so using format() here would mean doubling
every one of them -- and the doubled-brace convention fails silently in the
other direction the moment somebody next edits the C by hand. `{{` is
unescaped and only the named placeholders are substituted, so ordinary C
braces pass through untouched.
"""
text = template.replace("{{", "{").replace("}}", "}")
for key, value in kw.items():
text = text.replace("{%s}" % key, str(value))
return text
def _rule_table_c(rules):
"""Render the JSON rule set as a C table."""
errno_map = {"ENOENT": 2, "EACCES": 13, "EINVAL": 22, "EPERM": 1}
action_map = {"deny": "MASK_DENY", "fake_size": "MASK_FAKE_SIZE",
"filter_line": "MASK_FILTER_LINE"}
lines = []
for rule in rules:
path = rule.get("path")
prefix = rule.get("prefix")
action = action_map.get(rule.get("action"), "MASK_DENY")
err = errno_map.get(rule.get("errno", "ENOENT"), 2)
value = rule.get("size", 0)
lines.append(' { %s, %s, %s, -%d, %d },'
% ('"%s"' % path if path else "0",
'"%s"' % prefix if prefix else "0",
action, err, value))
return "\n".join(lines) if lines else " /* no rules configured */"
def cmd_gates(args):
"""Report the kernel-route gate table, and probe this host where it can be probed."""
print("== kernel-side routes and their gates ==\n")
for g in GATES:
print(" %-28s %s" % (g["route"], g["gate"]))
print(" %-28s why: %s" % ("", g["why"]))
print(" %-28s check: %s" % ("", g["check"]))
print()
print("== host probe (what can be checked from here) ==\n")
cc = None
import shutil
for cand in ("aarch64-none-elf-gcc", "aarch64-linux-gnu-gcc", "clang"):
found = shutil.which(cand)
if found:
cc = (cand, found)
break
if cc:
print(" bare-metal ARM64-ish compiler: %s -> %s" % cc)
if cc[0] == "clang":
print(" note: clang can target aarch64-none-elf, so this is "
"potentially usable for the KPM route")
else:
print(" bare-metal ARM64 compiler: NOT FOUND")
print(" the KPM route cannot be built from this host as configured")
for name in ("ndk-build", "make"):
print(" %-12s %s" % (name + ":", shutil.which(name) or "NOT FOUND"))
print("\n A kernel source tree, a patched boot image and the device's own "
"kernel headers cannot be checked from the host; they are device facts.")
print(" Run the `check` command for each route on the device before "
"believing any of this applies to it.")
return 0
def cmd_generate(args):
out = args.out
rules = DEFAULT_RULES
if args.rules:
with open(args.rules, encoding="utf-8") as fh:
rules = json.load(fh)
desc = args.description
os.makedirs(os.path.join(out, "config"), exist_ok=True)
written = []
def put(rel, text, mode=None):
path = os.path.join(out, rel)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8", newline="\n") as fh:
fh.write(text)
if mode is not None:
try:
os.chmod(path, mode)
except OSError:
pass
written.append(rel)
put("module.prop", _render(MODULE_PROP,
id=args.id, name=args.name, version=args.version,
description=desc))
put("customize.sh", CUSTOMIZE, 0o755)
put("post-fs-data.sh", _render(POST_FS_DATA, id=args.id), 0o755)
put("service.sh", _render(SERVICE, id=args.id), 0o755)
put("uninstall.sh", _render(UNINSTALL, id=args.id), 0o755)
put("config/syscall_mask.json", json.dumps(
{"version": 1, "id": args.id, "package": args.package,
"enabled": True, "note": "read by the kernel side, not by the shell "
"scripts -- see the module README",
"rules": rules}, indent=2) + "\n")
want = set(args.targets.split(","))
if "kpm" in want:
body = _render(KPM_C, id=args.id, version=args.version, description=desc,
RULE_TABLE=_rule_table_c(rules))
put("kpm/%s.c" % args.id, body)
put("kpm/Makefile", _render(KPM_MAKEFILE, id=args.id))
if "lkm" in want:
put("lkm/%s.c" % args.id, _render(LKM_C, id=args.id, description=desc))
if "ebpf" in want:
put("ebpf/syscall_mask.bpf.c", EBPF_C)
gate_rows = "\n".join("| %s | %s | `%s` | %s |"
% (g["route"], g["gate"], g["check"], g["why"])
for g in GATES)
put("README.md", _render(MODULE_README, name=args.name, gates=gate_rows,
id=args.id))
print("generated module scaffold in %s" % out)
for rel in written:
size = os.path.getsize(os.path.join(out, rel))
print(" %-28s %7d B" % (rel, size))
print()
print("STATUS: the userspace module is installable; every kernel-side file is "
"an UNVERIFIED template.")
print(" No kernel code was compiled or loaded. Run `gates` and read the "
"device README")
print(" section before assuming any route is open on your target.")
return 0
def _strip_c_comments(text):
"""Remove /* */ and // comments before any token-level check.
Not cosmetic: the first version of this check reported a false positive
because the template's own comment explains why `extern` is forbidden, and a
naive `extern.*kfunc_def` regex happily matched the prose and then ran on to
the real declaration's semicolon. A linter that reads comments as code
produces exactly the kind of report nobody can trust.
"""
text = re.sub(r"/\*.*?\*/", "", text, flags=re.S)
return re.sub(r"//[^\n]*", "", text)
def cmd_verify(args):
"""Read-only checks over a generated directory. Everything here is local."""
d = args.dir
problems = []
print("== verifying %s ==" % d)
prop = os.path.join(d, "module.prop")
if not os.path.isfile(prop):
problems.append("module.prop missing")
else:
fields = {}
with open(prop, encoding="utf-8") as fh:
for line in fh:
if "=" in line:
k, v = line.rstrip("\n").split("=", 1)
fields[k] = v
for req in ("id", "name", "version", "versionCode"):
if req not in fields:
problems.append("module.prop missing required field %r" % req)
if "id" in fields and not re.fullmatch(r"[a-zA-Z][a-zA-Z0-9._-]*", fields["id"]):
problems.append("module.prop id %r is not a valid module id" % fields["id"])
print(" module.prop: %s" % ", ".join("%s=%s" % kv for kv in sorted(fields.items())))
conf = os.path.join(d, "config", "syscall_mask.json")
if not os.path.isfile(conf):
problems.append("config/syscall_mask.json missing")
else:
try:
with open(conf, encoding="utf-8") as fh:
doc = json.load(fh)
rules = doc.get("rules", [])
known = {"deny", "fake_size", "filter_line"}
for i, r in enumerate(rules):
if r.get("action") not in known:
problems.append("rule %d has unknown action %r" % (i, r.get("action")))
if not r.get("syscall"):
problems.append("rule %d names no syscall" % i)
print(" config: %d rule(s), actions=%s"
% (len(rules), sorted({r.get("action") for r in rules})))
except (ValueError, OSError) as exc:
problems.append("config/syscall_mask.json unreadable: %s" % exc)
for rel, want_exec in (("post-fs-data.sh", True), ("service.sh", True),
("customize.sh", True), ("uninstall.sh", True)):
p = os.path.join(d, rel)
if not os.path.isfile(p):
problems.append("%s missing" % rel)
continue
if want_exec and not os.access(p, os.X_OK):
# Not fatal on a filesystem without an exec bit; the installer sets it.
print(" note: %s is not executable here (the installer chmods it)" % rel)
for rel in ("kpm", "lkm", "ebpf"):
p = os.path.join(d, rel)
if os.path.isdir(p):
print(" %-4s target present: %s"
% (rel, ", ".join(sorted(os.listdir(p)))))
print(" status: UNVERIFIED TEMPLATE -- not compiled, not loaded")
# The one static check worth doing on the C: the KPM interface has a
# documented trap (an extern kfunc declaration compiles to *UND* and the
# loader refuses the module), so catch it here rather than on a device.
kpm_dir = os.path.join(d, "kpm")
if os.path.isdir(kpm_dir):
for name in sorted(os.listdir(kpm_dir)):
if not name.endswith(".c"):
continue
with open(os.path.join(kpm_dir, name), encoding="utf-8") as fh:
text = fh.read()
code = _strip_c_comments(text)
if re.search(r"\bextern\b[^;]*\bkfunc_def\b", code):
problems.append("%s declares kfunc_def with `extern`, which the "
"loader rejects as an unknown symbol" % name)
for macro in ("KPM_NAME", "KPM_INIT", "KPM_EXIT"):
if macro not in code:
problems.append("%s is missing %s" % (name, macro))
print(" kpm/%s: KPM_* lifecycle macros present, kfunc_def not extern"
% name)
print()
if problems:
print("== %d problem(s) ==" % len(problems))
for p in problems:
print(" - %s" % p)
return 1
print("== 0 problem(s): structure is consistent ==")
print(" (this says nothing about whether the kernel side compiles or loads)")
return 0
def main(argv=None):
parser = argparse.ArgumentParser(
prog="kernelsu_syscall_mask.py",
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
sub = parser.add_subparsers(dest="cmd")
p = sub.add_parser("gates", help="print the kernel-route gate table and probe this host",
description="The gate table, plus whatever part of it can be "
"checked from this machine. Device facts still have "
"to be checked on the device.")
p.set_defaults(func=cmd_gates)
p = sub.add_parser("generate", help="write the module scaffold",
description="Generate a KernelSU/APatch-compatible module "
"directory. The userspace half is installable; the "
"kernel half is a template.")
p.add_argument("--out", required=True, help="directory to write into")
p.add_argument("--id", default="syscall-mask", help="module id / artifact name")
p.add_argument("--name", default="Syscall Mask", help="human-readable module name")
p.add_argument("--version", default="0.1.0", help="module version string")
p.add_argument("--description", default="config-driven syscall return masking "
"(openat/read/stat on /proc/self/*)",
help="module description")
p.add_argument("--package", default="<PKG>", help="target package placeholder")
p.add_argument("--rules", default=None,
help="JSON file with a {rules:[...]} rule set (default: built-in)")
p.add_argument("--targets", default="kpm,lkm,ebpf",
help="comma-separated kernel targets to emit (kpm,lkm,ebpf)")
p.set_defaults(func=cmd_generate)
p = sub.add_parser("verify", help="read-only consistency checks over a generated dir",
description="Check module.prop fields, the rule JSON, and the "
"static traps in the kernel template. Says nothing "
"about compilation or loading.")
p.add_argument("--dir", required=True, help="a directory from `generate`")
p.set_defaults(func=cmd_verify)
args = parser.parse_args(argv)
if not getattr(args, "cmd", None):
parser.print_help()
return 2
return args.func(args)
if __name__ == "__main__":
sys.exit(main())

216
vendor-new/apk-reverse/skills/apk-reverse/scripts/lib_map.py Normal file
View File

@@ -0,0 +1,216 @@
#!/usr/bin/env python3
"""Report which native libraries are ACTUALLY mapped into a live process.
`getprop ro.product.cpu.abi` tells you what the device claims to be. It does not
tell you what is executing. Those differ more often than people expect:
* an x86_64 emulator running an app whose only native libraries are arm64,
because an ARM translation layer is present;
* a hardened app whose real libraries are materialized into its data
directory at runtime, so the paths you see are not the paths inside the APK;
* a fat APK where the package manager picked a different ABI than you assumed,
so you patch a `.so` that never loads.
This script reads the live mapping table and reports, per library: the path, the
base address, and the architecture of the actual ELF on disk. It classifies each
path so the interesting ones stand out (system / from-APK / runtime-materialized).
Requires root to read another process's maps.
Usage:
python lib_map.py --pkg com.example.app
python lib_map.py --pkg com.example.app --app-only
python lib_map.py --pid 1234 --all
python lib_map.py --pkg com.example.app --json
python lib_map.py --pkg com.example.app --grep ssl
Reading it:
* A library whose path sits under the app's data dir (not under the APK's
lib/<abi> dir) was written at runtime. That is a packer/dropper fingerprint
and it will not be present in a repacked APK -- patch or replace with care.
* If app libraries are 64-bit ARM but the device primary ABI is x86_64,
a translator is in play. Timing and some native checks will differ.
* If the library you patched does not appear here at all, your patch cannot
matter. That is a plan problem, not a patch problem.
"""
import argparse
import json
import os
import re
import subprocess
import sys
E_MACHINE = {
0x03: "x86", 0x3E: "x86_64", 0x28: "arm", 0xB7: "aarch64",
0x08: "mips", 0xF3: "riscv", 0x16: "s390", 0x15: "ppc",
}
TRANSLATOR_MARKERS = ("houdini", "libndk", "native_bridge", "libntv", "armtrans",
"libhoudini", "ndk_translation", "libarm", "exagear", "box64")
def run(cmd: list[str], timeout: int = 60) -> str:
try:
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
return (p.stdout or "") + (p.stderr or "")
except Exception as e:
return "ERROR: %s" % e
def classify(path: str, pkg: str | None) -> str:
if path.startswith(("/system/", "/apex/", "/vendor/", "/product/", "/odm/", "/system_ext/")):
return "system"
if re.search(r"/data/app/", path):
return "apk-lib"
if pkg and (("/data/data/%s" % pkg) in path or ("/data/user/0/%s" % pkg) in path):
return "materialized"
if "/data/" in path:
return "data-other"
return "other"
def read_elf_arch(sh_fn, path: str) -> str:
"""Read the ELF header on the device and name the machine. '?' means the read failed."""
hdr = sh_fn("dd if='%s' bs=20 count=1 2>/dev/null | od -An -tx1 2>/dev/null" % path)
hexs = re.sub(r"[^0-9a-fA-F]", "", hdr)
if len(hexs) < 40:
return "?"
b = bytes.fromhex(hexs[:40])
if b[:4] != b"\x7fELF":
return "not-elf"
cls = {1: 32, 2: 64}.get(b[4], 0)
little = b[5] == 1
mach = int.from_bytes(b[18:20], "little" if little else "big")
if mach in E_MACHINE:
return E_MACHINE[mach] # the machine name already implies the width
return "0x%02x/%db" % (mach, cls or 0)
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
tgt = ap.add_mutually_exclusive_group(required=True)
tgt.add_argument("--pkg", help="target package name")
tgt.add_argument("--pid", type=int, help="target pid")
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"))
ap.add_argument("--serial", default=None)
ap.add_argument("--all", action="store_true", help="include system libraries")
ap.add_argument("--app-only", action="store_true", help="only APK libs and runtime-materialized libs")
ap.add_argument("--grep", default=None, help="only paths matching this regex")
ap.add_argument("--no-arch", action="store_true", help="skip reading ELF headers (faster)")
ap.add_argument("--max-libs", type=int, default=120, help="cap how many libraries get an arch probe")
ap.add_argument("--json", action="store_true", dest="as_json")
a = ap.parse_args()
def sh(cmd: str, timeout: int = 60) -> str:
base = [a.adb] + (["-s", a.serial] if a.serial else []) + ["shell", cmd]
return run(base, timeout)
def su(cmd: str, timeout: int = 90) -> str:
base = [a.adb] + (["-s", a.serial] if a.serial else []) + ["shell", 'su -c "%s"' % cmd]
return run(base, timeout)
pid = a.pid
if a.pkg and not pid:
out = sh("pidof %s" % a.pkg).strip()
if not out:
print("no running process for %s -- launch it and let it settle first" % a.pkg, file=sys.stderr)
return 2
pid = int(out.split()[0])
maps = su("cat /proc/%d/maps" % pid)
if not maps.strip() or "No such file" in maps:
print("cannot read /proc/%d/maps (root required, or the process is gone)" % pid, file=sys.stderr)
return 2
entries: dict[str, dict] = {}
for line in maps.splitlines():
m = re.match(r"^([0-9a-f]+)-([0-9a-f]+)\s+(\S+)\s+([0-9a-f]+)\s+\S+\s+\S+\s*(.*)$", line)
if not m:
continue
start, end, perms, off, path = m.groups()
path = path.strip()
if not path or path.startswith("["):
continue
if not path.endswith(".so") and ".so." not in path:
continue
e = entries.setdefault(path, {"path": path, "perms": set(), "base": start, "count": 0})
e["perms"].add(perms)
e["count"] += 1
if perms.startswith("r-x") or perms.startswith("r--"):
e["base"] = min(e["base"], start)
dev_abi = sh("getprop ro.product.cpu.abi").strip()
abilist = sh("getprop ro.product.cpu.abilist").strip()
pkg = a.pkg
if not pkg:
cmdline = su("cat /proc/%d/cmdline" % pid).split("\x00")[0].strip()
pkg = cmdline or None
rows = []
probed = 0
for path, e in sorted(entries.items()):
kind = classify(path, pkg)
if a.app_only and kind not in ("apk-lib", "materialized"):
continue
# --grep is an explicit narrow request, so honour it across all kinds
if not a.all and not a.app_only and not a.grep and kind == "system":
continue
if a.grep and not re.search(a.grep, path):
continue
rows.append({"path": path, "base": e["base"], "kind": kind,
"perms": "/".join(sorted(e["perms"])), "maps": e["count"], "arch": "?"})
if not a.no_arch:
for r in rows[:a.max_libs]:
r["arch"] = read_elf_arch(su, r["path"])
probed += 1
# translation markers
markers = sorted({p for p in entries if any(t in p.lower() for t in TRANSLATOR_MARKERS)})
app_archs = sorted({r["arch"] for r in rows if r["kind"] in ("apk-lib", "materialized") and r["arch"] not in ("?", "not-elf")})
host_is_x86 = "x86" in dev_abi
translated = bool(markers) or (host_is_x86 and any(x.startswith(("arm", "aarch64")) for x in app_archs))
if a.as_json:
print(json.dumps({"pid": pid, "pkg": pkg, "device_abi": dev_abi, "abilist": abilist,
"translation_suspected": translated, "translator_markers": markers,
"libs": rows}, indent=2, ensure_ascii=False))
return 0
print("pid=%s pkg=%s" % (pid, pkg or "?"))
print("device primary abi=%s abilist=[%s]" % (dev_abi, abilist))
print("%-13s %-9s %-6s %-6s %s" % ("arch", "kind", "perms", "maps", "path"))
print("-" * 100)
for r in rows:
print("%-13s %-9s %-6s %-6d %s" % (r["arch"], r["kind"], r["perms"], r["maps"], r["path"]))
print("-" * 100)
print("mapped libraries shown: %d" % len(rows))
if not a.no_arch and probed < len(rows):
print("(architecture probed for the first %d; raise --max-libs to cover the rest -- '?' means not probed)"
% probed)
if markers:
print("\nTRANSLATOR MARKERS PRESENT:")
for mk in markers:
print(" " + mk)
print(" -> native ARM code is being translated. Timing differs from a real ARM device,")
print(" and native integrity checks may behave differently. Verify the final artifact")
print(" on the ABI the user actually runs.")
if app_archs:
print("\napp library architectures: %s" % ", ".join(app_archs))
if translated:
print("translation suspected: YES")
mat = [r["path"] for r in rows if r["kind"] == "materialized"]
if mat:
print("\nRUNTIME-MATERIALIZED libraries (not inside the APK):")
for p in mat[:20]:
print(" " + p)
print(" -> written at runtime by the app or its shell. A repacked APK will not contain")
print(" them in this location; account for that before assuming your patch is loaded.")
if not rows:
print("\nnothing matched. Widen with --all, or check that the process is the one that draws the UI.")
return 0
if __name__ == "__main__":
sys.exit(main())

438
vendor-new/apk-reverse/skills/apk-reverse/scripts/lsposed_scaffold.py Normal file
View File

@@ -0,0 +1,438 @@
#!/usr/bin/env python3
"""Scaffold a minimal LSPosed module project that builds without Gradle.
Why this exists
---------------
Repackaging a hardened APK is not always the right route. When the app is
integrity-checked, signature-derived, or simply cheaper to hook than to rebuild,
the deliverable becomes a *module* instead of a patched APK, and the first
obstacle is that every tutorial assumes Android Studio. This script writes the
smallest project that a plain ``javac`` + ``d8`` + ``aapt2`` + ``apksigner``
chain can turn into an installable module, and the README it writes contains
that exact chain.
What it writes
--------------
<out>/AndroidManifest.xml the three xposed meta-data keys LSPosed reads
<out>/assets/xposed_init one line: the fully qualified entry class
<out>/src/<pkg path>/<Class>.java the entry class (IXposedHookLoadPackage)
<out>/README.md build, install, enable, verify, failure modes
The generated hook does two things on purpose, because both are diagnostics
before they are features: it logs on every load of a scoped process (so
"is the module injected at all" is answerable from logcat alone), and it hooks
``Application.attach`` / ``Activity.onCreate`` (so the log names the real
Activity classes even when a packer swaps the Application at runtime).
Note on scope: LSPosed decides *which* processes to inject into from the scope
list in its own configuration, not from anything this project declares. A
generated ``TARGETS`` array is a second gate inside the module and must not be
mistaken for the scope.
Usage
-----
lsposed_scaffold.py --package com.example.probe --name "Example probe" \\
--hook-target com.example.target --out ./module
Requirements: python3 only. No third-party imports, no network.
"""
import argparse
import os
import re
import sys
MANIFEST_TMPL = """<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="__PACKAGE__">
<uses-sdk android:minSdkVersion="__MINSDK__" android:targetSdkVersion="__TARGETSDK__" />
<application android:label="__NAME__" android:hasCode="true">
<!-- These three keys are what LSPosed Manager reads. Without
xposedmodule=true the APK installs as an ordinary app and never
appears in the module list at all. xposedminversion=82 is the
classic XposedBridge API level and is accepted by LSPosed. -->
<meta-data android:name="xposedmodule" android:value="true" />
<meta-data android:name="xposeddescription" android:value="__DESC__" />
<meta-data android:name="xposedminversion" android:value="82" />
</application>
</manifest>
"""
JAVA_TMPL = """package __PACKAGE__;
import android.app.Activity;
import android.app.Application;
import android.content.Context;
import android.os.Bundle;
import de.robv.android.xposed.IXposedHookLoadPackage;
import de.robv.android.xposed.XC_MethodHook;
import de.robv.android.xposed.XposedBridge;
import de.robv.android.xposed.XposedHelpers;
import de.robv.android.xposed.callbacks.XC_LoadPackage;
/**
* Entry class named by assets/xposed_init.
*
* Logcat is the evidence channel: LSPosed pushes XposedBridge.log() into the
* system log, so `adb logcat -s __TAG__` answers "did this module get injected"
* without attaching a debugger.
*/
public class __CLASS__ implements IXposedHookLoadPackage {
/** Short enough to stay readable in logcat. */
private static final String TAG = "__TAG__";
/**
* In-module gate, NOT the LSPosed scope. LSPosed already decides which
* processes receive handleLoadPackage; this array only keeps the hook body
* from acting on unintended processes if the scope is widened later.
*/
private static final String[] TARGETS = new String[] { __TARGETS__ };
private static boolean isTarget(String pkg) {
for (String t : TARGETS) {
if (t.equals(pkg)) {
return true;
}
}
return false;
}
@Override
public void handleLoadPackage(final XC_LoadPackage.LoadPackageParam lpp) throws Throwable {
// Fires once per scoped process, before any of the app's own code runs.
// This single line is the injection proof: if it is absent for a
// process you scoped, the problem is scope/enable state or the target
// process was never restarted, not your hook body.
try {
XposedBridge.log(TAG + " injected: " + lpp.packageName
+ "/" + lpp.processName + " cl=" + shortCl(lpp.classLoader));
} catch (Throwable t) {
XposedBridge.log(t);
}
if (!isTarget(lpp.packageName)) {
return;
}
// 1) Application.attach(Context) runs before the app's onCreate. Under a
// packer thisObject is the stub Application, not the manifest name.
try {
XposedHelpers.findAndHookMethod(Application.class, "attach", Context.class,
new XC_MethodHook() {
@Override
protected void afterHookedMethod(MethodHookParam param) throws Throwable {
XposedBridge.log(TAG + " Application.attach -> "
+ safeClass(param.thisObject));
}
});
XposedBridge.log(TAG + " hook installed: Application.attach");
} catch (Throwable t) {
XposedBridge.log(TAG + " attach-hook FAILED: " + t);
}
// 2) Activity lifecycle. Boot-classloader targets, so these install even
// when the app's own classes are not yet resolvable.
try {
XposedHelpers.findAndHookMethod(Activity.class, "onCreate", Bundle.class,
new XC_MethodHook() {
@Override
protected void afterHookedMethod(MethodHookParam param) throws Throwable {
XposedBridge.log(TAG + " Activity.onCreate -> "
+ safeClass(param.thisObject));
}
});
XposedBridge.log(TAG + " hook installed: Activity.onCreate");
} catch (Throwable t) {
XposedBridge.log(TAG + " activity-hook FAILED: " + t);
}
__HOOK_BLOCK__
}
private static String safeClass(Object o) {
try {
return o == null ? "null" : o.getClass().getName();
} catch (Throwable t) {
return "<class-name-threw " + t.getClass().getSimpleName() + ">";
}
}
private static String shortCl(ClassLoader cl) {
try {
if (cl == null) {
return "null";
}
return cl.getClass().getName() + "@"
+ Integer.toHexString(System.identityHashCode(cl));
} catch (Throwable t) {
return "<cl-threw>";
}
}
}
"""
HOOK_BLOCK_TMPL = """
// 3) Replace this with the method you actually care about. Find it at
// runtime first (enumerate loaded classes / read a stack) rather than
// guessing an obfuscated name from a decompiler.
try {
XposedHelpers.findAndHookMethod("__HOOKCLASS__", lpp.classLoader,
"__HOOKMETHOD__", __PARAMTYPES__,
new XC_MethodHook() {
@Override
protected void beforeHookedMethod(MethodHookParam param) throws Throwable {
XposedBridge.log(TAG + " __HOOKCLASS__.__HOOKMETHOD__ entered");
}
});
XposedBridge.log(TAG + " hook installed: __HOOKCLASS__.__HOOKMETHOD__");
} catch (Throwable t) {
XposedBridge.log(TAG + " target-hook FAILED: " + t);
}
"""
README_TMPL = """# __NAME__
Minimal LSPosed module generated by `lsposed_scaffold.py`.
- module package: `__PACKAGE__`
- entry class: `__PACKAGE__.__CLASS__` (named in `assets/xposed_init`)
- in-module target gate: __TARGETLIST__
- minSdk __MINSDK__ / targetSdk __TARGETSDK__
The LSPosed *scope* is separate from the target gate above: scope is configured
in LSPosed Manager, and a module is injected into a process only when the scope
contains that package **and** the module is enabled.
## Build without Gradle
You need a JDK (17 works), Android build-tools, an `android.jar` for the SDK you
declare, and the Xposed API stub jar (`de.robv.android.xposed:api:82`, about
25 KB). Set the three variables, then run the steps in order. There is no
resource to compile; the module ships only a manifest, assets and a dex.
```sh
BT=/path/to/build-tools/34.0.0
AJ=/path/to/android.jar # e.g. platforms/android-28/android.jar
API=/path/to/api-82.jar
OUT=build
# 1. Java -> class files. -source/-target 8 keeps d8 happy; Android API classes
# come from -cp, not from the JDK. (`-bootclasspath` is gone in JDK 9+.)
javac -encoding UTF-8 -source 8 -target 8 -nowarn \\
-cp "$AJ:$API" -d "$OUT/classes" src/__PATH__/__CLASS__.java
# 2. d8 wants a jar, NOT a directory -- a directory dies with
# "Unsupported source file type".
jar cf "$OUT/classes.jar" -C "$OUT/classes" .
"$BT/d8" --min-api __MINSDK__ --lib "$AJ" --output "$OUT" "$OUT/classes.jar"
# 3. manifest + assets -> base apk (no res/ in this project)
"$BT/aapt2" link -o "$OUT/base.apk" -I "$AJ" \\
--manifest AndroidManifest.xml \\
--min-sdk-version __MINSDK__ --target-sdk-version __TARGETSDK__ \\
-A assets
# 4. add the dex. aapt resolves the path relative to the cwd and stores the same
# name, so run it from $OUT with classes.dex sitting there.
cp "$OUT/base.apk" "$OUT/unsigned.apk"
cp "$OUT/classes.dex" "$OUT/classes.dex.tmp" && mv "$OUT/classes.dex.tmp" "$OUT/classes.dex"
(cd "$OUT" && "$BT/aapt" add unsigned.apk classes.dex)
# 5. align, then sign (v2 is enough for Android 7+)
"$BT/zipalign" -f -p 4 "$OUT/unsigned.apk" "$OUT/aligned.apk"
keytool -genkeypair -keystore "$OUT/test.keystore" -alias mod -keyalg RSA \\
-keysize 2048 -validity 10000 -storepass android -keypass android \\
-dname "CN=module, O=local"
"$BT/apksigner" sign --ks "$OUT/test.keystore" --ks-pass pass:android \\
--key-pass pass:android --v2-signing-enabled true \\
--out "$OUT/module.apk" "$OUT/aligned.apk"
"$BT/apksigner" verify --print-certs "$OUT/module.apk"
```
Windows: same steps; `d8`, `apksigner` and `aapt` ship as `.bat` wrappers, and
the classpath separator is `;`. Note that `jar` and `keytool` are frequently
**not** on `PATH` even when `javac` is (Oracle's `javapath` shim exposes only
`java`/`javac`), so call them from the JDK's `bin` directory explicitly.
## Install, enable, verify
```sh
adb install -r build/module.apk
adb shell pm list packages | grep __PACKAGE__ # installed
adb shell dumpsys package __PACKAGE__ | grep enabled # enabled=0 means DISABLED by the PM
adb shell pm enable __PACKAGE__ # only if it shows enabled=0
```
`pm enable` is about the *package manager* state; it is not the LSPosed module
switch. Then, in LSPosed Manager: open the module list, enable the module, and
tick the scope entry for the target package.
Verification has two channels. Use the platform log where it works, and always the module log:
```sh
adb -s <serial> shell "su -c 'cat /data/adb/lspd/log/modules_<timestamp>.log'" # module log
adb -s <serial> logcat -s __TAG__ # platform log
```
Expected first line in either: `__TAG__ injected: <target>/<process> ...`. The platform log can be
**completely empty** on a device whose `logd` route is broken (a log file that opens with
`Logd maybe crashed (err=Socket operation on non-socket)` is the tell); the module log is written
either way, so check it before concluding the module never ran. No line in either channel means the
module was not injected into that process. Check, in this order: module
enabled in LSPosed Manager, package present in scope, module APK still installed,
and **the target process restarted after the scope change** -- scope changes are
read for newly forked processes, so a running process keeps the old decision.
## Failure modes worth knowing
- `d8` on a directory: `Unsupported source file type`. Jar the classes first.
- `aapt add` says the file is missing: it resolves relative to the cwd and keeps
the given name, so `classes.dex` must exist in the directory you run it from.
- Module never appears in LSPosed Manager: the `xposedmodule` meta-data is
missing or mistyped in the merged manifest.
- Module listed but injection logs never appear: enabled state, scope, or a stale
target process -- in that order.
- Module injected but your `TARGETS` array skips everything: that is the in-module
gate, not LSPosed. Read the gate before blaming the framework.
"""
def build_parser():
p = argparse.ArgumentParser(
prog="lsposed_scaffold.py",
description="Write a minimal, Gradle-free LSPosed module project "
"(manifest, xposed_init, entry class, build README).",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="Example:\n"
" lsposed_scaffold.py --package com.example.probe \\\n"
" --name \"Example probe\" --hook-target com.example.target \\\n"
" --hook-class com.example.target.Helper --hook-method check \\\n"
" --out ./module\n")
p.add_argument("--package", required=True,
help="module package name, e.g. com.example.probe")
p.add_argument("--name", required=True,
help="human-readable module name shown in LSPosed Manager")
p.add_argument("--hook-target", action="append", default=None, dest="targets",
help="package the module acts on; repeat for several. "
"This is the in-module gate, not the LSPosed scope")
p.add_argument("--out", required=True, help="output project directory")
p.add_argument("--class-name", default="MainHook",
help="entry class name (default %(default)s)")
p.add_argument("--tag", default=None,
help="logcat tag; default derived from --class-name (max 20 chars)")
p.add_argument("--description", default="Hook module generated by lsposed_scaffold.py",
help="value of xposeddescription")
p.add_argument("--min-sdk", type=int, default=24, help="minSdkVersion (default %(default)s)")
p.add_argument("--target-sdk", type=int, default=28, help="targetSdkVersion (default %(default)s)")
p.add_argument("--hook-class", default=None,
help="optional concrete class to hook (adds a worked example block)")
p.add_argument("--hook-method", default=None,
help="method name to hook (requires --hook-class)")
p.add_argument("--hook-params", default="",
help="comma-separated parameter types for --hook-method, "
"e.g. 'java.lang.String,android.content.Context' (default: none)")
p.add_argument("--force", action="store_true",
help="overwrite files that already exist in --out")
return p
PKG_RE = re.compile(r'^[a-zA-Z][a-zA-Z0-9_]*(\.[a-zA-Z][a-zA-Z0-9_]*)+$')
CLASS_RE = re.compile(r'^[A-Za-z_][A-Za-z0-9_]*$')
def fail(msg):
print("[!] %s" % msg, file=sys.stderr)
return 2
def write_file(path, text, force):
if os.path.exists(path) and not force:
print("[=] exists, left alone: %s" % path)
return False
d = os.path.dirname(path)
if d:
os.makedirs(d, exist_ok=True)
# newline="\n" so a scaffold written on Windows still builds on POSIX
with open(path, 'w', encoding='utf-8', newline='\n') as fh:
fh.write(text)
print("[+] %s" % path)
return True
def params_java(spec):
"""'java.lang.String,android.content.Context' -> 'java.lang.String.class,
android.content.Context.class'."""
parts = [x.strip() for x in spec.split(',') if x.strip()]
if not parts:
return ''
return ', '.join('%s.class' % x for x in parts)
def main(argv=None):
args = build_parser().parse_args(argv)
if not PKG_RE.match(args.package):
return fail("--package %r is not a valid Java package name" % args.package)
if not CLASS_RE.match(args.class_name):
return fail("--class-name %r is not a valid Java identifier" % args.class_name)
targets = args.targets or ['<target.package>']
for t in targets:
if not PKG_RE.match(t) and t != '<target.package>':
return fail("--hook-target %r is not a valid package name" % t)
if args.hook_method and not args.hook_class:
return fail("--hook-method requires --hook-class")
tag = args.tag or args.class_name.upper()[:20]
hook_block = ''
if args.hook_class and args.hook_method:
hook_block = (HOOK_BLOCK_TMPL
.replace('__HOOKCLASS__', args.hook_class)
.replace('__HOOKMETHOD__', args.hook_method)
.replace('__PARAMTYPES__', params_java(args.hook_params)))
java = (JAVA_TMPL
.replace('__PACKAGE__', args.package)
.replace('__CLASS__', args.class_name)
.replace('__TAG__', tag)
.replace('__TARGETS__', ', '.join('"%s"' % t for t in targets))
.replace('__HOOK_BLOCK__', hook_block))
manifest = (MANIFEST_TMPL
.replace('__PACKAGE__', args.package)
.replace('__NAME__', args.name)
.replace('__DESC__', args.description)
.replace('__MINSDK__', str(args.min_sdk))
.replace('__TARGETSDK__', str(args.target_sdk)))
readme = (README_TMPL
.replace('__NAME__', args.name)
.replace('__PACKAGE__', args.package)
.replace('__CLASS__', args.class_name)
.replace('__TAG__', tag)
.replace('__PATH__', args.package.replace('.', '/'))
.replace('__TARGETLIST__', ', '.join('`%s`' % t for t in targets))
.replace('__MINSDK__', str(args.min_sdk))
.replace('__TARGETSDK__', str(args.target_sdk)))
src = os.path.join(args.out, 'src', args.package.replace('.', os.sep),
'%s.java' % args.class_name)
write_file(os.path.join(args.out, 'AndroidManifest.xml'), manifest, args.force)
write_file(os.path.join(args.out, 'assets', 'xposed_init'),
'%s.%s\n' % (args.package, args.class_name), args.force)
write_file(src, java, args.force)
write_file(os.path.join(args.out, 'README.md'), readme, args.force)
print("")
print("Next: build it with the chain in %s/README.md, then install, enable"
" the module in LSPosed Manager, and add the target to its scope."
% args.out)
print("Do not confuse the scope (LSPosed Manager) with the in-module gate"
" (TARGETS in %s.java)." % args.class_name)
return 0
if __name__ == '__main__':
sys.exit(main())

224
vendor-new/apk-reverse/skills/apk-reverse/scripts/mt_mcp_probe.py Normal file
View File

@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""Probe the MT Manager built-in APK MCP server (Streamable HTTP).
MT Manager (bin.mt.plus) ships an "APK MCP" service that exposes its APK
analysis/edit/repack/sign capabilities to MCP clients over Streamable HTTP,
by default on device port 8787 (reachable from the PC via
``adb forward tcp:8787 tcp:8787``).
This script does the MCP handshake by hand (JSON-RPC 2.0 over plain HTTP
POSTs, no mcp SDK dependency):
1. ``initialize`` -> serverInfo + capabilities
2. ``notifications/initialized``
3. ``tools/list`` -> tool inventory
Output is a human-readable capability report grouping the ``mt_apk_*`` tools
by category. When the server is not up yet (it must be started by hand in
the MT UI; adb cannot start it), the script prints waiting instructions and
exits 2, so it can also be used as a "is it up yet" check in a loop.
Verification status of the protocol shape: measured against the MT 2.26.9
implementation on 2026-09 (see docs/tool-verification/EXTENSION-kernel-ondevice.md).
"""
import argparse
import json
import sys
import urllib.error
import urllib.request
DEFAULT_URL = "http://127.0.0.1:8787/mcp"
# Tool categories for the report, keyed by name prefix within mt_apk_*.
CATEGORIES = [
("open", "APK open/selection"),
("list", "Entry listing"),
("search", "Text/string search"),
("read", "Content read"),
("dex", "Dex analysis"),
("resource", "Resources"),
("native", "Native (.so) static analysis"),
("edit", "Edit sessions"),
("patch", "Byte/instruction patching"),
("build", "Repack + sign"),
("close", "Cleanup"),
]
def http_post(url, payload, session_id=None, timeout=10.0):
"""POST one JSON-RPC message; return (status, headers, parsed-body-or-None).
Handles both plain application/json responses and text/event-stream
(SSE-framed) responses, which Streamable HTTP servers may use for
request bodies. Returns the first JSON object found in either shape.
"""
body = json.dumps(payload).encode("utf-8")
headers = {
"Content-Type": "application/json",
"Accept": "application/json, text/event-stream",
}
if session_id:
headers["Mcp-Session-Id"] = session_id
req = urllib.request.Request(url, data=body, headers=headers, method="POST")
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read().decode("utf-8", "replace")
sid = resp.headers.get("Mcp-Session-Id")
return resp.status, sid, parse_body(raw, resp.headers.get("Content-Type", ""))
except urllib.error.HTTPError as e:
raw = e.read().decode("utf-8", "replace")
return e.code, None, parse_body(raw, e.headers.get("Content-Type", ""))
def parse_body(raw, ctype):
"""Extract the first JSON message from a JSON or SSE-framed body."""
raw = raw.strip()
if not raw:
return None
if "text/event-stream" in (ctype or ""):
for line in raw.splitlines():
if line.startswith("data:"):
chunk = line[len("data:"):].strip()
try:
return json.loads(chunk)
except ValueError:
continue
return None
try:
return json.loads(raw)
except ValueError:
return None
def print_waiting_help(url):
print("[waiting] MCP server is not answering at %s" % url)
print()
print("The MT APK MCP must be started by hand in the MT UI (adb cannot start it):")
print(" 1. On the phone: MT Manager -> side drawer -> Tools -> APK MCP -> Start")
print(" 2. Note the address shown on that page (host:port, default 8787)")
print(" 3. On the PC, make the port reachable over USB:")
print(" adb forward tcp:8787 tcp:8787")
print(" 4. Re-run this probe: python mt_mcp_probe.py")
print()
print("(LAN alternative: connect the PC and phone to the same network and pass")
print(" --url http://<phone-ip>:8787/mcp)")
def main():
ap = argparse.ArgumentParser(
description="Probe the MT Manager APK MCP (Streamable HTTP) and list its tools.")
ap.add_argument("--url", default=DEFAULT_URL,
help="MCP endpoint (default: %(default)s)")
ap.add_argument("--timeout", type=float, default=10.0,
help="per-request timeout in seconds (default: %(default)s)")
ap.add_argument("--json", action="store_true",
help="print raw JSON-RPC responses instead of the report")
args = ap.parse_args()
# 1) initialize
init = {
"jsonrpc": "2.0", "id": 1, "method": "initialize",
"params": {
"protocolVersion": "2024-11-05",
"capabilities": {},
"clientInfo": {"name": "mt-mcp-probe", "version": "0.1"},
},
}
try:
status, session_id, body = http_post(args.url, init, timeout=args.timeout)
except (urllib.error.URLError, OSError) as e:
print_waiting_help(args.url)
if args.json:
print("[error] %s" % e)
return 2
if body is None or "result" not in (body or {}):
print("[unexpected] HTTP %d, no MCP initialize result in response" % status)
if args.json:
print(json.dumps(body, indent=2) if body is not None else "(empty body)")
return 1
result = body["result"]
server = result.get("serverInfo", {})
proto = result.get("protocolVersion", "?")
print("[online] %s | server %s %s | protocol %s | HTTP %d%s" % (
args.url, server.get("name", "?"), server.get("version", "?"),
proto, status, " | session %s" % session_id if session_id else ""))
if args.json:
print(json.dumps(result, indent=2))
# 2) initialized notification (no id -> notification; 202 expected, body ignored)
note = {"jsonrpc": "2.0", "method": "notifications/initialized"}
try:
http_post(args.url, note, session_id=session_id, timeout=args.timeout)
except (urllib.error.URLError, OSError):
pass # some servers accept it silently; tools/list will tell us
# 3) tools/list
listing = {"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}}
try:
status, _, body = http_post(args.url, listing, session_id=session_id,
timeout=args.timeout)
except (urllib.error.URLError, OSError) as e:
print("[error] tools/list failed after successful initialize: %s" % e)
return 1
if body is None or "result" not in (body or {}):
print("[unexpected] HTTP %d, no tools/list result" % status)
if args.json:
print(json.dumps(body, indent=2) if body is not None else "(empty body)")
return 1
tools = body["result"].get("tools", [])
if args.json:
print(json.dumps(body, indent=2))
# 4) capability report
print()
print("== MCP tool inventory: %d tool(s) ==" % len(tools))
mt_tools = [t for t in tools if t.get("name", "").startswith("mt_apk_")]
other = [t for t in tools if not t.get("name", "").startswith("mt_apk_")]
grouped = {prefix: [] for prefix, _ in CATEGORIES}
for t in sorted(mt_tools, key=lambda x: x.get("name", "")):
name = t.get("name", "")
suffix = name[len("mt_apk_"):]
placed = False
for prefix, label in CATEGORIES:
if suffix == prefix or suffix.startswith(prefix + "_"):
grouped[prefix].append(t)
placed = True
break
if not placed:
grouped.setdefault("misc", []).append(t)
for prefix, label in CATEGORIES:
items = grouped.get(prefix, [])
if not items:
continue
print("\n[%s] %s" % (label, len(items)))
for t in items:
desc = (t.get("description") or "").strip().splitlines()
first = desc[0].strip() if desc else ""
print(" %-42s %s" % (t.get("name"), first[:90]))
if grouped.get("misc"):
print("\n[other mt_apk_*] %d" % len(grouped["misc"]))
for t in grouped["misc"]:
print(" %s" % t.get("name"))
if other:
print("\n[non-mt_apk tools] %d" % len(other))
for t in other:
print(" %s" % t.get("name"))
print()
print("Constraints (from the official MT MCP docs, see on-device-tooling.md):")
print(" - no Java decompilation; read smali directly instead")
print(" - .so support is static analysis only (no dynamic run, no pseudo-C)")
print(" - resources.arsc: existing entries editable, no new locales/entries")
return 0
if __name__ == "__main__":
sys.exit(main())

279
vendor-new/apk-reverse/skills/apk-reverse/scripts/native_crash.py Normal file
View File

@@ -0,0 +1,279 @@
#!/usr/bin/env python3
"""Extract native crash blocks from a log or tombstone and locate the fault.
Native deaths are the expensive kind: no Java stack, often no obvious cause, and
the interesting question ("which library, which offset, is this a real fault or
an arranged one?") is buried in a debug dump that is tedious to read by hand.
This reads a saved `logcat` capture or a tombstone file and reports, per crash:
* the signal, fault address, and cause
* the register state that matters (the faulting pointer)
* the backtrace, split into your own libraries vs system libraries
* a verdict hint when the fault address looks *arranged* rather than accidental
With `--lib NAME=PATH` it also resolves a frame to a local copy of the library
and prints the bytes at that offset, plus a disassembly window when a decoder is
available.
Why the verdict hint matters: a hardening layer that decides the environment is
hostile rarely calls `kill`. It more often arranges a fault -- load a small
constant, use it as a pointer -- so the death looks like an ordinary bug. The
tell is a fault address that is a small integer combined with a register holding
that same integer. This script flags that shape so you look at the right place
instead of hunting a bug that is not there.
Usage
-----
python native_crash.py capture.txt
python native_crash.py tombstone_07
python native_crash.py capture.txt --lib libfoo.so=/path/to/local/libfoo.so
python native_crash.py capture.txt --lib libfoo.so=./libfoo.so --disasm 16
"""
import argparse
import os
import re
import struct
import sys
SIGNAL_RE = re.compile(
r"signal\s+(\d+)\s+\((\w+)\)\s*,?\s*code\s+(\d+)\s*\(([^)]+)\)\s*,?\s*fault addr\s+(0x[0-9a-fA-F]+)")
PID_RE = re.compile(r"pid:\s*(\d+),\s*tid:\s*(\d+),\s*name:\s*(\S+)\s*>>>\s*(\S+)")
PROC_RE = re.compile(r"Cmdline:\s*(\S+)")
UPTIME_RE = re.compile(r"Process uptime:\s*(\d+)")
FRAME_RE = re.compile(r"#(\d+)\s+pc\s+([0-9a-fA-F]+)\s+(\S+)")
REG_RE = re.compile(r"^\s*([xX]\d{1,2}|sp|lr|pc|pst)\s+([0-9a-fA-F]{8,16})")
SYSTEM_LIB_RE = re.compile(r"^/(apex|system|vendor|data/dalvik-cache|memfd)")
def is_system_lib(path):
if not path.startswith("/"):
return True
return bool(SYSTEM_LIB_RE.match(path))
def parse(text):
"""Yield crash dicts."""
lines = text.splitlines()
crashes = []
i = 0
while i < len(lines):
m = SIGNAL_RE.search(lines[i])
# some ROMs print the signal line without "F DEBUG"; accept both
if not m:
i += 1
continue
block = {"signal": int(m.group(1)), "signame": m.group(2),
"si_code": m.group(4), "fault": m.group(5),
"frames": [], "regs": {}, "raw": []}
# walk up a little to pick up pid/cmdline/uptime printed just before
for j in range(max(0, i - 25), i):
ln = lines[j]
mm = PID_RE.search(ln)
if mm and "pid" not in block:
block["pid"], block["tid"], block["tname"], block["proc"] = mm.groups()
mm = PROC_RE.search(ln)
if mm and "proc" not in block:
block["proc"] = mm.group(1)
mm = UPTIME_RE.search(ln)
if mm:
block["uptime"] = mm.group(1)
if "Cause:" in ln and "cause" not in block:
block["cause"] = ln.split("Cause:", 1)[1].strip()
# walk down through the dump
j = i
while j < len(lines) and j < i + 120:
ln = lines[j]
if j > i + 3 and SIGNAL_RE.search(ln) and j > i + 3:
break
block["raw"].append(ln.strip())
mm = PID_RE.search(ln)
if mm and "pid" not in block:
block["pid"], block["tid"], block["tname"], block["proc"] = mm.groups()
mm = PROC_RE.search(ln)
if mm and "proc" not in block:
block["proc"] = mm.group(1)
mm = UPTIME_RE.search(ln)
if mm:
block["uptime"] = mm.group(1)
if "Cause:" in ln and "cause" not in block:
block["cause"] = ln.split("Cause:", 1)[1].strip()
mm = FRAME_RE.search(ln)
if mm:
# Backtrace lines look like either of:
# #00 pc 00000000000128cc /data/app/.../lib/arm64/libfoo.so
# #02 pc 00000000000cccd0 /apex/.../libc.so (__pthread_start+256)
# The path is group(3); anything after it is a symbol in parens.
path = mm.group(3)
rest = ln.split(path, 1)[1].strip() if path in ln else ""
sym = ""
m2 = re.search(r"\(([^)]+)\)", rest)
if m2:
sym = m2.group(1)
block["frames"].append({
"n": mm.group(1), "pc": int(mm.group(2), 16),
"path": path, "symbol": sym,
})
# Register dumps put several registers on one line, so collect all of
# them rather than only the leading one.
for k, v in re.findall(r"\b([xX]\d{1,2}|sp|lr|pc|pst)\s+([0-9a-fA-F]{8,16})\b",
ln.replace("F DEBUG :", " ")):
block["regs"][k] = int(v, 16)
if "backtrace:" in ln:
pass
j += 1
i = j if j > i else i + 1
crashes.append(block)
return crashes
def verdict_hint(crash):
hints = []
fault = int(crash["fault"], 16)
regs = crash["regs"]
if fault <= 0x100:
holder = [k for k, v in regs.items() if v == fault]
hints.append("fault address is a small integer (%s)." % crash["fault"])
if holder:
hints.append("register(s) %s hold exactly that value -- this is the shape of an "
"ARRANGED fault, not an accident." % ", ".join(sorted(holder)))
hints.append("check whether the instruction at the faulting pc loaded that constant "
"a few instructions earlier and used it as a pointer.")
if crash["signame"] == "SIGKILL":
hints.append("SIGKILL with no tombstone: this is a deliberate termination, not a bug. "
"No crash dump exists because nothing faulted.")
if crash.get("cause") and "null pointer" in crash["cause"] and fault > 0x100:
hints.append("a genuine null-ish dereference; treat as an ordinary bug unless a small "
"constant is involved.")
return hints
def load_libs(specs):
out = {}
for s in specs or []:
if "=" not in s:
sys.exit("--lib expects NAME=PATH, got %r" % s)
name, path = s.split("=", 1)
with open(path, "rb") as fh:
out[name] = fh.read()
return out
def maybe_disasm(data, offset, count):
try:
from capstone import Cs, CS_ARCH_ARM64, CS_MODE_ARM, CS_ARCH_X86, CS_MODE_64
except Exception:
return None
# guess arch from ELF e_machine
if data[:4] != b"\x7fELF":
return None
machine = struct.unpack_from("<H", data, 18)[0]
if machine == 183:
md = Cs(CS_ARCH_ARM64, CS_MODE_ARM)
start = offset & ~3
elif machine == 62:
md = Cs(CS_ARCH_X86, CS_MODE_64)
start = max(0, offset - 16)
else:
return None
out = []
for ins in md.disasm(data[start:start + count * 8], start):
mark = " <<<" if ins.address <= offset < ins.address + ins.size else ""
out.append(" 0x%06x %-10s %s%s" % (ins.address, ins.mnemonic, ins.op_str, mark))
if len(out) >= count:
break
if not out:
# capstone stopped immediately -- say so rather than implying emptiness
return [" (decoder produced nothing from this offset; a silent stop is not "
"evidence there is no code here)"]
return out
def main():
ap = argparse.ArgumentParser(description="Locate native crashes from a log or tombstone.")
ap.add_argument("logfile", help="logcat capture or tombstone file")
ap.add_argument("--lib", action="append", metavar="NAME=PATH",
help="resolve frames for this library name to a local file")
ap.add_argument("--disasm", type=int, default=0, metavar="N",
help="disassemble N instructions around the faulting pc")
ap.add_argument("--frames", type=int, default=12, help="how many frames to print")
args = ap.parse_args()
if not os.path.exists(args.logfile):
sys.exit("no such file: %s" % args.logfile)
with open(args.logfile, "r", encoding="utf-8", errors="replace") as fh:
text = fh.read()
crashes = parse(text)
if not crashes:
print("no native crash block found in %s" % args.logfile)
print("")
print("This is a real negative only if the capture covers the death. Check:")
print(" - was the crash buffer cleared *after* the build under test was installed?")
print(" - does the capture include the tombstone? (a SIGKILL leaves none)")
print(" - did the process die at all, or did it merely restart?")
return
libs = load_libs(args.lib)
for idx, c in enumerate(crashes, 1):
print("=" * 72)
print("CRASH %d" % idx)
print(" process : %s" % c.get("proc", "?"))
print(" pid/tid : %s / %s" % (c.get("pid", "?"), c.get("tid", "?")))
if c.get("uptime"):
print(" uptime : %ss <-- time from launch to death; your test window must "
"exceed this" % c["uptime"])
print(" signal : %s (%d), si_code=%s" % (c["signame"], c["signal"], c["si_code"]))
print(" fault addr: %s" % c["fault"])
if c.get("cause"):
print(" cause : %s" % c["cause"])
hints = verdict_hint(c)
if hints:
print(" HINTS")
for h in hints:
print(" - %s" % h)
app_frames = [f for f in c["frames"] if not is_system_lib(f["path"])]
if app_frames:
print(" backtrace (your libraries):")
for f in app_frames[:args.frames]:
print(" #%s pc 0x%x %s%s"
% (f["n"], f["pc"], os.path.basename(f["path"]),
(" (%s)" % f["symbol"]) if f["symbol"] else ""))
else:
print(" backtrace: no frame inside an app library")
print(" -> the fault is in the runtime or a system library; an app-side")
print(" patch is unlikely to be the cause. Look at what called in.")
if c["frames"]:
print(" full backtrace:")
for f in c["frames"][:args.frames]:
tag = "system" if is_system_lib(f["path"]) else "APP"
print(" #%s [%s] pc 0x%x %s%s"
% (f["n"], tag, f["pc"], (f["path"] or "?"),
(" (%s)" % f["symbol"]) if f["symbol"] else ""))
# resolve against local libraries
for f in c["frames"]:
base = os.path.basename(f["path"])
for name, data in libs.items():
if name in base or base in name:
print(" local %s @ 0x%x:" % (name, f["pc"]))
if args.disasm and f is c["frames"][0]:
for line in maybe_disasm(data, f["pc"], args.disasm) or []:
print(line)
else:
print(" raw: %s" % data[f["pc"]:f["pc"] + 16].hex(" "))
print("")
print("=" * 72)
print("RECORD THIS: signal, fault addr, and the time-to-death above. They are the")
print("baseline an attempted fix must be measured against -- without them a later")
print("run cannot distinguish 'fixed' from 'the timing changed'.")
if __name__ == "__main__":
main()

103
vendor-new/apk-reverse/skills/apk-reverse/scripts/patch_smali.py Normal file
View File

@@ -0,0 +1,103 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Method-level smali patcher.
Design notes (each one earned the hard way):
- Only the instruction body is replaced: from the line after the `.method` declaration up to
(but not including) `.end method`. The `.method` / `.end method` / `.annotation` sections are
left untouched, so modifiers and annotations survive.
- The patch manifest must state a self-consistent `.registers` value: the assembler rejects
out-of-range registers, so `registers` must be >= the number of parameter registers and
>= the highest register index used by any instruction + 1. Parameter registers:
static method = number of parameters; instance method = number of parameters + 1 (this).
- The manifest is a JSON list; each entry:
{
"file": "com/example/Helper.smali", # path relative to the smali tree root
"method": ".method public final show(Landroid/app/Activity;)V", # exact .method line
"registers": 3,
"body": ["invoke-interface {p3}, ...;", "return-void"],
"note": "why this patch exists" # optional, report only
}
A literal-match tool: whitespace, operand punctuation and instruction names must match the
source exactly, and the anchor must be unique. See references/patch-audit.md section 3.
Usage:
python patch_smali.py <smali_tree> <patch.json> [--dry-run]
"""
import json
import os
import sys
def split_methods(text):
"""Split smali text into [(method_decl, start_idx, end_idx)] using line indices."""
lines = text.split('\n')
out = []
start = None
decl = None
for i, ln in enumerate(lines):
s = ln.strip()
if s.startswith('.method ') and start is None:
start, decl = i, s
elif s == '.end method' and start is not None:
out.append((decl, start, i))
start, decl = None, None
return lines, out
def patch_one(path, decl_wanted, registers, body):
with open(path, 'r', encoding='utf-8') as f:
text = f.read()
lines, methods = split_methods(text)
hits = [m for m in methods if m[0] == decl_wanted.strip()]
if not hits:
return False, 'method not found: %s' % decl_wanted
if len(hits) > 1:
return False, 'ambiguous (%d matches): %s' % (len(hits), decl_wanted)
_, start, end = hits[0]
# Keep the .method line; rewrite the interior: .registers + body
new_inner = [' .registers %d' % registers, '']
for b in body:
new_inner.append(' ' + b)
new_lines = lines[:start + 1] + new_inner + [''] + lines[end:]
with open(path, 'w', encoding='utf-8', newline='\n') as f:
f.write('\n'.join(new_lines))
return True, 'ok'
def main():
if len(sys.argv) < 3:
print(__doc__)
return 2
tree = sys.argv[1]
spec = sys.argv[2]
dry = '--dry-run' in sys.argv
with open(spec, 'r', encoding='utf-8') as f:
patches = json.load(f)
ok = fail = 0
for p in patches:
full = os.path.join(tree, p['file'].replace('/', os.sep))
if not os.path.isfile(full):
print('[MISS-FILE] %s' % p['file'])
fail += 1
continue
if dry:
with open(full, 'r', encoding='utf-8') as f:
_, methods = split_methods(f.read())
found = any(m[0] == p['method'].strip() for m in methods)
print('[%s] %s :: %s' % ('DRY-OK' if found else 'DRY-MISS', p['file'], p['method']))
ok += 1 if found else 0
fail += 0 if found else 1
continue
okk, msg = patch_one(full, p['method'], p['registers'], p['body'])
print('[%s] %s :: %s (%s)' % ('OK' if okk else 'FAIL', p['file'], p['method'].split('(')[0].replace('.method ', ''), msg))
ok += 1 if okk else 0
fail += 0 if okk else 1
print('\npatched=%d failed=%d%s' % (ok, fail, ' [dry-run]' if dry else ''))
return 0 if fail == 0 else 1
if __name__ == '__main__':
sys.exit(main())

251
vendor-new/apk-reverse/skills/apk-reverse/scripts/preflight.py Normal file
View File

@@ -0,0 +1,251 @@
#!/usr/bin/env python3
"""Preflight: prove the environment is sane BEFORE you blame your patch.
Run this at the start of every experiment block, and again whenever something
fails in a way you did not expect. A large share of "my patch broke the app"
turns out to be device state, a dead device server, a leftover proxy setting, or
a clock skew -- all of which look exactly like a broken artifact.
It is deliberately read-only: it changes nothing except optional cleanup of a
device-wide proxy setting and port forwards it created itself.
Usage:
python preflight.py
python preflight.py --serial <serial> --pkg com.example.app
python preflight.py --pkg com.example.app --expect-root
python preflight.py --cleanup # remove proxy + frida port forwards it finds
python preflight.py --json # machine-readable summary
Exit code is 0 when no BLOCKER was found, 1 otherwise. WARN never fails the run.
"""
import argparse
import json
import os
import re
import shutil
import subprocess
import sys
BLOCKER, WARN, OK, INFO = "BLOCKER", "WARN", "OK", "INFO"
class Report:
def __init__(self) -> None:
self.rows: list[tuple[str, str, str, str]] = []
def add(self, level: str, area: str, detail: str, fix: str = "") -> None:
self.rows.append((level, area, detail, fix))
def worst(self) -> str:
for lv in (BLOCKER, WARN, OK, INFO):
if any(r[0] == lv for r in self.rows):
return lv
return OK
def run(cmd: list[str], timeout: int = 60) -> tuple[int, str]:
try:
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
return p.returncode, (p.stdout or "") + (p.stderr or "")
except FileNotFoundError:
return 127, "not found: %s" % cmd[0]
except subprocess.TimeoutExpired:
return 124, "TIMEOUT after %ss: %s" % (timeout, " ".join(cmd[:4]))
except Exception as e: # pragma: no cover - defensive
return 1, "error: %s" % e
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"), help="adb executable (default: $ADB or 'adb')")
ap.add_argument("--serial", default=None, help="device serial; required when more than one is attached")
ap.add_argument("--pkg", default=None, help="target package to check (optional)")
ap.add_argument("--expect-root", action="store_true", help="treat missing root as a BLOCKER")
ap.add_argument("--cleanup", action="store_true", help="remove leftover proxy setting and frida forwards")
ap.add_argument("--json", action="store_true", dest="as_json")
a = ap.parse_args()
r = Report()
ADB = a.adb
# ---- host tools -------------------------------------------------------
for tool, why in (
("java", "dexlib2 patcher, apksigner"),
("python", "all scripts"),
):
found = shutil.which(tool)
r.add(OK if found else WARN, "host:" + tool, found or "not on PATH",
"" if found else "install it if you need " + why)
for tool in ("apksigner", "zipalign", "aapt", "aapt2"):
found = shutil.which(tool)
r.add(OK if found else INFO, "host:" + tool, found or "not on PATH",
"" if found else "only needed for signing/manifest work; build-tools on PATH helps")
# ---- adb + device selection ------------------------------------------
if shutil.which(ADB) is None and not os.path.exists(ADB):
r.add(BLOCKER, "adb", "executable not found: %s" % ADB, "pass --adb <path> or set $ADB")
return finish(r, a.as_json)
rc, out = run([ADB, "devices", "-l"])
if rc != 0:
r.add(BLOCKER, "adb", "adb devices failed: %s" % out.strip()[:160], "start the adb server; check USB/emulator")
return finish(r, a.as_json)
devices = []
for line in out.splitlines()[1:]:
line = line.strip()
if not line:
continue
parts = line.split()
if len(parts) >= 2:
devices.append((parts[0], parts[1], line))
online = [d for d in devices if d[1] == "device"]
if not devices:
r.add(BLOCKER, "device", "no device attached", "boot the emulator/connect the device, then re-run")
return finish(r, a.as_json)
if len(online) > 1 and not a.serial:
r.add(BLOCKER, "device", "%d devices online but no --serial given" % len(online),
"pass --serial explicitly, or adb will pick one at random and you will debug the wrong target")
if not online:
bad = ", ".join("%s(%s)" % (d[0], d[1]) for d in devices)
r.add(BLOCKER, "device", "attached but not usable: %s" % bad,
"unauthorized -> accept the USB debugging prompt; offline -> reconnect/reboot the emulator")
return finish(r, a.as_json)
serial = a.serial or online[0][0]
r.add(OK, "device", "using %s (of %d online)" % (serial, len(online)), "")
def sh(cmd: str, timeout: int = 60) -> str:
return run([ADB, "-s", serial, "shell", cmd], timeout=timeout)[1]
def su(cmd: str, timeout: int = 90) -> str:
return run([ADB, "-s", serial, "shell", 'su -c "%s"' % cmd], timeout=timeout)[1]
# ---- device facts -----------------------------------------------------
abi = sh("getprop ro.product.cpu.abi").strip()
abilist = sh("getprop ro.product.cpu.abilist").strip()
rel = sh("getprop ro.build.version.release").strip()
sdk = sh("getprop ro.build.version.sdk").strip()
model = sh("getprop ro.product.model").strip()
sec = sh("getenforce").strip()
r.add(OK, "device:os", "Android %s (sdk %s) %s" % (rel, sdk, model), "")
r.add(OK, "device:abi", "%s abilist=[%s]" % (abi, abilist), "")
r.add(OK if sec.lower() in ("permissive", "") else INFO, "device:selinux", sec or "?", "")
# translation layer / emulator hints -- these change how native code behaves
trans = []
for probe in ("ro.dalvik.vm.native.bridge", "ro.enable.native.bridge.exec", "ro.boot.native_bridge"):
v = sh("getprop " + probe).strip()
if v and v != "0":
trans.append("%s=%s" % (probe, v))
r.add(WARN if trans else OK, "device:translation",
"; ".join(trans) if trans else "no ARM translation declared",
"native arm libraries run through a translator here: timing differs and some native checks misbehave" if trans else "")
if re.search(r"^1$", sh("getprop ro.kernel.qemu").strip()):
r.add(INFO, "device:type", "emulator (ro.kernel.qemu=1)",
"final verification belongs on the real ABI the user will run")
# ---- root -------------------------------------------------------------
idout = su("id")
has_root = "uid=0" in idout
lvl = OK if has_root else (BLOCKER if a.expect_root else WARN)
r.add(lvl, "root", "available" if has_root else "NOT available (%s)" % idout.strip()[:60],
"" if has_root else "anything beyond static analysis needs root: data dirs, Frida, file locks")
# ---- clock (silently breaks TLS work) --------------------------------
dev_epoch = sh("date +%s").strip()
if dev_epoch.isdigit():
drift = abs(int(dev_epoch) - int(__import__("time").time()))
r.add(OK if drift < 120 else WARN, "clock",
"device/host drift %ds" % drift,
"" if drift < 120 else "an expired/wrong certificate can be nothing but this clock; fix before TLS triage")
# ---- leftover state that fakes a failure -----------------------------
proxy = sh("settings get global http_proxy").strip()
if proxy and proxy not in ("null", ":0"):
r.add(WARN, "state:proxy", "device-wide http_proxy=%s" % proxy,
"a leftover proxy makes every request fail; clear with: settings put global http_proxy :0")
if a.cleanup:
sh("settings put global http_proxy :0")
r.add(INFO, "state:proxy", "cleared (--cleanup)", "")
else:
r.add(OK, "state:proxy", "not set", "")
rc_f, fwd = run([ADB, "-s", serial, "forward", "--list"])
if fwd.strip():
r.add(INFO, "state:forwards", fwd.strip().replace("\n", " | ")[:200],
"stale forwards point frida at the wrong place; 'adb forward --remove-all' resets them")
if a.cleanup:
run([ADB, "-s", serial, "forward", "--remove-all"])
r.add(INFO, "state:forwards", "removed (--cleanup)", "")
# ---- frida server reachability ---------------------------------------
srv = su("pgrep -f frida-server || pgrep -f kwork")
if srv.strip().isdigit() or srv.strip().split("\n")[0].strip().isdigit():
r.add(OK, "frida:server", "device server process alive (pid %s)" % srv.strip().split("\n")[0].strip(), "")
else:
r.add(INFO, "frida:server", "no device server process found",
"only needed for dynamic work; see references/dynamic-frida.md (name it out of obvious paths)")
# ---- target package ---------------------------------------------------
if a.pkg:
pm = sh("pm list packages | grep -F %s" % a.pkg)
installed = ("package:" + a.pkg) in pm or a.pkg in pm
r.add(OK if installed else INFO, "target", "installed" if installed else "not installed", "")
if installed:
path = sh("pm path %s" % a.pkg).strip()
uid = su("dumpsys package %s | grep -m1 userId=" % a.pkg).strip()
r.add(INFO, "target:apk", path[:160] or "?", "")
r.add(INFO, "target:uid", uid or "?", "re-check after every reinstall: uid increments")
# the native library dir the package manager will use for this device
nat = sh("dumpsys package %s | grep -m3 primaryCpuAbi" % a.pkg).strip()
if nat:
r.add(INFO, "target:abi", nat.replace("\n", " | ")[:160],
"this is the ABI the package manager chose for THIS device")
maps = su("cat /proc/$(pidof %s | awk '{print $1}')/maps 2>/dev/null | head -1" % a.pkg)
if maps.strip():
r.add(INFO, "target:running", "process is running",
"prefer a clean cold start before measuring anything")
else:
r.add(OK, "target:running", "not running", "")
# ---- free space -------------------------------------------------------
# Report by device node, not by the mount-point column: a block device can be
# mounted at more than one path (an ARM translation shim does exactly this on
# some emulators), and `df` will then label /data with an unrelated path.
space = ""
for line in sh("df -h /data 2>/dev/null").splitlines()[1:]:
cols = line.split()
if len(cols) >= 5:
space = "%s: %s available of %s (%s used)" % (cols[0], cols[3], cols[1], cols[4])
break
if space:
r.add(INFO, "device:space", space, "")
return finish(r, a.as_json)
def finish(r: Report, as_json: bool) -> int:
if as_json:
print(json.dumps({"worst": r.worst(),
"rows": [dict(level=lv, area=ar, detail=d, fix=f) for lv, ar, d, f in r.rows]},
indent=2, ensure_ascii=False))
return 1 if r.worst() == BLOCKER else 0
width = max(len(x[1]) for x in r.rows) if r.rows else 10
for level, area, detail, fix in r.rows:
print("%-8s %-*s %s" % ("[" + level + "]", width, area, detail))
if fix:
print(" " * 9 + "%-*s -> %s" % (width, "", fix))
print("\nworst: %s" % r.worst())
if r.worst() == BLOCKER:
print("Fix every BLOCKER before running another experiment. Do not attribute a failure to your patch yet.")
return 1
return 0
if __name__ == "__main__":
sys.exit(main())

121
vendor-new/apk-reverse/skills/apk-reverse/scripts/probe_api.py Normal file
View File

@@ -0,0 +1,121 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Probe an app's HTTP API the way the app does.
Purpose-built for the common frustration: your request gets 403 while the app
works fine. Almost always the app sends a header you are not sending (frequently
one custom header), or your egress IP differs from the device's.
Usage
-----
python probe_api.py --base https://api.example.com --path /health
python probe_api.py --base https://api.example.com --path /adverts \
--param position=banner --header 'X-App-Name: myapp'
python probe_api.py --base https://api.example.com \
--path "/items?page=1" --raw
Options
-------
--base Base URL (required)
--path Path, may already contain a query string (required)
--param k=v Repeatable; appended as query parameters
--header k=v Repeatable; sent as-is. Always send the app's real headers.
--token T Convenience: adds 'Authorization: Bearer T'
--ua UA User-Agent (default: okhttp/4.12.0)
--no-proxy Ignore environment proxy settings (recommended)
--show Print the first N bytes of the body (default 800)
Interpreting results
--------------------
200 + data -> alive; read the envelope shape
200 + empty list -> valid input, nothing to return. Do NOT assume "ad removed":
it may simply be that no item is configured.
400 + message -> input is validated; the message often lists valid values
401/403, no creds -> auth-gated (server-side ownership)
401/403, forged -> token is verified server-side; client patching cannot mint one
403 + HTML body -> a WAF/CDN refused your *request shape* or egress, not the endpoint
404 -> exact path mismatch
"""
import argparse
import json
import urllib.error
import urllib.parse
import urllib.request
def build_opener(no_proxy):
if no_proxy:
return urllib.request.build_opener(urllib.request.ProxyHandler({}))
return urllib.request.build_opener()
def main():
ap = argparse.ArgumentParser(add_help=True)
ap.add_argument('--base', required=True)
ap.add_argument('--path', required=True)
ap.add_argument('--param', action='append', default=[])
ap.add_argument('--header', action='append', default=[])
ap.add_argument('--token')
ap.add_argument('--ua', default='okhttp/4.12.0')
ap.add_argument('--no-proxy', action='store_true')
ap.add_argument('--show', type=int, default=800)
a = ap.parse_args()
url = a.base.rstrip('/') + a.path
if a.param:
params = []
for p in a.param:
if '=' not in p:
print('[FAIL] --param expects k=v, got %r' % p)
return 2
k, v = p.split('=', 1)
params.append((k, v))
sep = '&' if '?' in url else '?'
url += sep + urllib.parse.urlencode(params)
headers = {
'User-Agent': a.ua,
'Accept': 'application/json',
}
for h in a.header:
if ':' not in h:
print('[FAIL] --header expects "Name: value", got %r' % h)
return 2
k, v = h.split(':', 1)
headers[k.strip()] = v.strip()
if a.token:
headers['Authorization'] = 'Bearer %s' % a.token
print('[req] GET %s' % url)
for k, v in headers.items():
shown = v if k.lower() != 'authorization' else v[:24] + '...'
print(' %s: %s' % (k, shown))
req = urllib.request.Request(url, headers=headers)
op = build_opener(a.no_proxy)
try:
with op.open(req, timeout=30) as r:
body = r.read().decode('utf-8', 'replace')
print('[res] %s %s' % (r.status, r.headers.get('Content-Type', '')))
print(body[:a.show])
if body.strip().startswith(('{', '[')):
try:
parsed = json.loads(body)
if isinstance(parsed, dict) and 'data' in parsed:
d = parsed['data']
if isinstance(d, dict) and 'list' in d:
print('[shape] data.list length=%s' % len(d['list'] or []))
except Exception:
pass
except urllib.error.HTTPError as e:
body = e.read().decode('utf-8', 'replace')
print('[res] HTTP %s' % e.code)
print(body[:a.show])
except Exception as e:
print('[ERR] %s' % e)
return 1
return 0
if __name__ == '__main__':
raise SystemExit(main())

1295
vendor-new/apk-reverse/skills/apk-reverse/scripts/protobuf_decode_raw.py Normal file
View File

File diff suppressed because it is too large Load Diff

298
vendor-new/apk-reverse/skills/apk-reverse/scripts/rasc_build.py Normal file
View File

@@ -0,0 +1,298 @@
#!/usr/bin/env python3
"""Build and verify `rasc`, the Rust re-implementation of ASC.
Why this script exists: `rasc` has no prebuilt artifact anywhere -- no GitHub release asset, no
crate, and `cargo install rasc` fetches an unrelated maths parser. Adopting it therefore means
building it, and a build that only one machine has ever performed is a build nobody else can
reproduce. This wraps the whole path: toolchain check, clone, build, and the smoke tests that decide
whether the binary is usable rather than merely present.
What it is for, and where it stops: `rasc` answers the same *location* questions as the Python
`droidasc` -- which classes exist, where a string/type/method/field is referenced, what the manifest
says, and the decompiled source of one class -- several times faster, with a Rust DEX decompiler
(droidsaw) behind `getclass`. It is not a replacement for a full decompiler on every class shape;
see `skills/apk-reverse/references/rasc-and-droidsaw.md` for the measured boundary.
Usage:
python rasc_build.py --check # is a usable rasc present? (fast)
python rasc_build.py --build # clone + build into a local tools/ tree
python rasc_build.py --verify <apk> # compare against droidasc on a real APK
python rasc_build.py --json
Requires, to build: git, and a Rust toolchain (rustup). On Windows the GNU host toolchain needs a
64-bit MinGW-w64 gcc on PATH -- the 32-bit MinGW that ships with some setups cannot link a 64-bit
binary and fails with "64-bit mode not compiled in".
"""
import argparse
import json
import os
import re
import shutil
import subprocess
import sys
import time
HERE = os.path.dirname(os.path.abspath(__file__))
def _repo_root():
"""Walk up from this script to the repository root, or None when there is not one.
The script lives at `<repo>/skills/apk-reverse/scripts/`, so `tools/_work` is three levels up --
but a skill installed by `npx skills add` has no repository around it at all. Resolving
`tools/_work` relative to the *script* directory (the first version of this did) looks for
`skills/apk-reverse/scripts/tools/_work`, which never exists, and then reports a working binary
as missing.
"""
cur = HERE
for _ in range(6):
if os.path.isdir(os.path.join(cur, 'tools', '_work')) or \
os.path.isdir(os.path.join(cur, '.git')):
return cur
parent = os.path.dirname(cur)
if parent == cur:
break
cur = parent
return None
ROOT = _repo_root()
# Falls back to a user-level directory when the skill runs outside a checkout, so the tool is not
# pinned to a layout the caller may not have.
DEFAULT_WORK = (os.path.join(ROOT, 'tools', '_work') if ROOT
else os.path.join(os.path.expanduser('~'), '.apk-reverse', 'work'))
# A cargo/rustc installed to a non-default CARGO_HOME is invisible to a non-interactive PATH, which
# is how this was first run: the toolchain was at <work>/rust/cargo/bin and the check said "missing".
TOOLCHAIN_HINTS = (
os.path.join(DEFAULT_WORK, 'rust', 'cargo', 'bin'),
os.path.expanduser('~/.cargo/bin'),
)
for _hint in TOOLCHAIN_HINTS:
if os.path.isdir(_hint) and _hint not in os.environ.get('PATH', ''):
os.environ['PATH'] = _hint + os.pathsep + os.environ.get('PATH', '')
REPO = 'https://github.com/MG1937/ASC.git'
BRANCH = 'rust'
RASC_HINTS = (
os.environ.get('RASC', ''),
os.path.join(DEFAULT_WORK, 'rust', 'target', 'release', 'rasc'),
os.path.join(DEFAULT_WORK, 'rust', 'target', 'release', 'rasc.exe'),
'rasc',
)
EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_CAPABILITY, EXIT_INTERNAL = 0, 1, 2, 3, 4
def find_rasc():
for hint in RASC_HINTS:
if not hint:
continue
if os.path.isabs(hint):
if os.path.isfile(hint):
return hint
continue
found = shutil.which(hint)
if found:
return found
return None
def rasc_version(path):
try:
out = subprocess.run([path, '--version'], capture_output=True, text=True, timeout=60)
except (OSError, subprocess.SubprocessError) as exc:
return None, str(exc)
if out.returncode != 0:
return None, (out.stderr or out.stdout).strip()[:200]
return (out.stdout or '').strip(), ''
def toolchain():
"""What this machine has that a build needs, reported rather than assumed."""
report = {}
for name in ('git', 'cargo', 'rustc', 'cc', 'gcc'):
report[name] = shutil.which(name)
if report['gcc']:
try:
out = subprocess.run([report['gcc'], '-dumpmachine'], capture_output=True, text=True,
timeout=60)
report['gcc_target'] = (out.stdout or '').strip()
except (OSError, subprocess.SubprocessError):
report['gcc_target'] = 'unknown'
return report
def build(workdir, jobs=None):
repo = os.path.join(workdir, 'rasc')
if not os.path.isdir(os.path.join(repo, '.git')):
os.makedirs(workdir, exist_ok=True)
cmd = ['git', 'clone', '--branch', BRANCH, '--depth', '1', REPO, repo]
print('+ %s' % ' '.join(cmd))
rc = subprocess.run(cmd).returncode
if rc != 0:
print('clone failed (rc=%d)' % rc, file=sys.stderr)
return None
env = dict(os.environ)
env.setdefault('CARGO_TARGET_DIR', os.path.join(workdir, 'rust', 'target'))
cmd = ['cargo', 'build', '--release'] + (['--jobs', str(jobs)] if jobs else [])
print('+ %s (in %s)' % (' '.join(cmd), repo))
t0 = time.perf_counter()
rc = subprocess.run(cmd, cwd=repo, env=env).returncode
print('build finished in %.1f s (rc=%d)' % (time.perf_counter() - t0, rc))
if rc != 0:
return None
for name in ('rasc', 'rasc.exe'):
candidate = os.path.join(env['CARGO_TARGET_DIR'], 'release', name)
if os.path.isfile(candidate):
return candidate
return None
def smoke(binary, apk):
"""Ask the same question of rasc and of the Python `droidasc`, and compare the answers.
Presence is not capability: the check that matters is whether the class-definition set agrees on
a real archive. A build that runs but answers differently is a build that must not be adopted,
which is why this lives next to the build rather than in a note.
"""
if not os.path.isfile(apk):
return {'result': 'usage', 'detail': 'apk not found: %s' % apk}
ours = subprocess.run([binary, 'classes', apk], capture_output=True, timeout=1800)
if ours.returncode != 0:
return {'result': 'failed', 'detail': 'rasc classes rc=%d %s'
% (ours.returncode, ours.stderr.decode('utf-8', 'replace')[:200])}
py = shutil.which('droidasc')
if not py:
n = len(re.findall(rb'L[^;|\s]+;', ours.stdout))
return {'result': 'partial', 'rasc_classes': n,
'detail': 'droidasc not on PATH, so only the rasc side was run'}
env = dict(os.environ, PYTHONIOENCODING='utf-8', PYTHONUTF8='1')
theirs = subprocess.run([py, 'listclass', apk], capture_output=True, timeout=1800, env=env)
if theirs.returncode != 0:
return {'result': 'partial', 'detail': 'droidasc rc=%d (its output is not UTF-8 safe on a '
'non-UTF-8 console without PYTHONUTF8=1)'
% theirs.returncode}
def descriptors(blob):
out = set()
for line in blob.decode('utf-8', 'replace').splitlines():
for tok in re.split(r'[\s|]+', line.strip()):
if re.fullmatch(r'L[^;]+;', tok):
out.add(tok)
break
return out
a, b = descriptors(ours.stdout), descriptors(theirs.stdout)
return {'result': 'ok' if a == b else 'failed',
'rasc_classes': len(a), 'droidasc_classes': len(b),
'only_rasc': len(a - b), 'only_droidasc': len(b - a)}
def main(argv=None):
ap = argparse.ArgumentParser(
prog='rasc_build.py',
description='Build and verify rasc, the Rust ASC re-implementation.',
epilog='exit codes: 0 ok, 1 verification failed, 2 usage, 3 a required capability is '
'missing, 4 internal error\n'
'examples:\n'
' rasc_build.py --check\n'
' rasc_build.py --build --work tools/_work\n'
' rasc_build.py --verify tools/_work/apks/sample.apk\n')
ap.add_argument('--check', action='store_true', help='report whether a usable rasc is present')
ap.add_argument('--build', action='store_true', help='clone and build rasc')
ap.add_argument('--verify', metavar='APK', help='compare rasc against droidasc on this APK')
ap.add_argument('--work', default=DEFAULT_WORK, help='where to clone and build (default: %s)'
% DEFAULT_WORK)
ap.add_argument('--jobs', type=int, default=None, help='parallel build jobs')
ap.add_argument('--json', action='store_true')
args = ap.parse_args(argv)
if not (args.check or args.build or args.verify):
ap.print_help()
print('RESULT=usage')
return EXIT_USAGE
payload = {'toolchain': {k: v for k, v in toolchain().items()}, 'work': args.work}
binary = find_rasc()
if binary:
version, err = rasc_version(binary)
payload['rasc'] = {'path': binary, 'version': version, 'error': err or None}
else:
payload['rasc'] = None
if args.build:
missing = [k for k in ('git', 'cargo', 'rustc') if not payload['toolchain'][k]]
if missing:
payload['result'] = 'capability_missing'
payload['next_action'] = ('install Rust (rustup) and git; missing: %s' % ', '.join(missing))
if args.json:
print(json.dumps(payload, indent=2))
print('cannot build: missing %s' % ', '.join(missing), file=sys.stderr)
print('RESULT=capability_missing')
return EXIT_CAPABILITY
gcc_target = payload['toolchain'].get('gcc_target') or ''
built = build(args.work, args.jobs)
if not built:
payload['result'] = 'build_failed'
# The hint is only useful when the linker target is the 32-bit one, which is the
# failure this actually hit: name the target rather than printing a generic list.
payload['next_action'] = (
'read the cargo output above; the GNU host toolchain needs a 64-bit MinGW-w64 '
'gcc and this machine reports %r' % (gcc_target or 'no gcc on PATH') if
'mingw32' in gcc_target or not gcc_target else
'read the cargo output above')
if args.json:
print(json.dumps(payload, indent=2))
print('RESULT=build_failed')
return EXIT_FAILED
binary = built
version, _ = rasc_version(binary)
payload['rasc'] = {'path': built, 'version': version, 'error': None}
if args.verify:
if not binary:
payload['result'] = 'capability_missing'
payload['next_action'] = 'run --build first, or set RASC to an existing binary'
if args.json:
print(json.dumps(payload, indent=2))
print('RESULT=capability_missing')
return EXIT_CAPABILITY
payload['verify'] = smoke(binary, args.verify)
payload['result'] = payload['verify']['result']
if args.check and 'result' not in payload:
payload['result'] = 'ok' if binary else 'capability_missing'
if not binary:
payload['next_action'] = 'run --build (needs git + rustup), or set RASC=<path>'
if args.json:
print(json.dumps(payload, indent=2))
else:
tc = payload['toolchain']
print('toolchain : git=%s cargo=%s rustc=%s gcc=%s%s'
% (tc.get('git') or '-', tc.get('cargo') or '-', tc.get('rustc') or '-',
tc.get('gcc') or '-',
('(%s)' % tc['gcc_target']) if tc.get('gcc_target') else ''))
print('rasc : %s' % (('%s %s' % (payload['rasc']['path'], payload['rasc']['version']))
if payload['rasc'] else 'not found'))
if 'verify' in payload:
print('verify : %s' % payload['verify'])
print('result : %s' % payload['result'])
if payload.get('next_action'):
print('next : %s' % payload['next_action'])
print('RESULT=%s' % payload['result'])
return {'ok': EXIT_OK, 'build_failed': EXIT_FAILED, 'failed': EXIT_FAILED,
'usage': EXIT_USAGE, 'capability_missing': EXIT_CAPABILITY,
'partial': EXIT_OK}.get(payload['result'], EXIT_INTERNAL)
if __name__ == '__main__':
if hasattr(sys.stdout, 'reconfigure'):
try:
sys.stdout.reconfigure(encoding='utf-8')
except (ValueError, OSError):
pass
sys.exit(main())

1497
vendor-new/apk-reverse/skills/apk-reverse/scripts/repack.py Normal file
View File

File diff suppressed because it is too large Load Diff

79
vendor-new/apk-reverse/skills/apk-reverse/scripts/rpc_template.js Normal file
View File

@@ -0,0 +1,79 @@
// rpc_template.js — editable rpc.exports example for frida_rpc_serve.py.
//
// Three shapes every target needs, already wired:
// 1. add — a pure-JS export, proves the transport with zero target knowledge
// 2. getpackagename — a harmless Java probe (calls a framework method via JNI)
// 3. callnative — a native-call placeholder: wrap any exported symbol of any
// loaded module and call it with explicit types
//
// Adapt 3 to your target: the hardened .so that computes a signature or an
// encryption is usually either exported (Module.findExportByName) or bound at
// runtime via RegisterNatives (hook RegisterNatives first to learn the address,
// then pass the raw address to callnativeaddr).
'use strict';
function log(ev, data) {
try { send(Object.assign({ t: Date.now(), ev: ev }, data || {})); } catch (e) {}
}
// Export lookup that works across frida 16/17 (Module.findExportByName(null,..)
// was removed in 17; getGlobalExportByName is the replacement).
function resolveExport(name) {
try { if (Module.getGlobalExportByName) return Module.getGlobalExportByName(name); } catch (e) {}
try { if (Module.findExportByName) return Module.findExportByName(null, name); } catch (e) {}
try {
const libc = Process.findModuleByName('libc.so');
if (libc) return libc.getExportByName(name);
} catch (e) {}
return null;
}
rpc.exports = {
// 1) pure-JS demo — sanity-check the RPC transport itself.
add: function (a, b) {
return a + b;
},
ping: function () {
return 'pong';
},
// 2) harmless Java probe: the package name of the process we live in.
// Works on any app process with a live Java VM; needs no target knowledge.
getpackagename: function () {
let out = null;
Java.perform(function () {
const app = Java.use('android.app.ActivityThread').currentApplication();
out = app.getPackageName();
});
return out;
},
// 3) NATIVE CALL PLACEHOLDER — adapt module/name/ret/arg types to your target.
// moduleName null means "search every loaded module" (global export lookup).
callnative: function (moduleName, exportName, retType, argTypes, args) {
let addr = null;
if (moduleName) {
const base = Module.findBaseAddress(moduleName);
if (base === null) throw new Error('module not loaded: ' + moduleName);
addr = Module.findExportByName(moduleName, exportName);
} else {
addr = resolveExport(exportName);
}
if (addr === null) throw new Error('export not found: ' + exportName);
const fn = new NativeFunction(addr, retType, argTypes);
return fn.apply(null, args || []);
},
// Variant for symbols that never hit the export table (dynamically registered
// JNI, or an offset inside a stripped .so): pass module name + raw offset.
callnativeaddr: function (moduleName, offset, retType, argTypes, args) {
const base = Module.findBaseAddress(moduleName);
if (base === null) throw new Error('module not loaded: ' + moduleName);
const fn = new NativeFunction(base.add(offset), retType, argTypes);
return fn.apply(null, args || []);
}
};
log('RPC-READY', { exports: Object.keys(rpc.exports) });

Some files were not shown because too many files have changed in this diff Show More