vlad/reverse-skills-max
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

reverse-skills-max: initial public release (restored, redacted, hardened)

Browse Source
This commit is contained in:
baradok609hecl1pq6qi
2026-10-20 02:17:15 +03:00
commit da24c2bd20
2227 changed files with 443931 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

75
vendor-new/apk-reverse/tests/README.md Normal file
View File

@@ -0,0 +1,75 @@
# tests/ - the executable half of this repository's evidence
`tests/benchmark.md` is a **record**: what happened when a route was run once, by hand,
against a real sample and a real device. This directory is the other half: checks that run
on every commit with nothing but the standard library and pytest.
The two are not substitutes. A benchmark row can establish that a route works on a hardened
target; it cannot tell you that yesterday's refactor broke the dex width table. A test in
here can tell you that every time, and it can never tell you whether the route achieves the
goal on a real app.
```
python -m pytest -q # whole suite
python tests/run_tests.py # same thing, with a clear message if pytest is missing
python -m pytest -q -m unit # one marker
python -m pytest -q -m cli # exit codes, flags, --json
python -m pytest -q -m integration # no-device flows
python -m pytest -q -ra # list every skip and xfail reason
```
## What the markers mean
- `unit` - pure functions and parsers of real shipped scripts: the dex width table and branch
arithmetic, the scheme-less protobuf decoder, the blob rotation search, the leak rules, and
the established `dex_patch_bytes` regression hash.
- `cli` - one subprocess run per script per invocation shape: `--help`, an unknown flag, no
arguments, and every input pointed at a file that does not exist.
- `integration` - flows that were made of several parts: adb argv construction under a fake
runner, logcat signal counting, tombstone parsing, the two-sample launch-health verdict.
## Nothing here needs a device
No test in this suite opens a frida session or touches the network. The device-dependent
behaviour is tested by intercepting the command that *would* have been run and asserting on the
argv list.
The exception is deliberate and enforced: `contract.DEVICE_OR_NETWORK_BOUND` names the 15
scripts that reach a device once their arguments parse, and those are **never executed** with
complete arguments — only `--help` and an unknown flag, which argparse answers before any
device call. That blocklist exists because a development host may well have a phone attached
(the machine this harness was written on did), and an earlier version of the CLI contract test
drove it for real. General safety net: every child runs in a private temporary working
directory, so a script that defaults an output path relative to cwd cannot write into the
shipped tree.
Anything that genuinely cannot be reached this way is not faked into a pass — it is listed as a
coverage gap in `docs/tool-verification/EXTENSION-test-harness.md`.
## Skipped is not passed
Two kinds of non-passing outcome are kept separate on purpose:
- **`skip`** - the check does not apply here. `tools/_work/bench/**` is git-ignored, so the
`dex_patch_bytes` regression hash self-skips on a fresh checkout and in CI, printing the
reason. A --json script that needs a sample to reach its JSON path skips too.
- **`xfail`** - the script is present and does **not** meet the repository contract. The
reason carries the measured value, for example
`contract says exit 2, measured 1`. These are work items, not flaky tests.
`pytest -q -ra` prints both lists. The xfail list for the CLI layer is the exit-code
unification work order; it is reproduced in `EXTENSION-test-harness.md` with owners.
## Ground rules
- **This suite never edits anything under `skills/`.** A defect found here is registered in
the harness report. Fixing a script's exit code or quoting strategy is its owner's change,
and doing it from a test would hide the drift the test exists to expose.
- **The bench tree is evidence and is read-only here.** The regression test copies its input
into `tmp_path` before patching, so `tools/_work/bench/l1l3/dex/classes.patched.dex` cannot
be overwritten by a test run.
- **No real target identity is committed.** The leak-scanner decoys are synthesised in
`tmp_path` at run time, and token-shaped decoys are assembled from fragments so that this
directory cannot itself become a finding.
- Fixtures in `tests/fixtures/` are small, synthetic sample files. `tests/fixtures/apk/**` is
owned by a different work item and is not touched from here.

57
vendor-new/apk-reverse/tests/benchmark.md Normal file
View File

@@ -0,0 +1,57 @@
# Benchmark set — the regression baseline for this skill
This file is a **test matrix**, not a sample collection. It answers one question per row: *when an
agent meets this shape of target, does the route this repository documents actually work, and what
is the strength of the evidence?* Every claim in the `Result` column must be reproducible from the
`Target` column plus the named script — nothing here is a summary of what "should" happen.
**Samples are never committed.** Each row names a public source; download it into `tools/_work/bench/`
(ignored by git) before running the row. Record the sample hash when you run it, because two rows on
different sample revisions are not comparable. Hashes used in this pass:
```
UnCrackable-Level1.apk 66651 B 1da8bf57d266109f…
UnCrackable-Level2.apk 901022 B 4c7980ef1f6cc295…
UnCrackable-Level3.apk 1460555 B 6827f776b7d28443…
r2pay-v1.0.apk 4293620 B 850d7c98bed8df53…
```
Strength labels follow the repository's own three-tier rule (`references/long-task-discipline.md`
reserves `observed` for a claim with an exact command and its output behind it):
- `observed` — a command was run and its output is quoted in the evidence file.
- `inferred` — reasoned from documented mechanism or from a neighbouring measurement.
- `unverified` — assumed, or reported by someone else and not reproduced here.
| # | Dimension | Target (public source) | Scripts / documents the row exercises | Result | Strength | Evidence |
|---|---|---|---|---|---|---|
| B1 | Equal-length dex patch + repack + re-sign installability (G2/G4) | MASTG `UnCrackable-Level1.apk` | `dex_patch_bytes.py`, `repack.py`, `byte-level-patching.md`, `repack-and-sign.md` | **The route works end to end, and the change is visible on screen.** 2 bytes of the first root-check branch in `MainActivity.onCreate` (`0x9d0: 39000e00` -> `00000000`) patched inside a 5528-byte dex; the whole-file diff is those 4 bytes plus bytes 8..32, and the written file self-verifies (`adler32(d[12:])` and `sha1(d[32:])` both reproduce, proving signature-before-checksum order). `repack.py` produced a STORED + 4-byte-aligned `resources.arsc`, apksigner reported v1+v2+v3 all true, the root install path succeeded with the on-device `base.apk` sha256 equal to the local build, and the blocking `Root detected!` dialog is gone from the screenshots. A zero-change control through the identical pipeline still shows it | observed | `EXTENSION-benchmark-l1-l3.md` §2 |
| B2 | Anti-debug + self-destruct race (patch before the check fires) | MASTG `UnCrackable-Level3.apk` | `spawn_patch_detach.py`, `hook_patch_only.js`, `dynamic-frida.md` | **Split: the tooling works, the patch does not.** Time-to-death with a probe attached is 5.34 s. After an ordering fix (resume -> wait for `PATCHED` -> detach) the probe wrote `libfoo.so` base+0x2ff0/+0x2ff8/+0x3080 to `mov w0,#0 ; ret` 0.04-0.05 s after resume and the writes survived the detach — the mechanism is real. The patch still does not stop the death: the patched run **and** a read-and-write-back control both die with `SIGABRT` at `libfoo.so (goodbye()+12)` 3.22 s after resume, so neutralising the terminate routine plus both of its branch inputs is insufficient; the region is a dense cluster of `mov w0,#imm ; ret` stubs and locating the real second site needs a full `.text` disassembly. The alternative ordering ("hold paused until `PATCHED`") cannot work — a frozen process has no libraries mapped, so `Process.findModuleByName` never returns (0.11 s after resume it appears). Independently, the pristine original already reports tamper on this device, so the row's assumed clean baseline does not exist here | observed (tooling) / unverified (a surviving target) | `EXTENSION-benchmark-l1-l3.md` §3 |
| B3 | Extraction-shell skeleton detection (`trivial-body` routing) | self-built 11-variant skeleton set from the `r2pay-v1.0.apk` dex, with a zero-change control | `dex_dump_validate.py`, `advanced-unpacking.md` | **The "stub% threshold" claim this skill used to make is wrong.** `stub%` is *bimodal*, not thresholded: a real skeleton reads ≈100% (`allstub` 100.0%) while a control reads 1.9%, and the shapes real shells actually use are invisible — nop-cleared bodies 0.0% (below the control), throw stubs 1.9% (indistinguishable), partial extraction at 25/50/75% 1.7/1.7/2.1%. `advanced-unpacking.md`'s original "tens of percent means a skeleton" had no support and was deleted. Three script defects were found and fixed (all-nop bodies ranked as real; a legal minimal body mislabelled malformed; the ranking pointing at a 25%-extracted image as "most likely original" — caught by independent review). The new `minimal-form` counter *does* track the extraction ratio monotonically (1299/2524/3736/4946) and now participates in the ranking | observed | `EXTENSION-extraction-shell-bench.md` §1–3 |
| B4 | Dex-VMP boundary: refuse-to-repack and downgrade the deliverable | DexPatcher `assets/ezAndroid.apk` | `advanced-unpacking.md`, `java2c-and-jni-sinking.md`, `lsposed_scaffold.py`, `frida_rpc_serve.py` | **The sample is not a Dex VMP — it is JNI sinking**: 7 `native` declarations, each with a matching exported `Java_*` symbol, no `JNI_OnLoad` / `RegisterNatives` symbol, 34,566 instructions with zero structural errors under the official `dexdump`. A zero-change repack succeeded (v2 signing, alignment, `classes.dex` CRC verified). Static criteria **can** rule a dex-VMP *out*; they cannot rule runtime-applied hardening out. The first verdict here was the opposite — a hand-written opcode table reported 1393/22424 undecodable bodies and called it a VMP; `dexdump` showed the table was missing an arm64 branch opcode, and that rule (never declare VMP without an official disassembler's failure count) is now in the reference. Installation / launch / refusal remain untested | observed (static + repack) / unverified (install, launch) | `EXTENSION-extraction-shell-bench.md` §4 |
| B5 | Java2C recognition (do not misdiagnose as an extraction shell) | self-built with `amimo/dcc` (`--no-build`) + real JNI samples | `java2c_probe.py`, `java2c-and-jni-sinking.md` | Native-declaration density separates the shapes by ~2000x: **0.03–0.04%** on three real JNI samples vs **82.76%** on a Java2C-shaped fixture, and `Java_*` symbols matched dex native-method counts 1:1 per ABI (2:2, 3:3). **No Dex-to-C compiler output was ever built** (no NDK, no host clang, no device clang, WSL unavailable), so every Java2C-specific criterion is inferred while the JNI-sinking side is measured. A symbol search fails silently on real Java2C: dcc emits `-fvisibility=hidden`, and C++ `jni.h` inlines `RegisterNatives` so it leaves no symbol | observed (density, JNI sinking) / inferred (Java2C-specific) | `EXTENSION-java2c.md` |
| B6 | Native tracing: does `Stalker.exclude()` change the outcome? | MASTG `UnCrackable-Level1.apk`, module `libc.so` followed on the main thread for 6 s | `stalker_trace.js`, `native-dbi-and-deobfuscation.md` | **Split result.** A no-follow baseline survives; the same follow with `excludeModules: []` **kills the process**; with 20 system modules excluded it survives — so exclusion is what keeps the target alive (`observed`, with the baseline arm ruling out "frida killed it anyway"). But the treatment arm still reported `blocks=0 blk=0 calls=0`: **exclusion does not fix the zero-event trace**, which partially contradicts the community framing that lists "no events" among its symptoms. The pass also fixed two harness traps that manufacture a false zero-event result (a load-time default follow, and a suspended spawn) and verified a new zero-event `WARN`. No real OLLVM trace was produced | observed (survival, delivery) / unverified (the multiplier, the cause of zero delivery) | `EXTENSION-stalker-exclude.md` |
| B7 | Native certificate pinning | CyReverse Cronet sample | `protocol-reverse.md` | not run this pass — the CyReverse Cronet sample needs a gradle+NDK build this machine cannot do | unverified | — |
| B8 | Schemaless protobuf decoding | synthetic fixtures + the official runtime + a real AndroidX DataStore container | `protobuf_decode_raw.py`, `protocol-reverse.md` | 26/26 built-in fixtures, **21/21 against the official protobuf runtime** (re-encode byte-identical to `SerializeToString()`), 8/8 framing checks. A real 81-byte DataStore file round-tripped byte-exact and an edited value re-encoded (81->80 B, length prefix recomputed) was read back by `datastore_inject.py`. Both traps reproduced on real bytes: `0801120178` is a valid nested message *and* a valid packed array; a non-presence `0` is byte-identical to "never assigned". Four script defects were found by these checks and fixed | observed | `EXTENSION-protobuf-raw.md` |
| B9 | Split APK / App Bundle handling | two real split sets pulled from a device (8 members = base + 7 density splits each carrying its own `resources.arsc`; 9 members = base + ABI split with only `lib/` + 7 density splits with 40-byte placeholder tables) | `repack.py` (split mode), `split-apk.md` | Both branches of the decision ran on real sets: after `resign` every member shares one certificate with v1+v2+v3 all true; `merge` **correctly refused** the set whose members carry their own `resources.arsc` (exit 1, before writing anything) and **succeeded** on the code/native-only set, folding 2 `.so` files into `base`. The single-APK legacy path still passes its regression (`--dexdir` replacement and a zero-change round trip). Signing has no `uber-apk-signer` on this host; the measured call is `zipalign -p -f 4` then `apksigner.jar --v1 --v2 --v3`, and the order cannot be reversed (`[-124]`). `adb install-multiple` and launch confirmation were **not** executed | observed (analyze/resign/merge) / unverified (install, launch) | `EXTENSION-split-apk.md` |
| B10 | Kernel-side syscall answer forging | KernelSU/APatch module template, host toolchain, device kernel | `kernel-and-environment-hardening.md`, `kernelsu_syscall_mask.py` | The generator produced an 11-file userspace module skeleton plus KPM/LKM/eBPF templates and `verify` reports 0 problems — but **no kernel-side artefact was compiled or loaded**: the `<DEVICE>` kernel is 4.14.186 (eBPF needs 5.10+), the host has no `aarch64-none-elf-gcc` / `make` / `ndk-build`, and there is no kernel source. Documentation correction with teeth: **an ordinary KernelSU module is a userspace module and structurally cannot change a syscall return value** | observed (generation) / unverified (kernel side) | `EXTENSION-kernel-weapons.md` |
| B11 | JNI dynamic registration discovery (`RegisterNatives` with no `Java_*` symbol) | a real system library + CyReverse `plugin/` | `java2c-and-jni-sinking.md`, `java2c_probe.py` | A real dynamically-registering library contains **zero `RegisterNatives` symbols** (the C++ `jni.h` inlines the call), so symbol-based discovery fails structurally. The combination that did fire on a real library is "**exports `JNI_OnLoad` and zero `Java_*`**" | observed | `EXTENSION-java2c.md` |
| B12 | Dart AOT string-table format across ABIs (the premise any string patch rests on) | a locally supplied hardened Flutter APK (360 shell + AOT payload, both ABIs; uncommitted, `tools/_work/dart/`) | `dart-aot.md` §7/§10, `scripts/dart_pool_strings.py` | **Two claims tested, both were wrong before this row.** (a) arm64 packed `[0x80\|(len<<1)][payload]` is confirmed — `dart_pool_strings.py` keeps 4,980 chained entries and the tag byte equals `0x80\|(len<<1)` at every literal checked (`AdmobBanner` 11 chars → `0x96`, `showSplashAd` and `AdPgl_Splash` 12 → `0x98`). (b) The armv7 form "`[len*2 u32le][UTF-16LE]`" is **refuted**: the same extractor keeps **0** entries on the 32-bit snapshot, and the byte layer shows why — each 4-byte-aligned record reads `[header u32][byte-count u32le][UTF-8 payload]`, with the two adjacent literals `SystemChrome.setEnabledSystemUIMode` (36) + `SystemChrome.setSystemUIOverlayStyle` (34) carrying lengths `0x46` (70) and `0x48` (72) that exactly equal their byte lengths. The extractor's zero is a format mismatch, not an empty string table. A third claim, "the engine banner gives `x.y.z (stable)`", is also refuted on this target: the banner is 66 bytes of `0.0.1` with a blank build id. Full detail and the raw probes: `EXTENSION-dart-aot-formats.md` | observed (both formats, banner) / unverified (an end-to-end device patch — no repack+install was run) | `EXTENSION-dart-aot-formats.md` |
| B13 | Naming the check that fires, and the environment it reads (anti-instrumentation triage) | MASTG `owasp.mstg.uncrackable3` (public target) for the three arms; device `libc.so` and a hardened shell library for `svc_scan.py` | `anti_detect_probe.js`, `detection-and-anti-analysis.md` Step 3, `svc_scan.py` | **The check was named, and the asymmetry is the finding.** Three arms on one target with a single variable: control (`spawn` + `resume`, no script) ends `alive_at_end=True`; the arm that also loads the observer probe ends `alive_at_end=False`, with the stream showing `strstr("frida")` at t=303 ms and, in an earlier same-shape arm, the kill itself at t=352 ms — process gone with **no tombstone, no crash and no ANR record**. The probe's own self-report on the same arm: `TracerPid=0` while the target's `/proc/self/maps` carries **4 frida-named mappings**, so a `TracerPid`-only detector passes and a maps/thread-name detector does not. Reproducibility is honestly bounded: 3 same-shape arms, 1 delivered a full sequence (a ≈300 ms self-destruct leaves a sub-second window). Separately, `svc_scan.py`'s two independent decoders — a hand-written word scan and capstone — produced an **identical 214-site set** (set difference empty), and a structural criterion fell out: the device `libc.so`'s 4 termination sites are libc's own exports (so a libc-level hook is structurally able to see them), while the hardened shell library's 21 `svc` sites are **all data** (neighbours are `udf`/NEON filler) with no termination site | observed (the arms, the named check, decode agreement) / unverified (dual-dumper byte agreement — no second producer existed; a dump from the hardened sample) | `EXTENSION-detection-pipeline.md` §1, §4, §5 |
## How to read a row
`Result` states what actually happened, including negative outcomes — "the route worked", "the route
worked only with `Stalker.exclude()`", "the tool refused to run at all". A negative result that is
recorded is worth more than a positive one that is assumed, because the next agent can route around
it without re-paying the cost.
A row whose `Strength` is `unverified` is a statement about *this repository's evidence*, not about
the mechanism's truth. It means: nobody has yet put a command and its output behind it here.
**Rows that changed a conclusion this repository had already recorded.** This is the outcome the
matrix exists to produce, so it is listed rather than buried: B3 (the trivial-body "threshold" became
a measured bimodal result), B4 (a VMP verdict was withdrawn after an official disassembler disagreed
with a hand-written opcode table), B6 (exclusion turns out to keep the process alive but does not fix
zero-event delivery, contradicting the community framing), and B1 (an earlier "`repack.py` has no
apksigner signing path" conclusion expired mid-pass when a second writer added one).

181
vendor-new/apk-reverse/tests/cli/contract.py Normal file
View File

@@ -0,0 +1,181 @@
"""Helpers for the CLI contract tests.
Everything here is derived at run time from the scripts' own `--help` output
rather than from a hard-coded list of argument names, so a new script is covered
automatically and a renamed flag does not silently stop being tested.
The contract under test is the repository-wide one recorded in
tools/_work/REBUILD-BRIEF.md section 4:
0 = success
1 = the target result failed
2 = bad arguments or input
3 = the environment lacks a capability
4 = internal tool error
A script that does not follow it yet is not a red test here: the test calls
``pytest.xfail`` with the measured code in the reason, so the summary line stays
green and the xfail list is exactly the work order for unifying exit codes.
"""
from __future__ import annotations
import json
from pathlib import Path
# A path that does not exist anywhere. Used to drive the "bad input" scenario.
NONEXISTENT = str(Path("__apkrev_missing__") / "does-not-exist.bin")
# The four contract codes, plus None for "did not return".
CONTRACT_EXIT_CODES = {0, 1, 2, 3, 4}
def parse_leading_json(text):
"""Parse the first JSON document in `text`, ignoring anything after it.
Several scripts print a `RESULT=<token>` line after their JSON document (the
repository's long-task discipline requires that token last), so a plain
json.loads on the whole stream fails. Only the first document is returned.
"""
starts = [i for i in (text.find("{"), text.find("[")) if i >= 0]
if not starts:
return None
try:
doc, _end = json.JSONDecoder().raw_decode(text[min(starts):])
except ValueError:
return None
return doc
def usage_block(help_text):
"""Return the joined `usage:` block of an argparse-rendered help text."""
lines = help_text.splitlines()
start = None
for i, line in enumerate(lines):
if line.lower().startswith("usage:"):
start = i
break
if start is None:
return None
block = [lines[start][len("usage:"):].strip()]
for line in lines[start + 1:]:
if not line.strip() or not line.startswith((" ", "\t")):
break
block.append(line.strip())
return " ".join(block)
def _strip_bracketed(block):
"""Split a usage line into tokens, dropping every bracketed (optional) group."""
tokens, depth, cur = [], 0, ""
for ch in block:
if ch in "[(":
depth += 1
continue
if ch in ")]":
depth -= 1
continue
if depth:
continue
if ch.isspace():
if cur:
tokens.append(cur)
cur = ""
else:
cur += ch
if cur:
tokens.append(cur)
return tokens
def usage_state(help_text):
"""Classify a script's usage line.
Returns one of:
"unreadable" - no argparse-style `usage:` block at all (hand-written usage)
"none-required" - a usage block exists but names no positional and no required option
"required" - at least one positional or required option is named
"""
block = usage_block(help_text)
if block is None:
return "unreadable"
return "required" if _strip_bracketed(block)[1:] else "none-required"
def missing_input_args(help_text):
"""Build an argv that points every positional and required option at a missing file.
Returns None when there is nothing to point at: either the usage block cannot
be recovered (the four scripts with a hand-written parser print their own
usage, not argparse's) or the script takes no required argument at all.
"""
if usage_state(help_text) != "required":
return None
tokens = _strip_bracketed(usage_block(help_text))
args, i = [], 1 # tokens[0] is the program name
while i < len(tokens):
token = tokens[i]
if token.startswith("-"):
args.append(token)
if i + 1 < len(tokens) and not tokens[i + 1].startswith("-"):
args.append(NONEXISTENT)
i += 1
else:
args.append(NONEXISTENT)
i += 1
return args or None
def has_option(help_text, option):
return option in help_text
# Scripts that reach a device, a live frida server or a network peer once their
# arguments parse. They are listed explicitly for one reason: this suite must not
# touch a device, and a development host may well have one attached. Measured on
# the machine this harness was written on, `adb devices` reported one online
# device, and running `coldstart.py --pkg <name>` really did start
# `adb logcat -v threadtime` and capture frames. So the exclusion is not
# theoretical: it is the difference between a unit test and a device test.
#
# `--help` and an unknown flag are still exercised for these scripts: argparse
# answers both before any device call is reached, which was verified for every
# one of them (`--help` returned in well under a second, an unknown flag returned
# 2). It is the argument-complete scenarios that are refused.
DEVICE_OR_NETWORK_BOUND = {
"coldstart.py", "datastore_inject.py", "devsh.py", "frida_rpc_serve.py",
"grab_crash.py", "install_test.py", "lib_map.py", "mt_mcp_probe.py",
"probe_api.py", "run_probe.py", "sig_probe.py", "snap.py",
"spawn_patch_detach.py", "tls_check.py", "usb_net_proxy.py",
}
def execution_refusal(name, scenario, help_text):
"""Why this script must not be executed for `scenario`, or None if it may be.
Returns a skip reason rather than a verdict: not executing a script is a
coverage limit, not a contract violation, and the two must not be reported
with the same word.
"""
if name in DEVICE_OR_NETWORK_BOUND:
return ("%s reaches a device or the network once its arguments parse; %s is not "
"executed because this suite does not touch a device" % (name, scenario))
state = usage_state(help_text)
if state == "unreadable":
return ("%s prints its own usage instead of argparse's, so %s cannot be derived "
"automatically" % (name, scenario))
if state == "none-required":
return ("%s requires no argument, so %s is legal and the contract does not apply"
% (name, scenario))
return None
# The scripts whose --json path is reachable on a bare checkout. Everything else
# that advertises --json needs a sample or a device to get that far; those are
# skipped with that reason rather than asserted against a guess.
JSON_RUNNABLE = {
"doctor.py": ["--json"],
"preflight.py": ["--json"],
}
# Keys the unified JSON contract requires at minimum.
JSON_REQUIRED_KEYS = ("status",)
JSON_CONTEXT_KEYS = ("exit_code", "capability", "evidence", "warnings", "next_action")

262
vendor-new/apk-reverse/tests/cli/test_cli_contract.py Normal file
View File

@@ -0,0 +1,262 @@
"""CLI contract tests: every shipped script, four invocation shapes each.
What is asserted, per the repository-wide contract in REBUILD-BRIEF section 4:
1. `--help` succeeds and never prints an uncaught traceback.
2. An unknown flag is a usage error: exit 2.
3. An invocation with required arguments missing, or with every input pointed
at a file that does not exist, is a usage error: exit 2.
4. A script that advertises `--json` emits a parseable JSON document that
states a `status`.
5. Every observed exit code is one of 0/1/2/3/4.
Two clean outcomes are distinguished, and the distinction is the whole point:
* `xfail` means the script is on the tree and does **not** meet the contract.
The reason carries the measured value. The xfail list this file produces is
the work order for exit-code unification -- it is not a defect to be fixed
here, because every one of those scripts has an owner.
* `skip` means the scenario is not applicable or not reachable on a machine
with no device, no network and no frida. Those are coverage gaps, and they
are reported as such in docs/tool-verification/EXTENSION-test-harness.md.
Each scenario runs once per script and is cached for the session, so the suite
spends its time on assertions rather than on re-spawning interpreters.
"""
from __future__ import annotations
import pytest
from conftest import all_scripts, run_script
from contract import (
CONTRACT_EXIT_CODES, DEVICE_OR_NETWORK_BOUND, JSON_REQUIRED_KEYS,
JSON_RUNNABLE, execution_refusal, has_option, missing_input_args,
parse_leading_json,
)
pytestmark = pytest.mark.cli
SCRIPTS = all_scripts()
BAD_FLAG = "--zzz-not-a-real-flag"
HELP_TIMEOUT = 60
SCENARIO_TIMEOUT = 12
def why(name):
if name in DEVICE_OR_NETWORK_BOUND:
return " (device/network bound: it may legitimately block)"
return ""
def require_returned(result, what):
if result.timed_out:
pytest.xfail("%s did not return within %ss%s; there is no exit code to check"
% (what, SCENARIO_TIMEOUT, why(result.argv[1])))
if result.returncode not in CONTRACT_EXIT_CODES:
pytest.fail("%s returned %r, outside the 0/1/2/3/4 contract vocabulary"
% (what, result.returncode))
def expect_exit(result, expected, what):
"""Pass when the measured code matches; otherwise record it as an xfail."""
require_returned(result, what)
if result.returncode != expected:
pytest.xfail("%s: contract says exit %d, measured %d%s"
% (what, expected, result.returncode, why(result.argv[1])))
# --------------------------------------------------------------------------- #
# one cached run per script per scenario
# --------------------------------------------------------------------------- #
@pytest.fixture(scope="session")
def help_results():
return {n: run_script(n, ["--help"], timeout=HELP_TIMEOUT) for n in SCRIPTS}
@pytest.fixture(scope="session")
def badflag_results():
return {n: run_script(n, [BAD_FLAG], timeout=SCENARIO_TIMEOUT) for n in SCRIPTS}
@pytest.fixture(scope="session")
def empty_results():
"""Bare invocations, for the scripts that may be run at all.
Device-bound scripts are excluded here too: their no-argument path is exactly
where several of them start doing device work (`usb_net_proxy.py` did not
return within 20 s when probed by hand), and this suite does not touch a device.
"""
return {n: run_script(n, [], timeout=SCENARIO_TIMEOUT) for n in SCRIPTS
if n not in DEVICE_OR_NETWORK_BOUND}
@pytest.fixture(scope="session")
def missing_results(help_results):
"""Argument-complete runs against a missing file, for the scripts that may be run at all.
Device-bound scripts are skipped here rather than executed and timed out: on
the host this harness was written on, `adb devices` reported an online device,
so running them would have touched real hardware.
"""
results = {}
for name in SCRIPTS:
if name in DEVICE_OR_NETWORK_BOUND:
continue
args = missing_input_args(help_results[name].output)
if args is None:
continue
results[name] = run_script(name, args, timeout=SCENARIO_TIMEOUT)
return results
# --------------------------------------------------------------------------- #
# 1. --help
# --------------------------------------------------------------------------- #
class TestHelp:
@pytest.mark.parametrize("name", SCRIPTS)
def test_help_never_crashes(self, name, help_results):
"""Hard assertion: an uncaught traceback from --help is a defect, not a drift."""
result = help_results[name]
assert not result.timed_out, "--help did not return for %s" % name
assert not result.has_traceback, result.output[-2000:]
@pytest.mark.parametrize("name", SCRIPTS)
def test_help_prints_something(self, name, help_results):
assert help_results[name].output.strip(), "--help printed nothing"
@pytest.mark.parametrize("name", SCRIPTS)
def test_help_exits_zero(self, name, help_results):
expect_exit(help_results[name], 0, "%s --help" % name)
# --------------------------------------------------------------------------- #
# 2. unknown flag
# --------------------------------------------------------------------------- #
class TestUnknownFlag:
@pytest.mark.parametrize("name", SCRIPTS)
def test_unknown_flag_is_never_success(self, name, badflag_results):
result = badflag_results[name]
require_returned(result, "%s %s" % (name, BAD_FLAG))
assert result.returncode != 0, (
"an unknown flag was reported as success:\n%s" % result.output[-2000:])
@pytest.mark.parametrize("name", SCRIPTS)
def test_unknown_flag_exits_two(self, name, badflag_results):
expect_exit(badflag_results[name], 2, "%s %s" % (name, BAD_FLAG))
# --------------------------------------------------------------------------- #
# 3a. empty invocation
# --------------------------------------------------------------------------- #
class TestEmptyInvocation:
@pytest.mark.parametrize("name", SCRIPTS)
def test_no_arguments_is_a_usage_error(self, name, help_results, empty_results):
"""Only applies when the script actually requires arguments.
A script whose usage block has no positional and no required option may
legitimately succeed with no arguments; demanding exit 2 from those would
invent a rule the contract does not state. Device-bound scripts are not run
at all -- see `contract.execution_refusal`.
"""
refusal = execution_refusal(name, "a bare invocation", help_results[name].output)
if refusal:
pytest.skip(refusal)
expect_exit(empty_results[name], 2, "%s with no arguments" % name)
# --------------------------------------------------------------------------- #
# 3b. bad input
# --------------------------------------------------------------------------- #
class TestMissingInput:
@pytest.mark.parametrize("name", SCRIPTS)
def test_missing_input_file_is_a_usage_error(self, name, help_results, missing_results):
if name not in missing_results:
pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
or "not executed for %s" % name)
expect_exit(missing_results[name], 2, "%s with a missing input" % name)
@pytest.mark.parametrize("name", SCRIPTS)
def test_missing_input_is_never_success(self, name, help_results, missing_results):
if name not in missing_results:
pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
or "not executed for %s" % name)
result = missing_results[name]
require_returned(result, "%s with a missing input" % name)
assert result.returncode != 0, (
"a missing input was reported as success:\n%s" % result.output[-2000:])
@pytest.mark.parametrize("name", SCRIPTS)
def test_bad_input_reports_cleanly_not_as_an_exception(self, name, help_results,
missing_results):
"""A missing file is an expected failure mode; it must not escape as an exception.
Several scripts currently let `FileNotFoundError` propagate. Each is
recorded here as an xfail naming the exception; the fix belongs to that
script's owner, not to this suite.
"""
if name not in missing_results:
pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
or "not executed for %s" % name)
result = missing_results[name]
require_returned(result, "%s with a missing input" % name)
if result.has_traceback:
tail = [l for l in result.output.splitlines() if l.strip()]
pytest.xfail("%s reports a missing input by letting an exception escape "
"instead of exiting 2 cleanly: %s"
% (name, (tail[-1] if tail else "uncaught exception")[:140]))
# --------------------------------------------------------------------------- #
# 4. the --json contract
# --------------------------------------------------------------------------- #
def json_capable(help_results):
return sorted(n for n, r in help_results.items() if has_option(r.output, "--json"))
def test_json_capable_scripts_are_discoverable(help_results):
found = json_capable(help_results)
assert found, "no script advertises --json; the discovery method is broken"
@pytest.mark.parametrize("name,args", sorted(JSON_RUNNABLE.items()))
def test_json_output_is_parseable_and_states_a_status(name, args):
result = run_script(name, args, timeout=120)
require_returned(result, "%s %s" % (name, " ".join(args)))
doc = parse_leading_json(result.stdout)
assert doc is not None, (
"%s %s printed no parseable JSON document:\n%s"
% (name, " ".join(args), result.output[-2000:]))
missing = [k for k in JSON_REQUIRED_KEYS if not isinstance(doc, dict) or k not in doc]
if missing:
present = sorted(doc) if isinstance(doc, dict) else type(doc).__name__
pytest.xfail("%s --json does not carry %s; top-level keys are %s"
% (name, missing, present))
@pytest.mark.parametrize("name", SCRIPTS)
def test_json_capable_but_unreachable_scripts_are_named(name, help_results):
"""Coverage honesty: a --json script is either exercised here or explicitly not."""
if not has_option(help_results[name].output, "--json"):
pytest.skip("%s does not advertise --json" % name)
if name in JSON_RUNNABLE:
return
pytest.skip("%s advertises --json, but reaching that path needs a device or a "
"sample; the gap is listed in EXTENSION-test-harness.md" % name)
# --------------------------------------------------------------------------- #
# 5. the exit-code vocabulary
# --------------------------------------------------------------------------- #
class TestExitCodeVocabulary:
@pytest.mark.parametrize("name", SCRIPTS)
def test_observed_codes_stay_inside_the_vocabulary(self, name, help_results,
badflag_results):
for label, result in (("--help", help_results[name]),
(BAD_FLAG, badflag_results[name])):
if result.returncode is not None:
assert result.returncode in CONTRACT_EXIT_CODES, (
"%s %s returned %r" % (name, label, result.returncode))
assert result.returncode is None or result.returncode >= 0, (
"a negative code is a crash signal, not an exit status: %s %s"
% (name, label))

174
vendor-new/apk-reverse/tests/conftest.py Normal file
View File

@@ -0,0 +1,174 @@
"""Shared fixtures for the apk-reverse regression suite.
The rules this directory follows (they are the point of the directory, not a
style preference):
* Nothing here may need a device, adb, frida, or the network. Routes that do
are recorded as experiments in ``tests/benchmark.md``; their coverage gap is
stated in ``docs/tool-verification/EXTENSION-test-harness.md`` rather than
simulated into a green tick.
* A missing fixture is ``pytest.skip`` with the reason on stdout, never a red
test. ``tools/_work/bench/**`` is git-ignored, so CI legitimately does not
have it and the regression hash test must degrade to a skip there.
* This suite observes behaviour. It never edits anything under
``skills/apk-reverse/scripts/`` -- a defect found here is written into the
report as a work item, not patched in place.
"""
from __future__ import annotations
import importlib.util
import os
import subprocess
import sys
import tempfile
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[1]
SCRIPTS_DIR = REPO_ROOT / "skills" / "apk-reverse" / "scripts"
FIXTURES_DIR = Path(__file__).resolve().parent / "fixtures"
BENCH_L1L3 = REPO_ROOT / "tools" / "_work" / "bench" / "l1l3"
# Some scripts import their siblings by bare name (`from dexutil import ...`);
# loading them as modules requires that directory on sys.path. The suite's own
# directory is added too, so `from conftest import load_script` works regardless
# of how pytest resolves the rootdir.
for _p in (SCRIPTS_DIR, Path(__file__).resolve().parent):
if str(_p) not in sys.path:
sys.path.insert(0, str(_p))
PYTHON = sys.executable
# Children run with this as their working directory, never the scripts directory.
# Measured reason: `coldstart.py` writes its capture under `--out` (default
# `coldstart/`), so a contract test that ran it with the scripts directory as cwd
# left a `skills/apk-reverse/scripts/coldstart/logcat.txt` behind. This suite must
# not write anywhere near the shipped tree, and a script that defaults an output
# path relative to cwd is exactly the case that catches a harness out.
ISOLATED_CWD = tempfile.mkdtemp(prefix="apkrev-suite-cwd-")
# The regression marker established before this harness existed. It is a hash,
# not a golden-file rewrite: scripts/dex_patch_bytes.py applied to the
# l1l3 fixture must still reproduce the file that the recorded benchmark row
# produced.
L1L3_PATCHED_SHA256 = "67ae1e58e7a44777e5ac8f9f823ab21e"
_loaded: dict[str, object] = {}
class ScriptResult:
"""Result of running one script as a subprocess."""
def __init__(self, argv, returncode, stdout, stderr, timed_out=False):
self.argv = argv
self.returncode = returncode
self.stdout = stdout
self.stderr = stderr
self.timed_out = timed_out
@property
def output(self) -> str:
return self.stdout + self.stderr
@property
def has_traceback(self) -> bool:
return "Traceback (most recent call last)" in self.output
def __repr__(self) -> str: # keeps assertion failures readable
return "ScriptResult(argv=%r, rc=%r, timed_out=%r)" % (
self.argv, self.returncode, self.timed_out)
def script_path(name: str) -> Path:
return SCRIPTS_DIR / name
def all_scripts() -> list[str]:
"""Every shipped script, sorted. Used to parameterise the CLI contract tests."""
return sorted(p.name for p in SCRIPTS_DIR.glob("*.py"))
def run_script(name, args=(), timeout=60, cwd=None, stdin=None, env=None):
"""Run one shipped script in a subprocess and capture everything.
The child runs with ``PYTHONIOENCODING=utf-8`` so that a non-UTF-8 console
(a Windows CP936 shell, a CI runner with C locale) does not turn a UTF-8
print into a UnicodeEncodeError and change the exit code under test, and with
an isolated working directory so that a script defaulting an output path
relative to cwd cannot write into the shipped tree.
"""
argv = [PYTHON, str(script_path(name))] + [str(a) for a in args]
child_env = dict(os.environ)
child_env["PYTHONIOENCODING"] = "utf-8"
child_env["PYTHONDONTWRITEBYTECODE"] = "1"
if env:
child_env.update(env)
try:
proc = subprocess.run(
argv, capture_output=True, text=True, encoding="utf-8", errors="replace",
timeout=timeout, cwd=str(cwd or ISOLATED_CWD), input=stdin, env=child_env,
)
except subprocess.TimeoutExpired as exc:
return ScriptResult(argv, None, _text(exc.stdout), _text(exc.stderr),
timed_out=True)
return ScriptResult(argv, proc.returncode, proc.stdout or "", proc.stderr or "")
def _text(blob):
if blob is None:
return ""
if isinstance(blob, bytes):
return blob.decode("utf-8", "replace")
return blob
def load_script(name):
"""Import a shipped script as a module, so pure functions can be unit-tested.
The module is registered under a namespaced name (``apkrev_<stem>``) because
several scripts have siblings with the same basename in other trees.
"""
if name in _loaded:
return _loaded[name]
path = script_path(name)
mod_name = "apkrev_" + path.stem
spec = importlib.util.spec_from_file_location(mod_name, str(path))
if spec is None or spec.loader is None:
raise ImportError("cannot load %s" % path)
module = importlib.util.module_from_spec(spec)
sys.modules[mod_name] = module
spec.loader.exec_module(module)
_loaded[name] = module
return module
def require_path(path: Path, why: str):
"""Skip with a printed reason when a git-ignored fixture is absent."""
if not Path(path).exists():
pytest.skip("fixture absent: %s (%s)" % (path, why))
return Path(path)
@pytest.fixture(scope="session")
def repo_root():
return REPO_ROOT
@pytest.fixture(scope="session")
def scripts_dir():
return SCRIPTS_DIR
@pytest.fixture(scope="session")
def fixtures_dir():
return FIXTURES_DIR
@pytest.fixture(scope="session")
def bench_l1l3():
return require_path(
BENCH_L1L3,
"tools/_work/bench is git-ignored; re-download the l1l3 sample to run the "
"dex_patch_bytes regression hash test",
)

9
vendor-new/apk-reverse/tests/fixtures/apk/.gitignore vendored Normal file
View File

@@ -0,0 +1,9 @@
# The APKs in this directory are committed on purpose.
#
# They are synthesised by ../make_fixture_apk.py (a few kilobytes each, no target data)
# and they are the input of tests/test_so_constpatch.py, which pins the contract that
# scripts/so_constpatch.py must keep when it rebuilds an archive: STORED stays STORED,
# extra / external_attr / create_system survive, and resources.arsc keeps its 4-byte
# alignment. The repository-wide `*.apk` rule exists to keep pulled samples out of the
# tree, which is a different thing from a generated fixture.
!*.apk

BIN
vendor-new/apk-reverse/tests/fixtures/apk/arsc_aligned.apk vendored Normal file
View File

Binary file not shown.

BIN
vendor-new/apk-reverse/tests/fixtures/apk/noextract.apk vendored Normal file
View File

Binary file not shown.

BIN
vendor-new/apk-reverse/tests/fixtures/apk/plain.apk vendored Normal file
View File

Binary file not shown.

BIN
vendor-new/apk-reverse/tests/fixtures/apk/services.apk vendored Normal file
View File

Binary file not shown.

11
vendor-new/apk-reverse/tests/fixtures/logcat_sample.txt vendored Normal file
View File

@@ -0,0 +1,11 @@
--------- beginning of main
01-01 10:00:01.100 1234 1234 I ActivityManager: Start proc 1234:com.example.app/u0a123 for activity
01-01 10:00:01.500 1234 1234 D AndroidRuntime: Shutting down VM
01-01 10:00:01.501 1234 1234 E AndroidRuntime: FATAL EXCEPTION: main
01-01 10:00:01.501 1234 1234 E AndroidRuntime: Process: com.example.app, PID: 1234
01-01 10:00:01.502 1234 1234 E AndroidRuntime: java.lang.VerifyError: Rejecting class Lprobe/Synthetic;
01-01 10:00:01.503 1234 1234 W dalvikvm: Failure to verify dex file: Bad checksum (12345678, expected 9abcdef0)
01-01 10:00:01.504 1234 1234 E AndroidRuntime: Caused by: java.lang.UnsatisfiedLinkError: dlopen failed
01-01 10:00:02.000 1000 1500 I ActivityManager: Displayed com.example.app/.MainActivity: +842ms
01-01 10:00:03.100 1000 1500 W ActivityManager: ANR in com.example.app (com.example.app/.MainActivity)
01-01 10:00:03.200 1000 1500 E ActivityManager: Reason: Input dispatching timed out

369
vendor-new/apk-reverse/tests/fixtures/make_fixture_apk.py vendored Normal file
View File

@@ -0,0 +1,369 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Build the four tiny APKs that pin the APK-rebuild contract of `so_constpatch.py`.
WHY THIS EXISTS
---------------
`so_constpatch.py` rebuilt an APK through `zipfile.ZipInfo(filename, date_time)`, which
drops `extra`, `external_attr`, `create_system` and re-deflates every entry. An entry
that must stay STORED or 4-byte aligned (`resources.arsc`, an uncompressed `lib/*.so`)
can therefore come back compressed or unaligned, and the result is refused by the
installer with `[-124]` -- or simply not loadable when the manifest asks for
`extractNativeLibs="false"`. A regression test for that fix needs inputs that actually
carry the metadata which was being dropped, at a size worth keeping in the repository:
plain.apk DEFLATED entries, no `resources.arsc` -> an unsafe rebuild is not
in play, so this is the "rebuild is allowed" control
arsc_aligned.apk `resources.arsc` STORED and 4-byte aligned, plus a STORED and a
DEFLATED `lib/*.so` -> the storage/alignment contract must survive
noextract.apk the manifest declares `android:extractNativeLibs="false"` and both
`resources.arsc` and `lib/*.so` are STORED+aligned -> a rebuild must
be REFUSED unless `--unsafe-rebuild` is passed
services.apk `META-INF/services/**` entries, a UTF-8 entry name and a non-empty
archive comment -> untouched entries must come back byte-identical
Nothing here is target data: every fixture is synthesised from the constants below. Two
details exist so the fixtures are not weaker than real input:
* `AndroidManifest.xml` is a real binary AXML document (string pool + start element), so
the manifest check reads the same bytes a compiler-produced manifest would give it.
* `resources.arsc` is a valid `RES_TABLE_TYPE` header plus filler: enough to pin the
storage/alignment contract, not a renderable resource table.
* the `lib/*.so` payload is a real (tiny) aarch64 shared object with `.rodata` and
`.shstrtab` section headers, so `--section-aware` exercises the section map instead of
falling back to "no sections".
USAGE
-----
python make_fixture_apk.py --out tests/fixtures/apk
The generated files are committed. Regenerate only when the contract itself changes;
the output is deterministic (fixed timestamps, fixed compression level, no randomness).
"""
from __future__ import annotations
import argparse
import hashlib
import os
import struct
import zlib
ALIGN = 4
STAMP_UNIX = 1577836800 # 2020-01-01T00:00:00Z
FORCE_UTF8 = 0x800 # general purpose bit 11
# ------------------------------------------------------------------ small builders
def _dos(dt):
"""(time, date) MS-DOS words, the way a zip local header stores them."""
y, mo, d, h, mi, s = dt
return (h << 11) | (mi << 5) | (s // 2), ((y - 1980) << 9) | (mo << 5) | d
def _deflate(data):
"""Raw deflate (no zlib wrapper) -- what a method-8 zip entry holds."""
c = zlib.compressobj(9, zlib.DEFLATED, -15)
return c.compress(data) + c.flush()
def ext_stamp(unix=STAMP_UNIX):
"""An extended-timestamp extra field (id 0x5455), so `extra` is never empty.
Real APKs carry this on entries written by a toolchain that stamps mtimes, and it is
exactly the field a rebuilt archive used to lose.
"""
return b'\x55\x54' + struct.pack('<H', 5) + b'\x01' + struct.pack('<I', unix)
def align_record(pad):
"""A private extra record of `4 + pad` bytes, i.e. `pad` bytes of alignment.
A zip extra area is a sequence of (id, 2-byte size, payload) records, so the smallest
record is 4 bytes and a 1-3 byte pad cannot be expressed on its own. Appending a
record of length `4 + pad` shifts the data offset by `pad` (mod 4), which is exactly
what is needed, without inserting a filler entry.
"""
if pad <= 0:
return b''
return b'\xca\xca' + struct.pack('<H', pad) + b'\x00' * pad
def make_elf(const=b'apkhuan', decoy=b'libapkhuanx.so'):
"""A minimal aarch64 ET_DYN object: ELF header, `.rodata`, `.shstrtab`, shdrs.
`const` sits alone between NULs (an isolated constant, safe to rewrite in place);
`decoy` contains the same bytes as a substring, which is the case `--find` must
report as NOT isolated.
"""
rodata = b'\x00' + const + b'\x00' + decoy + b'\x00'
shstr = b'\x00.rodata\x00.shstrtab\x00'
ehsize, shentsize, shnum, shstrndx = 64, 64, 3, 2
ro_off = ehsize
shstr_off = ro_off + len(rodata)
sh_off = (shstr_off + len(shstr) + 7) & ~7
ident = b'\x7fELF' + bytes([2, 1, 1, 0]) + b'\x00' * 8
ehdr = ident + struct.pack('<HHIQQQIHHHHHH',
3, # e_type = ET_DYN
183, # e_machine= EM_AARCH64
1, # e_version
0, 0, # e_entry, e_phoff
sh_off, # e_shoff
0, # e_flags
ehsize, 0, 0, shentsize, shnum, shstrndx)
def shdr(name, typ, flags, addr, off, size, align=1):
return struct.pack('<IIQQQQIIQQ', name, typ, flags, addr, off, size,
0, 0, align, 0)
shdrs = (shdr(0, 0, 0, 0, 0, 0)
+ shdr(1, 1, 0x2, 0x1000, ro_off, len(rodata)) # .rodata PROGBITS
+ shdr(9, 3, 0, 0, shstr_off, len(shstr))) # .shstrtab STRTAB
pad = b'\x00' * (sh_off - (shstr_off + len(shstr)))
return ehdr + rodata + shstr + pad + shdrs
def _string_pool(strings):
"""A UTF-8 (flag 0x100) RES_STRING_POOL chunk."""
offsets, blob = [], b''
for s in strings:
raw = s.encode('utf-8')
assert len(raw) < 0x80 and len(s) < 0x80, 'fixture pool is ASCII/short on purpose'
offsets.append(len(blob))
blob += bytes([len(raw), len(s)]) + raw + b'\x00'
strings_start = 28 + 4 * len(strings)
header = struct.pack('<HHI', 0x0001, 28, strings_start + len(blob))
header += struct.pack('<IIIII', len(strings), 0, 0x100, strings_start, 0)
return header + b''.join(struct.pack('<I', o) for o in offsets) + blob
def make_axml(package='com.example.fixture', extract_native_libs=None):
"""A minimal but structurally valid binary AndroidManifest.xml.
Layout: file header, one string pool, one START_ELEMENT chunk for <manifest>. The
attribute shape matches what aapt2 emits: `extractNativeLibs` is a boolean
(dataType 0x12) with no raw string, namespaced to the android URI.
"""
android_uri = 'http://schemas.android.com/apk/res/android'
strings = ['android', android_uri, 'manifest', 'package', 'extractNativeLibs', package]
s = {v: i for i, v in enumerate(strings)}
attrs = [(0xFFFFFFFF, s['package'], 0xFFFFFFFF, 0x03, s[package])]
if extract_native_libs is not None:
attrs.append((s[android_uri], s['extractNativeLibs'], 0xFFFFFFFF, 0x12,
1 if extract_native_libs else 0))
pool = _string_pool(strings)
def chunk(ctype, node, tail):
# chunk size covers the 8-byte chunk header as well as `node` and `tail`.
body = node + tail
return struct.pack('<HHI', ctype, 16, 8 + len(body)) + body
node = struct.pack('<II', 1, 0xFFFFFFFF) # lineNumber, comment
body = struct.pack('<II', 0xFFFFFFFF, s['manifest']) # ns, name
body += struct.pack('<HHHHHH', 20, 20, len(attrs), 0, 0, 0)
for ns, name, raw, dtype, data in attrs:
body += struct.pack('<III', ns, name, raw)
body += struct.pack('<HBBI', 8, 0, dtype, data)
element = chunk(0x0102, node, body)
end = chunk(0x0103, node, struct.pack('<II', 0xFFFFFFFF, s['manifest']))
total = 8 + len(pool) + len(element) + len(end)
return struct.pack('<HHI', 0x0003, 8, total) + pool + element + end
def make_arsc(filler=60):
"""A `RES_TABLE_TYPE` header with zero packages, plus filler bytes."""
body = struct.pack('<I', 0) + b'\x00' * filler
return struct.pack('<HHI', 0x0002, 12, 12 + len(body)) + body
def make_dex(classes=2):
"""A dex header whose file_size matches, so a reader does not reject it outright.
Only the header is written: the fixtures exist to pin zip behaviour, not dex
semantics, and a full dex would be the largest file in the repository's test tree.
"""
header_size, endian = 0x70, 0x12345678
size = header_size + 4 * classes
h = bytearray(header_size)
h[0:8] = b'dex\n035\x00'
h[32:36] = struct.pack('<I', size)
h[36:40] = struct.pack('<I', header_size)
h[40:44] = struct.pack('<I', endian)
h[104:108] = struct.pack('<I', 0) # data_size
h[108:112] = struct.pack('<I', header_size)
return bytes(h) + b'\x00' * (size - header_size)
# ------------------------------------------------------------------ zip writer
class Entry(object):
"""One fixture entry, with the metadata the rebuild must carry over."""
def __init__(self, name, data, method=8, date_time=(2020, 1, 1, 0, 0, 0),
extra=None, external_attr=0o100644 << 16, internal_attr=0,
create_system=3, comment=b'', align=False, utf8=False):
self.name = name
self.data = data
self.method = method
self.date_time = date_time
self.extra = ext_stamp() if extra is None else extra
self.external_attr = external_attr
self.internal_attr = internal_attr
self.create_system = create_system
self.comment = comment
self.align = align
self.utf8 = utf8
@property
def flags(self):
return FORCE_UTF8 if self.utf8 else 0
def write_apk(path, entries, archive_comment=b''):
"""Write a zip the way zipalign does: STORED stays STORED, aligned stays aligned.
Written by hand rather than through `zipfile` because alignment is a property of the
local header length, which `zipfile` cannot express -- the same reason `repack.py`
hand-rolls its writer.
"""
out = bytearray()
cd = []
for e in entries:
name_b = e.name.encode('utf-8')
payload = e.data if e.method == 0 else _deflate(e.data)
crc = zlib.crc32(e.data) & 0xFFFFFFFF
extra = e.extra
if e.align:
head = len(out) + 30 + len(name_b) + len(extra)
extra += align_record((ALIGN - head % ALIGN) % ALIGN)
t, d = _dos(e.date_time)
local_off = len(out)
out += struct.pack('<IHHHHHIIIHH', 0x04034B50, 20, e.flags, e.method, t, d,
crc, len(payload), len(e.data), len(name_b), len(extra))
out += name_b + extra + payload
data_off = local_off + 30 + len(name_b) + len(extra)
if e.align and data_off % ALIGN:
raise RuntimeError('%s is not %d-byte aligned' % (e.name, ALIGN))
cd.append((e, name_b, crc, len(payload), len(extra), local_off))
cd_off = len(out)
for e, name_b, crc, csize, elen, local_off in cd:
t, d = _dos(e.date_time)
out += struct.pack('<IHHHHHHIIIHHHHHII', 0x02014B50,
(e.create_system << 8) | 20, 20, e.flags, e.method, t, d,
crc, csize, len(e.data), len(name_b), len(e.extra),
len(e.comment), 0, e.internal_attr, e.external_attr,
local_off)
out += name_b + e.extra + e.comment
cd_size = len(out) - cd_off
out += struct.pack('<IHHHHIIH', 0x06054B50, 0, 0, len(cd), len(cd), cd_size,
cd_off, len(archive_comment))
out += archive_comment
folder = os.path.dirname(os.path.abspath(path))
if folder:
os.makedirs(folder, exist_ok=True)
with open(path, 'wb') as fh:
fh.write(bytes(out))
return bytes(out)
# ------------------------------------------------------------------ the fixtures
def definitions():
"""name -> (entries, archive comment). Deterministic; no timestamps from the clock.
Every STORED entry is 4-byte aligned, the way `zipalign` lays a real APK out: that is
what makes "the rebuild did not break alignment" a checkable claim, and what keeps a
rebuilt fixture acceptable to `zipalign -c`.
"""
elf = make_elf()
dex = make_dex()
strings_xml = b'<?xml version="1.0" encoding="utf-8"?>\n<resources/>\n'
plain_manifest = make_axml('com.example.plain')
noextract_manifest = make_axml('com.example.noextract', extract_native_libs=False)
st = dict(method=0, align=True, date_time=(2021, 6, 15, 12, 34, 56))
plain = [
Entry('AndroidManifest.xml', plain_manifest, **st),
Entry('classes.dex', dex, external_attr=0o100600 << 16),
Entry('res/values/strings.xml', strings_xml),
Entry('lib/arm64-v8a/libfoo.so', elf),
Entry('META-INF/MANIFEST.MF', b'Manifest-Version: 1.0\r\n\r\n'),
]
arsc_aligned = [
Entry('AndroidManifest.xml', plain_manifest, **st),
Entry('classes.dex', dex),
# DEFLATED and patched: its compressed size moves, so resources.arsc has to be
# re-aligned on output. This is the case the old writer silently broke.
Entry('lib/arm64-v8a/libfoo.so', elf),
# STORED: must stay STORED, must not be re-compressed, must stay aligned.
Entry('lib/arm64-v8a/libbar.so', elf, method=0, align=True,
external_attr=0o100755 << 16),
Entry('resources.arsc', make_arsc(), method=0, align=True),
Entry('res/values/strings.xml', strings_xml),
Entry('META-INF/services/com.example.Provider',
b'com.example.ProviderImpl\n'),
]
noextract = [
Entry('AndroidManifest.xml', noextract_manifest, **st),
Entry('classes.dex', dex),
Entry('lib/arm64-v8a/libfoo.so', elf, method=0, align=True),
Entry('resources.arsc', make_arsc(), method=0, align=True),
]
services = [
Entry('AndroidManifest.xml', plain_manifest, **st),
Entry('META-INF/services/com.example.Provider', b'com.example.ProviderImpl\n'),
Entry('META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory',
b'kotlinx.coroutines.internal.MainDispatcherFactoryImpl\n', method=0,
align=True),
Entry('lib/x86_64/libfoo.so', elf, external_attr=0),
# A non-ASCII name with the UTF-8 flag (bit 11) set: the raw name bytes and the
# flag must both survive a rebuild, and this entry must not be touched at all.
Entry('res/raw/\u6d4b\u8bd5.bin', b'fixture-payload\x00', utf8=True,
create_system=0, external_attr=0, internal_attr=1,
comment=b'fixture comment'),
]
return {
'plain.apk': (plain, b''),
'arsc_aligned.apk': (arsc_aligned, b''),
'noextract.apk': (noextract, b''),
'services.apk': (services, b'fixture-archive-comment'),
}
def main():
ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])
ap.add_argument('--out', default=os.path.join(os.path.dirname(os.path.abspath(__file__)),
'apk'),
help='directory to write the fixtures into')
ap.add_argument('--list', action='store_true', help='print names and exit')
a = ap.parse_args()
defs = definitions()
if a.list:
for name in sorted(defs):
print(name)
return 0
for name in sorted(defs):
entries, comment = defs[name]
path = os.path.join(a.out, name)
blob = write_apk(path, entries, comment)
print('%-18s %6d bytes %2d entries sha256=%s'
% (name, len(blob), len(entries), hashlib.sha256(blob).hexdigest()[:16]))
print('wrote %d fixture(s) to %s' % (len(defs), os.path.abspath(a.out)))
return 0
if __name__ == '__main__':
raise SystemExit(main())

20
vendor-new/apk-reverse/tests/fixtures/tombstone_sample.txt vendored Normal file
View File

@@ -0,0 +1,20 @@
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'google/sdk_gphone64_arm64/sdk_gphone64_arm64:14/UE1A.230829.036/11228888:userdebug/dev-keys'
Revision: '0'
ABI: 'arm64'
Timestamp: 2026-01-01 10:00:01.500000000+0000
Process uptime: 5
Cmdline: com.example.app
pid: 12345, tid: 12345, name: com.example.app >>> com.example.app <<<
uid: 10123
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0000000000000010
x0 0000000000000010 x1 0000007a12345678 x2 0000000000000000 x3 0000000000000020
x4 0000007a98765432 x5 0000000000000001 x6 0000000000000000 x7 0000000000000008
x8 0000000000000098 x9 0000000000000000 x10 0000007ffabc1234 x11 0000000000000000
sp 0000007ffabc1110 lr 0000007a11112222 pc 0000007a111128cc pst 0000000060001000
backtrace:
#00 pc 00000000000128cc /data/app/~~AbCdEf==/com.example.app-1/lib/arm64/libfoo.so
#01 pc 0000000000020f40 /data/app/~~AbCdEf==/com.example.app-1/lib/arm64/libfoo.so (Java_com_example_app_Native_check+120)
#02 pc 00000000000cccd0 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start+256)
#03 pc 0000000000055488 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68)
Cause: null pointer dereference? no -- read of a small integer address

334
vendor-new/apk-reverse/tests/integration/test_adb_command_construction.py Normal file
View File

@@ -0,0 +1,334 @@
"""Device-side command construction, exercised without a device.
The behaviour under test is the one the repository's own documentation says is
the reason these helpers exist: a device command is built as an **argv list** and
handed to `subprocess` directly, so the host shell never expands `$`, `|`, `>`,
`;` or quotes before adb sees them. Every assertion here inspects the argv list
that would have been executed; none of them launches adb.
"""
from __future__ import annotations
import re
import subprocess
import sys
import pytest
from conftest import load_script
devsh = load_script("devsh.py")
coldstart = load_script("coldstart.py")
install_test = load_script("install_test.py")
pytestmark = pytest.mark.integration
class Recorder:
"""Captures the argv lists handed to a patched subprocess runner."""
def __init__(self, stdout="", stderr="", returncode=0):
self.calls = []
self.stdout = stdout
self.stderr = stderr
self.returncode = returncode
def __call__(self, cmd, *args, **kwargs):
self.calls.append((list(cmd), kwargs))
return subprocess.CompletedProcess(cmd, self.returncode, self.stdout, self.stderr)
@property
def last(self):
return self.calls[-1][0]
# --------------------------------------------------------------------------- #
# devsh.py
# --------------------------------------------------------------------------- #
@pytest.fixture
def devsh_recorder(monkeypatch):
rec = Recorder(stdout="ok\n")
monkeypatch.setattr(devsh.subprocess, "run", rec)
monkeypatch.delenv("ADB_PATH", raising=False)
monkeypatch.delenv("ADB_SERIAL", raising=False)
return rec
def run_devsh(monkeypatch, rec, argv):
monkeypatch.setattr(sys, "argv", ["devsh.py"] + argv)
return devsh.main()
class TestDevshCommandConstruction:
def test_shell_command_is_one_argv_element_not_a_shell_string(self, monkeypatch, devsh_recorder):
"""`|` and `grep` must survive as literal text inside a single argument."""
assert run_devsh(monkeypatch, devsh_recorder, ["sh", "pm list packages | grep myapp"]) == 0
assert devsh_recorder.last == ["adb", "shell", "pm list packages | grep myapp"]
def test_serial_is_injected_as_dash_s(self, monkeypatch, devsh_recorder):
run_devsh(monkeypatch, devsh_recorder, ["--serial", "SERIAL123", "sh", "id"])
assert devsh_recorder.last == ["adb", "-s", "SERIAL123", "shell", "id"]
def test_serial_equals_form_is_accepted(self, monkeypatch, devsh_recorder):
run_devsh(monkeypatch, devsh_recorder, ["--adb=x", "--serial=SERIAL123", "sh", "id"])
assert devsh_recorder.last == ["x", "-s", "SERIAL123", "shell", "id"]
def test_adb_path_comes_from_the_environment_when_not_given(self, monkeypatch, devsh_recorder):
monkeypatch.setenv("ADB_PATH", "/opt/platform-tools/adb")
monkeypatch.setenv("ADB_SERIAL", "ENVSERIAL")
run_devsh(monkeypatch, devsh_recorder, ["sh", "id"])
assert devsh_recorder.last == ["/opt/platform-tools/adb", "-s", "ENVSERIAL", "shell", "id"]
def test_device_listing_explicitly_ignores_the_serial(self, monkeypatch, devsh_recorder):
run_devsh(monkeypatch, devsh_recorder, ["dev"])
assert devsh_recorder.last == ["adb", "devices", "-l"]
def test_pull_and_push_keep_their_arguments_positional(self, monkeypatch, devsh_recorder):
run_devsh(monkeypatch, devsh_recorder, ["pull", "/data/local/tmp/x.apk", "./x.apk"])
assert devsh_recorder.last == ["adb", "pull", "/data/local/tmp/x.apk", "./x.apk"]
run_devsh(monkeypatch, devsh_recorder, ["push", "./x.apk", "/data/local/tmp/x.apk"])
assert devsh_recorder.last == ["adb", "push", "./x.apk", "/data/local/tmp/x.apk"]
def test_a_nonzero_device_status_is_reported_as_one(self, monkeypatch, devsh_recorder):
devsh_recorder.returncode = 1
assert run_devsh(monkeypatch, devsh_recorder, ["sh", "false"]) == 1
def test_the_su_wrapper_is_posix_single_quoted(self, monkeypatch, devsh_recorder):
# Assert the *property*, not the representation. The previous version of this test pinned
# `su -c "..."` with `\"` escapes, which is exactly the form that let `;`, `$( )` and
# backticks through to the device shell -- the test was guarding the defect.
run_devsh(monkeypatch, devsh_recorder, ["su", 'pm grant com.example.app "android.permission.X"'])
payload = devsh_recorder.last[-1]
assert payload.startswith("su -c '") and payload.endswith("'")
inner = payload[len("su -c '"):-1]
# A POSIX single-quoted string: no bare quote may appear except as the '\'' escape.
assert re.fullmatch(r"[^']*('\\''[^']*)*", inner), payload
assert '"' in inner # the caller's double quotes survive as literal characters
def test_an_unknown_mode_prints_usage_and_exits_two(self, monkeypatch, devsh_recorder):
assert run_devsh(monkeypatch, devsh_recorder, ["frobnicate", "x"]) == 2
def test_no_arguments_prints_usage_and_exits_two(self, monkeypatch, devsh_recorder):
assert run_devsh(monkeypatch, devsh_recorder, []) == 2
class TestDevshSuQuotingGap:
"""The quoting contract, now unified through `scripts/device_shell.py`.
This class used to record a *gap*: `devsh.su` wrapped the command in double quotes and escaped
only `"`, so `$`, backticks, `;` and `|` from a caller-supplied value still reached the device
shell, while `coldstart.Device.shell` used the POSIX single-quote escape and did neutralise them.
Both now go through the shared module; the tests below pin the property from both sides, so a
future edit that reintroduces double-quote escaping fails here instead of on a device.
"""
@pytest.mark.parametrize("payload", [
"id; rm -rf /sdcard/x",
"id$(touch /data/local/tmp/pwned)",
"id`touch /data/local/tmp/pwned`",
"id | tee /data/local/tmp/pwned",
"id 'quoted'",
])
def test_su_wrapper_neutralises_shell_metacharacters(self, monkeypatch, devsh_recorder, payload):
run_devsh(monkeypatch, devsh_recorder, ["su", payload])
wrapped = devsh_recorder.last[-1]
assert wrapped.startswith("su -c '"), wrapped
inner = wrapped[len("su -c '"):-1]
# The payload must be present verbatim, with only the '\'' sequence able to break out --
# i.e. the whole payload is one argument to sh, not syntax.
assert re.fullmatch(r"[^']*('\\''[^']*)*", inner), wrapped
assert payload.replace("'", "'\\''") == inner
def test_a_malformed_path_is_refused_before_the_device_shell(self, monkeypatch, devsh_recorder):
"""`suf` interpolates a path, so it validates rather than quotes-and-hopes."""
rc = run_devsh(monkeypatch, devsh_recorder, ["suf", "/data/local/tmp/x;id", "200"])
assert rc == 2
assert not devsh_recorder.calls, "the refused command must never reach adb"
def test_coldstart_uses_the_posix_single_quote_escape(self, monkeypatch):
"""The contrasting implementation: this one does neutralise metacharacters."""
rec = Recorder()
monkeypatch.setattr(coldstart, "run", rec)
dev = coldstart.Device(None, use_su=False)
dev.su = True
dev.shell("echo 'hi'; rm -rf /sdcard/x")
payload = rec.last[-1]
assert payload.startswith("su -c '")
assert payload.endswith("'")
assert "'\\''" in payload, "the POSIX single-quote escape is missing"
# --------------------------------------------------------------------------- #
# coldstart.Device: command shape and screenshot handling
# --------------------------------------------------------------------------- #
class TestDeviceHelperCommands:
def make_device(self, monkeypatch, serial=None):
rec = Recorder()
monkeypatch.setattr(coldstart, "run", rec)
dev = coldstart.Device(serial, use_su=False)
return dev, rec
def test_shell_without_root_passes_the_command_as_one_argument(self, monkeypatch):
dev, rec = self.make_device(monkeypatch, serial="SERIAL123")
dev.shell("dumpsys window | grep mCurrentFocus")
assert rec.last == ["adb", "-s", "SERIAL123", "shell", "dumpsys window | grep mCurrentFocus"]
def test_shell_without_a_serial_omits_dash_s(self, monkeypatch):
dev, rec = self.make_device(monkeypatch)
dev.shell("id")
assert rec.last == ["adb", "shell", "id"]
def test_foreground_activity_parses_the_resumed_component(self, monkeypatch):
dev, rec = self.make_device(monkeypatch, serial="S")
rec.stdout = ("mResumedActivity: ActivityRecord{9f2c1 u0 "
"com.example.app/.MainActivity t42}\n")
assert coldstart.foreground_activity(dev) == "com.example.app/.MainActivity"
def test_foreground_activity_returns_none_when_absent(self, monkeypatch):
dev, rec = self.make_device(monkeypatch, serial="S")
rec.stdout = "no matches\n"
assert coldstart.foreground_activity(dev) is None
def test_installed_facts_extracts_version_and_uid(self, monkeypatch):
dev, rec = self.make_device(monkeypatch, serial="S")
def respond(cmd, *args, **kwargs):
rec.calls.append((list(cmd), kwargs))
if "versionCode" in cmd[-1]:
out = " versionCode=42 minSdk=24 targetSdk=34\n versionName=1.2.3\n"
elif "sha256sum" in cmd[-1]:
out = "a" * 64 + " /data/app/base.apk\n"
elif "base.apk" in cmd[-1]:
out = "/data/app/~~x==/com.example.app-1/base.apk\n"
elif "userId=" in cmd[-1]:
out = " userId=10123\n"
else:
out = ""
return subprocess.CompletedProcess(cmd, 0, out, "")
monkeypatch.setattr(coldstart, "run", respond)
facts = coldstart.installed_facts(dev, "com.example.app")
assert facts["versionName"] == "1.2.3"
assert facts["uid"] == "10123"
assert facts["apk_sha256"] == "a" * 64
def test_version_code_swallows_the_rest_of_the_line(self, monkeypatch):
"""Measured sub-string behaviour of `installed_facts`, recorded as a work item.
The extraction is `r'\\s*(versionCode|...)=(.*)'`, so on a real
`dumpsys package` line -- `versionCode=42 minSdk=24 targetSdk=34` --
`facts['versionCode']` is the whole remainder, not `42`. It is still a
usable before/after fingerprint, which is how the caller uses it, but the
field name overstates what it holds.
"""
dev = coldstart.Device(None, use_su=False)
def respond(cmd, *args, **kwargs):
if "versionCode" in cmd[-1]:
out = " versionCode=42 minSdk=24 targetSdk=34\n"
else:
out = ""
return subprocess.CompletedProcess(cmd, 0, out, "")
monkeypatch.setattr(coldstart, "run", respond)
facts = coldstart.installed_facts(dev, "com.example.app")
assert facts["versionCode"] == "42 minSdk=24 targetSdk=34"
class TestScreenshotValidation:
"""`screencap` is judged by the bytes on disk, not by the command's exit status."""
PNG_MAGIC = b"\x89PNG\r\n\x1a\n"
def make_device(self, monkeypatch):
dev = coldstart.Device(None, use_su=False)
return dev
def test_a_png_header_is_accepted_and_the_size_returned(self, monkeypatch, tmp_path):
dev = self.make_device(monkeypatch)
payload = self.PNG_MAGIC + b"\x00" * 100
def fake_run(cmd, *args, **kwargs):
kwargs["stdout"].write(payload)
return subprocess.CompletedProcess(cmd, 0, "", "")
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
out = tmp_path / "shot.png"
assert dev.screencap(str(out)) == len(payload)
assert out.read_bytes() == payload
def test_the_capture_uses_exec_out_not_shell_redirect(self, monkeypatch, tmp_path):
dev = self.make_device(monkeypatch)
seen = []
def fake_run(cmd, *args, **kwargs):
seen.append(list(cmd))
kwargs["stdout"].write(self.PNG_MAGIC + b"\x00" * 100)
return subprocess.CompletedProcess(cmd, 0, "", "")
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
dev.screencap(str(tmp_path / "shot.png"))
assert seen[0] == ["adb", "exec-out", "screencap", "-p"]
def test_a_non_png_body_is_rejected(self, monkeypatch, tmp_path):
"""A device that printed an error into the stream must not be read as a frame."""
dev = self.make_device(monkeypatch)
def fake_run(cmd, *args, **kwargs):
kwargs["stdout"].write(b"error: device offline\n")
return subprocess.CompletedProcess(cmd, 0, "", "")
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
assert dev.screencap(str(tmp_path / "shot.png")) == -1
def test_a_truncated_body_is_rejected(self, monkeypatch, tmp_path):
dev = self.make_device(monkeypatch)
def fake_run(cmd, *args, **kwargs):
kwargs["stdout"].write(b"\x89PNG")
return subprocess.CompletedProcess(cmd, 0, "", "")
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
assert dev.screencap(str(tmp_path / "shot.png")) == -1
def test_a_timeout_is_reported_as_a_failed_capture(self, monkeypatch, tmp_path):
dev = self.make_device(monkeypatch)
def fake_run(cmd, *args, **kwargs):
raise subprocess.TimeoutExpired(cmd, 30)
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
assert dev.screencap(str(tmp_path / "shot.png")) == -1
# --------------------------------------------------------------------------- #
# install_test.py: argument plumbing
# --------------------------------------------------------------------------- #
class TestInstallTestAdbWrappers:
def test_serial_is_injected_as_dash_s(self, monkeypatch):
rec = Recorder()
monkeypatch.setattr(install_test.subprocess, "run", rec)
install_test.adb(["shell", "id"], "SERIAL")
assert rec.last == ["adb", "-s", "SERIAL", "shell", "id"]
def test_without_a_serial_no_dash_s_is_added(self, monkeypatch):
rec = Recorder()
monkeypatch.setattr(install_test.subprocess, "run", rec)
install_test.adb(["devices", "-l"])
assert rec.last == ["adb", "devices", "-l"]
def test_stderr_is_appended_so_device_errors_are_not_silently_dropped(self, monkeypatch):
rec = Recorder(stdout="out", stderr="error: device offline")
monkeypatch.setattr(install_test.subprocess, "run", rec)
combined = install_test.adb(["devices"])
assert "out" in combined
assert "device offline" in combined
def test_su_wrapper_keeps_the_command_as_one_argument(self, monkeypatch):
rec = Recorder()
monkeypatch.setattr(install_test.subprocess, "run", rec)
install_test.su('pm grant com.example.app "android.permission.X"', "S")
assert rec.last == [
"adb", "-s", "S", "shell",
'su -c "pm grant com.example.app \\"android.permission.X\\""',
]

199
vendor-new/apk-reverse/tests/integration/test_check_commands_gate.py Normal file
View File

@@ -0,0 +1,199 @@
"""Regression tests for the documented-command gate itself.
`check_commands.py` is the gate that proves other documentation is runnable, so a blind spot in it
is worth more than a blind spot elsewhere: it turns "146 commands checked, 0 drift" into a claim the
repository cannot back. One such blind spot is pinned here, exactly as it was found.
**What was wrong.** `resolve_script()` tried the *document's own directory* first when resolving a
relative path. A bare `python analyze_pair.py` written inside `docs/tool-verification/` therefore
resolved to `tools/_work/bench/unpack/b3/analyze_pair.py` -- a real file, inside the gitignored
workbench -- and scored `ok`. Seven commands in two evidence files used that form while the same
files wrote the correct path elsewhere, and a reader copying one from the repository root gets
"No such file". The gate said the documentation was clean.
The tests below drive the resolver and the finding classifier directly rather than through the CLI,
so they stay fast and do not depend on the current state of the documentation.
"""
from __future__ import annotations
import sys
from pathlib import Path
import pytest
from conftest import REPO_ROOT
pytestmark = pytest.mark.integration
CHECK_COMMANDS = REPO_ROOT / "check_commands.py"
@pytest.fixture(scope="module")
def gate():
"""The gate as a module, so its internals can be driven without a subprocess."""
sys.path.insert(0, str(REPO_ROOT))
import importlib
return importlib.import_module("check_commands")
def _workbench_script_name(gate):
"""A `.py` basename that exists only under tools/, or skip if the workbench is absent."""
names = sorted(gate._workbench_names())
if not names:
pytest.skip("no workbench present (tools/ is gitignored and may not exist in a checkout)")
shipped = {p.name for p in (REPO_ROOT / "skills" / "apk-reverse" / "scripts").glob("*.py")}
root = {p.name for p in REPO_ROOT.glob("*.py")}
for name in names:
if name not in shipped and name not in root:
return name
pytest.skip("every workbench script name is also shipped; nothing to test with")
def test_a_bare_name_that_only_exists_in_the_workbench_is_drift(gate):
"""The exact defect: the name exists *somewhere*, which used to be enough to pass it."""
name = _workbench_script_name(gate)
doc = str(REPO_ROOT / "docs" / "tool-verification" / "EXTENSION-extraction-shell-bench.md")
path, why = gate.resolve_script(name, doc, include_workbench=False)
assert path is None
assert why == "unqualified-workbench", why
text = "```\n$ python %s --flag\n```\n" % name
findings = gate.command_findings(doc, text, {}, False)
kinds = [f["kind"] for f in findings]
assert "unqualified-workbench" in kinds, findings
def test_a_path_qualified_workbench_reference_is_a_skip_not_drift(gate):
"""`tools/_work/...` is deliberate in an evidence record: a skip, reported with its reason."""
name = _workbench_script_name(gate)
doc = str(REPO_ROOT / "docs" / "tool-verification" / "EXTENSION-extraction-shell-bench.md")
_path, why = gate.resolve_script("tools/_work/bench/unpack/b3/%s" % name, doc,
include_workbench=False)
assert why == "workbench"
@pytest.fixture
def fake_repo(gate, tmp_path, monkeypatch):
"""A repository-shaped tree with its own workbench, so resolution can be tested anywhere.
This exists because the workbench-dependent tests above cannot run on a clean checkout: the
`tools/` tree is gitignored, so on CI they skip and the gate's most important property goes
unverified exactly where it matters. The tree built here has the layout the resolver assumes
(a root, a skill dir, a gitignored `tools/`) and nothing else.
"""
root = tmp_path / "repo"
(root / "skills" / "apk-reverse" / "scripts").mkdir(parents=True)
(root / "docs" / "tool-verification").mkdir(parents=True)
(root / "tools" / "_work" / "bench").mkdir(parents=True)
(root / "README.md").write_text("# root\n", encoding="utf-8")
(root / "skills" / "apk-reverse" / "scripts" / "shipped.py").write_text(
'"""capability: x\n"""\nimport argparse\np = argparse.ArgumentParser()\n'
'p.add_argument("--flag")\n', encoding="utf-8")
(root / "tools" / "_work" / "bench" / "solo_in_workbench.py").write_text(
"print('x')\n", encoding="utf-8")
(root / "docs" / "tool-verification" / "record.md").write_text("# record\n", encoding="utf-8")
monkeypatch.setattr(gate, "ROOT", str(root))
monkeypatch.setattr(gate, "SKILL_DIR", str(root / "skills" / "apk-reverse"))
monkeypatch.setattr(gate, "SCRIPT_DIR", str(root / "skills" / "apk-reverse" / "scripts"))
gate._WORKBENCH_INDEX.clear()
yield root
gate._WORKBENCH_INDEX.clear()
def test_resolution_order_is_root_then_skill_then_document(gate, fake_repo):
"""The corrected order, asserted against a tree that exists on any machine."""
doc = str(fake_repo / "docs" / "tool-verification" / "record.md")
path, why = gate.resolve_script("README.md", doc, include_workbench=False)
assert why == "ok" and path.endswith("README.md")
path, why = gate.resolve_script("shipped.py", doc, include_workbench=False)
assert why == "ok" and path.endswith("shipped.py")
def test_a_workbench_only_name_is_drift_and_a_qualified_one_is_not(gate, fake_repo):
doc = str(fake_repo / "docs" / "tool-verification" / "record.md")
path, why = gate.resolve_script("solo_in_workbench.py", doc, include_workbench=False)
assert (path, why) == (None, "unqualified-workbench")
path, why = gate.resolve_script("tools/_work/bench/solo_in_workbench.py", doc,
include_workbench=False)
assert (path, why) == (None, "workbench")
def test_a_bare_name_the_document_paths_elsewhere_is_a_skip(gate, fake_repo):
"""The fix that made the gate behave the same on CI as on a populated machine.
`solo_in_workbench.py` appears once bare and once with a path in the same document. The
document has already told the reader where it lives, so the bare line is a shorthand -- and
that verdict must not depend on the gitignored tree being visible, which is how the first
version of this fix passed locally and failed on CI.
"""
doc = str(fake_repo / "docs" / "tool-verification" / "record.md")
text = ("Path: `tools/_work/bench/solo_in_workbench.py` explains it.\n\n"
"```\n$ python solo_in_workbench.py\n```\n")
findings = gate.command_findings(doc, text, {}, False)
kinds = [f["kind"] for f in findings]
assert "unqualified-workbench" not in kinds, findings
assert "missing-script" not in kinds, findings
assert "skipped" in kinds, findings
def test_a_bare_name_after_cd_with_no_resolution_is_skipped_not_drift(gate, fake_repo):
"""`cd <somewhere cloned>` then a bare name: unverifiable, and reported as such."""
doc = str(fake_repo / "docs" / "tool-verification" / "record.md")
text = "```\n$ cd tools/_work/bench/repos/elsewhere\n$ python upstream_tool.py --x\n```\n"
findings = gate.command_findings(doc, text, {}, False)
kinds = [f["kind"] for f in findings]
assert "missing-script" not in kinds, findings
assert "after-cd-unresolved" in kinds, findings
def test_the_external_dcc_entry_point_is_never_drift(gate):
"""`python dcc.py` after `cd tools/_work/bench/repos/dcc` runs inside the external dcc
checkout, so it is a skip on every machine -- including one without the workbench.
The name is declared in EXTERNAL_SCRIPTS because recognition by filename needs `tools/` to
exist: on a clean checkout `dcc.py` degraded to `missing-script` drift and the gate failed away
from the machine that wrote the record. Asserted as a *property* rather than as one specific
reason, because two mechanisms can legitimately produce it -- the declared entry point and the
workbench-after-`cd` rule -- and pinning either one alone makes the test machine-dependent,
which is the very defect it is guarding against.
"""
doc = str(REPO_ROOT / "docs" / "tool-verification" / "EXTENSION-java2c.md")
_path, why = gate.resolve_script("dcc.py", doc, include_workbench=False, chdir=True)
assert why in ("external", "workbench-after-cd"), why
text = "```\n$ cd tools/_work/bench/repos/dcc\n$ python dcc.py --no-build\n```\n"
findings = gate.command_findings(doc, text, {}, False)
kinds = [f["kind"] for f in findings]
assert "missing-script" not in kinds, findings
assert "unqualified-workbench" not in kinds, findings
assert any(k in ("workbench-after-cd", "external") for k in kinds), findings
def test_a_shipped_script_resolves_from_the_repository_root(gate):
"""The order that matters: root, then the skill's scripts, then the document's own directory."""
doc = str(REPO_ROOT / "docs" / "tool-verification" / "README.md")
path, why = gate.resolve_script("check_repo.py", doc, include_workbench=False)
assert why == "ok" and path and path.endswith("check_repo.py")
path, why = gate.resolve_script("skills/apk-reverse/scripts/scan_leaks.py", doc,
include_workbench=False)
assert why == "ok" and path and path.endswith("scan_leaks.py")
def test_the_committed_tree_reports_no_drift(gate, tmp_path):
"""The gate's own verdict on this repository, run in-process so it cannot drift silently."""
docs = gate.collect_docs(None)
findings = []
cache = {}
for doc in docs:
try:
text = Path(doc).read_text(encoding="utf-8")
except OSError:
continue
findings.extend(gate.command_findings(doc, text, cache, False))
drift = [f for f in findings
if f["kind"] in ("unknown-flag", "missing-script", "unqualified-workbench")]
detail = "\n".join(" %s:%s %s (%s)" % (f["doc"], f["line"], f["command"], f["kind"])
for f in drift[:20])
assert not drift, "documented commands that cannot run:\n%s" % detail

252
vendor-new/apk-reverse/tests/integration/test_no_device_flows.py Normal file
View File

@@ -0,0 +1,252 @@
"""No-device flows: launch-health verdicts, log parsing, tombstone parsing.
Everything here runs on a bare checkout. The device commands are intercepted, so
what is being tested is the *decision logic* -- which is exactly the part that a
device test would otherwise hide behind "it looked fine on screen".
"""
from __future__ import annotations
import sys
import pytest
from conftest import load_script, run_script
install_test = load_script("install_test.py")
coldstart = load_script("coldstart.py")
native_crash = load_script("native_crash.py")
pytestmark = pytest.mark.integration
TOMBSTONE = "tombstone_sample.txt"
LOGCAT = "logcat_sample.txt"
# --------------------------------------------------------------------------- #
# install_test.py: the two-sample pid verdict
# --------------------------------------------------------------------------- #
def drive(monkeypatch, pids, log="", capsys=None):
"""Run install_test.main() with the device replaced by a scripted responder.
`pids` is consumed in order by successive `pidof` calls, which is the whole
point of the check: one sample cannot tell a running app from a crash loop.
"""
pids = list(pids)
calls = []
def fake_adb(args, serial=None, timeout=600):
calls.append(list(args))
if list(args)[:1] == ["logcat"] and "-d" in args:
return log
return ""
def fake_sh(cmd, serial=None):
calls.append(cmd)
if cmd.startswith("pidof"):
return pids.pop(0) if pids else ""
if "mCurrentFocus" in cmd:
return "mCurrentFocus=Window{a1b2 u0 com.example.app/.MainActivity}"
return ""
monkeypatch.setattr(install_test, "adb", fake_adb)
monkeypatch.setattr(install_test, "sh", fake_sh)
monkeypatch.setattr(install_test.time, "sleep", lambda _s: None)
monkeypatch.setattr(sys, "argv", [
"install_test.py", "--apk", "out/app.apk", "--pkg", "com.example.app",
"--activity", ".MainActivity",
])
return install_test.main(), calls
class TestLaunchHealthVerdict:
def test_a_stable_pid_with_no_fatal_signature_is_ok(self, monkeypatch, capsys):
code, calls = drive(monkeypatch, ["1234", "1234"])
assert code == 0, capsys.readouterr().out
def test_a_changed_pid_is_a_failure_even_though_the_app_is_running(self, monkeypatch, capsys):
"""The measured shape of a crash loop: alive at the first sample, restarted by the second."""
code, _calls = drive(monkeypatch, ["1234", "5678"])
out = capsys.readouterr().out
assert code == 1
assert "pid @+6s : 1234" in out
assert "5678" in out
def test_a_dead_app_is_a_failure(self, monkeypatch, capsys):
code, _calls = drive(monkeypatch, ["", ""])
assert code == 1
assert "(dead)" in capsys.readouterr().out
def test_a_fatal_signature_fails_even_with_a_stable_pid(self, monkeypatch, capsys):
"""Stability is not health: a caught-but-fatal error still fails the check."""
code, _calls = drive(monkeypatch, ["1234", "1234"],
log="01-01 10:00:01 E AndroidRuntime: FATAL EXCEPTION: main\n")
assert code == 1
assert "FATAL EXCEPTION" in capsys.readouterr().out
def test_both_pid_samples_are_actually_taken(self, monkeypatch, capsys):
_code, calls = drive(monkeypatch, ["1234", "1234"])
capsys.readouterr()
pidof_calls = [c for c in calls if isinstance(c, str) and c.startswith("pidof")]
assert len(pidof_calls) == 2, "the second sample is what catches a crash loop"
def test_the_launch_uses_the_package_slash_activity_component(self, monkeypatch, capsys):
_code, calls = drive(monkeypatch, ["1234", "1234"])
capsys.readouterr()
assert any("am start -n com.example.app/.MainActivity" == c for c in calls
if isinstance(c, str))
# --------------------------------------------------------------------------- #
# coldstart.py: logcat signal counting
# --------------------------------------------------------------------------- #
class TestLogcatSignalCounting:
@pytest.fixture
def counts(self, fixtures_dir):
path = fixtures_dir / LOGCAT
assert path.exists(), "the logcat fixture is missing from tests/fixtures"
return coldstart.signal_counts(str(path), coldstart.SIGNAL_PATTERNS)
@pytest.mark.parametrize("name,expected", [
("FATAL EXCEPTION", 1),
("VerifyError", 1),
("Bad checksum", 1),
("ANR", 1),
("Displayed", 1),
("IncompatibleClassChangeError", 0),
("ClassNotFoundException", 0),
("uncaughtException", 0),
("am_crash", 0),
])
def test_counts(self, counts, name, expected):
assert counts[name] == expected
def test_every_pattern_in_the_table_is_reported(self, counts):
assert set(counts) == set(coldstart.SIGNAL_PATTERNS)
def test_a_missing_log_yields_no_counts_rather_than_raising(self, tmp_path):
assert coldstart.signal_counts(str(tmp_path / "nope.txt"),
coldstart.SIGNAL_PATTERNS) == {}
def test_the_displayed_pattern_captures_the_milliseconds(self):
import re
match = re.search(coldstart.SIGNAL_PATTERNS["Displayed"],
"I ActivityManager: Displayed com.example.app/.MainActivity: +842ms")
assert match is not None
assert match.group(1) == "842"
# --------------------------------------------------------------------------- #
# native_crash.py: tombstone parsing
# --------------------------------------------------------------------------- #
@pytest.fixture
def crash(fixtures_dir):
text = (fixtures_dir / TOMBSTONE).read_text(encoding="utf-8")
crashes = native_crash.parse(text)
assert len(crashes) == 1, "the fixture holds exactly one crash block"
return crashes[0]
class TestTombstoneParsing:
def test_signal_and_code(self, crash):
assert crash["signal"] == 11
assert crash["signame"] == "SIGSEGV"
assert crash["si_code"] == "SEGV_MAPERR"
assert crash["fault"] == "0x0000000000000010"
def test_process_identity_is_picked_up(self, crash):
assert crash["pid"] == "12345"
assert crash["tid"] == "12345"
assert crash["proc"] == "com.example.app"
def test_uptime_and_cause(self, crash):
assert crash["uptime"] == "5"
assert "null pointer" in crash["cause"]
def test_backtrace_frames_are_complete(self, crash):
assert len(crash["frames"]) == 4
assert crash["frames"][0]["path"].endswith("libfoo.so")
assert crash["frames"][0]["symbol"] == ""
assert crash["frames"][1]["symbol"] == "Java_com_example_app_Native_check+120"
assert crash["frames"][2]["path"].startswith("/apex/")
assert crash["frames"][2]["symbol"] == "__pthread_start+256"
def test_registers_are_collected_from_the_dump_line(self, crash):
assert crash["regs"]["x0"] == 0x10
assert crash["regs"]["x1"] == 0x7A12345678
assert crash["regs"]["sp"] == 0x7FFABC1110
assert crash["regs"]["lr"] == 0x7A11112222
def test_the_pc_key_is_overwritten_by_the_last_backtrace_frame(self, crash):
"""Measured behaviour, recorded rather than smoothed over.
Register extraction scans every line of the block with one permissive
pattern, so a `#NN pc <hex>` backtrace line also matches and the last one
wins. The register dump's own `pc 0000007a111128cc` is therefore not what
`regs['pc']` holds. It is a reporting wart, not a wrong verdict -- but it
is a wart, so it is pinned here and listed in the harness report.
"""
assert crash["regs"]["pc"] == 0x55488, "the last frame's pc"
assert crash["regs"]["pc"] != 0x7A111128CC, "the register dump's pc"
def test_empty_input_produces_no_crashes(self):
assert native_crash.parse("") == []
def test_a_log_without_a_signal_line_produces_no_crashes(self):
assert native_crash.parse("I/whatever: nothing to see\n") == []
class TestLibraryClassification:
@pytest.mark.parametrize("path,expected", [
("/apex/com.android.runtime/lib64/bionic/libc.so", True),
("/system/lib64/libart.so", True),
("/vendor/lib64/libqcom.so", True),
("/data/dalvik-cache/arm64/system@framework@boot.oat", True),
("/data/app/~~AbCdEf==/com.example.app-1/lib/arm64/libfoo.so", False),
("/data/local/tmp/libpayload.so", False),
])
def test_classification(self, path, expected):
assert native_crash.is_system_lib(path) is expected
def test_a_bare_name_is_treated_as_system(self):
"""Measured behaviour: a path that is not absolute cannot be attributed to the app."""
assert native_crash.is_system_lib("libfoo.so") is True
class TestVerdictHint:
def test_a_small_fault_address_held_in_a_register_is_flagged_as_arranged(self, crash):
hints = " ".join(native_crash.verdict_hint(crash))
assert "small integer" in hints
assert "ARRANGED" in hints
assert "x0" in hints
def test_an_ordinary_null_dereference_gets_no_arranged_hint(self):
ordinary = {"fault": "0x0", "regs": {"x1": 0x7A111128CC}, "signame": "SIGSEGV"}
hints = native_crash.verdict_hint(ordinary)
assert any("small integer" in h for h in hints)
assert not any("ARRANGED" in h for h in hints)
def test_sigkill_is_called_out_as_a_deliberate_termination(self):
hints = native_crash.verdict_hint({"fault": "0x0", "regs": {}, "signame": "SIGKILL"})
assert any("deliberate termination" in h for h in hints)
class TestNativeCrashCli:
def test_the_fixture_tombstone_is_parsed_by_the_cli(self, fixtures_dir):
result = run_script("native_crash.py", [str(fixtures_dir / TOMBSTONE)], timeout=120)
assert not result.timed_out
assert not result.has_traceback, result.output
assert result.returncode == 0, result.output
assert "libfoo.so" in result.stdout
assert "SIGSEGV" in result.stdout
def test_a_missing_log_is_reported_without_a_traceback(self, tmp_path):
result = run_script("native_crash.py", [str(tmp_path / "absent.txt")])
assert not result.has_traceback, result.output
assert result.returncode != 0
assert "no such file" in result.output
def test_a_malformed_lib_spec_is_refused(self, fixtures_dir):
result = run_script("native_crash.py",
[str(fixtures_dir / TOMBSTONE), "--lib", "notapair"])
assert result.returncode != 0
assert "NAME=PATH" in result.output

41
vendor-new/apk-reverse/tests/run_tests.py Normal file
View File

@@ -0,0 +1,41 @@
#!/usr/bin/env python3
"""Single entry point for the regression suite.
Why this file exists in addition to ``pytest.ini``: a fresh checkout without
pytest should say so in one line, with the install command, instead of dying on
an import error. It exits 3 (environment capability insufficient) per the
repository's exit-code policy, so a wrapper script can tell "no pytest here"
apart from "tests failed" (1).
Usage:
python tests/run_tests.py # whole suite, quiet summary
python tests/run_tests.py -m unit # one marker
python tests/run_tests.py -k dexutil -vv # anything pytest accepts
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
def main(argv):
try:
import pytest
except ImportError:
sys.stderr.write(
"pytest is not installed; this suite cannot run.\n"
" install: python -m pip install pytest\n"
" the suite is stdlib-only otherwise (no device, no network, no adb)\n")
return 3
# pytest.ini lives at the repository root and is found from there.
os.chdir(REPO_ROOT)
args = list(argv[1:]) or ["-q"]
return int(pytest.main(args))
if __name__ == "__main__":
sys.exit(main(sys.argv))

686
vendor-new/apk-reverse/tests/test_so_constpatch.py Normal file
View File

@@ -0,0 +1,686 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Regression tests for `skills/apk-reverse/scripts/so_constpatch.py`.
What is being locked down
-------------------------
The script rewrites one zip entry inside an APK. The first implementation rebuilt the
whole archive through `zipfile.ZipInfo(filename, date_time)` and re-deflated everything,
which dropped `extra`, `external_attr` and `create_system`, and could push
`resources.arsc` off its STORED/4-byte-aligned layout -- an APK the installer refuses
with `[-124]`, and one that may not load at all when the manifest asks for
`extractNativeLibs="false"`.
These tests run the CLI the way a caller does (subprocess, exit code, RESULT token, JSON)
and re-read the written archive to check the contract, so they cannot pass by trusting
the writer's own bookkeeping. No device, no Android toolchain, no network.
The fixtures in `tests/fixtures/apk/` are committed; `tests/fixtures/make_fixture_apk.py`
regenerates them and is exercised too, so a fixture that stops carrying the metadata
under test fails here rather than silently weakening the suite.
"""
from __future__ import annotations
import hashlib
import importlib.util
import json
import os
import struct
import subprocess
import sys
import zipfile
from pathlib import Path
import pytest
HERE = Path(__file__).resolve().parent
REPO = HERE.parent
SCRIPT = REPO / 'skills' / 'apk-reverse' / 'scripts' / 'so_constpatch.py'
SCRIPTS_DIR = SCRIPT.parent
FIXTURES = HERE / 'fixtures' / 'apk'
MAKE_FIXTURES = HERE / 'fixtures' / 'make_fixture_apk.py'
APKHUNAN = b'apkhuan'
ANDROID = b'android'
SO_ENTRY = 'lib/arm64-v8a/libfoo.so'
STORE_ENTRY = 'lib/arm64-v8a/libbar.so'
X64_SO_ENTRY = 'lib/x86_64/libfoo.so'
SERVICES_ENTRIES = (
'META-INF/services/com.example.Provider',
'META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory',
)
UTF8_ENTRY = 'res/raw/\u6d4b\u8bd5.bin'
# ------------------------------------------------------------------ helpers
def run_cli(*args, cwd=None):
return subprocess.run([sys.executable, str(SCRIPT)] + [str(a) for a in args],
capture_output=True, text=True, encoding='utf-8',
errors='replace', cwd=cwd)
def token_of(proc):
for line in reversed(proc.stdout.splitlines()):
if line.startswith('RESULT='):
return line.split('=', 1)[1].strip()
return None
def json_of(proc):
lines = proc.stdout.strip().splitlines()
assert lines and lines[-1].startswith('RESULT='), proc.stdout
return json.loads('\n'.join(lines[:-1]))
def load_named(name, path):
"""Import a module by path, so the tests do not depend on sys.path state."""
spec = importlib.util.spec_from_file_location(name, path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
def load_module():
return load_named('so_constpatch_under_test', SCRIPT)
def local_extra(path, name):
"""The local-header extra field, which is what alignment lives in."""
with zipfile.ZipFile(path) as z, open(path, 'rb') as fh:
info = z.getinfo(name)
fh.seek(info.header_offset)
head = fh.read(30)
nlen, elen = struct.unpack_from('<HH', head, 26)
return fh.read(nlen + elen)[nlen:]
def data_offset(path, name):
with zipfile.ZipFile(path) as z, open(path, 'rb') as fh:
info = z.getinfo(name)
fh.seek(info.header_offset)
head = fh.read(30)
nlen, elen = struct.unpack_from('<HH', head, 26)
return info.header_offset + 30 + nlen + elen
def zip_dump(path):
"""Everything a rebuild must preserve, keyed by entry name."""
out = {}
with zipfile.ZipFile(path) as z, open(path, 'rb') as fh:
for info in z.infolist():
fh.seek(info.header_offset)
head = fh.read(30)
nlen, elen = struct.unpack_from('<HH', head, 26)
raw_name = fh.read(nlen)
local_extra_bytes = fh.read(elen)
out[info.filename] = {
'info': info,
'raw_name': raw_name,
'local_extra': local_extra_bytes,
'method': info.compress_type,
'extra': info.extra,
'external_attr': info.external_attr,
'internal_attr': info.internal_attr,
'create_system': info.create_system,
'comment': info.comment,
'flags': info.flag_bits,
'date_time': info.date_time,
'crc': info.CRC,
'compress_size': info.compress_size,
'file_size': info.file_size,
'data_offset': info.header_offset + 30 + nlen + elen,
'sha256': hashlib.sha256(z.read(info.filename)).hexdigest(),
}
return out
def fixture(name):
p = FIXTURES / name
assert p.is_file(), 'missing fixture %s; run tests/fixtures/make_fixture_apk.py' % p
return p
def so_bytes(apk, entry=SO_ENTRY):
with zipfile.ZipFile(apk) as z:
return z.read(entry)
# ------------------------------------------------------------------ fixtures themselves
def test_fixtures_carry_the_metadata_under_test():
"""A fixture that lost its `extra`/`external_attr` would make the suite vacuous."""
for name in ('plain.apk', 'arsc_aligned.apk', 'noextract.apk', 'services.apk'):
dump = zip_dump(fixture(name))
assert dump, name
for entry, row in dump.items():
assert row['extra'], '%s!%s has no extra field to preserve' % (name, entry)
assert any(row['external_attr'] for row in dump.values()), \
'%s has no non-zero external_attr to preserve' % name
# And one entry that must keep a zero external_attr / create_system=0 too.
zeroed = zip_dump(fixture('services.apk'))[UTF8_ENTRY]
assert zeroed['external_attr'] == 0 and zeroed['create_system'] == 0
arsc = zip_dump(fixture('arsc_aligned.apk'))['resources.arsc']
assert arsc['method'] == 0
assert arsc['data_offset'] % 4 == 0
noextract = zip_dump(fixture('noextract.apk'))
assert noextract[SO_ENTRY]['method'] == 0
assert noextract[SO_ENTRY]['data_offset'] % 4 == 0
def test_fixture_manifest_is_binary_axml_with_the_declaration():
"""The manifest is real AXML, and the script's reader agrees with repack.py's.
Two independent readers on the same bytes: if either drifts, this fails instead of
one tool quietly disagreeing with the other about `extractNativeLibs`.
"""
sys.path.insert(0, str(SCRIPTS_DIR))
try:
import repack
finally:
sys.path.pop(0)
with zipfile.ZipFile(fixture('noextract.apk')) as z:
blob = z.read('AndroidManifest.xml')
assert blob[:2] == b'\x03\x00', 'not a binary AXML document'
attrs = repack.axml_root_attrs(blob)
assert attrs['extractNativeLibs'] == 'false'
assert attrs['package'] == 'com.example.noextract'
mod = load_module()
assert mod.axml_root_attributes(blob)['extractNativeLibs'] == 'false'
assert mod.manifest_extract_native_libs(blob) == 'false'
with zipfile.ZipFile(fixture('plain.apk')) as z:
assert mod.manifest_extract_native_libs(z.read('AndroidManifest.xml')) is None
assert mod.manifest_extract_native_libs(
b'<manifest xmlns:android="x" android:extractNativeLibs="false">') == 'false'
assert mod.manifest_extract_native_libs(b'</bad>') == 'unreadable'
def test_alignment_predicate_matches_a_real_abi_directory():
"""`lib/<abi>/x.so` has two slashes; a `[^/]+` pattern would match nothing."""
mod = load_module()
for name in ('lib/arm64-v8a/libfoo.so', 'lib/x86/libfoo.so', 'lib/x86_64/a.so'):
assert mod.needs_alignment(name, 0, 8) is True, name
# `resources.arsc` is mandatory whatever its input offset is.
assert mod.needs_alignment('resources.arsc', 0, 10) is True
assert mod.needs_alignment('resources.arsc', 8, 10) is True
# An uncompressed .so is mandatory too (extractNativeLibs=false maps it in place).
assert mod.needs_alignment('lib/arm64-v8a/libfoo.so', 0, 10) is True
# Any other STORED entry keeps the alignment it arrived with, and is not re-laid out.
assert mod.needs_alignment('assets/blob.bin', 0, 8) is True
assert mod.needs_alignment('assets/blob.bin', 0, 10) is False
# Deflated entries have no alignment requirement.
assert mod.needs_alignment('classes.dex', 8, 8) is False
assert mod.needs_alignment('lib/arm64-v8a/libfoo.so', 8, 8) is False
def test_fixture_generator_reproduces_the_committed_archives(tmp_path):
proc = subprocess.run([sys.executable, str(MAKE_FIXTURES), '--out', str(tmp_path)],
capture_output=True, text=True)
assert proc.returncode == 0, proc.stderr
for name in ('plain.apk', 'arsc_aligned.apk', 'noextract.apk', 'services.apk'):
regenerated = tmp_path / name
assert regenerated.is_file()
assert list(zip_dump(regenerated)) == list(zip_dump(fixture(name))), name
# Structure must match even if a different zlib produced different bytes.
assert os.path.getsize(regenerated) < 8192, 'fixtures must stay tiny'
# ------------------------------------------------------------------ the bare-.so path
def test_bare_so_rewrite_is_byte_exact(tmp_path):
"""The `.so` path predates the container work and must not move a single byte.
The expected output is constructed here from the input, so this is not a
self-consistency check: patching `apkhuan` at its reported offset is the whole
contract. The sha256 of this same output was measured identical under the
pre-change script (see tools/_work/zip-safety-EVIDENCE.md).
"""
src = so_bytes(fixture('plain.apk'))
so_path = tmp_path / 'libfoo.so'
so_path.write_bytes(src)
out = tmp_path / 'libfoo.patched.so'
proc = run_cli(so_path, '--replace', 'apkhuan=android', '-o', out)
assert proc.returncode == 0, proc.stdout + proc.stderr
assert token_of(proc) == 'patched'
got = out.read_bytes()
expected = bytearray(src)
expected[0x41:0x41 + len(APKHUNAN)] = ANDROID
assert got == bytes(expected)
assert len(got) == len(src)
assert hashlib.sha256(got).hexdigest() == \
hashlib.sha256(bytes(expected)).hexdigest()
def test_bare_so_reports_the_bytes_actually_changed(tmp_path):
"""`N byte(s) actually changed`: the window is 7, the changed bytes are 6."""
so_path = tmp_path / 'libfoo.so'
so_path.write_bytes(so_bytes(fixture('plain.apk')))
proc = run_cli(so_path, '--replace', 'apkhuan=android', '-o', tmp_path / 'o.so')
assert '6 byte(s) actually changed' in proc.stdout, proc.stdout
# ------------------------------------------------------------------ APK rebuild: metadata
def test_deflated_entry_rebuild_preserves_every_other_entry(tmp_path):
src = fixture('plain.apk')
out = tmp_path / 'plain.patched.apk'
before = zip_dump(src)
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out,
'--verbose')
assert proc.returncode == 0, proc.stdout + proc.stderr
after = zip_dump(out)
assert list(after) == list(before), 'entry order changed'
assert len(after) == len(before)
changed = after[SO_ENTRY]
assert changed['sha256'] != before[SO_ENTRY]['sha256'], 'the patched entry did not change'
assert changed['method'] == 8
for name, row in before.items():
if name == SO_ENTRY:
continue
got = after[name]
assert got['sha256'] == row['sha256'], '%s payload changed' % name
assert got['crc'] == row['crc'], '%s CRC changed' % name
assert got['compress_size'] == row['compress_size'], '%s compress_size changed' % name
assert got['extra'] == row['extra'], '%s central-directory extra changed' % name
assert got['local_extra'] == row['local_extra'], '%s local extra changed' % name
assert got['external_attr'] == row['external_attr'], '%s external_attr changed' % name
assert got['internal_attr'] == row['internal_attr']
assert got['create_system'] == row['create_system']
assert got['comment'] == row['comment']
assert got['date_time'] == row['date_time']
assert got['flags'] == row['flags']
def test_store_stays_store_and_is_not_recompressed(tmp_path):
"""A STORED entry must come back STORED with the same layout, not re-deflated."""
src = fixture('arsc_aligned.apk')
out = tmp_path / 'arsc_store.patched.apk'
before = zip_dump(src)
proc = run_cli(src, '--entry', STORE_ENTRY, '--replace', 'apkhuan=android', '-o', out)
assert proc.returncode == 0, proc.stdout + proc.stderr
after = zip_dump(out)
got, want = after[STORE_ENTRY], before[STORE_ENTRY]
assert want['method'] == 0
assert got['method'] == 0, 'a STORED entry was re-compressed'
assert got['compress_size'] == want['compress_size'] == want['file_size']
assert got['file_size'] == want['file_size']
assert got['sha256'] != want['sha256']
# Nothing moved, so alignment needed no new padding and every extra field is
# byte-identical -- the strongest form of "metadata preserved".
assert got['data_offset'] == want['data_offset']
assert got['extra'] == want['extra']
assert got['local_extra'] == want['local_extra']
for name, row in before.items():
assert after[name]['data_offset'] == row['data_offset'], name
assert after[name]['local_extra'] == row['local_extra'], name
def test_deflated_change_keeps_arsc_and_stored_so_aligned(tmp_path):
"""The case the old writer broke: a size change before `resources.arsc`."""
src = fixture('arsc_aligned.apk')
out = tmp_path / 'arsc_deflate.patched.apk'
before = zip_dump(src)
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out)
assert proc.returncode == 0, proc.stdout + proc.stderr
after = zip_dump(out)
assert after[SO_ENTRY]['compress_size'] != before[SO_ENTRY]['compress_size'], \
'this fixture must make the patched entry change size, or padding is not exercised'
for name, row in before.items():
got = after[name]
if row['method'] != 0:
continue
# Every uncompressed entry that was aligned stays aligned (zipalign semantics),
# and `resources.arsc` is aligned even if it had not been.
assert got['data_offset'] % 4 == 0, '%s lost 4-byte alignment' % name
assert got['method'] == 0, '%s is no longer STORED' % name
if name != SO_ENTRY:
# The entry itself is untouched...
assert got['sha256'] == row['sha256'], name
assert got['crc'] == row['crc'], name
assert got['extra'] == row['extra'], name
assert got['external_attr'] == row['external_attr'], name
# ...and its original local extra survives as a prefix of the new one:
# padding is appended, never substituted.
assert got['local_extra'].startswith(row['local_extra']), name
# `libbar.so` is STORED and sits before `resources.arsc`, so the size change shifts
# it by an unaligned amount: the writer must restore its alignment with a pad record.
bar_pad = len(after[STORE_ENTRY]['local_extra']) \
- len(before[STORE_ENTRY]['local_extra'])
assert bar_pad > 0, 'a shifted STORED .so was left unaligned without a pad record'
pad = bar_pad - 4
assert 1 <= pad <= 3, 'the appended record claims a pad of %d bytes' % pad
assert bar_pad % 4 == pad, 'the appended record does not land on the boundary'
appended = after[STORE_ENTRY]['local_extra'][
len(before[STORE_ENTRY]['local_extra']):]
assert struct.unpack_from('<H', appended, 2)[0] == pad, 'malformed extra record'
assert 'alignment padding' in proc.stdout
assert 'resources.arsc' in proc.stdout
assert '4-byte aligned' in proc.stdout
def test_an_unaligned_stored_entry_is_not_silently_relaid(tmp_path):
"""This tool preserves alignment; it does not re-lay-out an archive it was not asked
to fix. A STORED entry that arrived unaligned stays that way, with its extra and
offsets untouched -- silently rewriting it would be the same class of change the
STORED/alignment work exists to prevent."""
fx = load_named('make_fixture_apk_under_test', MAKE_FIXTURES)
src = tmp_path / 'handmade.apk'
entries = [
fx.Entry('classes.dex', fx.make_dex(), method=8),
fx.Entry('lib/arm64-v8a/libfoo.so', fx.make_elf(), method=8),
# STORED, deliberately NOT aligned (the generator aligns only when asked).
fx.Entry('AndroidManifest.xml', fx.make_axml('com.example.unaligned'), method=0),
]
fx.write_apk(str(src), entries)
before = zip_dump(src)
assert before['AndroidManifest.xml']['data_offset'] % 4 != 0, \
'the handmade archive is supposed to be unaligned here'
out = tmp_path / 'handmade.patched.apk'
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out)
assert proc.returncode == 0, proc.stdout + proc.stderr
after = zip_dump(out)
assert after[SO_ENTRY]['compress_size'] != before[SO_ENTRY]['compress_size']
got = after['AndroidManifest.xml']
assert got['data_offset'] % 4 != 0, 'an unaligned STORED entry was silently moved'
assert got['local_extra'] == before['AndroidManifest.xml']['local_extra']
assert got['sha256'] == before['AndroidManifest.xml']['sha256']
assert got['extra'] == before['AndroidManifest.xml']['extra']
assert 'NOT 4-byte aligned in the input' in proc.stdout
def test_self_check_is_printed_and_reads_back_the_written_file(tmp_path):
src = fixture('arsc_aligned.apk')
out = tmp_path / 'selfcheck.apk'
proc = run_cli(src, '--entry', STORE_ENTRY, '--replace', 'apkhuan=android', '-o', out)
assert proc.returncode == 0, proc.stdout + proc.stderr
text = proc.stdout
assert 'rebuild self-check' in text
assert 'entry count' in text and 'identical' in text
assert 'CRC+size identical' in text
assert 'sha256 identical' in text
assert 'header_offset' in text
def test_services_and_utf8_entries_are_untouched(tmp_path):
src = fixture('services.apk')
out = tmp_path / 'services.patched.apk'
before = zip_dump(src)
proc = run_cli(src, '--entry', X64_SO_ENTRY, '--replace', 'apkhuan=android', '-o', out,
'--verbose')
assert proc.returncode == 0, proc.stdout + proc.stderr
after = zip_dump(out)
for name in SERVICES_ENTRIES + (UTF8_ENTRY, 'AndroidManifest.xml'):
assert after[name]['sha256'] == before[name]['sha256'], '%s changed' % name
assert after[name]['extra'] == before[name]['extra'], '%s extra changed' % name
assert after[name]['external_attr'] == before[name]['external_attr']
assert after[name]['create_system'] == before[name]['create_system']
assert after[name]['comment'] == before[name]['comment']
utf8 = after[UTF8_ENTRY]
assert utf8['info'].filename == UTF8_ENTRY
assert utf8['flags'] & 0x800, 'the UTF-8 name flag was dropped'
assert utf8['create_system'] == 0
assert utf8['internal_attr'] == 1
assert utf8['comment'] == b'fixture comment'
with zipfile.ZipFile(src) as a, zipfile.ZipFile(out) as b:
assert a.comment == b.comment == b'fixture-archive-comment'
# ------------------------------------------------------------------ the refusal gate
def test_extract_native_libs_false_is_refused_by_default(tmp_path):
src = fixture('noextract.apk')
out = tmp_path / 'noextract.patched.apk'
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out)
assert proc.returncode == 1, proc.stdout + proc.stderr
assert token_of(proc) == 'refused_unsafe_rebuild'
assert not out.exists(), 'a refused rebuild must write nothing'
assert 'REFUSED' in proc.stdout
assert 'extractNativeLibs' in proc.stdout
assert 'scripts/repack.py' in proc.stdout, 'the refusal must name the right tool'
assert '--unsafe-rebuild' in proc.stdout
def test_unsafe_rebuild_is_explicit_and_reported(tmp_path):
src = fixture('noextract.apk')
out = tmp_path / 'noextract.unsafe.apk'
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out,
'--unsafe-rebuild')
assert proc.returncode == 0, proc.stdout + proc.stderr
assert token_of(proc) == 'patched'
assert 'WARNING: --unsafe-rebuild accepted' in proc.stdout
after = zip_dump(out)
assert after[SO_ENTRY]['method'] == 0
assert after[SO_ENTRY]['data_offset'] % 4 == 0
assert after['resources.arsc']['method'] == 0
assert after['resources.arsc']['data_offset'] % 4 == 0
def test_a_compressed_arsc_is_refused(tmp_path):
"""An arsc that is not STORED+aligned cannot be made compliant by this writer."""
mod = load_module()
broken = tmp_path / 'compressed_arsc.apk'
with zipfile.ZipFile(fixture('arsc_aligned.apk')) as src_z:
plan = [(i.filename, src_z.read(i.filename), i.compress_type, i.extra,
i.external_attr, i.create_system, i.date_time)
for i in src_z.infolist()]
with zipfile.ZipFile(broken, 'w') as dst:
for name, blob, method, extra, eattr, csys, dt in plan:
zi = zipfile.ZipInfo(name, date_time=dt)
zi.compress_type = zipfile.ZIP_DEFLATED if name == 'resources.arsc' else method
zi.extra = extra
zi.external_attr = eattr
zi.create_system = csys
dst.writestr(zi, blob)
out = tmp_path / 'x.apk'
proc = run_cli(broken, '--entry', STORE_ENTRY, '--replace', 'apkhuan=android',
'-o', out)
assert proc.returncode == 1, proc.stdout + proc.stderr
assert token_of(proc) == 'refused_unsafe_rebuild'
assert 'resources.arsc' in proc.stdout
assert not out.exists()
# The audit reads the storage state without needing a rebuild at all.
facts = mod.audit_apk_for_rebuild(str(broken), mod.Reporter(as_json=True))
assert facts['arsc']['stored'] is False
assert facts['unsafe']
# ------------------------------------------------------------------ find / usage errors
def test_find_reports_isolated_and_non_isolated(tmp_path):
src = fixture('plain.apk')
proc = run_cli(src, '--entry', SO_ENTRY, '--find', 'apkhuan')
assert proc.returncode == 0
assert token_of(proc) == 'found'
assert 'isolated=True' in proc.stdout and 'isolated=False' in proc.stdout
assert 'isolated (safe to rewrite in place): 1 / 2' in proc.stdout
def test_find_without_hits_is_a_negative_finding():
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--find', 'nosuchthing')
assert proc.returncode == 1
assert token_of(proc) == 'not_found'
def test_equal_length_is_mandatory():
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=and',
'-o', 'out.apk')
assert proc.returncode == 2
assert 'Equal length is mandatory' in proc.stdout
def test_non_isolated_and_multi_hits_need_an_explicit_flag():
# The decoy `libapkhuanx.so` makes a second, non-isolated hit.
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=android',
'-o', 'out.apk', '--allow-nonisolated')
assert proc.returncode == 2
assert '--force' in proc.stdout
assert token_of(proc) == 'usage_error'
def test_missing_entry_and_missing_out_are_usage_errors(tmp_path):
p = run_cli(fixture('plain.apk'), '--entry', 'lib/arm64-v8a/nope.so',
'--replace', 'apkhuan=android', '-o', tmp_path / 'x.apk')
assert p.returncode == 2
assert 'available .so entries' in p.stdout
p = run_cli(fixture('plain.apk'), '--replace', 'apkhuan=android')
assert p.returncode == 2
p = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=android')
assert p.returncode == 2, 'patching an APK without -o must be rejected'
p = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=android',
'-o', fixture('plain.apk'))
assert p.returncode == 2, 'overwriting the input in place must be rejected'
# ------------------------------------------------------------------ json contract
def test_json_contract_on_success(tmp_path):
out = tmp_path / 'j.apk'
proc = run_cli(fixture('arsc_aligned.apk'), '--entry', STORE_ENTRY, '--replace',
'apkhuan=android', '-o', out, '--json')
assert proc.returncode == 0, proc.stdout + proc.stderr
payload = json_of(proc)
assert payload['status'] == 'ok'
assert payload['exit_code'] == 0
assert payload['capability'] == 'native_const_patch'
assert payload['result'] == 'patched'
for key in ('evidence', 'warnings', 'next_action'):
assert key in payload
assert isinstance(payload['evidence'], list) and payload['evidence']
assert payload['verify']['entries'] == payload['verify']['untouched'] + 1
assert payload['verify']['sha256_identical'] == payload['verify']['untouched']
assert payload['audit']['arsc']['stored'] is True
assert payload['audit']['unsafe'] == []
def test_json_contract_on_refusal(tmp_path):
proc = run_cli(fixture('noextract.apk'), '--entry', SO_ENTRY, '--replace',
'apkhuan=android', '-o', tmp_path / 'r.apk', '--json')
assert proc.returncode == 1
payload = json_of(proc)
assert payload['status'] == 'refused'
assert payload['exit_code'] == 1
assert payload['result'] == 'refused_unsafe_rebuild'
assert payload['audit']['extract_native_libs'] == 'false'
assert payload['audit']['unsafe']
assert 'repack.py' in payload['next_action']
def test_json_contract_on_negative_find():
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--find', 'nope', '--json')
assert proc.returncode == 1
payload = json_of(proc)
assert payload['status'] == 'negative'
assert payload['result'] == 'not_found'
assert payload['exit_code'] == 1
# ------------------------------------------------------------------ streaming archives
def test_data_descriptors_are_resolved_into_the_local_header(tmp_path):
"""An entry written while streaming carries flag bit 3 and no CRC in its local
header. The rebuild resolves it from the central directory, so the output is
readable by tools that never look at the central directory."""
class _Sink(object):
"""A file object without seek/tell, which is what makes zipfile stream."""
def __init__(self, fh):
self._fh = fh
def write(self, data):
return self._fh.write(data)
def flush(self):
self._fh.flush()
def close(self):
self._fh.close()
streaming = tmp_path / 'streaming.apk'
with zipfile.ZipFile(fixture('plain.apk')) as zin:
with open(streaming, 'wb') as raw:
with zipfile.ZipFile(_Sink(raw), 'w', zipfile.ZIP_DEFLATED) as zout:
for info in zin.infolist():
zi = zipfile.ZipInfo(info.filename, date_time=info.date_time)
zi.compress_type = info.compress_type
with zout.open(zi, 'w') as fh:
fh.write(zin.read(info.filename))
before = zip_dump(streaming)
assert all(row['flags'] & 0x08 for row in before.values()), \
'this fixture must carry data descriptors, or the branch is not covered'
assert all(row['local_extra'] == b'' or True for row in before.values())
out = tmp_path / 'streaming.patched.apk'
proc = run_cli(streaming, '--entry', SO_ENTRY, '--replace', 'apkhuan=android',
'-o', out)
assert proc.returncode == 0, proc.stdout + proc.stderr
after = zip_dump(out)
assert 'data descriptors' in proc.stdout
for name, row in before.items():
got = after[name]
assert not (got['flags'] & 0x08), '%s still declares a data descriptor' % name
if name != SO_ENTRY:
assert got['sha256'] == row['sha256']
assert got['crc'] == row['crc']
with zipfile.ZipFile(out) as z:
assert z.testzip() is None
# ------------------------------------------------------------------ internal invariants
def test_padding_record_is_four_byte_expressible():
"""A zip extra record is (id, size, payload) and its minimum length is 4 bytes."""
mod = load_module()
assert mod._pad_extra(0) == b''
for pad in (1, 2, 3):
rec = mod._pad_extra(pad)
assert len(rec) == 4 + pad
assert len(rec) % 4 == pad
assert struct.unpack_from('<H', rec, 2)[0] == pad
def test_eocd_and_unsupported_containers_are_reported(tmp_path):
mod = load_module()
junk = tmp_path / 'not-a-zip.apk'
junk.write_bytes(b'not a zip at all')
with pytest.raises(mod.Refused) as exc:
mod.audit_apk_for_rebuild(str(junk), mod.Reporter(as_json=True))
assert exc.value.exit_code == 3
assert exc.value.token == 'unsupported_container'
proc = run_cli(junk, '--entry', 'anything', '--replace', 'a=b', '-o', tmp_path / 'o.apk')
assert proc.returncode == 3
assert token_of(proc) == 'unsupported_container'

186
vendor-new/apk-reverse/tests/unit/test_blob_decode.py Normal file
View File

@@ -0,0 +1,186 @@
"""Unit tests for skills/apk-reverse/scripts/blob_decode.py.
The encoded-blob search is the part worth pinning: the whole method exists
because a single un-rotated inflate attempt fails and the failure looks like
"this is encrypted". The tests build blobs with known parameters and require the
search to recover them.
"""
from __future__ import annotations
import gzip
import json
import zlib
import pytest
from conftest import load_script, run_script
blob_decode = load_script("blob_decode.py")
pytestmark = pytest.mark.unit
PAYLOAD = json.dumps({
"flags": {"newCheckout": True, "minVersion": 4210},
"endpoints": ["https://api.example.invalid/v3/config"],
"notes": "synthetic payload for the rotation search regression" * 3,
}).encode("utf-8")
# --------------------------------------------------------------------------- #
# outer encodings
# --------------------------------------------------------------------------- #
class TestOuterEncodings:
@pytest.mark.parametrize("which", ["base64", "base64url", "hex"])
def test_round_trip(self, which):
data = bytes(range(64))
encoded = blob_decode.outer_encode(data, which)
assert blob_decode.outer_decode(encoded, which) == data
def test_unknown_encoding_raises(self):
with pytest.raises(ValueError):
blob_decode.outer_encode(b"x", "rot13")
def test_undecodable_hex_returns_empty_not_an_exception(self):
"""The search calls this on candidate encodings; a raise would abort it."""
assert blob_decode.outer_decode("zzzz", "hex") == b""
def test_whitespace_and_newlines_are_stripped_before_decoding(self):
encoded = blob_decode.outer_encode(b"hello world", "base64")
wrapped = "\n".join([encoded[:4], encoded[4:]])
assert blob_decode.outer_decode(wrapped, "base64") == b"hello world"
def test_sniff_offers_all_three_encodings(self):
guesses = blob_decode.sniff_outer("aGVsbG8gd29ybGQgdGhpcyBpcyBhIHRlc3Q=")
assert set(guesses) >= {"base64", "base64url", "hex"}
def test_sniff_puts_hex_first_for_hex_shaped_text(self):
assert blob_decode.sniff_outer("deadbeef" * 8)[0] == "hex"
# --------------------------------------------------------------------------- #
# inner transforms
# --------------------------------------------------------------------------- #
class TestInnerTransforms:
def test_inflate_raw_round_trips_through_raw_deflate(self):
body = blob_decode.inflate_raw(PAYLOAD)
assert zlib.decompress(body, -15) == PAYLOAD
def test_inflate_raw_output_has_no_zlib_header(self):
"""A raw stream is what the rotation search needs: no header to mis-align."""
body = blob_decode.inflate_raw(b"x" * 200)
with pytest.raises(zlib.error):
zlib.decompress(body) # default wbits expects a zlib header
@pytest.mark.parametrize("name,blob", [
("raw", zlib.compressobj(9, zlib.DEFLATED, -15)),
("zlib", zlib.compressobj(9, zlib.DEFLATED, 15)),
("gzip", zlib.compressobj(9, zlib.DEFLATED, 31)),
])
def test_try_inflate_names_each_inner_transform(self, name, blob):
body = blob.compress(PAYLOAD) + blob.flush()
got = blob_decode.try_inflate(body)
assert got is not None
assert got[0] == name
assert got[1] == PAYLOAD
def test_try_inflate_returns_none_for_noise(self):
assert blob_decode.try_inflate(bytes(range(256))) is None
def test_gzip_is_accepted_via_the_stdlib_helper(self):
got = blob_decode.try_inflate(gzip.compress(PAYLOAD))
assert got is not None and got[0] == "gzip" and got[1] == PAYLOAD
# --------------------------------------------------------------------------- #
# scoring
# --------------------------------------------------------------------------- #
class TestScoring:
def test_structured_text_scores_above_random_bytes(self):
structured = blob_decode.score(PAYLOAD)
noise = blob_decode.score(bytes(range(256)) * 4)
assert structured > noise
def test_a_json_header_is_rewarded(self):
assert blob_decode.score(b'{"a": 1}') > blob_decode.score(b"aaaaaaaa")
def test_empty_output_scores_negative(self):
assert blob_decode.score(b"") < 0
# --------------------------------------------------------------------------- #
# the rotation search
# --------------------------------------------------------------------------- #
def rotate(body, cut):
return body[-cut:] + body[:-cut] if cut else body
class TestRotationSearch:
def _hits(self, blob, max_skip=0):
return blob_decode.search(blob, max_skip, lambda _msg: None)
def test_search_recovers_a_rotated_raw_stream(self):
body = blob_decode.inflate_raw(PAYLOAD)
cut = 7
hits = self._hits(rotate(body, cut))
matching = [h for h in hits if h["data"] == PAYLOAD]
assert matching, "the rotation with cut=%d was not recovered" % cut
assert matching[0]["cut"] == cut
assert matching[0]["inner"] == "raw"
assert matching[0]["size"] == len(PAYLOAD)
def test_search_recovers_a_prefixed_and_rotated_stream(self):
"""The `skip` arm: a short header in front of the compressed stream."""
body = blob_decode.inflate_raw(PAYLOAD)
blob = b"\x01\x02" + rotate(body, 5)
hits = self._hits(blob, max_skip=4)
matching = [h for h in hits if h["data"] == PAYLOAD]
assert matching, "skip=2 + cut=5 was not recovered"
assert matching[0]["skip"] == 2
def test_search_on_noise_returns_no_hits(self):
assert self._hits(bytes(range(64))) == []
def test_scores_are_reported_rounded(self):
body = blob_decode.inflate_raw(PAYLOAD)
hits = self._hits(body)
assert hits
assert all(isinstance(h["score"], float) for h in hits)
def test_the_shortest_accepted_body_is_eight_bytes(self):
"""Documented floor: `m < 8` breaks the loop, so tiny blobs yield nothing."""
assert self._hits(b"\x01\x02\x03") == []
# --------------------------------------------------------------------------- #
# the CLI-level write-back contract
# --------------------------------------------------------------------------- #
class TestEncodePath:
def test_encode_then_decode_recovers_the_payload(self, tmp_path):
"""The write-back line the tool prints must actually round-trip."""
payload_file = tmp_path / "payload.bin"
payload_file.write_bytes(PAYLOAD)
encoded_file = tmp_path / "value.txt"
enc = run_script("blob_decode.py",
["--encode", "--file", str(payload_file), "--inner", "raw",
"--outer", "base64", "--cut", "0", "--out", str(encoded_file)])
assert enc.returncode == 0, enc.output
back = run_script("blob_decode.py",
["--file", str(encoded_file), "--outer", "base64",
"--min-size", "1", "--out", str(tmp_path / "back.bin")])
assert back.returncode == 0, back.output
assert (tmp_path / "back.bin").read_bytes() == PAYLOAD
def test_an_undecodable_blob_exits_one(self, tmp_path):
value = tmp_path / "value.txt"
value.write_text("not-a-blob-at-all-just-prose-text\n", encoding="utf-8")
result = run_script("blob_decode.py", ["--file", str(value), "--outer", "hex"])
assert result.returncode == 1, result.output
assert not result.has_traceback
def test_empty_stdin_is_reported_as_invalid_input(self):
"""Exit code 2, not 1: an empty input is invalid, not a failed decode."""
result = run_script("blob_decode.py", [], stdin=" ")
assert result.returncode == 2, result.output
assert "empty input" in result.output

90
vendor-new/apk-reverse/tests/unit/test_budget_measurement.py Normal file
View File

@@ -0,0 +1,90 @@
"""The budget gate must reach the same verdict whether or not `tiktoken` is installed.
This is a small test for a defect that cost two red CI runs. `check_budget.py` measures the
always-loaded body in tokens, preferring `tiktoken` and falling back to `characters / N` when it is
absent. The first version's `N` was guessed (3.6) instead of calibrated, so a machine with tiktoken
saw ~10,900 tokens and passed while a machine without it saw ~13,200 and warned -- and this
repository's gate runner (CI's 3.11 job, and any contributor's box) is the one without it.
A gate whose answer depends on which packages the runner happens to have installed is not a gate, so
both paths are forced here and compared.
"""
from __future__ import annotations
import importlib
import sys
from pathlib import Path
import pytest
from conftest import REPO_ROOT
pytestmark = pytest.mark.unit
@pytest.fixture
def budget():
sys.path.insert(0, str(REPO_ROOT))
return importlib.import_module("check_budget")
def _has_tiktoken():
"""Whether a real tokenizer is available, reported rather than assumed.
The comparison needs *two* implementations; where there is only one, the honest outcome is a
skip with the install command, not a failure and not a silent pass. CI installs it, so the
property is verified where it matters; a contributor without it has both gates agreeing anyway,
because the fallback is calibrated to the same constant.
"""
try:
import tiktoken # noqa: F401
return True
except Exception:
return False
pytestmark = pytest.mark.unit
def test_the_fallback_estimate_agrees_with_the_real_tokenizer(budget):
"""Both measurement paths must land on the same side of the soft budget."""
if not _has_tiktoken():
pytest.skip("tiktoken is not installed here, so there is no measurement to compare against; "
"install it with `python -m pip install tiktoken`")
text = (REPO_ROOT / "skills" / "apk-reverse" / "SKILL.md").read_text(encoding="utf-8")
real, how_real = budget.token_estimate(text)
assert "tiktoken" in how_real, how_real
# Force the fallback path by handing the module a tokenizer that cannot be imported.
saved = sys.modules.get("tiktoken")
sys.modules["tiktoken"] = None # `import tiktoken` raises here
try:
fallback, how_fallback = budget.token_estimate(text)
finally:
if saved is not None:
sys.modules["tiktoken"] = saved
else:
sys.modules.pop("tiktoken", None)
assert "estimate" in how_fallback, how_fallback
drift = abs(real - fallback) / max(1, real)
assert drift < 0.05, (
"the fallback is %.1f%% away from tiktoken (%d vs %d): recalibrate TOKENS_PER_CHAR "
"against a fresh measurement rather than a guess" % (100 * drift, fallback, real))
assert (real <= budget.TOKEN_SOFT) == (fallback <= budget.TOKEN_SOFT), (
"the two paths straddle the soft budget (%d real / %d fallback vs %d): the gate would "
"answer differently on CI than on a developer machine"
% (real, fallback, budget.TOKEN_SOFT))
def test_the_fallback_constant_is_the_calibrated_one(budget):
"""Pin the constant to a measurement: `chars / tokens` on SKILL.md, recomputed here."""
text = (REPO_ROOT / "skills" / "apk-reverse" / "SKILL.md").read_text(encoding="utf-8")
real, how = budget.token_estimate(text)
if "tiktoken" not in how:
pytest.skip("no tiktoken available, so there is no measurement to calibrate against")
expected = len(text) / real
assert abs(budget.TOKENS_PER_CHAR - expected) < 0.25, (
"TOKENS_PER_CHAR is %.3f but this SKILL.md measures %.3f characters per token -- the "
"document changed, so recalibrate" % (budget.TOKENS_PER_CHAR, expected))

216
vendor-new/apk-reverse/tests/unit/test_dex_patch_bytes_regression.py Normal file
View File

@@ -0,0 +1,216 @@
"""The established regression for scripts/dex_patch_bytes.py.
This is the repository's pre-existing red line, not a test invented here: the
l1l3 benchmark row recorded that patching the first root check in
`MainActivity.onCreate` produces a specific file, and any refactor of the dex
reader, the spec matcher, the header fixer or the writer must reproduce it
byte for byte.
`tools/_work/bench/**` is git-ignored, so these tests skip with a printed reason
on a fresh checkout. That is deliberate: a regression marker that turns red
because a sample is absent is a marker people learn to ignore.
"""
from __future__ import annotations
import hashlib
import json
import pytest
from conftest import L1L3_PATCHED_SHA256, load_script, require_path, run_script
dexutil = load_script("dexutil.py")
pytestmark = pytest.mark.unit
SPEC_NAME = "l1_root.spec.json"
SITE_OFF = 0x9D0
OLD_BYTES = "39000e00"
NEW_BYTES = "00000000"
@pytest.fixture
def sample(bench_l1l3):
"""Paths to the recorded input, spec and expected output."""
dex = require_path(bench_l1l3 / "dex" / "classes.dex",
"git-ignored bench tree")
spec = require_path(bench_l1l3 / SPEC_NAME, "git-ignored bench tree")
expected = require_path(bench_l1l3 / "dex" / "classes.patched.dex",
"git-ignored bench tree")
return {"dex": dex, "spec": spec, "expected": expected}
def patch(sample, tmp_path, extra=()):
out = tmp_path / "out.patched.dex"
report = tmp_path / "report.json"
result = run_script("dex_patch_bytes.py",
[str(sample["dex"]), "--spec", str(sample["spec"]),
"-o", str(out), "--report", str(report)] + list(extra))
return result, out, report
# --------------------------------------------------------------------------- #
# the hash marker
# --------------------------------------------------------------------------- #
class TestRecordedRegression:
def test_the_patch_runs_and_exits_zero(self, sample, tmp_path):
result, _out, _report = patch(sample, tmp_path)
assert not result.timed_out, repr(result)
assert not result.has_traceback, result.output
assert result.returncode == 0, result.output
def test_output_matches_the_recorded_sha256(self, sample, tmp_path):
_result, out, _report = patch(sample, tmp_path)
digest = hashlib.sha256(out.read_bytes()).hexdigest()
assert digest.startswith(L1L3_PATCHED_SHA256), (
"the recorded marker moved: got %s, expected prefix %s"
% (digest, L1L3_PATCHED_SHA256))
def test_output_is_byte_identical_to_the_stored_expectation(self, sample, tmp_path):
_result, out, _report = patch(sample, tmp_path)
assert out.read_bytes() == sample["expected"].read_bytes()
def test_only_the_four_patch_bytes_and_the_header_change(self, sample, tmp_path):
"""Pin *where* the difference is, so a stray write cannot pass as a hash match.
The allowed window is bytes 8..32 (checksum + signature, recomputed) plus
the four bytes of the patched instruction. Two of those four keep their
original value (`0e00` -> `0000` clears one byte and leaves the zero
byte), and a recomputed checksum may keep a byte by coincidence, so the
assertion is containment plus the changes that must occur.
"""
_result, out, _report = patch(sample, tmp_path)
before = sample["dex"].read_bytes()
after = out.read_bytes()
assert len(before) == len(after), "an equal-length patch must not resize the file"
diffs = {i for i, (a, b) in enumerate(zip(before, after)) if a != b}
window = set(range(8, 32)) | {SITE_OFF, SITE_OFF + 1, SITE_OFF + 2, SITE_OFF + 3}
assert diffs <= window, "bytes changed outside the owned window: %r" % sorted(
diffs - window)
# The two bytes that differ between 39000e00 and 00000000 must have moved.
assert {SITE_OFF, SITE_OFF + 2} <= diffs
# Both header fields are recomputed; each must have changed somewhere.
assert diffs & set(range(8, 12)), "no checksum byte changed"
assert diffs & set(range(12, 32)), "no signature byte changed"
def test_the_written_dex_self_verifies(self, sample, tmp_path):
_result, out, _report = patch(sample, tmp_path)
data = bytearray(out.read_bytes())
assert dexutil.verify_dex_header(data) == (True, True)
def test_the_result_still_parses_and_decodes_cleanly(self, sample, tmp_path):
_result, out, _report = patch(sample, tmp_path)
dex, _entry = dexutil.load_dex(str(out))
assert dex.check() == []
found = dex.find_method("Lsg/vantagepoint/uncrackable1/MainActivity;",
"onCreate", "(Landroid/os/Bundle;)V")
assert found
_section, _idx, code_off = found
_insns, clean, expected = dex.decode_all(code_off)
assert clean, "the patched method must still decode to exactly insns_size units"
# --------------------------------------------------------------------------- #
# the report the tool writes
# --------------------------------------------------------------------------- #
class TestPatchReport:
def test_report_records_the_site_and_the_polarity_check(self, sample, tmp_path):
_result, _out, report = patch(sample, tmp_path)
reports = json.loads(report.read_text(encoding="utf-8"))
assert len(reports) == 1
entry = reports[0]
assert entry["ok"] is True
assert entry["site_off"] == SITE_OFF
assert entry["old_bytes"] == OLD_BYTES
assert entry["new_bytes"] == NEW_BYTES
assert entry["expect_next_ok"] is True
assert entry["size_delta"] == 0
def test_report_confirms_the_post_edit_instruction_boundary(self, sample, tmp_path):
_result, _out, report = patch(sample, tmp_path)
entry = json.loads(report.read_text(encoding="utf-8"))[0]
assert entry["post_decode_clean"] is True
assert entry["verifier_edges"] == 0
def test_stdout_states_the_polarity_check_passed(self, sample, tmp_path):
result, _out, _report = patch(sample, tmp_path)
assert "polarity: expect_next satisfied" in result.stdout
def test_dry_run_reports_without_writing(self, sample, tmp_path):
out = tmp_path / "should-not-exist.dex"
result = run_script("dex_patch_bytes.py",
[str(sample["dex"]), "--spec", str(sample["spec"]),
"-o", str(out), "--dry-run"])
assert result.returncode == 0, result.output
assert "dry run: nothing written" in result.stdout
assert not out.exists()
def test_default_output_name_is_derived_from_the_input(self, sample, tmp_path):
"""Run on a copy: the bench tree is evidence and is never written by this suite."""
local = tmp_path / "classes.dex"
local.write_bytes(sample["dex"].read_bytes())
result = run_script("dex_patch_bytes.py",
[str(local), "--spec", str(sample["spec"])])
assert result.returncode == 0, result.output
derived = tmp_path / "classes.patched.dex"
assert derived.exists(), result.output
assert hashlib.sha256(derived.read_bytes()).hexdigest() == \
hashlib.sha256(sample["expected"].read_bytes()).hexdigest()
# --------------------------------------------------------------------------- #
# refusal paths: a wrong spec must fail loudly, not write a plausible file
# --------------------------------------------------------------------------- #
class TestRefusals:
def test_a_dry_run_against_a_missing_match_fails(self, sample, tmp_path):
spec = tmp_path / "bad.spec.json"
spec.write_text(json.dumps([{
"name": "no such site",
"class": "Lsg/vantagepoint/uncrackable1/MainActivity;",
"method": "onCreate",
"desc": "(Landroid/os/Bundle;)V",
"match": {"kind": "if-testz", "bytes": "deadbeef"},
"replace": {"kind": "nops"},
}]), encoding="utf-8")
result = run_script("dex_patch_bytes.py",
[str(sample["dex"]), "--spec", str(spec), "--dry-run"])
assert result.returncode == 1, result.output
assert "no instruction matched" in result.output
def test_a_wrong_polarity_pin_is_refused(self, sample, tmp_path):
"""expect_next exists to make the wrong side of a branch impossible to patch."""
spec = tmp_path / "bad-polarity.spec.json"
spec.write_text(json.dumps([{
"name": "wrong expected neighbour",
"class": "Lsg/vantagepoint/uncrackable1/MainActivity;",
"method": "onCreate",
"desc": "(Landroid/os/Bundle;)V",
"match": {"kind": "if-testz", "bytes": OLD_BYTES},
"expect_next": {"kind": "return"},
"replace": {"kind": "nops"},
}]), encoding="utf-8")
result = run_script("dex_patch_bytes.py",
[str(sample["dex"]), "--spec", str(spec), "--dry-run"])
assert result.returncode == 1, result.output
assert "expect_next not satisfied" in result.output
def test_a_missing_dex_is_a_usage_error(self, tmp_path, sample):
result = run_script("dex_patch_bytes.py",
[str(tmp_path / "nope.dex"), "--spec", str(sample["spec"])])
assert result.returncode == 2, result.output
assert "no such file" in result.output
def test_a_missing_spec_argument_is_a_usage_error(self, sample):
result = run_script("dex_patch_bytes.py", [str(sample["dex"])])
assert result.returncode == 2, result.output
def test_a_non_dex_input_is_refused_with_code_1(self, tmp_path, sample):
"""Structurally invalid input is a target-side failure (1), not a usage error (2)."""
bogus = tmp_path / "bogus.dex"
bogus.write_bytes(b"not a dex at all" * 8)
result = run_script("dex_patch_bytes.py",
[str(bogus), "--spec", str(sample["spec"]), "--dry-run"])
assert result.returncode == 1, result.output
assert "not a dex" in result.output

393
vendor-new/apk-reverse/tests/unit/test_dexutil.py Normal file
View File

@@ -0,0 +1,393 @@
"""Unit tests for skills/apk-reverse/scripts/dexutil.py.
The dex format table is the one place where a wrong entry silently shifts every
offset after it, so widths are asserted opcode by opcode against the cases the
script's own docstring calls out. No device and no sample file are needed for
the table itself; the sample-backed assertions skip when the git-ignored bench
tree is absent.
"""
from __future__ import annotations
import hashlib
import zlib
import pytest
from conftest import load_script, require_path
dexutil = load_script("dexutil.py")
pytestmark = pytest.mark.unit
# --------------------------------------------------------------------------- #
# little-endian scalar readers
# --------------------------------------------------------------------------- #
class TestScalarReaders:
def test_u16_and_u32_are_little_endian(self):
assert dexutil.u16(b"\x01\x02", 0) == 0x0201
assert dexutil.u32(b"\x01\x02\x03\x04", 0) == 0x04030201
def test_u16_and_u32_read_from_an_offset(self):
buf = b"\xff\xff\x34\x12\x78\x56\x34\x12"
assert dexutil.u16(buf, 2) == 0x1234
assert dexutil.u32(buf, 4) == 0x12345678
@pytest.mark.parametrize("value,expected", [
(0x0000, 0),
(0x7FFF, 0x7FFF),
(0x8000, -0x8000),
(0xFFFF, -1),
])
def test_s16_boundaries(self, value, expected):
assert dexutil.s16(value) == expected
def test_read_uleb_single_byte(self):
assert dexutil.read_uleb(b"\x7f", 0) == (0x7F, 1)
def test_read_uleb_two_bytes_and_offset(self):
value, off = dexutil.read_uleb(b"\x00\x80\x01", 1)
assert value == 128
assert off == 3
def test_read_uleb_multi_byte(self):
# 0xE5 0x8E 0x26 -> 624485, the example from the format specification
value, off = dexutil.read_uleb(b"\xe5\x8e\x26", 0)
assert value == 624485
assert off == 3
# --------------------------------------------------------------------------- #
# instruction width table
# --------------------------------------------------------------------------- #
# Each case is a trap the module docstring names explicitly.
WIDTH_CASES = [
(0x00, 1, "plain nop is one unit"),
(0x02, 2, "move/from16"),
(0x03, 3, "move/16"),
(0x13, 2, "const/16"),
(0x14, 3, "const"),
(0x16, 2, "const-wide/16"),
(0x17, 3, "const-wide/32"),
(0x18, 5, "const-wide"),
(0x1A, 2, "const-string (21c) carries a 16-bit index"),
(0x1B, 3, "const-string/jumbo (31c) carries a 32-bit index"),
(0x1C, 2, "const-class"),
(0x1E, 1, "monitor-exit breaks the same-family reading"),
(0x20, 2, "instance-of"),
(0x21, 1, "array-length breaks the same-family reading"),
(0x27, 1, "throw"),
(0x28, 1, "goto (10t) is one unit"),
(0x29, 2, "goto/16 (20t)"),
(0x2A, 3, "goto/32 (30t)"),
(0x32, 2, "if-test (22t) is two units, not one"),
(0x37, 2, "last if-test"),
(0x38, 2, "if-testz (21t) is two units, not one"),
(0x3D, 2, "last if-testz"),
(0x6E, 3, "invoke-kind (35c)"),
(0x74, 3, "invoke-kind/range (3rc)"),
(0x7B, 1, "neg-int"),
(0x90, 2, "add-int (23x)"),
(0xD0, 2, "add-int/lit16 (22s)"),
(0xFA, 4, "invoke-polymorphic (45cc)"),
(0xFC, 3, "invoke-custom (35c)"),
]
class TestInstructionWidths:
@pytest.mark.parametrize("op,units,why", WIDTH_CASES,
ids=["%02x" % c[0] for c in WIDTH_CASES])
def test_width(self, op, units, why):
assert dexutil.insn_units(op, b"\x00" * 32, 0, 32) == units, why
def test_table_and_accessor_agree_for_every_listed_opcode(self):
"""insn_units must not diverge from OP_UNITS for any allocated opcode."""
buf = b"\x00" * 32
drifted = []
for op, units in sorted(dexutil.OP_UNITS.items()):
got = dexutil.insn_units(op, buf, 0, len(buf))
if got != units:
drifted.append((hex(op), units, got))
assert drifted == []
@pytest.mark.parametrize("op", [0x3E, 0x43, 0x73, 0x79, 0x7A, 0xE3, 0xF9])
def test_unallocated_opcodes_still_terminate(self, op):
"""An unlisted opcode must advance by one unit, not run off the end."""
assert dexutil.insn_units(op, b"\x00" * 8, 0, 8) == 1
def test_packed_switch_payload_width(self):
# 00 01 | size=3 -> 4 + 3*2 = 10 units
buf = bytes([0x00, 0x01]) + (3).to_bytes(2, "little") + b"\x00" * 12
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 10
def test_sparse_switch_payload_width(self):
# 00 02 | size=3 -> 2 + 3*4 = 14 units
buf = bytes([0x00, 0x02]) + (3).to_bytes(2, "little") + b"\x00" * 24
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 14
def test_fill_array_data_payload_width(self):
# 00 03 | element_width=4 | size=30 -> 4 + ceil(30*4/2) = 64 units.
# The measured case named in the docstring: counting this as 5 units is
# what desynchronises the rest of the walk.
buf = bytes([0x00, 0x03]) + (4).to_bytes(2, "little") + (30).to_bytes(4, "little")
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 64
def test_fill_array_data_payload_pads_to_an_even_byte_count(self):
# element_width=1, size=3 -> 3 bytes pads to 4 -> 4 + 2 = 6 units
buf = bytes([0x00, 0x03]) + (1).to_bytes(2, "little") + (3).to_bytes(4, "little")
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 6
def test_plain_nop_is_not_treated_as_a_payload(self):
assert dexutil.insn_units(0x00, b"\x00\x00\x00\x00", 0, 4) == 1
# --------------------------------------------------------------------------- #
# header parsing and the structural check
# --------------------------------------------------------------------------- #
# Verified dex header offsets, as declared by dexutil.OFFSETS. Asserted rather
# than assumed: these are the values a hand-written reader most often gets wrong.
HEADER_OFFSETS = {
"file_size": 0x20, "header_size": 0x24, "endian_tag": 0x28,
"map_off": 0x34, "string_ids_off": 0x3C, "type_ids_off": 0x44,
"proto_ids_off": 0x4C, "field_ids_off": 0x54, "method_ids_off": 0x5C,
"class_defs_off": 0x64,
}
# The six table pointers dex.check() requires to be inside the file.
TABLE_POINTERS = ("string_ids_off", "type_ids_off", "proto_ids_off",
"field_ids_off", "method_ids_off", "class_defs_off")
def synthetic_dex(size=512):
"""A dex-shaped buffer whose header is internally consistent.
Enough for the structural reader: magic, a matching file_size, and the six
table pointers inside the file. It is deliberately not a runnable dex, so
nothing here claims parse-level validity of a real image.
"""
buf = bytearray(size)
def put(offset, value, width=4):
buf[offset:offset + width] = value.to_bytes(width, "little")
buf[0:8] = b"dex\n035\x00"
put(HEADER_OFFSETS["file_size"], size)
put(HEADER_OFFSETS["header_size"], 0x70)
put(HEADER_OFFSETS["endian_tag"], 0x12345678)
put(HEADER_OFFSETS["map_off"], 0x70)
for key in TABLE_POINTERS:
# size 0, pointer 0x70: every pointer lands inside the file.
put(HEADER_OFFSETS[key] - 4, 0)
put(HEADER_OFFSETS[key], 0x70)
return buf
class TestHeaderParsing:
def test_the_offset_table_matches_the_format_specification(self):
for key, expected in HEADER_OFFSETS.items():
assert dexutil.OFFSETS[key] == expected, key
def test_header_fields_are_parsed_out_of_the_buffer(self):
dex = dexutil.Dex(bytes(synthetic_dex(512)))
assert dex.header["file_size"] == 512
assert dex.header["header_size"] == 0x70
assert dex.header["string_ids_off"] == 0x70
def test_magic_is_parsed(self):
dex = dexutil.Dex(bytes(synthetic_dex(256)))
assert dex.d[:4] == b"dex\n"
def test_consistent_header_reports_no_problems(self):
dex = dexutil.Dex(bytes(synthetic_dex(512)))
assert dex.check() == []
def test_bad_magic_is_reported(self):
buf = synthetic_dex(512)
buf[0:4] = b"zip\n"
assert any("not a dex" in p for p in dexutil.Dex(bytes(buf)).check())
def test_file_size_mismatch_is_reported(self):
buf = synthetic_dex(512)
buf[HEADER_OFFSETS["file_size"]:HEADER_OFFSETS["file_size"] + 4] = \
(999).to_bytes(4, "little")
assert any("file_size" in p for p in dexutil.Dex(bytes(buf)).check())
def test_out_of_range_table_pointer_is_reported(self):
buf = synthetic_dex(512)
at = HEADER_OFFSETS["string_ids_off"]
buf[at:at + 4] = (0).to_bytes(4, "little")
assert any("string_ids_off" in p for p in dexutil.Dex(bytes(buf)).check())
def test_string_safe_never_raises_on_a_junk_index(self):
"""A desynced decode passes junk indices; the reader must not throw."""
dex = dexutil.Dex(bytes(synthetic_dex(512)))
assert dex.string_safe(10**6).startswith("<string_idx")
# --------------------------------------------------------------------------- #
# branch targets
# --------------------------------------------------------------------------- #
def fake_dex(buf):
"""A Dex instance over raw bytes, without header parsing.
branch_target needs only self.d plus the instruction dict, so this keeps the
displacement arithmetic testable without a whole valid dex image.
"""
dex = dexutil.Dex.__new__(dexutil.Dex)
dex.d = bytes(buf)
dex.name = "<synthetic>"
return dex
def insn_at_zero(buf, op, raw_len):
return {"off": 0, "op": op, "raw": bytes(buf[:raw_len]),
"units": raw_len // 2, "name": "synthetic"}
class TestBranchTargets:
def test_if_testz_negative_displacement(self):
buf = bytes([0x38, 0x00]) + (-1).to_bytes(2, "little", signed=True)
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x38, 4)) == -2
def test_if_testz_positive_displacement(self):
buf = bytes([0x3D, 0x01]) + (2).to_bytes(2, "little", signed=True)
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x3D, 4)) == 4
def test_if_test_two_registers(self):
buf = bytes([0x32, 0x10]) + (-4).to_bytes(2, "little", signed=True)
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x32, 4)) == -8
def test_goto_signed_byte(self):
# 10t: op, int8. 0xFE -> -2 units -> -4 bytes from the instruction.
buf = bytes([0x28, 0xFE])
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x28, 2)) == -4
def test_goto16(self):
buf = bytes([0x29, 0x00]) + (3).to_bytes(2, "little", signed=True)
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x29, 4)) == 6
def test_goto32_negative(self):
buf = bytes([0x2A, 0x00]) + (-1).to_bytes(4, "little", signed=True)
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x2A, 6)) == -2
def test_non_branch_returns_none(self):
buf = bytes([0x0E, 0x00])
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x0E, 2)) is None
def test_branch_regs_if_testz_reads_one_register(self):
buf = bytes([0x38, 0x07]) + (-1).to_bytes(2, "little", signed=True)
regs, kind = fake_dex(buf).branch_regs(insn_at_zero(buf, 0x38, 4))
assert regs == [7]
assert kind == "if-testz"
def test_branch_regs_if_test_reads_two_registers(self):
buf = bytes([0x32, 0x21]) + (-1).to_bytes(2, "little", signed=True)
regs, kind = fake_dex(buf).branch_regs(insn_at_zero(buf, 0x32, 4))
assert regs == [2, 1]
assert kind == "if-test"
def test_branch_regs_is_empty_for_a_non_branch(self):
buf = bytes([0x0E, 0x00])
assert fake_dex(buf).branch_regs(insn_at_zero(buf, 0x0E, 2)) == ([], None)
# --------------------------------------------------------------------------- #
# header integrity: signature before checksum
# --------------------------------------------------------------------------- #
class TestHeaderIntegrity:
def test_fix_then_verify_round_trips(self):
data = bytearray(synthetic_dex(512))
dexutil.fix_dex_header(data)
assert dexutil.verify_dex_header(data) == (True, True)
def test_signature_covers_the_bytes_after_the_signature_field(self):
data = bytearray(synthetic_dex(512))
dexutil.fix_dex_header(data)
assert bytes(data[12:32]) == hashlib.sha1(bytes(data[32:])).digest()
def test_checksum_covers_the_signature_field(self):
"""The order requirement: adler32 over d[12:] includes the written signature."""
data = bytearray(synthetic_dex(512))
dexutil.fix_dex_header(data)
stored = int.from_bytes(data[8:12], "little")
assert stored == zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF
def test_checksum_taken_over_a_zeroed_signature_does_not_verify(self):
"""The failure mode the module docstring names: the header never verifies."""
data = bytearray(synthetic_dex(512))
data[12:32] = b"\x00" * 20
data[8:12] = (zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF).to_bytes(4, "little")
data[12:32] = hashlib.sha1(bytes(data[32:])).digest()
checksum_ok, _signature_ok = dexutil.verify_dex_header(data)
assert checksum_ok is False
def test_a_tampered_body_fails_verification(self):
data = bytearray(synthetic_dex(512))
dexutil.fix_dex_header(data)
data[-1] ^= 0xFF
assert dexutil.verify_dex_header(data) == (False, False)
def test_fix_reports_before_and_after_values(self):
data = bytearray(synthetic_dex(512))
report = dexutil.fix_dex_header(data)
assert set(report) == {"before_checksum", "after_checksum",
"before_signature", "after_signature"}
assert report["before_checksum"] != report["after_checksum"]
# --------------------------------------------------------------------------- #
# sample-backed assertions (skip when the git-ignored bench tree is absent)
# --------------------------------------------------------------------------- #
SAMPLE_CLASS = "Lsg/vantagepoint/uncrackable1/MainActivity;"
SAMPLE_METHOD = ("onCreate", "(Landroid/os/Bundle;)V")
SAMPLE_SITE = 0x9D0
SAMPLE_SITE_BYTES = "39000e00"
class TestAgainstTheRecordedSample:
"""The l1l3 fixture is git-ignored, so these skip in CI by design."""
@pytest.fixture(autouse=True)
def _need_sample(self, bench_l1l3):
self.dex_path = require_path(
bench_l1l3 / "dex" / "classes.dex",
"the l1l3 sample is a download into the git-ignored bench tree")
def _method(self):
dex, _entry = dexutil.load_dex(str(self.dex_path))
found = dex.find_method(SAMPLE_CLASS, *SAMPLE_METHOD)
assert found, "the recorded fixture must still contain MainActivity.onCreate"
_section, _idx, code_off = found
return dex, code_off
def test_sample_dex_parses_cleanly(self):
dex, _entry = dexutil.load_dex(str(self.dex_path))
assert dex.check() == []
def test_decode_walks_exactly_insns_size_units(self):
dex, code_off = self._method()
insns, clean, expected = dex.decode_all(code_off)
assert clean, "decode did not land on 0x%x" % expected
assert insns
def test_the_recorded_root_check_branch_is_at_0x9d0(self):
dex, code_off = self._method()
target = dex.insn_at(code_off, SAMPLE_SITE)
assert target is not None, "instruction boundary at 0x%x disappeared" % SAMPLE_SITE
assert target["raw"].hex() == SAMPLE_SITE_BYTES
assert target["op"] in dexutil.IF_TESTZ
def test_branch_target_resolves_to_an_instruction_boundary(self):
dex, code_off = self._method()
target = dex.insn_at(code_off, SAMPLE_SITE)
dest = dex.branch_target(target)
assert dest is not None
assert dex.insn_at(code_off, dest) is not None, (
"branch target 0x%x is not an instruction boundary" % dest)
def test_every_branch_target_is_an_instruction_boundary(self):
"""A decode desync shows up here first: a branch into the middle of an insn."""
dex, code_off = self._method()
boundaries = {i["off"] for i in dex.decode(code_off)}
stray = sorted(t for t in dex.all_targets(code_off) if t not in boundaries)
assert stray == [], "branch targets outside the instruction stream: %r" % stray

192
vendor-new/apk-reverse/tests/unit/test_protobuf_decode_raw.py Normal file
View File

@@ -0,0 +1,192 @@
"""Unit tests for skills/apk-reverse/scripts/protobuf_decode_raw.py.
Two properties are pinned here, because both are claims the reference material
makes and neither is visible from reading the decoder:
* the built-in known-answer fixture set passes in full (26/26), and
* a decode -> re-encode round trip is byte-identical for every fixture except
the one fixture that is deliberately non-canonical, which must instead
report *why* it differs rather than silently normalising.
"""
from __future__ import annotations
import pytest
from conftest import load_script, run_script
pb = load_script("protobuf_decode_raw.py")
pytestmark = pytest.mark.unit
# 12 fixtures x (fixture check + round trip) + 2 wire-format trap checks.
EXPECTED_FIXTURE_COUNT = 12
EXPECTED_TOTAL_CHECKS = 26
@pytest.fixture(scope="module")
def fixtures():
return pb._fixtures()
# --------------------------------------------------------------------------- #
# varint primitives
# --------------------------------------------------------------------------- #
class TestVarintPrimitives:
def test_canonical_example(self):
assert pb.encode_varint(150) == bytes.fromhex("9601")
def test_read_varint_returns_info_next_index_and_no_error(self):
info, nxt, err = pb.read_varint(bytes.fromhex("9601"), 0, 2)
assert err is None
assert info["value"] == 150
assert info["bytes"] == 2
assert nxt == 2
def test_a_truncated_varint_is_an_error_not_a_value(self):
info, nxt, err = pb.read_varint(bytes.fromhex("96"), 0, 1)
assert info is None
assert err and "truncated" in err
assert nxt == 0
def test_a_non_canonical_encoding_is_flagged(self):
"""A padded varint must be reported: it is why re-encode cannot be assumed equal."""
info, _nxt, _err = pb.read_varint(bytes.fromhex("8000"), 0, 2)
assert info["value"] == 0
assert info["non_canonical"] is True
assert info["canonical_bytes"] == 1
def test_varint_len_matches_the_encoding(self):
for value in (0, 1, 127, 128, 300, 2 ** 31, 2 ** 63 - 1):
assert pb.varint_len(value) == len(pb.encode_varint(value))
def test_zigzag(self):
assert pb.zigzag_decode(0) == 0
assert pb.zigzag_decode(1) == -1
assert pb.zigzag_decode(2) == 1
assert pb.zigzag_decode(4294967294) == 2147483647
def test_int_views_wrap(self):
assert pb.int32_view(0xFFFFFFFF) == -1
assert pb.int64_view(0xFFFFFFFFFFFFFFFF) == -1
# --------------------------------------------------------------------------- #
# the built-in fixture set
# --------------------------------------------------------------------------- #
class TestFixtureSet:
def test_the_selftest_function_passes_end_to_end(self, capsys):
assert pb.selftest(None) == 0
out = capsys.readouterr().out
assert "selftest: %d/%d PASS" % (EXPECTED_TOTAL_CHECKS, EXPECTED_TOTAL_CHECKS) in out
assert "FAIL" not in out
def test_fixture_count_is_stable(self, fixtures):
"""The 26/26 headline is 12 fixtures x 2 + 2; a silent drop would hide."""
assert len(fixtures) == EXPECTED_FIXTURE_COUNT
def test_every_fixture_has_a_name_payload_and_expectations(self, fixtures):
for name, payload, expects, _rt in fixtures:
assert isinstance(name, str) and name
assert isinstance(payload, (bytes, bytearray)) and payload
assert expects, "fixture %s has no assertions" % name
def test_each_declared_expectation_holds(self, fixtures):
failures = []
for name, payload, expects, _rt in fixtures:
ctx = pb.Ctx(max_depth=6)
frames = pb._fixture_frames(name, payload)
fields = []
for start, end, _p in frames:
found, _n, _s = pb.walk(payload, start, end, 0, ctx, "f", analyse=True)
fields.extend(found)
for label, num, expected in expects:
got = pb._selftest_lookup(fields, frames, payload, label, num)
if got != expected:
failures.append((name, label, expected, got))
assert failures == []
# --------------------------------------------------------------------------- #
# round trip
# --------------------------------------------------------------------------- #
class TestRoundTrip:
def _reencode(self, name, payload):
frames = pb._fixture_frames(name, payload)
tree = {"frames": [{"fields": pb.walk(payload, s, e, 0, pb.Ctx(6), "f",
analyse=True)[0]}
for s, e, _p in frames]}
report = {"fields": 0, "identical": 0, "mismatched": []}
out = pb.reencode_tree(tree, report)
expected = b"".join(payload[s:e] for s, e, _p in frames)
return out, expected, report
def test_canonical_fixtures_reencode_byte_identically(self, fixtures):
differing = []
for name, payload, _expects, rt in fixtures:
if rt == "non-canonical":
continue
out, expected, _report = self._reencode(name, payload)
if out != expected:
differing.append((name, expected.hex(), out.hex()))
assert differing == []
def test_the_non_canonical_fixture_reports_why_it_differs(self, fixtures):
"""A normalising re-encode must say so, not return different bytes quietly."""
targets = [f for f in fixtures if f[3] == "non-canonical"]
assert targets, "the non-canonical fixture disappeared from the set"
for name, payload, _expects, _rt in targets:
out, expected, report = self._reencode(name, payload)
assert out != expected
assert out.hex() == "089601"
assert report["mismatched"], "the mismatch must be recorded"
assert "non-canonical" in report["mismatched"][0]["reason"]
def test_reencode_accounts_for_every_field(self, fixtures):
for name, payload, _expects, rt in fixtures:
_out, _expected, report = self._reencode(name, payload)
assert report["fields"] > 0, name
# --------------------------------------------------------------------------- #
# the two wire-format traps, asserted rather than narrated
# --------------------------------------------------------------------------- #
class TestWireFormatTraps:
def test_explicit_zero_is_not_the_same_bytes_as_absent(self):
zero = pb.tag_0(5, 0) + pb.encode_varint(0)
assert zero != b""
def test_explicit_zero_decodes_to_value_zero(self):
zero = pb.tag_0(5, 0) + pb.encode_varint(0)
fields = pb.walk(zero, 0, len(zero), 0, pb.Ctx(6), "f")[0]
assert fields[0]["value"] == 0
def test_absent_field_decodes_to_no_fields(self):
assert pb.walk(b"", 0, 0, 0, pb.Ctx(6), "f")[0] == []
def test_a_packed_payload_offers_both_readings(self):
"""Three varints are also a valid nested message; only a schema decides."""
packed = b"".join(pb.encode_varint(v) for v in (3, 270, 86942))
cands, view = pb.analyse_payload(packed, 0, pb.Ctx(6), "f", 0)
live = sorted(c["kind"] for c in cands if c["confidence"] > 0.0)
all_kinds = sorted(c["kind"] for c in cands)
assert "packed_varint" in live
assert "nested_message" in all_kinds
assert view == "packed_varint"
# --------------------------------------------------------------------------- #
# the subprocess contract
# --------------------------------------------------------------------------- #
class TestSelftestCli:
def test_selftest_subcommand_exits_zero_and_prints_the_tally(self):
result = run_script("protobuf_decode_raw.py", ["--selftest"], timeout=180)
assert not result.timed_out
assert not result.has_traceback, result.output
assert result.returncode == 0, result.output
assert "selftest: %d/%d PASS" % (EXPECTED_TOTAL_CHECKS, EXPECTED_TOTAL_CHECKS) \
in result.stdout
def test_help_lists_the_selftest_flag(self):
result = run_script("protobuf_decode_raw.py", ["--help"])
assert result.returncode == 0
assert "--selftest" in result.stdout

198
vendor-new/apk-reverse/tests/unit/test_scan_leaks.py Normal file
View File

@@ -0,0 +1,198 @@
"""Unit tests for skills/apk-reverse/scripts/scan_leaks.py.
Decoys are built in a temporary directory at run time. No real target identity
is committed here, and the token-shaped decoys are assembled from fragments so
that this test file itself cannot become a finding if the scanner is ever
pointed at the tests tree.
"""
from __future__ import annotations
import json
import pytest
from conftest import load_script, run_script
scan_leaks = load_script("scan_leaks.py")
pytestmark = pytest.mark.unit
# Synthetic identifiers: reverse-domain-shaped, not derived from any real target.
DECOY_PACKAGE = "com." + "acme" + ".targetapp"
DECOY_TOKEN = "AbCdEf" + "0123456789" + "XyZ"
DECOY_USER = "mkessler"
def write(root, name, body):
path = root / name
path.write_text(body, encoding="utf-8")
return path
def scan(root, *extra, timeout=120):
"""Run the scanner over a temporary root and parse its JSON report."""
result = run_script("scan_leaks.py",
["--root", str(root), "--format", "json"] + list(extra),
timeout=timeout)
assert not result.timed_out, "scan_leaks.py did not finish: %r" % result
assert not result.has_traceback, result.output
# The JSON document is printed first, then the RESULT token line.
lines = result.stdout.splitlines()
token = lines[-1] if lines else ""
doc = json.loads(result.stdout[:result.stdout.rindex("}") + 1])
return result, doc, token
# --------------------------------------------------------------------------- #
# the exemption list as data
# --------------------------------------------------------------------------- #
class TestExemptions:
@pytest.mark.parametrize("value", [
"com.android.internal",
"androidx.core.app",
"com.google.firebase",
"org.example.demo",
"<pkg>",
"${PKG}",
"sg.vantagepoint.uncrackable1",
])
def test_benign_values_carry_an_exemption_reason(self, value):
assert scan_leaks._benign_package(value) != "", value
@pytest.mark.parametrize("value", [
DECOY_PACKAGE,
"com.unknownvendor.someapp",
])
def test_unknown_reverse_domain_values_are_reportable(self, value):
assert scan_leaks._benign_package(value) == "", value
def test_loopback_and_documentation_addresses_are_distinguished(self):
"""Loopback is exempt; an RFC 5737 documentation address is reported on purpose."""
assert scan_leaks._benign_ip("127.0.0.1:8080") != ""
assert scan_leaks._benign_ip("192.0.2.10:8080") == ""
def test_placeholder_user_paths_are_exempt(self):
assert scan_leaks._benign_path("C:\\Users\\<user>\\work") != ""
assert scan_leaks._benign_path("/home/" + DECOY_USER + "/work") == ""
# --------------------------------------------------------------------------- #
# decoys that must be reported
# --------------------------------------------------------------------------- #
class TestDecoyDetection:
def test_strong_decoys_are_reported(self, tmp_path):
write(tmp_path, "notes.md", "\n".join([
'package="%s"' % DECOY_PACKAGE,
"Authorization: Bearer " + DECOY_TOKEN,
"/home/%s/work/build/classes.dex" % DECOY_USER,
]) + "\n")
result, doc, token = scan(tmp_path)
rules = sorted({f["rule"] for f in doc["findings"]})
assert "bundle_pkg_attr" in rules
assert "token_authorization_header" in rules
assert "user_path_posix" in rules
assert all(f["strength"] in ("strong", "certain") for f in doc["findings"])
assert token == "RESULT=leaks_found"
assert result.returncode == 1, result.output
def test_findings_carry_context_not_just_a_line_number(self, tmp_path):
write(tmp_path, "notes.md", 'package="%s"\n' % DECOY_PACKAGE)
_result, doc, _token = scan(tmp_path)
finding = doc["findings"][0]
assert finding["context"].strip()
assert finding["match"] == DECOY_PACKAGE
assert finding["line"] == 1
assert finding["file"] == "notes.md"
def test_device_serial_shaped_token_is_reported(self, tmp_path):
# 16 uppercase alphanumerics with at least one letter, next to a device word.
serial = "R58M" + "12ABCDEF" + "9K7Q"
assert len(serial) == 16
write(tmp_path, "adb.txt", "device %s\n" % serial)
_result, doc, token = scan(tmp_path)
assert any(f["rule"].startswith("device_serial") for f in doc["findings"]), doc
assert token == "RESULT=leaks_found"
# --------------------------------------------------------------------------- #
# decoys that must not be reported
# --------------------------------------------------------------------------- #
class TestExemptDecoysStaySilent:
def test_benign_identifiers_produce_no_findings(self, tmp_path):
write(tmp_path, "notes.md", "\n".join([
'package="com.android.internal"',
'package="com.google.firebase.messaging"',
'package="<pkg>"',
"endpoint 127.0.0.1:8080",
"path C:\\Users\\<user>\\work\\x.dex",
"public crackme sg.vantagepoint.uncrackable1",
]) + "\n")
result, doc, token = scan(tmp_path)
assert doc["findings"] == [], doc["findings"]
assert token == "RESULT=clean"
assert result.returncode == 0
def test_exempted_hits_are_kept_and_auditable_with_show_exempt(self, tmp_path):
"""The exemption list must be inspectable, not just silent."""
write(tmp_path, "notes.md", 'package="com.android.internal"\n')
_result, doc, _token = scan(tmp_path, "--show-exempt")
assert doc["findings"] == []
assert doc["exempted"], "an exempted hit must still be recorded"
assert doc["exempted"][0]["exempt_reason"]
def test_a_weak_only_hit_keeps_the_default_gate_green(self, tmp_path):
"""Documented behaviour: an RFC 5737 address is reported but does not fail --fail-on strong.
Note the measured detail: `ip:port` satisfies both endpoint rules, so one
address is reported twice (once as the `ip:port` form, once as the bare
literal). Recorded here rather than smoothed over -- it is a report-volume
question, not a missed or invented finding.
"""
write(tmp_path, "notes.md", "endpoint 192.0.2.10:8080\n")
result, doc, token = scan(tmp_path)
assert {f["strength"] for f in doc["findings"]} == {"weak"}
assert {f["rule"] for f in doc["findings"]} == {"plain_addr", "plain_addr_ipliteral"}
assert token == "RESULT=leaks_found_strong_only"
assert result.returncode == 0
strict, _doc2, strict_token = scan(tmp_path, "--fail-on", "any")
assert strict_token == "RESULT=leaks_found"
assert strict.returncode == 1
# --------------------------------------------------------------------------- #
# the machine-readable contract
# --------------------------------------------------------------------------- #
class TestReportContract:
def test_json_report_has_the_documented_keys(self, tmp_path):
write(tmp_path, "notes.md", 'package="%s"\n' % DECOY_PACKAGE)
_result, doc, _token = scan(tmp_path)
assert set(doc) >= {"root", "files_scanned", "findings", "exempted",
"errors", "result", "fail_on"}
assert doc["result"] == "leaks_found"
assert doc["fail_on"] == "strong"
def test_result_token_is_always_the_last_line(self, tmp_path):
write(tmp_path, "notes.md", 'package="%s"\n' % DECOY_PACKAGE)
result, _doc, token = scan(tmp_path)
assert result.stdout.splitlines()[-1] == token
assert token.startswith("RESULT=")
def test_list_rules_prints_the_table_and_exits_zero(self):
result = run_script("scan_leaks.py", ["--list-rules"])
assert result.returncode == 0, result.output
assert not result.has_traceback
for rule_id in ("bundle_pkg_attr", "device_serial", "token_github_pat",
"appkey_assignment", "user_path_posix"):
assert rule_id in result.stdout
def test_unknown_category_is_a_usage_error(self, tmp_path):
result = run_script("scan_leaks.py",
["--root", str(tmp_path), "--only", "nosuchcategory"])
assert result.returncode == 2, result.output