reverse-skills-max: initial public release (restored, redacted, hardened)

vendor-new/analyst-ai-pack/skills/measuring-section-entropy-to-detect-packing/LICENSE

@@ -0,0 +1,9 @@
+Apache License 2.0 — see the repository root LICENSE file for the full text.
+
+Copyright the AnalystAIPack contributors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0

vendor-new/analyst-ai-pack/skills/measuring-section-entropy-to-detect-packing/SKILL.md

@@ -0,0 +1,89 @@
+---
+name: measuring-section-entropy-to-detect-packing
+description: 'Detects packing and encryption by measuring Shannon entropy across a binary''s
+  sections and regions: flagging high-entropy executable sections, entropy spikes, and
+  size/raw-vs-virtual anomalies that indicate compression or encryption. Activates for requests
+  to measure entropy, detect packing, or assess whether a sample is compressed/encrypted.'
+domain: cybersecurity
+subdomain: malware-analysis
+tags:
+  - malware-analysis
+  - entropy
+  - packing
+  - static-analysis
+  - pe
+version: 1.0.0
+author: analyst-ai-pack
+license: Apache-2.0
+mitre_attack:
+  - T1027.002
+  - T1027
+d3fend:
+  - D3-FCR
+references:
+  - 'Shannon, A Mathematical Theory of Communication (entropy) — https://doi.org/10.1002/j.1538-7305.1948.tb01338.x'
+  - 'Microsoft PE/COFF specification — https://learn.microsoft.com/windows/win32/debug/pe-format'
+---
+
+# Measuring Section Entropy to Detect Packing
+
+## When to Use
+
+- You want a quick statistical signal of whether a sample is packed or encrypted.
+- You are triaging a PE and need to flag suspicious high-entropy executable sections.
+- You are deciding whether to route a sample to an unpacking workflow.
+
+**Do not use** entropy as a verdict — legitimate compressed resources and installers also show
+high entropy; combine with imports, section names, and raw/virtual size anomalies.
+
+## Prerequisites
+
+- Python (`math` stdlib); optionally `pefile` for per-section analysis.
+
+## Safety & Handling
+
+- Compute entropy on the inert file; never execute the sample.
+- Keep the sample password-protected at rest and reference it by hash.
+
+## Workflow
+
+### Step 1: Compute whole-file and sliding-window entropy
+
+Calculate Shannon entropy (0–8 bits/byte). Sliding-window entropy reveals localized
+high-entropy regions even when the overall value is moderate.
+
+```bash
+python scripts/analyst.py entropy sample.exe
+```
+
+### Step 2: Compute per-section entropy (PE)
+
+For each PE section, compute entropy and compare raw vs. virtual size. Executable sections with
+entropy > ~7.0 are a packing indicator.
+
+### Step 3: Correlate structural anomalies
+
+Flag classic packer signs: high-entropy `.text`, unusual/renamed sections (`UPX0`, random
+names), tiny raw size but large virtual size, and an entry point outside `.text`.
+
+### Step 4: Decide routing
+
+If indicators stack (high entropy + thin IAT + odd sections), route to unpacking; otherwise
+proceed with normal static analysis.
+
+## Validation
+
+- High-entropy executable sections are corroborated by other packing signs before concluding.
+- Benign high-entropy cases (compressed resources) are not misclassified as packers.
+- The routing decision (unpack vs. proceed) is justified by combined indicators.
+
+## Pitfalls
+
+- Calling any high-entropy file "packed" without structural corroboration.
+- Ignoring sliding-window entropy and missing a localized encrypted blob.
+- Overlooking raw-vs-virtual size mismatch, a strong unpacking-at-runtime hint.
+
+## References
+
+- See [`references/api-reference.md`](references/api-reference.md) for the entropy tool.
+- Shannon entropy and the PE/COFF spec (linked in frontmatter).

vendor-new/analyst-ai-pack/skills/measuring-section-entropy-to-detect-packing/references/api-reference.md

@@ -0,0 +1,40 @@
+# API Reference: Measuring Section Entropy to Detect Packing
+
+## Overview
+
+`scripts/analyst.py` computes whole-file Shannon entropy, sliding-window peak entropy, and (for PE
+files, via `pefile`) per-section entropy with raw/virtual size comparison, flagging high-entropy
+executable sections. It reads the inert file and never executes it.
+
+## Dependencies
+
+| Package | Version | Purpose |
+|---------|---------|---------|
+| (stdlib) | — | math (entropy), file IO |
+| pefile | >= 2023.2.7 | per-section analysis (optional) |
+
+## Core functions
+
+### `shannon(data)`
+Shannon entropy in bits/byte (0–8).
+
+### `window_stats(data, window)`
+Returns the peak entropy and the fraction of windows above the packing threshold; a sustained
+high fraction (not a single window) indicates packing when section data is unavailable.
+
+### `pe_sections(path)`
+Per-section entropy, size, executable flag, and a suspicious flag.
+
+## Usage
+
+```bash
+python scripts/analyst.py entropy sample.exe
+```
+
+## Sources
+
+- Shannon, "A Mathematical Theory of Communication" (entropy) —
+  https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
+- Microsoft PE/COFF specification (section characteristics) —
+  https://learn.microsoft.com/windows/win32/debug/pe-format
+- Entropy > ~7.0 in executable sections as a packing heuristic, established practice

vendor-new/analyst-ai-pack/skills/measuring-section-entropy-to-detect-packing/scripts/analyst.py

@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""Measure Shannon entropy of a file and (for PE) its sections to detect packing.
+
+Computes whole-file entropy and sliding-window entropy to locate high-entropy regions, and, when
+``pefile`` is available and the input is a PE, per-section entropy with raw/virtual size
+comparison. Higher entropy (toward 8.0 bits/byte) indicates compression/encryption. It reads the
+inert file and never executes it.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import sys
+from collections import Counter
+from pathlib import Path
+
+PACK_THRESHOLD = 7.0
+
+
+def shannon(data: bytes) -> float:
+    if not data:
+        return 0.0
+    counts = Counter(data)
+    n = len(data)
+    return -sum((c / n) * math.log2(c / n) for c in counts.values())
+
+
+def window_stats(data: bytes, window: int = 4096) -> tuple[float, float]:
+    """Return (peak entropy, fraction of windows above PACK_THRESHOLD)."""
+    if len(data) <= window:
+        ent = shannon(data)
+        return ent, 1.0 if ent > PACK_THRESHOLD else 0.0
+    peak = 0.0
+    high = 0
+    total = 0
+    for i in range(0, len(data) - window, window):
+        ent = shannon(data[i:i + window])
+        peak = max(peak, ent)
+        high += ent > PACK_THRESHOLD
+        total += 1
+    return peak, (high / total if total else 0.0)
+
+
+def pe_sections(path: str) -> list[dict]:
+    try:
+        import pefile  # type: ignore
+    except ImportError:
+        return []
+    out = []
+    pe = pefile.PE(path)
+    for sec in pe.sections:
+        name = sec.Name.rstrip(b"\x00").decode("latin-1")
+        raw = sec.get_data()
+        ent = shannon(raw)
+        executable = bool(sec.Characteristics & 0x20000000)  # MEM_EXECUTE
+        out.append({
+            "name": name,
+            "entropy": round(ent, 3),
+            "raw_size": sec.SizeOfRawData,
+            "virtual_size": sec.Misc_VirtualSize,
+            "executable": executable,
+            "suspicious": ent > PACK_THRESHOLD and executable,
+        })
+    return out
+
+
+def analyze(path: str) -> dict:
+    data = Path(path).read_bytes()
+    sections = pe_sections(path) if data[:2] == b"MZ" else []
+    peak, high_fraction = window_stats(data)
+    # Section-level evidence is authoritative; otherwise require a sustained
+    # high-entropy fraction (a few high-entropy windows are usually packed resources).
+    if sections:
+        packed = any(s["suspicious"] for s in sections)
+    else:
+        packed = high_fraction > 0.5
+    return {
+        "file_entropy": round(shannon(data), 3),
+        "window_peak_entropy": round(peak, 3),
+        "high_entropy_window_fraction": round(high_fraction, 3),
+        "sections": sections,
+        "likely_packed": packed,
+    }
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description="Measure entropy / detect packing")
+    sub = parser.add_subparsers(dest="cmd", required=True)
+    p = sub.add_parser("entropy")
+    p.add_argument("file")
+    args = parser.parse_args(argv)
+    print(json.dumps(analyze(args.file), indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())