reverse-skills-max: initial public release (restored, redacted, hardened)

vendor-new/analyst-ai-pack/skills/hashing-and-fingerprinting-files/LICENSE

@@ -0,0 +1,9 @@
+Apache License 2.0 — see the repository root LICENSE file for the full text.
+
+Copyright the AnalystAIPack contributors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0

vendor-new/analyst-ai-pack/skills/hashing-and-fingerprinting-files/SKILL.md

@@ -0,0 +1,90 @@
+---
+name: hashing-and-fingerprinting-files
+description: 'Computes cryptographic and fuzzy fingerprints for malware samples: MD5/SHA-1/SHA-256
+  for exact identity, plus import hashing (imphash), section hashing, and fuzzy hashes (ssdeep,
+  TLSH) to cluster related variants. Activates for requests to hash a sample, compute imphash,
+  or fingerprint files for similarity clustering.'
+domain: cybersecurity
+subdomain: lab-foundations
+tags:
+  - lab-foundations
+  - hashing
+  - fingerprinting
+  - imphash
+  - fuzzy-hashing
+  - clustering
+version: 1.0.0
+author: analyst-ai-pack
+license: Apache-2.0
+mitre_attack:
+  - T1027
+d3fend:
+  - D3-FH
+references:
+  - 'ssdeep fuzzy hashing — https://ssdeep-project.github.io/ssdeep/'
+  - 'TLSH locality-sensitive hashing — https://github.com/trendmicro/tlsh'
+---
+
+# Hashing and Fingerprinting Files
+
+## When to Use
+
+- You need a stable identity for a sample to look up, deduplicate, or share as an IOC.
+- You want to cluster related variants that differ slightly using fuzzy/similarity hashes.
+- You are computing imphash or section hashes to pivot across a malware family.
+
+**Do not use** cryptographic hashes alone to judge similarity — a single byte change yields a
+completely different SHA-256; use fuzzy/import hashing for relatedness.
+
+## Prerequisites
+
+- Python with `hashlib` (stdlib); optionally `ssdeep`/`python-tlsh` for fuzzy hashes and
+  `pefile` for imphash.
+- Samples handled in the isolated lab per the safe-handling skill.
+
+## Safety & Handling
+
+- Treat every sample as live: never execute it during hashing; operate on the inert file only.
+- Store samples encrypted/password-protected and reference them by hash, not by original name.
+
+## Workflow
+
+### Step 1: Compute cryptographic hashes
+
+Generate MD5, SHA-1, and SHA-256. SHA-256 is the canonical identity for sharing; MD5/SHA-1 aid
+lookups in legacy feeds.
+
+```bash
+python scripts/analyst.py hash sample.bin
+```
+
+### Step 2: Compute structural hashes (PE)
+
+For PE files, compute imphash (hash of the import table) and per-section hashes to pivot across
+samples built from the same toolchain.
+
+### Step 3: Compute fuzzy/similarity hashes
+
+Generate ssdeep and/or TLSH digests so near-duplicates can be matched even when bytes differ.
+
+### Step 4: Record and cross-reference
+
+Store all digests with the sample metadata; query threat-intel feeds by SHA-256 and cluster by
+imphash/fuzzy hash.
+
+## Validation
+
+- The same input always yields identical cryptographic hashes (deterministic).
+- Imphash matches across known-related samples; fuzzy hashes score high similarity for variants.
+- Digests are recorded alongside sample metadata for later pivoting.
+
+## Pitfalls
+
+- Treating MD5 collisions as identity proof; use SHA-256 as canonical.
+- Assuming different cryptographic hashes mean unrelated files; check fuzzy/import hashes.
+- Computing imphash on packed samples (imports are stubbed) and over-trusting the result.
+
+## References
+
+- See [`references/api-reference.md`](references/api-reference.md) for the hashing helper.
+- ssdeep and TLSH (linked in frontmatter).

vendor-new/analyst-ai-pack/skills/hashing-and-fingerprinting-files/references/api-reference.md

@@ -0,0 +1,40 @@
+# API Reference: Hashing and Fingerprinting Files
+
+## Overview
+
+`scripts/analyst.py` computes MD5/SHA-1/SHA-256 with stdlib `hashlib`, adds PE imphash and
+per-section SHA-256 when `pefile` is present, and includes ssdeep/TLSH fuzzy hashes when their
+libraries are installed. It reads inert bytes and never executes the sample.
+
+## Dependencies
+
+| Package | Version | Purpose |
+|---------|---------|---------|
+| (stdlib) | — | hashlib cryptographic hashes |
+| pefile | >= 2023.2.7 | imphash and section hashing (optional) |
+| ssdeep | >= 3.4 | Fuzzy/context-triggered piecewise hashing (optional) |
+| python-tlsh | >= 4.5 | Locality-sensitive similarity hashing (optional) |
+
+## Core functions
+
+### `crypto_hashes(data)`
+Returns MD5, SHA-1, SHA-256, and size.
+
+### `pe_hashes(path)`
+Returns imphash and per-section SHA-256 for PE files.
+
+### `fuzzy_hashes(data)`
+Returns ssdeep/TLSH digests when available.
+
+## Usage
+
+```bash
+python scripts/analyst.py hash sample.bin
+```
+
+## Sources
+
+- ssdeep fuzzy hashing — https://ssdeep-project.github.io/ssdeep/
+- TLSH locality-sensitive hashing — https://github.com/trendmicro/tlsh
+- imphash concept (import-table hash for toolchain clustering) — Mandiant/FireEye research
+- SHA-256 as canonical sample identity, general malware-handling practice

vendor-new/analyst-ai-pack/skills/hashing-and-fingerprinting-files/scripts/analyst.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""Compute cryptographic and structural fingerprints for a file.
+
+Always computes MD5/SHA-1/SHA-256 with stdlib ``hashlib``. If ``pefile`` is available and the
+input is a PE, also computes imphash and per-section SHA-256. Optional fuzzy hashes (ssdeep,
+TLSH) are included when their libraries are installed. It reads the file as inert bytes and
+never executes it.
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import sys
+from pathlib import Path
+
+
+def crypto_hashes(data: bytes) -> dict:
+    return {
+        "md5": hashlib.md5(data).hexdigest(),
+        "sha1": hashlib.sha1(data).hexdigest(),
+        "sha256": hashlib.sha256(data).hexdigest(),
+        "size": len(data),
+    }
+
+
+def pe_hashes(path: str) -> dict:
+    try:
+        import pefile  # type: ignore
+    except ImportError:
+        return {}
+    try:
+        pe = pefile.PE(path, fast_load=True)
+        pe.parse_data_directories(
+            directories=[pefile.DIRECTORY_ENTRY["IMAGE_DIRECTORY_ENTRY_IMPORT"]])
+        out = {"imphash": pe.get_imphash() or None}
+        sections = {}
+        for sec in pe.sections:
+            name = sec.Name.rstrip(b"\x00").decode("latin-1")
+            sections[name] = hashlib.sha256(sec.get_data()).hexdigest()
+        out["section_sha256"] = sections
+        return out
+    except Exception as exc:  # noqa: BLE001 - pefile raises many types
+        return {"pe_error": str(exc)}
+
+
+def fuzzy_hashes(data: bytes) -> dict:
+    out = {}
+    try:
+        import ssdeep  # type: ignore
+        out["ssdeep"] = ssdeep.hash(data)
+    except ImportError:
+        pass
+    try:
+        import tlsh  # type: ignore
+        out["tlsh"] = tlsh.hash(data)
+    except ImportError:
+        pass
+    return out
+
+
+def fingerprint(path: str) -> dict:
+    data = Path(path).read_bytes()
+    result = crypto_hashes(data)
+    is_pe = data[:2] == b"MZ"
+    if is_pe:
+        result.update(pe_hashes(path))
+    result.update(fuzzy_hashes(data))
+    return result
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description="Fingerprint a file")
+    sub = parser.add_subparsers(dest="cmd", required=True)
+    p = sub.add_parser("hash")
+    p.add_argument("file")
+    args = parser.parse_args(argv)
+    print(json.dumps(fingerprint(args.file), indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())