reverse-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
baradok609hecl1pq6qi
@@ -0,0 +1,9 @@
|
||||
Apache License 2.0 — see the repository root LICENSE file for the full text.
|
||||
|
||||
Copyright the AnalystAIPack contributors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
@@ -0,0 +1,119 @@
|
||||
---
|
||||
name: triaging-an-unknown-sample
|
||||
description: 'Performs fast first-pass triage of an unknown file to decide depth of analysis:
|
||||
file type identification, hashing, reputation lookup, packing/entropy check, and string
|
||||
review. Activates for requests to triage, classify, or do initial assessment of an unknown
|
||||
or suspicious file.'
|
||||
domain: cybersecurity
|
||||
subdomain: lab-foundations
|
||||
tags:
|
||||
- malware
|
||||
- triage
|
||||
- file-identification
|
||||
- entropy
|
||||
- strings
|
||||
- reputation
|
||||
version: 1.0.0
|
||||
author: analyst-ai-pack
|
||||
license: Apache-2.0
|
||||
mitre_attack:
|
||||
- T1027
|
||||
- T1140
|
||||
d3fend:
|
||||
- D3-FA
|
||||
- D3-FCR
|
||||
references:
|
||||
- 'file/libmagic — https://man7.org/linux/man-pages/man1/file.1.html'
|
||||
- 'VirusTotal API documentation — https://docs.virustotal.com/reference/overview'
|
||||
---
|
||||
|
||||
# Triaging an Unknown Sample
|
||||
|
||||
## When to Use
|
||||
|
||||
- A suspicious file has arrived and you need to quickly decide whether it warrants deep
|
||||
static/dynamic analysis or can be dismissed.
|
||||
- You want a structured first-pass record (type, hashes, packing, notable strings) before
|
||||
committing analyst time.
|
||||
- You are batch-triaging many files and need to rank them by suspicion.
|
||||
|
||||
**Do not use** triage results as a final verdict. A clean string review or unknown reputation
|
||||
does not mean benign; it means proceed to deeper analysis.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `file` (libmagic) for type identification, or the bundled magic-based detection.
|
||||
- Optional VirusTotal API key for reputation (hash lookups only — never upload someone
|
||||
else's data without authorization).
|
||||
- The sample in a neutralized, non-executable form inside the lab.
|
||||
|
||||
## Safety & Handling
|
||||
|
||||
- Triage is **static**: read bytes, never execute. Keep the sample with a neutralized
|
||||
extension.
|
||||
- Submit only the **hash** to reputation services unless you have authorization to upload the
|
||||
file itself; uploads can expose sensitive or attributable data.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Identify the true file type
|
||||
|
||||
Do not trust the extension. Identify by magic bytes:
|
||||
|
||||
```bash
|
||||
python scripts/analyst.py triage sample.bin
|
||||
# reports: detected type, magic, hashes, entropy, suspicious strings
|
||||
```
|
||||
|
||||
A `.pdf` that is really a PE, or a `.jpg` that is a script, is itself a finding.
|
||||
|
||||
### Step 2: Hash and check reputation
|
||||
|
||||
Compute the SHA-256 and look it up (hash-only) to see if it is known:
|
||||
|
||||
```bash
|
||||
python scripts/analyst.py reputation --sha256 <sha256> --vt-key $VT_API_KEY
|
||||
```
|
||||
|
||||
Known-bad with many detections → escalate. Unknown → continue triage; absence of detections
|
||||
is not exoneration.
|
||||
|
||||
### Step 3: Estimate packing via entropy
|
||||
|
||||
High, uniform entropy across the whole file or a code section suggests packing or encryption:
|
||||
|
||||
```text
|
||||
entropy ~7.9 / 8.0 over most of the file -> likely packed/encrypted
|
||||
mixed entropy with readable strings -> likely unpacked
|
||||
```
|
||||
|
||||
### Step 4: Skim strings for quick signal
|
||||
|
||||
Look for URLs, IPs, registry paths, suspicious API names, and embedded PE headers (`MZ`).
|
||||
The script extracts and categorizes printable strings.
|
||||
|
||||
### Step 5: Decide and record
|
||||
|
||||
Classify as: dismiss, deep static, or detonate. Emit a triage record for the case file.
|
||||
|
||||
## Validation
|
||||
|
||||
- The detected type matches the file's structure (e.g. PE confirmed by `MZ`/`PE\0\0`).
|
||||
- Entropy reading is consistent with the strings observed (packed files yield few readable
|
||||
strings).
|
||||
- The triage record captures type, hashes, entropy, reputation, and the escalation decision.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Equating "0 detections on VirusTotal" with benign — fresh or targeted malware is often
|
||||
unknown.
|
||||
- Treating high entropy alone as proof of malice; installers and compressed media are also
|
||||
high-entropy. Combine signals.
|
||||
- Uploading samples to third parties without authorization, leaking sensitive data.
|
||||
- Stopping at triage for a file that is clearly suspicious but "not yet detected."
|
||||
|
||||
## References
|
||||
|
||||
- See [`references/api-reference.md`](references/api-reference.md) for the triage tool and
|
||||
scoring logic.
|
||||
- libmagic and VirusTotal API documentation (linked in frontmatter).
|
||||
@@ -0,0 +1,48 @@
|
||||
# API Reference: Triaging an Unknown Sample
|
||||
|
||||
## Overview
|
||||
|
||||
`scripts/analyst.py` performs static first-pass triage: magic-byte type identification,
|
||||
hashing, Shannon entropy (packing signal), categorized strings, and a simple suspicion
|
||||
score with an escalation recommendation. An optional VirusTotal hash lookup adds reputation.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| (stdlib) | — | Hashing, entropy, regex, optional `urllib` for VT |
|
||||
|
||||
## External tools
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| file / libmagic | Authoritative type identification beyond the built-in magic set |
|
||||
| VirusTotal API | Hash reputation (lookup only; no upload) |
|
||||
|
||||
## Core functions
|
||||
|
||||
### `triage(filepath)`
|
||||
Returns type, size, sha256, entropy, `likely_packed`, `embedded_pe`, suspicious strings,
|
||||
a `triage_score`, and a `recommendation` (`low-signal` / `review` / `deep-analysis`).
|
||||
|
||||
### `shannon_entropy(data)`
|
||||
Byte-level Shannon entropy in bits (0–8). Values above ~7.2 suggest packing/encryption.
|
||||
|
||||
### `identify(data)`
|
||||
Magic-byte file-type label.
|
||||
|
||||
### `reputation(sha256, vt_key)`
|
||||
Looks up a hash on VirusTotal v3 and returns `last_analysis_stats`. Never uploads the file.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python scripts/analyst.py triage sample.bin
|
||||
python scripts/analyst.py reputation --sha256 <sha256> --vt-key "$VT_API_KEY"
|
||||
```
|
||||
|
||||
## Sources
|
||||
|
||||
- file(1) / libmagic manual — https://man7.org/linux/man-pages/man1/file.1.html
|
||||
- VirusTotal API v3 reference — https://docs.virustotal.com/reference/overview
|
||||
- Shannon entropy as a packing indicator (information theory; standard practice)
|
||||
@@ -0,0 +1,121 @@
|
||||
#!/usr/bin/env python3
|
||||
"""First-pass triage for an unknown file: type, hashes, entropy, and strings.
|
||||
|
||||
Pure static analysis on file bytes. Optionally checks a hash against VirusTotal if a key is
|
||||
provided (hash lookup only; it never uploads the file). It never executes the sample.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import sys
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
# magic-byte signatures -> label
|
||||
_MAGIC = [
|
||||
(b"MZ", "PE/DOS executable"),
|
||||
(b"\x7fELF", "ELF executable"),
|
||||
(b"\xca\xfe\xba\xbe", "Mach-O (fat) / Java class"),
|
||||
(b"\xfe\xed\xfa", "Mach-O"),
|
||||
(b"%PDF", "PDF document"),
|
||||
(b"PK\x03\x04", "ZIP/OOXML/JAR archive"),
|
||||
(b"\xd0\xcf\x11\xe0", "OLE2 (legacy Office)"),
|
||||
(b"Rar!", "RAR archive"),
|
||||
(b"\x1f\x8b", "gzip"),
|
||||
]
|
||||
|
||||
_SUSPICIOUS = re.compile(
|
||||
rb"(https?://|cmd\.exe|powershell|VirtualAlloc|CreateRemoteThread|WScript|"
|
||||
rb"RegSetValue|URLDownloadToFile|\\\\\.\\pipe\\|-enc |FromBase64String)",
|
||||
re.I,
|
||||
)
|
||||
|
||||
|
||||
def shannon_entropy(data: bytes) -> float:
|
||||
if not data:
|
||||
return 0.0
|
||||
counts = Counter(data)
|
||||
length = len(data)
|
||||
return -sum((c / length) * math.log2(c / length) for c in counts.values())
|
||||
|
||||
|
||||
def identify(data: bytes) -> str:
|
||||
for magic, label in _MAGIC:
|
||||
if data.startswith(magic):
|
||||
return label
|
||||
return "unknown"
|
||||
|
||||
|
||||
def extract_strings(data: bytes, minlen: int = 5) -> list[str]:
|
||||
found = re.findall(rb"[\x20-\x7e]{%d,}" % minlen, data)
|
||||
return [s.decode("ascii", "replace") for s in found]
|
||||
|
||||
|
||||
def suspicious_strings(data: bytes) -> list[str]:
|
||||
return sorted({m.group(0).decode("ascii", "replace") for m in _SUSPICIOUS.finditer(data)})
|
||||
|
||||
|
||||
def triage(filepath: str | Path) -> dict:
|
||||
data = Path(filepath).read_bytes()
|
||||
ent = shannon_entropy(data)
|
||||
sus = suspicious_strings(data)
|
||||
embedded_pe = data.count(b"MZ") > 1
|
||||
score = 0
|
||||
score += 2 if ent > 7.2 else 0
|
||||
score += 2 if sus else 0
|
||||
score += 2 if embedded_pe else 0
|
||||
verdict = "deep-analysis" if score >= 4 else "review" if score >= 2 else "low-signal"
|
||||
return {
|
||||
"file": str(filepath),
|
||||
"type": identify(data),
|
||||
"size": len(data),
|
||||
"sha256": hashlib.sha256(data).hexdigest(),
|
||||
"entropy": round(ent, 3),
|
||||
"likely_packed": ent > 7.2,
|
||||
"embedded_pe": embedded_pe,
|
||||
"suspicious_strings": sus,
|
||||
"triage_score": score,
|
||||
"recommendation": verdict,
|
||||
}
|
||||
|
||||
|
||||
def reputation(sha256: str, vt_key: str) -> dict:
|
||||
try:
|
||||
import urllib.request
|
||||
req = urllib.request.Request(
|
||||
f"https://www.virustotal.com/api/v3/files/{sha256}",
|
||||
headers={"x-apikey": vt_key},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=20) as resp: # noqa: S310
|
||||
body = json.loads(resp.read())
|
||||
stats = body["data"]["attributes"]["last_analysis_stats"]
|
||||
return {"sha256": sha256, "found": True, "stats": stats}
|
||||
except Exception as exc: # noqa: BLE001 - report, do not crash triage
|
||||
return {"sha256": sha256, "found": False, "error": str(exc)}
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description="Triage an unknown sample")
|
||||
sub = parser.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
p_tri = sub.add_parser("triage", help="static triage of a file")
|
||||
p_tri.add_argument("file")
|
||||
|
||||
p_rep = sub.add_parser("reputation", help="VirusTotal hash lookup")
|
||||
p_rep.add_argument("--sha256", required=True)
|
||||
p_rep.add_argument("--vt-key", required=True)
|
||||
|
||||
args = parser.parse_args(argv)
|
||||
if args.cmd == "triage":
|
||||
print(json.dumps(triage(args.file), indent=2))
|
||||
elif args.cmd == "reputation":
|
||||
print(json.dumps(reputation(args.sha256, args.vt_key), indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user