reverse-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
Apache License 2.0 — see the repository root LICENSE file for the full text.
|
||||
|
||||
Copyright the AnalystAIPack contributors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
name: building-a-sample-management-workflow
|
||||
description: 'Establishes a disciplined malware sample repository: content-addressed storage by
|
||||
hash, encrypted/password-protected archiving, consistent metadata records, and chain-of-custody
|
||||
tracking so samples are reproducible and safe to handle. Activates for requests to organize a
|
||||
malware repository, manage samples, or track sample metadata and provenance.'
|
||||
domain: cybersecurity
|
||||
subdomain: lab-foundations
|
||||
tags:
|
||||
- lab-foundations
|
||||
- sample-management
|
||||
- storage
|
||||
- metadata
|
||||
- chain-of-custody
|
||||
version: 1.0.0
|
||||
author: analyst-ai-pack
|
||||
license: Apache-2.0
|
||||
mitre_attack:
|
||||
- T1587
|
||||
d3fend:
|
||||
- D3-FH
|
||||
references:
|
||||
- 'NIST SP 800-86 Guide to Integrating Forensic Techniques into Incident Response — https://csrc.nist.gov/pubs/sp/800/86/final'
|
||||
- 'MalwareBazaar sample conventions — https://bazaar.abuse.ch/'
|
||||
---
|
||||
|
||||
# Building a Sample Management Workflow
|
||||
|
||||
## When to Use
|
||||
|
||||
- You are accumulating malware samples and need a repeatable, safe way to store and find them.
|
||||
- You need metadata and provenance (source, date, case) attached to every sample.
|
||||
- You want content-addressed storage so the same sample is never duplicated or misnamed.
|
||||
|
||||
**Do not use** original filenames or loose folders as the organizing principle — names are
|
||||
attacker-controlled and collide; address samples by hash.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The hashing skill for content addressing; encrypted/password-protected archive tooling.
|
||||
- A defined storage location inside the isolated lab.
|
||||
|
||||
## Safety & Handling
|
||||
|
||||
- Store every sample inside a password-protected archive (commonly `infected`) so it cannot
|
||||
auto-execute or be scanned/quarantined by host AV.
|
||||
- Strip the executable bit / neutralize the extension at rest; restore only inside the lab.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Content-address on intake
|
||||
|
||||
On receipt, compute SHA-256 and store the sample under a path derived from its hash
|
||||
(e.g., `samples/ab/cd/<sha256>`), preventing duplicates and name collisions.
|
||||
|
||||
```bash
|
||||
python scripts/analyst.py intake sample.bin --source "phishing-case-42" --store ./samples
|
||||
```
|
||||
|
||||
### Step 2: Record metadata
|
||||
|
||||
Write a metadata record per sample: hashes, original name, source, intake date, case ID,
|
||||
file type, and analyst — kept next to the sample or in an index.
|
||||
|
||||
### Step 3: Archive safely
|
||||
|
||||
Wrap the sample in a password-protected archive; keep the inert copy out of host AV's reach and
|
||||
the metadata in plaintext for searchability.
|
||||
|
||||
### Step 4: Maintain chain of custody
|
||||
|
||||
Append-only log each access/action (who, when, what) so the sample's handling is auditable.
|
||||
|
||||
### Step 5: Index and search
|
||||
|
||||
Build a searchable index over metadata so samples can be found by hash, family, source, or case
|
||||
without touching the raw bytes.
|
||||
|
||||
## Validation
|
||||
|
||||
- The same sample always lands at the same hash-derived path (idempotent intake).
|
||||
- Every sample has a complete metadata record and an access log entry.
|
||||
- Samples are stored password-protected; metadata is searchable in plaintext.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Organizing by attacker-supplied filename, causing collisions and confusion.
|
||||
- Storing live samples unarchived where host AV deletes them mid-analysis.
|
||||
- Losing provenance, so an indicator cannot be traced back to its case.
|
||||
|
||||
## References
|
||||
|
||||
- See [`references/api-reference.md`](references/api-reference.md) for the intake/indexer.
|
||||
- NIST SP 800-86 and MalwareBazaar conventions (linked in frontmatter).
|
||||
@@ -0,0 +1,32 @@
|
||||
# API Reference: Building a Sample Management Workflow
|
||||
|
||||
## Overview
|
||||
|
||||
`scripts/analyst.py` performs idempotent content-addressed intake: it computes SHA-256, copies the
|
||||
sample to a hash-derived path, writes a JSON metadata record, and appends a chain-of-custody log
|
||||
entry. It reads inert bytes and never executes the sample.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| (stdlib) | — | hashlib, shutil, JSON, file IO |
|
||||
|
||||
## Core functions
|
||||
|
||||
### `intake(path, store, source, case, analyst)`
|
||||
Stores the sample at `<store>/<aa>/<bb>/<sha256>`, writes `<sha256>.meta.json`, and appends to
|
||||
`custody.log`. Returns the metadata record (with a `duplicate` flag).
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
python scripts/analyst.py intake sample.bin --source "phishing-case-42" --store ./samples
|
||||
```
|
||||
|
||||
## Sources
|
||||
|
||||
- NIST SP 800-86 Guide to Integrating Forensic Techniques into Incident Response —
|
||||
https://csrc.nist.gov/pubs/sp/800/86/final
|
||||
- MalwareBazaar sample conventions (hash-named, password-protected) — https://bazaar.abuse.ch/
|
||||
- Content-addressed storage and chain-of-custody, general DFIR practice
|
||||
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Intake a sample into a content-addressed store with metadata.
|
||||
|
||||
Computes SHA-256, copies the sample to a hash-derived path (``<store>/ab/cd/<sha256>``), writes
|
||||
a JSON metadata record, and appends a chain-of-custody log entry. Intake is idempotent: the same
|
||||
bytes always land at the same path. It reads the sample as inert bytes and never executes it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as _dt
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def intake(path: str, store: str, source: str, case: str, analyst: str) -> dict:
|
||||
data = Path(path).read_bytes()
|
||||
sha256 = hashlib.sha256(data).hexdigest()
|
||||
store_root = Path(store)
|
||||
dest_dir = store_root / sha256[:2] / sha256[2:4]
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
dest = dest_dir / sha256
|
||||
already = dest.exists()
|
||||
if not already:
|
||||
shutil.copy2(path, dest)
|
||||
|
||||
record = {
|
||||
"sha256": sha256,
|
||||
"md5": hashlib.md5(data).hexdigest(),
|
||||
"sha1": hashlib.sha1(data).hexdigest(),
|
||||
"size": len(data),
|
||||
"original_name": Path(path).name,
|
||||
"source": source,
|
||||
"case": case,
|
||||
"intake_utc": _dt.datetime.now(_dt.timezone.utc).isoformat(),
|
||||
"stored_path": str(dest),
|
||||
}
|
||||
(dest_dir / f"{sha256}.meta.json").write_text(json.dumps(record, indent=2),
|
||||
encoding="utf-8")
|
||||
|
||||
custody = store_root / "custody.log"
|
||||
entry = (f"{record['intake_utc']}\t{analyst}\tINTAKE\t{sha256}\t"
|
||||
f"source={source}\tcase={case}\n")
|
||||
with custody.open("a", encoding="utf-8") as fh:
|
||||
fh.write(entry)
|
||||
|
||||
record["duplicate"] = already
|
||||
return record
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description="Sample intake into content-addressed store")
|
||||
sub = parser.add_subparsers(dest="cmd", required=True)
|
||||
p = sub.add_parser("intake")
|
||||
p.add_argument("file")
|
||||
p.add_argument("--store", default="./samples")
|
||||
p.add_argument("--source", default="unknown")
|
||||
p.add_argument("--case", default="")
|
||||
p.add_argument("--analyst", default="analyst-ai-pack")
|
||||
args = parser.parse_args(argv)
|
||||
print(json.dumps(intake(args.file, args.store, args.source, args.case, args.analyst),
|
||||
indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user