reverse-skills-max: initial public release (restored, redacted, hardened)
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
Apache License 2.0 — see the repository root LICENSE file for the full text.
|
||||
|
||||
Copyright the AnalystAIPack contributors.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
@@ -0,0 +1,99 @@
|
||||
---
|
||||
name: scanning-samples-with-yara
|
||||
description: 'Uses YARA to classify and triage samples at scale: applying rule sets, reading
|
||||
matches and string offsets, tuning for false positives, and organizing rules for malware
|
||||
family identification. Activates for requests to scan files with YARA, apply YARA rules, or
|
||||
classify samples by signature.'
|
||||
domain: cybersecurity
|
||||
subdomain: malware-analysis
|
||||
tags:
|
||||
- malware
|
||||
- yara
|
||||
- signatures
|
||||
- classification
|
||||
- triage
|
||||
- detection
|
||||
version: 1.0.0
|
||||
author: analyst-ai-pack
|
||||
license: Apache-2.0
|
||||
mitre_attack:
|
||||
- T1027
|
||||
- T1587.001
|
||||
d3fend:
|
||||
- D3-FCR
|
||||
- D3-FA
|
||||
references:
|
||||
- 'YARA documentation — https://yara.readthedocs.io/'
|
||||
- 'yara-python — https://github.com/VirusTotal/yara-python'
|
||||
---
|
||||
|
||||
# Scanning Samples with YARA
|
||||
|
||||
## When to Use
|
||||
|
||||
- You want to classify one or many samples against known malware-family or capability rules.
|
||||
- You need to confirm a family hypothesis with a targeted rule and read which strings matched.
|
||||
- You are triaging a directory of files and want to flag the suspicious ones.
|
||||
|
||||
**Do not use** a YARA match as proof of family attribution on its own — public rules vary in
|
||||
quality and can over-match. Corroborate with behavior or code analysis.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `yara` CLI or `yara-python` (`pip install yara-python`).
|
||||
- A curated rule set (your own plus vetted public rules). Avoid blindly merging large noisy
|
||||
collections.
|
||||
- Samples in neutralized form inside the lab.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Organize rules
|
||||
|
||||
Keep rules in categories (family, capability, packer, anomaly) and compile them once for
|
||||
speed. Tag rules so matches are self-describing.
|
||||
|
||||
### Step 2: Scan and read matches
|
||||
|
||||
```bash
|
||||
python scripts/analyst.py scan rules/ sample.bin
|
||||
# prints matched rule names, tags, and matched strings with offsets
|
||||
```
|
||||
|
||||
The matched **string offsets** matter: they tell you where in the file the signal is (header,
|
||||
overlay, resource), which guides deeper analysis.
|
||||
|
||||
### Step 3: Triage a directory
|
||||
|
||||
```bash
|
||||
python scripts/analyst.py scan rules/ ./samples --recursive --summary
|
||||
```
|
||||
|
||||
Rank files by number/severity of matches to prioritize analyst time.
|
||||
|
||||
### Step 4: Tune for false positives
|
||||
|
||||
If a rule fires on benign files, tighten it: require multiple strings (`2 of ($a*)`),
|
||||
anchor to file structure (`uint16(0) == 0x5A4D`), or raise the condition specificity.
|
||||
|
||||
### Step 5: Promote good signals to detection
|
||||
|
||||
A rule that reliably identifies a family or capability becomes a hunting/detection artifact —
|
||||
hand it to the detection-engineering workflow.
|
||||
|
||||
## Validation
|
||||
|
||||
- Matches reproduce across runs and tools (CLI and `yara-python` agree).
|
||||
- Each kept rule has an acceptable false-positive rate against a known-clean corpus.
|
||||
- Match offsets correspond to meaningful file regions, not incidental byte coincidences.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Merging huge public rule packs without curation — noise and false positives explode.
|
||||
- Writing rules on packed bytes that change with every build; prefer unpacked or structural
|
||||
features.
|
||||
- Over-broad strings (common library text) that match unrelated software.
|
||||
|
||||
## References
|
||||
|
||||
- See [`references/api-reference.md`](references/api-reference.md) for the scanner tool.
|
||||
- YARA and yara-python documentation (linked in frontmatter).
|
||||
@@ -0,0 +1,44 @@
|
||||
# API Reference: Scanning Samples with YARA
|
||||
|
||||
## Overview
|
||||
|
||||
`scripts/analyst.py` compiles a YARA rule file or directory and scans a file or directory,
|
||||
reporting matched rules, tags, and matched-string offsets. A `--summary` mode ranks files
|
||||
with matches for triage.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| yara-python | >= 4.3 | Compile and match YARA rules |
|
||||
|
||||
## External tools
|
||||
|
||||
| Tool | Purpose |
|
||||
|------|---------|
|
||||
| yara (CLI) | Cross-check matches outside Python |
|
||||
|
||||
## Core functions
|
||||
|
||||
### `compile_rules(rules_path)`
|
||||
Compiles a single rule file, or all `.yar`/`.yara` files under a directory (namespaced by
|
||||
filename stem).
|
||||
|
||||
### `scan_file(rules, filepath)`
|
||||
Returns matched rules with tags and per-string offsets for one file.
|
||||
|
||||
### `scan_path(rules_path, target, recursive)`
|
||||
Scans a file or directory (optionally recursive) and returns per-file results.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
pip install yara-python
|
||||
python scripts/analyst.py rules/ sample.bin
|
||||
python scripts/analyst.py rules/ ./samples --recursive --summary
|
||||
```
|
||||
|
||||
## Sources
|
||||
|
||||
- YARA documentation — https://yara.readthedocs.io/
|
||||
- yara-python — https://github.com/VirusTotal/yara-python
|
||||
@@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Scan files and directories with YARA rules and summarize matches.
|
||||
|
||||
Wraps ``yara-python`` to compile a rules path and report matched rule names, tags, and string
|
||||
offsets. Reads files as data; it never executes a sample.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def compile_rules(rules_path: str):
|
||||
import yara # type: ignore
|
||||
p = Path(rules_path)
|
||||
if p.is_dir():
|
||||
filepaths = {f.stem: str(f) for f in p.rglob("*.yar")}
|
||||
filepaths.update({f.stem: str(f) for f in p.rglob("*.yara")})
|
||||
if not filepaths:
|
||||
raise FileNotFoundError(f"no .yar/.yara files under {rules_path}")
|
||||
return yara.compile(filepaths=filepaths)
|
||||
return yara.compile(filepath=str(p))
|
||||
|
||||
|
||||
def scan_file(rules, filepath: str) -> dict:
|
||||
matches = rules.match(filepath, timeout=60)
|
||||
return {
|
||||
"file": filepath,
|
||||
"matches": [{
|
||||
"rule": m.rule,
|
||||
"tags": list(m.tags),
|
||||
"strings": [
|
||||
{"id": s.identifier, "offset": inst.offset}
|
||||
for s in m.strings for inst in s.instances
|
||||
] if hasattr(m, "strings") else [],
|
||||
} for m in matches],
|
||||
}
|
||||
|
||||
|
||||
def scan_path(rules_path: str, target: str, recursive: bool) -> list[dict]:
|
||||
rules = compile_rules(rules_path)
|
||||
target_p = Path(target)
|
||||
files = ([f for f in target_p.rglob("*") if f.is_file()] if recursive and target_p.is_dir()
|
||||
else [f for f in target_p.iterdir() if f.is_file()] if target_p.is_dir()
|
||||
else [target_p])
|
||||
return [scan_file(rules, str(f)) for f in files]
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description="Scan with YARA")
|
||||
parser.add_argument("rules", help="rule file or directory")
|
||||
parser.add_argument("target", help="file or directory to scan")
|
||||
parser.add_argument("--recursive", action="store_true")
|
||||
parser.add_argument("--summary", action="store_true")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
try:
|
||||
results = scan_path(args.rules, args.target, args.recursive)
|
||||
except ImportError:
|
||||
print("yara-python not installed: pip install yara-python", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
if args.summary:
|
||||
hit = [r for r in results if r["matches"]]
|
||||
print(json.dumps({
|
||||
"scanned": len(results),
|
||||
"with_matches": len(hit),
|
||||
"files": [{"file": r["file"],
|
||||
"rules": [m["rule"] for m in r["matches"]]} for r in hit],
|
||||
}, indent=2))
|
||||
else:
|
||||
print(json.dumps(results, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user