Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 16 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,34 +38,31 @@

## 📚 Documentation

DeepZero features extensive, exhaustive documentation covering architecture, pipeline schemas, CLI references, and custom processor development.
The documentation covers architecture, pipeline schemas, CLI references, and custom processor development.

👉 **[Read the Official Documentation here](https://blog.ahmadz.ai/DeepZero/)**

---

## ⚡️ Quickstart

DeepZero requires a target corpus of files to analyze and a pipeline configuration detailing how to process them.
Try a complete local run with the included text samples. **No API keys, Ghidra, or driver corpus needed.** Requires Python 3.11+.

1. **Clone & Install (Python 3.11+)**
```bash
git clone https://github.com/416rehman/DeepZero.git
cd DeepZero
pip install -e .
```
```sh
git clone https://github.com/416rehman/DeepZero.git
cd DeepZero
python -m pip install -e .
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

These commands work in PowerShell and POSIX shells. For an isolated installation, create and activate a Python virtual environment before installing.

2. **Configure Environment**
```bash
cp .env.example .env
```
The demo discovers two harmless text files, keeps one, filters the smaller one, and generates a browsable HTML report. Run the same pipeline command again to resume from saved state. This demonstrates the engine; it does not run vulnerability analysis. See the [demo walkthrough](pipelines/demo/README.md) for expected results and configuration experiments.

3. **Run a Pipeline**
```bash
deepzero run C:\drivers -p .\pipelines\loldrivers\pipeline.yaml
```
For the driver analysis pipeline, follow the [full setup guide](https://blog.ahmadz.ai/DeepZero/en/overview/quickstart.html) and [pipeline prerequisites](https://blog.ahmadz.ai/DeepZero/en/reference/included-pipeline.html). Optional integrations require their own dependencies and configuration.

For detailed setup instructions and example corpora, see the [Quickstart Documentation](https://blog.ahmadz.ai/DeepZero/en/overview/quickstart.html).
If DeepZero is useful to your work, **star this repository** to help others discover it. Feedback on your first run is welcome in the [issue tracker](https://github.com/416rehman/DeepZero/issues).

---

Expand All @@ -84,6 +81,7 @@ processors/ # external processors (shipped as examples)
└── semgrep_scanner/ # semgrep batch scanner (BulkMapProcessor)

pipelines/
├── demo/ # local first run with harmless text files; no API keys
└── loldrivers/ # BYOVD kernel driver vulnerability research pipeline
├── pipeline.yaml
├── assessment.j2 # LLM prompt template
Expand All @@ -97,7 +95,7 @@ tests/ # pytest suite

## 🤝 Contributing

CI runs on Python 3.11 and 3.12 via GitHub Actions.
CI runs on Python 3.11, 3.12, 3.13, and 3.14 via GitHub Actions.

Run linting and security checks before submitting:

Expand Down
13 changes: 13 additions & 0 deletions docs/en/overview/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ order: 1

DeepZero requires a target corpus of files to analyze and a [pipeline configuration]({{ '/reference/pipeline-yaml.html' | relative_url }}) detailing how to process them. We provide a [complete example pipeline]({{ '/reference/included-pipeline.html' | relative_url }}) designed to hunt for new BYOVD (Bring Your Own Vulnerable Driver) candidates across raw binary datasets (e.g., the Snappy Driver Installer corpus) by explicitly filtering out known hashes using the [LOLDrivers project](https://www.loldrivers.io/).

### Try the local demo first

After installing the base package, run the included harmless text samples from the repository root:

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero status -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

The demo needs no API keys, external tools, or driver corpus. It discovers two files, keeps `hello.txt`, filters `tiny.txt` by size, and writes an HTML report. It demonstrates orchestration and saved state, not vulnerability detection. Repeating the run command resumes saved results. See the [demo walkthrough](https://github.com/416rehman/DeepZero/tree/main/pipelines/demo) for details.


### 1. Installation

DeepZero requires **Python 3.11+**.
Expand Down
13 changes: 13 additions & 0 deletions docs/es/overview/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ order: 1

DeepZero requiere un corpus de archivos de destino para analizar y una [configuración de canalización]({{ '/es/reference/pipeline-yaml.html' | relative_url }}) que detalla cómo procesarlos. Proporcionamos una [canalización de ejemplo completa]({{ '/es/reference/included-pipeline.html' | relative_url }}) diseñada para buscar nuevos candidatos BYOVD en conjuntos de datos binarios sin procesar.

### Try the local demo first

After installing the base package, run the included harmless text samples from the repository root:

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero status -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

The demo needs no API keys, external tools, or driver corpus. It discovers two files, keeps `hello.txt`, filters `tiny.txt` by size, and writes an HTML report. It demonstrates orchestration and saved state, not vulnerability detection. Repeating the run command resumes saved results. See the [demo walkthrough](https://github.com/416rehman/DeepZero/tree/main/pipelines/demo) for details.


### 1. Instalación

DeepZero requiere **Python 3.11+**.
Expand Down
13 changes: 13 additions & 0 deletions docs/fr/overview/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ order: 1

DeepZero nécessite un corpus cible de fichiers à analyser et une [configuration de pipeline]({{ '/fr/reference/pipeline-yaml.html' | relative_url }}) détaillant la manière de les traiter. Nous fournissons un [pipeline complet d'exemple]({{ '/fr/reference/included-pipeline.html' | relative_url }}) conçu pour rechercher de nouveaux candidats BYOVD (Bring Your Own Vulnerable Driver) dans des ensembles de données binaires bruts (par exemple, le corpus Snappy Driver Installer) en filtrant explicitement les hachages connus à l'aide du [projet LOLDrivers](https://www.loldrivers.io/).

### Try the local demo first

After installing the base package, run the included harmless text samples from the repository root:

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero status -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

The demo needs no API keys, external tools, or driver corpus. It discovers two files, keeps `hello.txt`, filters `tiny.txt` by size, and writes an HTML report. It demonstrates orchestration and saved state, not vulnerability detection. Repeating the run command resumes saved results. See the [demo walkthrough](https://github.com/416rehman/DeepZero/tree/main/pipelines/demo) for details.


### 1. Installation

DeepZero nécessite **Python 3.11+**.
Expand Down
13 changes: 13 additions & 0 deletions docs/ja/overview/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ order: 1

DeepZero は、分析するためのターゲットとなるファイルコーパスと、その処理方法を詳述する[パイプライン設定]({{ '/ja/reference/pipeline-yaml.html' | relative_url }})を必要とします。[LOLDrivers](https://www.loldrivers.io/) プロジェクトからの既知のハッシュをフィルタリングすることにより、未加工のバイナリデータセットの中から新しい脆弱なドライバー(BYOVD)候補を見つけるように設計された[完全なサンプルパイプライン]({{ '/ja/reference/included-pipeline.html' | relative_url }})を提供しています。

### Try the local demo first

After installing the base package, run the included harmless text samples from the repository root:

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero status -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

The demo needs no API keys, external tools, or driver corpus. It discovers two files, keeps `hello.txt`, filters `tiny.txt` by size, and writes an HTML report. It demonstrates orchestration and saved state, not vulnerability detection. Repeating the run command resumes saved results. See the [demo walkthrough](https://github.com/416rehman/DeepZero/tree/main/pipelines/demo) for details.


### 1. インストール

DeepZero には **Python 3.11+** が必要です。
Expand Down
13 changes: 13 additions & 0 deletions docs/ru/overview/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ order: 1

DeepZero требует целевого корпуса файлов для анализа и [конфигурации конвейера]({{ '/ru/reference/pipeline-yaml.html' | relative_url }}), детализирующей их обработку. Мы предоставляем [полный пример конвейера]({{ '/ru/reference/included-pipeline.html' | relative_url }}), предназначенный для поиска новых уязвимых драйверов (BYOVD) в необработанных бинарных наборах данных, путем фильтрации известных хешей с использованием проекта [LOLDrivers](https://www.loldrivers.io/).

### Try the local demo first

After installing the base package, run the included harmless text samples from the repository root:

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero status -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

The demo needs no API keys, external tools, or driver corpus. It discovers two files, keeps `hello.txt`, filters `tiny.txt` by size, and writes an HTML report. It demonstrates orchestration and saved state, not vulnerability detection. Repeating the run command resumes saved results. See the [demo walkthrough](https://github.com/416rehman/DeepZero/tree/main/pipelines/demo) for details.


### 1. Установка

DeepZero требует **Python 3.11+**.
Expand Down
13 changes: 13 additions & 0 deletions docs/zh-CN/overview/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,19 @@ order: 1

DeepZero 需要一个待分析的目标文件语料库,以及一个详细说明如何处理这些文件的 [流水线配置]({{ '/zh-CN/reference/pipeline-yaml.html' | relative_url }})。我们提供了一个 [完整的示例流水线]({{ '/zh-CN/reference/included-pipeline.html' | relative_url }}),它旨在通过使用 [LOLDrivers 项目](https://www.loldrivers.io/) 显式过滤已知的哈希值,从而在原始二进制数据集 (例如,Snappy Driver Installer 语料库) 中寻找新的 BYOVD (自带易受攻击驱动程序) 候选目标。

### Try the local demo first

After installing the base package, run the included harmless text samples from the repository root:

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero status -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

The demo needs no API keys, external tools, or driver corpus. It discovers two files, keeps `hello.txt`, filters `tiny.txt` by size, and writes an HTML report. It demonstrates orchestration and saved state, not vulnerability detection. Repeating the run command resumes saved results. See the [demo walkthrough](https://github.com/416rehman/DeepZero/tree/main/pipelines/demo) for details.


### 1. 安装

DeepZero 要求使用 **Python 3.11+**。
Expand Down
33 changes: 33 additions & 0 deletions pipelines/demo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Try DeepZero locally

This example demonstrates pipeline execution, metadata filtering, persisted state,
and HTML reports using two harmless text files. It does not perform vulnerability
analysis or call an LLM. After installing DeepZero's base dependencies, the demo
needs no network access, API keys, Ghidra, or driver corpus.

Run these commands from the repository root (PowerShell or a POSIX shell):

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero status -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open
```

The `discover` stage finds two files and records their SHA-256 hashes and sizes.
The `keep_larger_files` stage keeps `hello.txt` and filters out `tiny.txt` because
it contains fewer than 32 bytes. A filtered file is an expected outcome, not an
error. The generated report shows the samples and their stage results; it does
not establish whether a file is safe or vulnerable.

Run the first command again to resume the saved run. DeepZero keeps state and
reports under `work/demo/<corpus-key>/`; the CLI prints the report location.

To explore the YAML configuration, change `min_size_bytes` and run with a new
work directory so previous results do not get reused:

```sh
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml -w work/demo-experiment
```

For real analysis prerequisites and pipeline configuration, see the
[documentation](https://blog.ahmadz.ai/DeepZero/en/overview/quickstart.html).
13 changes: 13 additions & 0 deletions pipelines/demo/pipeline.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: demo
description: Local file discovery and filtering demo; no external tools or API keys.

stages:
- name: discover
processor: file_discovery
config:
extensions: [txt]

- name: keep_larger_files
processor: metadata_filter
config:
min_size_bytes: 32
1 change: 1 addition & 0 deletions pipelines/demo/samples/hello.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Hello from DeepZero. This harmless text file is large enough to pass the demo's size filter.
1 change: 1 addition & 0 deletions pipelines/demo/samples/tiny.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Small sample.
39 changes: 39 additions & 0 deletions tests/test_demo.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
from pathlib import Path

from click.testing import CliRunner

from deepzero.cli import main
from deepzero.engine.pipeline import corpus_segment
from deepzero.engine.state import StateStore


def test_shipped_demo_runs_resumes_and_reports(tmp_path):
root = Path(__file__).resolve().parents[1]
pipeline = root / "pipelines/demo/pipeline.yaml"
samples = root / "pipelines/demo/samples"
work = tmp_path / "work"
cli = CliRunner()
args = ["run", str(samples), "-p", str(pipeline), "-w", str(work)]

first = cli.invoke(main, args)
assert first.exit_code == 0, first.output
run_dir = work / "demo" / corpus_segment(samples)
store = StateStore(run_dir)
run = store.load_run()
assert run.status == "completed"
states = {sample.filename: sample for sample in store.list_samples()}
assert set(states) == {"hello.txt", "tiny.txt"}
assert states["hello.txt"].history["keep_larger_files"].status == "completed"
assert states["hello.txt"].history["keep_larger_files"].verdict == "continue"
assert states["tiny.txt"].verdict == "filtered"
assert (run_dir / "report/index.html").is_file()

resumed = cli.invoke(main, args)
assert resumed.exit_code == 0, resumed.output
assert "resuming run" in resumed.output
assert store.load_run().run_id == run.run_id
assert len(store.list_samples()) == 2

report = cli.invoke(main, ["report", "-w", str(run_dir)])
assert report.exit_code == 0, report.output
assert (run_dir / "report/report.json").is_file()
Loading