A3-N/xpfarm
GitHub: A3-N/xpfarm
一个封装多款安全工具的自动化漏洞扫描平台，通过统一Web UI 与 AI 辅助二进制分析提升测试效率与可见性。
Stars: 40 | Forks: 45
# XPFarm
一个开源漏洞扫描器，将知名的安全工具封装在统一的 Web UI 中。
## 
### 索引
| 部分 | 描述 |
|---|---|
| [为什么](#why) | XPFarm 的动机与理念 |
| [封装工具](#wrapped-tools) | XPFarm 编排的 10 个开源工具 |
| [架构图](#architecture-map) | 完整系统架构、扫描流程、数据流与 AI 子系统 |
| [Overlord - AI 二进制分析](#overlord---ai-binary-analysis) | 内置用于二进制/恶意软件分析的 AI 代理 |
| [安装](#setup) | 构建与部署说明（Docker / 源码） |
| [随机截图](#random-screenshots) | 扫描与日志的 UI 截图 |
| [待办事项](#todo) | 计划中的功能与路线图 |
## 架构图
```
flowchart TB
subgraph ENTRY["Entrypoint - main.go"]
M1["Parse Flags
-debug mode"] M2["InitDB
SQLite + WAL + GORM"] M3["InitModules
Register 10 tool wrappers"] M4["Health Check
Auto-install missing tools"] M5["RunUpdates
-up flag on all PD tools"] M6["CheckAndIndexTemplates
Nuclei template versioning"] M7["StartServer
Gin on :8888"] M1 --> M2 --> M3 --> M4 --> M5 --> M6 --> M7 end subgraph UI_LAYER["Web UI - internal/ui/server.go"] direction TB GIN["Gin HTTP Server
Embedded templates + static"] subgraph Pages["HTML Pages"] P1["Dashboard"] P2["Assets"] P3["Asset Details"] P4["Target Details"] P5["Modules"] P6["Settings"] P7["Overlord Chat"] P8["Overlord Binary"] P9["Search"] P10["Advanced Scan"] P11["Scan Settings"] end subgraph REST["REST API"] direction LR A1["POST /api/scan"] A2["POST /api/search"] A3["GET/POST /api/overlord/*"] A4["POST /assets/create|delete"] A5["POST /settings/*"] A6["GET /api/active-scans"] A7["POST /api/search/save|delete"] A8["GET /api/overlord/events
SSE Proxy"] end GIN --> Pages GIN --> REST end subgraph SCAN_ENGINE["Scan Engine - internal/core/"] direction TB SM["ScanManager
Singleton, mutex-guarded
Active scan tracking"] subgraph PIPELINE["8-Stage Scanning Pipeline - manager.go"] direction TB S0["Target Input
ParseTarget + NormalizeToHostname"] S1["Stage 1: Subfinder
Subdomain Discovery"] S2["Stage 2: Filter & Save
ResolveAndCheck per subdomain
Cloudflare / Localhost / Alive"] S3["Stage 3: Naabu
Port Scanning
5-worker pool via channel"] S4["Stage 4: Nmap
Service Enumeration
Version + Script detection"] S5["Stage 5: Httpx
HTTP Probing
Rich metadata extraction"] S6["Stage 6: Parallel Web Processing"] S7["Stage 7: CVEMap
CVE lookup by product/tech"] S8["Stage 8: Nuclei
Vulnerability Scanning"] S0 --> S1 --> S2 --> S3 --> S4 --> S5 --> S6 --> S7 --> S8 end subgraph STAGE6_DETAIL["Stage 6 - Parallel Web Asset Processing"] direction LR GW["Gowitness
Screenshots"] KAT["Katana
JS Crawling, depth=5"] UF["URLFinder
URL Discovery"] WAP["Wappalyzer
Tech Detection
Header + Body analysis"] end subgraph NUCLEI_PLAN["Stage 8 - Nuclei Scan Plan"] direction TB NP1["BuildNucleiPlan
serviceTagMap lookup"] NP2["Network Scans
Per-port, service tags"] NP3["Web Auto Scan
-as wappalyzer mode"] NP4["Fallback Scan
Unmapped services"] NP5["Enabled Mode
Custom template workflow"] NP1 --> NP2 NP1 --> NP3 NP1 --> NP4 NP1 --> NP5 end subgraph RESOLVE["Target Resolution - target.go"] direction LR R1["ParseTarget
IP / CIDR / Domain / URL"] R2["ResolveAndCheck
DNS + Cloudflare + Localhost"] R3["DNS Cache
sync.Map, 5min TTL"] R1 --> R2 --> R3 end subgraph SEARCH["Search Engine - search.go"] SRCH["GlobalSearch
Regex filter engine
SQL joins across 5 tables
AND/OR/Negate chaining"] end subgraph TEMPLATE_IDX["Template Indexer - template_indexer.go"] TI["IndexNucleiTemplates
Walk filesystem, batch upsert
Version-tracked re-index"] end SM --> PIPELINE S6 --> STAGE6_DETAIL S8 --> NUCLEI_PLAN end subgraph MODULES["Module System - internal/modules/"] direction TB IFACE["Module Interface
Name / Description
CheckInstalled / Install / Run"] subgraph REGISTRY["Registry - 10 Registered Modules"] direction LR T1["Subfinder
Subdomain enum"] T2["Naabu
Port scan"] T3["Nmap
Service detection"] T4["Httpx
HTTP probe"] T5["Gowitness
Screenshots"] T6["Katana
Web crawling"] T7["URLFinder
URL discovery"] T8["Wappalyzer
Tech fingerprint"] T9["Nuclei
Vuln scanning"] T10["CVEMap
CVE lookup"] end IFACE --> REGISTRY end subgraph DATABASE["Database - internal/database/"] direction TB DB["SQLite + WAL Mode
GORM ORM
Single writer connection
30s busy timeout"] subgraph MODELS["Data Models"] direction LR DM1["Asset
name, advanced_mode"] DM2["Target
value, type, is_alive
is_cloudflare, is_localhost"] DM3["Port
port, protocol
service, product, version"] DM4["WebAsset
url, title, tech_stack
screenshot, paths"] DM5["Vulnerability
name, severity
template_id, matcher"] DM6["CVE
cve_id, severity
cvss, epss, is_kev"] DM7["ScanResult
tool_name, output"] DM8["ScanProfile
Feature toggles
Port/Web/Vuln scope"] DM9["NucleiTemplate
template_id, file_path"] DM10["SavedSearch
name, query_data"] DM11["Setting
key-value config"] end DM1 -->|has many| DM2 DM2 -->|has many| DM3 DM2 -->|has many| DM4 DM2 -->|has many| DM5 DM2 -->|has many| DM6 DM2 -->|has many| DM7 DM1 -->|has one| DM8 DB --> MODELS end subgraph OVERLORD["Overlord - AI Agent Subsystem"] direction TB OV_PROXY["Overlord Proxy
internal/overlord/overlord.go"] subgraph OV_API["OpenCode Serve API"] direction LR OA1["GET /session
List sessions"] OA2["POST /session
Create session"] OA3["POST /session/:id/prompt_async
Send message"] OA4["POST /session/:id/abort
Stop analysis"] OA5["GET /event
SSE stream"] end subgraph OV_FILES["File Management"] direction LR OF1["Binary Upload
overlord/binaries/"] OF2["Analysis Output
overlord/output/"] OF3["Auth JSON
API key storage"] end subgraph PROVIDERS["21 AI Providers"] direction LR PR1["Anthropic
OpenAI
Groq"] PR2["DeepSeek
OpenRouter
xAI"] PR3["Ollama Local
Cerebras
Together"] PR4["OpenCode Zen/Go
+ 12 more"] end OV_PROXY --> OV_API OV_PROXY --> OV_FILES OV_PROXY --> PROVIDERS end subgraph DOCKER["Docker Deployment - docker-compose.yml"] direction LR DC1["xpfarm container
Go app on :8888"] DC2["overlord container
OpenCode serve :3000
radare2 + analysis tools"] DC3["mobsf container
Mobile scan :8000
optional profile"] DC1 <-->|xpfarm-network| DC2 DC1 <-->|xpfarm-network| DC3 end subgraph NOTIFICATIONS["Notifications - internal/notifications/"] direction LR N1["Discord Bot
Embed notifications
Scan start/stop alerts"] N2["Telegram Bot
Markdown messages
Scan start/stop alerts"] end subgraph UTILS["Utilities - pkg/utils/"] direction LR U1["Gradient Logger
Color-coded terminal output"] U2["Cloudflare IP Detector
IPv4 + IPv6 CIDR matching"] U3["Binary Resolver
PATH + GOPATH fallback"] end %% Cross-component connections M7 --> GIN A1 --> SM A2 --> SRCH A3 --> OV_PROXY SM -->|uses| IFACE SM -->|reads/writes| DB SM -->|callbacks| N1 SM -->|callbacks| N2 PIPELINE -->|ResolveAndCheck| RESOLVE PIPELINE -->|recordResult| DB S5 -->|analyzes response| WAP A8 -->|proxies| OA5 ``` ## 为什么 像 [Assetnote](https://www.assetnote.io/) 这样的工具非常出色——维护良好、更新及时、漏洞识别透明。但它们并非开源，也没有必要重复造轮子，因为已经存在大量成熟的开源工具。XPFarm 只是将它们封装在一起，让你拥有一个开源且更少商业色彩的漏洞扫描器。 重点在于构建一个你可以看到后台哪些步骤失败或被移除的漏洞扫描器，而不是对神秘行为感到困惑。 ## 封装工具 - [Subfinder](https://github.com/projectdiscovery/subfinder) - 子域名发现 - [Naabu](https://github.com/projectdiscovery/naabu) - 端口扫描 - [Httpx](https://github.com/projectdiscovery/httpx) - HTTP 探测 - [Nuclei](https://github.com/projectdiscovery/nuclei) - 漏洞扫描 - [Nmap](https://nmap.org/) - 网络扫描 - [Katana](https://github.com/projectdiscovery/katana) - 爬取 - [URLFinder](https://github.com/projectdiscovery/urlfinder) - URL 发现 - [Gowitness](https://github.com/sensepost/gowitness) - 截图 - [Wappalyzer](https://github.com/projectdiscovery/wappalyzergo) - 技术识别 - [CVEMap](https://github.com/projectdiscovery/cvemap) - CVE 映射  ## Overlord - AI 二进制分析 ## 安装 ``` # 使用辅助脚本（推荐） ./xpfarm.sh build # Build all containers ./xpfarm.sh up # Start everything # Windows .\xpfarm.ps1 build .\xpfarm.ps1 up # 标准 Docker docker compose up --build # 从源代码构建（无 Overlord） go build -o xpfarm ./xpfarm ./xpfarm -debug ```  ## 随机截图     ## 待办事项 - [ ] SecretFinder JS 4 Web - [ ] 从域名检测/扫描仓库
-debug mode"] M2["InitDB
SQLite + WAL + GORM"] M3["InitModules
Register 10 tool wrappers"] M4["Health Check
Auto-install missing tools"] M5["RunUpdates
-up flag on all PD tools"] M6["CheckAndIndexTemplates
Nuclei template versioning"] M7["StartServer
Gin on :8888"] M1 --> M2 --> M3 --> M4 --> M5 --> M6 --> M7 end subgraph UI_LAYER["Web UI - internal/ui/server.go"] direction TB GIN["Gin HTTP Server
Embedded templates + static"] subgraph Pages["HTML Pages"] P1["Dashboard"] P2["Assets"] P3["Asset Details"] P4["Target Details"] P5["Modules"] P6["Settings"] P7["Overlord Chat"] P8["Overlord Binary"] P9["Search"] P10["Advanced Scan"] P11["Scan Settings"] end subgraph REST["REST API"] direction LR A1["POST /api/scan"] A2["POST /api/search"] A3["GET/POST /api/overlord/*"] A4["POST /assets/create|delete"] A5["POST /settings/*"] A6["GET /api/active-scans"] A7["POST /api/search/save|delete"] A8["GET /api/overlord/events
SSE Proxy"] end GIN --> Pages GIN --> REST end subgraph SCAN_ENGINE["Scan Engine - internal/core/"] direction TB SM["ScanManager
Singleton, mutex-guarded
Active scan tracking"] subgraph PIPELINE["8-Stage Scanning Pipeline - manager.go"] direction TB S0["Target Input
ParseTarget + NormalizeToHostname"] S1["Stage 1: Subfinder
Subdomain Discovery"] S2["Stage 2: Filter & Save
ResolveAndCheck per subdomain
Cloudflare / Localhost / Alive"] S3["Stage 3: Naabu
Port Scanning
5-worker pool via channel"] S4["Stage 4: Nmap
Service Enumeration
Version + Script detection"] S5["Stage 5: Httpx
HTTP Probing
Rich metadata extraction"] S6["Stage 6: Parallel Web Processing"] S7["Stage 7: CVEMap
CVE lookup by product/tech"] S8["Stage 8: Nuclei
Vulnerability Scanning"] S0 --> S1 --> S2 --> S3 --> S4 --> S5 --> S6 --> S7 --> S8 end subgraph STAGE6_DETAIL["Stage 6 - Parallel Web Asset Processing"] direction LR GW["Gowitness
Screenshots"] KAT["Katana
JS Crawling, depth=5"] UF["URLFinder
URL Discovery"] WAP["Wappalyzer
Tech Detection
Header + Body analysis"] end subgraph NUCLEI_PLAN["Stage 8 - Nuclei Scan Plan"] direction TB NP1["BuildNucleiPlan
serviceTagMap lookup"] NP2["Network Scans
Per-port, service tags"] NP3["Web Auto Scan
-as wappalyzer mode"] NP4["Fallback Scan
Unmapped services"] NP5["Enabled Mode
Custom template workflow"] NP1 --> NP2 NP1 --> NP3 NP1 --> NP4 NP1 --> NP5 end subgraph RESOLVE["Target Resolution - target.go"] direction LR R1["ParseTarget
IP / CIDR / Domain / URL"] R2["ResolveAndCheck
DNS + Cloudflare + Localhost"] R3["DNS Cache
sync.Map, 5min TTL"] R1 --> R2 --> R3 end subgraph SEARCH["Search Engine - search.go"] SRCH["GlobalSearch
Regex filter engine
SQL joins across 5 tables
AND/OR/Negate chaining"] end subgraph TEMPLATE_IDX["Template Indexer - template_indexer.go"] TI["IndexNucleiTemplates
Walk filesystem, batch upsert
Version-tracked re-index"] end SM --> PIPELINE S6 --> STAGE6_DETAIL S8 --> NUCLEI_PLAN end subgraph MODULES["Module System - internal/modules/"] direction TB IFACE["Module Interface
Name / Description
CheckInstalled / Install / Run"] subgraph REGISTRY["Registry - 10 Registered Modules"] direction LR T1["Subfinder
Subdomain enum"] T2["Naabu
Port scan"] T3["Nmap
Service detection"] T4["Httpx
HTTP probe"] T5["Gowitness
Screenshots"] T6["Katana
Web crawling"] T7["URLFinder
URL discovery"] T8["Wappalyzer
Tech fingerprint"] T9["Nuclei
Vuln scanning"] T10["CVEMap
CVE lookup"] end IFACE --> REGISTRY end subgraph DATABASE["Database - internal/database/"] direction TB DB["SQLite + WAL Mode
GORM ORM
Single writer connection
30s busy timeout"] subgraph MODELS["Data Models"] direction LR DM1["Asset
name, advanced_mode"] DM2["Target
value, type, is_alive
is_cloudflare, is_localhost"] DM3["Port
port, protocol
service, product, version"] DM4["WebAsset
url, title, tech_stack
screenshot, paths"] DM5["Vulnerability
name, severity
template_id, matcher"] DM6["CVE
cve_id, severity
cvss, epss, is_kev"] DM7["ScanResult
tool_name, output"] DM8["ScanProfile
Feature toggles
Port/Web/Vuln scope"] DM9["NucleiTemplate
template_id, file_path"] DM10["SavedSearch
name, query_data"] DM11["Setting
key-value config"] end DM1 -->|has many| DM2 DM2 -->|has many| DM3 DM2 -->|has many| DM4 DM2 -->|has many| DM5 DM2 -->|has many| DM6 DM2 -->|has many| DM7 DM1 -->|has one| DM8 DB --> MODELS end subgraph OVERLORD["Overlord - AI Agent Subsystem"] direction TB OV_PROXY["Overlord Proxy
internal/overlord/overlord.go"] subgraph OV_API["OpenCode Serve API"] direction LR OA1["GET /session
List sessions"] OA2["POST /session
Create session"] OA3["POST /session/:id/prompt_async
Send message"] OA4["POST /session/:id/abort
Stop analysis"] OA5["GET /event
SSE stream"] end subgraph OV_FILES["File Management"] direction LR OF1["Binary Upload
overlord/binaries/"] OF2["Analysis Output
overlord/output/"] OF3["Auth JSON
API key storage"] end subgraph PROVIDERS["21 AI Providers"] direction LR PR1["Anthropic
OpenAI
Groq"] PR2["DeepSeek
OpenRouter
xAI"] PR3["Ollama Local
Cerebras
Together"] PR4["OpenCode Zen/Go
+ 12 more"] end OV_PROXY --> OV_API OV_PROXY --> OV_FILES OV_PROXY --> PROVIDERS end subgraph DOCKER["Docker Deployment - docker-compose.yml"] direction LR DC1["xpfarm container
Go app on :8888"] DC2["overlord container
OpenCode serve :3000
radare2 + analysis tools"] DC3["mobsf container
Mobile scan :8000
optional profile"] DC1 <-->|xpfarm-network| DC2 DC1 <-->|xpfarm-network| DC3 end subgraph NOTIFICATIONS["Notifications - internal/notifications/"] direction LR N1["Discord Bot
Embed notifications
Scan start/stop alerts"] N2["Telegram Bot
Markdown messages
Scan start/stop alerts"] end subgraph UTILS["Utilities - pkg/utils/"] direction LR U1["Gradient Logger
Color-coded terminal output"] U2["Cloudflare IP Detector
IPv4 + IPv6 CIDR matching"] U3["Binary Resolver
PATH + GOPATH fallback"] end %% Cross-component connections M7 --> GIN A1 --> SM A2 --> SRCH A3 --> OV_PROXY SM -->|uses| IFACE SM -->|reads/writes| DB SM -->|callbacks| N1 SM -->|callbacks| N2 PIPELINE -->|ResolveAndCheck| RESOLVE PIPELINE -->|recordResult| DB S5 -->|analyzes response| WAP A8 -->|proxies| OA5 ``` ## 为什么 像 [Assetnote](https://www.assetnote.io/) 这样的工具非常出色——维护良好、更新及时、漏洞识别透明。但它们并非开源，也没有必要重复造轮子，因为已经存在大量成熟的开源工具。XPFarm 只是将它们封装在一起，让你拥有一个开源且更少商业色彩的漏洞扫描器。 重点在于构建一个你可以看到后台哪些步骤失败或被移除的漏洞扫描器，而不是对神秘行为感到困惑。 ## 封装工具 - [Subfinder](https://github.com/projectdiscovery/subfinder) - 子域名发现 - [Naabu](https://github.com/projectdiscovery/naabu) - 端口扫描 - [Httpx](https://github.com/projectdiscovery/httpx) - HTTP 探测 - [Nuclei](https://github.com/projectdiscovery/nuclei) - 漏洞扫描 - [Nmap](https://nmap.org/) - 网络扫描 - [Katana](https://github.com/projectdiscovery/katana) - 爬取 - [URLFinder](https://github.com/projectdiscovery/urlfinder) - URL 发现 - [Gowitness](https://github.com/sensepost/gowitness) - 截图 - [Wappalyzer](https://github.com/projectdiscovery/wappalyzergo) - 技术识别 - [CVEMap](https://github.com/projectdiscovery/cvemap) - CVE 映射  ## Overlord - AI 二进制分析 ## 安装 ``` # 使用辅助脚本（推荐） ./xpfarm.sh build # Build all containers ./xpfarm.sh up # Start everything # Windows .\xpfarm.ps1 build .\xpfarm.ps1 up # 标准 Docker docker compose up --build # 从源代码构建（无 Overlord） go build -o xpfarm ./xpfarm ./xpfarm -debug ```  ## 随机截图     ## 待办事项 - [ ] SecretFinder JS 4 Web - [ ] 从域名检测/扫描仓库
标签：ADB, AI 二进制分析, Android 动态分析, CTI, DAST, Docker, Gin, Go, Google, GORM, Nuclei, Ruby工具, SEO 漏洞扫描器, SQLite, TCP SYN 扫描, Web UI, 仪表盘, 健康检查, 动态插桩, 后端开发, 安全扫描, 安全防御评估, 开源安全工具, 开源安全编排, 恶意软件分析, 日志可视化, 日志审计, 时序注入, 模块化扫描, 索引与版本控制, 自动化渗透测试, 自动更新, 请求拦截, 逆向工程平台