clivoa/DaC

# Detection as Code — Splunk 一个完整的 Detection as Code (DaC) 流水线，用于将 Splunk 检测规则作为受版本控制的 YAML 文件进行管理，并通过 GitHub Actions 实现自动验证与部署。 [](https://github.com/clivoa/DaC/actions/workflows/validate-pr.yml) ## 工作原理 ``` Analyst creates or edits a detection YAML file │ ▼ git push → Pull Request → dev branch │ ▼ GitHub Actions (self-hosted runner) ┌────────────────────────────────────────┐ │ 1. YAML syntax check │ │ 2. JSON Schema validation │ │ 3. SPL syntax check (Splunk REST API) │ │ → PR blocked if any check fails │ └────────────────────────────────────────┘ │ ▼ Code review + approval → merge to dev │ ▼ PR: dev → main (production promotion) │ ▼ GitHub Actions deploy ┌────────────────────────────────────────┐ │ 1. Verify: came from a merged PR │ │ 2. Dry-run preview │ │ 3. Create/update Splunk saved search │ └────────────────────────────────────────┘ ``` 有关完整的架构和设计理念，请参阅 [docs/architecture.md](docs/architecture.md)。 ## 仓库结构 ``` . ├── detections/ │ ├── endpoint/ # Endpoint detections (Windows, Linux, macOS) │ ├── network/ # Network-based detections │ ├── identity/ # Identity and access detections │ ├── cloud/ # Cloud provider detections │ └── application/ # Application-layer detections ├── docs/ │ ├── architecture.md # Full DaC design, pipeline, and concepts │ ├── github-governance.md # Branch model and protection policy │ ├── splunk-setup.md # Splunk local setup with Orbstack/Docker │ └── runner-setup.md # Self-hosted runner container setup ├── runner/ │ ├── Dockerfile # Ubuntu 24.04 image with GitHub Actions runner │ └── entrypoint.sh # Runner registration and graceful deregistration ├── schemas/ │ └── detection.schema.json # JSON Schema for detection YAML files ├── scripts/ │ ├── validate.py # Schema + SPL syntax validation │ ├── deploy.py # Deploy detections via Splunk REST API │ └── splunk_client.py # Splunk REST API client ├── .github/ │ ├── workflows/ │ │ ├── validate-pr.yml # Required check on PRs to dev and main │ │ ├── deploy.yml # Deploy on merge to main │ │ └── cleanup-branches.yml # Weekly stale branch cleanup │ └── PULL_REQUEST_TEMPLATE.md ├── docker-compose.yml # Orchestrates the self-hosted runner container ├── .env.example # Environment variable reference (no real values) ├── CONTRIBUTING.md # How to write and submit detections └── SECURITY.md # Security policy and sensitive data guidelines ``` ## 检测文件格式 ``` name: "Detect " id: "" # python3 -c "import uuid; print(uuid.uuid4())" version: 1 # increment on every change status: draft # lifecycle hint: draft | testing | production | deprecated author: "" date: "YYYY-MM-DD" modified: "YYYY-MM-DD" description: "What this detects and why it matters" type: alert # alert | report | scheduled_report search: | index= EventCode= | stats count by ComputerName, User | where count > 0 schedule: cron: "*/15 * * * *" earliest: "-15m" latest: "now" alert: condition: "search count > 0" severity: high # informational | low | medium | high | critical suppress: false tags: mitre_attack: - T1059.001 # T#### or T####.### platform: - Windows # Windows | Linux | macOS | AWS | Azure | GCP | Network category: endpoint # endpoint | network | identity | cloud | application splunk_app: "search" ``` ## 基础设施设置 此流水线需要两个在 Orbstack 上运行的本地组件。只需设置一次： | 组件 | 指南 | |---|---| | Splunk (Docker) | [docs/splunk-setup.md](docs/splunk-setup.md) | | GitHub Actions runner (Docker) | [docs/runner-setup.md](docs/runner-setup.md) | ### 必需的 GitHub secrets 前往 **Settings → Secrets and variables → Actions**： | 类型 | 名称 | 值 | |---|---|---| | Secret | `SPLUNK_URL` | `https://splunk:8089` | | Secret | `SPLUNK_TOKEN` | 来自 Splunk → Settings → Tokens 的 token | | Variable | `SPLUNK_APP` | `search`（或您的自定义应用名称） | ## 本地开发 ### 设置 ``` # 安装 Python 依赖和 pre-commit hooks make setup # 本地 SPL 验证所需的必需环境变量 export SPLUNK_URL="https://splunk:8089" # or https://localhost:8089 if port 8089 is published export SPLUNK_TOKEN="" ``` ### 验证检测 ``` # 仅 Schema 验证（不需要 Splunk） make validate # Schema + 实时 SPL 语法检查 make validate-splunk # 单文件 python3 scripts/validate.py detections/endpoint/detect_new_local_admin.yml ``` ### 手动部署 ``` make deploy-dry # preview without making changes make deploy # deploy all detections using each YAML status # 在不编辑 detection YAML 的情况下覆盖部署状态 DEPLOY_STATUS=production make deploy python3 scripts/deploy.py --all --deploy-status testing ``` ## 分析师工作流 ``` # 1. 从 dev 创建分支 git checkout dev && git pull origin dev git checkout -b dev// # 2. 编写 detection 文件 # （为 id 字段生成一个新的 UUID） python3 -c "import uuid; print(uuid.uuid4())" # 3. 提交前在本地验证 python3 scripts/validate.py --no-splunk detections//.yml # 4. 提交并推送 git add detections//.yml git commit -m "feat: describe the detection" git push origin dev// # 5. 向 dev 发起 PR → CI 会自动验证 # 6. 审查并合并后 → 通过 PR 提升：dev → main → 使用 DEPLOY_STATUS=production 进行自动部署 ``` 有关完整的编写指南和检测质量指南，请参阅 [CONTRIBUTING.md](CONTRIBUTING.md)。 ## 已验证的字段 | 字段 | 必需 | 规则 | |---|---|---| | `name` | 是 | 最少 5 个字符 | | `id` | 是 | UUID v4 格式 | | `version` | 是 | 整数 ≥ 1 | | `status` | 是 | `draft` \| `testing` \| `production` \| `deprecated` | | `author` | 是 | 字符串 | | `description` | 是 | 最少 10 个字符 | | `type` | 是 | `alert` \| `report` \| `scheduled_report` | | `search` | 是 | 有效的 SPL（通过 Splunk REST API 实时检查） | | `tags.category` | 是 | 枚举值 | | `tags.mitre_attack` | 否 | `T####` 或 `T####.###` 格式 — 如缺失则发出警告 | ## GitHub 治理 `main` 和 `dev` 是受保护分支。所有更改必须通过 pull request 提交。有关完整的策略、分支保护设置以及 GitHub 套餐要求说明，请参阅 [docs/github-governance.md](docs/github-governance.md)。 ## 许可证 [MIT](LICENSE)