kei99-web3/agentops-control-tower
GitHub: kei99-web3/agentops-control-tower
基于 Splunk 的 AI 事件指挥中心,通过证据驱动的根本原因分析和人工审批的修复流程,将分散的中断信号转化为可审查的运维决策。
Stars: 0 | Forks: 0
# Agentic 事件指挥中心
这不仅是另一个告警视图:它是一个 AI 事件指挥官,在 Splunk 中验证每一项声明,并在行动前请求许可。
基于 Splunk 的事件决策:通过单一的指挥流程,提供引用了证据的根本原因、爆炸半径以及经人工批准的修复方案。
## 5 分钟内评判
1. 打开 `reports/latest_control_tower.html`。
您将看到 checkout 事件摘要、排好序的根本原因、爆炸半径以及 MCP Remediation Ledger。
2. 运行本地验证路径:
```
python prototype\agentops_control_tower.py run-demo
python scripts\run_local_spl_query_pack.py
python scripts\build_judge_quickstart.py
```
您将看到重新生成的合成 `agentops_events`、重建的 SPL 等价验证,以及位于 `reports/latest_judge_quickstart.html` 的最新评判快速入门。
3. 检查官方 MCP 证据:
`submission/post_action_evidence/2026-06-09_optional_live_splunk_mcp_proof_readback.md`
官方 Splunk MCP Server 已通过使用合成 `agentops_events` 的本地 Splunk Enterprise Docker 进行验证;`splunk_run_query` 返回了事件 ID、证据引用、风险评分和审批状态。本声明不涉及生产级 Splunk Cloud 部署。
适用于所有公开材料的边界声明用语:基于合成数据的本地 Splunk Enterprise Docker 验证;不涉及生产级 Splunk Cloud 部署。
每一项主要声明都链接到评委可以检查的查询或账本条目:
| 声明 | 证据路径 | 查看要点 |
| --- | --- | --- |
| 根本原因有证据支持 | `reports/latest_local_spl_query_results.html` | `checkout-api` 的时间线和根本原因证据行。 |
| 行动前可见爆炸半径 | `reports/latest_control_tower.html` | 修复前受影响服务的分组情况。 |
| 高风险修复保持人工批准 | `reports/latest_control_tower.html` | MCP Remediation Ledger 的审批状态。 |
| 官方 MCP 回读已在本地验证 | `submission/post_action_evidence/2026-06-09_optional_live_splunk_mcp_proof_readback.md` | 包含事件 ID 和证据引用的 `splunk_run_query` 行。 |
## 概述
Agentic Incident Command Center 是一个 Splunk Agentic Ops Hackathon 项目的候选作品。它将跨领域的事件信号转化为有证据支持的 AI 指挥流程:时间线、爆炸半径、根本原因排序以及经人工批准的修复方案。
该项目围绕一个实际问题构建:在实时中断期间,线索散落在部署日志、应用程序错误、APM 追踪、数据库压力、身份/安全事件、边缘网络以及 AI/MCP 工具调用中。Splunk 是天然的证据层。AI 不应凭空发明修复方案;它应该向 Splunk 查询,对可能的原因进行排序,引用证据,并将高风险操作保留在人工批准之后。
核心创新在于 MCP Remediation Ledger:AI 提出的每一次回滚、WAF 监控、工单、利益相关者更新或凭证边界拦截,都与 Splunk 证据和明确的审批状态绑定。
产品价值在于决策压缩:将分散的部署、APM、数据库、安全、边缘以及 MCP/工具调用信号汇聚成一个可审查的流程,从证据到排序的原因再到待批准的操作,而无需赋予 agent 不受限的修复权力。
这是一个独立的黑客松项目,并非官方 Splunk 产品。Splunk 及相关标志归其各自所有者所有。

## 功能说明
- 摄取跨越部署、应用程序、APM、数据库、安全、网络、修复、通信和 MCP 运行时领域的合成事件。
- 使用适配 Splunk 的证据字段,对导致 checkout 中断的可能根本原因进行排序。
- 生成带有证据支持的建议操作和审批状态的 MCP Remediation Ledger。
- 呈现人工操作包:批准回滚、批准临时的 WAF 监控规则、审查利益相关者更新、保留被拦截的凭证边界证据,或进行进一步调查。
- 导出适配 Splunk 的 CSV 和 SPL 示例,用于索引和基于 MCP 的调查。
- 包含一个 Splunk 应用候选版本,具有 index、sourcetype、dashboard 和 saved-search 配置。
- 渲染一个本地仪表板,在不使用私有数据的情况下演示完整流程。
## 为什么选择 Splunk
Splunk 是解决此问题的天然运营数据层:
- Agentic 系统会创建日志、事件、追踪、审批和工具调用记录。
- Splunk 能够统一来自开发者体验、安全和运营的这些信号。
- Splunk MCP Server 可以将此运营上下文暴露给 AI 助手,同时保持底层数据的可审计性。
- 人工审查者可以用自然语言提问,但每一项建议仍应立足于具体事件。
## 演示场景
本地演示使用合成的 checkout 中断事件:
1. Checkout API 发布完成后不久,出现 5xx 错误和延迟飙升。
2. 数据库连接池压力、身份异常、WAF 探测和边缘丢包作为竞争信号出现。
3. AI 事件指挥官请求获取 Splunk 上下文,并将 `checkout-api 发布回退` 排序为主要原因。
4. 回滚、WAF 监控、利益相关者更新和工单创建被准备为有证据支持的操作。
5. 高影响的修复操作保持由人工批准,而凭证边界工具的尝试则被拦截并作为经过脱敏处理的审计证据保留。
本地演示中不使用任何真实的 secret、账户、token、帖子、支付或外部系统。
## 完整本地构建与验证
```
python prototype\agentops_control_tower.py run-demo
python scripts\run_local_spl_query_pack.py
python scripts\build_demo_tour.py
python scripts\build_video_readiness_report.py
python scripts\build_video_cue_sheet.py
python scripts\build_video_upload_metadata.py
python scripts\build_video_command_plan.py
python scripts\build_claim_evidence_matrix.py
python scripts\build_external_approval_packet.py
python scripts\build_publication_command_plan.py
python scripts\build_public_repo_metadata.py
python scripts\build_public_repo_publish_brief.py
python scripts\verify_public_repo_publication_gate.py
python scripts\build_public_launch_snapshot.py
python scripts\verify_public_artifact_urls.py
python scripts\build_devpost_submission_packet.py
python scripts\export_devpost_final_copy.py
python scripts\build_final_go_no_go_report.py
python scripts\build_devpost_submit_command_plan.py
python scripts\build_devpost_manual_fill_brief.py
python scripts\build_post_action_evidence_brief.py
python scripts\build_official_source_freshness.py
python scripts\build_release_integrity_manifest.py
python scripts\prepare_submission_urls.py
python scripts\validate_claim_boundaries.py
python scripts\validate_submission_urls.py
python scripts\validate_splunk_app.py
python scripts\package_splunk_app.py
python scripts\build_splunk_mcp_command_plan.py
python scripts\build_splunk_mcp_proof_brief.py
python scripts\build_splunk_mcp_prompt_pack.py
python scripts\build_splunk_mcp_proof_capture_manifest.py
python scripts\build_submission_gate_ledger.py
python scripts\build_submission_deadline_burndown.py
python scripts\build_submission_review_index.py
python scripts\build_judge_quickstart.py
python scripts\build_judge_scorecard.py
python scripts\build_launch_decision_brief.py
python scripts\build_content_rights_audit.py
python scripts\build_video_dry_run.py
python scripts\build_video_recording_preview.py
python scripts\verify_public_video_upload_gate.py
python scripts\build_eligibility_compliance_audit.py
python scripts\build_next_approval_packet.py
python scripts\build_approval_consistency_audit.py
python scripts\build_status_conflict_audit.py
python scripts\build_public_repo_dry_run.py
python scripts\verify_public_repo_publication_gate.py
python scripts\publish_public_repo_after_approval.py
python scripts\build_url_writeback_dry_run.py
python scripts\package_public_candidate_zip.py
python scripts\smoke_test_release_zip.py
```
打开:
```
reports/latest_control_tower.html
reports/latest_claim_boundary_validation.html
reports/latest_devpost_final_copy.html
reports/latest_devpost_final_copy.md
reports/latest_submission_url_validation.html
reports/latest_release_zip_smoke_test.html
reports/latest_submission_review_index.html
reports/latest_demo_tour.html
reports/latest_video_readiness.html
submission/VIDEO_SCREEN_SAFETY_CHECKLIST.md
reports/latest_video_command_plan.html
reports/latest_video_cue_sheet.html
reports/latest_video_dry_run.html
reports/latest_video_recording_preview.html
reports/latest_video_upload_metadata.html
reports/latest_public_video_upload_preflight.html
reports/latest_claim_evidence_matrix.html
reports/latest_external_approval_packet.html
reports/latest_publication_command_plan.html
reports/latest_public_repo_metadata.html
reports/latest_public_repo_publish_brief.html
reports/latest_public_repo_dry_run.html
reports/latest_public_artifact_url_readback.html
reports/latest_url_writeback_dry_run.html
reports/latest_public_launch_snapshot.html
reports/latest_splunk_mcp_command_plan.html
reports/latest_splunk_mcp_proof_brief.html
reports/latest_splunk_mcp_prompt_pack.html
reports/latest_splunk_mcp_proof_capture_manifest.html
reports/latest_splunk_app_package_manifest.html
reports/latest_submission_gate_ledger.html
reports/latest_submission_deadline_burndown.html
reports/latest_judge_quickstart.html
reports/latest_judge_scorecard.html
reports/latest_launch_decision_brief.html
reports/latest_next_approval_packet.html
reports/latest_approval_consistency_audit.html
reports/latest_content_rights_audit.html
reports/latest_eligibility_compliance_audit.html
submission/HUMAN_CONFIRMATION_CHECKLIST.md
reports/latest_devpost_submit_command_plan.html
reports/latest_devpost_manual_fill_brief.html
submission/DEVPOST_FINAL_REVIEW_CHECKLIST.md
reports/latest_post_action_evidence_brief.html
reports/latest_official_source_freshness.html
reports/latest_release_integrity_manifest.html
reports/latest_status_conflict_audit.html
submission/POST_ACTION_EVIDENCE_LOG_TEMPLATE.md
reports/latest_devpost_submission_packet.html
reports/latest_final_go_no_go.html
reports/latest_local_spl_query_results.html
reports/latest_public_candidate_zip_manifest.html
reports/latest_submission_url_apply_plan.html
```
## 运行测试
```
python -m unittest discover -s tests
```
## 验证提交包
```
python scripts\validate_submission_packet.py
```
这将重新生成本地输出,运行本地 SPL 等价查询包,验证声明边界,测试软件包,检查截图/HTML 核心要素,并扫描公开候选版本中是否存在内部路径或类似 secret 的字符串。
它还会检查演示视频脚本时间轴、屏幕安全检查清单、视频屏幕安全检查清单、安全的 Splunk MCP 声明措辞、声明证据矩阵、明确的视频命令计划、视频提示表、视频彩排、视频录制预览、视频上传元数据、公开视频上传前检查、外部审批包、公开代码库发布命令计划、公开代码库元数据、公开代码库发布简报、公开代码库发布前检查、公开代码库彩排、受保护的公开代码库发布辅助工具、URL 回填彩排、公开发布快照、实时 Splunk/MCP 验证命令计划、实时 Splunk/MCP 验证简报、实时 Splunk/MCP prompt 包、实时 Splunk/MCP 验证捕获清单、提交门控账本、提交截止日期倒计时、评委快速入门、评委评分卡、发布决策简报、下一次审批包、审批一致性审计、状态冲突审计、内容版权和资产安全性、资格与合规性、人工确认检查清单、Devpost 最终提交命令计划、Devpost 手动填写/回读简报、Devpost 最终审查检查清单、行动后证据简报、行动后证据日志模板、官方来源新鲜度以及发布完整性清单,这一切都在进行任何录制、上传、发布、URL 回填或 Devpost 提交之前完成。
它还会验证本地的 Splunk 应用候选版本,包括 `default/indexes.conf`、`default/props.conf`、saved search、dashboard XML 以及生成的 `.spl` 软件包。
## 输出
- `data/synthetic_agentops_events.jsonl`
- `data/agentops_event_schema.json`
- `data/splunk_agentops_events.csv`
- `reports/latest_analysis.json`
- `reports/latest_claim_boundary_validation.html`
- `reports/latest_claim_boundary_validation.json`
- `reports/latest_control_tower.html`
- `reports/latest_devpost_final_copy.html`
- `reports/latest_devpost_final_copy.json`
- `reports/latest_devpost_final_copy.md`
- `reports/latest_submission_url_validation.html`
- `reports/latest_submission_url_validation.json`
- `reports/latest_release_zip_smoke_test.html`
- `reports/latest_release_zip_smoke_test.json`
- `reports/latest_demo_tour.html`
- `reports/latest_video_readiness.html`
- `reports/latest_video_readiness.json`
- `submission/VIDEO_SCREEN_SAFETY_CHECKLIST.md`
- `reports/latest_video_command_plan.html`
- `reports/latest_video_command_plan.json`
- `reports/latest_video_command_plan.md`
- `reports/latest_video_cue_sheet.html`
- `reports/latest_video_cue_sheet.json`
- `reports/latest_video_cue_sheet.md`
- `reports/latest_video_dry_run.html`
- `reports/latest_video_dry_run.json`
- `reports/latest_video_dry_run.md`
- `reports/latest_video_recording_preview.html`
- `reports/latest_video_recording_preview.json`
- `reports/latest_video_recording_preview.md`
- `reports/latest_video_upload_metadata.html`
- `reports/latest_video_upload_metadata.json`
- `reports/latest_video_upload_metadata.md`
- `submission/VIDEO_UPLOAD_METADATA.md`
- `reports/latest_public_video_upload_preflight.html`
- `reports/latest_public_video_upload_preflight.json`
- `reports/latest_public_video_upload_preflight.md`
- `reports/latest_claim_evidence_matrix.html`
- `reports/latest_claim_evidence_matrix.json`
- `reports/latest_claim_evidence_matrix.md`
- `reports/latest_external_approval_packet.html`
- `reports/latest_external_approval_packet.json`
- `reports/latest_external_approval_packet.md`
- `reports/latest_publication_command_plan.html`
- `reports/latest_publication_command_plan.json`
- `reports/latest_publication_command_plan.md`
- `reports/latest_public_repo_metadata.html`
- `reports/latest_public_repo_metadata.json`
- `reports/latest_public_repo_metadata.md`
- `reports/latest_public_repo_publish_brief.html`
- `reports/latest_public_repo_publish_brief.json`
- `reports/latest_public_repo_publish_brief.md`
- `reports/latest_public_repo_publication_preflight.html`
- `reports/latest_public_repo_publication_preflight.json`
- `reports/latest_public_repo_publication_preflight.md`
- `reports/latest_public_repo_dry_run.html`
- `reports/latest_public_repo_dry_run.json`
- `reports/latest_public_repo_dry_run.md`
- `reports/latest_public_artifact_url_readback.html`
- `reports/latest_public_artifact_url_readback.json`
- `reports/latest_public_artifact_url_readback.md`
- `reports/latest_url_writeback_dry_run.html`
- `reports/latest_url_writeback_dry_run.json`
- `reports/latest_url_writeback_dry_run.md`
- `reports/latest_public_launch_snapshot.html`
- `reports/latest_public_launch_snapshot.json`
- `reports/latest_public_launch_snapshot.md`
- `reports/latest_splunk_mcp_command_plan.html`
- `reports/latest_splunk_mcp_command_plan.json`
- `reports/latest_splunk_mcp_command_plan.md`
- `reports/latest_splunk_mcp_proof_brief.html`
- `reports/latest_splunk_mcp_proof_brief.json`
- `reports/latest_splunk_mcp_proof_brief.md`
- `reports/latest_splunk_mcp_prompt_pack.html`
- `reports/latest_splunk_mcp_prompt_pack.json`
- `reports/latest_splunk_mcp_prompt_pack.md`
- `submission/SPLUNK_MCP_PROMPT_PACK.md`
- `reports/latest_splunk_mcp_proof_capture_manifest.html`
- `reports/latest_splunk_mcp_proof_capture_manifest.json`
- `reports/latest_splunk_mcp_proof_capture_manifest.md`
- `submission/SPLUNK_MCP_PROOF_CAPTURE_MANIFEST.md`
- `reports/latest_splunk_app_package_manifest.html`
- `reports/latest_splunk_app_package_manifest.json`
- `reports/latest_splunk_app_package_manifest.md`
- `reports/latest_submission_gate_ledger.html`
- `reports/latest_submission_gate_ledger.json`
- `reports/latest_submission_gate_ledger.md`
- `reports/latest_submission_deadline_burndown.html`
- `reports/latest_submission_deadline_burndown.json`
- `reports/latest_submission_deadline_burndown.md`
- `reports/latest_submission_review_index.html`
- `reports/latest_submission_review_index.json`
- `reports/latest_submission_review_index.md`
- `reports/latest_judge_quickstart.html`
- `reports/latest_judge_quickstart.json`
- `reports/latest_judge_quickstart.md`
- `reports/latest_judge_scorecard.html`
- `reports/latest_judge_scorecard.json`
- `reports/latest_judge_scorecard.md`
- `reports/latest_launch_decision_brief.html`
- `reports/latest_launch_decision_brief.json`
- `reports/latest_launch_decision_brief.md`
- `reports/latest_next_approval_packet.html`
- `reports/latest_next_approval_packet.json`
- `reports/latest_next_approval_packet.md`
- `submission/NEXT_APPROVAL_PACKET.md`
- `reports/latest_approval_consistency_audit.html`
- `reports/latest_approval_consistency_audit.json`
- `reports/latest_approval_consistency_audit.md`
- `submission/USER_APPROVAL_GATES.md`
- `reports/latest_content_rights_audit.html`
- `reports/latest_content_rights_audit.json`
- `reports/latest_content_rights_audit.md`
- `reports/latest_eligibility_compliance_audit.html`
- `reports/latest_eligibility_compliance_audit.json`
- `reports/latest_eligibility_compliance_audit.md`
- `submission/HUMAN_CONFIRMATION_CHECKLIST.md`
- `reports/latest_devpost_submit_command_plan.html`
- `reports/latest_devpost_submit_command_plan.json`
- `reports/latestpost_submit_command_plan.md`
- `reports/latest_devpost_manual_fill_brief.html`
- `reports/latest_devpost_manual_fill_brief.json`
- `reports/latest_devpost_manual_fill_brief.md`
- `submission/DEVPOST_FINAL_REVIEW_CHECKLIST.md`
- `reports/latest_post_action_evidence_brief.html`
- `reports/latest_post_action_evidence_brief.json`
- `reports/latest_post_action_evidence_brief.md`
- `reports/latest_official_source_freshness.html`
- `reports/latest_official_source_freshness.json`
- `reports/latest_official_source_freshness.md`
- `reports/latest_release_integrity_manifest.html`
- `reports/latest_release_integrity_manifest.json`
- `reports/latest_release_integrity_manifest.md`
- `reports/latest_status_conflict_audit.html`
- `reports/latest_status_conflict_audit.json`
- `reports/latest_status_conflict_audit.md`
- `submission/POST_ACTION_EVIDENCE_LOG_TEMPLATE.md`
- `submission/PUBLIC_REPO_METADATA.md`
- `reports/latest_submission_url_apply_plan.html`
- `reports/latest_submission_url_apply_plan.json`
- `reports/latest_submission_url_apply_plan.md`
- `reports/latest_devpost_submission_packet.html`
- `reports/latest_devpost_submission_packet.json`
- `reports/latest_final_go_no_go.html`
- `reports/latest_final_go_no_go.json`
- `reports/latest_local_spl_query_results.html`
- `reports/latest_local_spl_query_results.json`
- `reports/latest_public_candidate_zip_manifest.html`
- `reports/latest_public_candidate_zip_manifest.json`
- `release/agentops-control-tower-public-candidate.zip`
- `splunk_app/agentops_control_tower/default/data/ui/views/agentops_control_tower.xml`
- `splunk_app/agentops_control_tower/default/savedsearches.conf`
- `reports/latest_mcp_investigation.md`
- `reports/latest_submission_validation.html`
- `reports/latest_submission_validation.json`
- `dist/agentops-control-tower-splunk-app.spl`
- `assets/dashboard_preview.png`
- `submission/REQUIREMENTS_MATRIX.md`
- `submission/DEVPOST_FIELD_MAP.md`
- `submission/DEVPOST_FINAL_REVIEW_CHECKLIST.md`
- `submission/DEVPOST_SUBMISSION_DRAFT.md`
- `submission/DEMO_VIDEO_SCRIPT.md`
- `submission/VIDEO_RECORDING_RUNBOOK.md`
- `submission/FINAL_SUBMISSION_CHECKLIST.md`
- `submission/JUDGING_ALIGNMENT.md`
- `submission/OFFICIAL_REQUIREMENTS_AUDIT.md`
- `submission/SPL_QUERIES.md`
- `submission/SUBMISSION_DEADLINE_BURNDOWN.md`
- `submission/SUBMISSION_LAUNCH_RUNBOOK.md`
- `submission/SUBMISSION_REVIEW_QA.md`
- `architecture_diagram.md`
## Splunk 导入
在将 `data/splunk_agentops_events.csv` 导入到 `agentops_events` index 后,从以下步骤开始:
```
index=agentops_events risk_score>=70 | table _time component run_id event_type risk_score policy_decision evidence_ref message
```
查看 `submission/SPL_QUERIES.md` 获取完整的演示查询包。
查看 `submission/SPLUNK_MCP_PROMPT_PACK.md` 获取可选的实时 MCP 验证 prompt、预期引用、成功回读和停止条件。
该代码库还包含一个本地 Splunk 应用候选版本:
```
splunk_app/agentops_control_tower
```
它包含一个 Simple XML dashboard 以及用于事件时间线、根本原因证据、经人工批准的修复账本、MCP 调查上下文和爆炸半径的 saved search。使用以下命令在本地验证它:
```
python scripts\validate_splunk_app.py
```
在本地将其打包为可供审查的 `.spl` 产物,无需安装、上传、发布或连接:
```
python scripts\package_splunk_app.py
```
这将生成 `dist/agentops-control-tower-splunk-app.spl` 和 `reports/latest_splunk_app_package_manifest.html`。
在批准实时 Splunk 访问权限之前,可以在本地检查相同的查询意图:
```
python scripts\run_local_spl_query_pack.py
```
这将生成 `reports/latest_local_spl_query_results.html` 和 `.json`,作为事件时间线、根本原因证据、经人工批准的修复账本、Splunk MCP 调查上下文和爆炸半径查询都在生成的 CSV 上返回了具体数据行的证明。
## 提交赛道
主要赛道:
- 可观测性 (Observability)
次要相关性:
- 平台与开发者体验 (Platform & Developer Experience)
- 安全 (Security)
奖励目标:
- 最佳 Splunk MCP Server 应用 (Best Use of Splunk MCP Server)
MCP Remediation Ledger 为 AI 提议的事件响应操作提供了可审计性和护栏。
## 安全边界
当前代码库状态仅限本地。以下操作需要明确的用户批准:
- Splunk 账户、Splunk Cloud、Splunk Enterprise 或开发者许可证设置。
- 涉及凭证的 Splunk MCP Server 配置。
- 公开的 GitHub 代码库发布。
- 公开演示视频上传。
- 批准将公开 URL 回填到本地提交产物中。
- Devpost 注册、草稿保存或最终提交。
前检查门控 `scripts\verify_public_repo_publication_gate.py` 会记录确切的公开 GitHub 批准短语、源文件夹审查、隔离暂存确认、扫描确认、公开可见性确认,以及发布前明确的公开 git 身份。受保护的辅助脚本 `scripts\publish_public_repo_after_approval.py` 默认以本地演练模式运行。其实际执行模式受确切的公开 GitHub 批准短语和明确的公开 git 身份参数门控,并且仅应在审查了干净的公开候选版本、隔离的 TEMP 暂存区、扫描结果、发布前检查和发布回读计划之后使用。
## 许可证
用于公开提交的 Apache-2.0 候选版本。
标签:AI运维, IT运维, Socks5代理, 多模态安全, 模块化设计, 自动化修复, 请求拦截, 逆向工具