murdok1982/Global-Intelligence

## 📋 目录 - [执行摘要](#-executive-summary) - [架构](#-architecture) - [功能特性](#-capabilities) - [安装说明](#-installation) - [配置](#-configuration) - [使用指南](#-usage) - [API 参考](#-api-reference) - [安全性](#-security) - [合规性](#-compliance) ## 🎯 执行摘要 **Global Intelligence** 是一个开源情报平台，专为需要**完全数据主权**的政府机构、武装部队和情报机构设计。 ``` ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ PROBLEM: Las plataformas de IA envían datos a proveedores │ │ externos → INCOMPATIBLE con información clasificada │ │ │ │ SOLUTION: Inferencia 100% local · LLM soberano · Air-gap ready │ │ │ │ RESULT: Análisis de inteligencia sin ceder soberanía │ │ │ └─────────────────────────────────────────────────────────────────────┘ ``` ### 核心优势 | 特性 | 描述 | |---------|-------------| | 🔐 **零外部数据泄露** | 所有处理均在本地进行。任何机密数据都不会离开内部网络边界。 | | 🧠 **ATALAYA LLM** | 专注于军事分析的 LLM，采用北约 PMESII-PT 方法论。 | | ⚔️ **军事数据库** | 包含 30 多种真实武器系统（F-35、Su-57、Leopard 2、Javelin 等），数据来源于 SIPRI/Jane's。 | | 📡 **9 大 OSINT 数据源** | RSS、YouTube、GDELT、SIPRI、FININT、CYBINT、GEOINT、SIGINT、ACLED。 | | 🗺️ **地理空间情报** | 交互式 Leaflet 地图，显示军事基地、冲突和武器转让。 | | 📱 **移动端就绪** | Flutter App，支持离线模式、生物特征身份验证和远程擦除。 | | 🚨 **早期预警** | 具有可配置阈值的自动化警报系统。 | | 📊 **预测性 ML** | 时间序列分析、态势升级检测和风险评分。 | ## 🏗️ 架构 ### 系统概览 ``` graph TB subgraph USER_LAYER["👤 USER LAYER"] WEB[Web Frontend
Next.js 14] MOBILE[Mobile App
Flutter] API_CLIENT[API Clients
REST/GraphQL] end subgraph GATEWAY["🔒 SECURITY GATEWAY"] AUTH[JWT + MFA
TOTP/WebAuthn] RLS[Row-Level Security
PostgreSQL] AUDIT[Audit Log
Hash Chain] end subgraph BACKEND["⚙️ BACKEND SERVICES"] FASTAPI[FastAPI
REST API] CELERY[Celery Workers
Async Tasks] subgraph AGENTS["🤖 INTELLIGENCE AGENTS"] OSINT_AGENT[OSINT Agent
9 Providers] SYNTHESIS[Synthesis Agent
ATALAYA LLM] SCENARIO[Scenario Agent
What-If] EAGLE[EagleEye
Satellite] MONEY[MoneyTrail
Financial] CYBER[CyberSentinel
Cyber Intel] NARRATIVE[NarrativeWatch
Disinfo] WARNING[EarlyWarning
Alerts] end subgraph SERVICES["🔧 CORE SERVICES"] PREDICTION[Prediction ML
Time Series] SNA[Network Analysis
Graph SNA] RAG[RAG Service
pgvector] EXPORT[Export Service
PDF/DOCX] EDGE[Edge Sync
Field Devices] ALERT[Alert System
Thresholds] end end subgraph DATA_LAYER["💾 DATA LAYER"] POSTGRES[(PostgreSQL 15
+ pgvector
+ RLS)] REDIS[(Redis
Cache/Queue)] subgraph MILITARY_DB["⚔️ MILITARY DATABASE"] WEAPONS[Weapon Systems
30+ entries] TRANSFERS[Arms Transfers
SIPRI data] BASES[Military Bases
Geolocated] BUDGETS[Defense Budgets
23 countries] end end subgraph LLM_LAYER["🧠 LLM INFERENCE"] OLLAMA[Ollama Local
ATALAYA Profile] VLLM[vLLM
Optional] OPENROUTER[OpenRouter
PUBLIC only] end subgraph OSINT_SOURCES["📡 OSINT SOURCES"] RSS[30 RSS Feeds
Reuters, BBC...] YT[10 YouTube
CSIS, RAND...] GDELT[GDELT v2
Global Events] SIPRI[SIPRI API
Arms Transfers] FININT_API[World Bank
Exchange Rates] CYBINT_API[Shodan, GreyNoise
CISA KEV] GEOINT_API[Sentinel-2, USGS
NASA EONET] SIGINT_API[AIS, ADS-B
OpenSky] ACLED_API[ACLED
Conflicts] end WEB --> AUTH MOBILE --> AUTH API_CLIENT --> AUTH AUTH --> FASTAPI FASTAPI --> AGENTS FASTAPI --> SERVICES AGENTS --> CELERY SERVICES --> CELERY CELERY --> POSTGRES CELERY --> REDIS CELERY --> LLM_LAYER OSINT_AGENT --> OSINT_SOURCES POSTGRES --> MILITARY_DB FASTAPI --> RLS FASTAPI --> AUDIT style USER_LAYER fill:#1a1a2e,color:#fff,stroke:#4a4a8a style GATEWAY fill:#2e1a1a,color:#fff,stroke:#8a2a2a style BACKEND fill:#0d2137,color:#fff,stroke:#1e5f8a style DATA_LAYER fill:#1a3a1a,color:#fff,stroke:#2a8a2a style LLM_LAYER fill:#2e2a1a,color:#fff,stroke:#8a7a2a style OSINT_SOURCES fill:#3a1a3a,color:#fff,stroke:#8a2a8a ``` ### 数据流 ``` sequenceDiagram participant U as Usuario participant API as FastAPI participant AG as Agentes participant LLM as ATALAYA LLM participant DB as PostgreSQL participant OSINT as OSINT Sources U->>API: Solicitud de análisis API->>API: Auth + MFA + RLS API->>AG: Dispatch a agentes par Parallel Execution AG->>OSINT: Collect signals (9 providers) OSINT-->>AG: Raw intelligence data AG->>DB: Query military database DB-->>AG: Weapon/transfer data end AG->>LLM: Synthesis + Analysis LLM-->>AG: Structured report AG->>DB: Store results DB-->>API: Response API-->>U: Intelligence product ``` ## ⚡ 功能特性 ### 情报收集矩阵 ``` ┌─────────────────────────────────────────────────────────────────────┐ │ INTELLIGENCE DISCIPLINES │ ├──────────────┬──────────────────────────────────────────────────────┤ │ DISCIPLINE │ SOURCES & CAPABILITIES │ ├──────────────┼──────────────────────────────────────────────────────┤ │ 📰 OSINT │ 30 RSS (Reuters, BBC, FT, RAND, SIPRI, Jane's) │ │ │ 10 YouTube (CSIS, Stratfor, CFR, Chatham House) │ │ │ GDELT v2 (global events, media, tone) │ ├──────────────┼──────────────────────────────────────────────────────┤ │ ⚔️ MILINT │ 30+ Weapon Systems (F-35, Su-57, J-20, Leopard 2) │ │ │ Arms Transfers (SIPRI database) │ │ │ Military Bases (geolocated, 8 strategic sites) │ │ │ Defense Budgets (23 countries, 2024 data) │ ├──────────────┼──────────────────────────────────────────────────────┤ │ 📡 SIGINT │ AIS Maritime (vessel tracking via AISHub) │ │ │ ADS-B Aircraft (OpenSky Network) │ │ │ SDR Spectrum (RTL-SDR, satellite signals) │ ├──────────────┼──────────────────────────────────────────────────────┤ │ 💰 FININT │ World Bank GDP/indicators │ │ │ Exchange rates (Open ER-API) │ │ │ Sanctions lists (OFAC, EU, UN) │ │ │ Commodity prices (oil, gas, wheat, rare metals) │ ├──────────────┼──────────────────────────────────────────────────────┤ │ 🌐 GEOINT │ Sentinel-2 (ESA satellite imagery) │ │ │ USGS Earthquakes (real-time seismic data) │ │ │ NASA EONET (natural events tracking) │ ├──────────────┼──────────────────────────────────────────────────────┤ │ 🔒 CYBINT │ Shodan (infrastructure exposure) │ │ │ GreyNoise (mass scanning detection) │ │ │ CISA KEV (known exploited vulnerabilities) │ ├──────────────┼──────────────────────────────────────────────────────┤ │ 📊 ANALYTICS │ SNA (NATO, CSTO, BRICS, SCO, AUKUS graphs) │ │ │ Predictive ML (time series, escalation detection) │ │ │ Risk scoring (multi-signal anomaly detection) │ ├──────────────┼──────────────────────────────────────────────────────┤ │ 📱 SOCIAL │ Telegram (public channels via Telethon) │ │ │ Reddit (subreddit monitoring) │ │ │ Twitter/X (API v2 if token available) │ └──────────────┴──────────────────────────────────────────────────────┘ ``` ### 军事数据库 ``` ┌─────────────────────────────────────────────────────────────────────┐ │ WEAPON SYSTEMS CATALOG │ ├──────────────────┬──────────────────────────────────────────────────┤ │ CATEGORY │ SYSTEMS │ ├──────────────────┼──────────────────────────────────────────────────┤ │ ✈️ Aircraft │ F-35A, Su-57, J-20, Eurofighter, Rafale │ │ 🚁 Helicopters │ AH-64 Apache, Mi-28 Havoc │ │ 🛩️ UAV/Drones │ MQ-9 Reaper, Bayraktar TB2, Shahed-136 │ │ 🛡️ Tanks │ M1A2 Abrams, Leopard 2A7, T-90M, Challenger 3 │ │ 💥 Artillery │ CAESAR, PzH 2000, HIMARS │ │ 🎯 Missiles │ Javelin, Patriot PAC-3, S-400, Tomahawk │ │ 🚢 Ships │ Ford-class CVN, Type 45 DDG, Astute SSN │ │ 📡 EW Systems │ Krasukha-4, SPECTRA │ │ ☢️ CBRN │ TOS-1A Solntsepyok │ │ 🔫 Small Arms │ HK416, AK-74M │ └──────────────────┴──────────────────────────────────────────────────┘ ``` ### 数据源验证 ``` ┌─────────────────────────────────────────────────────────────────────┐ │ SOURCE RELIABILITY (NATO STANAG 2511) │ ├──────────┬──────────────────────────────────────────────────────────┤ │ A (Full) │ Reuters, BBC, FT, RAND, SIPRI, IISS, Jane's, CSIS, │ │ │ Chatham House, CFR, Brookings, AP News, El País │ ├──────────┼──────────────────────────────────────────────────────────┤ │ B (Gen) │ Bellingcat, Middle East Eye, Real Vision │ ├──────────┼──────────────────────────────────────────────────────────┤ │ C (Mod) │ RT, Xinhua, TASS (state media, cross-verify) │ └──────────┴──────────────────────────────────────────────────────────┘ ``` ## 🚀 安装说明 ### 前置条件 ``` ┌─────────────────────────────────────────────────────────────────────┐ │ SYSTEM REQUIREMENTS │ ├──────────────────┬──────────────────────────────────────────────────┤ │ OS │ Linux (Ubuntu 22.04+), macOS, Windows (WSL2) │ │ CPU │ 4+ cores (8+ recommended) │ │ RAM │ 16 GB minimum (32 GB recommended) │ │ Storage │ 50 GB SSD (100 GB for full OSINT cache) │ │ GPU │ Optional: NVIDIA with CUDA 12+ (10x speedup) │ │ Docker │ 24.0+ with Compose plugin │ │ Python │ 3.11+ │ │ Node.js │ 20+ (for frontend) │ └──────────────────┴──────────────────────────────────────────────────┘ ``` ### 快速开始 ``` # Clone repository git clone https://github.com/murdok1982/Global-Intelligence.git cd Global-Intelligence # 配置环境 cp backend/.env.example backend/.env # 使用您的 secrets 编辑 backend/.env # 启动所有服务 docker compose up -d --build # 拉取 LLM model docker compose exec ollama ollama pull llama3.1:8b-instruct-q4_K_M # 创建 ATALAYA profile (military analyst) docker compose exec ollama ollama create atalaya-geoint -f osint_pipeline/ollama_geoint.modelfile # 运行数据库 migrations docker compose exec backend alembic upgrade head # 填充 military database docker compose exec backend python -m app.db.seeds.seed_military # 访问平台 # Web UI: http://localhost:3000 # API: http://localhost:8000/api/v1/ # API Docs: http://localhost:8000/docs ``` ### 手动安装 ``` # Backend cd backend python -m venv venv source venv/bin/activate # Linux/Mac # venv\Scripts\activate # Windows pip install -r requirements.txt # 配置环境 cp .env.example .env # 使用您的 secrets 编辑 .env # Database # 确保已安装并运行带有 pgvector extension 的 PostgreSQL 15+ export DATABASE_URL="postgresql+asyncpg://user:pass@localhost:5432/global_intelligence" alembic upgrade head python -m app.db.seeds.seed_military # 启动 backend uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload # Frontend (separate terminal) cd ../frontend npm install npm run dev ``` ### 移动应用 ``` cd mobile_app flutter pub get flutter run ``` ## ⚙️ 配置 ### 环境变量 ``` # ═══════════════════════════════════════════════════════════════════ # 安全 (必需) # ═══════════════════════════════════════════════════════════════════ SECRET_KEY= REFRESH_SECRET_KEY= POSTGRES_PASSWORD= # ═══════════════════════════════════════════════════════════════════ # STATE-GRADE MODE (classified operations 必需) # ═══════════════════════════════════════════════════════════════════ STATE_GRADE_MODE=true MFA_ENCRYPTION_KEY= MFA_CHALLENGE_SECRET= ENABLE_OPENROUTER_FALLBACK=false # NEVER enable for classified data # ═══════════════════════════════════════════════════════════════════ # LLM 提供商 # ═══════════════════════════════════════════════════════════════════ OLLAMA_BASE_URL=http://ollama:11434 OLLAMA_MODEL=atalaya-geoint # or llama3.1:8b-instruct-q4_K_M # ═══════════════════════════════════════════════════════════════════ # OSINT 提供商 (可选 - 增强 data collection) # ═══════════════════════════════════════════════════════════════════ NEWSAPI_API_KEY= SHODAN_API_KEY= GREYNOISE_API_KEY= ACLED_API_KEY= ACLED_EMAIL= TWITTER_BEARER_TOKEN= TELEGRAM_API_ID= TELEGRAM_API_HASH= COPERNICUS_USERNAME= COPERNICUS_PASSWORD= # ═══════════════════════════════════════════════════════════════════ # 报告签名 (Ed25519) # ═══════════════════════════════════════════════════════════════════ SIGNING_PRIVATE_KEY_PATH=/var/lib/gi/signing/ed25519_private.pem SIGNING_PUBLIC_KEY_PATH=/var/lib/gi/signing/ed25519_public.pem ``` ### 生成签名密钥 ``` mkdir -p /var/lib/gi/signing openssl genpkey -algorithm Ed25519 -out /var/lib/gi/signing/ed25519_private.pem openssl pkey -in /var/lib/gi/signing/ed25519_private.pem -pubout -out /var/lib/gi/signing/ed25519_public.pem ``` ## 📖 使用指南 ### Web 界面 1. 在 `http://localhost:3000/auth` **登录** 2. **仪表盘** — 包含风险指标的全局概览 3. **军事情报** — 武器系统、武器转让、军事基地 4. **报告** — 每日情报综合分析 5. **情景模拟** — 使用 ATALAYA 进行假设分析 ### API 示例 ``` # 列出 weapon systems curl -H "Authorization: Bearer $TOKEN" \ http://localhost:8000/api/v1/military/weapons?category=aircraft # 获取 arms transfers curl -H "Authorization: Bearer $TOKEN" \ http://localhost:8000/api/v1/military/transfers?recipient_iso=UA&year_from=2022 # 列出 military bases curl -H "Authorization: Bearer $TOKEN" \ http://localhost:8000/api/v1/military/bases?strategic_only=true # 生成 intelligence report curl -X POST -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"country_iso": "UA", "days_back": 7}' \ http://localhost:8000/api/v1/reports/generate # 将报告导出为 PDF curl -H "Authorization: Bearer $TOKEN" \ http://localhost:8000/api/v1/export/report/{id}/pdf ``` ### OSINT Pipeline（独立运行） ``` cd osint_pipeline ./setup.sh # Install Ollama + ATALAYA profile python main.py # Generate weekly bilingüal report ``` ## 🔌 API 参考 ### Endpoints | Method | Endpoint | Description | |--------|----------|-------------| | `GET` | `/api/v1/military/weapons` | 列出武器系统 | | `GET` | `/api/v1/military/weapons/{id}` | 获取武器详情 | | `GET` | `/api/v1/military/transfers` | 列出武器转让记录 | | `GET` | `/api/v1/military/bases` | 列出军事基地 | | `GET` | `/api/v1/military/budgets` | 国防预算 | | `GET` | `/api/v1/military/stats` | 全球军事统计数据 | | `POST` | `/api/v1/reports/generate` | 生成情报报告 | | `GET` | `/api/v1/export/report/{id}/pdf` | 导出为 PDF | | `GET` | `/api/v1/export/report/{id}/docx` | 导出为 DOCX | | `POST` | `/api/v1/export/offline-package` | 准备离线数据 | ## 🔒 安全性 ``` ┌─────────────────────────────────────────────────────────────────────┐ │ SECURITY ARCHITECTURE │ ├─────────────────────────────────────────────────────────────────────┤ │ │ │ 🔐 AUTHENTICATION │ │ • JWT (HS256) with 15-min access tokens │ │ • Refresh tokens (7-day) with secure cookie storage │ │ • MFA mandatory (TOTP + WebAuthn optional) │ │ • Biometric auth (mobile) │ │ │ │ 🛡️ AUTHORIZATION │ │ • Row-Level Security (PostgreSQL) │ │ • Classification levels: PUBLIC → RESTRICTED → CONFIDENTIAL │ │ • Organization-based compartmentation │ │ • Clearance enforcement at DB level │ │ │ │ 📝 AUDIT │ │ • Immutable audit log with SHA-256 hash chain │ │ • Every access logged (who, what, when, from where) │ │ • Tamper-evident (deletion/modification detectable) │ │ │ │ 🔏 CRYPTOGRAPHY │ │ • Ed25519 report signing (non-repudiation) │ │ • AES-256-GCM (MFA secrets at rest) │ │ • bcrypt password hashing │ │ • Invisible watermarks on exports │ │ │ │ 🌐 NETWORK │ │ • SSRF protection (block private IPs) │ │ • CORS strict policy │ │ • Rate limiting (slowapi) │ │ • TLS termination at reverse proxy │ │ │ │ 🚫 DATA SOVEREIGNTY │ │ • LLM inference 100% local (Ollama/vLLM) │ │ • External providers forbidden for CONFIDENTIAL+ │ │ • Fail-closed: 503 if local LLM unavailable │ │ • Air-gap compatible │ │ │ └─────────────────────────────────────────────────────────────────────┘ ``` ## 📜 合规性 | 框架 | 状态 | 备注 | |-----------|--------|-------| | **ENS (西班牙)** | ALTA | 国家安全方案 | | **ISO 27001:2022** | Ready | 信息安全管理 | | **NIST 800-53 Rev.5** | Ready | 美国联邦安全控制 | | **FedRAMP** | Compatible | 云安全授权 | | **STANAG 2511** | Native | 北约 Admiralty 代码 | | **TLP v2.0** | Native | 交通信号灯协议 | | **GDPR** | Compliant | 欧盟数据保护 | ## 📞 联系方式 **General Murdok** (Gustavo Lobato Clara) 📧 gustavolobatoclara@gmail.com 🔗 [GitHub](https://github.com/murdok1982) 🔗 [LinkedIn](https://www.linkedin.com/in/gustavo-lobato-clara-2b446b102/)

``` ╔══════════════════════════════════════════════════════════════════╗ ║ ║ ║ ██╗ ██╗ █████╗ ███████╗████████╗███████╗██████╗ ║ ║ ██║ ██║██╔══██╗██╔════╝╚══██╔══╝██╔════╝██╔══██╗ ║ ║ ███████║███████║███████╗ ██║ █████╗ ██████╔╝ ║ ║ ██╔══██║██╔══██║╚════██║ ██║ ██╔══╝ ██╔══██╗ ║ ║ ██║ ██║██║ ██║███████║ ██║ ███████╗██║ ██║ ║ ║ ╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝ ╚═╝ ╚══════╝╚═╝ ╚═╝ ║ ║ ║ ║ S T A Y I N F O R M E D · S T A Y A H E A D ║ ║ ║ ╚══════════════════════════════════════════════════════════════════╝ ``` *机密 · 主权 · 可运行*

## 支持 / 赞助本项目 我致力于开发专注于应用 AI、自动化和数据情报的开源项目。 在我的 GitHub 上，你会发现诸如 AI 驱动的分析引擎、用于开源研究的 OSINT 平台、Windows 自动化工具以及语言模型实验等项目。 所有内容都是公开免费的，任何人都可以使用、学习或在此基础上进行构建。github.com/murdok1982 维持这些项目的运转需要耗费大量时间。如果其中任何一个项目对你有所帮助，或者你只是喜欢我正在做的事情，你可以请我喝杯咖啡来支持我：ko-fi.com/murdok1982 每一份贡献都将直接用于发布更多的开源代码。