sami-tor/IntelForge-beta

# IntelForge-beta [](LICENSE) [](https://nextjs.org/) [](https://www.typescriptlang.org/) **Repository：** `IntelForge-beta` (公测版) **文档：** [`docs/`](docs/) · [安装指南](docs/SETUP.md) · [API 参考](docs/API-REFERENCE.md) · [架构](docs/CODEBASE-DEEP-STRUCTURE.md) ## 项目结构 ``` intelforge/ │ ├── src/ # All application source code │ ├── app/ # Next.js 16 App Router — pages & API routes │ │ ├── intelligence/ # 30+ threat intelligence modules │ │ ├── admin/ # Multi-tenant admin panel │ │ ├── api/ # REST API endpoints (auth, intel, admin, search) │ │ ├── dashboard/ # User dashboard │ │ ├── cases/ # Case/investigation management │ │ ├── reports/ # Report generation │ │ ├── search/ # Global search UI │ │ └── (auth pages) # Login, Register, Password reset │ ├── components/ # React component library │ │ ├── intelligence/ # Threat score gauge, forecast charts, geo-heatmap │ │ ├── search/ # Advanced search modals, archive viewer │ │ └── ui/ # Reusable UI primitives (shadcn/ui) │ ├── lib/ # Core backend logic │ │ ├── intel/ # Intelligence modules │ │ │ ├── automation/ # Action queue, correlator, briefing generator, threat score │ │ │ ├── connectors/ # WHOIS, DNS lookup connectors │ │ │ └── fetchers/ # CVE, ransomware, malware, phishing, darknet fetchers │ │ ├── integrations/ # MISP, SIEM formatter, webhook dispatcher │ │ ├── db.ts # PostgreSQL connection pool │ │ ├── auth.ts # JWT authentication (access + refresh tokens) │ │ ├── security.ts # Security middleware & audit logging │ │ └── validation.ts # Zod-based request validation schemas │ ├── hooks/ # Custom React hooks (useIsMobile, useToast) │ ├── styles/ # Global CSS styles │ ├── public/ # Static assets │ ├── tests/ # Node.js test suite (12 suites, 84 test cases) │ ├── proxy.ts # Next.js 16 proxy (middleware) — auth, rate limiting, security headers │ ├── next.config.mjs # Next.js configuration │ ├── tailwind.config.ts # Tailwind CSS configuration │ ├── tsconfig.json # TypeScript configuration │ ├── components.json # shadcn/ui component registry │ ├── postcss.config.mjs # PostCSS configuration │ └── AGENTS.md # Project conventions & learned preferences │ ├── scrapers/ # All scraping & data collection services │ ├── services/ # Python scraping services │ │ ├── face_scraper/ # Face detection, embedding, deduplication pipeline │ │ ├── image_indexer/ # Image embedding and Milvus indexing pipeline │ │ ├── threads_scraper/ # Threads/social media scraping utilities │ │ └── forum_monitor/ # Dark web forum monitoring service │ ├── visual-search/ # FastAPI visual/face search microservice │ │ └── service/app.py # Face search, visual search API endpoints │ ├── threadcoreface/ # Standalone face identification utility tool │ └── telegram-scraper/ # Standalone Telegram data scraper │ ├── indexer/ # All indexing & search services │ ├── cmd/ # Go microservices │ │ ├── search-api/ # REST search API (Quickwit/OpenSearch backend) │ │ └── indexer/ # Background indexing service │ ├── internal/ # Go internal packages (logger, search, security, etc.) │ ├── quickwit-config/ # Quickwit full-text search index schemas │ ├── indexer.py # Python standalone indexer │ └── auto-indexer.py # Python auto-indexing pipeline │ ├── data/ # All data — scraped intel, samples, cache │ ├── intel/ # Aggregated threat intelligence data │ │ └── dns_ip_cert/ # DNS, IP, certificate intelligence │ ├── scraped/ # Scraped data from various sources │ │ ├── forums/ # Dark web forum scrapes │ │ ├── github/ # GitHub secret scanning results │ │ ├── paste_sites/ # Paste site monitoring │ │ ├── reddit/ # Reddit OSINT │ │ ├── telegram/ # Telegram channel scrapes │ │ └── twitter/ # Twitter/X OSINT │ ├── leaks/ # Breach/leak data │ ├── stealer_logs/ # Stealer log data │ ├── ulp/ # User-level persistence data │ ├── database/ # SQLite/archive database files │ └── image/ # Sample/test images for visual search │ ├── userfiles/ # User-uploaded & generated files │ ├── uploads/ # User file uploads (CSV, JSON, etc.) │ ├── exports/ # Exported data, IOC dumps │ ├── reports/ # Generated PDF/HTML reports │ └── profiles/ # User profile images, avatars │ ├── scripts/ # Database migrations, seed data, utilities ├── docs/ # Setup, API reference, architecture docs ├── diagram/ # PlantUML architecture diagrams └── docker-compose.quickwit.yml # Quickwit search cluster Docker compose ``` ## 技术栈 | 层级 | 技术 | |---|---| | 前端 | Next.js 16 (App Router), React 19, TypeScript, Tailwind CSS 4 | | API | Next.js API Routes (REST), Go (search-api) | | 数据库 | PostgreSQL 16 (情报缓存表) | | 搜索 | Quickwit (全文), Milvus (向量/视觉) | | 认证 | JWT (access + refresh), 2FA (speakeasy), 响应签名 | | 视觉搜索 | FastAPI (Python), 人脸 embedding 模型 | | 爬虫 | Python (Threads, Telegram, 论坛, GitHub 凭据) | | 图表 | Recharts, 自定义威胁评分仪表盘 | ## 快速开始 查看完整指南：**[docs/SETUP.md](docs/SETUP.md)** ``` # 1. 安装依赖 npm install --legacy-peer-deps # 2. 复制环境模板（切勿提交 .env.local） cp .env.example .env.local # 3. 生成密钥并在 .env.local 中配置 DATABASE_URL npm run env:secrets # 4. 应用数据库迁移 npm run db:migrate # 5. 启动开发服务器 npm run dev ``` 打开 [http://localhost:3000](http://localhost:3000) ``` # Production build npm run build && npm start # 运行测试套件 npm run defence:test ``` ## 核心情报模块 (30+) | 模块 | 描述 | |---|---| | **指挥中心** | 实时全球威胁评分，自动关联的集群 | | **行动队列** | 自动生成、已确定优先级的分析师任务 | | **高管简报** | 每日 AI 生成的威胁报告 | | **CVE 情报** | EPSS 评分、KEV 标记、CVSS 严重程度 | | **勒索软件追踪器** | 活跃组织、受害者披露、行业分析 | | **IOC 查询** | IP、域名、哈希、URL 信誉检查 | | **暗网监控** | 勒索软件博客与论坛监控 | | **钓鱼情报** | OpenPhish 与 PhishTank 品牌冒充追踪 | | **供应链情报** | OSV.dev 依赖项漏洞数据 | | **GitHub 凭据** | 暴露的 API key 与凭据扫描 | | **漏洞优先级排序** | 综合风险评分 (CVSS + EPSS + KEV) | | **检测覆盖率** | 映射到 Sigma/YARA 规则的 MITRE ATT&CK | | **关系图谱** | 交互式的 actor–malware–CVE 图谱 | | **AI 分析师工作区** | 针对所有情报的自然语言 CTI 查询 | | **攻击面** | 证书透明度、域名误植、凭据、钓鱼 | ## 数据库表 (情报缓存) `intel_news_cache` · `intel_cve_cache` · `intel_ransomware_groups` · `intel_ransomware_victims` · `intel_malware_cache` · `intel_mitre_groups` · `intel_mitre_techniques` · `intel_phishing_cache` · `intel_darknet_posts` · `intel_yara_rules` · `intel_feed_sync_log` ## 管理员访问 运行迁移 (`npm run db:migrate`) 后，将植入一个默认管理员账号。 首次登录后，请**立即更改默认密码**。 - 管理面板：`/admin-portal` 或 `/admin` - 设置详情：[docs/SETUP.md#admin-account](docs/SETUP.md#admin-account) ## 文档索引 | 文档 | 描述 | |----------|-------------| | [docs/SETUP.md](docs/SETUP.md) | 安装、Docker、环境变量、首次运行 | | [docs/API-REFERENCE.md](docs/API-REFERENCE.md) | 所有 REST API (137 个路由) | | [docs/CODEBASE-DEEP-STRUCTURE.md](docs/CODEBASE-DEEP-STRUCTURE.md) | 完整架构深度剖析 | | [docs/GITHUB-PUBLISH-PLAN.md](docs/GITHUB-PUBLISH-PLAN.md) | 本仓库是如何为 GitHub 准备的 | ## 许可证 MIT 许可证 — 见 [LICENSE](LICENSE)。仅限用于授权的安全研究和教育用途。

标签：ESC4, OSINT, PostgreSQL, 威胁情报, 安全分析平台, 开发者工具, 日志审计, 测试用例, 自动化攻击, 请求拦截, 逆向工具