Henrinnes/RansKnow

# RansKnow：勒索软件知识数据集 这是一个精选的数据集，包含来自 **50 个网络安全 YouTube 频道** 的 **440 个视频字幕**，旨在支持专注于勒索软件的知识提取、NLP 研究和威胁情报分析。 ## 背景 勒索软件攻击是破坏性最大的网络威胁类别之一，然而关于其战术、工具和受影响平台的知识却散布于各类会议演讲、应急响应报告和安全简报中。RansKnow 系统地从公开的视频内容中收集并结构化这些知识。 视频是基于关键词包含标准（与勒索软件相关的术语）进行筛选的，经过技术深度的过滤，并通过 Knowledge Agent pipeline 进行处理，以提取与 MITRE ATT&CK 框架对齐的结构化特征。 ## 仓库结构 ``` RansKnow/ ├── transcripts/ # 440 videos across 50 channels │ ├── C01_The_DFIR_Report/ │ │ ├── V0001.txt # Raw transcript (YouTube timestamped format) │ │ ├── V0001.meta.json # Video metadata │ │ └── ... │ ├── C02_SANS_Digital_Forensics_and_Incident_Response/ │ └── ... (C01–C50) │ ├── outputs/ # Knowledge Agent extraction results │ ├── Knowledge_Agent_Features_307.csv │ ├── Knowledge_Agent_Features_307.xlsx │ ├── Knowledge_Agent_Output.csv │ └── Knowledge_Agent_Uncertainty_Distribution.png │ ├── RansKnow_v1/ │ └── Knowledge_Agent_Features_v1.csv # v1 feature CSV (307 rows) │ ├── Scripts/ # Data pipeline notebooks and scripts │ ├── 01_Transcript_Dataset_Construction.ipynb │ ├── fetch_transcripts_keywords.ipynb │ ├── Data_extraction_inclusion.ipynb │ ├── Inclusin_Criteria_2.ipynb │ ├── Knowledge_Agent_Modelling.ipynb │ ├── fill_video_selection_rubric_openpyxl.py │ └── populate_video_registry_from_transcripts.py │ ├── rubrics/ # Scoring rubrics and channel registry │ ├── Dataset_Channel_Registry_Populated_25.xlsx │ ├── Dataset_Channel_Registry_Updated_50_fixed_urls.xlsx │ ├── Ransomware_Family_Coverage_List.xlsx │ └── Video_Selection_Rubric_*.xlsx │ ├── Figures/ # Visualisations ├── Progress_Mapping/ # Weekly progress tracking ├── Channel_Registry_1.xlsx # Master registry of all 50 channels ├── Family_Coverage_Targets_Rules.docx ├── Ransomware_Transcript_Dataset_Summary.pdf └── Video_Selection_Rubric_AutoScore_Filled_6.xlsx ``` ## Knowledge Agent 特征 Schema 440 个视频中有 307 个已经过 Knowledge Agent pipeline 的全面处理，生成了 **33 个结构化特征列**： | 类别 | 列名 | |---|---| | **标识符** | `Video_ID`, `Channel_ID`, `Channel_Name`, `Video_Title`, `YouTube_URL`, `Transcript_Path` | | **勒索软件家族** | `Family_Count`, `Family_List` | | **MITRE ATT&CK 战术** | `Tactic_Initial_Access`, `Tactic_Execution`, `Tactic_Persistence`, `Tactic_Privilege_Escalation`, `Tactic_Credential_Access`, `Tactic_Lateral_Movement`, `Tactic_Discovery`, `Tactic_Command_and_Control`, `Tactic_Exfiltration`, `Tactic_Impact`, `Tactic_Total_Mentions`, `Dominant_Tactic` | | **工具** | `Tool_Cobalt_Strike`, `Tool_Mimikatz`, `Tool_PsExec`, `Tool_Rclone`, `Tool_MegaNZ`, `Tool_AnyDesk`, `Tool_TeamViewer`, `Tool_BloodHound`, `Tool_Total_Mentions`, `Tool_List` | | **平台** | `Platform_Signal`, `Platform_Windows`, `Platform_Linux`, `Platform_ESXi` | ## 涵盖频道 (C01–C50) | ID | 频道 | |---|---| | C01 | The DFIR Report | | C02 | SANS Digital Forensics and Incident Response | | C03 | Black Hat | | C04 | DEF CON Conference | | C05 | CrowdStrike | | C06 | Mandiant / Google Cloud Security | | C07 | Sophos X-Ops | | C08 | Red Canary | | C09 | Huntress | | C10 | John Hammond | | C11 | Secureworks | | C12 | Kaspersky | | C13 | Palo Alto Networks Unit 42 | | C14 | Elastic Security | | C15 | RSA Conference | | C16 | USENIX Security | | C17 | FIRST Conference | | C18 | Malware Analysis for Hedgehogs | | C19 | OALabs | | C20 | LiveOverflow | | C21 | CyberWire | | C22 | Threatpost | | C23 | Microsoft Security | | C24 | Cisco Talos Intelligence Group | | C25 | Recorded Future | | C26 | MalwareTech | | C27 | VX-Underground | | C28 | Black Hills Information Security | | C29 | Security Onion Solutions | | C30 | SANS Institute | | C31 | Mandiant | | C32 | Sophos | | C33 | ESET | | C34 | Elastic | | C35 | Splunk | | C36 | Wazuh | | C37 | TrustedSec | | C38 | Blue Team Village | | C39 | MITRE ATT&CK | | C40 | Magnet Forensics | | C41 | Belkasoft | | C42 | DFIR Science | | C43 | Active Countermeasures | | C44 | VMware Carbon Black | | C45 | Arctic Wolf | | C46 | Dragos Inc | | C47 | Cybereason | | C48 | ThreatLocker | | C49 | LogRhythm | | C50 | Darktrace | 大多数频道各贡献了 **10 个视频**，这些视频的选择旨在最大化涵盖勒索软件家族和技术深度。 ## 字幕格式 每个 `.txt` 文件都包含带有时间戳格式的原始 YouTube 字幕： ``` 0:00 Welcome to the DFIR report... 0:05 Today we are covering a LockBit intrusion... ``` 每个 `.meta.json` 文件包含结构化的元数据： ``` { "Video_ID": "V0001", "Channel_ID": "C01", "Channel_Name": "The DFIR Report", "Channel_UC": "UC6R2MPMkkCqFxvAdQAI_23A", "YouTube_Video_ID": "xxxxxx", "Video_Title": "...", "Year": 2024, "PublishedAt": "2024-03-15T14:00:00Z", "DurationSeconds": 1843, "DurationISO": "PT30M43S", "Matched_Keywords": "ransomware;ir", "Transcript_Available": true } ``` ## 预期用例 - 勒索软件威胁情报研究 - 针对网络安全文本的 NLP 和信息提取 - MITRE ATT&CK 战术分类 - 工具和平台归属建模 - 勒索软件知识图谱构建 - 网络安全教育的课程开发 ## Kaggle 上的数据集 完整的数据集（包括字幕压缩包、脚本和评分标准）也可在 Kaggle 上获取： [https://www.kaggle.com/datasets/henrykabuye/ransknow-v1](https://www.kaggle.com/datasets/henrykabuye/ransknow-v1) ## 版本历史 | 版本 | 日期 | 描述 | |---|---|---| | v1 | 2026 年 5 月 | 初始发布 — Knowledge Agent CSV（307 行） | | v2 | 2026 年 7 月 | 完整数据集 — 440 个字幕、所有脚本、评分标准、图表和文档 | ## 许可证 本数据集根据 [CC0 1.0 Universal (Public Domain)](https://creativecommons.org/publicdomain/zero/1.0/) 许可证发布。

标签：Homebrew安装, NoSQL, 勒索软件, 威胁情报, 开发者工具, 知识抽取, 逆向工具