phani-kb/dns-toolkit

GitHub: phani-kb/dns-toolkit

一款基于 Go 的 DNS 屏蔽列表生成与管理工具,支持从多来源下载、解析、整合并输出兼容 Pi-hole、AdGuard Home 等平台的白名单和屏蔽列表。

Stars: 5 | Forks: 0

## dns-toolkit ![dns-toolkit logo](https://static.pigsec.cn/wp-content/uploads/repos/cas/ec/ec60fac6db04cbbeff67860fa97d904923f964a15e3037b15525ea8ea2ac6db6.png) [![Go CI Workflow](https://static.pigsec.cn/wp-content/uploads/repos/cas/20/206def293b1092faf7b224bfa50ab7cda3bc662c3b9f5d2d99a547df7384482f.svg)](https://github.com/phani-kb/dns-toolkit/actions/workflows/ci.yml) [![Go Report Card](https://goreportcard.com/badge/github.com/phani-kb/dns-toolkit)](https://goreportcard.com/report/github.com/phani-kb/dns-toolkit) [![codecov](https://codecov.io/gh/phani-kb/dns-toolkit/branch/release/1.0.0/graph/badge.svg)](https://codecov.io/gh/phani-kb/dns-toolkit) [![GoDoc](https://godoc.org/github.com/phani-kb/dns-toolkit?status.svg)](https://godoc.org/github.com/phani-kb/dns-toolkit) [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0) [![Go Version](https://img.shields.io/badge/Go-1.24-blue.svg)](https://golang.org/doc/go1.24) [![GitHub commit activity](https://img.shields.io/github/commit-activity/y/phani-kb/dns-toolkit)](https://github.com/phani-kb/dns-toolkit/graphs/commit-activity) [![GitHub repo size](https://img.shields.io/github/repo-size/phani-kb/dns-toolkit)](https://github.com/phani-kb/dns-toolkit) **高性能的 Go DNS 屏蔽列表 pipeline** DNS Toolkit 是一个基于 Go 的命令行实用程序,用于从多个来源下载、处理、解析和整合屏蔽列表(blocklist)和白名单(allowlist)。它执行 DNS 到 IP 的解析、反向查找以及重叠检测。生成可随时用于 DNS sinkhole、Pi-hole、AdGuard Home 和其他网络安全工具的列表。 ## Pipeline 流程 ``` --- config: theme: mc --- flowchart LR B["Process
via Processors"] --> C["Consolidate
Group
Categorize
Top Entries
Overlaps"] C --> D["Blocklist/Allowlist Files"] & E["Main README"] & F["Overlap Analysis"] & G["Summary Report"] & H["Statistics Report"] & n2["Format Conversion
via Converters"] & n4["Graph Generation
via Graphers"] D --> I["Archive"] E --> I F --> I H --> I G --> I n1["Data files"] --> B A["Download
via Downloaders"] --> n1 n2 --> n3["Converted Files"] n4 --> n5["Graph Images"] n3 --> I n5 --> I B@{ shape: procs} C@{ shape: h-cyl} D@{ shape: docs} n2@{ shape: h-cyl} n4@{ shape: paper-tape} n1@{ shape: docs} n3@{ shape: docs} style C stroke-width:4px,stroke-dasharray: 0 style n2 stroke-width:2px,stroke-dasharray: 2 ``` ## 发布的输出 **可直接使用的屏蔽列表文件每天发布到 [`output`](https://github.com/phani-kb/dns-toolkit/tree/output) 分支:** - 兼容 Pi-hole、pfBlockerNG、AdGuard Home 的域名和 IP 屏蔽列表/白名单 - 按大小(mini、lite、normal、big)和类别(广告、恶意软件、隐私)组织的列表 - 根据来源频率排名的顶级条目,提供高置信度的拦截 **⚡ 快速使用:** 将 `https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/[filename]` 添加到您的 DNS 过滤工具(Pi-hole、AdGuard Home、pfBlockerNG)中。 最常用的输出;点击任意链接即可打开文件。 | 类别 | 文件(点击打开) | | -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 🗂️ 合并列表 | 🛑 **屏蔽列表:** [adguard](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/adguard_blocklist.txt) · [cidr_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/cidr_ipv4_blocklist.txt) · [domain](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/domain_blocklist.txt) · [ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/ipv4_blocklist.txt) · [ipv6](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/ipv6_blocklist.txt)
✅ **白名单:** [adguard (白名单)](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/adguard_allowlist.txt) · [domain (白名单)](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/domain_allowlist.txt) · [ipv4 (白名单)](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/ipv4_allowlist.txt) | | 📏 Mini — 低误报率 🟢 | [mini_adguard](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/mini_adguard_blocklist.txt), [mini_cidr_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/mini_cidr_ipv4_blocklist.txt), [mini_domain](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/mini_domain_blocklist.txt), [mini_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/mini_ipv4_blocklist.txt) | | 📏 Lite — 均衡防护 🟡 | [lite_adguard](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/lite_adguard_blocklist.txt), [lite_cidr_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/lite_cidr_ipv4_blocklist.txt), [lite_domain](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/lite_domain_blocklist.txt), [lite_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/lite_ipv4_blocklist.txt) | | 📏 Normal — 更广防护 🔵 | [normal_adguard](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/normal_adguard_blocklist.txt), [normal_cidr_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/normal_cidr_ipv4_blocklist.txt), [normal_domain](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/normal_domain_blocklist.txt), [normal_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/normal_ipv4_blocklist.txt) | | 📏 Big — 激进覆盖 🔴 | [big_adguard](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/big_adguard_blocklist.txt), [big_cidr_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/big_cidr_ipv4_blocklist.txt), [big_domain](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/big_domain_blocklist.txt), [big_ipv4](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/big_ipv4_blocklist.txt), [big_ipv6](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/groups/big_ipv6_blocklist.txt) | | ⭐ 热门列表 — min3 至 min12 | [top_adguard_min3](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/top/top_adguard_blocklist_min3.txt), [top_adguard_min5](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/top/top_adguard_blocklist_min5.txt) · [top_domain_min3](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/top/top_domain_blocklist_min3.txt), [top_domain_min5](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/top/top_domain_blocklist_min5.txt) · [top_ipv4_min3](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/top/top_ipv4_blocklist_min3.txt), [top_ipv4_min5](https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/top/top_ipv4_blocklist_min5.txt) (`min3` 表示“至少来自 3 个来源”,`min5` 表示“至少来自 5 个来源”,以此类推;`minN` 值越高 → 置信度越高,条目越少。) | **[查看详细的重叠分析](https://github.com/phani-kb/dns-toolkit/blob/output/overlap.md)** 展示不同 DNS 来源之间如何共享条目的综合分析。 **[冲突报告(白名单 vs. 屏蔽列表)](https://github.com/phani-kb/dns-toolkit/blob/output/conflicts.md)** 每日生成的报告,列出在白名单和屏蔽列表中同时找到的条目,包括发现它们的来源。 **处理摘要和元数据归档在 [`summaries`](https://github.com/phani-kb/dns-toolkit/tree/summaries) 分支中,保留期为 1 年。** ## 快速添加(可复制的 URL) 将以下原始 URL 之一直接复制到您的 DNS 过滤工具中。 ### Pi-hole(域名列表) ``` https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/domain_blocklist.txt ``` ### AdGuard Home(AdGuard 格式) ``` https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/adguard_blocklist.txt ``` ### pfBlockerNG(pfSense) 使用“DNSBL”>“DNSBL Groups”部分,并使用原始 URL 添加自定义列表。示例域名列表(粘贴到 pfBlockerNG 自定义列表中): ``` https://raw.githubusercontent.com/phani-kb/dns-toolkit/output/domain_blocklist.txt ``` ## 分支大小 **注意:** 上方的仓库大小徽章仅反映默认分支(`release/1.0.0`)。 - **Output 分支大小:** 501.02 MB - **Summaries 分支大小:** 11.34 MB ## 来源统计 *从来源配置文件自动生成的统计信息* | 指标 | 数量 | 详情 | |--------|-------|---------| | **总来源数** | 183 | 151 个启用,32 个禁用 | | **屏蔽列表来源** | 155 | 提供屏蔽规则的来源 | | **白名单来源** | 39 | 提供例外规则的来源 | | **类别** | 41 | ads, adult, ai, annoyance, anonymizer, bot, botnet, browser, cryptocurrency, dating, discord, dns, doh, fake, fakenews, finance, gambling, issues, kad, local, mac, malicious, malware, mobile, others, phishing, privacy, proxy, ransomware, scam, smarttv, social, spam, spyware, threat, topdomains, tor, torrent_trackers, trackers, url_shorteners, windows | | **来源类型** | 30 | adguard, adguard_csv_http_url_find, adguard_domain, adguard_http_url, cidr_ipv4, domain, domain_adguard, domain_comment, domain_csv_http_url_find, domain_custom_csv_blackbook, domain_custom_csv_maltrail, domain_custom_html_ccam, domain_custom_html_puppyscams, domain_http_url, domain_top, domain_url, domain_with_comment_suffix, hostname, ipv4, ipv4_cidr_expand, ipv4_csv_http_url_find, ipv4_custom_html_ccam, ipv4_find, ipv4_from_domain, ipv4_http_url, ipv4_range_expand, ipv4_url, ipv6, ipv6_find, ipv6_htaccess | | **地理覆盖范围** | 21 个国家 | CN, CZ, DE, ES, FI, FR, HU, ID, IL, IT, KR, LV, MY, NL, RO, RU, SA, SK, UA, US, VN | | **最后更新** | 2026-07-07 18:12:10 UTC | 统计信息生成时间 | ## 来源配置(重要!) 来源在 `data/config/sources*.json` 文件中配置。每个来源指定: - 下载 URL 和频率 - 来源类型(域名、IPv4、IPv6、AdGuard 等) - 类别(广告、恶意软件、隐私等) - 许可证和网站信息 ## 关于 Top 域名的特别说明 (tranco-list.eu) 来自 tranco-list.eu 列表(`domain_top` 类型)的顶级域名将被视为白名单处理。 ## 白名单生成流程 ``` --- config: theme: mc --- flowchart LR subgraph horizontal[" "] direction LR OutputIPv4["allowlist_ipv4.txt"] end ReadSources["Read Sources"] --> ExtractDomains["Extract Domains from Source URLs"] ExtractDomains --> LoadCustom["Load Custom Files"] LoadCustom --> Combine["Combine & Deduplicate"] Combine --> OutputDomains["allowlist_domains.txt"] & OutputAdGuard["allowlist_adg.txt
(@@||domain^ format)"] & ResolveDNS["Resolve to IPv4"] ResolveDNS --> OutputIPv4 OutputIPv4@{ shape: doc} ReadSources@{ shape: procs} ExtractDomains@{ shape: procs} OutputDomains@{ shape: doc} OutputAdGuard@{ shape: doc} ResolveDNS@{ shape: proc} style Combine stroke-width:2px,stroke-dasharray: 2 style horizontal fill:transparent,stroke:transparent ``` ## 安装说明 ``` git clone https://github.com/phani-kb/dns-toolkit.git cd dns-toolkit go build -o bin/dns-toolkit main.go ``` ## 快速开始 ``` # 下载并处理所有 sources dns-toolkit download dns-toolkit process dns-toolkit consolidate dns-toolkit generate output # 搜索 domain dns-toolkit search example.com # 分析 sources 之间的重叠 dns-toolkit overlap # 查找跨 sources 的热门条目 dns-toolkit top ``` ## 核心命令 ``` DNS Toolkit Usage: dns-toolkit [command] Available Commands: archive Archive DNS toolkit data consolidate Consolidate processed files download Download enabled sources generate Generate different types of outputs help Help about any command overlap Find overlap between source files process Process downloaded files search Search for a domain or IP in the processed files sts Prints the source types summary top Find top entry(s) in each generic source type validate-sources Validate the sources configuration version Print the version number of DNS Toolkit Flags: -h, --help help for dns-toolkit Use "dns-toolkit [command] --help" for more information about a command. ``` ## 输出结构 ``` data/output/ ├── *_blocklist.txt # Blocklists for various source types (adguard, domain, ipv4, ipv6, cidr) ├── *_allowlist.txt # Allowlists for various source types (adguard, domain, etc.) ├── categories/ # Lists by category (ads, malware, privacy, etc.) ├── groups/ # Lists by size (mini, lite, normal, big) ├── top/ # Top entries based on source frequency └── summaries/ # Processing metadata and statistics ``` ## 问题 如果您遇到错误、有功能请求或想提出改进建议,请在 [GitHub Issues](https://github.com/phani-kb/dns-toolkit/issues) 页面上提交 issue。 ## 许可证 本项目根据 LICENSE 文件中指定的条款进行许可。
标签:DNS, Docker 部署, EVTX分析, Go, Linux安全, Ruby工具, 广告拦截, 日志审计, 网络运维, 黑白名单