Quad9DNS/threat-intel-api

GitHub: Quad9DNS/threat-intel-api

Quad9 为威胁情报提供商提供的 WebSocket 流式 API 示例代码，用于检索合作伙伴提交的恶意域名所产生的近实时 DNS 拦截遥测数据。

Stars: 72 | Forks: 8

# threat-intel-api 本代码库包含威胁情报提供商的示例代码，这些提供商向 Quad9 提供由恶意域名组成的威胁情报源，随后这些域名将被 Quad9 具备拦截功能的服务地址集拦截。合作伙伴应联系 Quad9 (support@quad9.net) 以获取有效的 API Key。 # 概述 threat-intel-api 是一个 HTTP/Websocket 服务，允许威胁情报 (TI) 提供商检索由其通过威胁情报源提供给 Quad9 的恶意域名所生成的遥测数据。本服务面向已与 Quad9 签署协议以提供威胁数据（域名）的 TI 提供商 - 它不是面向 Quad9 的通用 API。如果您是威胁情报提供商并且需要密钥，请通过 support@quad9.net 联系 Quad9。客户端软件将通过 websocket 访问 API。如果客户端有多个威胁列表，系统将为每个列表提供独立的 URL，并使用相同的身份验证凭据。 ## 要求客户端需要： - websocket 的 URL (https://tiapi.quad9.net/<由 Quad9 为每个提供商列表提供的 URL 路径>) - 由 Quad9 提供的身份验证凭据（用户名/密码） - 流式 websocket 客户端或支持 TLS 加密 websocket 源 (wss:) 的客户端库 ## API 业务规则编写 threat-intel-api 时遵循了以下业务规则： - 事件数据仅传输给在事件中提供了特定域名的 TI 提供商 - 多个 TI 提供商可能会提供相同的域名；每个提供商都将获得事件数据的独立副本 - 数据以 JSON 格式传输 - 数据以近实时方式流式传输（通常端到端延迟不到 5 秒） - 事件在环形缓冲区中排队，缓冲区最多可容纳 100000 个事件。如果客户端断开连接，只要断开连接后发生的事件不超过 100000 个，并且在重连时包含了最后成功接收到的 message-id，就可以继续接收而不会丢失消息。 - 客户端可以在不指定“last-seen”（最后可见）项目的情况下进行连接，系统将传送最新的事件及后续事件 # 简单的 wsget 示例 Python、Go、Rust 和大多数其他语言都有用于更高级 API 连接的 websocket 库。使用“wscat”() 开源连接工具，可以进行简单的测试以验证源状态。其他用于命令行 websocket 连接的选项包括 websocat () 或 curl 8.14.0 及更高版本等工具。示例： ``` wscat --auth yoyodyne-labs:UeBX9mXEpq33Mu3 --connect "wss://tapi4.quad9.net/tiapi/yoyodyne-KCWk7HRP3AR/" ``` 如果运行正常，该命令将生成如下输出： ``` Connected (press CTRL+C to quit) < {"blockedNameData":{"addressesV4":{"answers":[{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"178.63.248.53","recordType":"A","recordTypeId":1,"ttl":126},{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"157.90.33.78","recordType":"A","recordTypeId":1,"ttl":126},{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"157.90.33.79","recordType":"A","recordTypeId":1,"ttl":126},{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"178.63.248.50","recordType":"A","recordTypeId":1,"ttl":126},{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"178.63.248.55","recordType":"A","recordTypeId":1,"ttl":126},{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"178.63.248.54","recordType":"A","recordTypeId":1,"ttl":126},{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"178.63.248.48","recordType":"A","recordTypeId":1,"ttl":126},{"asn":{"autonomous_system_number":24940,"autonomous_system_organization":"HETZNER ONLINE G.M.B.H."},"class":"IN","domainName":"one-new-message-okay.com","geoip":{"city":{"names":{"en":"Auerbach"}},"country":{"iso_code":"DE"},"location":{"latitude":50.51,"longitude":12.4},"subdivisions":[{"iso_code":"13"}],"traits":{"autonomous_system_number":24940,"autonomous_system_organization":"Hetzner Online GmbH","user_type":"hosting"}},"rData":"178.63.248.49","recordType":"A","recordTypeId":1,"ttl":126}]},"dataAge":256,"nameservers":{"answers":[{"class":"IN","domainName":"one-new-message-okay.com","rData":"aspen.ns.cloudflare.com.","recordType":"NS","recordTypeId":2,"ttl":9631},{"class":"IN","domainName":"one-new-message-okay.com","rData":"braden.ns.cloudflare.com.","recordType":"NS","recordTypeId":2,"ttl":9631}],"dataAge":1784564933}},"geoip":{"city":{"names":{"en":"Jakarta"}},"country":{"iso_code":"ID"},"location":{"latitude":-6.21,"longitude":106.84},"subdivisions":[{"iso_code":"04"}],"traits":{"autonomous_system_number":139428,"autonomous_system_organization":"PT.LEXXA DATA INDONUSA","user_type":"residential"}},"messageType":"ClientQuery","message_id":"019f805b-e990-73b3-b4dd-9f7cf5b601ba","pop_code":"jkt","requestData":{"fullRcode":0,"header":{"aa":false,"ad":true,"anCount":0,"arCount":1,"cd":false,"id":0,"nsCount":0,"opcode":0,"qdCount":1,"qr":0,"ra":false,"rcode":0,"rd":true,"tc":false},"opt":{"do":true,"ednsVersion":0,"extendedRcode":0,"udpPayloadSize":8192},"question":[{"class":"IN","domainName":"one-new-message-okay.com.","etld":{"etld":"com.","etld_plus":"one-new-message-okay.com.","known_suffix":true,"tld":"com."},"questionType":"A","questionTypeId":1}],"rcodeName":"NoError"},"responseAddress":"9.9.9.9","responsePort":853,"serverId":"res130.jkt","socketFamily":"INET","socketProtocol":"DOT","timestamp":"2026-07-20T16:28:53.284Z"} ``` 单条记录示例（为了便于阅读，已通过“jq -r”解析） ``` { "blockedNameData": { "addressesV4": { "answers": [ { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "178.63.248.53", "recordType": "A", "recordTypeId": 1, "ttl": 126 }, { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "157.90.33.78", "recordType": "A", "recordTypeId": 1, "ttl": 126 }, { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "157.90.33.79", "recordType": "A", "recordTypeId": 1, "ttl": 126 }, { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "178.63.248.50", "recordType": "A", "recordTypeId": 1, "ttl": 126 }, { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "178.63.248.55", "recordType": "A", "recordTypeId": 1, "ttl": 126 }, { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "178.63.248.54", "recordType": "A", "recordTypeId": 1, "ttl": 126 }, { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "178.63.248.48", "recordType": "A", "recordTypeId": 1, "ttl": 126 }, { "asn": { "autonomous_system_number": 24940, "autonomous_system_organization": "HETZNER ONLINE G.M.B.H." }, "class": "IN", "domainName": "one-new-message-okay.com", "geoip": { "city": { "names": { "en": "Auerbach" } }, "country": { "iso_code": "DE" }, "location": { "latitude": 50.51, "longitude": 12.4 }, "subdivisions": [ { "iso_code": "13" } ], "traits": { "autonomous_system_number": 24940, "autonomous_system_organization": "Hetzner Online GmbH", "user_type": "hosting" } }, "rData": "178.63.248.49", "recordType": "A", "recordTypeId": 1, "ttl": 126 } ] }, "dataAge": 256, "nameservers": { "answers": [ { "class": "IN", "domainName": "one-new-message-okay.com", "rData": "aspen.ns.cloudflare.com.", "recordType": "NS", "recordTypeId": 2, "ttl": 9631 }, { "class": "IN", "domainName": "one-new-message-okay.com", "rData": "braden.ns.cloudflare.com.", "recordType": "NS", "recordTypeId": 2, "ttl": 9631 } ], "dataAge": 1784564933 } }, "geoip": { "city": { "names": { "en": "Jakarta" } }, "country": { "iso_code": "ID" }, "location": { "latitude": -6.21, "longitude": 106.84 }, "subdivisions": [ { "iso_code": "04" } ], "traits": { "autonomous_system_number": 139428, "autonomous_system_organization": "PT.LEXXA DATA INDONUSA", "user_type": "residential" } }, "messageType": "ClientQuery", "message_id": "019f805b-e990-73b3-b4dd-9f7cf5b601ba", "pop_code": "jkt", "requestData": { "header": { "aa": false, "ad": true, "anCount": 0, "arCount": 1, "cd": false, "id": 0, "nsCount": 0, "opcode": 0, "qdCount": 1, "qr": 0, "ra": false, "rcode": 0, "rd": true, "tc": false }, "opt": { "do": true, "ednsVersion": 0, "extendedRcode": 0, "udpPayloadSize": 8192 }, "question": [ { "class": "IN", "domainName": "one-new-message-okay.com.", "etld": { "etld": "com.", "etld_plus": "one-new-message-okay.com.", "known_suffix": true, "tld": "com." }, "questionType": "A", "questionTypeId": 1 } ] }, "responseAddress": "9.9.9.9", "responsePort": 853, "serverId": "res130.jkt", "socketFamily": "INET", "socketProtocol": "DOT", "timestamp": "2026-07-20T16:28:53.284Z" } ``` 通常，Quad9 会提供有关已被拦截的各个 DNS 查询事件的信息，这些信息在保护隐私的同时，让威胁情报提供商能够深入了解已被阻止的恶意活动。这些数据包括：经过模糊处理的受害者信息、关于恶意目标的详细信息，以及有关事件流的元信息。不包含或引用任何关于客户端的个人身份信息 (PII)，也绝对不包含任何客户端 IP 地址数据。上述所有的 IP 地址均为目标 IP 地址 - 即对恶意域名的 A 或 AAAA 记录进行 DNS 查询的结果。请参阅管辖 Quad9 在这些以及所有遥测聚合中传输的数据的 Quad9 隐私政策：并非所有的查询都是为了获取 A 或 AAAA 结果。有些查询是为了获取 NS、TXT 或其他结果。根据该特定记录类型可用的信息不同，这些记录将包含不同程度附加信息。数据字段简要概述： - `pop_code` = 查询来源的 POP。有关 POP 名称，请参见 www.quad9.net/locations，但通常 POP 名称中会包含三个字母的 IATA 代码。 - `socketProtocol` = DOT、DOH、UDP、TCP、DOQ 等，随着新标准的出现，可能还会包含其他协议 - `responsePort` = 用于入站连接的端口 - `responseAddress` = 客户端发送查询的目标 Quad9 服务地址 - `requestData` = DNS 请求信息 - `geoip` = 关于客户端（受害者）经过模糊处理的地理、网络和人口统计数据。目前，地名仅提供英文名称。 - `blockedNameData` = 被拦截查询的 IPv4、IPv6 地址以及被拦截查询的域名服务器。如果查询被允许，这就是本应交付给受害者的内容。 - `message_id` = 此 websocket 流中的序列号。 *_注意：我们有多个地理数据来源，但我们会将位置对齐/模糊到高于最低人口数量的最近城市中心 - 它绝不会移出一个地区（通常是国家），但您不会获得小城市的细粒度地理信息。这是为了确保最终用户的隐私。目标数据使用相同的地理数据库。_* ## 断开与重连为了确保完整交付所有消息，您可能希望保留 `message_id` 记录的状态。此标识符在随后的每条消息中都会发生变化。它允许数据源在下次连接时，通过在 wss URL 中使用“last_received”参数从上次中断的地方继续接收。示例： ``` wscat --auth yoyodyne-labs:UeBX9mXEpq33Mu3 --connect "wss://tapi4.quad9.net/tiapi/yoyodyne-KCWk7HRP3AR?last_received=019f805b-e990-73b3-b4dd-9f7cf5b601ba" ``` 如果流量不足，客户端库可能会断开 Websocket - 请确保将 ping 间隔设置为不到 20 秒。Quad9 不会因为不活跃而断开会话，但客户端库可能默认进行了此类配置。请确保如果发生断开连接，您的客户端会立即重连。如果指定的 message-id 指针已从队列中过期，您的 websocket 连接将首先传输环形缓冲区中最旧的项目。在相同的身份验证和 URL 条件下连接的多个客户端可能会导致意外的重复，尽管这一点尚未经过详细测试。请使用单一 socket 连接。如果您的威胁数据集内发生了该数量的拦截，socket 能够轻松实现每秒数万个事件的传输；请确保接收端有足够的缓冲区。 v2027/07/21.01 jtodd

标签：Homebrew安装, IP 地址批量处理, WebSocket, 依赖分析, 威胁情报, 开发者工具, 网络安全, 遥测数据, 隐私保护