使用Gitleaks保护、发现泄露的密钥 🔑

作者：Sec-Labs | 发布时间：2023-02-20 15:20:37

项目地址

Gitleaks

┌─○───┐
│ │╲  │
│ │ ○ │
│ ○ ░ │
└─░───┘

Gitleaks 是一个 SAST 工具，用于检测和防止 git repos 中的硬编码秘密，如密码、api 密钥和令牌。 Gitleaks 是一种 易于使用的一体化解决方案， 用于检测代码中过去或现在的秘密。

➜  ~/code(master) gitleaks detect --source . -v

    ○
    │╲
    │ ○
    ○ ░
    ░    gitleaks

Finding:     "export BUNDLE_ENTERPRISE__CONTRIBSYS__COM=cafebabe:deadbeef",
Secret:      cafebabe:deadbeef
RuleID:      sidekiq-secret
Entropy:     2.609850
File:        cmd/generate/config/rules/sidekiq.go
Line:        23
Commit:      cd5226711335c68be1e720b318b7bc3135a30eb2
Author:      John
Email:       john@users.noreply.github.com
Date:        2022-08-03T12:31:40Z
Fingerprint: cd5226711335c68be1e720b318b7bc3135a30eb2:cmd/generate/config/rules/sidekiq.go:sidekiq-secret:23

入门

可以使用 Homebrew、Docker 或 Go 安装 Gitleaks。 Gitleaks 还以二进制形式提供，适用于发布页面上的许多流行平台和操作系统类型。此外，Gitleaks 可以作为预提交挂钩直接在您的存储库中实现，或者作为 GitHub 操作使用 Gitleaks-Action 实现。

安装中

# MacOS
brew install gitleaks

# Docker (DockerHub)
docker pull zricethezav/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest [COMMAND] --source="/path" [OPTIONS]

# Docker (ghcr.io)
docker pull ghcr.io/zricethezav/gitleaks:latest
docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest [COMMAND] --source="/path" [OPTIONS]

# From Source
git clone https://github.com/zricethezav/gitleaks.git
cd gitleaks
make build

GitHub Action

查看官方 Gitleaks GitHub Action

name: gitleaks
on: [pull_request, push, workflow_dispatch]
jobs:
  scan:
    name: gitleaks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          GITLEAKS_LICENSE: ${{ secrets.GITLEAKS_LICENSE}} # Only required for Organizations, not personal accounts.

预提交

从https://pre-commit.com/#install 安装预提交
.pre-commit-config.yaml 使用以下内容在存储库的根目录中创建一个文件： repos: - repo: https://github.com/zricethezav/gitleaks rev: v8.15.3 hooks: - id: gitleaks 用于本地执行 GitLeaks 或使用 gitleaks-docker 预提交 ID 使用官方 Docker 图像执行 GitLeaks
通过执行自动将配置更新到最新的 repos 版本 pre-commit autoupdate
安装 pre-commit install
现在你已经准备好了！

➜ git commit -m "this commit contains a secret"
Detect hardcoded secrets.................................................Failed

注意：要禁用 gitleaks 预提交挂钩，您可以 SKIP=gitleaks 在提交命令前加上它，它将跳过运行 gitleaks

➜ SKIP=gitleaks git commit -m "skip gitleaks check"
Detect hardcoded secrets................................................Skipped

用法

Usage:
  gitleaks [command]

Available Commands:
  completion  generate the autocompletion script for the specified shell
  detect      detect secrets in code
  help        Help about any command
  protect     protect secrets in code
  version     display gitleaks version

Flags:
  -b, --baseline-path string       path to baseline with issues that can be ignored
  -c, --config string              config file path
                                   order of precedence:
                                   1. --config/-c
                                   2. env var GITLEAKS_CONFIG
                                   3. (--source/-s)/.gitleaks.toml
                                   If none of the three options are used, then gitleaks will use the default config
      --exit-code int              exit code when leaks have been encountered (default 1)
  -h, --help                       help for gitleaks
  -l, --log-level string           log level (trace, debug, info, warn, error, fatal) (default "info")
      --max-target-megabytes int   files larger than this will be skipped
      --no-banner                  suppress banner
      --redact                     redact secrets from logs and stdout
  -f, --report-format string       output format (json, csv, sarif) (default "json")
  -r, --report-path string         report file
  -s, --source string              path to source (default: $PWD) (default ".")
  -v, --verbose                    show verbose output from scan

Use "gitleaks [command] --help" for more information about a command.

命令

您将使用两个命令来检测秘密； detect 和 protect 。

探测

该 detect 命令用于扫描存储库、目录和文件。此命令可用于开发人员机器和 CI 环境。

在 git 存储库上运行时 detect ，gitleaks 将解析命令的输出 git log -p （您可以在此处查看其执行方式）。生成gitleaks 将用来检测秘密的 git log -p 补丁。您可以 git log 使用标志配置提交范围 --log-opts 。 --log-opts 接受的任何选项 git log -p 。例如，如果您想在一系列提交上运行 gitleaks，您可以使用以下命令： gitleaks detect --source . --log-opts="--all commitA..commitB" . 有关详细信息，请参阅 git log 文档。

您可以使用该 --no-git 选项扫描文件和目录。

保护

该 protect 命令用于 git 存储库中未提交的更改。此命令应根据安全性左移在开发人员机器上使用。在 git 存储库上运行时 protect ，gitleaks 将解析命令的输出 git diff （您可以在此处查看其执行方式）。您可以设置 --staged 标志以检查已 git add 编辑的提交中的更改。 --staged 将 Gitleaks 作为预提交运行时应使用该标志。

注意：该 protect 命令只能在 git repos 上使用， protect 在文件或目录上运行将导致错误消息。

创建基线

扫描大型存储库或历史悠久的存储库时，使用基线会很方便。使用基线时，gitleaks 将忽略基线中存在的任何旧发现。基线可以是任何 gitleaks 报告。要创建 gitleaks 报告，请使用 --report-path 参数运行 gitleaks。

gitleaks detect --report-path gitleaks-report.json # This will save the report in a file called gitleaks-report.json

一旦创建了基线，就可以在再次运行检测命令时应用它：

gitleaks detect --baseline-path gitleaks-report.json --report-path findings.json

使用 --baseline-path 参数运行检测命令后，报告输出 (findings.json) 将仅包含新问题。

验证结果

您可以使用命令验证 gitleaks 发现的结果 git log 。示例输出：

Finding:     aws_secret="AKIAIMNOJVGFDXXXE4OA"
RuleID:      aws-access-token
Secret       AKIAIMNOJVGFDXXXE4OA
Entropy:     3.65
File:        checks_test.go
Line:        37
Commit:      ec2fc9d6cb0954fb3b57201cf6133c48d8ca0d29
Author:      Zachary Rice
Email:       z@email.com
Date:        2018-01-28T17:39:00Z
Fingerprint: ec2fc9d6cb0954fb3b57201cf6133c48d8ca0d29:checks_test.go:aws-access-token:37

我们可以使用以下格式来验证泄漏：

git log -L {StartLine,EndLine}:{File} {Commit}

所以在这个例子中它看起来像：

git log -L 37,37:checks_test.go ec2fc9d6cb0954fb3b57201cf6133c48d8ca0d29

这给了我们：

commit ec2fc9d6cb0954fb3b57201cf6133c48d8ca0d29
Author: zricethezav <thisispublicanyways@gmail.com>
Date:   Sun Jan 28 17:39:00 2018 -0500

    [update] entropy check

diff --git a/checks_test.go b/checks_test.go
--- a/checks_test.go
+++ b/checks_test.go
@@ -28,0 +37,1 @@
+               "aws_secret= \"AKIAIMNOJVGFDXXXE4OA\"":          true,

预提交挂钩

pre-commit.py 您可以通过将示例脚本复制到您的 .git/hooks/ 目录中来运行 Gitleaks 作为预提交挂钩。

配置

Gitleaks 提供了一种配置格式，您可以按照这种格式编写自己的秘密检测规则：

# Title for the gitleaks configuration file.
title = "Gitleaks title"

# Extend the base (this) configuration. When you extend a configuration
# the base rules take precendence over the extended rules. I.e, if there are
# duplicate rules in both the base configuration and the extended configuration
# the base rules will override the extended rules.
# Another thing to know with extending configurations is you can chain together
# multiple configuration files to a depth of 2. Allowlist arrays are appended
# and can contain duplicates.
# useDefault and path can NOT be used at the same time. Choose one.
[extend]
# useDefault will extend the base configuration with the default gitleaks config:
# https://github.com/zricethezav/gitleaks/blob/master/config/gitleaks.toml
useDefault = true
# or you can supply a path to a configuration. Path is relative to where gitleaks
# was invoked, not the location of the base config.
path = "common_config.toml"

# An array of tables that contain information that define instructions
# on how to detect secrets
[[rules]]

# Unique identifier for this rule
id = "awesome-rule-1"

# Short human readable description of the rule.
description = "awesome rule 1"

# Golang regular expression used to detect secrets. Note Golang's regex engine
# does not support lookaheads.
regex = '''one-go-style-regex-for-this-rule'''

# Golang regular expression used to match paths. This can be used as a standalone rule or it can be used
# in conjunction with a valid `regex` entry.
path = '''a-file-path-regex'''

# Array of strings used for metadata and reporting purposes.
tags = ["tag","another tag"]

# Int used to extract secret from regex match and used as the group that will have
# its entropy checked if `entropy` is set.
secretGroup = 3

# Float representing the minimum shannon entropy a regex group must have to be considered a secret.
entropy = 3.5

# Keywords are used for pre-regex check filtering. Rules that contain
# keywords will perform a quick string compare check to make sure the
# keyword(s) are in the content being scanned. Ideally these values should
# either be part of the idenitifer or unique strings specific to the rule's regex
# (introduced in v8.6.0)
keywords = [
  "auth",
  "password",
  "token",
]

# You can include an allowlist table for a single rule to reduce false positives or ignore commits
# with known/rotated secrets
[rules.allowlist]
description = "ignore commit A"
commits = [ "commit-A", "commit-B"]
paths = [
  '''go\.mod''',
  '''go\.sum'''
]
regexes = [
  '''process''',
  '''getenv''',
]
# note: stopwords targets the extracted secret, not the entire regex match
# like 'regexes' does. (stopwords introduced in 8.8.0)
stopwords = [
  '''client''',
  '''endpoint''',
]


# This is a global allowlist which has a higher order of precedence than rule-specific allowlists.
# If a commit listed in the `commits` field below is encountered then that commit will be skipped and no
# secrets will be detected for said commit. The same logic applies for regexes and paths.
[allowlist]
description = "global allow list"
commits = [ "commit-A", "commit-B", "commit-C"]
paths = [
  '''gitleaks\.toml''',
  '''(.*?)(jpg|gif|doc)'''
]
regexes = [
  '''219-09-9999''',
  '''078-05-1120''',
  '''(9[0-9]{2}|666)-\d{2}-\d{4}''',
]
# note: stopwords targets the extracted secret, not the entire regex match
# like 'regexes' does. (stopwords introduced in 8.8.0)
stopwords = [
  '''client''',
  '''endpoint''',
]

请参阅默认的 gitleaks 配置以获取示例，或者如果您想为默认配置做出贡献，请遵循贡献指南。此外，您可以查看这篇涵盖高级配置设置的 gitleaks 博客文章。

附加配置

gitleaks：允许

如果你故意提交一个 gitleaks 会捕获的测试秘密，你可以在 gitleaks:allow 该行添加一条注释，指示 gitleaks 忽略该秘密。前任：

class CustomClass:
    discord_client_secret = '8dyfuiRyq=vVc3RRr_edRk-fK__JItpZ'  #gitleaks:allow

.gitleaksignore

.gitleaksignore 您可以通过在存储库的根目录下创建一个文件来忽略特定的发现。在版本 v8.10.0 中，Gitleaks Fingerprint 向 Gitleaks 报告添加了一个值。每个泄漏或发现都有一个唯一标识秘密的指纹。将此指纹添加到 .gitleaksignore 文件中以忽略该特定秘密。有关示例，请参见 Gitleaks 的 .gitleaksignore 。注意：此功能是实验性的，将来可能会发生变化。

由 Jit 保护

我们使用 Jit 来保护我们的代码库，使用世界上最好的 OSS 安全工具来实现全自动、全栈的持续安全。

标签：工具分享, 敏感信息查询工具, API安全, API密钥泄露