ClickHouse/ClickBOM
GitHub: ClickHouse/ClickBOM
一款专注于 SBOM 自动化采集、格式转换与集中存储的 GitHub Action，帮助团队从多个安全平台汇聚软件物料清单到 S3 和 ClickHouse 进行统一管理与分析。
Stars: 9 | Forks: 1
[](https://github.com/ClickHouse/ClickBOM/actions/workflows/tests.yml) [](https://github.com/ClickHouse/ClickBOM/actions/workflows/docker-security.yml)
# ClickBOM
从 GitHub、Mend 和 Wiz 下载 SBOM。上传到 S3 和 ClickHouse。
- [输入参数](#inputs)
- [GitHub](#github)
- [Mend](#mend)
- [Wiz](#wiz)
- [AWS](#aws)
- [ClickHouse](#clickhouse)
- [常规](#general)
- [用法](#usage)
- [同一仓库](#same-repository)
- [同一仓库配合 ClickHouse](#same-repository-with-clickhouse)
- [同一仓库配合 GitHub App](#same-repository-with-github-app)
- [多个仓库](#multiple-repositories)
- [合并在 S3 中存储的 SBOM](#merging-sboms-stored-in-s3)
- [使用包含/排除过滤条件合并 SBOM](#merging-sboms-with-includeexclude-filters)
- [从 Mend 下载 SBOM](#downloading-an-sbom-from-mend)
- [从 Wiz 下载 SBOM](#downloading-an-sbom-from-wiz)
- [创建 GitHub App](#creating-a-github-app)
## 输入参数
### GitHub
| 名称 | 描述 | 默认值 | 必填 | 敏感 |
| --------------------- | ----------------------------------- | -------------- | -------- | --------- |
| github-token | GitHub Token | | false | true |
| repository | 要下载 SBOM 的仓库 | | false | false |
- `github-token` 可以是内置的 `${{ secrets.GITHUB_TOKEN }}` 或由 GitHub App 生成的 token。如果您使用 GitHub App，请参阅[创建 GitHub App](#creating-a-github-app)。
### Mend
| 名称 | 描述 | 默认值 | 必填 | 敏感 |
| ------------------- | ------------------------------------------------------------- | ------------------------ | -------- | --------- |
| mend-email | Mend 用户邮箱地址 | | false | true |
| mend-org-uuid | Mend 组织 UUID | | false | true |
| mend-user-key | Mend 用户密钥 | | false | true |
| mend-base-url | Mend 基础 URL | https://api-saas.mend.io | false | false |
| mend-product-uuid | 用于产品范围 SBOM 的 Mend 产品 UUID | | false | true |
| mend-project-uuid | 用于项目范围 SBOM 的 Mend 项目 UUID | | false | true |
| mend-org-scope-uuid | 用于组织范围 SBOM 的 Mend 组织 UUID | | false | true |
| mend-project-uuids | 要包含的特定项目 UUID 的逗号分隔列表 | | false | true |
| mend-max-wait-time | 等待 Mend 报告生成的最大时间（秒） | 1800 | false | false |
| mend-poll-interval | Mend 报告状态的轮询间隔（秒） | 30 | false | false |
- `mend-org-scope-uuid` 用于组织范围的 SBOM，这与用于身份验证的 `mend-org-uuid` 不同。
- ClickBOM 仅支持从 Mend 下载 CycloneDX v1.5 格式的 SBOM。如果您需要将 SBOM 转换为 SPDX，可以使用 `sbom-format` 输入参数。（即将支持 SPDX）
### Wiz
| 名称 | 描述 | 默认值 | 必填 | 敏感 |
| ----------------- | ----------------- | ------- | -------- | --------- |
| wiz-auth-endpoint | Wiz Auth Endpoint | | false | true |
| wiz-api-endpoint | Wiz API Endpoint | | false | true |
| wiz-client-id | Wiz Client ID | | false | true |
| wiz-client-secret | Wiz Client Secret | | false | true |
| wiz-report-id | Wiz Report ID | | false | true |
### AWS
| 名称 | 描述 | 默认值 | 必填 | 敏感 |
| --------------------- | --------------------------------------------------------------------------- | --------- | -------- | --------- |
| aws-access-key-id | AWS Access Key ID。**已弃用 — 推荐使用 OIDC**（见下方示例）。 | | false | true |
| aws-secret-access-key | AWS Secret Access Key。**已弃用 — 推荐使用 OIDC**（见下方示例）。 | | false | true |
| aws-region | AWS Region。**已弃用 — 推荐使用 OIDC**（通过 configure-aws-credentials 设置）。 | us-east-1 | false | false |
| s3-bucket | S3 Bucket 名称 | | false | false |
| s3-key | S3 Key 前缀 | sbom.json | false | false |
- 建议为 ClickBOM 创建一个专用的 S3 bucket。
- 保留 `aws-*` 输入参数是为了与该 action 的 bash 版本向后兼容。推荐的路径是配合 GitHub OIDC 使用 [`aws-actions/configure-aws-credentials@v4`](https://github.com/aws-actions/configure-aws-credentials))；该 action 会将 `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / `AWS_DEFAULT_REGION` 导出为作业级别的环境变量，AWS SDK 会自动识别它们——无需将它们作为输入参数传递。
### ClickHouse
| 名称 | 描述 | 默认值 | 必填 | 敏感 |
| ------------------- | ----------------------------------- | -------------- | -------- | --------- |
| clickhouse-url | ClickHouse URL | | false | true |
| clickhouse-database | ClickHouse Database 名称 | default | false | false |
| clickhouse-username | ClickHouse 用户名 | default | false | false |
| clickhouse-password | ClickHouse 密码 | (空) | false | true |
| truncate-table | 插入前清空表 | false | false | false |
- 目前，仅支持通过 HTTP 进行 ClickHouse 数据摄取。
### 常规
| 名称 | 描述 | 默认值 | 必填 | 敏感 |
| ----------- | --------------------------------------------------------------------- | --------- | -------- | --------- |
| sbom-source | SBOM 来源 (github, mend, wiz) | github | false | false |
| sbom-format | SBOM 格式 (spdxjson 或 cyclonedx) | cyclonedx | false | false |
| merge | 合并在 S3 中存储的 SBOM | false | false | false |
| include | 合并时要包含的文件名或模式的逗号分隔列表 | (空) | false | false |
| exclude | 合并时要排除的文件名或模式的逗号分隔列表 | (空) | false | false |
| debug | 启用调试日志记录 | false | false | false |
- `sbom-format` 指定您希望最终 SBOM 采用的格式。例如，GitHub 仅支持 SPDX，将此输入设置为 `cyclonedx` 会将 SBOM 转换为 CycloneDX 格式。
- `include` 和 `exclude` 仅在 `merge` 设置为 `true` 时使用。它们允许您过滤 S3 bucket 中应包含在合并操作中的文件。
- `include` 和 `exclude` 都支持精确的文件名匹配和通配符模式（例如 `file*.json`、`*-prod.json`）。
- 如果指定了 `include`，则只会处理匹配包含模式的文件。
- 如果指定了 `exclude`，则会跳过匹配排除模式的文件。
- `exclude` 会在 `include` 之后应用，因此如果一个文件**同时**匹配包含和排除模式，它将被*排除*。
## 用法
### 同一仓库
从同一仓库下载 SBOM 并将其上传到 S3 的简单示例。将 SBOM 转换为 CycloneDX 格式。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom:
name: ClickBOM
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM
uses: ClickHouse/ClickBom@v1.0.10
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: clickbom.json
repository: ${{ github.repository_owner }}/${{ github.repository }}
```
### 同一仓库配合 ClickHouse
从同一仓库下载 SBOM 并将其上传到 S3。将 SBOM 转换为 CycloneDX 格式。同时将 SBOM 上传到 ClickHouse。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom:
name: ClickBOM
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM
uses: ClickHouse/ClickBom@v1.0.10
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: clickbom.json
repository: ${{ github.repository_owner }}/${{ github.repository }}
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
```
### 同一仓库配合 GitHub App
从同一仓库下载 SBOM 并将其上传到 S3。保留 SBOM 为 SPDX 格式。使用 GitHub App 进行身份验证。请参阅[创建 GitHub App](#creating-a-github-app)。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom:
name: ClickBOM
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Generate Token
id: generate-token
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.CLICKBOM_AUTH_APP_ID }}
private-key: ${{ secrets.CLICKBOM_AUTH_PRIVATE_KEY }}
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM
uses: ClickHouse/ClickBom@v1.0.10
with:
github-token: ${{ steps.generate-token.outputs.token }}
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
sbom-format: spdxjson
s3-bucket: my-sbom-bucket
s3-key: clickbom.json
repository: ${{ github.repository_owner }}/${{ github.repository }}
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
```
### 多个仓库
从多个仓库下载 SBOM（必须安装 GitHub App），将 SBOM 转换为 CycloneDX 格式，并将它们上传到 S3 和 ClickHouse。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom:
strategy:
fail-fast: false
matrix:
repository: [
"repository-one",
"repository-two",
"repository-three"
]
name: ClickBOM
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Generate Token
id: generate-token
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.CLICKBOM_AUTH_APP_ID }}
private-key: ${{ secrets.CLICKBOM_AUTH_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: ${{ matrix.repository }}
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM
uses: ClickHouse/ClickBom@v1.0.10
with:
github-token: ${{ steps.generate-token.outputs.token }}
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: ${{ matrix.repository }}.json
repository: ${{ github.repository_owner }}/${{ matrix.repository }}
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
```
### 合并在 S3 中存储的 SBOM
此示例在上一个示例的基础上，合并在 S3 中存储的 SBOM。它从 S3 下载 SBOM，将它们合并，然后将合并后的 SBOM 重新上传到 S3 和 ClickHouse。仅 CycloneDX 格式支持合并。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom:
strategy:
fail-fast: false
matrix:
repository: [
"repository-one",
"repository-two",
"repository-three"
]
name: ClickBOM
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Generate Token
id: generate-token
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.CLICKBOM_AUTH_APP_ID }}
private-key: ${{ secrets.CLICKBOM_AUTH_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: ${{ matrix.repository }}
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM
uses: ClickHouse/ClickBom@v1.0.10
with:
github-token: ${{ steps.generate-token.outputs.token }}
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: ${{ matrix.repository }}.json
repository: ${{ github.repository_owner }}/${{ matrix.repository }}
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
clickbom_merge:
needs: clickbom
name: ClickBOM Merge
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Generate Token
id: generate-token
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.CLICKBOM_AUTH_APP_ID }}
private-key: ${{ secrets.CLICKBOM_AUTH_PRIVATE_KEY }}
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM
uses: ClickHouse/ClickBom@v1.0.10
with:
github-token: ${{ steps.generate-token.outputs.token }}
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: clickbom.json
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
merge: true
```
### 使用包含/排除过滤条件合并 SBOM
此示例展示了在合并 SBOM 时如何使用 `include` 和 `exclude` 过滤条件。当您只想合并 S3 bucket 中的特定文件时，这非常有用。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom_merge:
name: ClickBOM Merge with Filters
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Generate Token
id: generate-token
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.CLICKBOM_AUTH_APP_ID }}
private-key: ${{ secrets.CLICKBOM_AUTH_PRIVATE_KEY }}
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Merge Production SBOMs Only
uses: ClickHouse/ClickBom@v1.0.10
with:
github-token: ${{ steps.generate-token.outputs.token }}
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: production-merged.json
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
merge: true
include: "*-prod.json,production-*.json"
exclude: "*-test.json,*-dev.json"
```
在此示例中：
- `include: "*-prod.json,production-*.json"` 将只处理匹配这些模式的文件
- `exclude: "*-test.json,*-dev.json"` 将跳过匹配这些模式的任何文件
- 结果是，只有与生产相关的 SBOM 会被合并，排除了测试和开发环境的 SBOM
### 从 Mend 下载 SBOM
如果您想从 Mend 下载 SBOM，可以使用以下示例。此示例假设您已在 GitHub Secrets 中配置了必要的 Mend 凭证。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom:
name: ClickBOM
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM from Mend
uses: ClickHouse/ClickBom@v1.0.10
with:
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: clickbom.json
sbom-source: mend
mend-email: ${{ secrets.CLICKBOM_MEND_EMAIL }}
mend-org-uuid: ${{ secrets.CLICKBOM_MEND_ORG_UUID }}
mend-user-key: ${{ secrets.CLICKBOM_MEND_USER_KEY }}
mend-product-uuid: ${{ secrets.CLICKBOM_MEND_PRODUCT_UUID }}
mend-project-uuid: ${{ secrets.CLICKBOM_MEND_PROJECT_UUID }}
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
```
### 从 Wiz 下载 SBOM
如果您想从 Wiz 下载 SBOM，可以使用以下示例。此示例假设您已在 GitHub Secrets 中配置了必要的 Wiz 凭证。
```
name: Upload SBOM
on:
push:
branches:
- main
jobs:
clickbom:
name: ClickBOM
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Configure AWS Credentials
id: aws-creds
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::012345678912:role/GitHubOIDCRole
role-session-name: clickbom-session
aws-region: us-east-1
- name: Upload SBOM from Wiz
uses: ClickHouse/ClickBom@v1.0.10
with:
aws-access-key-id: ${{ steps.aws-creds.outputs.aws-access-key-id }}
aws-secret-access-key: ${{ steps.aws-creds.outputs.aws-secret-access-key }}
s3-bucket: my-sbom-bucket
s3-key: clickbom.json
sbom-source: wiz
wiz-auth-endpoint: ${{ secrets.CLICKBOM_WIZ_AUTH_ENDPOINT }}
wiz-api-endpoint: ${{ secrets.CLICKBOM_WIZ_API_ENDPOINT }}
wiz-client-id: ${{ secrets.CLICKBOM_WIZ_CLIENT_ID }}
wiz-client-secret: ${{ secrets.CLICKBOM_WIZ_CLIENT_SECRET }}
wiz-report-id: ${{ secrets.CLICKBOM_WIZ_REPORT_ID }}
clickhouse-url: ${{ secrets.CLICKHOUSE_URL }}
clickhouse-database: ${{ secrets.CLICKHOUSE_DATABASE }}
clickhouse-username: ${{ secrets.CLICKHOUSE_USERNAME }}
clickhouse-password: ${{ secrets.CLICKHOUSE_PASSWORD }}
```
## 创建 GitHub App
- 按照[此处](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app)的说明创建 GitHub App。
- 确保为该 App 授予对 `Contents` 和 `Metadata` 的 `Read access`（读取权限）。
- 将该 App 安装在您想要使用它的仓库上。
- 为该 App 生成一个私钥，并将其保存在安全的地方，例如 GitHub Secrets。
标签：API集成, AWS S3, CICD集成, CISA项目, ClickHouse, DevSecOps, DNS解析, EVTX分析, GPT, Mend, SBOM管理, WebSocket, Wiz, 上游代理, 云存储, 代码示例, 依赖分析, 可观测性, 安全合规, 开源项目, 数据分析, 数据导入, 漏洞管理, 组件分析, 网络代理, 自动化安全工具, 跌倒检测, 软件开发, 软件物料清单