birne420/hiwi-mitm-al4-ingest-demo
GitHub: birne420/hiwi-mitm-al4-ingest-demo
基于 Docker 搭建的演示环境，利用 mitmproxy 截获网络流量并通过 AssemblyLine4 自动导入流水线，实现高级 YARA 规则对网络流量的检测评估。
Stars: 0 | Forks: 0
# MITM AssemblyLine 导入演示
## AssemblyLine 安装 (`al4_install.sh`)
1. Docker 引擎前置条件
sudo apt update && \
sudo apt install ca-certificates curl && \
sudo install -m 0755 -d /etc/apt/keyrings && \
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc && \
sudo chmod a+r /etc/apt/keyrings/docker.asc && \
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null && \
sudo apt update && \
sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin && \
sudo systemctl start docker && \
sudo groupadd docker ; \
sudo usermod -aG docker $USER && \
newgrp docker
- https://docs.docker.com/engine/install/debian/
- https://docs.docker.com/engine/install/
2. Docker 引擎配置（地址空间）
echo "{\"insecure-registries\": [\"172.17.0.1:5000\"], \"default-address-pools\": [{\"base\": \"172.17.0.0/16\", \"size\": 24}]}" | sudo tee /etc/docker/daemon.json && sudo systemctl restart docker
3. AssemblyLine 安装
git clone https://github.com/CybercentreCanada/assemblyline-docker-compose.git ~/deployments/assemblyline && \
openssl req -nodes -x509 -newkey rsa:4096 -keyout ~/deployments/assemblyline/config/nginx.key -out ~/deployments/assemblyline/config/nginx.crt -days 365 -subj "/C=CA/ST=Ontario/L=Ottawa/O=CCCS/CN=assemblyline.local"
cd ~/deployments/assemblyline && \
sudo docker compose pull --ignore-buildable && \
sudo env COMPOSE_BAKE=true docker compose build && \
sudo docker compose -f bootstrap-compose.yaml pull
4. 设置 YaraCustom 服务
git clone https://gitti.cs.uni-magdeburg.de/birnbaum/mitm-ingest-demo ~/git/mitm-ingest-demo && \
cd ~/git/mitm-ingest-demo/al4_yara_custom && \
docker build -t testing/assemblyline-service-yara-custom . && \
cd ~/git/mitm-ingest-demo && \
docker compose build
## 启动带有 YaraCustom 的 AssemblyLine (`al4_start.sh`)
```
docker stop registry 2> /dev/null ; \
docker rm registry 2> /dev/null ; \
docker run -d --name registry --restart=always -p 5000:5000 registry
```
```
cd ~/git/mitm-ingest-demo/al4_yara_custom && \
docker build -t testing/assemblyline-service-yara-custom . && \
docker tag testing/assemblyline-service-yara-custom localhost:5000/testing/assemblyline-service-yara-custom:latest && \
docker tag testing/assemblyline-service-yara-custom localhost:5000/testing/assemblyline-service-yara-custom:stable && \
docker push --all-tags localhost:5000/testing/assemblyline-service-yara-custom
```
```
cd ~/deployments/assemblyline && \
sudo docker compose up -d --wait && \
sudo docker compose -f bootstrap-compose.yaml up
```
## 首次启动设置（手动步骤）
- 访问 'https://localhost' 并使用 'admin:admin' 登录
- 访问 'https://localhost/admin/services'，点击添加服务符号，将 `service_manifest.yml` 的内容粘贴到文本字段中（`cat ~/git/mitm-ingest-demo/al4_yara_custom/service_manifest.yml`）
- 访问 'https://localhost/account'，在 'Security' 下找到 'Manage API Keys'，创建名为 `devkey` 的 api key，将其放入 `api.key`（`echo \"devkey:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\" > ~/git/mitm-ingest-demo/watchdog/api.key`）
## 启动 YaraCustom 自动导入演示 (firefox+mitmproxy, `./yc_autoingest.sh`)
```
cd ~/git/mitm-ingest-demo && \
sudo rm -drf ./_data ; \
docker compose build && \
docker compose up --remove-orphans
```
- 通过 'http://localhost:5800' 访问 firefox 实例
## 关闭 AssemblyLine (`al4_stop.sh`)
```
cd ~/deployments/assemblyline && \
docker stop registry; docker rm registry; \
sudo docker compose down --remove-orphans
```
## 安装脚本（下载并放在桌面上）
```
git clone https://gitti.cs.uni-magdeburg.de/birnbaum/mitm-ingest-demo /tmp/gitmid && mv /tmp/gitmid/*.sh ~/Desktop/. && rm -drf /tmp/gitmid/
```
标签：AL4, AssemblyLine4, Cutter, DNS信息、DNS暴力破解, Docker, Docker Compose, GitHub, MITM, YARA, YARA规则, 中间人攻击, 云资产可视化, 反取证, 安全编排与自动化, 安全评估, 安全防御评估, 沙箱, 版权保护, 网络基础设施, 网络安全, 自动 ingest 管道, 自动化分析, 规则评估, 请求拦截, 跨站脚本, 逆向工具, 防御绕过, 隐私保护