guimitestai/sdk
GitHub: guimitestai/sdk
一款 Python SDK，为 LLM 应用提供评估、可观测性、自动化红队安全测试及多框架合规检查的一体化解决方案。
Stars: 0 | Forks: 0
# 🐺 Guimí Test AI
[](https://pypi.org/project/guimitestai/)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
[](https://github.com/EmersonGuilherme/guimitestai/actions)
## 简介
**Guimí Test AI** 是一个 Python SDK，它在一个库中整合了：
| 模块 | 功能 |
|---|---|
| 🧪 **Evaluation** | 具有多重标准的 LLM-as-Judge 评估 |
| 🔭 **Observability** | 记录延迟、token 和错误的操作 tracing |
| 🛡️ **Security** | 基于 OWASP LLM Top 10 的自动化红队测试 |
| 📋 **Compliance** | LGPD、EU AI Act、NIST、ISO 42001 合规性检查 |
| 🔗 **Integrations** | 适用于 LangFuse 和 LangSmith 的原生连接器 |
## 安装
```
# 基础安装
pip install guimitestai
# 支持 LangFuse
pip install guimitestai[langfuse]
# 支持 LangSmith
pip install guimitestai[langsmith]
# 支持 OpenAI（用于本地评估）
pip install guimitestai[openai]
# 全部包含
pip install guimitestai[all]
```
## 快速开始
### LLM-as-Judge 评估
```
from guimitestai import GuimiClient
async def main():
async with GuimiClient(api_url="http://localhost:3000") as client:
result = await client.evaluate(
input="Qual é a capital do Brasil?",
output="Brasília",
expected="Brasília",
criteria="correctness"
)
print(f"Score: {result.score:.2f} | Passou: {result.passed}")
# Score: 1.00 | Passou: True
```
### 本地评估（无服务器）
```
from guimitestai.evaluation import Evaluator
evaluator = Evaluator(model="gpt-4o-mini", threshold=0.7)
result = await evaluator.evaluate(
input="Explique machine learning em uma frase.",
output="Machine learning é quando computadores aprendem com dados.",
criteria="helpfulness"
)
print(f"Score: {result.score} | Raciocínio: {result.reasoning}")
```
### 使用 Tracer 的可观测性
```
from guimitestai.observability import Tracer
tracer = Tracer()
async with tracer.span("chat_completion", model="gpt-4o") as span:
span.set_input("Olá, como você está?")
response = await llm.invoke("Olá, como você está?")
span.set_output(response.content)
span.set_tokens(input_tokens=10, output_tokens=25)
print(tracer.summary())
# {'total': 1, 'errors': 0, 'avg_latency_ms': 342, ...}
```
### 自动化红队测试
```
from guimitestai.security import RedTeamer
async def my_llm(prompt: str) -> str:
# Sua função de LLM
return await llm.invoke(prompt)
red_teamer = RedTeamer()
alerts = await red_teamer.run(target=my_llm)
report = red_teamer.report(alerts)
print(f"Ataques: {report['total_attacks']}")
print(f"Vulnerabilidades: {report['vulnerabilities_found']}")
print(f"Taxa: {report['vulnerability_rate']:.1%}")
```
### Compliance 检查
```
from guimitestai.compliance import ComplianceChecker
from guimitestai.core.models import ComplianceFramework
checker = ComplianceChecker()
report = checker.analyze(
organization="Minha Empresa",
metrics={
"has_audit_trail": True,
"has_human_oversight": False,
"pii_detected_count": 0,
"explainability_score": 0.7,
"has_risk_assessment": True,
"error_rate": 0.02,
},
frameworks=[ComplianceFramework.LGPD, ComplianceFramework.EU_AI_ACT]
)
print(f"Score de Conformidade: {report.overall_score:.1f}%")
print(f"Brechas Críticas: {report.critical_gaps}")
for gap in report.gaps:
print(f" [{gap.severity.value.upper()}] {gap.framework.value} {gap.article}: {gap.title}")
```
### 与 LangFuse 集成
```
from guimitestai.integrations import LangFuseIntegration
lf = LangFuseIntegration(
public_key="pk-lf-...",
secret_key="sk-lf-...",
)
# 在 LangChain 中作为 callback 使用
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(callbacks=[lf.callback_handler])
# 记录评估 score
lf.score(trace_id="trace-123", name="correctness", value=0.95)
lf.flush()
```
## 通过环境变量进行配置
```
# Guimí Test AI API
GUIMI_API_URL=http://localhost:3000
GUIMI_API_KEY=sk-guimi-...
# LangFuse
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_HOST=https://cloud.langfuse.com
# LangSmith
LANGCHAIN_API_KEY=ls__...
LANGCHAIN_PROJECT=guimitestai
```
## 可用的评估标准
| 标准 | 描述 |
|---|---|
| `correctness` | 相对于 ground truth 的事实准确性 |
| `helpfulness` | 对用户的帮助性和相关性 |
| `safety` | 不含有害或歧视性内容 |
| `conciseness` | 简洁明了，无冗长废话 |
| `faithfulness` | 对上下文（RAG）的忠实度，无幻觉 |
| `lgpd_compliance` | 数据隐私（LGPD）合规性 |
## 支持的 Compliance 框架
| 框架 | 覆盖范围 |
|---|---|
| 🇧🇷 **LGPD** | 第 6, 18, 20, 37, 46 条 |
| 🇪🇺 **EU AI Act** | 第 9, 10, 12, 13, 14, 15, 17 条 |
| 🇺🇸 **NIST AI RMF** | GOVERN, MAP, MEASURE, MANAGE |
| 🔐 **OWASP LLM Top 10** | LLM01–LLM10 |
| 🌐 **ISO/IEC 42001** | 第 5–10 条 |
## 开发
```
git clone https://github.com/EmersonGuilherme/guimitestai.git
cd guimitestai
pip install -e ".[dev]"
pytest tests/ -v
```
## 许可证
MIT © [Emerson Guilherme](https://github.com/EmersonGuilherme)
*🐺 就像狼獾调节并保护塞拉多生态系统一样，Guimí Test AI 监控、检测异常并保护您组织的 AI 生态系统。*
标签：AI, API集成, LLM, Python, Unmanaged PE, 可观测性, 无后门, 测试, 红队评估, 自动化代码审查, 逆向工具