epappas/llmtrace

# LLMTrace [](https://github.com/epappas/llmtrace/actions/workflows/ci.yml) [](https://github.com/epappas/llmtrace/actions/workflows/security.yml) [](https://crates.io/crates/llmtrace) [](https://pypi.org/project/llmtracing/) [](LICENSE) [](https://www.rust-lang.org/) [](https://github.com/epappas/llmtrace/stargazers) **Zero-code LLM observability and security for production.** LLMTrace is a transparent proxy that captures, analyzes, and secures your LLM interactions in real-time. Drop it between your app and any OpenAI-compatible API to get instant visibility into prompt injection attacks, PII leaks, cost overruns, and performance bottlenecks — without changing a single line of code. ## 为什么选择 LLMTrace？ Production LLM applications face three critical blind spots: - **Security vulnerabilities** — Prompt injection, data leakage, PII exposure - **Cost runaway** — Uncontrolled API spend, inefficient token usage - **Performance opacity** — No visibility into latency, failure rates, or user behavior LLMTrace solves this by sitting transparently between your application and LLM providers, giving you complete observability and control. ## 关键特性 - **Transparent Proxy** — Drop-in replacement for any OpenAI-compatible API - **ML Ensemble Detection** — Multi-detector majority voting (regex, DeBERTa, InjecGuard, PIGuard) - **Real-time Security** — Prompt injection detection, PII scanning, data leakage prevention - **Performance Monitoring** — Latency, token usage, streaming metrics (TTFT), error tracking - **Cost Control** — Per-agent budgets, rate limits, anomaly detection - **Multi-tenant Ready** — Isolated per API key or custom tenant headers - **High Performance** — Built in Rust, handles streaming responses, circuit breaker protection ## 安全性与性能 | Metric | Value | |-----------|-------| | Accuracy | 87.6% | | Precision | 95.5% | | F1 Score | 86.9% | | Recall | 79.7% | Tested on a 153-sample adversarial corpus across 12 attack categories including CyberSecEval2, BIPIA, TensorTrust, and InjecAgent. See [benchmarks/](benchmarks/) for methodology and full results. ## 快速开始 ### 1. 安装 ``` curl -sS https://raw.githubusercontent.com/epappas/llmtrace/main/scripts/install.sh | bash ``` Or use one of the other methods: ``` cargo install llmtrace # from crates.io docker pull ghcr.io/epappas/llmtrace-proxy:latest # Docker ``` ### 2. 运行 ``` export OPENAI_API_KEY="sk-..." llmtrace-proxy --config config.yaml ``` ### 3. 使用现有代码尝试 ``` import openai # 在此之前：直接指向 OpenAI client = openai.OpenAI() # 之后：指向 LLMTrace 代理（仅此而已！） client = openai.OpenAI(base_url="http://localhost:8080/v1") # 您的代码完全不变 response = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": "Hello!"}] ) ``` ### 4. 查看追踪记录 ``` # 查看最近活动 curl http://localhost:8080/api/v1/traces | jq '.[0]' # 检查安全发现 curl http://localhost:8080/api/v1/security/findings | jq # 监控成本 curl http://localhost:8080/api/v1/costs/current | jq ``` **That's it!** You now have full observability into your LLM interactions. ## 架构 ``` graph LR A[Your Application] -->|HTTP| B[LLMTrace Proxy] B -->|HTTP| C[OpenAI/LLM Provider] B -->|Async| D[Security Engine] B -->|Async| E[Storage Engine] D --> F[SQLite/PostgreSQL] E --> F D --> G[Real-time Alerts] H[Dashboard] -->|REST API| B I[Monitoring] -->|Metrics API| B style B fill:#e1f5fe style D fill:#fff3e0 style E fill:#f3e5f5 ``` **How it works:** 1. **Transparent Proxy** — Your app sends requests to LLMTrace instead of OpenAI 2. **Pass-through** — LLMTrace forwards requests to the real LLM provider 3. **Background Analysis** — Security analysis and trace capture happen asynchronously 4. **Zero Impact** — Your application never waits for analysis, even if something fails ## 集成示例 ### OpenAI Python SDK ``` import openai # 只需更改 base_url client = openai.OpenAI( base_url="http://localhost:8080/v1", api_key="your-openai-key" ) ``` ### OpenAI Node.js SDK ``` import OpenAI from 'openai'; const openai = new OpenAI({ baseURL: 'http://localhost:8080/v1', apiKey: 'your-openai-key' }); ``` ### LangChain ``` from langchain_openai import ChatOpenAI llm = ChatOpenAI( base_url="http://localhost:8080/v1", api_key="your-openai-key" ) ``` ### curl ``` curl http://localhost:8080/v1/chat/completions \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello!"}]}' ``` **[View all integration guides ->](docs/README.md#guides)** ## 仪表板与监控 LLMTrace includes a built-in dashboard for visualizing traces, security findings, and costs: ``` # 访问仪表板 open http://localhost:3000 # 或使用 REST API curl http://localhost:8080/api/v1/traces curl http://localhost:8080/api/v1/security/findings curl http://localhost:8080/api/v1/costs/current ``` **Dashboard features:** - Real-time trace visualization - Security incident timeline - Cost breakdown by model/agent - Performance metrics and alerts ## 配置 ### 最小配置 ``` # config.yaml upstream_url: "https://api.openai.com" listen_addr: "0.0.0.0:8080" storage: profile: "lite" # SQLite for simple deployments security: enable_prompt_injection_detection: true enable_pii_detection: true ``` ### 生产配置 ``` # config.yaml upstream_url: "https://api.openai.com" listen_addr: "0.0.0.0:8080" storage: profile: "production" postgres_url: "postgresql://user:pass@localhost/llmtrace" clickhouse_url: "http://localhost:8123" redis_url: "redis://localhost:6379" security: enable_prompt_injection_detection: true enable_pii_detection: true enable_streaming_analysis: true cost_control: daily_budget_usd: 1000 per_agent_daily_budget_usd: 100 alerts: slack: webhook_url: "https://hooks.slack.com/..." rate_limiting: requests_per_minute: 1000 burst_capacity: 2000 ``` **[Full configuration guide ->](docs/getting-started/configuration.md)** ## API 参考 | Endpoint | Description | |----------|-------------| | `GET /api/v1/traces` | List recent traces | | `GET /api/v1/traces/{id}` | Get specific trace details | | `GET /api/v1/security/findings` | List security incidents | | `GET /api/v1/costs/current` | Cost breakdown and usage | | `GET /health` | Health check and circuit breaker status | | `POST /policies/validate` | Validate custom security policies | **[Full API documentation ->](docs/guides/API.md)** ## 安装 ### Cargo (Rust 代理) ``` cargo install llmtrace llmtrace-proxy --config config.yaml ``` ### Pip (Python SDK) ``` pip install llmtracing ``` ``` import llmtrace tracer = llmtrace.configure({"enable_security": True}) span = tracer.start_span("chat_completion", "openai", "gpt-4") span.set_prompt("Hello!") span.set_response("Hi there!") print(span.to_dict()) ``` ### Docker ``` docker pull ghcr.io/epappas/llmtrace-proxy:latest docker run -p 8080:8080 ghcr.io/epappas/llmtrace-proxy:latest ``` ### 带依赖的 Docker Compose ``` curl -o compose.yaml https://raw.githubusercontent.com/epappas/llmtrace/main/compose.yaml docker compose up -d ``` ### 从源码构建 ``` git clone https://github.com/epappas/llmtrace cd llmtrace cargo build --release --features ml ./target/release/llmtrace-proxy --config config.yaml ``` ### Kubernetes ``` helm install llmtrace ./deployments/helm/llmtrace ``` **[Installation guide with all methods ->](docs/getting-started/installation.md)** ## 生产部署 ### 高可用性设置 - **Load Balancer** -> Multiple LLMTrace instances - **PostgreSQL** for persistent trace storage - **ClickHouse** for high-volume analytics - **Redis** for caching and rate limiting ### 安全最佳实践 - API key validation and tenant isolation - TLS termination at load balancer - Network segmentation between components - Regular security policy updates ### 监控与告警 - Prometheus metrics export - Grafana dashboards - PagerDuty/Slack integration - OWASP LLM Top 10 compliance reporting **[Production deployment guide ->](docs/deployment/kubernetes.md)** ## 贡献 We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details. ### 开发环境设置 ``` git clone https://github.com/epappas/llmtrace cd llmtrace cargo build --workspace cargo test --workspace ``` ### 项目结构 | Crate | Package | Purpose | |-------|---------|---------| | `llmtrace-core` | - | Shared types and traits | | `llmtrace` | [crates.io](https://crates.io/crates/llmtrace) | HTTP proxy server (`cargo install llmtrace`) | | `llmtrace-security` | - | Security analysis engine (regex + DeBERTa + InjecGuard + PIGuard ensemble) | | `llmtrace-storage` | - | Storage backends (SQLite, PostgreSQL, ClickHouse, Redis) | | `llmtrace-python` | [PyPI](https://pypi.org/project/llmtracing/) | Python SDK (`pip install llmtracing`, imports as `import llmtrace`) | **[Development guide ->](CONTRIBUTING.md)** ## 许可证 [MIT](LICENSE) - Free for commercial and personal use. **Star this repo** if LLMTrace helps secure your LLM applications! **Found a bug?** [Open an issue](https://github.com/epappas/llmtrace/issues) **Questions?** [Start a discussion](https://github.com/epappas/llmtrace/discussions)

标签：API安全, API集成, DeBERTa, InjecGuard, JSON输出, PIGuard, PII扫描, Rust实现, SEO关键词, 代理服务, 令牌用量, 可观测性, 可视化界面, 安全合规, 实时检测, 延迟监控, 开源代理, 性能监控, 成本控制, 搜索引擎查询, 数据泄露防护, 机器学习集成, 正则检测, 生产级监控, 网络代理, 网络探测, 透明中间件, 透明代理, 通知系统, 零代码安全