dottxt-ai/outlines
GitHub: dottxt-ai/outlines
Outlines 确保大语言模型在生成阶段直接输出符合指定类型与结构的有效数据，消除事后解析的不确定性。
Stars: 14623 | Forks: 789
🗒️ *面向 LLM 的结构化输出* 🗒️
由 [.txt](https://dottxt.co) 团队用 ❤👷️ 打造
受到 NVIDIA、Cohere、HuggingFace、vLLM 等信赖 [][pypi] [][pypistats] [][stars] [][discord] [][dottxt-blog] [][twitter]
.txt API 目前处于早期访问阶段。**[在此处申请访问权限 →](https://h1xbpbfsf0w.typeform.com/to/fwQNWmS8?utm_source=github&utm_medium=organic&utm_campaign=outlines)**
🙋♂️ 客户支持分类
此示例展示了如何将自由格式的客户电子邮件转换为结构化的服务工单。通过解析优先级、类别和升级标志等属性，该代码能够实现对支持问题的自动化路由和处理。
```
import outlines
from enum import Enum
from pydantic import BaseModel
from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import List
MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
AutoTokenizer.from_pretrained(MODEL_NAME)
)
def alert_manager(ticket):
print("Alert!", ticket)
class TicketPriority(str, Enum):
low = "low"
medium = "medium"
high = "high"
urgent = "urgent"
class ServiceTicket(BaseModel):
priority: TicketPriority
category: str
requires_manager: bool
summary: str
action_items: List[str]
customer_email = """
Subject: URGENT - Cannot access my account after payment
I paid for the premium plan 3 hours ago and still can't access any features.
I've tried logging out and back in multiple times. This is unacceptable as I
have a client presentation in an hour and need the analytics dashboard.
Please fix this immediately or refund my payment.
"""
prompt = f"""
<|im_start|>user
Analyze this customer email:
{customer_email}
<|im_end|>
<|im_start|>assistant
"""
ticket = model(
prompt,
ServiceTicket,
max_new_tokens=500
)
# 使用 structured data 来路由 ticket
ticket = ServiceTicket.model_validate_json(ticket)
if ticket.priority == "urgent" or ticket.requires_manager:
alert_manager(ticket)
```
📦 电子商务产品分类
此用例演示了 outlines 如何将产品描述转换为结构化的分类数据（例如，主类别、子类别和属性），从而简化库存管理等任务。每个产品描述都会被自动处理，减少了人工分类的开销。
```
import outlines
from pydantic import BaseModel
from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import List, Optional
MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
AutoTokenizer.from_pretrained(MODEL_NAME)
)
def update_inventory(product, category, sub_category):
print(f"Updated {product.split(',')[0]} in category {category}/{sub_category}")
class ProductCategory(BaseModel):
main_category: str
sub_category: str
attributes: List[str]
brand_match: Optional[str]
# 批量处理产品描述
product_descriptions = [
"Apple iPhone 15 Pro Max 256GB Titanium, 6.7-inch Super Retina XDR display with ProMotion",
"Organic Cotton T-Shirt, Men's Medium, Navy Blue, 100% Sustainable Materials",
"KitchenAid Stand Mixer, 5 Quart, Red, 10-Speed Settings with Dough Hook Attachment"
]
template = outlines.Template.from_string("""
<|im_start|>user
Categorize this product:
{{ description }}
<|im_end|>
<|im_start|>assistant
""")
# 获取所有产品的 structured 分类
categories = model(
[template(description=desc) for desc in product_descriptions],
ProductCategory,
max_new_tokens=200
)
# 将分类用于库存管理
categories = [
ProductCategory.model_validate_json(category) for category in categories
]
for product, category in zip(product_descriptions, categories):
update_inventory(product, category.main_category, category.sub_category)
```
📊 从不完整数据中解析事件详情
此示例使用 outlines 将事件描述解析为结构化信息（如事件名称、日期、地点、类型和主题），甚至能处理数据不完整的情况。它利用 union types 返回结构化的事件数据或备选的“我不知道”答案，确保在各种场景下都能进行稳健的提取。
```
import outlines
from typing import Union, List, Literal
from pydantic import BaseModel
from enum import Enum
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
AutoTokenizer.from_pretrained(MODEL_NAME)
)
class EventType(str, Enum):
conference = "conference"
webinar = "webinar"
workshop = "workshop"
meetup = "meetup"
other = "other"
class EventInfo(BaseModel):
"""Structured information about a tech event"""
name: str
date: str
location: str
event_type: EventType
topics: List[str]
registration_required: bool
# 创建一个可以是 structured EventInfo 或 "I don't know" 的 union type
EventResponse = Union[EventInfo, Literal["I don't know"]]
# 示例事件描述
event_descriptions = [
# Complete information
"""
Join us for DevCon 2023, the premier developer conference happening on November 15-17, 2023
at the San Francisco Convention Center. Topics include AI/ML, cloud infrastructure, and web3.
Registration is required.
""",
# Insufficient information
"""
Tech event next week. More details coming soon!
"""
]
# 处理事件
results = []
for description in event_descriptions:
prompt = f"""
<|im_start>system
You are a helpful assistant
<|im_end|>
<|im_start>user
Extract structured information about this tech event:
{description}
If there is enough information, return a JSON object with the following fields:
- name: The name of the event
- date: The date where the event is taking place
- location: Where the event is taking place
- event_type: either 'conference', 'webinar', 'workshop', 'meetup' or 'other'
- topics: a list of topics of the conference
- registration_required: a boolean that indicates whether registration is required
If the information available does not allow you to fill this JSON, and only then, answer 'I don't know'.
<|im_end|>
<|im_start|>assistant
"""
# Union type allows the model to return structured data or "I don't know"
result = model(prompt, EventResponse, max_new_tokens=200)
results.append(result)
# 显示结果
for i, result in enumerate(results):
print(f"Event {i+1}:")
if isinstance(result, str):
print(f" {result}")
else:
# It's an EventInfo object
print(f" Name: {result.name}")
print(f" Type: {result.event_type}")
print(f" Date: {result.date}")
print(f" Topics: {', '.join(result.topics)}")
print()
# 在 downstream processing 中使用 structured data
structured_count = sum(1 for r in results if isinstance(r, EventInfo))
print(f"Successfully extracted data for {structured_count} of {len(results)} events")
```
🗂️ 将文档分类为预定义类型
在这个案例中，outlines 使用 literal 类型规范将文档分类到预定义的类别（例如，“财务报告”、“法律合同”）中。生成的分类结果以表格格式和类别分布摘要的形式展示，说明了结构化输出如何简化内容管理。
```
import outlines
from typing import Literal, List
import pandas as pd
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_NAME = "microsoft/Phi-3-mini-4k-instruct"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
AutoTokenizer.from_pretrained(MODEL_NAME)
)
# 使用 Literal 定义分类类别
DocumentCategory = Literal[
"Financial Report",
"Legal Contract",
"Technical Documentation",
"Marketing Material",
"Personal Correspondence"
]
# 待分类的示例文档
documents = [
"Q3 Financial Summary: Revenue increased by 15% year-over-year to $12.4M. EBITDA margin improved to 23% compared to 19% in Q3 last year. Operating expenses...",
"This agreement is made between Party A and Party B, hereinafter referred to as 'the Parties', on this day of...",
"The API accepts POST requests with JSON payloads. Required parameters include 'user_id' and 'transaction_type'. The endpoint returns a 200 status code on success."
]
template = outlines.Template.from_string("""
<|im_start|>user
Classify the following document into exactly one category among the following categories:
- Financial Report
- Legal Contract
- Technical Documentation
- Marketing Material
- Personal Correspondence
Document:
{{ document }}
<|im_end|>
<|im_start|>assistant
""")
# 对文档进行分类
def classify_documents(texts: List[str]) -> List[DocumentCategory]:
results = []
for text in texts:
prompt = template(document=text)
# The model must return one of the predefined categories
category = model(prompt, DocumentCategory, max_new_tokens=200)
results.append(category)
return results
# 执行分类
classifications = classify_documents(documents)
# 创建简单的 results table
results_df = pd.DataFrame({
"Document": [doc[:50] + "..." for doc in documents],
"Classification": classifications
})
print(results_df)
# 按类别统计文档
category_counts = pd.Series(classifications).value_counts()
print("\nCategory Distribution:")
print(category_counts)
```
📅 通过 Function Calling 根据请求安排会议
此示例演示了 outlines 如何解释自然语言的会议请求，并将其转换为与预定义 function 参数相匹配的结构化格式。提取出会议详情（例如，标题、日期、持续时间、参与者）后，这些信息将被用于自动安排会议。
```
import outlines
import json
from typing import List, Optional
from datetime import date
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_NAME = "microsoft/phi-4"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
AutoTokenizer.from_pretrained(MODEL_NAME)
)
# 定义带有 typed parameters 的 function
def schedule_meeting(
title: str,
date: date,
duration_minutes: int,
attendees: List[str],
location: Optional[str] = None,
agenda_items: Optional[List[str]] = None
):
"""Schedule a meeting with the specified details"""
# In a real app, this would create the meeting
meeting = {
"title": title,
"date": date,
"duration_minutes": duration_minutes,
"attendees": attendees,
"location": location,
"agenda_items": agenda_items
}
return f"Meeting '{title}' scheduled for {date} with {len(attendees)} attendees"
# Natural language 请求
user_request = """
I need to set up a product roadmap review with the engineering team for next
Tuesday at 2pm. It should last 90 minutes. Please invite john@example.com,
sarah@example.com, and the product team at product@example.com.
"""
# Outlines 自动从 function signature 推断所需的结构
prompt = f"""
<|im_start|>user
Extract the meeting details from this request:
{user_request}
<|im_end|>
<|im_start|>assistant
"""
meeting_params = model(prompt, schedule_meeting, max_new_tokens=200)
# 结果是一个与 function parameters 匹配的 dictionary
meeting_params = json.loads(meeting_params)
print(meeting_params)
# 使用提取的参数调用 function
result = schedule_meeting(**meeting_params)
print(result)
# "Meeting 'Product Roadmap Review' 安排在 2023-10-17，有 3 位 attendees"
```
📝 使用可重用模板动态生成 prompt
使用基于 Jinja 的模板，此示例展示了如何为情感分析等任务生成动态 prompt。它说明了如何轻松地为不同内容类型重用和自定义 prompt（包括 few-shot learning 策略），同时确保输出保持结构化。
```
import outlines
from typing import List, Literal
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_NAME = "microsoft/phi-4"
model = outlines.from_transformers(
AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto"),
AutoTokenizer.from_pretrained(MODEL_NAME)
)
# 1. 使用 Jinja syntax 创建可重用的 template
sentiment_template = outlines.Template.from_string("""
<|im_start>user
Analyze the sentiment of the following {{ content_type }}:
{{ text }}
Provide your analysis as either "Positive", "Negative", or "Neutral".
<|im_end>
<|im_start>assistant
""")
# 2. 使用不同的参数生成 prompts
review = "This restaurant exceeded all my expectations. Fantastic service!"
prompt = sentiment_template(content_type="review", text=review)
# 3. 将 templated prompt 用于 structured generation
result = model(prompt, Literal["Positive", "Negative", "Neutral"])
print(result) # "Positive"
# Templates 也可以从文件加载
example_template = outlines.Template.from_file("templates/few_shot.txt")
# 与 examples 结合用于 few-shot learning
examples = [
("The food was cold", "Negative"),
("The staff was friendly", "Positive")
]
few_shot_prompt = example_template(examples=examples, query="Service was slow")
print(few_shot_prompt)
```
标签：AI风险缓解, DLL 劫持, Petitpotam, Python, Schema约束, SOC Prime, 人工智能, 大语言模型, 开发工具, 无后门, 用户模式Hook绕过, 系统调用监控, 结构化输出, 逆向工具