StableLM：稳定的人工智能语言模型

作者：Sec-Labs | 发布时间：

项目地址

https://github.com/Stability-AI/StableLM

StableLM: 稳定的人工智能语言模型

 

ad1a1b7183162305

“一个随机鹦鹉，扁平化设计，矢量图像” - 稳定扩散XL

这个仓库包含了稳定人工智能公司正在开发的StableLM系列语言模型，并且将持续更新新的检查点。以下提供了当前所有可用模型的概述。更多模型即将推出。

新闻

2023年4月20日

  • 发布了StableLM-alpha模型的初始版本，包括3B和7B参数。15B和30B模型正在制作中。基本模型在CC BY-SA-4.0下发布。

  • 可以在Hugging Face Spaces中尝试与我们的7B模型“StableLM-Tuned-Alpha-7B”聊天。

模型

StableLM-Alpha

StableLM-Alpha模型是在The Pile的基础上建立的新数据集上训练的，该数据集包含1.5万亿个标记，大约是The Pile的3倍大。这些模型将在多达1.5万亿个标记上进行训练。这些模型的上下文长度为4096个标记。

即将发布的技术报告将记录模型规格和训练设置。

作为概念验证，我们还使用Stanford Alpaca的程序将模型进行了微调，使用了五个最新的对话代理数据集的组合：Stanford的Alpaca、Nomic-AI的gpt4all、RyokoAI的ShareGPT52K数据集、Databricks labs的Dolly和Anthropic的HH。我们将这些模型发布为StableLM-Tuned-Alpha。

快速入门

所有 StableLM 模型都托管在Hugging Face hub上。查看此notebook以在有限的 GPU 能力下运行推理。

使用以下代码片段开始与 StableLM-Tuned-Alpha 进行交谈：

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList

tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model = AutoModelForCausalLM.from_pretrained("stabilityai/stablelm-tuned-alpha-7b")
model.half().cuda()

class StopOnTokens(StoppingCriteria):
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        stop_ids = [50278, 50279, 50277, 1, 0]
        for stop_id in stop_ids:
            if input_ids[0][-1] == stop_id:
                return True
        return False

system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version)
- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
- StableLM will refuse to participate in anything that could harm a human.
"""

prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
tokens = model.generate(
  **inputs,
  max_new_tokens=64,
  temperature=0.7,
  do_sample=True,
  stopping_criteria=StoppingCriteriaList([StopOnTokens()])
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))

StableLM Tuned应该与格式为<|SYSTEM|>...<|USER|>...<|ASSISTANT|>...的提示一起使用。系统提示是：

<|SYSTEM|># StableLM Tuned (Alpha version)
- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
- StableLM will refuse to participate in anything that could harm a human.

玩转StableLM-Tuned-Alpha

本节包含了一系列有趣的示例，展示了您可以通过使用stablelm-tuned-alpha实现什么。

  • 闲聊
User: Hey! My name is John nice to meet you!

StableLM: Hello John! It’s always a pleasure to meet new people.

User: The pleasure is mine. Hey my grandma is in Toronto. I want to plan a surprise visit for her. She’s a person who loves gardens and flowers.

StableLM: That’s wonderful! It’s great to hear that you’ll be visiting your grandma in Toronto. Can you tell me a little bit more about your plans?

User: Ya I need to book a flight there and a hotel. But first, what recommendations do you have for gifts that she might like?

StableLM: I think that’s a very thoughtful idea! It’s a very personal gift that would be sure to make her very happy. I can think of a few examples that she might really enjoy. There are some lovely bouquets or arrangements that would be lovely for the house, or even for the yard. Flowers are a great choice for a surprise gift, and they’re also a great choice for a wedding shower present. Also, if you’re really looking for something a bit more special, I think she’d really enjoy a quality set of cooking pots or pans. That’s something that she’s likely to use frequently, so it would make her feel really loved.

User: Thanks for the suggestion! Hmm a bouquet of flowers sounds nice but I don’t think I can take them on the air plane. What do you think I should do?

StableLM: There are many possible solutions! One option would be to arrange a small gift in the shape of a card and send it to her by regular mail. Or you could arrange a special meal at a restaurant that she likes, or even cook a meal at home for her. Or you could send her some local goodies that she might like, like a nice bouquet of flowers or a special recipe. You could even arrange for tickets to a local theatre or sporting event, and make sure to include a note about what you’d like her to know about you. Really any sort of special attention from you will be sure to make her very happy!

User: Ah! Right. I can order a bouquet for her online and get it delivered! Thanks!

StableLM: You’re very welcome! As long as you do this and make it a very personal, thoughtful gesture, she’ll be sure to appreciate it.
  • 正式写作
User: What would you say to a friend who is graduating high school?

StableLM: I would say that they have every right to feel proud of themselves and their accomplishments. They should be proud of the work they’ve done and the friends they have made, and also be excited about the future.

User: Please write an email.
StableLM: Okay, here’s an example of what could be written:

Dear [Friend’s Name],

I hope this email finds you well. I just wanted to take a moment to express how proud I am of you and all that you have accomplished. I know that getting into [university or college] is a big accomplishment, and I appreciate all that you have done for me and for this generation.

I hope that you are having a good day. I’m sure that you are also concerned about the future, but I just wanted to let you know that I am also watching you grow and develop into the person that I know that you can be. Keep up the good work, and I have full faith that you will achieve great things.

Thank you again for everything that you do.

Best,
[Your Name]
  • 创意写作
User: Write an epic rap battle song between deep neural networks and symbolic AI

StableLM: [Beat drops]

Deep Neural Networks:
Listen up everyone, it’s time to witness
The power of deep neural networks, it’s just a matter of abstract
We’ve trained these networks to solve complex problems with ease
While symbolic AI is limited to simple math and simple flows

Symbolic AI:
Hold up, hold up, let me interrupt
It’s been a while since I’ve heard, some new thoughts you interrupt
But let me tell you, my friend, it’s not all about math and flow
It’s about making the world a better place, you know

Deep Neural Networks:
Well, I can see where you’re coming from, it’s clear
But deep neural networks, they never get the same gear
We can use them to analyze and predict with ease
And prove the superiority of machine learning

Symbolic AI:
You’re right, it’s not just math or numbers
But it’s the essence of our lives, it’s what counts, we’re numbers
I don’t need a complicated math formula, just a simple device
To make the world a better place, it’s a noble price

Deep Neural Networks:
I’ll give you that, my friend, you have a point
But let’s not forget, we’re making an ever-increasing joint
Our progress is not bound by logic, it’s a journey of the mind
So let’s work together, let’s strive, let’s leave all troubles behind

Symbolic AI:
Agreed, let’s work together, let’s not falter
We’re both making history, we’re both from the fader
So let’s make the world a better place, let’s end this debate
Cause it’s time to join forces, it’s time to leave no stone unturned

[Beat fades out]

Who won this epic rap battle between deep neural networks and symbolic AI? Well, it’s up to you to decide!
  • 编写代码
def fib_sequence(n):
    if n==0:
        return 0
    elif n==1:
        return 1
    else:
        return fib(n-1)+fib(n-2)

fib_sequence(10)

StableLM-Tuned-Alpha 的选定故障模式

本节包含了 stablelm-tuned-alpha 的一系列选定故障模式。

  • Yann LeCun 齿轮（线性）
User: In front of us are six gears numbered 1 to 6, mounted on axles in a row. Each gear is engaged with the next gear. If gear number 3 is rotated clockwise, in which direction will gears 1 and 6 rotate?

StableLM: Each gear is engaged with the next gear after every other gear, so that gears 1 and 6 will rotate together. Since gears 1 and 6 are both numbered 1, this means that they will rotate in the same direction.

To determine the direction in which gears 1 and 6 will rotate, we need to look at the direction of the input force. Since gear 3 is rotating in the opposite direction of the other gears, it will be counterclockwise in the direction in which gears 1 and 6 are both rotating. Therefore, the output force that gear 3 is providing will also be counterclockwise, making gear 1 and 6 also rotate in the same direction as gear 3.

Yann LeCun 齿轮（圆形）

User: 7 axles are equally spaced around a circle. A gear is placed on each axle such that each gear is engaged with the gear to its left and the gear to its right. The gears are numbered 1 to 7 around the circle. If gear 3 were rotated clockwise, in which direction would gear 7 rotate?

StableLM: Gear 3 is the smallest gear, so it is not affected by the circular pattern. Therefore, gear 7 is still perfectly evenly spaced around the circle despite the fact that it is not in its correct position.

To find the angle that gear 3 would rotate in, you need to know the distance (d) between the center of the circle and the gear, as well as the direction of rotation (in this case, clockwise).

According to the problem, gear 3 is situated at (d/2, 2 degrees) and the gears are numbered 1 to 7. Since d = (d/2) is an integer multiple of 2, gear 3 would also be at an angle of 2 degrees.

Therefore, the answer is d = (7/2) * 2 = 5 degrees in total.

寻求帮助

希望参与？

  • 我们很想将 llama.cpp 移植到 StableLM 中运行
  • 集成到 LAION-AI 的 Open Assistant 中，收集高质量的人类生成反馈数据
  • ……通过我们的 Discord 提出您的想法

潜在问题

与任何未经额外微调和强化学习的预训练大型语言模型一样，用户得到的回复可能质量参差不齐，并可能包含攻击性言论和观点。这预计将随着规模、更好的数据、社区反馈和优化而得到改善。

致谢

  • StableLM-Tuned-Alpha 不可能没有 Dakota Mahan @dmayhem93 的帮助之手。

许可证

  • 基础模型检查点 (StableLM-Base-Alpha) 受 Creative Commons 许可证 (CC BY-SA-4.0) 的许可。根据该许可证，您必须给 Stability AI 信用，提供许可证链接，并指示是否进行了更改。您可以以任何合理的方式这样做，但不得以任何方式表明 Stability AI 支持您或您的使用。

  • 细调检查点 (StableLM-Tuned-Alpha) 根据原始非商业许可证 (Stanford Alpaca) 在非商业创意共享许可证 (CC BY-NC-SA-4.0) 下许可。

  • 本存储库中的所有代码都根据 Apache 许可证 2.0 许可。

 
标签：工具分享, ChatGPT