huggingface pipeline使用清华智谱的模型THUDM/chatglm3-6b_automodelforcausallm 加载"清华智谱"大模型

以下代码成功运在CPU机器上，需要32G内存：
**Google colab 中运行报内存不足 * *
第一次运行会自动下载模型文件，需要比较长的时间。

from transformers import AutoTokenizer, AutoModel
from transformers import AutoModelForSeq2SeqLM
from huggingface_hub.hf_api import HfFolder

HfFolder.save_token('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')
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True, device='cpu')
model = model.eval()
response = model.chat(tokenizer, "你好", history=[])
print(response)
1
2
3
4
5
6
7
8
9
10

以下使用GPU运行chatglm3-6b：

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True).half().cuda()
model = model.eval()
for i in range(10):
    response, history = model.chat(tokenizer, "来一首唐诗", history=[])
    print(response)
1
2
3
4
5
6
7