当前位置:   article > 正文

chatglm llm实时流api接口及post访问_chatglm api

chatglm api

参考:
https://github.com/THUDM/ChatGLM-6B/pull/573/commits/02947052eefe392fd9f9632894e9551a805c6109
https://github.com/THUDM/ChatGLM-6B/pull/573

1、代码:

提前安装:
sse_starlette、fastapi

python  fastapi_api.py
  • 1

fastapi_api.py

from fastapi import FastAPI
from pydantic import BaseModel
from typing import Optional
from sse_starlette.sse import EventSourceResponse
from transformers import AutoTokenizer, AutoModel
import uvicorn
import torch

'''
 此脚本实现模型响应结果的流式传输,让用户无需等待完整内容的响应。
 This script implements the streaming transmission of model response results, eliminating the need for users to wait for a complete response of the content.
 访问接口时它将返回event-stream流,你需要在客户端接收并处理它。
 When accessing the interface, it will return an 'event-stream' stream, which you need to receive and process on the client.
 POST http://127.0.0.1:8010
 { "i
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/你好赵伟/article/detail/348534
推荐阅读
相关标签
  

闽ICP备14008679号