Qwen-7B推理教程【Python调用+web端部署】

前提

• 操作系统为Ubuntu22.04.4 LTS
• 安装Anaconda（本人安装教程如下）

Ubuntu22.04.4 LTS系统/安装Anaconda【GPU版】-CSDN博客

• 安装python3.9/pytorch/torchvision（本人安装教程如下）

Ubuntu22.04.4系统/安装python3.9/pytorch/torchvision【GPU版】-CSDN博客

1、conda虚拟环境

conda activate QwenChat

2、下载Qwen代码

git clone https://github.com/QwenLM/Qwen.git

3、安装依赖环境

cd Qwen

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

pip install -r requirements_web_demo.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

4、安装魔搭

pip install modelscope transformers -i https://pypi.tuna.tsinghua.edu.cn/simple

5、测试模型

5.1 Python调用

5.1.1 创建chat-7b.py

touch chat-7b.py

5.1.2 编辑chat-7b.py

vi chat-7b.py

chat-7b.py文件内容如下

from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig
#可选的模型包括: "qwen/Qwen-7B-Chat", "qwen/Qwen-14B-Chat"
tokenizer = AutoTokenizer.from_pretrained("qwen/Qwen-7B-Chat", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("qwen/Qwen-7B-Chat", device_map="auto", trust_remote_code=True, fp16=True).eval()
model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B-Chat", trust_remote_code=True) # 可指定不同的生成长度、top_p等相关超参
response, history = model.chat(tokenizer, "你好", history=None)
print(response)
response, history = model.chat(tokenizer, "上海好玩吗？", history=history)
print(response)
response, history = model.chat(tokenizer, "七月份去上海，有推荐的旅游攻略吗？", history=history)
print(response)

5.1.3 运行chat-7b.py

python chat-7b.py

5.1.4 测试成功

5.2 web端部署

5.2.1 编辑web_demo.py

vi web_demo.py

修改内容如下，将transformers修改为modelscope

...
import gradio as gr
import mdtex2html
import torch
#from transformers import AutoModelForCausalLM, AutoTokenizer
#from transformers.generation import GenerationConfig
from modelscope import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
DEFAULT_CKPT_PATH = 'qwen/Qwen-7B-Chat-Int4'
...

5.2.2 其他安装需求

pip install auto-gptq
pip install optimum
pip install --upgrade gradio

5.2.3 运行web_demo.py

python web_demo.py

5.2.4 测试成功

本人安装浏览器的教程，自取链接如下

Ubuntu下载安装chrome浏览器-CSDN博客

参考链接

【Ubuntu20.04部署通义千问Qwen-7B，实测成功】_ubuntu上部署qwen-CSDN博客^v100^pc_search_result_base2&spm=1018.2226.3001.4187