Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记

Atlas 300I duo mindie 部署qwen3-vl-30b-a3b笔记

第一步 mindie镜像下载安装

镜像地址：https://www.hiascend.com/developer/ascendhub/detail/af85b724a7e5469ebd7ea13c3439d48f
docker pull --platform=arm64 swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:3.0.0b2-300I-Duo-py311-openeuler24.03-lts

第二步

dockerrun-it-d\--shm-size=64g\--namemindie_310\--device=/dev/davinci0\--device=/dev/davinci1\--device=/dev/davinci2\--device=/dev/davinci3\--device=/dev/davinci4\--device=/dev/davinci5\--device=/dev/davinci_manager\--device=/dev/hisi_hdc\-v/usr/local/Ascend/driver:/usr/local/Ascend/driver:ro\-v/usr/local/dcmi:/usr/local/dcmi:ro\-v/usr/local/bin/npu-smi:/usr/local/bin/npu-smi:ro\-v/usr/local/Ascend/firmware:/usr/local/Ascend/firmware:ro\-v/usr/local/sbin:/usr/local/sbin:ro\-v/data/alg:/root/alg\-p29511:29511\-p29512:29512\-p29513:29513\-p1025:1025\-p1026:1026\-p1027:1027\-p1121:1121\swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:3.0.0-300I-Duo-py311-openeuler24.03-lts\bash

第三步

dockerexec-itmindie_310 /bin/bashcd/usr/local/Ascend/atb-models/requirements/models pipinstall-rrequirements_qwen3vl.txt

第四步

cd/usr/local/Ascend/mindie/latestchmod750mindie-servicechmod-R550mindie-service/binchmod550mindie-service/libchmod440mindie-service/lib/*chmod550mindie-service/lib/grpcchmod440mindie-service/lib/grpc/*chmod-R550mindie-service/includechmod-R550mindie-service/scriptschmod750mindie-service/logschmod750mindie-service/confchmod640mindie-service/conf/config.jsonchmod700mindie-service/securitychmod-R700mindie-service/security/*

第五步

# 配置CANN环境，默认安装在/usr/local目录下source/usr/local/Ascend/ascend-toolkit/set_env.sh# 配置加速库环境source/usr/local/Ascend/nnal/atb/set_env.sh# 配置模型仓环境变量source/usr/local/Ascend/atb-models/set_env.sh# MindIEsource/usr/local/Ascend/mindie/latest/mindie-llm/set_env.shsource/usr/local/Ascend/mindie/latest/mindie-service/set_env.sh# 开启MindIE日志打印exportMINDIE_LOG_TO_STDOUT="true"

第六步

cd/usr/local/Ascend/mindie/latest/mindie-servicecdmindie-service/confvimconfig.json 修改"httpsEnabled":false修改"modelName":"qwen2-7b","modelWeightPath":"/home/weight",

其余参数视情况修改
注意

"maxSeqLen":1280000,#2560倍数"maxInputTokenLen":1024000,1024倍数"maxPrefillTokens":1024000,需大于maxInputTokenLen

第七步 启动服务

nohup./bin/mindieservice_daemon>output.log2>&1&或 ./bin/mindieservice_daemon 回显 Daemon start success!表明启动成功

测试

importtimeimportrequestsimportjsondeftest_generation_speed(prompt="输出工地安全准则",max_tokens=1204000,runs=3):url="http://127.0.0.1:1025/v1/chat/completions"total_tokens=0total_time=0foriinrange(runs):print(f"\n--- Run{i+1}---")start_time=time.time()generated_tokens=0response=requests.post(url,headers={"Content-Type":"application/json"},json={"model":"qwen3-vl-qwen3-vl-a3b","messages":[{"role":"user","content":prompt}],"max_tokens":max_tokens,# 增加生成长度"temperature":0.7,"stream":True},timeout=3600,# 增加超时时间stream=True)ifresponse.status_code==200:print("生成内容: ",end="",flush=True)forlineinresponse.iter_lines():ifline:line=line.decode('utf-8')ifline.startswith('data: '):ifline.strip()=='data: [DONE]':breaktry:chunk=json.loads(line[6:])content=chunk['choices'][0]['delta'].get('content','')ifcontent:print(content,end="",flush=True)generated_tokens+=len(content.split())except:passprint()# 换行else:print(f"Run{i+1}: Error{response.status_code}-{response.text}")continueduration=time.time()-start_time speed=generated_tokens/durationifduration>0else0print(f"Run{i+1}: ~{generated_tokens}tokens in{duration:.2f}s →{speed:.2f}tokens/s")total_tokens+=generated_tokens total_time+=durationiftotal_time>0:print(f"\nAverage:{total_tokens/total_time:.2f}tokens/s over{runs}runs")if__name__=="__main__":test_generation_speed()

输出

其他：容器简单推理测化命令

torchrun--standalone--nnodes=1--nproc_per_node=1\examples/run_pa.py\--model_path/root/alg/qwen3-vl-8b\--input_texts"什么是深度学习？""深度学习和机器学习有什么区别？""请举例说明深度学习的应用场景。"\--max_output_length512\--max_batch_size3\--trust_remote_code

参考连接

https://www.hiascend.com/document/detail/zh/mindie/300/envdeployment/instg/docs/zh/user_guide/install/source/image_usage_guide.md
https://gitcode.com/Ascend/MindIE-LLM/blob/v3.0.0/docs/zh/user_guide/quick_start/quick_start.md