Skip to content

FastDeploy完整启动代码 , fastdeploy.entrypoints.openai.api_server #4019

@qinghuacui99-web

Description

@qinghuacui99-web

使用下面的启动命令:
命令1:
python -m fastdeploy.entrypoints.openai.api_server
--model Qwen/Qwen3-0.6B
--port 8180
--metrics-port 8181
--engine-worker-queue-port 8182
--max-model-len 8192
--max-num-seqs 4
--load_choices "default_v1"
命令2:
python -m fastdeploy.entrypoints.openai.api_server
--model baidu/ERNIE-4.5-0.3B-Paddle
--port 8180
--metrics-port 8181
--engine-worker-queue-port 8182
--max-model-len 8192
--max-num-seqs 4

都出现加载模型权重错误,报错如下:
ValueError: (InvalidArgument) matmul(): argument 'x' (position 0) must be Tensor, but got list (at /paddle/paddle/fluid/pybind/eager_utils.cc:1508)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions