Skip to content

[Serve] Add timeout parameter for deploy  #25433

@ArzelaAscoIi

Description

@ArzelaAscoIi

Description

It would be helpful to have a public-facing timeout for deploy.

Optionally: rollback the deployment request.

Use case

Since this parameter is currently set to -1 by default and it is not passed to the deploy method. Without being able to set this parameter your process might end up waiting for more worker nodes, but your cluster already started 10/10 nodes.
In this case it would be helpful to set this timeout and retry later, since other deployments might have been removed.

You can find a draft of this here: master...ArzelaAscoIi:enhancement/timeoutForServeDeployments
(This does not include some rollback of the request)

Workarounds are:
1. First checking if there are sufficient available resources
2. Adding a manual timeout implementation <- this would not roll back the deployment request

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issue, but not time-criticalenhancementRequest for new feature and/or capabilitypending-cleanupThis issue is pending cleanup. It will be removed in 2 weeks after being assigned.serveRay Serve Related Issue

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions