Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: vllm-project/guidellm
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: vllm-project/guidellm
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: feat/adv_prefix
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 20 commits
  • 14 files changed
  • 3 contributors

Commits on Sep 30, 2025

  1. Add turns support to synthetic dataset 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    1fa186f View commit details
    Browse the repository at this point in the history

  2. Add basic multiturn loader support 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    7efb7b1 View commit details
    Browse the repository at this point in the history

  3. Make dict encoding recursive 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    3f0cdbc View commit details
    Browse the repository at this point in the history

  4. Use details for next request in chain 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    220377e View commit details
    Browse the repository at this point in the history

  5. Implement worker support for multiturn 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    3ac4df6 View commit details
    Browse the repository at this point in the history

  6. Cancel requests in conversation 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    a7bf690 View commit details
    Browse the repository at this point in the history

  7. Cancel whole conversation 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    e276f6c View commit details
    Browse the repository at this point in the history

  8. Implement multiturn history in openai backend 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    1de1c64 View commit details
    Browse the repository at this point in the history

  9. Add wait_then_requeue behavior 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    0e8713c View commit details
    Browse the repository at this point in the history

  10. Type cleanup 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    cd43b2c View commit details
    Browse the repository at this point in the history

  11. Add fixed prefix option to synthetic data 

    Signed-off-by: Samuel Monson <smonson@redhat.com>

Add prefix before decode

Signed-off-by: Samuel Monson <smonson@redhat.com>

Add unique single-token prefix to every request

Co-authored-by: Mehul <MEHTMEHUL@GMAIL.COM>
Co-authored-by: Samuel Monson <smonson@redhat.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson @MML-coder
    sjmonson and MML-coder committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    eade3a2 View commit details
    Browse the repository at this point in the history

  12. Update tests for new prefix patch and reduce the number of mocks 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    5795c02 View commit details
    Browse the repository at this point in the history

  13. Add more prefix bucket testcases 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    66b5311 View commit details
    Browse the repository at this point in the history

  14. Append prefix tokens only to first turn 

    Signed-off-by: Samuel Monson <smonson@redhat.com>
    @sjmonson
    sjmonson committed Sep 30, 2025
    Configuration menu
    Copy the full SHA
    5ddb73c View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2025

  1. Backport new prefix_tokens bucketting algorithm

    @sjmonson
    sjmonson committed Oct 7, 2025
    Configuration menu
    Copy the full SHA
    3902a28 View commit details
    Browse the repository at this point in the history

  2. Fix incorrect field name in validator

    @sjmonson
    sjmonson committed Oct 7, 2025
    Configuration menu
    Copy the full SHA
    98a0e37 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2025

  1. Map history only from previous request

    @sjmonson
    sjmonson committed Oct 13, 2025
    Configuration menu
    Copy the full SHA
    213d801 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2025

  1. Only track the last response in multi-turn

    @sjmonson
    sjmonson committed Oct 20, 2025
    Configuration menu
    Copy the full SHA
    79caf54 View commit details
    Browse the repository at this point in the history

  2. Clear conversation on premature exit

    @sjmonson
    sjmonson committed Oct 20, 2025
    Configuration menu
    Copy the full SHA
    23f8186 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2025

  1. Configurable max_tokens/max_completion_tokens key (#399) 

    <!--
Include a short paragraph of the changes introduced in this PR.
If this PR requires additional context or rationale, explain why
the changes are necessary.
-->

Makes the `max_tokens` request key configurable through an environment
variable per endpoint type. Defaults to `max_tokens` for legacy
`completions` and `max_completion_tokens` for `chat/completions`

<!--
Provide a detailed list of all changes introduced in this pull request.
-->
- Add the `GUIDELLM__OPENAI__MAX_OUTPUT_KEY` config option which is a
dict mapping from route name -> output tokens key. Default is
`{"text_completions": "max_tokens", "chat_completions":
"max_completion_tokens"}`

<!--
List the steps needed to test this PR.
-->
-

<!--
Link any relevant issues that this PR addresses.
-->
- Closes #395
- Closes #269
- Related #210

---

- [x] "I certify that all code in this PR is my own, except as noted
below."

- [ ] Includes AI-assisted code completion
- [ ] Includes code generated by an AI application
- [ ] Includes AI-generated tests (NOTE: AI written tests should have a
docstring that includes `## WRITTEN BY AI ##`)

---------

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Samuel Monson <smonson@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
    @sjmonson @tlrmchlsmth
    sjmonson and tlrmchlsmth committed Oct 21, 2025
    Configuration menu
    Copy the full SHA
    e6c7e55 View commit details
    Browse the repository at this point in the history
Loading