trinity.common.models.vllm_patch.worker_patch module

Contents

trinity.common.models.vllm_patch.worker_patch module#

trinity.common.models.vllm_patch.worker_patch.patch_vllm_prompt_logprobs(model_runner: GPUModelRunner)[source]#: Patch vLLM model runner to support prompt logprobs extraction.