trinity.common.models.vllm_patch package

Submodules

• trinity.common.models.vllm_patch.api_patch module
  • PatchedChatCompletionResponseChoice
    • PatchedChatCompletionResponseChoice.token_ids
    • PatchedChatCompletionResponseChoice.model_config
  • PatchedChatCompletionResponse
    • PatchedChatCompletionResponse.prompt_token_ids
    • PatchedChatCompletionResponse.choices
    • PatchedChatCompletionResponse.model_config
  • chat_completion_full_generator()
  • run_server_in_ray()
  • dummy_add_signal_handler()
  • patch_and_serve_http()
  • run_api_server_in_ray_actor()
• trinity.common.models.vllm_patch.worker_patch module
  • patch_vllm_prompt_logprobs()

Module contents

trinity.common.models.vllm_patch.get_vllm_version()