trinity.common.workflows.rubric_judge_workflow module

A workflow with LLM-as-a-judge.

class trinity.common.workflows.rubric_judge_workflow.RubricJudgeWorkflow(*, task: Task, model: ModelWrapper, auxiliary_models: List[ModelWrapper] | None = None)

基类：SimpleWorkflow

A workflow using LLM-as-a-judge and rubrics to get the reward.

Adapted from https://arxiv.org/pdf/2507.17746

__init__(*, task: Task, model: ModelWrapper, auxiliary_models: List[ModelWrapper] | None = None)

reset(task: Task)

Modified from SimpleWorkflow.reset

run() → List[Experience]

Modified from SimpleWorkflow.run

get_judge_reward(response: str, judger: OpenAI) → Tuple[bool, float]

Get rewards with LLM-as-a-judge The prompts are adapted from RAR-IMPLICIT method in https://arxiv.org/pdf/2507.17746