trinity.trainer.verl.utils module

trinity.trainer.verl.utils module#

Utils for ccompatibility issues with verl.

trinity.trainer.verl.utils.to_data_proto(experiences: List[Experience], pad_token_id: int, logger: Logger) → DataProto[源代码]#

Convert List[Experience] to verl DataProto.

trinity.trainer.verl.utils.compute_data_metrics(batch: DataProto) → dict[源代码]#

Computes various metrics from a batch of data for PPO training. Modified from verl.trainer.ppo.metric_utils.compute_data_metrics

This function calculates metrics related to scores, rewards, advantages, returns, values, and sequence lengths from a batch of data. It provides statistical information (mean, max, min) for each metric category.

参数:

batch -- A DataProto object containing batch data with token-level scores, rewards, advantages, etc.

返回:

  • critic/score/mean, max, min: Statistics about sequence scores

  • critic/rewards/mean, max, min: Statistics about sequence rewards

  • critic/advantages/mean, max, min: Statistics about advantages

  • critic/returns/mean, max, min: Statistics about returns

  • critic/values/mean, max, min: Statistics about critic values

  • critic/vf_explained_var: Explained variance of the value function

  • response_length/mean, max, min, clip_ratio: Statistics about response lengths

  • prompt_length/mean, max, min, clip_ratio: Statistics about prompt lengths

返回类型:

A dictionary of metrics including

trinity.trainer.verl.utils.get_latest_hf_checkpoint_path(config: Config)[源代码]#

Get the latest huggingface checkpoint path

目录