trinity.algorithm.policy_loss_fn package

Submodules

• trinity.algorithm.policy_loss_fn.chord_policy_loss module
  • mu_schedule_function()
  • SFTISLossFn
    • SFTISLossFn.__init__()
    • SFTISLossFn.default_args()
    • SFTISLossFn.select_keys
  • phi_function()
  • SFTPhiLossFn
    • SFTPhiLossFn.__init__()
    • SFTPhiLossFn.default_args()
    • SFTPhiLossFn.select_keys
  • MIXCHORDPolicyLossFn
    • MIXCHORDPolicyLossFn.__init__()
    • MIXCHORDPolicyLossFn.default_args()
    • MIXCHORDPolicyLossFn.select_keys
• trinity.algorithm.policy_loss_fn.cispo_policy_loss module
  • CISPOPolicyLossFn
    • CISPOPolicyLossFn.__init__()
    • CISPOPolicyLossFn.default_args()
    • CISPOPolicyLossFn.select_keys
• trinity.algorithm.policy_loss_fn.dpo_loss module
  • DPOLossFn
    • DPOLossFn.__init__()
    • DPOLossFn.default_args()
    • DPOLossFn.select_keys
• trinity.algorithm.policy_loss_fn.gspo_policy_loss module
  • GSPOLossFn
    • GSPOLossFn.__init__()
    • GSPOLossFn.default_args()
    • GSPOLossFn.select_keys
• trinity.algorithm.policy_loss_fn.mix_policy_loss module
  • MIXPolicyLossFn
    • MIXPolicyLossFn.__init__()
    • MIXPolicyLossFn.default_args()
    • MIXPolicyLossFn.select_keys
• trinity.algorithm.policy_loss_fn.opmd_policy_loss module
  • OPMDPolicyLossFn
    • OPMDPolicyLossFn.__init__()
    • OPMDPolicyLossFn.default_args()
    • OPMDPolicyLossFn.select_keys
• trinity.algorithm.policy_loss_fn.policy_loss_fn module
  • PolicyLossFnMeta
    • PolicyLossFnMeta.ignore_keys
  • PolicyLossFn
    • PolicyLossFn.__init__()
    • PolicyLossFn.default_args()
    • PolicyLossFn.select_keys
• trinity.algorithm.policy_loss_fn.ppo_policy_loss module
  • PPOPolicyLossFn
    • PPOPolicyLossFn.__init__()
    • PPOPolicyLossFn.select_keys
    • PPOPolicyLossFn.default_args()
• trinity.algorithm.policy_loss_fn.rec_policy_loss module
  • RECPolicyLossFn
    • RECPolicyLossFn.__init__()
    • RECPolicyLossFn.default_args()
    • RECPolicyLossFn.select_keys
• trinity.algorithm.policy_loss_fn.sft_loss module
  • SFTLossFn
    • SFTLossFn.__init__()
    • SFTLossFn.default_args()
    • SFTLossFn.select_keys
• trinity.algorithm.policy_loss_fn.sppo_loss_fn module
  • sPPOPolicyLossFn
    • sPPOPolicyLossFn.__init__()
    • sPPOPolicyLossFn.default_args()
    • sPPOPolicyLossFn.select_keys
• trinity.algorithm.policy_loss_fn.topr_policy_loss module
  • TOPRPolicyLossFn
    • TOPRPolicyLossFn.__init__()
    • TOPRPolicyLossFn.default_args()
    • TOPRPolicyLossFn.select_keys

Module contents

class trinity.algorithm.policy_loss_fn.PolicyLossFn(backend: str = 'verl')

Bases: ABC

Abstract base class for policy loss functions.

This class provides the interface for implementing different policy gradient loss functions while handling parameter name mapping between different training frameworks.

__init__(backend: str = 'verl')

Initialize the policy loss function.

Parameters:

backend – The training framework/backend to use (e.g., “verl”)

abstract classmethod default_args() → Dict

Get default initialization arguments for this loss function.

Returns:

The default init arguments for the policy loss function.

Return type:

Dict

property select_keys

Returns parameter keys mapped to the specific training framework’s naming convention.