trinity package

Subpackages

• trinity.algorithm package
  • Subpackages
    • trinity.algorithm.advantage_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.entropy_loss_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.kl_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.policy_loss_fn package
      • Submodules
      • Module contents
    • trinity.algorithm.sample_strategy package
      • Submodules
      • Module contents
  • Submodules
    • trinity.algorithm.algorithm module
      • ConstantMeta
      • AlgorithmType
      • SFTAlgorithm
      • PPOAlgorithm
      • GRPOAlgorithm
      • OPMDAlgorithm
      • AsymREAlgorithm
      • DPOAlgorithm
      • TOPRAlgorithm
      • CISPOAlgorithm
      • MIXAlgorithm
      • MIXCHORDAlgorithm
      • RAFTAlgorithm
      • sPPOAlgorithm
    • trinity.algorithm.algorithm_manager module
      • AlgorithmManager
    • trinity.algorithm.key_mapper module
      • KeyMapper
    • trinity.algorithm.utils module
      • masked_sum()
      • masked_mean()
      • masked_var()
      • masked_whiten()
      • prefix_metrics()
  • Module contents
    • AlgorithmType
      • AlgorithmType.use_critic
      • AlgorithmType.use_reference
      • AlgorithmType.compute_advantage_in_trainer
      • AlgorithmType.can_balance_batch
      • AlgorithmType.schema
      • AlgorithmType.default_config()
      • AlgorithmType.name()
      • AlgorithmType.check_config()
    • AdvantageFn
      • AdvantageFn.default_args()
      • AdvantageFn.compute_in_trainer()
    • PolicyLossFn
      • PolicyLossFn.__init__()
      • PolicyLossFn.default_args()
      • PolicyLossFn.select_keys
    • KLFn
      • KLFn.__init__()
      • KLFn.update_kl_coef()
      • KLFn.apply_kl_penalty_to_reward()
      • KLFn.calculate_kl_loss()
      • KLFn.calculate_kl()
      • KLFn.default_args()
    • EntropyLossFn
      • EntropyLossFn.default_args()
    • SampleStrategy
      • SampleStrategy.__init__()
      • SampleStrategy.sample()
      • SampleStrategy.default_args()
• trinity.buffer package
  • Subpackages
    • trinity.buffer.operators package
      • Subpackages
      • Submodules
      • Module contents
    • trinity.buffer.pipelines package
      • Submodules
      • Module contents
    • trinity.buffer.reader package
      • Submodules
      • Module contents
    • trinity.buffer.schema package
      • Submodules
      • Module contents
    • trinity.buffer.storage package
      • Submodules
      • Module contents
    • trinity.buffer.writer package
      • Submodules
      • Module contents
  • Submodules
    • trinity.buffer.buffer module
      • get_buffer_reader()
      • get_buffer_writer()
    • trinity.buffer.buffer_reader module
      • BufferReader
    • trinity.buffer.buffer_writer module
      • BufferWriter
    • trinity.buffer.utils module
      • retry_session()
      • default_storage_path()
  • Module contents
    • BufferReader
      • BufferReader.read()
      • BufferReader.read_async()
    • BufferWriter
      • BufferWriter.write()
      • BufferWriter.write_async()
      • BufferWriter.acquire()
      • BufferWriter.release()
    • get_buffer_reader()
    • get_buffer_writer()
• trinity.common package
  • Subpackages
    • trinity.common.models package
      • Submodules
      • Module contents
    • trinity.common.rewards package
      • Submodules
      • Module contents
    • trinity.common.workflows package
      • Submodules
      • Module contents
  • Submodules
    • trinity.common.config module
      • FormatConfig
      • GenerationConfig
      • StorageConfig
      • OperatorConfig
      • ExperiencePipelineConfig
      • TaskPipelineConfig
      • DataProcessorConfig
      • ModelConfig
      • InferenceModelConfig
      • AlgorithmConfig
      • ClusterConfig
      • ExplorerInput
      • TrainerInput
      • BufferConfig
      • ExplorerConfig
      • TrainerConfig
      • MonitorConfig
      • SynchronizerConfig
      • DataJuicerServiceConfig
      • ServiceConfig
      • LogConfig
      • Config
      • load_config()
    • trinity.common.constants module
      • CaseInsensitiveEnumMeta
      • CaseInsensitiveEnum
      • PromptType
      • StorageType
      • SyncMethodEnumMeta
      • SyncMethod
      • RunningStatus
      • OpType
      • SyncStyle
    • trinity.common.experience module
      • EID
      • CustomField
      • Experience
      • split_dpo_experience_to_single_turn()
      • Experiences
      • empty_experiences()
      • gather_token_ids()
      • gather_action_masks()
      • gather_attention_masks()
      • gather_logprobs()
      • gather_advantages()
      • gather_returns()
      • gather_multi_modal_inputs()
      • group_by()
      • to_hf_datasets()
      • from_hf_datasets()
    • trinity.common.verl_config module
      • Data
      • FusedKernelOptions
      • ActorModel
      • Optim
      • WrapPolicy
      • FSDPConfig
      • Checkpoint
      • OverrideTransformerConfig
      • MegatronConfig
      • ProfileConfig
      • Actor
      • Ref
      • Rollout
      • ActorRolloutRef
      • CriticModel
      • Critic
      • RewardModel
      • CustomRewardFunction
      • KL_Ctrl
      • Algorithm
      • Trainer
      • veRLConfig
      • load_config()
  • Module contents
• trinity.explorer package
  • Submodules
    • trinity.explorer.explorer module
      • Explorer
    • trinity.explorer.scheduler module
      • TaskWrapper
      • RunnerWrapper
      • sort_batch_id()
      • Scheduler
    • trinity.explorer.workflow_runner module
      • Status
      • WorkflowRunner
  • Module contents
    • Explorer
      • Explorer.__init__()
      • Explorer.setup_weight_sync_group()
      • Explorer.prepare()
      • Explorer.get_weight()
      • Explorer.explore()
      • Explorer.explore_step()
      • Explorer.need_sync()
      • Explorer.need_eval()
      • Explorer.eval()
      • Explorer.benchmark()
      • Explorer.save_checkpoint()
      • Explorer.sync_weight()
      • Explorer.shutdown()
      • Explorer.is_alive()
      • Explorer.get_actor()
• trinity.manager package
  • Subpackages
    • trinity.manager.config_registry package
      • Submodules
      • Module contents
  • Submodules
    • trinity.manager.config_manager module
      • ConfigManager
    • trinity.manager.state_manager module
      • StateManager
    • trinity.manager.synchronizer module
      • Synchronizer
  • Module contents
    • StateManager
      • StateManager.__init__()
      • StateManager.save_explorer()
      • StateManager.load_explorer()
      • StateManager.save_trainer()
      • StateManager.load_trainer()
    • Synchronizer
      • Synchronizer.trainer_status
      • Synchronizer.explorer_status_counts
      • Synchronizer._ready_condition
      • Synchronizer.model_state_dict
      • Synchronizer.model_version
      • Synchronizer.checkpoint_shard_counter
      • Synchronizer.__init__()
      • Synchronizer.add_module()
      • Synchronizer.set_trainer_status()
      • Synchronizer.get_trainer_status()
      • Synchronizer.set_explorer_status()
      • Synchronizer.get_explorer_status_counts()
      • Synchronizer.set_model_state_dict_with_step_num()
      • Synchronizer.set_model_state_dict()
      • Synchronizer.get_model_state_dict()
      • Synchronizer.get_state_dict_meta()
      • Synchronizer.setup_weight_sync_group()
      • Synchronizer.wait_new_model_state_dict()
      • Synchronizer.ready_to_nccl_sync()
      • Synchronizer.get_actor()
• trinity.plugins package
  • Module contents
• trinity.service package
  • Subpackages
    • trinity.service.data_juicer package
      • Subpackages
      • Submodules
      • Module contents
  • Module contents
    • DataJuicerClient
      • DataJuicerClient.__init__()
      • DataJuicerClient.initialize()
      • DataJuicerClient.process_experience()
      • DataJuicerClient.process_task()
      • DataJuicerClient.close()
• trinity.trainer package
  • Subpackages
    • trinity.trainer.verl package
      • Submodules
      • Module contents
  • Submodules
    • trinity.trainer.trainer module
      • Trainer
      • TrainEngineWrapper
      • get_trainer_wrapper()
    • trinity.trainer.verl_trainer module
      • VerlPPOTrainerWrapper
  • Module contents
    • Trainer
      • Trainer.__init__()
      • Trainer.prepare()
      • Trainer.train()
      • Trainer.train_step()
      • Trainer.need_sync()
      • Trainer.sync_weight()
      • Trainer.save_checkpoint()
      • Trainer.shutdown()
      • Trainer.train_step_num
      • Trainer.is_alive()
      • Trainer.get_actor()
    • TrainEngineWrapper
      • TrainEngineWrapper.prepare()
      • TrainEngineWrapper.train_step_num
      • TrainEngineWrapper.train_step()
      • TrainEngineWrapper.save_checkpoint()
      • TrainEngineWrapper.sync_weight()
      • TrainEngineWrapper.upload_state_dict()
      • TrainEngineWrapper.save_state_dict()
    • get_trainer_wrapper()
• trinity.utils package
  • Submodules
    • trinity.utils.annotations module
      • Experimental()
      • Deprecated()
    • trinity.utils.distributed module
      • is_ipv6_address()
      • get_available_port()
      • is_port_available()
      • init_process_group()
    • trinity.utils.dlc_utils module
      • ClusterStatus
      • get_dlc_env_vars()
      • is_running()
      • wait_for_ray_setup()
      • wait_for_ray_worker_nodes()
      • setup_ray_cluster()
      • stop_ray_cluster()
    • trinity.utils.eval_utils module
      • simple_answer_parser()
      • find_boxed_answer()
      • extract_solution()
      • validate_equation()
      • evaluate_equation()
      • validate_think_pattern()
      • compute_score()
      • is_equiv()
      • remove_boxed()
      • last_boxed_only_string()
      • remove_right_units()
    • trinity.utils.log module
      • NewLineFormatter
      • get_logger()
    • trinity.utils.math_eval_utils module
      • verify_math_answer()
      • extract_answer()
      • strip_string()
      • fix_fracs()
      • fix_a_slash_b()
      • fix_sqrt()
      • convert_word_number()
      • math_equal()
      • numeric_equal()
      • symbolic_equal()
    • trinity.utils.monitor module
      • gather_metrics()
      • Monitor
      • TensorboardMonitor
      • WandbMonitor
      • MlflowMonitor
    • trinity.utils.plugin_loader module
      • load_plugins()
      • load_plugin_from_dirs()
      • load_from_file()
    • trinity.utils.registry module
      • Registry
    • trinity.utils.timer module
      • Timer
  • Module contents

Module contents

Trinity-RFT (Reinforcement Fine-Tuning)