trinity
Subpackages
- trinity.algorithm
- Subpackages
- trinity.algorithm.add_strategy
- trinity.algorithm.advantage_fn
- Submodules
- trinity.algorithm.advantage_fn.advantage_fn module
- trinity.algorithm.advantage_fn.grpo_advantage module
- trinity.algorithm.advantage_fn.opmd_advantage module
- trinity.algorithm.advantage_fn.ppo_advantage module
- trinity.algorithm.advantage_fn.reinforce_plus_plus_advantage module
- trinity.algorithm.advantage_fn.remax_advantage module
- trinity.algorithm.advantage_fn.rloo_advantage module
- Module contents
- trinity.algorithm.entropy_loss_fn
- trinity.algorithm.kl_fn
- trinity.algorithm.policy_loss_fn
- Submodules
- trinity.algorithm.policy_loss_fn.dpo_loss module
- trinity.algorithm.policy_loss_fn.gspo_policy_loss module
- trinity.algorithm.policy_loss_fn.mix_policy_loss module
- trinity.algorithm.policy_loss_fn.opmd_policy_loss module
- trinity.algorithm.policy_loss_fn.policy_loss_fn module
- trinity.algorithm.policy_loss_fn.ppo_policy_loss module
- trinity.algorithm.policy_loss_fn.sft_loss module
- Module contents
- trinity.algorithm.sample_strategy
- Submodules
- trinity.algorithm.algorithm module
- trinity.algorithm.algorithm_manager module
- trinity.algorithm.key_mapper module
- trinity.algorithm.utils module
- Module contents
- Subpackages
- trinity.buffer
- Subpackages
- Submodules
- trinity.buffer.buffer module
- trinity.buffer.buffer_reader module
- trinity.buffer.buffer_writer module
- trinity.buffer.queue module
- trinity.buffer.ray_wrapper module
- trinity.buffer.utils module
- Module contents
- trinity.common
- Subpackages
- trinity.common.models
- trinity.common.rewards
- Submodules
- trinity.common.rewards.accuracy_reward module
- trinity.common.rewards.agents_reward module
- trinity.common.rewards.countdown_reward module
- trinity.common.rewards.dapo_reward module
- trinity.common.rewards.format_reward module
- trinity.common.rewards.human_reward module
- trinity.common.rewards.math_reward module
- trinity.common.rewards.reward_fn module
- trinity.common.rewards.tool_reward module
- trinity.common.rewards.utils module
- Module contents
- trinity.common.workflows
- Submodules
- trinity.common.workflows.customized_math_workflows module
- trinity.common.workflows.customized_toolcall_workflows module
- trinity.common.workflows.eval_workflow module
- trinity.common.workflows.math_rm_workflow module
- trinity.common.workflows.step_wise_workflow module
- trinity.common.workflows.workflow module
- Module contents
- Submodules
- trinity.common.config module
FormatConfigFormatConfig.prompt_typeFormatConfig.prompt_keyFormatConfig.response_keyFormatConfig.messages_keyFormatConfig.chat_templateFormatConfig.system_promptFormatConfig.reply_prefixFormatConfig.reward_fn_keyFormatConfig.workflow_keyFormatConfig.solution_keyFormatConfig.reward_keyFormatConfig.chosen_keyFormatConfig.rejected_keyFormatConfig.label_keyFormatConfig.__init__()
GenerationConfigStorageConfigStorageConfig.nameStorageConfig.storage_typeStorageConfig.pathStorageConfig.repeat_timesStorageConfig.rawStorageConfig.splitStorageConfig.subset_nameStorageConfig.formatStorageConfig.indexStorageConfig.wrap_in_rayStorageConfig.capacityStorageConfig.max_read_timeoutStorageConfig.use_priority_queueStorageConfig.reuse_cooldown_timeStorageConfig.replay_buffer_kwargsStorageConfig.default_workflow_typeStorageConfig.default_eval_workflow_typeStorageConfig.default_reward_fn_typeStorageConfig.rollout_argsStorageConfig.workflow_argsStorageConfig.reward_fn_argsStorageConfig.enable_progress_barStorageConfig.ray_namespaceStorageConfig.algorithm_typeStorageConfig.total_epochsStorageConfig.total_stepsStorageConfig.task_typeStorageConfig.__init__()
RewardShapingConfigDataPipelineConfigDataPipelineConfig.input_buffersDataPipelineConfig.output_bufferDataPipelineConfig.formatDataPipelineConfig.dj_config_pathDataPipelineConfig.dj_process_descDataPipelineConfig.agent_model_nameDataPipelineConfig.clean_strategyDataPipelineConfig.min_size_ratioDataPipelineConfig.min_priority_scoreDataPipelineConfig.priority_weightsDataPipelineConfig.data_distDataPipelineConfig.reward_shapingDataPipelineConfig.__init__()
DataProcessorConfigModelConfigInferenceModelConfigInferenceModelConfig.model_pathInferenceModelConfig.engine_typeInferenceModelConfig.engine_numInferenceModelConfig.tensor_parallel_sizeInferenceModelConfig.use_v1InferenceModelConfig.enforce_eagerInferenceModelConfig.enable_prefix_cachingInferenceModelConfig.enable_chunked_prefillInferenceModelConfig.gpu_memory_utilizationInferenceModelConfig.dtypeInferenceModelConfig.seedInferenceModelConfig.max_model_lenInferenceModelConfig.max_prompt_tokensInferenceModelConfig.max_response_tokensInferenceModelConfig.chat_templateInferenceModelConfig.enable_thinkingInferenceModelConfig.enable_historyInferenceModelConfig.enable_openai_apiInferenceModelConfig.enable_auto_tool_choiceInferenceModelConfig.tool_call_parserInferenceModelConfig.reasoning_parserInferenceModelConfig.bundle_indicesInferenceModelConfig.__init__()
AlgorithmConfigAlgorithmConfig.algorithm_typeAlgorithmConfig.repeat_timesAlgorithmConfig.add_strategyAlgorithmConfig.add_strategy_argsAlgorithmConfig.sample_strategyAlgorithmConfig.sample_strategy_argsAlgorithmConfig.advantage_fnAlgorithmConfig.advantage_fn_argsAlgorithmConfig.kl_penalty_fnAlgorithmConfig.kl_penalty_fn_argsAlgorithmConfig.policy_loss_fnAlgorithmConfig.policy_loss_fn_argsAlgorithmConfig.kl_loss_fnAlgorithmConfig.kl_loss_fn_argsAlgorithmConfig.entropy_loss_fnAlgorithmConfig.entropy_loss_fn_argsAlgorithmConfig.use_token_level_lossAlgorithmConfig.__init__()
ClusterConfigExplorerInputTrainerInputBufferConfigBufferConfig.batch_sizeBufferConfig.train_batch_sizeBufferConfig.total_epochsBufferConfig.total_stepsBufferConfig.explorer_inputBufferConfig.explorer_outputBufferConfig.trainer_inputBufferConfig.max_retry_timesBufferConfig.max_retry_intervalBufferConfig.tokenizer_pathBufferConfig.pad_token_idBufferConfig.cache_dirBufferConfig.__init__()
ExplorerConfigExplorerConfig.nameExplorerConfig.runner_per_modelExplorerConfig.max_timeoutExplorerConfig.max_retry_timesExplorerConfig.env_varsExplorerConfig.max_repeat_times_per_runnerExplorerConfig.runner_numExplorerConfig.rollout_modelExplorerConfig.auxiliary_modelsExplorerConfig.eval_intervalExplorerConfig.eval_on_startupExplorerConfig.bench_on_latest_checkpointExplorerConfig.collect_experiencesExplorerConfig.__init__()
TrainerConfigMonitorConfigSynchronizerConfigConfigConfig.modeConfig.projectConfig.groupConfig.nameConfig.checkpoint_root_dirConfig.checkpoint_job_dirConfig.ray_namespaceConfig.continue_from_checkpointConfig.algorithmConfig.data_processorConfig.modelConfig.clusterConfig.bufferConfig.explorerConfig.trainerConfig.monitorConfig.synchronizerConfig.save()Config.check_and_update()Config.flatten()Config.__init__()
load_config()
- trinity.common.constants module
- trinity.common.experience module
EIDExperienceTypeCustomFieldExperienceExperience.__init__()Experience.eidExperience.rewardExperience.advantagesExperience.returnsExperience.experience_typeExperience.infoExperience.metricsExperience.prompt_lengthExperience.response_textExperience.prompt_textExperience.messagesExperience.chosen_textExperience.rejected_textExperience.tokensExperience.logprobsExperience.action_maskExperience.chosenExperience.rejectedExperience.serialize()Experience.deserialize()Experience.to_dict()Experience.gather()
split_dpo_experience_to_single_turn()ExperiencesExperiences.__init__()Experiences.eidsExperiences.tokensExperiences.rewardsExperiences.advantagesExperiences.returnsExperiences.attention_masksExperiences.action_masksExperiences.prompt_lengthExperiences.logprobsExperiences.custom_fieldsExperiences.batch_sizeExperiences.gather_experiences()
empty_experiences()gather_token_ids()gather_action_masks()gather_attention_masks()gather_logprobs()gather_advantages()gather_returns()
- trinity.common.verl_config module
DataFusedKernelOptionsActorModelOptimWrapPolicyFSDPConfigCheckpointActorActor.strategyActor.ppo_mini_batch_sizeActor.ppo_micro_batch_sizeActor.ppo_micro_batch_size_per_gpuActor.use_dynamic_bszActor.ppo_max_token_len_per_gpuActor.grad_clipActor.ppo_epochsActor.shuffleActor.ulysses_sequence_parallel_sizeActor.entropy_from_logits_with_chunkingActor.entropy_checkpointingActor.checkpointActor.optimActor.fsdp_configActor.loss_agg_modeActor.clip_ratioActor.entropy_coeffActor.use_kl_lossActor.kl_loss_coefActor.kl_loss_typeActor.__init__()
RefRolloutActorRolloutRefCriticModelCriticCritic.strategyCritic.optimCritic.modelCritic.ppo_mini_batch_sizeCritic.ppo_micro_batch_sizeCritic.ppo_micro_batch_size_per_gpuCritic.forward_micro_batch_sizeCritic.forward_micro_batch_size_per_gpuCritic.use_dynamic_bszCritic.ppo_max_token_len_per_gpuCritic.forward_max_token_len_per_gpuCritic.ulysses_sequence_parallel_sizeCritic.ppo_epochsCritic.shuffleCritic.grad_clipCritic.cliprange_valueCritic.checkpointCritic.rollout_nCritic.loss_agg_modeCritic.__init__()
RewardModelCustomRewardFunctionKL_CtrlAlgorithmTrainerTrainer.balance_batchTrainer.total_epochsTrainer.total_training_stepsTrainer.project_nameTrainer.group_nameTrainer.experiment_nameTrainer.loggerTrainer.val_generations_to_log_to_wandbTrainer.nnodesTrainer.n_gpus_per_nodeTrainer.save_freqTrainer.resume_modeTrainer.resume_from_pathTrainer.test_freqTrainer.critic_warmupTrainer.default_hdfs_dirTrainer.remove_previous_ckpt_in_saveTrainer.del_local_ckpt_after_loadTrainer.default_local_dirTrainer.val_before_trainTrainer.training_rollout_modeTrainer.enable_exp_bufferTrainer.sync_freqTrainer.sft_warmup_stepsTrainer.max_actor_ckpt_to_keepTrainer.max_critic_ckpt_to_keepTrainer.__init__()
veRLConfigload_config()
- Module contents
- Subpackages
- trinity.explorer
- Submodules
- trinity.explorer.explorer module
ExplorerExplorer.__init__()Explorer.setup_weight_sync_group()Explorer.prepare()Explorer.get_weight()Explorer.explore()Explorer.explore_step()Explorer.need_sync()Explorer.need_eval()Explorer.eval()Explorer.benchmark()Explorer.save_checkpoint()Explorer.sync_weight()Explorer.shutdown()Explorer.is_alive()
- trinity.explorer.scheduler module
- trinity.explorer.workflow_runner module
- Module contents
ExplorerExplorer.__init__()Explorer.setup_weight_sync_group()Explorer.prepare()Explorer.get_weight()Explorer.explore()Explorer.explore_step()Explorer.need_sync()Explorer.need_eval()Explorer.eval()Explorer.benchmark()Explorer.save_checkpoint()Explorer.sync_weight()Explorer.shutdown()Explorer.is_alive()
- trinity.manager
- Subpackages
- trinity.manager.config_registry
- Submodules
- trinity.manager.config_registry.algorithm_config_manager module
- trinity.manager.config_registry.buffer_config_manager module
- trinity.manager.config_registry.config_registry module
- trinity.manager.config_registry.explorer_config_manager module
- trinity.manager.config_registry.model_config_manager module
- trinity.manager.config_registry.trainer_config_manager module
- Module contents
- trinity.manager.config_registry
- Submodules
- trinity.manager.config_manager module
- trinity.manager.manager module
- trinity.manager.synchronizer module
SynchronizerSynchronizer.trainer_statusSynchronizer.explorer_status_countsSynchronizer._ready_conditionSynchronizer.model_state_dictSynchronizer.model_versionSynchronizer.checkpoint_shard_counterSynchronizer.__init__()Synchronizer.add_module()Synchronizer.set_trainer_status()Synchronizer.get_trainer_status()Synchronizer.set_explorer_status()Synchronizer.get_explorer_status_counts()Synchronizer.set_model_state_dict_with_step_num()Synchronizer.set_model_state_dict()Synchronizer.get_model_state_dict()Synchronizer.get_state_dict_meta()Synchronizer.setup_weight_sync_group()Synchronizer.wait_new_model_state_dict()Synchronizer.ready_to_nccl_sync()Synchronizer.get_actor()
- Module contents
CacheManagerSynchronizerSynchronizer.trainer_statusSynchronizer.explorer_status_countsSynchronizer._ready_conditionSynchronizer.model_state_dictSynchronizer.model_versionSynchronizer.checkpoint_shard_counterSynchronizer.__init__()Synchronizer.add_module()Synchronizer.set_trainer_status()Synchronizer.get_trainer_status()Synchronizer.set_explorer_status()Synchronizer.get_explorer_status_counts()Synchronizer.set_model_state_dict_with_step_num()Synchronizer.set_model_state_dict()Synchronizer.get_model_state_dict()Synchronizer.get_state_dict_meta()Synchronizer.setup_weight_sync_group()Synchronizer.wait_new_model_state_dict()Synchronizer.ready_to_nccl_sync()Synchronizer.get_actor()
- Subpackages
- trinity.plugins
- trinity.trainer
- Subpackages
- Submodules
- trinity.trainer.trainer module
- trinity.trainer.verl_trainer module
VerlPPOTrainerWrapperVerlPPOTrainerWrapper.__init__()VerlPPOTrainerWrapper.init_workers()VerlPPOTrainerWrapper.train_step_numVerlPPOTrainerWrapper.prepare()VerlPPOTrainerWrapper.save_state_dict()VerlPPOTrainerWrapper.upload_state_dict()VerlPPOTrainerWrapper.train_step()VerlPPOTrainerWrapper.save_checkpoint()VerlPPOTrainerWrapper.sync_weight()VerlPPOTrainerWrapper.sft_to_rft()
- Module contents
- trinity.utils
- Submodules
- trinity.utils.distributed module
- trinity.utils.dlc_utils module
- trinity.utils.eval_utils module
- trinity.utils.log module
- trinity.utils.math_eval_utils module
- trinity.utils.monitor module
- trinity.utils.plugin_loader module
- trinity.utils.registry module
- trinity.utils.timer module
- Module contents
Module contents
Trinity-RFT (Reinforcement Fine-Tuning)