trinity
Subpackages
- trinity.buffer
- trinity.common
- Subpackages
- trinity.common.models
- trinity.common.rewards
- Submodules
- trinity.common.rewards.accuracy_reward module
- trinity.common.rewards.agents_reward module
- trinity.common.rewards.base module
- trinity.common.rewards.composite_reward module
- trinity.common.rewards.format_reward module
- trinity.common.rewards.human_reward module
- trinity.common.rewards.reward_fn module
- trinity.common.rewards.tool_reward module
- Module contents
- trinity.common.workflows
- Submodules
- trinity.common.config module
FormatConfigFormatConfig.prompt_typeFormatConfig.prompt_keyFormatConfig.response_keyFormatConfig.messages_keyFormatConfig.chat_templateFormatConfig.system_promptFormatConfig.reply_prefixFormatConfig.reward_fn_keyFormatConfig.workflow_keyFormatConfig.solution_keyFormatConfig.reward_keyFormatConfig.chosen_keyFormatConfig.rejected_keyFormatConfig.label_keyFormatConfig.__init__()
GenerationConfigStorageConfigStorageConfig.nameStorageConfig.storage_typeStorageConfig.pathStorageConfig.splitStorageConfig.subset_nameStorageConfig.formatStorageConfig.indexStorageConfig.default_workflow_typeStorageConfig.default_reward_fn_typeStorageConfig.rollout_argsStorageConfig.algorithm_typeStorageConfig.total_epochsStorageConfig.task_typeStorageConfig.__init__()
DataProcessorConfigDataProcessorConfig.data_workflow_urlDataProcessorConfig.source_data_pathDataProcessorConfig.formatDataProcessorConfig.load_kwargsDataProcessorConfig.dj_config_pathDataProcessorConfig.dj_process_descDataProcessorConfig.agent_model_nameDataProcessorConfig.agent_model_configDataProcessorConfig.clean_strategyDataProcessorConfig.min_size_ratioDataProcessorConfig.min_priority_scoreDataProcessorConfig.priority_weightsDataProcessorConfig.data_distDataProcessorConfig.db_urlDataProcessorConfig.max_retry_timesDataProcessorConfig.max_retry_intervalDataProcessorConfig.__init__()
ModelConfigInferenceModelConfigInferenceModelConfig.model_pathInferenceModelConfig.engine_typeInferenceModelConfig.engine_numInferenceModelConfig.tensor_parallel_sizeInferenceModelConfig.use_v1InferenceModelConfig.enforce_eagerInferenceModelConfig.enable_prefix_cachingInferenceModelConfig.enable_chunked_prefillInferenceModelConfig.gpu_memory_utilizationInferenceModelConfig.dtypeInferenceModelConfig.seedInferenceModelConfig.max_prompt_tokensInferenceModelConfig.max_response_tokensInferenceModelConfig.chat_templateInferenceModelConfig.enable_thinkingInferenceModelConfig.enable_openai_apiInferenceModelConfig.bundle_indicesInferenceModelConfig.__init__()
AlgorithmConfigClusterConfigExplorerInputTrainerInputBufferConfigBufferConfig.batch_sizeBufferConfig.total_epochsBufferConfig.explorer_inputBufferConfig.explorer_outputBufferConfig.trainer_inputBufferConfig.max_retry_timesBufferConfig.max_retry_intervalBufferConfig.read_batch_sizeBufferConfig.tokenizer_pathBufferConfig.pad_token_idBufferConfig.__init__()
ExplorerConfigTrainerConfigTrainerConfig.trainer_typeTrainerConfig.save_intervalTrainerConfig.enable_previewTrainerConfig.actor_use_kl_lossTrainerConfig.actor_kl_loss_coefTrainerConfig.actor_entropy_coefTrainerConfig.actor_grad_clipTrainerConfig.actor_clip_ratioTrainerConfig.trainer_configTrainerConfig.trainer_config_pathTrainerConfig.__init__()
MonitorConfigSynchronizerConfigConfigload_config()
- trinity.common.constants module
- trinity.common.experience module
ExperienceExperience.tokensExperience.prompt_lengthExperience.logprobsExperience.rewardExperience.prompt_textExperience.response_textExperience.action_maskExperience.chosenExperience.rejectedExperience.infoExperience.metricsExperience.run_idExperience.serialize()Experience.deserialize()Experience.__init__()
Experiences
- trinity.common.schema module
RftDatasetModelRftDatasetModel.idRftDatasetModel.consumed_cntRftDatasetModel.last_modified_dateRftDatasetModel.from_idRftDatasetModel.from_modelRftDatasetModel.from_recipeRftDatasetModel.promptRftDatasetModel.responseRftDatasetModel.solutionRftDatasetModel.rewardRftDatasetModel.chosenRftDatasetModel.rejectedRftDatasetModel.labelRftDatasetModel.quality_scoreRftDatasetModel.quality_score_detailRftDatasetModel.difficulty_scoreRftDatasetModel.difficulty_score_detailRftDatasetModel.diversity_scoreRftDatasetModel.diversity_score_detailRftDatasetModel.priorityRftDatasetModel.reward_fnRftDatasetModel.workflowRftDatasetModel.to_dict()RftDatasetModel.__init__()
TaskModelExperienceModelSFTDataModelDPODataModel
- trinity.common.verl_config module
DataActorModelOptimWrapPolicyFSDPConfigCheckpointActorActor.strategyActor.ppo_mini_batch_sizeActor.ppo_micro_batch_sizeActor.ppo_micro_batch_size_per_gpuActor.use_dynamic_bszActor.ppo_max_token_len_per_gpuActor.grad_clipActor.clip_ratioActor.entropy_coeffActor.use_kl_lossActor.kl_loss_coefActor.kl_loss_typeActor.ppo_epochsActor.shuffleActor.ulysses_sequence_parallel_sizeActor.checkpointActor.optimActor.fsdp_configActor.algorithm_typeActor.tauActor.opmd_baselineActor.use_uidActor.__init__()
RefRolloutActorRolloutRefCriticModelCriticCritic.strategyCritic.optimCritic.modelCritic.ppo_mini_batch_sizeCritic.ppo_micro_batch_sizeCritic.ppo_micro_batch_size_per_gpuCritic.forward_micro_batch_sizeCritic.forward_micro_batch_size_per_gpuCritic.use_dynamic_bszCritic.ppo_max_token_len_per_gpuCritic.forward_max_token_len_per_gpuCritic.ulysses_sequence_parallel_sizeCritic.ppo_epochsCritic.shuffleCritic.grad_clipCritic.cliprange_valueCritic.checkpointCritic.rollout_nCritic.__init__()
RewardModelCustomRewardFunctionKL_CtrlAlgorithmTrainerTrainer.balance_batchTrainer.total_epochsTrainer.total_training_stepsTrainer.project_nameTrainer.experiment_nameTrainer.loggerTrainer.val_generations_to_log_to_wandbTrainer.nnodesTrainer.n_gpus_per_nodeTrainer.save_freqTrainer.resume_modeTrainer.resume_from_pathTrainer.test_freqTrainer.critic_warmupTrainer.default_hdfs_dirTrainer.remove_previous_ckpt_in_saveTrainer.del_local_ckpt_after_loadTrainer.default_local_dirTrainer.val_before_trainTrainer.training_rollout_modeTrainer.enable_exp_bufferTrainer.sync_freqTrainer.sft_warmup_stepsTrainer.max_actor_ckpt_to_keepTrainer.max_critic_ckpt_to_keepTrainer.__init__()
veRLConfigload_config()
- Module contents
- Subpackages
- trinity.explorer
- trinity.manager
- Subpackages
- trinity.manager.config_registry
- Submodules
- trinity.manager.config_registry.buffer_config_manager module
- trinity.manager.config_registry.config_registry module
- trinity.manager.config_registry.explorer_config_manager module
- trinity.manager.config_registry.model_config_manager module
- trinity.manager.config_registry.trainer_config_manager module
- Module contents
- trinity.manager.config_registry
- Submodules
- trinity.manager.config_manager module
- trinity.manager.manager module
- Module contents
- Subpackages
- trinity.trainer
- Subpackages
- Submodules
- trinity.trainer.trainer module
- trinity.trainer.verl_trainer module
VerlPPOTrainerWrapperVerlPPOTrainerWrapper.__init__()VerlPPOTrainerWrapper.reset_experiences_example_table()VerlPPOTrainerWrapper.prepare()VerlPPOTrainerWrapper.train_dpo_step()VerlPPOTrainerWrapper.train_sft_step()VerlPPOTrainerWrapper.train_rft_step()VerlPPOTrainerWrapper.save_checkpoint()VerlPPOTrainerWrapper.sync_weight()VerlPPOTrainerWrapper.set_mode()VerlPPOTrainerWrapper.sft_to_rft()VerlPPOTrainerWrapper.shutdown()
- Module contents
- trinity.utils
Module contents
Trinity-RFT (Reinforcement Fine-Tuning)