Twinkle Twinkle
  • Home
  • Quick Start
  • Documentation
  • Blog
  • Cookbook
  • Community
EN 中文
ESC

Searching...

No results found

↑↓ Navigate ↵ Select
Powered by Hugo Blox
  • Blog
    • Sequence Parallel & Ring Attention: Training with Ultra-Long Contexts
    • Ascend NPU Support: Fused Operators and Flash Linear Attention
    • Two Execution Modes: torchrun (Local) vs Ray (Distributed)
    • Multi-LoRA: Concurrent Multi-Tenant Training on Shared GPUs
    • TUI & Auto-Research: An AI Agent for Training Control
    • OpenEnv Integration: Connecting External Environments to RL Training
    • Free LLM Training on ModelScope: Twinkle Training-as-a-Service
  • Documentation
    • Quick Start
    • Concepts & Architecture
      • Components
      • Runtime Modes
      • Multi-Tenancy
      • Server & Client
      • Training as a Service
      • Cookbook
      • NPU Support
      • Architecture
    • API Reference
    • Usage Guide
      • Training Guide
      • Twinkle Installation
      • Server and Client
        • Overview
        • Server
        • Observability
        • Twinkle Client
        • Tinker Client
      • NPU (Ascend) Quick Start Guide
      • Twinkle Training Service on ModelScope
      • Qwen3.5 Training Best Practices
      • Embedding Training
    • Components
      • Dataset
        • Basic Dataset Components
        • Lazy Loading Dataset
        • Fixed-Length Packing Dataset
        • Streaming Dataset
        • Streaming Fixed-Length Packing Dataset
      • Data Format
        • Message
        • Trajectory
        • Model Input
        • Model Output
        • Sampling Output
        • Model Output
      • Template
        • DeepSeek-V4 Template
        • Tool Call Parsers
        • Template
      • Preprocessor and Filter
        • Preprocessor
        • Built-in Preprocessors
        • Filter
      • Data Loading
        • DataLoader
      • Task Processor
        • InputProcessor
      • Model
        • Supported Models
        • TwinkleModel
        • TransformersModel
        • MultiLoraTransformersModel
        • MegatronModel
        • MultiLoraMegatronModel
      • Sampler
        • Sampler
        • vLLMSampler
        • TorchSampler
      • Reward
        • Reward
        • GSM8K Reward
        • MultiModal Reward
        • OlympiadBench Reward
      • Advantage
        • Advantage
        • GRPOAdvantage
        • RLOOAdvantage
      • Hub
        • Hub
      • Checkpoint Engine
        • CheckpointEngine
        • NCCLCheckpointEngine
        • HCCLCheckpointEngine
      • Metrics
        • TrainMetric
        • LossMetric
        • Accuracy
        • CompletionRewardMetric
        • DPOMetric
        • GRPOMetric
        • EmbeddingMetric
        • Building Metrics
      • Loss
        • InfoNCE Loss
        • Cross Entropy
        • Chunked Cross Entropy
        • DPO Loss
        • GKD Loss
        • GRPO Loss
        • MSE Loss
        • Building New Loss
      • Loss Scale
        • Loss Scale
      • LRScheduler
        • CosineWarmupScheduler
        • LinearWarmupScheduler
      • Patch
        • Patch
      • Plugin
        • Plugin
      • Kernel
        • Twinkle Kernel Module
      • Training Middleware
        • DeviceMesh/DeviceGroup
        • Expert Parallel (EP)
        • Sequence Parallel (SP)
        • Padding-Free Training
        • RemoteClass
        • TwinkleClient
      • CLI
        • CLI
      • Notifier
        • Notifier
      • Agentic
        • Agentic Preprocessor
        • Protocol
        • Multi-Turn Rollout
        • Tools & ToolManager
        • Environments (Envs)
        • Multi-Turn Tool Usage Guide
      • Auto
        • Auto-Research
        • SkillProvider
  • Community
  • Cookbook
    • Shell Launch
    • SFT (FSDP2)
    • Megatron TP
    • NPU (Ascend)
    • EP + MoE
    • GRPO
    • Embedding
    • GKD Distill
    • Multi-Turn RL
    • DPO
    • Multimodal
  • Need help?
  • Community
  • GitHub ↗
  • Quick Start
  • Concepts & Architecture
    • Components
    • Runtime Modes
    • Multi-Tenancy
    • Server & Client
    • Training as a Service
    • Cookbook
    • NPU Support
    • Architecture
  • API Reference
  • Usage Guide
    • Training Guide
    • Twinkle Installation
    • Server and Client
      • Overview
      • Server
      • Observability
      • Twinkle Client
      • Tinker Client
    • NPU (Ascend) Quick Start Guide
    • Twinkle Training Service on ModelScope
    • Qwen3.5 Training Best Practices
    • Embedding Training
  • Components
    • Dataset
      • Basic Dataset Components
      • Lazy Loading Dataset
      • Fixed-Length Packing Dataset
      • Streaming Dataset
      • Streaming Fixed-Length Packing Dataset
    • Data Format
      • Message
      • Trajectory
      • Model Input
      • Model Output
      • Sampling Output
      • Model Output
    • Template
      • DeepSeek-V4 Template
      • Tool Call Parsers
      • Template
    • Preprocessor and Filter
      • Preprocessor
      • Built-in Preprocessors
      • Filter
    • Data Loading
      • DataLoader
    • Task Processor
      • InputProcessor
    • Model
      • Supported Models
      • TwinkleModel
      • TransformersModel
      • MultiLoraTransformersModel
      • MegatronModel
      • MultiLoraMegatronModel
    • Sampler
      • Sampler
      • vLLMSampler
      • TorchSampler
    • Reward
      • Reward
      • GSM8K Reward
      • MultiModal Reward
      • OlympiadBench Reward
    • Advantage
      • Advantage
      • GRPOAdvantage
      • RLOOAdvantage
    • Hub
      • Hub
    • Checkpoint Engine
      • CheckpointEngine
      • NCCLCheckpointEngine
      • HCCLCheckpointEngine
    • Metrics
      • TrainMetric
      • LossMetric
      • Accuracy
      • CompletionRewardMetric
      • DPOMetric
      • GRPOMetric
      • EmbeddingMetric
      • Building Metrics
    • Loss
      • InfoNCE Loss
      • Cross Entropy
      • Chunked Cross Entropy
      • DPO Loss
      • GKD Loss
      • GRPO Loss
      • MSE Loss
      • Building New Loss
    • Loss Scale
      • Loss Scale
    • LRScheduler
      • CosineWarmupScheduler
      • LinearWarmupScheduler
    • Patch
      • Patch
    • Plugin
      • Plugin
    • Kernel
      • Twinkle Kernel Module
    • Training Middleware
      • DeviceMesh/DeviceGroup
      • Expert Parallel (EP)
      • Sequence Parallel (SP)
      • Padding-Free Training
      • RemoteClass
      • TwinkleClient
    • CLI
      • CLI
    • Notifier
      • Notifier
    • Agentic
      • Agentic Preprocessor
      • Protocol
      • Multi-Turn Rollout
      • Tools & ToolManager
      • Environments (Envs)
      • Multi-Turn Tool Usage Guide
    • Auto
      • Auto-Research
      • SkillProvider
  • Need help?
  • Community
  • GitHub ↗
Edit this page
Documentation
Components
Advantage

Advantage

Advantage GRPOAdvantage RLOOAdvantage
Advantage GRPOAdvantage RLOOAdvantage
docs

© 2026 ModelScope. Licensed under Apache License 2.0.

Made with Hugo Blox. Create yours →