Multimodal SFT (VLM)
Vision-language model fine-tuning with image inputs (e.g. LaTeX OCR, Gemma4). View full source → import twinkle from twinkle import DeviceMesh from twinkle.cli import CLI from …
•
1 min read
Vision-language model fine-tuning with image inputs (e.g. LaTeX OCR, Gemma4). View full source → import twinkle from twinkle import DeviceMesh from twinkle.cli import CLI from …
Direct Preference Optimization — align models using human preference data without reward modeling. Supports sigmoid/hinge/IPO/SimPO/ORPO/CPO variants. View full source → import …
Multi-turn GRPO with interactive environments — the agent takes actions via tool calls and learns from episode rewards. View full source → import twinkle from twinkle import …
Generalized Knowledge Distillation: student generates on-policy, teacher provides top-k logprobs, student learns to match teacher’s distribution. View full source → import twinkle …
Train embedding models with InfoNCE contrastive loss. Supports both full-parameter and LoRA fine-tuning. View full source → import twinkle from twinkle import DeviceMesh from …
Group Relative Policy Optimization with vLLM sampling and custom reward functions (e.g. GSM8K math). View full source → import twinkle from twinkle import DeviceMesh, DeviceGroup, …
Expert-parallel + FSDP2 for Mixture-of-Experts models like DeepSeek V4 and Qwen3.5 MoE. View full source → import twinkle from twinkle import DeviceMesh, Platform, get_logger from …
Training on Huawei Ascend NPUs using the Megatron backend with MindSpeed integration. Twinkle automatically applies fused NPU operators (RMSNorm, RoPE, SwiGLU, SDPA) via …
Tensor-parallel training via Megatron backend — ideal for large models that don’t fit on a single GPU. View full source → from peft import LoraConfig import twinkle from twinkle …
Supervised fine-tuning with FSDP2 sharding and the Muon optimizer. Supports both full-parameter and LoRA training. View full source → from torch.optim import Muon import twinkle …
The standard way to launch local multi-GPU training with torchrun: #!/usr/bin/env bash CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \ torchrun --nproc_per_node=8 fsdp2.py \ --model-id …