Auto-Research

Twinkle Auto is a terminal-based intelligent training assistant that lets you control, monitor, and debug ML training through natural language. It combines a chat-driven AI agent with an automated health monitor that can detect and fix training failures autonomously.

Architecture Overview

┌──────────────────────────────────────────────────────────┐
│ TwinkleAuto (asyncio chat loop)                          │
│                                                          │
│ Components:                                              │
│   AgentLoop  ─── LLM tool-calling loop                   │
│   TrainingMonitor ─── periodic health check & auto-fix   │
│   LocalConnection ─── file-system based communication   │
│   SkillManager ─── async plugin loading                 │
└──────────────────────────────────────────────────────────┘

Installation & Launch

Auto is part of the twinkle-client package:

pip install twinkle-client

Command-Line Usage

# Basic launch (uses default local Ollama endpoint)
twinkle-auto

# Specify LLM backend
twinkle-auto --llm-base-url http://localhost:11434/v1 --llm-model qwen3.5

# Attach to an existing training run
twinkle-auto --run-id my-grpo-run

# Use a remote API (e.g., OpenAI-compatible)
twinkle-auto --llm-base-url https://api.example.com/v1 --llm-api-key sk-xxx --llm-model gpt-4o

# Enable debug logging
twinkle-auto --verbose

Or run as a Python module:

python -m twinkle_client.auto

CLI Options

Chat Agent

The core of Auto is an LLM-powered tool-calling agent (AgentLoop) that processes natural language commands through an OpenAI-compatible API. The agent maintains conversation history with automatic pruning (last 50 messages) and supports up to 10 tool-calling rounds per interaction.

What You Can Say

Training lifecycle:

• “List my training runs”
• “Start a new GRPO training with Qwen3.5-4B on gsm8k”
• “Pause the current run”
• “Resume training”
• “Stop training”

Server management:

• “Start the server with Qwen3.5-4B and a Qwen3.5-72B sampler on 2 GPUs”
• “Shut down the server”
• “How many GPUs are available?”

Monitoring & analysis:

• “How is the training going?”
• “Show me the reward-related metrics”
• “Zoom into steps 100-200”
• “Reset the chart view”

Search:

• “Search for math datasets”
• “Find Qwen models on ModelScope”

Available Tools

The agent has access to 13 built-in tools:

Server Startup

The start_server tool automates a multi-step pipeline:

1. GPU detection — nvidia-smi hardware scan
2. GPU allocation — partition GPUs between training model and samplers
3. Config generation — auto-create server_config.yaml
4. Ray cluster startup — multi-node GPU partitioning with isolated CUDA_VISIBLE_DEVICES
5. Server launch — start Twinkle Server as background process
6. Health check — poll /api/v1/healthz until ready

Multi-model topology is supported: 1 training model + N sampler/teacher models.

Skills System

Auto supports extensible skill plugins loaded from three sources:

1. Bundled skills — shipped inside twinkle_client/skills/bundled/
2. User-local skills — ~/.cache/twinkle/auto/skills/local/
3. Community skills — fetched from ModelScope (best-effort, 10s timeout)

Skills are loaded asynchronously after startup and injected into the agent’s system prompt. The agent is usable immediately even before skills finish loading.

Training Monitor (Auto-Fix)

The TrainingMonitor is a background service that runs every 30 seconds, collecting all available signals about the current training run and feeding them to the LLM for analysis.

Collected Signals

• Process status: alive / dead / unknown
• output.log tail: last 1500 chars (prioritizes tracebacks)
• Metrics: recent entries + first-half vs second-half trend analysis
• Stall duration: seconds since last metric was produced
• Current train.py: full script source (for accurate fixes)

Decision Framework

The LLM classifies each check into one of three actions:

Auto-Fix Pipeline

When a FIX is needed:

1. LLM outputs diagnosis + complete fixed script
2. Monitor archives the old train.py as train_v{N}.py
3. Writes the fixed script as the new train.py
4. Re-launches training via resume_training
5. Resets stall tracking for the new attempt

Safety guardrails:

• Max 3 auto-fix attempts per run (prevents infinite retry loops)
• Fix attempts are tracked per run_id
• Snapshot deduplication avoids re-analyzing unchanged states

File-Based Connection

Auto communicates with training processes through the local filesystem:

~/.cache/twinkle/{run_id}/
├── meta.json       — run metadata (model_id, config, status, pid)
├── metrics.jsonl   — one JSON object per step (incremental)
├── output.log      — combined stdout+stderr from training
├── train.py        — current active training script
└── train_v{N}.py   — archived previous script versions

Training Control Model

In Server Mode, the Twinkle Server retains all model/optimizer state in GPU memory:

• Pause = kill client process (SIGKILL) — server state preserved
• Resume = re-launch client script — seamlessly continues training
• Stop = SIGTERM — triggers checkpoint saving then exits
• Shut down server = releases GPU resources, destroys model state

TrainingRuntime (Script Integration)

Training scripts use TrainingRuntime to integrate with Auto:

from twinkle_client.auto.runtime import TrainingRuntime

rt = TrainingRuntime(run_id='my-grpo-run')
rt.start(model_id='Qwen/Qwen3.5-4B', config={'lr': 1e-5})
rt.register_graceful_shutdown(model, dataloader)

for step, batch in enumerate(dataloader):
    # ... training logic ...
    rt.log_metrics(step=step, loss=loss, reward=reward, grad_norm=gn, lr=lr)
    rt.log(f'Completed step {step}, loss={loss:.4f}')

rt.finish()

Key Methods

Resume Support

TrainingRuntime automatically saves training progress to meta.json (throttled to every 5 seconds). Scripts can use get_resume_info() to resume from the last saved step:

rt = TrainingRuntime(run_id='my-run')
resume = rt.get_resume_info()
global_step = resume['last_step']

if global_step > 0:
    dataloader.skip_consumed_samples(global_step * BATCH_SIZE)
    print(f'Resuming from step {global_step}')

Graceful Shutdown

When register_graceful_shutdown() is called, a SIGTERM handler is installed that:

1. Saves model checkpoint (LoRA weights + optimizer state)
2. Saves dataloader position (consumed_train_samples)
3. Logs the checkpoint path
4. Marks training as stopped and exits

Logging

All logs are written to ./auto.log (current working directory):

• Rotated at 5MB with 3 backups
• No console output — all output goes to the log file
• Use --verbose for DEBUG level logging