Agentic | Twinkle

Agentic Preprocessor

Mon, 01 Jan 0001 00:00:00 +0000

The agentic preprocessor module provides a pipeline-based data quality filtering framework for multi-turn conversation datasets. It is designed for cleaning and filtering training data before RLHF / agentic fine-tuning.

QualityPreprocessor

QualityPreprocessor is a thin pipeline runner that accepts a list of filter callables and runs them in sequence. Each step receives a list of rows, returns (kept, dropped), and the pipeline logs per-step statistics.

from twinkle_agentic.preprocessor import QualityPreprocessor, HardFilter, DeadLoopFilter

pipeline = [
 HardFilter(min_user_chars=10),
 DeadLoopFilter(),
]
preprocessor = QualityPreprocessor(pipeline, dropped_log_path='dropped.jsonl')

# rows is a dict of columns (Dataset.map format)
cleaned = preprocessor(rows)

Parameters

Parameter	Type	Description
`pipeline`	`List[Callable]`	Ordered list of filter steps. Each step takes `List[Dict]` and returns `(kept, dropped)`.
`dropped_log_path`	`str`	Optional JSONL file path for logging dropped rows with step name and reason.

Built-in Filters

HardFilter

Rule-based filter that removes trivially bad rows using deterministic rules. Supports multi-language detection (EN/ZH/JA/KO).

from twinkle_agentic.preprocessor import HardFilter

f = HardFilter(
 min_user_chars=10, # Min chars for non-CJK user query
 min_user_chars_cjk=6, # Min chars for CJK user query
 min_assistant_chars_2turn=80, # Min assistant reply length (2-turn)
 min_thinking_chars=200, # Min thinking chain length to exempt
 system_deny_keywords=['hack', 'exploit'],
 max_chars_per_round=50000,
 max_total_chars=200000,
 max_rounds=50,
)

Drop reasons: trivial_single_turn, shallow_reply, all_empty_assistant, system_deny_keyword, round_too_long, total_too_long, too_many_rounds

DeadLoopFilter

Detects assistant messages exhibiting hesitation or dead-loop patterns — repetitive self-corrections, cascading corrections, and high n-gram repetition.

from twinkle_agentic.preprocessor import DeadLoopFilter

f = DeadLoopFilter(
 hesitation_density_threshold=7.0, # Markers per 1000 chars (response)
 cascade_threshold=5, # Cascade markers in window
 cascade_window=800, # Window size in chars
 repetition_threshold=0.45, # N-gram repetition ratio
 think_hesitation_density_threshold=15.0, # Laxer for <think> blocks
 think_repetition_threshold=0.65,
)

Uses separate threshold profiles for <think> reasoning blocks (laxer, free to ramble) and visible response (stricter).

DedupFilter

Global longest-wins deduplication. The signature is derived from the first real user turn (head+tail) and the first assistant reply.

from twinkle_agentic.preprocessor import DedupFilter

f = DedupFilter(prefix_chars=100, asst_chars=100)
kept, dropped = f(all_rows) # Must see entire dataset in one call

Note: DedupFilter requires the full dataset in a single call. Do not place it inside QualityPreprocessor (which processes per-batch). Run it separately before or after the pipeline.

RefuseFilter

Detects self-referential refusals in the first assistant reply (e.g., “I cannot help with that”). Multi-language pattern matching (EN/ZH/JA/KO).

from twinkle_agentic.preprocessor import RefuseFilter

f = RefuseFilter(check_window=600) # Only check first N chars

TokenSoupFilter

Detects garbled / token-soup output by checking for replacement characters, control characters, private-use Unicode, leaked special tokens, single-character repetition, and script chaos.

from twinkle_agentic.preprocessor import TokenSoupFilter

f = TokenSoupFilter(
 replacement_char_ratio=0.02,
 special_token_count=20,
 script_chaos_threshold=0.55,
)

PIIPresidioFilter

Multi-language PII detection and rewriting using Microsoft Presidio + spaCy NER + Faker. Detects and replaces personal identifiable information (names, emails, phone numbers, addresses, etc.).

from twinkle_agentic.preprocessor import PIIPresidioFilter

f = PIIPresidioFilter(languages=['en', 'zh'])

IntentClassifier

Heuristic intent classifier that tags each row with detected intents. Pluggable detector pipeline.

from twinkle_agentic.preprocessor import IntentClassifier

classifier = IntentClassifier()

Intent categories: tool_call, code, math, complex_logic, reasoning, user_dissatisfaction, other

ScoreFilter

Pluggable scorer-based filter with built-in scorers for character-level metrics, semantic similarity, and code execution.

from twinkle_agentic.preprocessor import ScoreFilter

f = ScoreFilter()

Built-in scorers: ChrMinScorer, SIFDScorer, PassNScorer, ParaphraseScorer

ModelFilter

Filters rows by model ID whitelist.

from twinkle_agentic.preprocessor import ModelFilter

f = ModelFilter(allowed_models=['qwen3.5-4b', 'qwen3.5-32b'])

MessageNormalizer

Three-pass message normalization: heartbeat stripping, tool-call rewriting, and consecutive same-role message merging.

from twinkle_agentic.preprocessor import MessageNormalizer

normalizer = MessageNormalizer()

Complete Pipeline Example

from twinkle_agentic.preprocessor import (
 QualityPreprocessor,
 HardFilter,
 DeadLoopFilter,
 RefuseFilter,
 TokenSoupFilter,
 MessageNormalizer,
 DedupFilter,
)

# Step 1: Global dedup (must run on full dataset)
dedup = DedupFilter()
rows, _ = dedup(all_rows)

# Step 2: Per-batch pipeline
pipeline = [
 HardFilter(min_user_chars=10, max_rounds=30),
 DeadLoopFilter(),
 RefuseFilter(),
 TokenSoupFilter(),
 MessageNormalizer(),
]
preprocessor = QualityPreprocessor(pipeline, dropped_log_path='dropped.jsonl')
cleaned = preprocessor(rows)

Protocol

Mon, 01 Jan 0001 00:00:00 +0000

The Protocol module provides an abstract LLM API client interface and its OpenAI-compatible implementation. It bridges Twinkle’s Trajectory / SamplingParams data types with external LLM inference services.

API Base Class

from abc import ABC, abstractmethod
from twinkle.data_format import Trajectory
from twinkle.data_format.message import Message
from twinkle.data_format.sampling import SamplingParams

class API(ABC):
 """Abstract LLM API client: Trajectory + SamplingParams -> assistant Message(s)."""

 @abstractmethod
 def __call__(
 self,
 trajectory: Trajectory,
 sampling_params: SamplingParams,
 **kwargs,
 ) -> Union[Message, List[Message]]:
 raise NotImplementedError()

The API class defines a simple contract: given a conversation trajectory and sampling parameters, return one or more assistant messages.

OpenAI

OpenAI is the built-in implementation that works with any endpoint speaking the /v1/chat/completions protocol (OpenAI, Azure OpenAI, vLLM, SGLang, Ollama, etc.).

from twinkle_agentic.protocol.openai import OpenAI

api = OpenAI(
 model='qwen3.5-32b',
 base_url='http://localhost:8000/v1',
 api_key='EMPTY',
)

Parameters

Parameter	Type	Description
`model`	`str`	Model name to pass in the API request.
`api_key`	`str`	API key. Defaults to the `OPENAI_API_KEY` environment variable.
`base_url`	`str`	Base URL of the API endpoint (e.g. `http://localhost:8000/v1`).
`client_kwargs`	`Dict`	Extra keyword arguments forwarded to the `openai.OpenAI` client constructor.

Usage

from twinkle.data_format import Trajectory
from twinkle.data_format.sampling import SamplingParams

trajectory = {
 'messages': [
 {'role': 'user', 'content': 'What is the capital of France?'},
 ]
}

sp = SamplingParams(temperature=0.7, max_tokens=512)
reply = api(trajectory, sp)
# reply is a Message dict: {'role': 'assistant', 'content': '...'}

Features

Tool calls: Automatically maps trajectory['tools'] to the API request and parses structured tool_calls from the response.
Reasoning content: Preserves reasoning_content from models that support it (e.g., o1-style reasoning).
Finish reason: Surfaces finish_reason on the returned message so multi-turn drivers can detect length-cap truncation.
Multi-sample: When sampling_params.num_samples > 1, returns a list of messages (one per choice).

Custom API Client

To integrate a non-OpenAI API, subclass API:

from twinkle_agentic.protocol.base import API

class MyCustomAPI(API):

 def __call__(self, trajectory, sampling_params, **kwargs):
 # Call your custom endpoint
 response = my_llm_client.chat(
 messages=trajectory['messages'],
 temperature=sampling_params.temperature,
 )
 return {'role': 'assistant', 'content': response.text}

Multi-Turn Rollout

Mon, 01 Jan 0001 00:00:00 +0000

The Rollout module provides multi-turn conversation rollout engines for agentic RLHF training. Two implementations are available: MultiTurnRollout for batched vLLM sampling and APIMultiTurnRollout for OpenAI-compatible API endpoints.

Rollout Base Class

from abc import ABC, abstractmethod
from twinkle.data_format import Trajectory

class Rollout(ABC):

 @abstractmethod
 def __call__(self, trajectories: List[Trajectory], **kwargs) -> List[Trajectory]:
 raise NotImplementedError()

All rollouts accept a list of trajectories and return the same number of trajectories with additional fields (messages, turns, stop_reason, truncated).

MultiTurnRollout

Batched multi-turn rollout engine that uses a vLLM sampler for generation. All active trajectories are sampled in a single batched call per turn for maximum throughput.

Per-turn Loop

Encode each trajectory into an InputFeature with a generation prompt
Batch sampler.sample(active_pifs) — all live trajectories in parallel
Check termination: stop_reason == 'length', no tool calls, or max turns reached
Dispatch tools via ToolManager, append tool responses
Compute bridge tokens (tool turns + generation prompt) with labels = -100
Repeat until all trajectories are done

from twinkle_agentic.rollout.multi_turn import MultiTurnRollout
from twinkle_agentic.tools.tool_manager import ToolManager
from twinkle.data_format.sampling import SamplingParams

rollout = MultiTurnRollout(
 sampler=vllm_sampler,
 template=template,
 tool_manager=tool_manager,
 sampling_params=SamplingParams(temperature=0.7, max_tokens=4096),
 max_turns=6,
 max_trajectory_tokens=8192,
 trace_dir='rollout_traces/',
)

# Run rollout
results = rollout(trajectories)

Parameters

Parameter	Type	Description
`sampler`	Sampler	vLLM sampler instance for batched generation.
`template`	`Template`	Chat template for encoding/decoding.
`tool_manager`	`ToolManager`	Tool dispatcher. Can also be passed per-call.
`sampling_params`	`SamplingParams`	Default sampling parameters.
`max_turns`	`int`	Maximum number of turns per trajectory (default: 6).
`max_trajectory_tokens`	`int`	Max total token length; exceeding truncates the trajectory.
`trace_dir`	`str`	Directory for per-trajectory JSON trace dumps.
`trace_callback`	`Callable`	Decides whether to store a trajectory trace.
`success_callback`	`Callable`	Decides filename prefix (`ok-` vs `fail-`).

Output Fields

Each output trajectory dict includes:

Field	Type	Description
`messages`	`List[Dict]`	Full conversation including tool turns.
`input_ids`	`List[int]`	Token IDs of the full sequence.
`labels`	`List[int]`	Training labels (`-100` for non-trainable tokens).
`turns`	`int`	Number of turns performed.
`stop_reason`	`str`	`'stop'` / `'length'`
`truncated`	`bool`	Whether the trajectory was truncated.
`logprobs`	`List`	Per-token log probabilities (if available).

Ray Remote Support

MultiTurnRollout is decorated with @remote_class(), enabling transparent deployment as a Ray actor:

# The rollout can run as a Ray remote actor
rollout_actor = MultiTurnRollout.remote(sampler=sampler, template=template, ...)
results = ray.get(rollout_actor.__call__.remote(trajectories))

APIMultiTurnRollout

Multi-turn rollout over an OpenAI-compatible chat-completions API. Each trajectory runs independently in a thread pool for network concurrency.

from twinkle_agentic.rollout.api_multi_turn import APIMultiTurnRollout
from twinkle_agentic.protocol.openai import OpenAI

api = OpenAI(model='qwen3.5-32b', base_url='http://localhost:8000/v1')

rollout = APIMultiTurnRollout(
 api=api,
 tool_manager=tool_manager,
 sampling_params=SamplingParams(temperature=0.7),
 max_turns=6,
 concurrency=8,
 trace_dir='api_traces/',
)

results = rollout(trajectories)

Parameters

Parameter	Type	Description
`api`	`OpenAI`	OpenAI-compatible API client.
`tool_manager`	`ToolManager`	Tool dispatcher (single or per-trajectory list).
`sampling_params`	`SamplingParams`	Default sampling parameters.
`max_turns`	`int`	Maximum turns per trajectory (default: 6).
`concurrency`	`int`	Thread pool size for parallel API calls (default: 8).
`extra_body`	`Dict`	Extra fields to include in API requests.
`trace_dir`	`str`	Directory for trace dumps.

Stop Reasons

Reason	Description
`stop`	Assistant responded without tool calls (natural end).
`length`	API returned `finish_reason='length'` (token limit).
`max_turns`	Reached `max_turns` limit.
`api_error`	API call or tool execution raised an exception.

Choosing Between Rollouts

Feature	MultiTurnRollout	APIMultiTurnRollout
Backend	vLLM sampler (local GPU)	OpenAI-compatible API
Training integration	Produces `input_ids` / `labels` for GRPO	Messages only (for data collection)
Batching	GPU-level batch parallelism	Network-level thread concurrency
Use case	Online RLHF training loop	Offline data generation / evaluation

Tools & ToolManager

Mon, 01 Jan 0001 00:00:00 +0000

The Tools module provides an abstract tool interface and a central tool dispatcher (ToolManager) for agentic multi-turn rollouts. Tools follow the OpenAI function-calling schema for seamless integration with LLM tool-use capabilities.

Tool Base Class

from abc import ABC, abstractmethod
from twinkle.data_format import Tool as ToolInfo

class Tool(ABC):

 @abstractmethod
 def __call__(self, tool_name: str, arguments: Dict[str, Any]) -> str:
 """Execute the tool and return a string result."""
 raise NotImplementedError

 @abstractmethod
 def tool_info(self) -> ToolInfo:
 """Return OpenAI-compatible tool schema."""
 raise NotImplementedError

Implementing a Custom Tool

from twinkle_agentic.tools.base import Tool

class SearchTool(Tool):

 def __call__(self, tool_name: str, arguments: dict) -> str:
 query = arguments.get('query', '')
 # Perform search logic
 return f'Search results for: {query}'

 def tool_info(self):
 return {
 'type': 'function',
 'function': {
 'name': 'search',
 'description': 'Search the web for information.',
 'parameters': {
 'type': 'object',
 'properties': {
 'query': {
 'type': 'string',
 'description': 'The search query.',
 },
 },
 'required': ['query'],
 },
 },
 }

ToolManager

ToolManager is a registry and dispatcher for tools. It resolves tool calls from the LLM’s structured output and routes them to the correct tool implementation.

from twinkle_agentic.tools.tool_manager import ToolManager

# Initialize with a list of Tool instances
manager = ToolManager([search_tool, calculator_tool])

# Or with a dict
manager = ToolManager({'search': search_tool, 'calc': calculator_tool})

# Or register dynamically
manager = ToolManager()
manager.register(search_tool)
manager.register(calculator_tool)

Key Methods

Method	Description
`register(tool)`	Register a tool (name extracted from `tool_info()`).
`unregister(name)`	Remove a tool by name.
`names()`	List all registered tool names.
`copy()`	Create a shallow copy of the manager.
`tool_infos()`	Return a list of all tool schemas (for API requests).
`__call__(tool_call)`	Dispatch a tool call and return the result string.

Dispatching Tool Calls

ToolManager accepts OpenAI-shaped tool call dicts:

tool_call = {
 'id': 'call_1',
 'type': 'function',
 'function': {
 'name': 'search',
 'arguments': '{"query": "Python tutorials"}',
 },
}

result = manager(tool_call)
# result: 'Search results for: Python tutorials'

Error handling: If the tool name is unknown, arguments are invalid JSON, or the tool raises an exception, ToolManager returns a descriptive error string instead of raising — this keeps the rollout loop running.

Integration with Rollout

from twinkle_agentic.rollout.multi_turn import MultiTurnRollout

rollout = MultiTurnRollout(
 sampler=sampler,
 template=template,
 tool_manager=manager, # Pass manager to rollout
 max_turns=6,
)

The rollout engine calls manager(tool_call) for each tool call generated by the model, and appends the result as a {'role': 'tool', 'content': result} message.

Environments (Envs)

Mon, 01 Jan 0001 00:00:00 +0000

The Envs module provides an RL execution environment abstraction for agentic training. Environments can participate in multi-turn rollouts interactively or evaluate completed trajectories in batch.

Env Base Class

from twinkle_agentic.envs.base import Env, StepResult

class Env(ABC):

 def reset(self, trajectory=None) -> StepResult:
 """Reset for a new episode."""

 @abstractmethod
 def step(self, tool_name: str, arguments: dict) -> StepResult:
 """Execute a single action, return observation + reward + done."""

 def tools(self) -> List[ToolInfo]:
 """Return tool definitions available in this environment."""

 def evaluate(self, trajectories, **kwargs) -> List[float]:
 """Batch-evaluate completed trajectories, return rewards."""

 def close(self) -> None:
 """Release resources."""

StepResult

@dataclass
class StepResult:
 observation: str = '' # Environment observation after the action
 reward: float = 0.0 # Scalar reward for this step
 done: bool = False # Whether the episode is terminated
 info: Dict[str, Any] = field(default_factory=dict) # Extra metadata

Two Usage Modes

Interactive mode (multi-turn rollout) — step-by-step execution:

env = MyEnv()
env.reset(trajectory)
result = env.step('search', {'query': 'Python'})
# ... repeat until result.done

Batch evaluation mode — evaluate completed trajectories:

rewards = env.evaluate(completed_trajectories)

EnvTool

EnvTool wraps an Env as a Tool, bridging the environment with ToolManager and MultiTurnRollout.

from twinkle_agentic.envs.env_tool import EnvTool
from twinkle_agentic.tools.tool_manager import ToolManager

env = MyEnv()

# Create one EnvTool per tool defined in the environment
env_tools = EnvTool.from_env(env)

# Register into ToolManager
manager = ToolManager(env_tools)

Key Features

Feature	Description
`from_env(env)`	Factory: creates one `EnvTool` per tool in `env.tools()`.
`last_result`	Stores the most recent `StepResult` for inspection.
`done`	Property: whether the last step terminated the episode.
`episode_reward`	Property: cumulative reward from `info['episode_reward']`.

Manual Construction

env_tool = EnvTool(
 env=my_env,
 tool_name='execute_code',
 description='Execute Python code in a sandbox.',
 parameters={
 'type': 'object',
 'properties': {
 'code': {'type': 'string', 'description': 'Python code to execute.'},
 },
 'required': ['code'],
 },
)

OpenEnv

OpenEnv adapts an WebSocket-based environment server as a synchronous Twinkle Env.

from twinkle_agentic.envs.openenv import OpenEnv

env = OpenEnv(
 base_url='http://localhost:8000',
 env_cls='coding_env.CodingEnv', # Optional typed client
 env_kwargs={'message_timeout_s': 30},
 tool_schema=[...], # Optional tool definitions
)

Parameters

Parameter	Type	Description
`base_url`	`str`	URL of the running OpenEnv server.
`env_cls`	`str` or class	Dotted import path or class for a typed client. `None` uses `GenericEnvClient`.
`env_kwargs`	`Dict`	Extra kwargs for the client constructor.
`tool_schema`	`List[ToolInfo]`	Tool definitions exposed via `tools()`.
`action_mapper`	`Callable`	Custom function to map `(tool_name, args)` to the action dict sent to the server.

Usage with Rollout

from twinkle_agentic.envs.openenv import OpenEnv
from twinkle_agentic.envs.env_tool import EnvTool
from twinkle_agentic.tools.tool_manager import ToolManager
from twinkle_agentic.rollout.api_multi_turn import APIMultiTurnRollout

# Set up environment
env = OpenEnv(base_url='http://localhost:8000', tool_schema=[...])
env.reset()

# Bridge to ToolManager
env_tools = EnvTool.from_env(env)
manager = ToolManager(env_tools)

# Use in rollout
rollout = APIMultiTurnRollout(api=api, tool_manager=manager, max_turns=10)
results = rollout(trajectories)

Implementing a Custom Environment

from twinkle_agentic.envs.base import Env, StepResult

class CodeExecutionEnv(Env):

 def reset(self, trajectory=None):
 self._sandbox = create_sandbox()
 return StepResult(observation='Sandbox ready.')

 def step(self, tool_name, arguments):
 code = arguments.get('code', '')
 output = self._sandbox.run(code)
 return StepResult(
 observation=output,
 reward=1.0 if 'error' not in output.lower() else 0.0,
 done=False,
 )

 def tools(self):
 return [{
 'type': 'function',
 'function': {
 'name': 'execute_code',
 'description': 'Run Python code.',
 'parameters': {
 'type': 'object',
 'properties': {
 'code': {'type': 'string'},
 },
 },
 },
 }]

 def close(self):
 self._sandbox.cleanup()

Multi-Turn Tool Usage Guide

Mon, 01 Jan 0001 00:00:00 +0000

This guide shows how to set up and run multi-turn agentic rollouts with tool use in Twinkle.

Architecture Overview

The agentic rollout pipeline consists of four key components:

Tool — implements a specific capability (search, code execution, etc.)
ToolManager — registers tools and dispatches LLM tool calls
Env (optional) — RL environment that exposes tools via EnvTool
Rollout — drives the multi-turn conversation loop

Quick Start: API-based Rollout

The simplest way to run a multi-turn tool-use rollout using an OpenAI-compatible API:

from twinkle_agentic.protocol.openai import OpenAI
from twinkle_agentic.tools.base import Tool
from twinkle_agentic.tools.tool_manager import ToolManager
from twinkle_agentic.rollout.api_multi_turn import APIMultiTurnRollout
from twinkle.data_format.sampling import SamplingParams

# 1. Define tools
class WeatherTool(Tool):
 def __call__(self, tool_name, arguments):
 city = arguments.get('city', 'unknown')
 return f'The weather in {city} is sunny, 25°C.'

 def tool_info(self):
 return {
 'type': 'function',
 'function': {
 'name': 'get_weather',
 'description': 'Get the current weather for a city.',
 'parameters': {
 'type': 'object',
 'properties': {
 'city': {'type': 'string', 'description': 'City name.'},
 },
 'required': ['city'],
 },
 },
 }

# 2. Set up ToolManager
manager = ToolManager([WeatherTool()])

# 3. Create API client
api = OpenAI(model='qwen3.5-32b', base_url='http://localhost:8000/v1')

# 4. Create rollout
rollout = APIMultiTurnRollout(
 api=api,
 tool_manager=manager,
 sampling_params=SamplingParams(temperature=0.7, max_tokens=2048),
 max_turns=6,
 concurrency=8,
)

# 5. Prepare trajectories
trajectories = [
 {
 'messages': [
 {'role': 'user', 'content': "What's the weather like in Beijing?"},
 ],
 },
]

# 6. Run rollout
results = rollout(trajectories)
for r in results:
 print(f"Turns: {r['turns']}, Stop: {r['stop_reason']}")
 for msg in r['messages']:
 print(f" [{msg['role']}] {msg.get('content', '')[:100]}")

Training Integration: vLLM-based Rollout

For RLHF training, use MultiTurnRollout which produces input_ids and labels:

from twinkle_agentic.rollout.multi_turn import MultiTurnRollout
from twinkle.data_format.sampling import SamplingParams

rollout = MultiTurnRollout(
 sampler=vllm_sampler, # vLLMSampler instance
 template=template, # Chat template
 tool_manager=manager,
 sampling_params=SamplingParams(temperature=0.7, max_tokens=4096),
 max_turns=6,
 max_trajectory_tokens=8192,
 trace_dir='rollout_traces/',
)

# In GRPO training loop
results = rollout(batch_trajectories)
# results contain input_ids, labels, logprobs for training

Using Environments as Tools

Bridge an RL environment into the tool pipeline:

from twinkle_agentic.envs.base import Env, StepResult
from twinkle_agentic.envs.env_tool import EnvTool
from twinkle_agentic.tools.tool_manager import ToolManager

# Define environment
class CodeEnv(Env):
 def step(self, tool_name, arguments):
 code = arguments.get('code', '')
 # Execute code in sandbox
 result = execute_in_sandbox(code)
 return StepResult(observation=result, reward=1.0, done=False)

 def tools(self):
 return [{
 'type': 'function',
 'function': {
 'name': 'run_python',
 'description': 'Execute Python code.',
 'parameters': {
 'type': 'object',
 'properties': {
 'code': {'type': 'string'},
 },
 'required': ['code'],
 },
 },
 }]

# Bridge Env -> Tool -> ToolManager
env = CodeEnv()
env_tools = EnvTool.from_env(env)
manager = ToolManager(env_tools)

# Use manager in rollout as usual
rollout = APIMultiTurnRollout(api=api, tool_manager=manager, max_turns=10)

Using OpenEnv Environments

Connect to a remote OpenEnv WebSocket server:

from twinkle_agentic.envs.openenv import OpenEnv
from twinkle_agentic.envs.env_tool import EnvTool

env = OpenEnv(
 base_url='http://localhost:8000',
 env_cls='coding_env.CodingEnv',
 tool_schema=[{
 'type': 'function',
 'function': {
 'name': 'submit',
 'description': 'Submit code solution.',
 'parameters': {
 'type': 'object',
 'properties': {
 'code': {'type': 'string'},
 },
 },
 },
 }],
)

env.reset()
env_tools = EnvTool.from_env(env)
manager = ToolManager(env_tools)

Per-Trajectory Tool Managers

For scenarios where each trajectory needs its own tool set (e.g., trajectory-bound state):

# Create per-trajectory managers
managers = []
for traj in trajectories:
 env = create_env_for(traj)
 env_tools = EnvTool.from_env(env)
 managers.append(ToolManager(env_tools))

# Pass as a list (aligned 1:1 with trajectories)
results = rollout(trajectories, tool_manager=managers)

Trace Debugging

Both rollout implementations support trace dumps for debugging:

rollout = APIMultiTurnRollout(
 api=api,
 tool_manager=manager,
 trace_dir='traces/',
 trace_callback=lambda t: t['turns'] > 1, # Only store multi-turn
 success_callback=lambda t: t.get('stop_reason') == 'stop',
)

Trace files are saved as {step}-{ok|fail}-{id}.json with the full conversation and metadata.