Documentation
Components
Data Format
Model Input

Model Input

The class used by Twinkle to represent model input is InputFeature, which is adapted to model structures such as transformers/megatron.

InputType = Union[List[List[int]], List[int], np.ndarray, Any]

class InputFeature(TypedDict, total=False):
    # Text-related fields
    input_ids: InputType
    attention_mask: InputType
    position_ids: InputType
    labels: InputType

InputFeature is essentially a Dict. Its input comes from the output of the Template component.

  • input_ids: Token list after List[Messages] is nested with a template
  • attention_mask: Attention mask
  • position_ids: Position encoding for sample distinction
  • labels: Training labels, which have already undergone a one-token left shift

In the case of packing or padding_free, fields such as input_ids are concatenated from lists of multiple samples. In multimodal scenarios, InputFeature contains other multimodal fields.

InputFeature is the standard interface for all template outputs and model inputs in Twinkle.

docs