data_juicer.ops.grouper¶
- class data_juicer.ops.grouper.KeyValueGrouper(group_by_keys: List[str] | None = None, *args, **kwargs)[source]¶
Bases:
Grouper
Group samples to batched samples according values in given keys.
- class data_juicer.ops.grouper.NaiveGrouper(*args, **kwargs)[source]¶
Bases:
Grouper
Group all samples to one batched sample.
- class data_juicer.ops.grouper.NaiveReverseGrouper(batch_meta_export_path=None, *args, **kwargs)[source]¶
Bases:
Grouper
Split batched samples to samples.