data_juicer.ops.mapper.query_intent_detection_mapper module

class data_juicer.ops.mapper.query_intent_detection_mapper.QueryIntentDetectionMapper(hf_model: str = 'bespin-global/klue-roberta-small-3i4k-intent-classification', zh_to_en_hf_model: str | None = 'Helsinki-NLP/opus-mt-zh-en', model_params: Dict = {}, zh_to_en_model_params: Dict = {}, *, label_key: str = 'query_intent_label', score_key: str = 'query_intent_label_score', **kwargs)

基类：Mapper

Mapper to predict user's Intent label in query. Input from query_key. Output intent label and corresponding score for the query.

__init__(hf_model: str = 'bespin-global/klue-roberta-small-3i4k-intent-classification', zh_to_en_hf_model: str | None = 'Helsinki-NLP/opus-mt-zh-en', model_params: Dict = {}, zh_to_en_model_params: Dict = {}, *, label_key: str = 'query_intent_label', score_key: str = 'query_intent_label_score', **kwargs)

Initialization method.

参数:

• hf_model -- Huggingface model ID to predict intent label.

• zh_to_en_hf_model -- Translation model from Chinese to English. If not None, translate the query from Chinese to English.

• model_params -- model param for hf_model.

• zh_to_en_model_params -- model param for zh_to_hf_model.

• label_key -- The key name in the meta field to store the output label. It is 'query_intent_label' in default.

• score_key -- The key name in the meta field to store the corresponding label score. It is 'query_intent_label_score' in default.

• kwargs -- Extra keyword arguments.

process_batched(samples, rank=None)