sentence_augmentation_mapper

Augments sentences by generating enhanced versions using a Hugging Face model. This operator enhances input sentences by generating new, augmented versions. It is designed to work best with individual sentences rather than full documents. For optimal results, ensure the input text is at the sentence level. The augmentation process uses a Hugging Face model, such as lmsys/vicuna-13b-v1.5 or Qwen/Qwen2-7B-Instruct. The operator requires specifying both the primary and secondary text keys, where the augmented sentence will be stored in the secondary key. The generation process can be customized with parameters like temperature, top-p sampling, and beam search size.

使用 Hugging Face 模型生成增强版本的句子以进行数据增强。该算子通过生成新的增强版本来增强输入句子。它设计为最适合处理单个句子而不是整个文档。为了获得最佳结果，请确保输入文本处于句子级别。增强过程使用 Hugging Face 模型，如 lmsys/vicuna-13b-v1.5 或 Qwen/Qwen2-7B-Instruct。算子要求指定主次文本键，其中增强后的句子将存储在次级键中。生成过程可以通过温度、top-p 采样和束搜索大小等参数进行自定义。

Type 算子类型: mapper

Tags 标签: cpu, hf, text

🔧 Parameter Configuration 参数配置

name 参数名

type 类型

default 默认值

desc 说明

hf_model

<class ‘str’>

'Qwen/Qwen2-7B-Instruct'

Huggingface model id.

system_prompt

<class ‘str’>

None

System prompt.

task_sentence

<class ‘str’>

None

The instruction for the current task.

max_new_tokens

256

the maximum number of new tokens generated by the model.

temperature

0.2

used to control the randomness of generated text. The higher the temperature, the more random and creative the generated text will be.

top_p

None

randomly select the next word from the group of words whose cumulative probability reaches p.

num_beams

1

the larger the beam search size, the higher the quality of the generated text.

text_key

None

the key name used to store the first sentence in the text pair. (optional, defalut=’text’)

text_key_second

None

the key name used to store the second sentence in the text pair.

args

''

extra args

kwargs

''

extra args

📊 Effect demonstration 效果演示

not available 暂无