optimize_response_mapper

Optimize response in question-answer pairs to be more detailed and specific.

This operator enhances the responses in question-answer pairs, making them more detailed and specific while ensuring they still address the original question. It uses a predefined system prompt for optimization. The optimized response is stripped of any leading or trailing whitespace before being returned. This mapper leverages a Hugging Face model for the optimization process, which is accelerated using CUDA.

优化问答对中的回答，使其更加详细和具体。

该算子增强问答对中的回答，使其更加详细和具体，同时确保仍然回答原始问题。它使用预定义的系统提示进行优化。优化后的回答在返回前会去除任何前导或尾随的空白字符。此映射器利用Hugging Face模型进行优化过程，并使用CUDA加速。

Type 算子类型: mapper

Tags 标签: cpu, vllm, hf, api

🔧 Parameter Configuration 参数配置

name 参数名

type 类型

default 默认值

desc 说明

api_or_hf_model

<class ‘str’>

'Qwen/Qwen2.5-7B-Instruct'

API or huggingface model name.

is_hf_model

<class ‘bool’>

True

If true, use huggingface model. Otherwise, use API.

api_endpoint

typing.Optional[str]

None

URL endpoint for the API.

response_path

typing.Optional[str]

None

Path to extract content from the API response. Defaults to ‘choices.0.message.content’.

system_prompt

typing.Optional[str]

None

System prompt for guiding the optimization task.

input_template

typing.Optional[str]

None

Template for building the input for the model. Please make sure the template contains one placeholder ‘{}’, which corresponds to the question and answer pair generated by param qa_pair_template.

qa_pair_template

typing.Optional[str]

None

Template for formatting the question and answer pair. Please make sure the template contains two ‘{}’ to format question and answer.

output_pattern

typing.Optional[str]

None

Regular expression pattern to extract question and answer from model response.

try_num

typing.Annotated[int, Gt(gt=0)]

3

The number of retry attempts when there is an API call error or output parsing error.

enable_vllm

<class ‘bool’>

False

Whether to use VLLM for inference acceleration.

model_params

typing.Optional[typing.Dict]

None

Parameters for initializing the model.

sampling_params

typing.Optional[typing.Dict]

None

Sampling parameters for text generation (e.g., {‘temperature’: 0.9, ‘top_p’: 0.95}).

kwargs

''

Extra keyword arguments.

📊 Effect demonstration 效果演示

not available 暂无