optimize_response_mapper¶
Optimize response in question-answer pairs to be more detailed and specific.
This operator enhances the responses in question-answer pairs, making them more detailed and specific while ensuring they still address the original question. It uses a predefined system prompt for optimization. The optimized response is stripped of any leading or trailing whitespace before being returned. This mapper leverages a Hugging Face model for the optimization process, which is accelerated using CUDA.
优化问答对中的回答,使其更加详细和具体。
该算子增强问答对中的回答,使其更加详细和具体,同时确保仍然回答原始问题。它使用预定义的系统提示进行优化。优化后的回答在返回前会去除任何前导或尾随的空白字符。此映射器利用Hugging Face模型进行优化过程,并使用CUDA加速。
Type 算子类型: mapper
Tags 标签: cpu, vllm, hf, api
🔧 Parameter Configuration 参数配置¶
name 参数名 |
type 类型 |
default 默认值 |
desc 说明 |
---|---|---|---|
|
<class 'str'> |
|
API or huggingface model name. |
|
<class 'bool'> |
|
If true, use huggingface model. Otherwise, use API. |
|
typing.Optional[str] |
|
URL endpoint for the API. |
|
typing.Optional[str] |
|
Path to extract content from the API response. Defaults to 'choices.0.message.content'. |
|
typing.Optional[str] |
|
System prompt for guiding the optimization task. |
|
typing.Optional[str] |
|
Template for building the input for the model. Please make sure the template contains one placeholder '{}', which corresponds to the question and answer pair generated by param |
|
typing.Optional[str] |
|
Template for formatting the question and answer pair. Please make sure the template contains two '{}' to format question and answer. |
|
typing.Optional[str] |
|
Regular expression pattern to extract question and answer from model response. |
|
typing.Annotated[int, Gt(gt=0)] |
|
The number of retry attempts when there is an API call error or output parsing error. |
|
<class 'bool'> |
|
Whether to use VLLM for inference acceleration. |
|
typing.Optional[typing.Dict] |
|
Parameters for initializing the model. |
|
typing.Optional[typing.Dict] |
|
Sampling parameters for text generation (e.g., {'temperature': 0.9, 'top_p': 0.95}). |
|
|
Extra keyword arguments. |
📊 Effect demonstration 效果演示¶
not available 暂无