trinity.utils.eval_utils module

trinity.utils.eval_utils.simple_answer_parser(response: str) → str

trinity.utils.eval_utils.find_boxed_answer(raw_answer, timeout=10)

Find answers from solutions where the answers are enclosed in LaTeX’s boxed tag

Parameters:

• raw_answer (str) – raw answer from model

• timeout (int) – timeout in seconds for regex

Returns:

answer if found, otherwise None

Return type:

str

trinity.utils.eval_utils.extract_solution(solution_str)

Extract the equation from the solution string.

trinity.utils.eval_utils.validate_equation(equation_str, available_numbers)

Validate that equation only uses available numbers and each number once.

trinity.utils.eval_utils.evaluate_equation(equation_str)

Safely evaluate the arithmetic equation using eval() with precautions.

trinity.utils.eval_utils.validate_think_pattern(text)

Validate whether the <think> </think> tag is properly formatted.

trinity.utils.eval_utils.compute_score(solution_str, ground_truth) → float

trinity.utils.eval_utils.is_equiv(str1, str2, verbose=False)

trinity.utils.eval_utils.remove_boxed(s)

trinity.utils.eval_utils.last_boxed_only_string(string)

trinity.utils.eval_utils.remove_right_units(string)