data_juicer.utils.logger_utils module

data_juicer.utils.logger_utils.get_caller_name(depth=0)

Get caller name by depth.

参数:

depth -- depth of caller context, use 0 for caller depth.

返回:

module name of the caller

class data_juicer.utils.logger_utils.StreamToLoguru(level='INFO', caller_names=('datasets', 'logging'))

基类：object

Stream object that redirects writes to a logger instance.

__init__(level='INFO', caller_names=('datasets', 'logging'))

Initialization method.

参数:

• level -- log level string of loguru. Default value: "INFO".

• caller_names -- caller names of redirected module. Default value: (apex, pycocotools).

fileno()

write(buf)

getvalue()

flush()

isatty()

data_juicer.utils.logger_utils.redirect_sys_output(log_level='INFO')

Redirect stdout/stderr to loguru with log level.

参数:

log_level -- log level string of loguru. Default value: "INFO".

data_juicer.utils.logger_utils.get_log_file_path()

Get the path to the location of the log file.

返回:

a location of log file.

data_juicer.utils.logger_utils.setup_logger(save_dir, distributed_rank=0, filename='log.txt', mode='o', level='INFO', redirect=True)

Setup logger for training and testing.

参数:

• save_dir -- location to save log file

• distributed_rank -- device rank when multi-gpu environment

• filename -- log file name to save

• mode -- log file write mode, append or override. default is o.

• level -- log severity level. It's "INFO" in default.

• redirect -- whether to redirect system output

返回:

logger instance.

data_juicer.utils.logger_utils.make_log_summarization(max_show_item=10)

class data_juicer.utils.logger_utils.HiddenPrints

基类：object

Define a range that hide the outputs within this range.