data_juicer.utils.resource_utils module¶

data_juicer.utils.resource_utils.query_cuda_info(query_key)[source]¶

data_juicer.utils.resource_utils.get_cpu_utilization()[source]¶

data_juicer.utils.resource_utils.query_mem_info(query_key)[source]¶

data_juicer.utils.resource_utils.cuda_device_count()[source]¶

data_juicer.utils.resource_utils.is_cuda_available()[source]¶

data_juicer.utils.resource_utils.cpu_count()[source]¶

data_juicer.utils.resource_utils.available_memories() → List[int][source]¶: Available memory for each node in MB.

data_juicer.utils.resource_utils.available_gpu_memories() → List[int][source]¶: Available gpu memory of each gpu card for each alive node in MB.