data_juicer.utils.resource_utils module¶

data_juicer.utils.resource_utils.query_cuda_info(query_key)[source]¶

data_juicer.utils.resource_utils.get_cpu_count()[source]¶

data_juicer.utils.resource_utils.get_cpu_utilization()[source]¶

data_juicer.utils.resource_utils.query_mem_info(query_key)[source]¶

data_juicer.utils.resource_utils.get_ray_gpu_count()[source]¶

Get the number of available GPUs in the Ray cluster.

Returns:: Number of available GPUs, or 0 if no GPUs are available or Ray is not initialized
Return type:: int

data_juicer.utils.resource_utils.get_ray_gpu_memory()[source]¶

Get the available GPU memory in the Ray cluster.

Returns:: Dictionary mapping GPU indices to available memory in MB, or empty dict if no GPUs available
Return type:: dict