data_juicer.utils.ray_utils module¶

data_juicer.utils.ray_utils.is_ray_mode()[source]¶

data_juicer.utils.ray_utils.initialize_ray(cfg=None, force=False)[source]¶

data_juicer.utils.ray_utils.check_and_initialize_ray(cfg=None)[source]¶

data_juicer.utils.ray_utils.get_ray_nodes_info(cfg=None)[source]¶

data_juicer.utils.ray_utils.ray_cpu_count()[source]¶

data_juicer.utils.ray_utils.ray_gpu_count()[source]¶

data_juicer.utils.ray_utils.ray_available_memories()[source]¶: Available memory for each alive node in MB.

data_juicer.utils.ray_utils.ray_available_gpu_memories()[source]¶: Available gpu memory of each gpu card for each alive node in MB.