You're reading the documentation from the main branch. For the latest released version, please have a look at v1.4.0.

Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
data-juicer
data-juicer

Tutorial

  • DJ-Cookbook
  • Installation Guide
  • Quick Start

docs

  • Operator Schemas 算子提要
  • Data Recipe Gallery
  • Dataset Configuration Guide
  • “Bad” Data Exhibition
  • DJ-SORA
  • DJ_service
  • How-to Guide for Developers
  • Distributed Data Processing in Data-Juicer
  • Sandbox
  • Awesome Data-Model Co-Development of MLLMs

demos

  • Demos

tools

  • Distributed Fuzzy Deduplication Tools
  • Auto Evaluation Toolkit
  • GPT EVAL: Evaluate your model with OpenAI API
  • Evaluation Results Recorder
  • Format Conversion Tools
  • Multimodal Tools
  • Post Tuning Tools
  • Hyper-parameter Optimization for Data Recipe
  • Label Studio Service Utility
  • Metrics for video generation
  • Postprocess tools
  • Preprocess Tools
  • Data Scoring

thirdparty

  • LLM Ecosystems
  • Third-party Model Library

API Reference

  • API Reference
    • data_juicer.core package
      • data_juicer.core.data package
      • data_juicer.core.executor package
    • data_juicer.ops package
      • data_juicer.ops.aggregator package
      • data_juicer.ops.common package
      • data_juicer.ops.deduplicator package
      • data_juicer.ops.filter package
      • data_juicer.ops.grouper package
      • data_juicer.ops.mapper package
        • data_juicer.ops.mapper.annotation package
      • data_juicer.ops.selector package
    • data_juicer.ops.filter package
    • data_juicer.ops.mapper package
      • data_juicer.ops.mapper.annotation package
    • data_juicer.ops.deduplicator package
    • data_juicer.ops.selector package
    • data_juicer.ops.common package
    • data_juicer.analysis package
    • data_juicer.config package
    • data_juicer.format package
en|main
Language
English 简体中文
Version
v1.4.0 main
Back to top
Copyright © 2024, Data-Juicer Team
Made with Sphinx and @pradyunsg's Furo