This section provides API reference documentation for NeMo Curator’s core classes and interfaces.
Core Classes
Pipeline
The main orchestrator for executing sequences of processing stages.
ProcessingStageBase class for all data processing stages in NeMo Curator.
CompositeStageHigh-level stages that decompose into multiple execution stages.
Task Types
DocumentBatch
Task type for text document processing.
ImageBatchTask type for image processing.
VideoTaskTask type for video processing.
AudioTaskTask type for audio processing.
Executors
XennaExecutor
Production executor using Cosmos-Xenna for distributed execution.
Experimental ExecutorsRay-based experimental executors.
Configuration
Source Code
For complete implementation details, see the NeMo Curator source code on GitHub.