Index

optimus_dl.modules.data.transforms

Modules and Sub-packages

• base: Base transform classes for data pipeline.
• basic_batcher: Configuration for basic token batching with dynamic padding.
• chunk_tokens: Configuration for chunking token sequences.
• composite: Configuration for a chain of transforms.
• flat_tokens_batcher: Configuration for token aggregation and batching.
• prefetch: Configuration for prefetching.
• shuffle: Configuration for data shuffling.
• to_device: Configuration for device transfers.
• tokenize: Configuration for text tokenization.