slimpajama

optimus_dl.modules.data.presets.slimpajama

Config dataclass

Bases: RegistryConfigStrict

Config(_name: str | None = None, split: str = 'train', streaming: bool = True)

Parameters:

Name	Type	Description	Default
split	str		'train'
streaming	bool		True

Source code in optimus_dl/modules/data/presets/slimpajama.py

@dataclass
class Config(RegistryConfigStrict):
    split: str = "train"
    streaming: bool = True