DatasetConfig

from modal_training_gym.common.dataset import DatasetConfig

Dataset fields and materialization behavior shared across training frameworks.

Attributes

dataset_id str

Dataset ID. Default: ""

input_key str

Prompt column name. Default: ""

label_key str

Ground-truth column name. Default: ""

output_format str

On-disk format written by prepare(), either parquet or jsonl. Default: "parquet"

apply_chat_template bool

Apply the model's chat template. Default: True

always_prepare bool

Rerun prepare() when the output path exists. Default: False

writes_eval_paths bool

Whether prepare() must materialize every eval_paths entry. Default: True

load

load(split: Literal['all', 'train', 'eval'] = 'all') -> Any

Load raw examples, optionally filtered by split.

Returns

Raw examples for split.

name

name: str

prepare

prepare(path: str, eval_paths: dict[str, str] | None = None) -> None

Materialize training data at path and evaluation data at eval_paths.

validate_prepared

validate_prepared(path: str) -> None

Validate the prepared file format and required columns.