View on GitHub
"```python\nfrom modal_training_gym.common.dataset import HarborDataset\n```\n\nDataset backed by a Harbor task directory structure.\n\nEach task folder contains an instruction file and optional label metadata.\nTasks are discovered by globbing the task_root directory.\n\n**Inherits from:** `DatasetConfig`\n\n## Fields\n\n| Field | Type | Default | Description |\n|-------|------|---------|-------------|\n| `dataset_id` | `str` | `\"\"` | |\n| `input_key` | `str` | `\"\"` | |\n| `label_key` | `str` | `\"\"` | |\n| `apply_chat_template` | `bool` | `True` | |\n| `always_prepare` | `bool` | `False` | |\n| `writes_eval_paths` | `bool` | `True` | |\n| `dataset_name` | `str` | `\"\"` | |\n| `path` | `str \\| None` | `None` | |\n| `task_root` | `str` | `\"\"` | |\n| `task_glob` | `str` | `\"*\"` | |\n| `task_names` | `list[str] \\| None` | `None` | |\n| `instruction_path` | `str` | `\"instruction.md\"` | |\n| `label_metadata_path` | `str \\| None` | `None` | |\n| `test_data_dir` | `str \\| None` | `None` | |\n| `output_format` | `str` | `\"parquet\"` | |\n| `prompt_template` | `str` | `\"{instruction}\"` | |\n| `system_prompt` | `str` | `\"\"` | |\n| `train_size` | `int \\| None` | `None` | |\n| `eval_size` | `int \\| None` | `None` | |\n| `train_repeats` | `int` | `1` | |\n| `eval_repeats` | `int` | `1` | |\n| `shuffle_tasks` | `bool` | `False` | |\n| `shuffle_seed` | `int` | `0` | |\n\n## Methods\n\n### `load(self, split: \"Literal['all', 'train', 'eval']\" = 'all') -> 'Any'`\n\nLoad raw examples, optionally filtered by split.\n\n### `prepare(self, path: 'str', eval_paths: 'dict[str, str] | None' = None) -> 'None'`\n\nMaterialize training data to `path` (and eval splits to `eval_paths`).\n\n### `to_pandas(self, *, formatted: 'bool' = False)`\n\n### `validate_prepared(self, path: 'str') -> 'None'`\n\nSniff what `prepare()` wrote and confirm the columns the framework will index.\n\n## Related Tutorials\n\n- [Code RL with Harbor hello-world and sandboxed verification](/tutorials/rl/001_sandboxes/)\n\n**Source:** [`modal_training_gym/common/dataset.py`](https://github.com/modal-projects/training-gym/blob/main/modal_training_gym/common/dataset.py)\n"
HarborDataset
Dataset backed by a Harbor task directory structure.
from modal_training_gym.common.dataset import HarborDatasetDataset backed by a Harbor task directory structure.
Each task folder contains an instruction file and optional label metadata. Tasks are discovered by globbing the task_root directory.
Inherits from: DatasetConfig
Fields
Section titled “Fields”| Field | Type | Default | Description |
|---|---|---|---|
dataset_id | str | "" | |
input_key | str | "" | |
label_key | str | "" | |
apply_chat_template | bool | True | |
always_prepare | bool | False | |
writes_eval_paths | bool | True | |
dataset_name | str | "" | |
path | str | None | None | |
task_root | str | "" | |
task_glob | str | "*" | |
task_names | list[str] | None | None | |
instruction_path | str | "instruction.md" | |
label_metadata_path | str | None | None | |
test_data_dir | str | None | None | |
output_format | str | "parquet" | |
prompt_template | str | "{instruction}" | |
system_prompt | str | "" | |
train_size | int | None | None | |
eval_size | int | None | None | |
train_repeats | int | 1 | |
eval_repeats | int | 1 | |
shuffle_tasks | bool | False | |
shuffle_seed | int | 0 |
Methods
Section titled “Methods”load(self, split: "Literal['all', 'train', 'eval']" = 'all') -> 'Any'
Section titled “load(self, split: "Literal['all', 'train', 'eval']" = 'all') -> 'Any'”Load raw examples, optionally filtered by split.
prepare(self, path: 'str', eval_paths: 'dict[str, str] | None' = None) -> 'None'
Section titled “prepare(self, path: 'str', eval_paths: 'dict[str, str] | None' = None) -> 'None'”Materialize training data to path (and eval splits to eval_paths).
to_pandas(self, *, formatted: 'bool' = False)
Section titled “to_pandas(self, *, formatted: 'bool' = False)”validate_prepared(self, path: 'str') -> 'None'
Section titled “validate_prepared(self, path: 'str') -> 'None'”Sniff what prepare() wrote and confirm the columns the framework will index.