ModalRayCluster
"# ModalRayCluster\n\n```python\nfrom modal_training_gym.common.ray_cluster import ModalRayCluster\n```\n\nStarts and coordinates a Ray cluster inside a Modal clustered function.\n\n**Attributes**\n\n<div class=\"tg-param\">\n<p><strong>n_nodes</strong> <code>int</code></p>\n<p>Total cluster node count.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>rank</strong> <code>int</code></p>\n<p>This container's rank, where zero is the head.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>head_addr</strong> <code>str</code></p>\n<p>IPv4 address of the head node.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>node_ip</strong> <code>str</code></p>\n<p>IPv4 address of this container.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>is_head</strong> <code>bool</code></p>\n<p>Whether this container is the head.</p>\n</div>\n\n## `client`\n\n```python\nclient\n```\n\nRay `JobSubmissionClient` for the head node, or `None` before startup.\n\n**Raises**\n\n- `RuntimeError`: Cluster discovery has not run or this container is not the head node.\n\n## `discover_cluster`\n\n```python\ndiscover_cluster(n_nodes: int) -> None\n```\n\nDiscover this container's Ray cluster state.\n\n**Parameters**\n\n<div class=\"tg-param\">\n<p><strong>n_nodes</strong> <code>int</code></p>\n<p>Number of containers in the cluster.</p>\n</div>\n\n## `forward_dashboard`\n\n```python\nforward_dashboard()\n```\n\nOpen a forwarding context for the Ray dashboard.\n\n**Returns**\n\nA context manager compatible with `with` and `async with`.\n\n**Raises**\n\n- `RuntimeError`: This container is not the head node.\n\n## `head_extra_start_args`\n\n```python\nhead_extra_start_args() -> list[str]\n```\n\nAdditional arguments for `ray start --head`.\n\n## `start`\n\n```python\nstart(n_nodes: int, *, init_retries: int = 30, worker_wait_retries: int = 60) -> None\n```\n\nDiscover the cluster and start Ray.\n\n## `start_ray`\n\n```python\nstart_ray(*, init_retries: int = 30, worker_wait_retries: int = 60) -> None\n```\n\nStart the Ray head or worker.\n\n**Parameters**\n\n<div class=\"tg-param\">\n<p><strong>init_retries</strong> <code>int</code></p>\n<p>Maximum Ray head initialization attempts. <span class=\"tg-param-default\">Default: 30</span></p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>worker_wait_retries</strong> <code>int</code></p>\n<p>Maximum attempts to wait for workers. <span class=\"tg-param-default\">Default: 60</span></p>\n</div>\n\n**Raises**\n\n- `RuntimeError`: `discover_cluster()` has not been called.\n\n## `submit_and_tail`\n\n```python\nsubmit_and_tail(entrypoint: str, *, runtime_env: dict | None = None, max_retries: int = 35) -> ModalRayJobResult\n```\n\nSubmit a Ray job and stream its logs.\n\n**Returns**\n\nThe final Ray job status.\n\n## `wait_forever`\n\n```python\nwait_forever(poll_seconds: float = 10) -> None\n```\n\nKeep a worker container alive until the head terminates the cluster.\n\n## `worker_extra_start_args`\n\n```python\nworker_extra_start_args() -> list[str]\n```\n\nAdditional arguments for `ray start` on worker ranks.\n"
from modal_training_gym.common.ray_cluster import ModalRayClusterStarts and coordinates a Ray cluster inside a Modal clustered function.
Attributes
n_nodes int
Total cluster node count.
rank int
This container's rank, where zero is the head.
head_addr str
IPv4 address of the head node.
node_ip str
IPv4 address of this container.
is_head bool
Whether this container is the head.
client
Section titled “client”clientRay JobSubmissionClient for the head node, or None before startup.
Raises
RuntimeError: Cluster discovery has not run or this container is not the head node.
discover_cluster
Section titled “discover_cluster”discover_cluster(n_nodes: int) -> NoneDiscover this container’s Ray cluster state.
Parameters
n_nodes int
Number of containers in the cluster.
forward_dashboard
Section titled “forward_dashboard”forward_dashboard()Open a forwarding context for the Ray dashboard.
Returns
A context manager compatible with with and async with.
Raises
RuntimeError: This container is not the head node.
head_extra_start_args
Section titled “head_extra_start_args”head_extra_start_args() -> list[str]Additional arguments for ray start --head.
start(n_nodes: int, *, init_retries: int = 30, worker_wait_retries: int = 60) -> NoneDiscover the cluster and start Ray.
start_ray
Section titled “start_ray”start_ray(*, init_retries: int = 30, worker_wait_retries: int = 60) -> NoneStart the Ray head or worker.
Parameters
init_retries int
Maximum Ray head initialization attempts. Default: 30
worker_wait_retries int
Maximum attempts to wait for workers. Default: 60
Raises
RuntimeError:discover_cluster()has not been called.
submit_and_tail
Section titled “submit_and_tail”submit_and_tail(entrypoint: str, *, runtime_env: dict | None = None, max_retries: int = 35) -> ModalRayJobResultSubmit a Ray job and stream its logs.
Returns
The final Ray job status.
wait_forever
Section titled “wait_forever”wait_forever(poll_seconds: float = 10) -> NoneKeep a worker container alive until the head terminates the cluster.
worker_extra_start_args
Section titled “worker_extra_start_args”worker_extra_start_args() -> list[str]Additional arguments for ray start on worker ranks.