Endpoint

from modal_training_gym.common.endpoint import Endpoint

Controls a Modal Endpoint that persists until stopped.

Attributes

url str

Base URL of the endpoint.

endpoint_name str

Modal Endpoint name.

model_name str

Base model ID sent in request bodies.

requires_proxy_auth bool

Whether a proxy token is required to use the endpoint.

Constructor

Endpoint(url: str, *, endpoint_name: str, model_name: str, requires_proxy_auth: bool)

chat

chat(messages: list[dict[str, Any]], timeout: int = 120, max_attempts: int = 4, **extra: Any) -> dict

Send a chat-completion request.

Parameters

messages list[dict[str, Any]]

OpenAI-compatible chat messages.

timeout int

Timeout in seconds for each request. Default: 120

max_attempts int

Maximum request attempts for transient failures. Default: 4

**extra Any

Additional Chat Completions request fields.

Returns

The assistant message dict.

Raises

• TrainingGymConfigError: Required proxy credentials are unavailable.
• RuntimeError: The endpoint rejects proxy credentials.
• httpx.HTTPError: The request fails.

launch

launch(model: ModelConfig | str, checkpoint: Checkpoint | None = None, *, endpoint_name: str | None = None, unauthenticated: bool = True, routing_region: str | None = None, environment: str | None = None, colocate_compute: bool = False, wait_timeout_sec: float = 300, recreate_if_existing: bool = False) -> Endpoint

Deploy model without waiting for readiness.

Parameters

model ModelConfig | str

Model configuration or Hugging Face model name.

checkpoint Checkpoint | None

Training checkpoint to convert and serve.

endpoint_name str | None

Endpoint name. Derived from the configuration when omitted.

unauthenticated bool

Whether the endpoint accepts requests without proxy credentials. Default: True

routing_region str | None

Endpoint traffic region.

environment str | None

Modal environment for the endpoint.

colocate_compute bool

Whether to place compute in the routing region. Default: False

wait_timeout_sec float

Maximum time to wait for an endpoint URL. Default: 300

recreate_if_existing bool

Whether to replace an endpoint with the same name. Default: False

Returns

The deployed Endpoint handle.

Raises

• TimeoutError: The endpoint does not publish a URL before wait_timeout_sec.

wait_until_ready

wait_until_ready(timeout: float = 1800) -> None

Wait until the endpoint can serve traffic.

Parameters

timeout float

Maximum number of seconds to wait. Default: 1800

Raises

• TrainingGymConfigError: Required proxy credentials are unavailable.
• RuntimeError: The endpoint rejects proxy credentials.
• TimeoutError: The endpoint is not ready before timeout.