Skip to content
GitHub
View on GitHub

ModelDeployment

A deployed model endpoint.

from modal_training_gym.common.deployment import ModelDeployment

A deployed model endpoint.

Returned by DeploymentConfig.serve(). Carries the live endpoint URL, Modal app id, and convenience methods for generation and evaluation.

ModelDeployment(**data)
ParameterTypeDefaultDescription
AttributeTypeDefaultDescription
deployment_idstr
deployment_configDeploymentConfig
modal_app_idstr
modal_app_urlstr
urlstr
statusstr

chat(self, messages: 'list[dict]', ensure_ready: 'bool' = True, max_attempts: 'int' = 4, timeout: 'int' = 120, **kwargs) -> 'dict'

Section titled “chat(self, messages: 'list[dict]', ensure_ready: 'bool' = True, max_attempts: 'int' = 4, timeout: 'int' = 120, **kwargs) -> 'dict'”

Return one OpenAI-compatible chat-completion message while

generate(self, prompt: 'str | list[dict]', ensure_ready: 'bool' = True, **kwargs) -> 'str'

Section titled “generate(self, prompt: 'str | list[dict]', ensure_ready: 'bool' = True, **kwargs) -> 'str'”

wait_until_ready(self, timeout: 'int' = 600) -> 'None'

Section titled “wait_until_ready(self, timeout: 'int' = 600) -> 'None'”

Source: modal_training_gym/common/deployment.py