VllmRecipe
"# VllmRecipe\n\n```python\nfrom modal_training_gym.deploy_recipes.vllm_recipe import VllmRecipe\n```\n\nvLLM server settings.\n\n**Attributes**\n\n<div class=\"tg-param\">\n<p><strong>recipe_type</strong> <code>DeployRecipeType</code></p>\n<p><span class=\"tg-param-default\">Default: vllm</span></p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>gpu</strong> <code>Literal['H100', 'H200', 'B200', 'B300'] | None</code></p>\n<p>GPU type for server containers.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>n_gpu</strong> <code>int | None</code></p>\n<p>Number of GPUs and tensor-parallel degree.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>extra_vllm_args</strong> <code>list[str] | None</code></p>\n<p>Additional arguments for <code>vllm serve</code>.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>environment_name</strong> <code>str | None</code></p>\n<p>Modal deployment environment.</p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>deploy_strategy</strong> <code>str</code></p>\n<p>Modal deployment strategy. <span class=\"tg-param-default\">Default: \"rolling\"</span></p>\n</div>\n\n<div class=\"tg-param\">\n<p><strong>startup_timeout</strong> <code>int</code></p>\n<p>Maximum time in seconds for container startup. <span class=\"tg-param-default\">Default: 1200</span></p>\n</div>\n"
from modal_training_gym.deploy_recipes.vllm_recipe import VllmRecipevLLM server settings.
Attributes
recipe_type DeployRecipeType
Default: vllm
gpu Literal['H100', 'H200', 'B200', 'B300'] | None
GPU type for server containers.
n_gpu int | None
Number of GPUs and tensor-parallel degree.
extra_vllm_args list[str] | None
Additional arguments for vllm serve.
environment_name str | None
Modal deployment environment.
deploy_strategy str
Modal deployment strategy. Default: "rolling"
startup_timeout int
Maximum time in seconds for container startup. Default: 1200