Qwen3_1_7B_SglangRecipe

from modal_training_gym.deploy_recipes.sglang_recipe.qwen3_1_7b import Qwen3_1_7B_SglangRecipe

Qwen3-1.7B SGLang recipe for 1×H100.

Attributes

recipe_type DeployRecipeType

Default: sglang

gpu Literal['H100', 'H200', 'B200', 'B300']

GPU type for server containers. Default: "H100"

tp int | None

Tensor-parallel degree and GPU count. Defaults to one.

dp int | None

Data-parallel degree passed as --dp-size. When set, the recipe also passes --enable-dp-attention.

context_length int | None

Maximum context length. The model default applies when unset.

mem_fraction_static float | None

Fraction of GPU memory reserved for model weights and the KV cache. SGLang chooses it when unset.

chunked_prefill_size int | None

Chunked-prefill token budget.

max_running_requests int | None

Maximum concurrent requests per worker.

sglang_image str

SGLang container image. Default: "lmsysorg/sglang:v0.5.12"

extra_server_args dict[str, str] | None

Additional sglang.launch_server arguments. Use an empty string for flag-only arguments.

env_vars dict[str, str]

Environment variables merged over the base Hugging Face cache environment. Default: {}

install_transformers_from_git bool

Install Transformers from GitHub in the image. Disable this if sglang_image already registers the model architecture because duplicate registrations can crash the server. Default: True

environment_name str | None

Modal deployment environment.

deploy_strategy str

Modal deployment strategy. Default: "rolling"

startup_timeout int

Maximum time in seconds for container startup and the SGLang readiness check. Default: 1200

server_args

server_args(*, served_model_name: str) -> dict[str, str]

Build SGLang launch arguments.

Returns

Launch arguments keyed by flag name.