Qwen3_4B_Recipe

from modal_training_gym.train_recipes.slime_recipe.qwen3_4b import Qwen3_4B_Recipe

Qwen3-4B GRPO recipe for 1 node with 8 H100 GPUs.

Attributes

sequence_parallel bool

Megatron sequence parallelism. Requires tensor parallelism greater than one. Default: False

rollout_max_response_len int

Max generated tokens per sample. Default: 4096

rollout_temperature float

Sampling temperature for rollout generation. Default: 1.0

save_interval int

Save a checkpoint every N rollout steps. Default: 10

gpu_type str

Modal GPU type for every node. Default: "H100"

colocate bool

Share GPUs between the trainer and rollout engines. Default: True

tensor_model_parallel_size int

Megatron tensor-parallel size for the actor. Default: 1

rollout_num_gpus_per_engine int

GPUs and tensor-parallel size per SGLang engine. Default: 1

num_rollout int

Training and rollout steps for the run. Default: 1

start_rollout_id int | None

Rollout step to start counting from. None continues from the iteration stored in load; TrainConfig(checkpoint=...) sets 0 so num_rollout counts the steps this run performs.

rollout_batch_size int

Prompts sampled per rollout step; each prompt is expanded into a group of sampled responses. Default: 16

name str

Modal app title. The launcher derives it from the recipe class when empty. Default: ""

app_tags dict

Extra tags merged into the Modal app metadata for dashboard auto-discovery. Default: {}

environment dict

Training-container environment variables such as Megatron PYTHONPATH and NCCL settings.

async_mode bool

Overlap rollout generation and training with slime's one-step off-policy train_async.py. Default: False

metrics MetricConfig | None

Metric tracker settings; expands to slime's W&B-compatible flags.

image_overlay Callable[[Image], Image] | None

Function that modifies the Modal image.

local_slime str | None

Local slime checkout mounted over the image copy without rebuilding it.

slime_git_repository str | None

Public HTTPS Git repository to overlay onto the image's slime checkout. Must be paired with slime_git_revision and is intended for reproducible fork-backed runs. The selected source must remain compatible with Training Gym's build-time Slime patches.

slime_git_revision str | None

Full 40-character commit SHA fetched from slime_git_repository. Branches and tags are rejected because they can move between runs.

memory int | tuple[int, int] | None

Modal Function memory request/limit in MiB.

cpu float | tuple[float, float] | None

Modal Function CPU request/limit in cores per container.

cloud str | None

Modal cloud provider to pin the cluster to.

region str | None

Modal region to pin the cluster to.

slime_model_script str

Slime script that defines MODEL_ARGS in place of the attached ModelConfig architecture. Default: ""

source_hf_checkpoint str | None

Source checkpoint when it differs from the model's own.

megatron_conversion_hf_checkpoint str | None

HF checkpoint used for the HF→Megatron conversion step instead of the training model's own weights.

patch_files list[str]

Local patch scripts applied to slime and Megatron sources. Default: []

image_run_commands list[str]

Extra shell commands run while building the image. Default: []

image_env dict[str, str]

Extra env vars baked into the image. Default: {}

train_function_kwargs dict[str, Any]

Additional Modal Function keyword arguments for the training function. Default: {}

substep_timing Literal['auto', 'off']

Default: "auto"

capture_trace bool

Attach sampled per-request execution traces to recorded rollouts. Default: False

trace_sample_limit int

Maximum traced samples per rollout when capture_trace is enabled. Default: 16

actor_num_nodes int

Megatron actor nodes. Default: 1

actor_num_gpus_per_node int

GPUs per actor node. Default: 8

rollout_num_gpus int | None

Total GPUs for rollout engines when disaggregated; None lets the allocation resolver size it.

use_critic bool

Train a separate critic model for PPO. GRPO does not use one. Default: False

critic_num_nodes int | None

Nodes for the critic when use_critic is set.

critic_num_gpus_per_node int | None

GPUs per critic node.

advantage_estimator str

Advantage estimator. Default: "grpo"

n_samples_per_prompt int

Responses sampled per prompt as one GRPO group. Default: 8

eps_clip float

PPO clip lower bound. Default: 0.2

eps_clip_high float

Upper PPO clip bound for asymmetric DAPO clipping. Default: 0.28

use_kl_loss bool

Add a per-token KL loss term against the reference model. Default: False

kl_loss_type str

KL formulation. Default: "low_var_kl"

kl_loss_coef float

Coefficient of the KL loss term. Default: 0.0

kl_coef float

KL penalty coefficient applied in the reward. Default: 0.0

entropy_coef float

Entropy bonus coefficient. Default: 0.0

calculate_per_token_loss bool

Average the loss over tokens instead of over samples. Default: False

ref_load str

Checkpoint read by the reference model for KL terms. Default: ""

over_sampling_batch_size int | None

Extra DAPO prompts sampled to replace filtered groups.

dynamic_sampling_filter_path str | None

Import path of the predicate that selects sample groups.

balance_data bool

Rebalance kept samples across data-parallel ranks. Default: False

rollout_shuffle bool

Shuffle the prompt dataset between epochs. Default: True

rollout_top_p float

Nucleus-sampling top-p for rollout generation. Default: 1.0

rollout_stop_token_ids list[int] | None

Extra token ids that terminate generation.

sglang_mem_fraction_static float

Fraction of GPU memory sglang reserves for weights + KV cache. Default: 0.75

use_fault_tolerance bool

Enable slime's fault tolerance so the run can recover from worker failures. Default: True

rollout_health_check_interval int

Interval in seconds between rollout engine /health_generate checks during generate/eval. Default: 30

rollout_health_check_timeout int

Timeout in seconds to wait for a rollout engine /health_generate response before killing it. Default: 30

rollout_health_check_first_wait int

Initial delay in seconds before health checks. DeepGEMM compilation may require a longer delay. Default: 300

global_batch_size int

Training samples per optim step. Default: 16

lr float

Learning rate. Default: 5e-07

lr_decay_style str

Learning-rate schedule. Default: "constant"

weight_decay float

Weight decay. Default: 0.1

adam_beta1 float

Adam beta1. Default: 0.9

adam_beta2 float

Adam beta2. Default: 0.98

optimizer str

Optimizer name. Default: "adam"

attention_dropout float

Attention dropout probability. Default: 0.0

hidden_dropout float

Hidden-layer dropout probability. Default: 0.0

attention_softmax_in_fp32 bool

Compute attention softmax in fp32. Default: True

accumulate_allreduce_grads_in_fp32 bool

Accumulate and all-reduce gradients in fp32. Default: True

use_distributed_optimizer bool

Shard optimizer state across data-parallel ranks with Megatron's distributed optimizer. Default: False

recompute_granularity str

Activation recomputation granularity: "full" or "selective". Default: "full"

recompute_method str

Recomputation method: "uniform" or "block". Default: "uniform"

recompute_num_layers int

Layers per recomputation chunk. Default: 1

use_dynamic_batch_size bool

Pack variable-length samples into micro-batches up to max_tokens_per_gpu instead of a fixed micro batch size. Default: True

max_tokens_per_gpu int

Token budget per GPU per micro-batch when dynamic batching is on. Default: 8192

qkv_format str

QKV layout for the Megatron backend, emitted as --qkv-format. Default: "thd"

eval_interval int | None

Run eval every N rollout steps; None disables eval. Default: 10

n_samples_per_eval_prompt int

Responses sampled per eval prompt. Default: 4

eval_max_response_len int

Max generated tokens per eval sample. Default: 4096

eval_top_p float

Nucleus-sampling top-p for eval generation. Default: 1.0

eval_config dict | None

Evaluation defaults and datasets written to --eval-config as YAML.

save str

Checkpoint output directory on the mounted /checkpoints volume. Default: "/checkpoints"

load str

Checkpoint directory to resume from; empty starts from the converted HF weights. Default: ""

no_save_optim bool

Omit optimizer state from checkpoints. The resulting checkpoints cannot resume the optimizer exactly. Default: False

no_load_optim bool

Default: False

megatron_to_hf_mode str

Export mode for saved Megatron checkpoints. An empty value disables export. Default: ""

freeze_params_name_list list[str] | None

Parameter-name patterns matched with re.search to select frozen weights.

update_weight_mode str

Weight synchronization mode. "full" sends all weights. "delta" sends byte-level changes from a CPU snapshot. Default: "full"

update_weight_transport str

"nccl" or "disk"; disk requires trainer and rollout engines to share a filesystem. Default: "nccl"

update_weight_encoding str

Encoding for delta payloads. Default: "indices"

update_weight_disk_dir str

Shared directory used by the disk transport. Default: ""

rm_type str | None

Built-in reward function name. Leave unset for a custom reward.

custom_rm_function Callable | None

Reward callable shipped by value to the containers and registered as slime's --custom-rm-path.

custom_generate_function Callable | None

Custom slime generation step shipped by value.

custom_reward_post_process_function Callable | None

Function applied to rewards after generation and shipped by value.

custom_rollout_log_function Callable | str | None

Function called with each rollout's data after dashboard and phase reporting.

custom_eval_rollout_log_function Callable | str | None

Function called with each evaluation rollout's data.

rollout_function Callable | str | None

Custom rollout loop passed through --rollout-function-path.

custom_megatron_before_log_prob_hook Callable | str | None

Hook run in the Megatron trainer before log-prob computation.

custom_megatron_before_train_step_hook Callable | str | None

Hook run in the Megatron trainer before each train step.

sglang_enable_dp_attention bool

Enable data-parallel attention across engine ranks. Default: False

sglang_dp_size int | None

Data-parallel size for the engines.

sglang_ep_size int | None

Expert-parallel size for MoE models.

sglang_enable_dp_lm_head bool

Data-parallel LM head paired with DP attention. Default: False

sglang_disable_custom_all_reduce bool

Fall back to NCCL all-reduce instead of sglang's custom kernel. Default: False

sglang_cuda_graph_bs list[int] | None

Batch sizes to capture CUDA graphs for.

sglang_cuda_graph_backend_prefill str | None

sglang_max_running_requests int | None

Cap on concurrent in-flight requests per engine.

sglang_tool_call_parser str | None

Tool-call output parser.

sglang_reasoning_parser str | None

Parser for reasoning/thinking output.

extra_config dict | None

Custom configuration written to YAML and passed as --custom-config-path. Keys become attributes on slime's parsed args and always override same-named recipe fields.

sglang_config dict | None

SGLang engine settings written to --sglang-config as YAML.

sglang_request_params dict | None

Additional parameters for SGLang generation requests.

apply_chat_template_kwargs dict | str

Keyword arguments for tokenizer apply_chat_template, passed as JSON. Default: ""

train_env_vars dict | str | None

Env vars for the training processes, passed as inline JSON.

multimodal_keys dict | str | None

Multimodal dataset columns passed as JSON.

cli_args

cli_args(dataset: DatasetConfig | None = None, model: ModelConfig | None = None) -> list[str]

get_base_recipe

get_base_recipe(model_config: ModelConfig) -> SlimeRecipe

Return the model preset for model_config.

Call this explicitly. TrainConfig uses the recipe it receives without applying a preset.

Returns

The model preset, or None when no preset is registered.

Raises

• TrainingGymConfigError: The recipe accepts only registered models and model_config is not registered.

gpu_allocation

gpu_allocation: GpuAllocation

total_nodes

total_nodes: int

validate_model_parallelism

validate_model_parallelism(model: ModelConfig) -> None

Validate the model’s parallelism settings.