SlimeRecipe

from modal_training_gym.train_recipes.slime_recipe.recipe import SlimeRecipe

Recipe dataclass for configuring slime GRPO training on Modal.

Fields fall into two categories:

1. Launcher instructions — consumed by the Modal launcher (image build, cluster topology, W&B, checkpoint conversion, callable shipping) and never forwarded to slime. These are exactly the names listed in _SLIME_SKIP.
2. slime CLI flags — every other field is emitted to slime’s train.py as --<field-name-with-dashes> <value> by BaseTrainRecipe.cli_args: None/False/"" omit the flag entirely, True emits a bare flag, lists become space-separated values, dicts in YAML_CONFIG_FIELDS are materialized to temp YAML files on the container and passed as paths, and dicts in JSON_CONFIG_FIELDS are passed as inline JSON. sglang_*-prefixed fields configure the rollout engines rather than Megatron: slime registers every sglang ServerArgs option under a --sglang- prefix and forwards it.

Passing a flag that has no field here. extra="forbid" rejects unknown constructor kwargs, but you never need to edit this class to pass a new slime or sglang flag:

• Declare it as a field on a subclass. Emission is purely name-based, so any slime flag maps 1:1 (--use-tis → a use_tis: bool = True field) and any sglang server arg maps via the prefix (--sglang-moe-dense-tp-size → sglang_moe_dense_tp_size: int = 1); see glm_4_7.py for a recipe that does both. This works even for flags newer than this package — if the slime/sglang version baked into SLIME_IMAGE understands the flag, it just works.
• Or put it in extra_config. The launcher writes the dict to YAML and passes --custom-config-path; slime sets each key as an attribute on its parsed args. Keys there always win over same-named top-level fields (the field’s CLI flag is dropped so the YAML value stands), and this is the only route for keys that aren’t argparse flags at all, e.g. settings a custom generate function reads via args.<key>.

When adding a launcher-only field to this class or a subclass, also add its name to _SLIME_SKIP — otherwise it leaks onto the slime command line and argparse aborts with “unrecognized arguments”.

Inherits from: BaseTrainRecipe

App Identity

Field	Type	Default	Description
recipe_type	RecipeType	slime	Discriminator marking this recipe as slime; never override.
name	str	""	Modal app title; when empty the launcher derives one from the recipe class.
app_tags	dict	{}	Extra tags merged into the Modal app metadata for dashboard auto-discovery.

Modal Launcher

Field	Type	Default	Description
environment	dict	{'PYTHONPATH': '/root/Megatron-LM/', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_NVLS_ENABLE': '1'}	Env vars set in the training containers (defaults include PYTHONPATH for Megatron and NCCL tuning).
async_mode	bool	False	Run slime’s train_async.py so rollout generation and training overlap (one-step off-policy) instead of alternating.
wandb	WandbConfig | None	None	W&B settings; expands to slime’s --use-wandb/--wandb-project/ --wandb-group flags.
image_overlay	collections.abc.Callable[[modal.image.Image], modal.image.Image] | None	None	Callable that customizes the Modal image (e.g. lambda img: img.pip_install("pkg")).
local_slime	str | None	None	Path to a local slime checkout mounted over the image’s copy — dev overlay for testing slime changes without an image rebuild.
memory	int | tuple[int, int] | None	None	Modal Function memory request/limit in MiB.
cloud	str | None	None	Modal cloud provider to pin the cluster to.
region	str | None	None	Modal region to pin the cluster to.
slime_model_script	str	""	Script path relative to the slime repo, sourced before train.py to provide MODEL_ARGS; when set, model-architecture flags are not emitted from the attached ModelConfig.
source_hf_checkpoint	str | None	None	HF repo fetched as the source checkpoint when it differs from the model’s own (used by some recipes).
megatron_conversion_hf_checkpoint	str | None	None	HF checkpoint used for the HF→Megatron conversion step instead of the training model’s own weights.
patch_files	list[str]	[]	Local patch scripts copied into the image at build time (applied to slime/Megatron sources).
image_run_commands	list[str]	[]	Extra shell commands run while building the image.
image_env	dict[str, str]	{}	Extra env vars baked into the image.
train_function_kwargs	dict[str, Any]	{}	Extra Modal Function options for the train function; supported keys: secrets, experimental_options, ephemeral_disk.
capture_trace	bool	False	Attach slime’s per-sample execution trace (generate/reward/tool-call timeline) to recorded rollouts for the dashboard.
trace_sample_limit	int	16	With capture_trace, number of samples per rollout that get a trace attached (sampling keeps the added data volume small).

Cluster and Parallelism

Field	Type	Default	Description
gpu_type	str		Modal GPU type for every node, e.g. "H100" or "B200".
colocate	bool		Trainer and rollout engines share the same GPUs, shifting memory between phases; False gives each its own GPUs (disaggregated).
actor_num_nodes	int	1	Number of nodes for the Megatron actor (trainer).
actor_num_gpus_per_node	int	8	GPUs per actor node.
rollout_num_gpus	int | None	None	Total GPUs for rollout engines when disaggregated; None lets the allocation resolver size it.
rollout_num_gpus_per_engine	int		GPUs per sglang engine — its tensor-parallel size.
tensor_model_parallel_size	int		Megatron tensor-parallel size for the actor.
sequence_parallel	bool		Megatron sequence parallelism (requires TP > 1).
use_critic	bool	False	Train a separate critic model (PPO-style; GRPO runs without one).
critic_num_nodes	int | None	None	Nodes for the critic when use_critic is set.
critic_num_gpus_per_node	int | None	None	GPUs per critic node.

Rollout and Sampling

Field	Type	Default	Description
num_rollout	int		Total rollout steps (= training steps) for the run.
rollout_batch_size	int		Prompts sampled per rollout step; each prompt is expanded into a group of sampled responses.
rollout_max_response_len	int		Max generated tokens per sample.
rollout_temperature	float		Sampling temperature for rollout generation.
rollout_shuffle	bool	True	Shuffle the prompt dataset between epochs.
rollout_top_p	float	1.0	Nucleus-sampling top-p for rollout generation.
rollout_stop_token_ids	list[int] | None	None	Extra token ids that terminate generation.

Fault Tolerance and Health Checks

Field	Type	Default	Description
use_fault_tolerance	bool	True	Enable slime’s fault tolerance so the run can recover from worker failures.
rollout_health_check_interval	int	30	Interval in seconds between rollout engine /health_generate checks during generate/eval.
rollout_health_check_timeout	int	30	Timeout in seconds to wait for a rollout engine /health_generate response before killing it.
rollout_health_check_first_wait	int	300	Initial grace period (in seconds) before starting health checks. This allows time for model compilation and initialization. Increase this value significantly when using deepgemm.

Checkpointing

Field	Type	Default	Description
save	str	"/checkpoints"	Checkpoint output directory (the mounted /checkpoints volume).
save_interval	int		Save a checkpoint every N rollout steps.
load	str	""	Checkpoint directory to resume from; empty starts from the converted HF weights.
no_save_optim	bool	False	Omit optim state from checkpoints (smaller, but no exact resume).
megatron_to_hf_mode	str	""	Mode used to export saved Megatron checkpoints back to HF format; empty disables the export step.
freeze_params_name_list	list[str] | None	None	Regex patterns (matched with re.search) of parameter names to freeze, e.g. a VL model’s vision tower so RL only updates the language backbone.

RL Algorithm

Field	Type	Default	Description
advantage_estimator	str	"grpo"	Advantage estimator, e.g. "grpo".
n_samples_per_prompt	int	2	Responses sampled per prompt (the GRPO group size).
eps_clip	float	0.2	PPO clip lower bound.
eps_clip_high	float	0.28	PPO clip upper bound (asymmetric DAPO-style clipping).
use_kl_loss	bool	False	Add a per-token KL loss term against the reference model.
kl_loss_type	str	"low_var_kl"	KL formulation, e.g. "low_var_kl".
kl_loss_coef	float	0.0	Coefficient of the KL loss term.
kl_coef	float	0.0	KL penalty coefficient applied in the reward.
entropy_coef	float	0.0	Entropy bonus coefficient.
calculate_per_token_loss	bool	False	Average the loss over tokens instead of over samples.
ref_load	str	""	Checkpoint path the reference model is read from (for KL terms).

Dynamic Sampling

Field	Type	Default	Description
over_sampling_batch_size	int | None	None	Prompts sampled beyond rollout_batch_size so groups rejected by the filter can be replaced (DAPO).
dynamic_sampling_filter_path	str | None	None	Import path of the predicate deciding which sample groups to keep, e.g. dropping all-equal-reward groups.
balance_data	bool	False	Rebalance kept samples across data-parallel ranks.

Training and Optimizer

Field	Type	Default	Description
global_batch_size	int	16	Training samples per optim step.
lr	float	1e-06	Learning rate.
lr_decay_style	str	"constant"	Schedule, e.g. "constant" or "cosine".
weight_decay	float	0.1	Weight decay.
adam_beta1	float	0.9	Adam beta1.
adam_beta2	float	0.98	Adam beta2.
optimizer	str	"adam"	Optimizer name, e.g. "adam".

Memory and Precision

Field	Type	Default	Description
attention_dropout	float	0.0	Attention dropout probability.
hidden_dropout	float	0.0	Hidden-layer dropout probability.
attention_softmax_in_fp32	bool	True	Compute attention softmax in fp32.
accumulate_allreduce_grads_in_fp32	bool	True	Accumulate and all-reduce gradients in fp32.
use_distributed_optimizer	bool	False	Shard optim state across data-parallel ranks (Megatron distributed optimizer).
recompute_granularity	str	"full"	Activation recomputation granularity ("full" or "selective").
recompute_method	str	"uniform"	Recomputation method ("uniform" or "block").
recompute_num_layers	int	1	Layers per recomputation chunk.
qkv_format	str	"thd"	QKV layout for the Megatron backend ("thd" or "bshd"), emitted as --qkv-format.

Dynamic Batching

Field	Type	Default	Description
use_dynamic_batch_size	bool	True	Pack variable-length samples into micro-batches up to max_tokens_per_gpu instead of a fixed micro batch size.
max_tokens_per_gpu	int	9216	Token budget per GPU per micro-batch when dynamic batching is on.

Eval

Field	Type	Default	Description
eval_interval	int | None	None	Run eval every N rollout steps; None disables eval.
n_samples_per_eval_prompt	int	4	Responses sampled per eval prompt.
eval_max_response_len	int	16384	Max generated tokens per eval sample.
eval_top_p	float	1.0	Nucleus-sampling top-p for eval generation.
eval_config	dict | None	None	Inline dict materialized to a YAML file and passed as --eval-config; holds eval defaults and the eval dataset list.

Weight Sync

Field	Type	Default	Description
update_weight_mode	str	"full"	"full" rebroadcasts all weights each sync; "delta" pin-snapshots the last broadcast on CPU and ships only byte-level changes (~5-10x faster for large MoE models whose weights barely move per rollout).
update_weight_transport	str	"nccl"	"nccl" or "disk"; disk requires trainer and rollout engines to share a filesystem.
update_weight_encoding	str	"indices"	Encoding for delta payloads, e.g. "indices".
update_weight_disk_dir	str	""	Shared directory used by the disk transport.

Reward Model

Field	Type	Default	Description
rm_type	str | None	None	Name of a slime built-in reward function (e.g. "deepscaler"); leave None when shipping a reward callable instead.

Custom Functions and Hooks

Field	Type	Default	Description
custom_rm_function	collections.abc.Callable | None	None	Reward callable shipped by value to the containers and registered as slime’s --custom-rm-path.
custom_generate_function	collections.abc.Callable | None	None	Callable replacing slime’s generate step; shipped by value and registered via its resolved import path.
custom_reward_post_process_function	collections.abc.Callable | None	None	Callable applied to rewards after generation. Prefer this over setting a raw dotted path yourself: functions defined in a __main__ tutorial script have no reliably importable module name, so slime’s own importlib.import_module on that path fails inside the Ray actor.
rollout_function	collections.abc.Callable | str | None	None	Replaces slime’s entire rollout loop (--rollout-function-path).
custom_rollout_log_function	collections.abc.Callable | str | None	None	Called with each rollout’s data for logging; the gym wraps it so phase reporting and dashboard capture still run.
custom_eval_rollout_log_function	collections.abc.Callable | str | None	None	Same as above, for eval rollouts.
custom_megatron_before_log_prob_hook	collections.abc.Callable | str | None	None	Hook run in the Megatron trainer before log-prob computation.
custom_megatron_before_train_step_hook	collections.abc.Callable | str | None	None	Hook run in the Megatron trainer before each train step.

Config Overrides

Field	Type	Default	Description
extra_config	dict | None	None	The primary escape hatch: dict written to YAML and passed as --custom-config-path. Keys become attributes on slime’s parsed args and always override same-named recipe fields.
sglang_config	dict | None	None	Dict written to YAML and passed as --sglang-config — structured sglang engine config that isn’t a flat flag (e.g. PD-disaggregation server_groups).
sglang_request_params	dict | None	None	Extra request parameters injected into sglang generate calls (shipped via extra_config, read as args.sglang_request_params by generate paths such as on-policy distillation).
apply_chat_template_kwargs	dict | str	""	Kwargs forwarded to the tokenizer’s apply_chat_template, passed as inline JSON.
train_env_vars	dict | str | None	None	Env vars for the training processes, passed as inline JSON.
multimodal_keys	dict | str | None	None	Dataset columns holding multimodal inputs, passed as inline JSON; auto-filled from the attached DatasetConfig.

SGLang Rollout Engine

Field	Type	Default	Description
sglang_mem_fraction_static	float	0.75	Fraction of GPU memory sglang reserves for weights + KV cache.
sglang_enable_dp_attention	bool	False	Enable data-parallel attention across engine ranks.
sglang_dp_size	int | None	None	Data-parallel size for the engines.
sglang_ep_size	int | None	None	Expert-parallel size for MoE models.
sglang_enable_dp_lm_head	bool	False	Data-parallel LM head (pairs with DP attention).
sglang_disable_custom_all_reduce	bool	False	Fall back to NCCL all-reduce instead of sglang’s custom kernel.
sglang_cuda_graph_bs	list[int] | None	None	Batch sizes to capture CUDA graphs for.
sglang_max_running_requests	int | None	None	Cap on concurrent in-flight requests per engine.
sglang_tool_call_parser	str | None	None	Parser for tool-call output, e.g. "qwen25".
sglang_reasoning_parser	str | None	None	Parser for reasoning/thinking output.

Other Fields

Field	Type	Default	Description
sglang_cuda_graph_backend_prefill	str | None	None

Methods

cli_args(self, dataset: 'DatasetConfig | None' = None, model: 'ModelConfig | None' = None) -> list[str]

get_base_recipe(model_config: modal_training_gym.common.models.base.ModelConfig) -> 'SlimeRecipe'

validate_model_parallelism(self, model: 'ModelConfig') -> None

Related Tutorials

• Qwen3-4B haiku evaluation with verifiable rewards — serve, evaluate, train, compare
• Code RL with Harbor hello-world and sandboxed verification
• Multi-turn number-guessing RL with custom generate and reward functions
• On-policy distillation on math — Qwen3-8B teacher, Qwen3-4B student
• DAPO on math with Qwen3-4B
• Sweep hyperparameters across runs with TrainingGroup

Source: modal_training_gym/train_recipes/slime_recipe/recipe.py