ATTUNE: Agentic Throughput Tuning for LLM Training

Executive Summary & Core Discovery

The throughput of LLM training jobs is a critical factor in the cost and efficiency of model development. Traditional automated tuners search a fixed space of knobs and only check if a job runs, missing potential levers and risking silent configuration changes. ATTUNE addresses these challenges by using a coding agent that tunes any working recipe unattended within a GPU-hour budget. Guided by a skill set of search rules, scripts, and lessons from past tuning campaigns, ATTUNE diagnoses each run and changes one lever at a time. A correctness gate outside the agent refuses any argument known to change the training objective, ensuring the integrity of the training process. ATTUNE has been successfully applied to Amazon’s production post-training recipes on H100, H200, and B200 clusters, raising throughput by 1.7x on average. It also beats the published throughput of four of NVIDIA’s own benchmark recipes on NVIDIA’s AutoModel framework by 6-17%. On models held out from its development, ATTUNE improves the strongest public recipes most where they were written for another setting, by up to 2.6x on MiniMax-M2, and least where the vendor tuned them for the same hardware. Given the same GPU-hour budget, ATTUNE outperforms existing automated tuners, including Bayesian optimization, random search, and OptiCo, a recent LLM-agent tuner. Its gain on GLM-4.5 is more than four times the best alternative’s. The same coding agent without ATTUNE’s skill gains less and fails half its launches.

Architectural Background & Theoretical Foundations

The throughput of an LLM training job depends on its recipe: how the model is split across GPUs, how memory is traded for compute, and which kernels and communication overlaps it uses. The best recipe changes with every model, accelerator, and framework version. Traditional automated tuners search a fixed space of knobs and only check if a job runs, missing levers outside that space and risking silent configuration changes. ATTUNE addresses these challenges by using a coding agent that tunes any working recipe unattended within a GPU-hour budget. Guided by a skill set of search rules, scripts, and lessons from past tuning campaigns, ATTUNE diagnoses each run and changes one lever at a time. A correctness gate outside the agent refuses any argument known to change the training objective, ensuring the integrity of the training process.

Benchmark Architecture & Evaluation Methodology

ATTUNE has been evaluated on Amazon’s production post-training recipes on H100, H200, and B200 clusters. It has also been tested on NVIDIA’s AutoModel framework, where it beats the published throughput of four of NVIDIA’s own benchmark recipes by 6-17%. On models held out from its development, ATTUNE improves the strongest public recipes most where they were written for another setting, by up to 2.6x on MiniMax-M2, and least where the vendor tuned them for the same hardware. Given the same GPU-hour budget, ATTUNE outperforms existing automated tuners, including Bayesian optimization, random search, and OptiCo, a recent LLM-agent tuner. Its gain on GLM-4.5 is more than four times the best alternative’s. The same coding agent without ATTUNE’s skill gains less and fails half its launches.

In-Depth Analysis of Core Limitations & Vulnerabilities

ATTUNE addresses several core limitations in existing LLM training optimization methods. It uses a coding agent that tunes any working recipe unattended within a GPU-hour budget, guided by a skill set of search rules, scripts, and lessons from past tuning campaigns. It diagnoses each run and changes one lever at a time, ensuring that each configuration change is verified to ensure training objective integrity. A correctness gate outside the agent refuses any argument known to change the training objective, ensuring the integrity of the training process.

1. Limited Knob Space

Traditional automated tuners search a fixed space of knobs and only check if a job runs, missing levers outside that space. ATTUNE addresses this limitation by using a coding agent that tunes any working recipe unattended within a GPU-hour budget, guided by a skill set of search rules, scripts, and lessons from past tuning campaigns.

2. Silent Configuration Changes

Traditional automated tuners risk silent configuration changes that can affect the training process. ATTUNE addresses this limitation by diagnosing each run and changing one lever at a time, ensuring that each configuration change is verified to ensure training objective integrity.

3. Inefficient Memory-Compute Trade-offs

Traditional automated tuners may not optimize memory-compute trade-offs efficiently. ATTUNE addresses this limitation by using a coding agent that tunes any working recipe unattended within a GPU-hour budget, guided by a skill set of search rules, scripts, and lessons from past tuning campaigns.

4. Changes to Training Objective

Traditional automated tuners may introduce changes to the training objective that can affect the model’s performance. ATTUNE addresses this limitation by using a correctness gate outside the agent that refuses any argument known to change the training objective, ensuring the integrity of the training process.

Enterprise Production & Red-Teaming Blueprint

ATTUNE’s agentic approach to throughput tuning offers a more efficient and reliable method for improving training throughput. It can be integrated into existing LLM training pipelines to optimize throughput and ensure configuration integrity. Enterprise teams can use ATTUNE to verify each configuration change and ensure that the training objective remains unchanged. Additionally, ATTUNE can be used to stress-test and red-team LLM training processes to identify potential vulnerabilities and ensure robustness.

Technical FAQ (Granular & Non-Repetitive)

ATTUNE represents a significant advancement in LLM training optimization, offering a more efficient and reliable method for improving training throughput. Its agentic approach, guided by a skill set of search rules and lessons from past tuning campaigns, ensures that each configuration change is verified to maintain training objective integrity. By addressing core limitations in existing methods, ATTUNE sets a new standard for LLM training optimization.