Gnani-Evon-v3.3-30B-A3B: A 5.2-Point Indic Reasoning Gain at 1.7-Point English Cost

Executive Summary & Core Discovery

The Gnani-Evon-v3.3-30B-A3B model, a hybrid Mamba-Transformer mixture-of-experts architecture, achieves a significant 5.2-point gain in Indic reasoning performance while sacrificing 1.7 points in English retention. This tradeoff underscores the challenges of optimizing for multilingual capabilities in large language models. The model’s post-training reinforcement learning phase, while improving mathematics, graduate-level science, and instruction following, inadvertently degraded Indic extractive question answering, with the share of Indic extraction items answered in English rising from 8.7% to 91.7%. These findings provide valuable insights into the complexities of balancing multilingual performance in advanced reasoning models.

Architectural Background & Theoretical Foundations

Gnani-Evon-v3.3-30B-A3B is built upon NVIDIA’s Nemotron-3-Nano-30B-A3B checkpoint, retaining its 52-layer backbone but modifying the vocabulary, training data, and post-training objectives. The model incorporates a hybrid architecture combining Mamba and Transformer components with a mixture-of-experts design. This architecture allows for efficient handling of long-context sequences and multilingual reasoning tasks. The model’s vocabulary was expanded from 131,072 to 163,072 tokens, with the new embedding rows initialized from their upstream decompositions and warmed before the backbone was updated. This expansion reduced the mean Indian-language fertility by 38.6%, indicating a more efficient use of the extended vocabulary space.

Benchmark Architecture & Evaluation Methodology

The evaluation of Gnani-Evon-v3.3-30B-A3B involved several key phases. Continual pretraining was conducted on a broad English-Indic dataset with high-quality annealing and direct training at a 131,072-token sequence length. Post-training reinforcement learning was performed across four verifiable task environments, focusing on mathematics, graduate-level science, and instruction following. The model was also tested on Indic extractive question answering, revealing a significant degradation in performance due to the lack of language-specific rewards. Two unreleased studies explored repairs, including a scoped Indic reward that restored language conventions and a 26B-A4B generative judge that outperformed a larger 235B-parameter model on RM-Bench. Distillation was used to restore capabilities displaced by optimization against the generative judge, achieving an 81% recovery rate.

Empirical Findings & Granular Performance Analysis

The empirical results of Gnani-Evon-v3.3-30B-A3B are striking. The model gains 5.2 points on an Indic aggregate while losing 1.7 points of English retention relative to the starting checkpoint. The unreleased Indic-aware branch shows even more pronounced gains, with a 14.6-point increase in Indic performance at a cost of 5.4 points in English. The planned FP8 release reduces the model’s artifact size to 32GB with near-zero mean change across 32 paired benchmarks, demonstrating the model’s efficiency and scalability. The post-training reinforcement learning phase improved mathematics, graduate-level science, and instruction following but degraded Indic extractive question answering, with the share of Indic extraction items answered in English rising from 8.7% to 91.7%. Accuracy conditional on answering in the target script roughly doubled, indicating preserved reading ability but measured on a self-selected subset.

In-Depth Analysis of Core Limitations & Vulnerabilities

The Gnani-Evon-v3.3-30B-A3B model presents several core limitations that must be addressed for further optimization. The most significant issue is the degradation of Indic extractive question answering due to the lack of language-specific rewards during the reinforcement learning phase. This highlights the need for more nuanced reward structures that account for multilingual requirements. Additionally, the tradeoff between Indic and English performance indicates the challenges of balancing multilingual capabilities in large language models. The unreleased studies suggest that scoped rewards and generative judges can help mitigate these issues, but further research is needed to fully understand and address these limitations.

1. Language-Specific Reward Misalignment

The degradation of Indic extractive question answering is primarily due to the lack of language-specific rewards during the reinforcement learning phase. This misalignment between the rewards and the target language requirements highlights the need for more sophisticated reward structures that can account for multilingual nuances.

2. Tradeoff Between Indic and English Performance

The significant tradeoff between Indic and English performance reveals the complexities of optimizing for multilingual capabilities in large language models. This tradeoff must be carefully managed to ensure that the model’s performance in both languages is balanced and meets the desired standards.

3. Reinforcement Learning Focus

The reinforcement learning phase focused on mathematics, graduate-level science, and instruction following, which improved these areas but inadvertently degraded Indic extractive question answering. This indicates the need for a more balanced approach to reinforcement learning that accounts for multilingual requirements.

4. Generative Judge Performance

The 26B-A4B generative judge outperformed a larger 235B-parameter model on RM-Bench, demonstrating the potential of smaller, more efficient models in certain evaluation contexts. However, the need for distillation to restore displaced capabilities highlights the challenges of optimizing generative judges for multilingual tasks.

Enterprise Production & Red-Teaming Blueprint

For enterprise production and red-teaming, the findings from Gnani-Evon-v3.3-30B-A3B highlight the importance of carefully managing the tradeoffs between multilingual capabilities. Developers should prioritize the use of language-specific rewards and generative judges to mitigate the degradation of Indic extractive question answering. Additionally, a more balanced approach to reinforcement learning is needed to ensure that the model’s performance in both Indic and English is optimized. The unreleased studies provide valuable insights into potential repair mechanisms, such as scoped rewards and distillation, which can be explored further to enhance the model’s multilingual capabilities.

Technical FAQ (Granular & Non-Repetitive)

Takeaway: The Gnani-Evon-v3.3-30B-A3B model’s performance highlights the critical tradeoffs in optimizing multilingual capabilities in advanced reasoning models. By carefully managing these tradeoffs and exploring repair mechanisms, developers can enhance the model’s capabilities and ensure balanced performance across languages.