Nanbeige4.2-3B on Apple Silicon: Fixing Deployment Bugs and Decreasing Looped Transformer Memory Overhead

The Core Issue: A Broken Model on Apple Hardware

Nanbeige4.2-3B ships broken. This 3-billion-parameter model, designed to run on Apple Silicon devices (via Metal Performance Shaders), contains five separate bugs that prevent it from even loading through standard Hugging Face tools—and when those bugs are fixed, it still can't handle the tasks it was built for. The authors identify and fix all of these issues, then show that the model's clever efficiency trick actually creates a hidden memory wall that blocks practical use.

Why the Model Exists (and Why It's Hard to Run)

Nanbeige4.2-3B uses a Looped Transformer architecture. Instead of stacking many layers, it reuses a single stack twice—running data through the same 12 layers in two forward passes sequentially. This cuts parameter count roughly in half compared to a traditional 3B model at similar depth, a valuable win for on-device deployment.

But Apple Silicon has tight memory constraints. The reuse strategy creates a problem: attention computation requires storing activations for both the first and second loop passes, effectively doubling peak memory use at the attention layer. A context window that fits in theory doesn't fit in practice.

Fixing and Optimizing

The five bugs ranged from data corruption (a Rotary Position Embedding buffer silently zeroing out on MPS) to deprecated API calls (the model tried to use Hugging Face transformer cache interfaces that no longer exist). Fixing these alone wasn't enough.

The authors introduce chunked-prefill: instead of processing the entire input context at once before the first loop, they split it into chunks and pipeline computation. This reduces peak attention memory by deferring some computation, allowing 2.7× longer context windows on 32 GB shared memory. They also patched MPS memory bugs and rewrote the system prompt handling.

What Works Now (and What Doesn't)

On MCPMark (a benchmark for multi-step agentic reasoning), the original model completed 0% of tasks. The patched version completes 30%. On BFCL (tool-calling evaluation), the model performs nearly perfectly on single-tool tasks but fails most multi-tool scenarios—suggesting the depth added by looping helps some reasoning chains but not others.

The gap between theory and practice is stark: even a small, parameter-efficient model needs careful systems work to become usable. The authors release patched weights, evaluation code, and tools for system prompt optimization.

Why This Matters

This paper reveals a gap in production ML: academic model architectures often assume ideal software stacks and infinite memory. Real deployment—especially on constrained Apple Silicon—requires debugging, optimization, and honest benchmarking. The chunked-prefill strategy is a general technique applicable to other Looped Transformer variants. Most importantly, the authors publish fixes and measurements that save downstream developers from rediscovering the same bugs.