3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering

The Problem: Too Many Tokens Waste Computation

3D vision-language models (3D VLMs) convert 3D scenes into tokens that preserve spatial geometry—a capability crucial for tasks like answering questions about 3D objects. But this approach creates thousands of tokens per scene, bloating memory use and slowing inference. While token compression has worked well for 2D images, those methods pick tokens based on semantic importance or attention patterns. They miss a key insight: 3D tokens have inherent spatial structure. Simply grouping nearby tokens doesn't solve the real problem—some objects end up over-represented while others are starved for tokens.

How 3DZip Works

3DZip attacks token redundancy in three stages:

Stage 1: Coarse Voxelization. First, the method divides 3D space into a coarse voxel grid and averages features within each cell. This strips away point-level noise without any learning required—a simple way to eliminate the most obvious duplication.

Stage 2: Diversity-Aware Selection. Next comes the core novelty: selecting representative anchor tokens using a Determinantal Point Process (DPP). Rather than picking tokens by semantic relevance alone, DPP encourages diversity in feature space. Intuitively, DPP ensures the selected tokens "spread out" in feature space, reducing overlap. This handles the object-level imbalance problem: even if nearby tokens are merged, diversity selection prevents one object from monopolizing the remaining token budget.

Stage 3: Spatial Merging. Finally, leftover tokens are merged with anchor tokens while respecting spatial constraints. This preserves geometric coherence—tokens that are far apart spatially won't be incorrectly merged, maintaining the 3D reasoning ability the model relies on.

Why It Works and What the Numbers Say

The three-stage design recognizes that 3D redundancy has multiple sources: point-level noise (fixed by voxelization), feature-space overlap (fixed by diversity selection), and geometric imbalance (fixed by spatial constraints). Existing compression methods ignore one or more of these.

Tested on three 3D question answering benchmarks, 3DZip retained 94.7% of the original model's performance while compressing to just 128 tokens—a massive reduction from thousands. The payoff: 1.92× faster inference. Crucially, 3DZip outperformed existing compression baselines, showing that spatial awareness and diversity matter more than simpler alternatives.

This matters because 3D VLMs are computationally hungry. Faster inference with negligible accuracy loss makes these models practical for real-world deployment, especially on resource-constrained devices. The insight—that 3D token redundancy is structurally different from 2D redundancy—opens a path for future compression research to exploit domain-specific geometry.