SUFLECA: Scaling Up Feature Learning for CAD-to-image Alignment

The Core Problem and Solution

CAD-to-image alignment—figuring out an object's exact 3D position, rotation, and scale from a single photo—matters for robotics and AR applications. Current zero-shot methods (those that don't require task-specific training) rely on visual foundation models to find matching points between image regions and 3D CAD models, but they fail when objects are partially hidden or when there's a gap between synthetic training data and real-world images.

The authors' key insight is that geometry matters more than appearance. Rather than matching based on what pixels look like, SUFLECA learns features that encode geometric structure—where on an object's surface each pixel maps to. This geometry-grounded approach is more robust to occlusion and domain shift.

How It Works

SUFLECA has two main pieces:

Geometry-aware feature learning: The authors take a pretrained visual foundation model and fine-tune it using Normalized Object Coordinates (NOCs)—a supervision signal that labels each pixel with its 3D location on the object's surface. They train on 674K images across 12 datasets (both real and synthetic), scaling up this weakly-supervised learning to teach the model to recognize geometry rather than just appearance. The result is compact, reusable features that transfer across different datasets and domains.

Geometrically consistent matching: Once you have image features and CAD model features, you need to find which image pixels correspond to which 3D points on the CAD model. A naive approach might match the closest features globally, creating inconsistencies—one pixel might claim to match a corner while its neighbor claims to match a distant edge. Their algorithm enforces geometric consistency, requiring that matched correspondences form a coherent 3D structure. This produces reliable one-to-one pairings without the need for slow iterative refinement.

Why This Matters

The results are substantial. On ScanNet25k (a standard benchmark), SUFLECA achieves 33.4% accuracy when evaluated by category and 42.3% when evaluated by individual instance. This beats the previous zero-shot state-of-the-art by over 10 percentage points—a large gap in this space. More strikingly, it even outperforms fully supervised methods that are task-trained on the same data, suggesting that geometry-grounded features + geometric consistency are genuinely superior principles, not just workarounds.

The method runs in under a second per object without iterative refinement, making it practical for real-time applications. The authors release code and models, enabling adoption in robotics pipelines where 6D/9D pose estimation is a bottleneck.

The underlying insight—that large-scale geometry supervision beats appearance-driven matching for cross-domain robustness—extends beyond CAD alignment. It suggests a direction for improving other vision tasks where domain shift is a challenge: ground features in geometric structure rather than visual appearance.