TESSERA v2: Scaling Pixel-wise Earth Foundation Models

The Scaling Problem Nobody Had Solved Yet

Pixel-wise Earth observation (EO) foundation models—neural networks trained on satellite imagery to produce spatial embeddings—are getting better, but researchers had no idea how to scale them efficiently. The authors ran 395 training experiments across massive hardware (1,024 GH200 superchips) to find out what actually works. Their finding upends conventional wisdom: pretraining loss—the metric most researchers use to select models—is nearly useless for predicting real-world performance. Models chosen by loss alone waste substantial compute.

How to Actually Scale These Models

The authors trained models within a fixed Barlow Twins architecture (a self-supervised learning framework) and tested each one on 15 downstream tasks: crop type classification, cloud detection, building segmentation, and others. They discovered a simple allocation rule: as your compute budget grows, scale the encoder (the main neural network) and the training data together, but keep the projector (a smaller output layer) fixed. This is counterintuitive—most scaling laws suggest everything grows in lockstep.

Armed with this rule, they trained models up to 1 billion parameters and then compressed them into smaller "student" models via distillation (training a small network to mimic a large one). The distilled 21-million-parameter version, TESSERA v2-1B-M, outperformed all tested open and proprietary competitors, many vastly larger.

Why This Matters

The compression strategy is particularly practical. The distilled models produce Matryoshka representations—embeddings structured so that shorter prefixes retain most of the information. A 16-dimensional prefix from the full 128-dimensional embedding keeps 92% of performance while cutting storage by 8×. For anyone deploying embeddings at scale, this translates directly to cost savings.

The broader insight is methodological: a controlled scaling study revealed that what works in computer vision (e.g., ImageNet pretraining) doesn't automatically transfer to satellite data. Downstream task performance, not pretraining metrics, should drive model selection—a labor-intensive principle but one that avoids the trap of overfitting to misleading loss curves.

The authors promise to release global v2 embeddings covering 2017–2025, making their foundational work accessible beyond the research setting. For the Earth observation community, this is significant: a concrete recipe (large encoder, curated data, distilled students) backed by 395 experiments removes guesswork from a costly process.

The core claim is simple but powerful: you can build smaller, cheaper EO models that work better if you abandon loss-based selection and adopt a principled scaling strategy informed by downstream evaluation.