SULAND v2: A Refined RGB Dataset and Deep Learning Object Detection Benchmark for UAV/UGV-Based SUrface LANDmine Detection Under Domain Shift

Surface Landmine Detection Needs Better Data and Honest Benchmarking

Detecting landmines with cheap RGB cameras on drones and ground robots should work in theory—it's practical and affordable. But object detectors built for this safety-critical task remain largely unexamined, and nobody knows whether they actually generalize when deployment conditions change. The bottleneck: public RGB landmine datasets are scarce, and the main one available, SULAND, contains annotation errors that undermine any benchmark built on it.

The authors found systematic problems in SULAND: missing annotations, imprecise bounding boxes, inconsistent labeling rules for when mines are visible enough to count, mislabeled classes, and temporal inconsistencies where the same mine received different labels across frames. They manually corrected these issues across all 33,771 images and 12,433 boxes, creating SULAND_v2. This matters because sloppy annotations don't just add noise—they create a false sense of model performance.

What the Refinement Reveals

The improvements from cleaning up annotations are dramatic. YOLOv8, one of the most popular detectors, jumped 14.6–19.6 percentage points in test accuracy (mAP@50) on in-distribution data—cases where training and test images come from the same conditions. Fixing the class-ID bug alone boosted out-of-distribution (OOD) performance—where test images come from different environmental or sensor conditions—by ~25 percentage points for YOLOv8.

The authors then benchmarked 35 detector configurations across nine families (YOLO variants, Faster R-CNN, EfficientDet, DETR-based models, and others). On SULAND_v2, YOLOv12-Small achieved the best in-distribution performance (0.908 mAP@50), but RF-DETR-Large delivered the strongest out-of-distribution results (0.799 mAP@50, 0.675 recall). This split reveals a critical operational insight: high test accuracy on the training domain doesn't guarantee the detector will work in the field under different lighting, terrain, or seasonal conditions.

Why This Matters for Mine Action

Landmine surveys happen in diverse real-world conditions—different seasons, lighting, soil types, and sensor platforms. A detector trained on one set of deployment scenarios must work on others. SULAND_v2 is designed to expose this domain-shift problem explicitly, separating in-distribution from out-of-distribution test splits. The data and benchmarks provide practitioners a way to evaluate whether their detector is robust enough for deployment, not just whether it scores well on a test set.

The refined dataset and honest benchmarking establish a standard that the mine-action community can build on. By publishing both the corrected annotations and the full benchmark results, the authors create accountability: future work can be measured against detectors that actually work in the field, not against inflated numbers from flawed annotations.