VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use

The Core Finding

VectraYX-Vision-1B failed to learn visual grounding despite a fully functional system. The model has all the right pieces—a frozen image encoder, a Spanish-language decoder, structured reasoning tokens, and tool-calling ability—but during training it learned to ignore images entirely, scoring near-zero (0.08) on tool identification benchmarks. Rather than hiding this, the authors diagnose the failure, propose fixes, and release the code to help others avoid the same trap.

What the Model Does (When It Works)

The architecture couples a SigLIP image encoder (frozen, pre-trained) to a 1.04-billion-parameter decoder trained specifically for Spanish and Latin American Spanish. Between them sits a simple MLP bridge. The model targets a niche audience: cybersecurity analysts working with tools like IDA (reverse-engineering), Ghidra, Wireshark (packet analysis), Nmap, Metasploit, and Volatility (memory forensics).

Beyond describing images, it outputs two special token types: <|think|> for structured reasoning steps and <|tool_call|> for invoking external tools via the Model Context Protocol. The entire system fits under 2 billion parameters and can run offline on CPU via llama.cpp's mmproj format—critical for air-gapped environments where sensitive security work happens.

The Problem and the Path Forward

The authors trained on 400 to 1,900 steps (16 million tokens) of supervised finetuning (SFT). Despite working pipelines, the model learned to emit plausible text without looking at images. They traced this to three likely culprits: (1) training was too short, (2) too little data replay (they recommend ≥60%), and (3) learning rate was too high. They also uncovered a checkpoint-loader bug where an unstripped llm. prefix masked what looked like training collapse—a cautionary tale for practitioners.

Rather than hand-wave, they release three ablations (V0, V1, V2) testing whether periodic "no-positional-encoding" (NoPE) layers—which drop position signals in some Transformer blocks—help or hurt attention over the 729-token visual feature sequence. This architectural question has been under-explored and deserves empirical data.

Why This Matters

This work is brutally honest about failure in a space where most papers hide missteps. A sub-2B model for cybersecurity in Spanish/LATAM is novel; the first fully specified implementation, including tool use and offline deployment, fills a real gap. Even though the current vision component doesn't work, the released code, weights, and 14,596 QA pairs across 10 security domains establish a baseline and benchmark for future builders.

The ablation matrix and diagnostic detail—pinpointing the checkpoint bug, specifying SFT parameters—make this useful beyond the specific model. Anyone scaling vision-language models will recognize these failure modes and debug strategies.

The authors open-source everything: model checkpoints, training configs, and trajectories. They've prioritized reproducibility and architectural clarity over claiming premature success.