This paper, stemming from Duality's work with NASA-JPL in DARPA’s RACER program, introduces a novel digital twin approach for building off-road autonomy — showing how synthetic data can fine-tune foundation models for semantic segmentation and outperform models trained solely on real-world datasets. By generating synthetic data from photorealistic, physics-accurate 3D environments with digital twins of autonomous vehicles, we demonstrate how domain-specific synthetic data enables more reliable perception training in novel, complex, and variable terrains.