SceneFrom3D: Geometry-Conditioned Outdoor 3D Scene Generation via View Scheduling with Object-Level Control

The Problem: View Scheduling for Outdoor 3D Scenes

Current methods for generating 3D outdoor scenes from user-provided geometry follow a three-stage pipeline: pick camera viewpoints, synthesize images from those viewpoints, then reconstruct a 3D model. The bottleneck is the first step—deciding which views to render. For outdoor scenes with large, sprawling, and unbounded geometry, this is hard. You need enough viewpoint coverage to see the whole scene while maintaining stable, consistent image generation. Manual view scheduling doesn't scale.

SceneFrom3D solves this by automatically determining the viewing strategy based on the input geometry itself, rather than requiring users to specify it upfront. This is the core contribution.

How It Works: The Generation Graph

The authors' key insight is to represent the viewing plan as a directed generation graph. Nodes in this graph are anchor views—strategically chosen camera positions that should each produce a full image. Edges represent interpolation trajectories—smooth camera paths between those anchor views where intermediate frames are synthesized to fill gaps.

This two-level structure elegantly sidesteps the coverage problem. Rather than densely sampling all possible viewpoints (computationally prohibitive) or sparsely guessing key views (unreliable), the system identifies critical anchor views and smoothly connects them. The graph structure also enforces an order to generation, enabling stable sequential synthesis where each new view conditions on previously generated content.

The paper doesn't detail the exact algorithm for constructing this graph from geometry, but the abstract indicates it analyzes the input geometry to determine coverage and stability—likely using metrics like occlusion, depth complexity, or viewpoint diversity.

Object-Level Control

Beyond automatic view scheduling, SceneFrom3D adds fine-grained control through two mechanisms:

1. Identity images: Users assign each object in the scene an appearance image (a reference photo or sketch), guiding what that object should look like in the final render.

2. Geometry-adherence parameters: A per-object scalar controls how strictly the generation respects input geometry versus allowing artistic deviation. High adherence locks the object to its original shape; low adherence gives the generative model more freedom.

This dual control is practical—users get automatic view planning (reducing manual burden) while retaining object-level artistic direction.

Why This Matters

Geometry-conditioned 3D scene generation is useful for urban planning, game asset creation, and architectural visualization—workflows where users have a rough layout but want photorealistic rendering. Previous methods required manual view scheduling, a tedious step that often failed on complex outdoor scenes.

By automating view selection based on geometry analysis, SceneFrom3D reduces friction and improves reliability. The object-level conditioning ensures the output isn't just geometrically correct but also visually controllable—critical for professional use cases where appearance matters as much as structure.

The experiments validate state-of-the-art quality on outdoor scenes, though the abstract doesn't specify dataset or baselines. The combination of automatic planning and manual per-object control strikes a practical balance between convenience and creative authority.