arxiv_cv 95% Match Research Paper 3D Vision Researchers,Game Developers,Urban Planners,AR/VR Developers,GIS Specialists 2 weeks ago

Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery

computer-vision › 3d-vision

📄 Abstract

Abstract: Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task in providing immersive and embodied applications. The challenges lie in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by synergizing the readily available satellite imagery that supplies realistic coarse geometry and the open-domain diffusion model for creating high-quality close-up appearances. We propose \textbf{Skyfall-GS}, the first city-block scale 3D scene creation framework without costly 3D annotations, also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic textures. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/

Authors (9)

Jie-Ying Lee

Yi-Ruei Liu

Shr-Ruei Tsai

Wei-Cheng Chang

Chung-Ho Wu

Jiewen Chan

+3 more

Submitted

October 17, 2025

arXiv Category

cs.CV

arXiv PDF

Key Contributions

Skyfall-GS is the first framework to synthesize city-block scale 3D urban scenes without costly 3D annotations by synergizing satellite imagery (for coarse geometry) and diffusion models (for high-quality appearances). It employs a curriculum-driven iterative refinement strategy to enhance geometric completeness and photorealism, enabling real-time, immersive 3D exploration.

Business Value

Enables the creation of detailed, explorable 3D urban environments for applications like virtual city tours, urban planning simulations, and immersive gaming experiences, reducing the need for expensive manual 3D modeling.

Paper Metadata

Innovation Type

Algorithmic Approach / Framework Design

Deployment Feasibility

Moderate. Requires significant computational resources for synthesis and rendering, but the output is explorable 3D scenes.

Limitations Addressed

Scarcity of large-scale, high-quality 3D scan data,Challenges in training generalizable 3D generative models,High cost of 3D annotations

Performance Gains

Provides improved cross-view consistent geometry and more realistic textures.

Technical Tags

3D scene synthesisurban scenessatellite imagerydiffusion modelsgenerative modelsgeometric accuracyphotorealistic texturescurriculum learningiterative refinement3D reconstruction

Research Topics

3D Computer VisionGenerative AIScene SynthesisGeospatial AIComputer Graphics

Methods & Architectures

Synergizing satellite imagery and diffusion modelsCurriculum-driven iterative refinementGenerating coarse geometry from satellite dataCreating high-quality appearances with diffusion modelsFramework without costly 3D annotations Diffusion modelsGenerative models

Applications & Tasks

Urban Planning Architecture Gaming Virtual Reality Augmented Reality Geographic Information Systems (GIS) Lack of large-scale, high-quality 3D scansDifficulty in training generalizable generative models for 3D scenesCostly 3D annotations 3D urban scene synthesisReal-time 3D explorationGenerating explorable 3D environments

Datasets & Benchmarks

Datasets

Satellite imagery

Related Fields

Computer VisionGenerative AI3D ReconstructionComputer GraphicsGeospatial AnalysisDeep Learning

Keywords

3D scene synthesisurban scenessatellite imagerydiffusion modelsgenerative modelsgeometric accuracyphotorealistic texturescurriculum learningiterative refinement3D reconstructionvirtual realitygaming

Academic Context

#3D Computer Vision#Generative AI#Scene Synthesis#Geospatial AI#Computer Graphics

Commercial Potential

Potential Products

Tools for generating virtual citiesPlatforms for architectural visualization3D environment creation tools for games and simulations

Target Industries

GamingReal EstateUrban PlanningArchitectureFilm and EntertainmentMetaverse

Use Case Examples

Creating realistic virtual environments for training simulations (e.g., emergency response)Generating explorable 3D models of cities for tourism or planningBuilding detailed backdrops for video games

Competitive Edge

Unique in its approach of combining satellite data with diffusion models for large-scale urban scene synthesis without 3D annotations, offering a scalable alternative to traditional methods.

Market Opportunity

Large and growing market for 3D content creation and virtual environments.

Revenue Models

Licensing of the synthesis frameworkservices for creating custom 3D environments.

Resource Requirements

Compute Needs

High, for training diffusion models and iterative refinement.

Data Requirements

Requires access to satellite imagery and potentially other geospatial data.

Deployment Constraints

Synthesis process can be computationally intensive and time-consuming.

Scalability

Scalability to city-block scale is a key feature, but larger scales might require further optimization.

Production Readiness

Maturity Level

Research

Time to Market

2-3 years for a robust platform.

Patent Potential

Moderate, for the novel combination of data sources and refinement strategy.

View Full Paper Back to Papers