arxiv_cv 90% Match Research Paper Robotics Engineers,Autonomous Systems Developers,Computer Vision Researchers,Sensor Fusion Specialists 3 weeks ago

Targetless LiDAR-Camera Calibration with Neural Gaussian Splatting

computer-vision › 3d-vision

📄 Abstract

Abstract: Accurate LiDAR-camera calibration is crucial for multi-sensor systems. However, traditional methods often rely on physical targets, which are impractical for real-world deployment. Moreover, even carefully calibrated extrinsics can degrade over time due to sensor drift or external disturbances, necessitating periodic recalibration. To address these challenges, we present a Targetless LiDAR-Camera Calibration (TLC-Calib) that jointly optimizes sensor poses with a neural Gaussian-based scene representation. Reliable LiDAR points are frozen as anchor Gaussians to preserve global structure, while auxiliary Gaussians prevent local overfitting under noisy initialization. Our fully differentiable pipeline with photometric and geometric regularization achieves robust and generalizable calibration, consistently outperforming existing targetless methods on KITTI-360, Waymo, and FAST-LIVO2, and surpassing even the provided calibrations in rendering quality.

Key Contributions

Presents TLC-Calib, a targetless LiDAR-camera calibration method that jointly optimizes sensor poses and a neural Gaussian-based scene representation. It achieves robust and generalizable calibration without physical targets.

Business Value

Enables more reliable and cost-effective deployment of multi-sensor systems (e.g., autonomous vehicles) by simplifying and improving the calibration process, reducing downtime and increasing safety.

Paper Metadata

Innovation Type

Algorithmic Approach and Representation

Deployment Feasibility

High, as it addresses a critical practical need in robotics and autonomous systems and offers a robust, targetless solution.

Limitations Addressed

Impracticality of physical targets for calibration, degradation of calibration over time, and limitations of existing targetless methods.

Performance Gains

Consistently outperforms existing targetless methods and surpasses provided calibrations in rendering quality.

Technical Tags

LiDAR-Camera CalibrationNeural Gaussian SplattingTargetless CalibrationSensor Fusion3D Scene RepresentationDifferentiable RenderingRobust Calibration

Research Topics

Computer VisionRoboticsSensor Fusion3D ReconstructionCalibration

Methods & Architectures

Neural Gaussian SplattingJoint optimization of poses and scene representationPhotometric and geometric regularizationDifferentiable rendering Neural Gaussian Splatting

Applications & Tasks

Autonomous Driving Robotics Augmented Reality 3D Mapping Sensor CalibrationReal-world Deployment ChallengesSensor DriftTargetless Methods LiDAR-Camera Calibration3D Scene Representation

Datasets & Benchmarks

Datasets

KITTI-360, Waymo, FAST-LIVO2

Benchmarks

Outperforms existing targetless methods on KITTI-360, Waymo, and FAST-LIVO2. • Surpasses provided calibrations in rendering quality.

Calibration AccuracyRendering QualityRobustnessGeneralizability

Related Fields

RoboticsComputer VisionSensor FusionAutonomous Driving3D VisionCalibration

Keywords

LiDAR-Camera CalibrationTargetless CalibrationNeural Gaussian SplattingSensor Fusion3D Scene RepresentationAutonomous DrivingRoboticsDifferentiable RenderingCalibrationKITTI-360

Academic Context

#Computer Vision#Robotics#Sensor Fusion#3D Reconstruction#Calibration

Commercial Potential

Potential Products

Automated calibration tools for autonomous vehicle fleetsSoftware modules for robot perception systems3D mapping and surveying solutions

Target Industries

AutomotiveRoboticsAerospaceMapping & SurveyingLogistics

Use Case Examples

Calibrating LiDAR and cameras on self-driving cars without needing specialized targets.Ensuring accurate sensor alignment for robotic manipulation tasks.

Competitive Edge

Offers a novel targetless calibration approach using neural representations, outperforming existing methods in robustness and accuracy.

Market Opportunity

Large and growing market for autonomous systems and robotics.

Revenue Models

Licensing of calibration softwareintegration into sensor suites.

Resource Requirements

Compute Needs

High, for training neural representations and optimization.

Data Requirements

Paired LiDAR and camera data from diverse scenes.

Deployment Constraints

Computational cost during calibration, potential need for large datasets.

Scalability

Scalability depends on the size of the scene and the complexity of the Gaussian representation.

Production Readiness

Maturity Level

Research

Time to Market

1-3 years for integration into commercial systems.

Patent Potential

High, for the novel targetless calibration method and its integration with neural representations.

View Full Paper Back to Papers