Accurate camera-LiDAR calibration is an essential prerequisite for robust multi-modal perception in robotics. Recent target-less methods based on learned pixel-point correspondences achieve high extrinsic calibration accuracy and generalize across environments. However, they operate on rectified images with known camera intrinsics, restricting their applicability to systems whose cameras have already been individually calibrated. We present a fully automatic, target-less framework that jointly estimates camera intrinsics, radial-tangential distortion, and camera-LiDAR extrinsics from raw images and LiDAR scans. Our method generalizes deep correspondence-based calibration from rectified images to raw cameras through intrinsic- and distortion-aware correspondence learning via synthetic augmentations. During calibration, we iteratively couple correspondence prediction and nonlinear optimization to jointly refine the camera parameters and relative sensor transformation. We evaluate our approach on an unseen camera-LiDAR configuration from the KITTI dataset and on three real robotic platforms in a zero-shot manner. The results demonstrate that joint calibration with our generalized pixel-point correspondence network improves extrinsic accuracy over correspondence-based methods using pre-calibrated cameras, while additionally recovering accurate intrinsics.

Our fully automatic framework calibrates a camera and LiDAR from synchronized raw images and point clouds, without calibration targets or manually specified initial parameters. First, COLMAP structure-from-motion provides initial camera intrinsics and radial-tangential distortion, while motion alignment of trajectories estimated with ORB-SLAM3 and FAST-LIO2 initializes the camera-LiDAR transformation. We extend CMRNext to predict pixel-point correspondences between RGB images and LiDAR scans projected using the current intrinsic, distortion, and extrinsic estimates. To generalize to previously unseen raw cameras, our training includes perturbed camera parameters and synthetically distorted images generated by camera-to-camera remapping with validity checks that avoid radial-distortion folding. During calibration, we select reliable correspondences and minimize robust reprojection errors using joint nonlinear optimization, including a weak principal-point prior. The optimized camera parameters and relative sensor pose are fed back to the correspondence network at each iteration. We jointly refine intrinsics and extrinsics for the first five iterations, then fix the intrinsics and continue refining the extrinsics for eleven iterations in total to improve stability.

Simon Bultmann, Daniele Cattaneo, and Abhinav Valada
Generalizable Target-Less Joint Intrinsic–Extrinsic Camera–LiDAR Calibration using Deep Pixel–Point Correspondences
arXiv preprint arXiv:2610.37350, 2026.

@article{bultmann2026generalizabletargetlessjointintrinsicextrinsic,
  title={Generalizable Target-Less Joint Intrinsic-Extrinsic Camera-LiDAR Calibration using Deep Pixel-Point Correspondences},
  author={Simon Bultmann and Daniele Cattaneo and Abhinav Valada},
  journal={arXiv preprint arXiv:2610.37350},
  year={2026},
  eprint={2610.37350},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2610.37350}
}