Intrinsic and extrinsic calibration of a UAV-based multi-sensor system
-
Arman Khami
,
Ingo Neumann
Abstract
The integration of multiple sensors, such as cameras and LiDARs, is increasingly employed in vehicle navigation and 3D environmental mapping. Accurate multi-sensor data fusion relies heavily on the precise estimation of extrinsic parameters, which define the geometric transformations between sensors. However, when low-cost sensors are used, systematic errors in their intrinsic characteristics often introduce inaccuracies in subsequent applications. To address this challenge, we investigate the impact of intrinsic calibration of a low-cost LiDAR on the accuracy of its extrinsic calibration parameters. Our study evaluates a generalized intrinsic calibration approach that assumes a single scale factor and bias for the LiDAR’s laser rays, in contrast to a detailed model where individual scale factors and biases are estimated for each ray. While the generalized approach exhibits noticeable deviations from the systematic parameters of some rays, it has a substantial positive impact on the estimated extrinsic parameters of the LiDAR, leading to improved mounting accuracy. This improvement highlights the critical role of intrinsic calibration in achieving reliable extrinsic calibration for multi-sensor systems. To ensure robust extrinsic calibration, we decouple the intrinsic and extrinsic calibration processes, thereby providing rectified input data for extrinsic calibration. Furthermore, we present an extrinsic calibration approach for a multi-sensor system comprising an inertial measurement unit, two cameras, and a LiDAR, where the cameras and LiDAR share only a narrow overlapping field of view, resulting in minimal shared information. By employing a high-accuracy reference instrument, we address the limited overlap issue and validate both the extrinsic calibration results and the impact of the LiDAR’s intrinsic calibration. Our findings demonstrate that untreated LiDAR intrinsic errors lead to significant deviations in the extrinsic calibration parameters, underscoring that intrinsic calibration is not merely a preliminary step but a vital factor in ensuring accurate extrinsic calibration in multi-sensor systems.
Funding source: Deutsche Forschungsgemeinschaft
Award Identifier / Grant number: GRK 2159
Award Identifier / Grant number: HE1822/37-1
Award Identifier / Grant number: NE1453/5-1
Acknowledgments
We would like to express our gratitude to Prof. Dr. Stephan Schulz and his team at the Department of Mechanical Engineering and Production Management, HAW Hamburg, for their collaboration in developing the platform. Last but not least, we extend our thanks to Johannes Link and Dmitri Diener for their valuable hardware support and time synchronization of the MSS.
-
Research ethics: Not applicable.
-
Informed consent: Not applicable.
-
Author contributions: AK, SV, IN; Experiments, Coding, Processing, Writing: AK; Proof-Reading: SV, IN; Project Supervision: SV, IN. The authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Use of Large Language Models, AI and Machine Learning Tools: None declared.
-
Conflict of interest: The authors state no conflict of interest.
-
Research funding: This work was funded by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) HE1822/37-1 for the Institute of Photogrammetry and GeoInformation and NE1453/5-1 for Geodetic Institute Hannover. This work is also partially supported by Graduiertenakademie (GRK 2159).
-
Data availability: The raw data can be obtained on request from the corresponding author.
References
1. Shaukat, A, Blacker, PC, Spiteri, C, Gao, Y. Towards camera-lidar fusion-based terrain modelling for planetary surfaces: review and analysis. Sensors 2016;16:1952. https://doi.org/10.3390/s16111952.Search in Google Scholar PubMed PubMed Central
2. Zhang, Q, Pless, R. Extrinsic calibration of a camera and laser range finder (improves camera calibration). In: 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE cat. no. 04CH37566); 2004, vol 3:2301–6 pp.10.1109/IROS.2004.1389752Search in Google Scholar
3. Douillard, B, Brooks, A, Ramos, F. A 3d laser and vision based classifier. In: 2009 international conference on intelligent sensors, sensor networks and information processing (ISSNIP); 2009:295–300 pp.10.1109/ISSNIP.2009.5416828Search in Google Scholar
4. Pang, S, Morris, D, Radha, H. Clocs: camera-lidar object candidates fusion for 3d object detection. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS); 2020:10386–93 pp.10.1109/IROS45743.2020.9341791Search in Google Scholar
5. Zhuang, Z, Li, R, Jia, K, Wang, Q, Li, Y, Tan, M. Perception-aware multi-sensor fusion for 3d lidar semantic segmentation. In: 2021 IEEE/CVF international conference on computer vision (ICCV); 2021:16260–70 pp.10.1109/ICCV48922.2021.01597Search in Google Scholar
6. Hartley, R, Zisserman, A. Multiple view geometry in computer vision. United States of America, New York: Cambridge University Press; 2004.10.1017/CBO9780511811685Search in Google Scholar
7. Foerstner, W, Wrobe, LBP. Photogrammetric computer vision: statistics, geometry, orientation and reconstruction. Switzerland, Cham: Springer International Publishing AG; 2018.Search in Google Scholar
8. Mirzaei, FM, Roumeliotis, SI. A Kalman filter-based algorithm for imu-camera calibration: observability analysis and performance evaluation. IEEE Trans Robot 2008;24:1143–56. https://doi.org/10.1109/tro.2008.2004486.Search in Google Scholar
9. Mikhail, EM, Ackermann, FE. Observations and least squares. United States of America, Washington, D.C: University Press of America; 1982.Search in Google Scholar
10. Glennie, CL, Kusari, A, Facchin, A. Calibration and stability analysis of the vlp-16 laser scanner. Int Arch Photogram Remote Sens Spatial Inf Sci 2016;XL-3/W4:55–60. https://doi.org/10.5194/isprsarchives-xl-3-w4-55-2016.Search in Google Scholar
11. Chan, T, Lichti, D, Roesler, G, Cosandier, D, Durgham, K. Range scale-factor calibration of the velodyne vlp-16 lidar system for position tracking applications. In: Proceedings of the 11th International Conference on Mobile Mapping. Shenzhen, China; 2020:6–8 pp.Search in Google Scholar
12. Schulz, T. Calibration of a terrestrial laser scanner for engineering geodesy [Ph.D. thesis]. ETH Zurich; 2008.Search in Google Scholar
13. Holst, C, Neuner, H, Wieser, A, Wunderlich, T, Kuhlmann, H. Calibration of terrestrial laser scanners. Allg Vermess Nachr (AVN) 2016;123:147–57.Search in Google Scholar
14. Lerma, JL, García-San-Miguel, D. Self-calibration of terrestrial laser scanners: selection of the best geometric additional parameters. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 2014;II-5:219–26. https://doi.org/10.5194/isprsannals-ii-5-219-2014.Search in Google Scholar
15. Medić, T. Efficient calibration strategies for panoramic terrestrial laser scanners [Ph.D. thesis]. Rheinische Friedrich-Wilhelms-Universität Bonn; 2021.Search in Google Scholar
16. Filin, S. Recovery of systematic biases in laser altimetry data using natural surfaces. Photogramm Eng Remote Sens 2003;69:1235–42. https://doi.org/10.14358/pers.69.11.1235.Search in Google Scholar
17. Skaloud, J, Lichti, D. Rigorous approach to bore-sight self-calibration in airborne laser scanning. ISPRS J Photogrammetry Remote Sens 2006;61:47–59. https://doi.org/10.1016/j.isprsjprs.2006.07.003.Search in Google Scholar
18. Glennie, C, Lichti, DD. Static calibration and analysis of the velodyne hdl-64e s2 for high accuracy mobile scanning. Remote Sens 2010;2:1610–24. https://doi.org/10.3390/rs2061610.Search in Google Scholar
19. Taylor, Z, Nieto, J. A mutual information approach to automatic calibration of camera and lidar in natural environments. In: Proceedings of Australasian Conference on Robotics and Automation. New Zealand, Wellington; 2012.Search in Google Scholar
20. Pandey, G, McBride, J, Savarese, S, Eustice, R. Automatic extrinsic calibration of vision and lidar by maximizing mutual information. J Field Robot 2014;32:696–722. https://doi.org/10.1002/rob.21542.Search in Google Scholar
21. Scaramuzza, D, Harati, A, Siegwart, R. Extrinsic self calibration of a camera and a 3d laser range finder from natural scenes. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. United States of America, San Diego; 2007:4164–9 pp.10.1109/IROS.2007.4399276Search in Google Scholar
22. Bileschi, S. Fully automatic calibration of lidar and video streams from a vehicle. In: 2009 IEEE 12th international conference on computer vision workshops, ICCV workshops; 2009:1457–64 pp.10.1109/ICCVW.2009.5457439Search in Google Scholar
23. Levinson, J, Thrun, S. Automatic online calibration of cameras and lasers. In: Robotics: science and systems; 2013.10.15607/RSS.2013.IX.029Search in Google Scholar
24. Moghadam, P, Bosse, M, Zlot, R. Line-based extrinsic calibration of range and image sensors. In: 2013 IEEE international conference on robotics and automation; 2013:3685–91 pp.10.1109/ICRA.2013.6631095Search in Google Scholar
25. Ravi, R, Lin, Y-J, Elbahnasawy, M, Shamseldin, T, Habib, A. Simultaneous system calibration of a multi-lidar multicamera mobile mapping platform. IEEE J Sel Top Appl Earth Obs Rem Sens 2018;11:1694–714. https://doi.org/10.1109/jstars.2018.2812796.Search in Google Scholar
26. Huang, L, Barth, M. A novel multi-planar lidar and computer vision calibration procedure using 2d patterns for automated navigation. In: 2009 IEEE intelligent vehicles symposium; 2009:117–22 pp.10.1109/IVS.2009.5164263Search in Google Scholar
27. Li, G, Liu, Y, Dong, L, Cai, X, Zhou, D. An algorithm for extrinsic parameters calibration of a camera and a laser range finder using line features. In: 2007 IEEE/RSJ international conference on intelligent robots and systems; 2007:3854–9 pp.10.1109/IROS.2007.4399041Search in Google Scholar
28. Aliakbarpour, H, Nunez, P, Prado, J, Khoshhal, K, Dias, J. An efficient algorithm for extrinsic calibration between a 3d laser range finder and a stereo camera for surveillance. In: 2009 international conference on advanced robotics; 2009:1–6 pp.Search in Google Scholar
29. Gong, X, Lin, Y, Liu, J. 3d lidar-camera extrinsic calibration using an arbitrary trihedron. Sensors (Basel, Switzerland) 2013;13:1902–18. https://doi.org/10.3390/s130201902.Search in Google Scholar PubMed PubMed Central
30. Strübing, T, Neumann, I. Positions-und orientierungsschätzung von lidar-sensoren auf multisensorplattformen. Z Geod Geoinf Landmanag 2013;138:210–21.Search in Google Scholar
31. Vogel, S, Ernst, D, Neumann, I, Alkhatib, H. Recursive gauss-helmert model with equality constraints applied to the efficient system calibration of a 3d laser scanner. J Appl Geodesy 2022;16:37–57. https://doi.org/10.1515/jag-2021-0026.Search in Google Scholar
32. Heinz, E, Eling, C, Wieland, M, Klingbeil, L, Kuhlmann, H. Analysis of different reference plane setups for the calibration of a mobile laser scanning system. In: Lienhart W. (Hrsg.): Ingenieurvermessung 17, Beiträge zum 18. Internationalen Ingenieurvermessungskurs. Austria, Graz; 2017:131–46 pp.Search in Google Scholar
33. Hartmann, J, von Gösseln, I, Schild, N-M, Dorndorf, A, Paffenholz, J-A, Neumann, I. Optimisation of the calibration process of a k-tls based multi-sensor-system by genetic algorithms. In: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. The Netherlands, Enschede; 2019, XLII-2/W13:1655–62 pp.10.5194/isprs-archives-XLII-2-W13-1655-2019Search in Google Scholar
34. Heinz, E, Holst, C, Kuhlmann, H, Klingbeil, L. Design and evaluation of a permanently installed plane-based calibration field for mobile laser scanning systems. Remote Sens 2020;12:555. https://doi.org/10.3390/rs12030555.Search in Google Scholar
35. Lenzmann, L, Lenzmann, E. Zur lösung des nichtlinearen gauss-markov-modells. Z Geod Geoinf Landmanag 2007;132:108–20.Search in Google Scholar
36. Neitzel, F, Petrovic, S. Total least squares (tls) im kontext der ausgleichung nach kleinsten quadranten am beispiel der ausgleichenden geraden. ZfV: Z Geod Geoinf Landmanag 2008;133:141–8.Search in Google Scholar
37. Ghilani, CD. Adjustment computations: spatial data analysis. United States of America, New Jersey: John Wiley & Sons, Ltd; 2018.10.1002/9781119390664Search in Google Scholar
38. Drixler, E. Analyse der Form und Lage von Objekten im Raum [Ph.D. thesis]; 1993. Muenchen 1993. (Deutsche Geodaetische Kommission bei der Bayerischen Akademie der Wissenschaften. Reihe C, Nr. 409.) Fak. f. Bauingenieur- und Vermessungswesen, Diss. v. 18.6.1993.Search in Google Scholar
39. Autodesk civil 3D; 2022. Version: 2022. https://www.autodesk.com/de/products/civil-3d/overview [Accessed 12 May 2025].Search in Google Scholar
40. Mohammadi, M, Khami, A, Rottensteiner, F, Neumann, I, Heipke, C. Mounting calibration of a multi-view camera system on a uav platform. ISPRS Ann Photogramm Remote Sens Spat Inf Sci 2021;V-1-2021:97–104. https://doi.org/10.5194/isprs-annals-v-1-2021-97-2021.Search in Google Scholar
41. Liu, J. Extrinsische kalibrierung einer inertialen messeinheit für ein multi-sensor system [unpublished] [Master’s thesis]. LUH: Leibniz University Hannover Geodetic Institute; 2021.Search in Google Scholar
42. Papcova, M, Neuner, H-B. Towards the influence of the angle of incidence and the surface roughness on distances in terrestrial laser scanning. In: FIG working week 2017. FIG; 2017.Search in Google Scholar
43. Hartmann, J, Heiken, M, Alkhatib, H, Neumann, I. Automatic quality assessment of terrestrial laser scans. J Appl Geodesy 2023;17:333–53. https://doi.org/10.1515/jag-2022-0030.Search in Google Scholar
44. Reshetyuk, Y. Investigation of the influence of surface reflectance on the measurements with the terrestrial laser scanner leica hds 3000. ZfV: Zeitschrift fur Geodasie, Geoinformation und Landmanagement 2006;2:96–103.Search in Google Scholar
45. Velodyne LiDAR. VLP-16; 2022. https://velodyneLiDAR.com [Accessed 11 May 2022].Search in Google Scholar
46. Ernst, D, Alkhatib, H, Neumann, I, Vogel, S. Analysis of multiple positions for the intrinsic and extrinsic calibration of a multi-beam lidar. In: 2022 25th international conference on information fusion (FUSION); 2022:01–8 pp.10.23919/FUSION49751.2022.9841366Search in Google Scholar
47. Leica, Geosystems. Leica nova ts60; 2022. https://leica-geosystems.com/de-de/products/total-stations/robotic-total-stations/leica-nova-ts60 [Accessed 11 May 2025].Search in Google Scholar
48. Amiri-Simkooei, AR, Jazaeri, S. Data-snooping procedure applied to errors-in-variables models. Studia Geophys Geod 2013;57:426–41. https://doi.org/10.1007/s11200-012-0474-2.Search in Google Scholar
49. Hexagon. Leica absolute tracker at960; 2022. https://hexagon.com/de/products/leica-absolute-tracker-at960 [Accessed 11 May 2025].Search in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
