Home Data fusion for automated driving: An introduction
Article
Licensed
Unlicensed Requires Authentication

Data fusion for automated driving: An introduction

  • Matthias Schreier

    Dr.-Ing. Matthias Schreier is Senior Expert and Team Lead in the area of comprehensive vehicle environment model fusion for automated driving at Continental and responsible for the topics traffic participant fusion, free space fusion, and road model fusion for Continental’s worldwide automated driving prototyping fleet. Additionally, he is a lecturer at TU Darmstadt (Lecture “Automated Driving”) and a recipient of the IEEE ITSS Best Ph. D. Dissertation Award 2016 (#2) as well as the Best Dissertation Award 2016 of the Department of Electrical Engineering and Information Technology, TU Darmstadt.

    EMAIL logo
Published/Copyright: March 11, 2022

Abstract

Data fusion is one of the key ingredients of Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS). The article provides an introduction into the field by highlighting different possible data fusion categorizations and architectures alongside pros and cons. Benefits, limitations, and pitfalls of exemplary data fusion systems are presented, and the consequences of specific design choices discussed.

Zusammenfassung

Datenfusion ist ein Schlüsselbestandteil automatisierter Fahrzeuge und Fahrerassistenzsysteme. Der Artikel liefert eine Einführung in das Themenfeld, indem unterschiedliche Datenfusionskategorisierungen und -architekturen sowie deren Vor- und Nachteile beleuchtet werden. Der Nutzen, aber auch die Limitierungen und Fallstricke beispielhafter Datenfusionssysteme werden dargestellt und Konsequenzen spezifischer Designentscheidungen diskutiert.


Dedicated to the 60th birthday of Prof. Dr.-Ing. Jürgen Adamy.


About the author

Matthias Schreier

Dr.-Ing. Matthias Schreier is Senior Expert and Team Lead in the area of comprehensive vehicle environment model fusion for automated driving at Continental and responsible for the topics traffic participant fusion, free space fusion, and road model fusion for Continental’s worldwide automated driving prototyping fleet. Additionally, he is a lecturer at TU Darmstadt (Lecture “Automated Driving”) and a recipient of the IEEE ITSS Best Ph. D. Dissertation Award 2016 (#2) as well as the Best Dissertation Award 2016 of the Department of Electrical Engineering and Information Technology, TU Darmstadt.

References

1. Bar-Shalom, Y., X.-R. Li and T. Kirubarajan. 2001. Estimation with Applications to Tracking and Navigation. Theory, Algorithms and Software. Wiley-Interscience, New York City, NY, USA.10.1002/0471221279Search in Google Scholar

2. Barnard, M. Tesla & Google Disagree About LIDAR – Which Is Right? CleanTechnica, 2016.Search in Google Scholar

3. Blackman, S. and R. Popoli. 1999. Design and Analysis of Modern Tracking Systems. Artech House, Boston, MA, USA.Search in Google Scholar

4. Castanedo, F. 2013. A Review of Data Fusion Techniques. The Scientific World Journal 2013: 1–19.10.1155/2013/704504Search in Google Scholar PubMed PubMed Central

5. Challa, S., M. R. Morelande, D. Musǐcki and R. J. Evans. 2011. Fundamentals of Object Tracking. Cambridge University Press, Cambridge, England.10.1017/CBO9780511975837Search in Google Scholar

6. Darms, M. 2016. Data Fusion of Environment-Perception Sensors for ADAS. In: Handbook of Driver Assistance Systems. Basic Information, Components and Systems for Active Safety and Comfort, Chapter 23. Springer, Switzerland, pp. 549–566.10.1007/978-3-319-12352-3_24Search in Google Scholar

7. Dasarathy, B. V. 1997. Sensor Fusion Potential Exploitation – Innovative Architectures and Illustrative Applications. Proc. of the IEEE 85(1): 24–38.10.1109/5.554206Search in Google Scholar

8. Dietmayer, K., A. Kirchner and N. Kämpchen. 2005. Fusionsarchitekturen zur Umfeldwahrnehmung für zukünftige Fahrerassistenzsysteme. In: Fahrerassistenzsysteme mit maschineller Wahrnehmung, Chapter 4. Springer, Berlin, Germany, pp. 59–88.10.1007/3-540-27137-6_4Search in Google Scholar

9. Durrant-Whyte, H. and T. C. Henderson. 2016. Multisensor Data Fusion. In: Handbook of Robotics, Chapter 35. Springer, pp. 867–892.10.1007/978-3-319-32552-1_35Search in Google Scholar

10. Durrant-Whyte, H. F. 1988. Sensor Models and Multisensor Integration. The International Journal of Robotics Research 7(6): 97–113.10.1007/978-1-4613-8997-2_7Search in Google Scholar

11. Endsley, M. R. Mar. 1995. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors 37(1): 32–64.10.1518/001872095779049543Search in Google Scholar

12. Esteban, J., A. Starr, R. Willetts, P. Hannah and P. Bryanston-Cross. 2005. A Review of Data Fusion Models and Architectures: Towards Engineering Guidelines. Neural Computing & Applications 14(4): 273–281.10.1007/s00521-004-0463-7Search in Google Scholar

13. Feng, D., C. Haase-Schütz, L. Rosenbaum, H. Hertzlein, C. Gläser, F. Timm, W. Wiesbeck and K. Dietmayer. 2021. Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges. IEEE Transactions on Intelligent Transportation Systems 22(3): 1341–1360.10.1109/TITS.2020.2972974Search in Google Scholar

14. Fowler, C. A. 1979. Comments on the Cost and Performance of Military Systems. IEEE Transactions on Aerospace and Electronic Systems 15(1): 2–10.10.1109/TAES.1979.308790Search in Google Scholar

15. Hall, D. L. and A. K. Garga. 1999. Pitfalls in Data Fusion (and How to Avoid Them). In: Proc. of the International Conference on Information Fusion. Sunnyvale, CA, USA, pp. 429–436.Search in Google Scholar

16. Jøsang, A. and S. Pope. Nov. 2012. Dempster’s Rule as Seen by Little Colored Balls. Computational Intelligence 28(4): 453–474.10.1111/j.1467-8640.2012.00421.xSearch in Google Scholar

17. Kálmán, R. E. 1960. A New Approach to Linear Filtering and Prediction Problems. Transactions of the ASME–Journal of Basic Engineering 82: 35–45.10.1115/1.3662552Search in Google Scholar

18. Khaleghi, B., A. Khamis, F. O. Karay and S. N. Razavi. 2013. Multisensor Data Fusion: A Review of the State-Of-The-Art. Information Fusion 14(1): 28–44.10.1016/j.inffus.2011.08.001Search in Google Scholar

19. Kunz, P. and M. Schreier. June 2017. Automated Detection of Construction Sites on Motorways. In: Proc. of the IEEE Intelligent Vehicles Symposium. Redondo Beach, CA, USA, pp. 1378–1385.10.1109/IVS.2017.7995903Search in Google Scholar

20. Lefèvre, S., D. Vasquez and C. Laugier. July 2014. A Survey on Motion Prediction and Risk Assessment for Intelligent Vehicles. ROBOMECH Journal 1(1): 1–14.10.1186/s40648-014-0001-zSearch in Google Scholar

21. Liggins, M. E., C.-Y. Chong, I. Kadar, M. G. Alford, V. Vannicola and S. Thomopoulos. 1997. Distributed Fusion Architectures and Algorithms for Target Tracking. Proc. of the IEEE 85(1): 95–107.10.1109/JPROC.1997.554211Search in Google Scholar

22. Luthardt, S., C. Han, V. Willert and M. Schreier. June 2017. Efficient Graph-Based V2V Free Space Fusion. In: Proc. of the IEEE Intelligent Vehicles Symposium. Redondo Beach, CA, USA, pp. 985–992.10.1109/IVS.2017.7995843Search in Google Scholar

23. Mozzaffari, S., O. Y. Al-Jarrah, M. Dianati, P. Jennings and A. Mouzakitis. 2020. Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review. IEEE Transactions on Intelligent Transportation Systems.10.1109/TITS.2020.3012034Search in Google Scholar

24. Nuss, D., S. Reuter, M. Thom, T. Yuan, G. Krehl, M. Maile, A. Gern and K. Dietmayer. 2018. A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time Application. The International Journal of Robotics Research 37(8): 841–866.10.1177/0278364918775523Search in Google Scholar

25. Ruser, H. 2007. Informationsfusion – Eine Übersicht. Technisches Messen 74(3): 93–102.10.1524/teme.2007.74.3.93Search in Google Scholar

26. Schreier, M. Dec. 2015. Bayesian Environment Representation, Prediction, and Criticality Assessment for Driver Assistance Systems. PhD thesis, Technische Universität Darmstadt, Darmstadt, Germany.Search in Google Scholar

27. Schreier, M. 2018. Environment Representations for Automated On-Road Vehicles. at – Automatisierungstechnik 66(2): 107–118.10.1515/auto-2017-0104Search in Google Scholar

28. Schreier, M. and R. Grewe. June 2017. A High-Level Road Model Information Fusion Framework and its Application to Multi-Lane Speed Limit Inference. In: Proc. of the IEEE Intelligent Vehicles Symposium. Redondo Beach, CA, USA, pp. 1201–1208.10.1109/IVS.2017.7995876Search in Google Scholar

29. Schreier, M., V. Willert and J. Adamy. Oct. 2016. An Integrated Approach to Maneuver-Based Trajectory Prediction and Criticality Assessment in Arbitrary Road Environments. IEEE Transactions on Intelligent Transportation Systems 17(10): 2751–2766.10.1109/TITS.2016.2522507Search in Google Scholar

30. Schreier, M., V. Willert and J. Adamy. Feb. 2016. Compact Representation of Dynamic Driving Environments for ADAS by Parametric Free Space and Dynamic Object Maps. IEEE Transactions on Intelligent Transportation Systems 17(2): 367–384.10.1109/TITS.2015.2472965Search in Google Scholar

31. Smith, D. and S. Singh. 2006. Approaches to Multisensor Data Fusion in Target Tracking: A Survey. IEEE Transactions on Knowledge and Data Engineering 18(2): 1696–1710.10.1109/TKDE.2006.183Search in Google Scholar

32. Steinberg, A. N. and C. L. Bowman. Revisions to the JDL Data Fusion Model. 2001. In: Handbook of Multisensor Data Fusion, Chapter 2. CRC Press, Boca Raton, FL, USA.10.1201/9781420038545.ch2Search in Google Scholar

33. Stiller, C., F. P. León and M. Kruse. 2011. Information Fusion for Automotive Applications – An Overview. Information Fusion 12(4): 244–252.10.1016/j.inffus.2011.03.005Search in Google Scholar

34. Tanzmeister, G. and D. Wollherr. June 2016. Evidential Grid-Based Tracking and Mapping. IEEE Transactions on Intelligent Transportation Systems 18(6): 1454–1467.10.1109/TITS.2016.2608919Search in Google Scholar

35. Unknown. 1916. The Moving Picture World 29(10): 1940.Search in Google Scholar

36. Whang, Z., Y. Wu and Q. Niu. 2019. Multi-Sensor Fusion in Automated Driving: A Survey. IEEE Access 8: 2847–2868.10.1109/ACCESS.2019.2962554Search in Google Scholar

37. Yeong, D. J., G. Velasco-Hernandez, J. Barry and J. Walsh. 2021. Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review. Sensors 21(6): 1–37.10.3390/s21062140Search in Google Scholar PubMed PubMed Central

Received: 2021-09-16
Accepted: 2022-02-02
Published Online: 2022-03-11
Published in Print: 2022-03-28

© 2022 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 25.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/auto-2021-0132/html
Scroll to top button