Automotive mass production of camera systems: Linking image quality to AI performance
-
Alexander Braun
Alexander Braun received his diploma in physics with a focus on laser fluorescence spectroscopy from the University of Göttingen in 2001. His PhD research in quantum optics and quantum computers was carried out at the University of Hamburg, resulting in a doctorate from the University of Siegen in 2007. He started working as an optical designer for camera-based ADAS with the company Kostal, and later became responsible for the optical quality of the series mass production. Next, he became a professor of physics at the University of Applied Sciences Düsseldorf in 2013, where he now researches optical metrology and optical models for simulation in the context of autonomous driving. He is a member of DPG, SPIE, IS&T and VDI, participating in norming efforts at IEEE (P2020) and VDI (FA 8.13), and currently serves on the advisory board for the AutoSens conference and for the VDI Optical Technologies (Fachbeirat 8).
Abstract
Artificial intelligence methods based on machine learning or artificial neural networks have become indispensable in camera-based driver assistance systems, and also represent an essential building block for future autonomous driving. However, the great successes of these evaluation methods in environment perception and also driving planning are accompanied by equally great challenges in the validation and verification of these systems. One of the essential aspects for this is the required guaranteed safety of the functions under mass production conditions of the vehicles. This article explains this point of view using a detailed example from the field of camera-based driver assistance systems: the determination of inspection limits at the end of the production line. The camera is one of the most important sensor modalities for vehicle environment sensing and as such, the quality of the camera systems plays a key role in the safety argumentation of the overall system. Several illustrative application examples (role of simulations, calibration, influence of the windshield) will be presented. The basic ideas presented can be well transferred to the other sensor modalities (lidar, radar, ToF, etc.). The investigations/evidence show that doubts are allowed whether or how fast autonomous driving on level L4/5 will take hold as robotaxis or – even more challenging – in private ownership on a larger scale.
Zusammenfassung
Methoden der künstlichen Intelligenz auf Basis von Machine Learning oder künstlichen neuronalen Netzen sind aus Kamera-basierten Fahrerassistenzsystemen nicht mehr wegzudenken, und stellen auch für das zukünftige autonome Fahren einen wesentlichen Baustein dar. Die großen Erfolge dieser Auswertemethoden in der Umfeldwahrnehmung und auch der Fahrplanung gehen jedoch mit ebenso großen Herausforderungen bei der Validierung und Verifikation dieser Systeme einher. Einer der wesentlichen Aspekte hierfür ist die benötigte garantierte Sicherheit der Funktionen unter Massenproduktionsbedingungen der Fahrzeuge. Dieser Artikel erläutert diese Sichtweise an Hand eines detaillierten Beispiels aus dem Bereich der Kamera-basierten Fahrerassistenzsysteme: der Bestimmung der Prüfgrenzen am Bandende der Produktion. Die Kamera ist eine der wichtigsten Sensormodalitäten für die Fahrzeug-Umfelderfassung und als solches spielt die Qualität der Kamerasysteme für die Sicherheitsargumentation des Gesamtsystems eine zentrale Rolle. Die Darstellung wird mit mehreren Anwendungsbeispielen erläutert (Rolle von Simulationen, Kalibrierung, Einfluss der Windschutzscheibe). Die vorgestellten Grundgedanken lassen sich gut auf die anderen Sensormodalitäten (Lidar, Radar, ToF etc.) übertragen. Die Untersuchungen/Erkenntnisse zeigen, dass Zweifel erlaubt sind, ob bzw. wie schnell autonomes Fahren auf Niveau L4/5 als Robotaxis oder – noch anspruchsvoller – im Privatbesitz in größerem Umfang Einzug halten wird.
About the author
Alexander Braun received his diploma in physics with a focus on laser fluorescence spectroscopy from the University of Göttingen in 2001. His PhD research in quantum optics and quantum computers was carried out at the University of Hamburg, resulting in a doctorate from the University of Siegen in 2007. He started working as an optical designer for camera-based ADAS with the company Kostal, and later became responsible for the optical quality of the series mass production. Next, he became a professor of physics at the University of Applied Sciences Düsseldorf in 2013, where he now researches optical metrology and optical models for simulation in the context of autonomous driving. He is a member of DPG, SPIE, IS&T and VDI, participating in norming efforts at IEEE (P2020) and VDI (FA 8.13), and currently serves on the advisory board for the AutoSens conference and for the VDI Optical Technologies (Fachbeirat 8).
References
1. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. http://www.deeplearningbook.org. MIT Press, 2016.Suche in Google Scholar
2. M. Ruhdorfer. ADAC Autotest. URL: https://www.adac.de/_ext/itr/tests/Autotest/AT6104_VW_ID_4_Pro_Performance_77_kWh_Max/VW_ID_4_Pro_Performance_77_kWh_Max.pdf (visited on 02/04/2022).Suche in Google Scholar
3. Tesla Inc. Autopilot and Full Self-Driving Capability. Feb. 13, 2019. URL: https://www.tesla.com/support/autopilot (visited on 02/14/2022).Suche in Google Scholar
4. Valeo Press Release. Valeo has produced its 100 millionth camera at its Tuam site in Ireland. Apr. 2022. URL: https://www.valeo.com/en/valeo-has-produced-its-100-millionth-camera-at-its-tuam-site-in-ireland/ (visited on 04/29/2022).Suche in Google Scholar
5. Sebastian Bach et al. “On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation”. In: PLoS ONE 10.7 (July 2015). URL: http://dx.doi.org/10.1371y%2Fjournal.pone.0130140.10.1371/journal.pone.0130140Suche in Google Scholar PubMed PubMed Central
6. Amina Adadi and Mohammed Berrada. “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)”. In: IEEE Access 6 (2018), pp. 52138–52160.10.1109/ACCESS.2018.2870052Suche in Google Scholar
7. Europäische Kommission and Generaldirektion Mobilität und Verkehr. Nächste Schritte auf dem Weg zur „Vision Null Straßenverkehrstote“: EU-Politikrahmen für die Strassenverkehrssicherheit im Zeitraum 2021 bis 2030. Publications Office, 2020.Suche in Google Scholar
8. VDA. VDA 6.3 Qualitätsmangement in der Automobil-Industrie. Dec. 2016.Suche in Google Scholar
9. Beuth. Quantitative methods in process improvement – Six Sigma – Part 1: DMAIC methodology. Berlin, 2011-09.Suche in Google Scholar
10. Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. “One Pixel Attack for Fooling Deep Neural Networks”. In: IEEE Transactions on Evolutionary Computation 23.5 (2019), pp. 828–841.10.1109/TEVC.2019.2890858Suche in Google Scholar
11. Danilo Vasconcellos Vargas and Jiawei Su. “Understanding the one-pixel attack: Propagation maps and locality analysis”. In: CEUR Workshop Proceedings 2640 (2020).Suche in Google Scholar
12. Ian Goodfellow et al. Attacking Machine Learning with Adversarial Examples. 2017. URL: https://openai.com/blog/adversarial-example-research/ (visited on 02/24/2017).Suche in Google Scholar
13. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. “Explaining and Harnessing Adversarial Examples”. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings. Ed. by Yoshua Bengio and Yann LeCun. 2015. URL: http://arxiv.org/abs/1412.6572.Suche in Google Scholar
14. Steve Povolny. Model Hacking ADAS to Pave Safer Roads for Autonomous Vehicles. 2020. URL: https://www.mcafee.com/blogs/other-blogs/mcafee-labs/model-hacking-adas-to-pave-safer-roads-for-autonomous-vehicles/.Suche in Google Scholar
15. Anurag Ranjan et al. “Attacking Optical Flow”. In: Proceedings International Conference on Computer Vision (ICCV). ISSN: 2380-7504. IEEE, Nov. 2019, pp. 2404–2413. URL: https://flowattack.is.tue.mpg.de/.10.1109/ICCV.2019.00249Suche in Google Scholar
16. Zachary Pezzementi et al. “Putting Image Manipulations in Context: Robustness Testing for Safe Perception”. In: 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). 2018, pp. 1–8.10.1109/SSRR.2018.8468619Suche in Google Scholar
17. Philip Koopman, Aaron Kane, and Jen Black. “Credible Autonomy Safety Argumentation”. In: Proceedings of the Safety-Critical Systems Symposium (2019), p. 27.Suche in Google Scholar
18. Philip Koopman. “The Heavy Tail Safety Ceiling”. In: Automated and Connected Vehicle Systems Testing Symposium. SAE. 2018.Suche in Google Scholar
19. Philip Koopman. Edge Cases and Autonomous Vehicle Safety. Lecture notes Bristol SSS 2019. Feb. 2019.Suche in Google Scholar
20. Martin Kords. Anzahl der Getöteten bei Straßenverkehrsunfällen in ausgewählten Ländern im Jahr 2018. 2020. URL: https://de.statista.com/statistik/daten/studie/163411/umfrage/unfallraten-auf-autobahnen-nach-ausgewaehlten-laendern/.Suche in Google Scholar
21. Nidhi Kalra and Susan M. Paddock. Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? Santa Monica, CA: RAND Corporation, 2016.10.7249/RR1478Suche in Google Scholar
22. IEEE Standards Association P2020 | Automotive Image Quality Working Group. URL: https://site.ieee.org/sagroups-2020/ (visited on 01/31/2022).Suche in Google Scholar
23. The Waymo Team. Introducing the Waymo Driver to New York City’s streets. Nov. 2021. URL: https://blog.waymo.com/2021/11/introducing-waymo-driver-to-new-york.html (visited on 11/03/2021).Suche in Google Scholar
24. Martin Kords. Statistiken zum Taxigewerbe. Jan. 2022. URL: https://de.statista.com/themen/2563/taxigewerbe/ (visited on 01/19/2022).Suche in Google Scholar
25. DEWESoft. The List of Automotive Proving Grounds for Vehicle Testing. Jan. 2021. URL: https://dewesoft.com/daq/list-of-automotive-proving-grounds (visited on 01/15/2021).Suche in Google Scholar
26. Satish Jeyachandran. Introducing the 5th-generation Waymo Driver: Informed by experience, designed for scale, engineered to tackle more environments. Mar. 2020. URL: https://blog.waymo.com/2020/03/introducing-5th-generation-waymo-driver.html (visited on 03/04/2020).Suche in Google Scholar
27. Insurance Institute for Highway Safety. Pedestrian crash avoidance systems cut crashes – but not in the dark. Feb. 2022. URL: https://www.iihs.org/news/detail/pedestrian-crash-avoidance-systems-cut-crashes–but-not-in-the-dark (visited on 02/03/2022).Suche in Google Scholar
28. Sanjay Gangadhara and Ludovic Manillier. Collaborate on Seamless Optics Design for ADAS. Jan. 2021. URL: https://www.ansys.com/de-de/advantage-magazine/volume-xv-issue-1-2021/collaborate-seamless-optics-design-adas.Suche in Google Scholar
29. Alexey Dosovitskiy. “CARLA: An Open Urban Driving Simulator”. In: Proceedings of 1st Conference on Robot Learning (CoRL 2017), Mountain View, United States (2017), p. 16.Suche in Google Scholar
30. Epic Games. Scene Fringe (Chromatic Aberration). URL: https://docs.unrealengine.com/4.27/en-US/RenderingAndGraphics/PostProcessEffects/SceneFringe/.Suche in Google Scholar
31. He Huang, Philip S. Yu, and Changhu Wang. An Introduction to Image Synthesis with Generative Adversarial Nets. 2018. arXiv: 1803.04469 [cs.CV].Suche in Google Scholar
32. Matthias Lehmann et al. “Resolution and accuracy of nonlinear regression of point spread function with artificial neural networks”. In: Optical Engineering 58.4 (2019), pp. 1–9.10.1117/1.OE.58.4.045101Suche in Google Scholar
33. Matthias Lehmann et al. “Modeling realistic optical aberrations to reuse existing drive scene recordings for autonomous driving validation”. In: Journal of Electronic Imaging 28.1 (2019), pp. 1–9.10.1117/1.JEI.28.1.013005Suche in Google Scholar
34. Christian Krebs, Patrick Müller, and Alexander Braun. “Impact of Windshield Optical Aberrations on Visual Range Camera Based Classification Tasks Performed by CNNs”. In: London Image Meeting. Society for Imaging Science and Technology, 2021.10.2352/issn.2694-118X.2021.LIM-83Suche in Google Scholar
35. Ethan Tseng et al. “Differentiable Compound Optics and Processing Pipeline Optimization for End-to-End Camera Design”. In: ACM Trans. Graph. 40.2 (June 2021).10.1145/3446791Suche in Google Scholar
36. Patrick Mueller, Matthias Lehmann, and Alexander Braun. “Simulating tests to test simulation”. In: International Symposium on Electronic Imaging 2020. Society for Imaging Science and Technology, 2020.Suche in Google Scholar
37. Patrick Müller, Mattis Brummel, and Alexander Braun. “Spatial Recall Index for Machine Learning Algorithms”. In: London Image Meeting. Society for Imaging Science and Technology, 2021.10.2352/issn.2694-118X.2021.LIM-58Suche in Google Scholar
38. Mattis Brummel, Patrick Müller, and Alexander Braun. “Spatial Precision and Recall Indices to assess the performance of Instance Segmentation algorithms”. In: Submitted for publication at Electronic Imaging 2022, IS&T, 2022.Suche in Google Scholar
39. Marius Dupuis et al. Feb. 2022. URL: https://www.simcert.org.Suche in Google Scholar
40. Sven Fleck et al. “Data driven degradation of automotive sensors and effect analysis”. In: Electronic Imaging 2021.17 (2021), pp. 180–1–180–8. URL: https://www.ingentaconnect.com/content/ist/ei/2021/00002021/00000017/art00010.10.2352/ISSN.2470-1173.2021.17.AVM-180Suche in Google Scholar
41. AAA Automotive. ADAS Sensor Calibration Increases Repair Costs. 2021. URL: https://www.aaa.com/autorepair/articles/adas-sensor-calibration-increases-repair-costs.Suche in Google Scholar
42. Andrew J. Hawkins. Waymo’s driverless car: ghost-riding in the back seat of a robot taxi. Dec. 2019. URL: https://www.theverge.com/2019/12/9/21000085/waymo-fully-driverless-car-self-driving-ride-hail-service-phoenix-arizona (visited on 12/09/2019).Suche in Google Scholar
43. Ashesh Jain, Lei Zhang, and Li Jiang. High-Fidelity Sensor Calibration for Autonomous Vehicles. Aug. 2019. URL: https://medium.com/lyftlevel5/high-fidelity-sensor-calibration-for-autonomous-vehicles-6af06eba4c26 (visited on 08/14/2019).Suche in Google Scholar
44. Juli Clover. Apple Self-Driving Car Needed a Realignment After Clipping Curb in Autonomous Mode. Oct. 2021. URL: https://www.macrumors.com/2021/10/12/apple-self-driving-car-clips-curb/ (visited on 10/12/2021).Suche in Google Scholar
45. German Institute for Standardization. DIN-52305: Determination of the optical deviation and refractive power of safety glass for vehicle glazing. June 1995. URL: https://www.beuth.de/de/norm/din-52305/2533097.Suche in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Frontmatter
- Editorial
- Measurement systems and sensors with cognitive features II
- Research Articles
- Uncertainty-aware automated machine learning toolbox
- Explainable AI for sensor-based sorting systems
- Review Article
- Perspectives on AI-driven systems for multiple sensor data fusion
- Research Articles
- A collection of machine learning assisted distributed fiber optic sensors for infrastructure monitoring
- Automated end-of-line quality assurance with visual inspection and convolutional neural networks
- Review Article
- Automotive mass production of camera systems: Linking image quality to AI performance
Artikel in diesem Heft
- Frontmatter
- Editorial
- Measurement systems and sensors with cognitive features II
- Research Articles
- Uncertainty-aware automated machine learning toolbox
- Explainable AI for sensor-based sorting systems
- Review Article
- Perspectives on AI-driven systems for multiple sensor data fusion
- Research Articles
- A collection of machine learning assisted distributed fiber optic sensors for infrastructure monitoring
- Automated end-of-line quality assurance with visual inspection and convolutional neural networks
- Review Article
- Automotive mass production of camera systems: Linking image quality to AI performance
