Personalised neural networks for a driver intention prediction: communication as enabler for automated driving
-
Johannes Reschke
,
Cornelius Neumann
Abstract
In everyday traffic, pedestrians rely on informal communication with other road users. In case of automated vehicles, this communication can be replaced by light signals, which need to be learned beforehand. Prior to an extensive introduction of automated vehicles, a learning phase for these light signals can be set up in manual driving with help of a driver intention prediction. Therefore, a three-staged algorithm consisting of a neural network, a random forest and a conditional stage, is implemented. Using this algorithm, a true-positive rate (TPR) of 94.0% for a 5.0% false-positive rate (FPR) can be achieved. To improve this process, a personalization procedure is implemented, using driver-specific behaviours, resulting in TPRs ranging from 91.5 to 96.6% for a FPR of 5.0%. Transfer learning of neural networks improves the prediction accuracy of almost all drivers. In order to introduce the implemented algorithm in today’s traffic, especially the FPR has to be improved considerably.
About the authors
Johannes Reschke studied Electrical Engineering and Applied Research in Engineering Sciences at the Ostbayerische Technische Hochschule Regensburg, Germany. He is currently working on his PhD thesis at the Light Technology Institute at Karlsruhe Institute of Technology, Germany. In parallel, he is a development engineer in the Department of Development Lighting Innovations/Functions at Audi, Germany. His research focuses on vehicle-pedestrian-communication and machine learning.
Cornelius Neumann studied Physics and Philosophy at the University of Bielefeld, Germany. After his PhD, he worked for the automotive supplier Hella in the advanced development for Automotive Lighting. During his time at Hella, he was responsible for signal lighting, LED application and acted as a director of the L-LAB, a laboratory for lighting and mechatronics in public private partnership with the University of Paderborn, Germany. In 2009, he became professor for Optical Technologies in Automotive and General Lighting and one of the two directors of the Light Technology Institute at the Karlsruhe Institute of Technology, Germany.
Stephan Berlitz studied Electrical Engineering at the Technical University of Munich, Germany. Afterwards, he started his professional carrier in the Development and Technical Coordination for Tail Lights at Schefenacker in Esslingen, Germany. He worked in the Department of Innovations Lighting at Audi, Germany, from 2001 to 2005. Since then, he is head of Lighting Innovations/Functions and therefore responsible for leading light innovations at Audi.
Acknowledgement
We would like to thank especially Nawel Attia, Benedict Schleyer and Thomas Hoess, whose master theses layed the foundation for this contribution.
Author contribution: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: None declared.
Competing interest: The authors declare no conflicts of interest regarding this article.
References
[1] K. Merten, Informelle Zeichengebung im Straßenverkehr: Bericht zum Forschungsprojekt 7521 der Bundesanstalt für Straßenwesen Bereich Unfallforschung, Köln: Bundesanstalt für Straßenwesen Bereich Unfallforschung, 1981.Suche in Google Scholar
[2] R. Risser, Kommunikation und Kultur des Straßenverkehrs, 1st ed. Wien, Österreich, Literas-Universitäts-Verlag, 1988.Suche in Google Scholar
[3] A. Rasouli, I. Kotseruba, and J. K. Tsotsos, “Understanding pedestrian behavior in complex traffic scenes,” IEEE Trans. Intell. Veh., vol. 3, pp. 61–70, 2018, https://doi.org/10.1109/itsc.2018.8569324.Suche in Google Scholar
[4] D. Rothenbücher, J. Li, D. Sirkin, B. Mok, and W. Ju, “Ghost driver: A field study investigating the interaction between pedestrians and driverless vehicles,” in 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), IEEE, ed., New York, NY, USA: IEEE, 2016, pp. 795–802.10.1109/ROMAN.2016.7745210Suche in Google Scholar
[5] V. M. Lundgren, A. Habibovic, J. Andersson, et al.., “Will there be new communication needs when introducing automated vehicles to the urban context?,” in Advances in Intelligent Systems and Computing, Advances in Human Aspects of Transportation: Proc. of the AHFE 2017 International Conf. on Human Factors in Transportation, vol. 597, Los Angeles, CA, USA, N. A. Stanton, Ed., Cham, Schweiz, Springer International Publishing, 2018, pp. 485–497.10.1007/978-3-319-41682-3_41Suche in Google Scholar
[6] National Highway Traffic Safety Administration and US, Department of Transportation, Washington, DC, USA, Traffic Safety Facts: Pedestrians. 2016 Data, 2018.Suche in Google Scholar
[7] C. R. Hudson, S. Deb, D. W. Carruth, J. McGinley, and D. Frey, “Pedestrian perception of autonomous vehicles with external interacting features,” in Advances in Human Factors and Systems Interaction: Proceedings of the AHFE, Orlando, FL, USA: Springer, 2018, pp. 33–39.10.1007/978-3-319-94334-3_5Suche in Google Scholar
[8] J. Domeyer, A. Dinparastdjadid, J. D. Lee, G. Douglas, A. Alsaid and M. Price., “Proxemics and Kinesics in Automated Vehicle–Pedestrian Communication: Representing Ethnographic Observations,” in vol. 2673, Transportation Research Record: Journal of the Transportation Research Board, National Academy of Sciences, EdSAGE journals., Washington, DC, 2019, pp. 70–81.10.1177/0361198119848413Suche in Google Scholar
[9] A. Schaudt and S. Russell, “Judging a car by its cover: Human factors implications for automated vehicle external communication,” in Lecture Notes in Mobility, Road Vehicle Automation 5, G. Meyer and S. Beiker, Eds., Cham, Schweiz, Springer International Publishing, 2018, pp. 69–76.10.1007/978-3-319-94896-6_6Suche in Google Scholar
[10] J.-H. Willrodt, H. Strothmann, and J. Wallaschek, “Optical car-to-human communication for automated vehicles,” in Darmstädter Lichttechnik, Proc. of the 12th International Symposium on Automotive Lighting: ISAL 2017, T. Q. Khanh, Ed., München: utzverlag, 2017, pp. 579–588.Suche in Google Scholar
[11] J. Reschke, P. Rabenau, M. Hamm, and C. Neumann, “Symbolische fahrzeug-fußgänger-interaktion,” in Optische Technologien in der Fahrzeugtechnik: Proc. der 8. VDI-Fachtagung, vol. 8, V. D. I Wissensforum GmbH, Ed., Düsseldorf, VDI Verlag GmbH, 2018, pp. 95–106.10.51202/9783181023235-95Suche in Google Scholar
[12] J. Reschke, M.-T. Auburger, R. Marichalar, and C. Neumann, “Kommunikation zwischen automatisierten Fahrzeugen und Fußgängern,” ATZ - Automob. Z., vol. 121, no. 9, pp. 16–21, 2019, https://doi.org/10.1007/s38311-019-0098-z.Suche in Google Scholar
[13] I. Othersen, S. Cramer, and C. Salomon, “HMI for external communication - kann die Fahrzeugbewegung als Kommunikationskanal zwischen einem Fahrzeug und einem Fußgänger dienen?,” in: VDI Wissensforum GmbH, editor. 10. VDI-Tagung Mensch-Maschine-Mobilität, Braunschweig: VDI Verlag GmbH; 2019, vol. 2360, pp. 145–154.10.51202/9783181023600-145Suche in Google Scholar
[14] J. Reschke, T. Höß, B. Schleyer, S. Berlitz, and C. Neumann, “How vehicles learn to display symbols to pedestrians,” in Darmstädter Lichttechnik Proc. of the 13th International Symposium on Automotive Lighting: ISAL 2019, vol. 18, T. Q. Khanh, Ed., München: utzverlag, 2019, pp. 590–599.Suche in Google Scholar
[15] J. Reschke, S. Prösl, M. Hamm, and C. Neumann, “Assistance system for vehicle-pedestrian-interaction: Deep learning and driver intention prediction,” in SIA VISION 2018: Vehicle Infrastructure Safety Improvement in Adverse Conditions and Night Driving, SIA - French Society of Automotive Engineers, Paris, Frankreich, 2018, pp. 19–26.Suche in Google Scholar
[16] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” in Neural Computation, Massachusetts Institute of Technology, vol. 9, Boston, MA, USA: MIT Press, 1997, pp. 1735–1780, https://doi.org/10.1162/neco.1997.9.8.1735.Suche in Google Scholar
[17] H. Salehinejad, S. Sankar, J. Barfett, E. Colak, and S. Valaee, “Recent advances in recurrent neural networks,” [Online]. Available at: https://arxiv.org/abs/1801.01078 [accessed: Nov. 28, 2019].Suche in Google Scholar
[18] B. Oancea and S. C. Ciucu, “Time series forecasting using neural networks,” [Online]. Available at: https://arxiv.org/abs/1401.1333 [accessed: May. 21, 2019].Suche in Google Scholar
[19] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on Knowledge and Data Engineering, Piscataway, NJ, USA: IEEE, 2010, vol. 22, pp. 1345–1359.10.1109/TKDE.2009.191Suche in Google Scholar
[20] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, Cambridge, Massachusetts, London, England, MIT Press, 2016.Suche in Google Scholar
[21] F. Gers, “Long short-term memory in recurrent neural networks,” Lausanne, Schweiz, Disseration, École polytechnique fédérale de Lausanne, 2001.Suche in Google Scholar
[22] Y. Guo, H. Shi, A. Kumar, et al., “SpotTune: transfer learning through adaptive fine-tuning,” [Online]. Available at: https://arxiv.org/abs/1811.08737 [accessed: Nov. 28, 2019].10.1109/CVPR.2019.00494Suche in Google Scholar
[23] S. Yoon, H. Yun, Y. Kim, G.-T. Park, and K. Jung, “Efficient transfer learning schemes for personalized language modeling using recurrent neural network,” 31st Association for the Advancement of Artificial Intelligence: Workshop on Crowdsourcing, Deep Learning and Artificial Intelligence Agents, San Francisco, CA, USA: AAAI Press, 2017, pp. 457–463.Suche in Google Scholar
[24] K. Pudenz, “Volkswagen zeigt Start-Stopp 2.0 und Freilauf-Motor-Aus,” [Online]. Available at: https://www.springerprofessional.de/automobil---motoren/volkswagen-zeigt-start-stopp-2-0-und-freilauf-motor-aus/6585236 [accessed: Jun. 03, 2019].Suche in Google Scholar
[25] K. R. Varshney, “Engineering safety in machine learning,” in 2016 Information Theory and Applications Workshop (ITA): Jan. 31, 2016-Feb. 5, 2016, IEEE, Piscataway, NJ, USA, IEEE, 2016.10.1109/ITA.2016.7888195Suche in Google Scholar
[26] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 3rd ed. Cambridge, MA, USA, MIT Press, 2009.Suche in Google Scholar
[27] F. Al-Khoury, “Safety of machine learning systems in autonomous driving,” Master thesis, Stockholm, Schweden, School of Industrial Engineering and Management, KTH Royal Institute of Technology, 2017.Suche in Google Scholar
[28] I. N. D. Silva, D. H. Spatti, R. Andrade Flauzino, L. H. B. Liboni, and S. F. D. Reis Alves, Artificial Neural Networks: A Practical Course, Cham, Schweiz, Springer, 2017.10.1007/978-3-319-43162-8Suche in Google Scholar
[29] Kraftfahrt-Bundesamt, “Verkehr in kilometern: inländerfahrleistung,” [Online]. Available at: https://www.kba.de/DE/Statistik/Kraftverkehr/VerkehrKilometer/verkehr_in_kilometern_node.html [accessed: Jan. 03, 2020].Suche in Google Scholar
[30] Bundestanstalt für Straßenwesen, “Ergebnisse Fahrleistungserhebung 2014: erhebung der Inländerfahrleistung und der Inlandsfahrleistung,” [Online]. Available at: https://www.bast.de/BASt_2017/DE/Verkehrssicherheit/Fachthemen/u2-fahrleistung-2014/u2-Fahrleistung-2014-ergebnisse.html [accessed: Jan. 03, 2020].Suche in Google Scholar
[31] T. Fawcett, “An introduction to ROC analysis,” Pattern Recogn. Lett., vol. 27, no. 8, pp. 861–874, 2006, https://doi.org/10.1016/j.patrec.2005.10.010.Suche in Google Scholar
[32] L. Zhu, S. Li, Y. Li, et al.., “Study on driver’s braking intention identification based on functional near-infrared spectroscopy,” J. Intell. Connected Veh., vol. 1, Emerald Publishing Limited, pp. 107–113, 2018, https://doi.org/10.1108/jicv-09-2018-0007.Suche in Google Scholar
[33] D. Tran, W. Sheng, L. Liu, and M. Liu, “A hidden markov model based driver intention prediction system,” in IEEE International Conf. on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER): 8 - 12 June 2015, Shenyang, China, IEEE, Piscataway, NJ, USA, IEEE, 2015, pp. 115–120. 2015.10.1109/CYBER.2015.7287920Suche in Google Scholar
[34] M. Garcia Ortiz, J. Fritsch, F. Kummert, and A. Gepperth, “Behavior prediction at multiple time-scales in inner-city scenarios,” in IEEE Intelligent Vehicles Symposium (IV), IEEE, Piscataway, NJ, USA, IEEE, 2011, pp. 1068–1073.10.1109/IVS.2011.5940524Suche in Google Scholar
[35] H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller, “Transfer learning for time series classification,” in IEEE International Conf. on Big Data: Proceedings: Dec 10 - Dec 13, 2018, Seattle, WA, USA, N. Abe, Ed., Piscataway, NJ, USA, IEEE, 2018, pp. 1367–1376.10.1109/BigData.2018.8621990Suche in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Frontmatter
- Community
- News
- Topical Issue: Automotive Lighting; Guest Editor: Cornelius Neumann
- Editorial
- Automotive lighting
- Letter
- Development of HD-headlamps with high-performance projectors
- Research Articles
- Imaging vehicle-to-vehicle communication using visible light
- Manufacturing of volume holographic cell arrays for usage with uncollimated LEDs in automotive applications
- Personalised neural networks for a driver intention prediction: communication as enabler for automated driving
- Development of a high resolution scanning RGB laser headlamp
- Review articles
- Temporal coherence properties of laser modules used in headlamps determined by a Michelson interferometer
- Influence of the perceived size of a light source on non-visual effects in humans
Artikel in diesem Heft
- Frontmatter
- Community
- News
- Topical Issue: Automotive Lighting; Guest Editor: Cornelius Neumann
- Editorial
- Automotive lighting
- Letter
- Development of HD-headlamps with high-performance projectors
- Research Articles
- Imaging vehicle-to-vehicle communication using visible light
- Manufacturing of volume holographic cell arrays for usage with uncollimated LEDs in automotive applications
- Personalised neural networks for a driver intention prediction: communication as enabler for automated driving
- Development of a high resolution scanning RGB laser headlamp
- Review articles
- Temporal coherence properties of laser modules used in headlamps determined by a Michelson interferometer
- Influence of the perceived size of a light source on non-visual effects in humans
