Application of virtual and augmented reality in automotive

Zdeněk Čujan; Gabriel Fedorko; Nikoleta Mikušová

doi:10.1515/eng-2020-0022

Article Open Access

Application of virtual and augmented reality in automotive

Zdeněk Čujan , Gabriel Fedorko and Nikoleta Mikušová

Published/Copyright: January 31, 2020

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information

From the journal Open Engineering Volume 10 Issue 1

Abstract

Nowadays, virtual reality enters engineering work. It is widely used in various sectors. It brings new possibilities that result is increasing of productivity and reliability of production, quality of products and processes. One of the areas where virtual reality has been used more and more is logistics. Virtual reality and especially augmented reality offer above all in the area of logistics a perspective related with increasing of effectiveness of processes.Wide possibilities for virtual reality use in logistics can be found in the automotive industry. The paper describes an application of the method of video-mapping in storage logistics. The paper also describes its practical use as a tool for increasing the efficiency of logistics processes, which was presented by a reduction of the processing time of the shipment picking by 10%. The paper also points out the possibilities for applying this method in other industrial areas, it is possible to use the experience gained in automotive and described in this paper.

Keywords: Virtual reality; augmented reality; logistics

1 Introduction

Virtual and augmented reality presents a powerful engineering tool that finds application in various engineering fields, including logistics. Authors Karkul and Stryhunivska present one of the possible examples of virtual reality application in logistics [1]. They dealt with detailed field oriented to facilitating logistic systems. However, in the context of logistics, the concept of augmented reality is increasingly being used instead of classical virtual reality which is characterized by an artificially created environment [2]. The great advantage of augmented reality is the fact that it offers, among other things, a direct view of the real environment [3]. Use of augmented reality is possible with special hardware tools, such as glasses Hololens [4] or Google Glasses. It is an approach that is used for example in storage logistics. This problem is in detail described by Guo et al. [5]. As one of the results of their research is the application of head-up displays which reduce errors in logistics processes and increases their efficiency. This problem is also in detail researched by Reif et al. [6]. These authors state that headset systems of augmented reality can significantly affect providing information to employees and indicate their use within the system Pick-by-Visio. The system Pick-by-Visio is used in various systems of storage logistics [7]. This process is described in more detail in the paper of the authors Schwerdtfeger et al. [8]. It is possible to state that virtual and augmented reality are a perspective are for information and communication technologies in the automotive industry [9]. It helps to increase their efficiency, eliminate bottlenecks and improve them. Virtual reality can be used for example for the planning of layout and allocation of production devices and equipment [10]. Virtual reality has also its place for logistics in the field of planning and projection of transport units. This activity can use knowledge, presented in [11]. Virtual and augmented reality dispose with perspective for optimization of processes within production logistics and this fact raises these technologies for automotive industry [12, 13]. Based on the present knowledge, it is possible to state, that virtual and augmented reality has a great potential for logistics. But it is needed to realize a continual research, verification of their application possibilities to logistics processes. Automotive industry is characterized by highly sophisticated logistics processes that can make application of virtual and augmented reality more efficient. Within the paper it will be described and analyzed ways of application of virtual and augmented reality in the field of storage logistics and processes of shipment picking. The presented example will point out the possibility of obtaining time savings by realization of the selected logistics operations.

2 Virtual reality in logistics

The use of augmented and virtual reality in production, processing and other industrial applications contributes to greater visibility, training and problem-solving. Improvement can be made in the area of design, assembly, quality control and safety. The resulting effect of operation and processes can increase the quality of products and accelerates their marketing.

Options of virtual and augmented reality can be beneficial in many operational areas:

Improvement of design solution
Complex assembly
Quality assurance
Maintenance
Professional support
Safety
Stock operation.

Logistics uses mainly augmented reality. It is used primarily in warehousing, transport optimization, distribution and in an expanded range of value-added services. The storage uses for example glasses with the display, portable computers and cameras. They enable flawless storage of material, it is finding in a warehouse position and flawless completion of orders. The system eliminates errors caused by material changes. In transport, augmented reality applies for completeness checking of supplies and loading of material.

Manual pallet recalculation is replaced by the scanner with 3D sensors, which quickly and accurately determines the number of pallets and package in a pallet. The system should also be able to detect damaged pieces in the package. In the distribution, we mean handing over the consignment to the addressee. Often there are problems with drivers, they cannot find the addressee due to bad or missing data. This should be avoided by using the face detection function to determine the authenticity of the recipient. But there is also a problem with the privacy law. The offer of services for data protection we mean assembly or repair. The system of augmented reality monitors and identifies errors in the process.

2.1 Augmented reality in logistics systems

The term augmented reality – AR, which is not as known as virtual reality, has been used since 1990. Augmented reality is the representation of reality and the subsequent addition of digital elements. Augmented reality is one of the four categories of virtual reality. Within it, the real worldis visible, either the immediate surroundings of the observer (real reality) or the projection of a distant place (presence in another place). In such a created real-world system are placed artificial images, which can, for example, to display commonly invisible object to the human eye. The term augmented reality is a general term used for a variety of technologies that are a combination of alphanumeric codes, graphic object (elements), symbols and information that appear in the real world.

In terms of logistics, the recognition, localization and projection augmented reality have the best supposition for use and application.

2.2 Projection type of augmented reality

The projection-type of augmented reality is characterized by the projection of graphic elements into the real environment. Graphic elements can be a monitor with the naked eye and projection technology is required for their projection. This type of virtual reality is mainly used for information purpose, it helps to realize navigation in open or closed objects and can display simple operating elements (Figure 1).

Figure 1

Example of a projection type of virtual reality [14, 15].

In terms of implementation, projection by augmented reality belongs to direct methods of augmented reality because it allows monitor of the physical scene directly and objects are added in a transparent form.

3 Application of virtual and augmented reality in automotive

Logistics in automotive companies is a key component, or it represents one of the main serving items of any automotive company. It cooperates very closely with production. Together they provide car production, production of service parts, aggregates and their components, production of tools and preparations.

Other tasks of logistics include ensuring the supply of parts and components and at the same time, it prepares the realization of new projects. Within logistics in automotive, activities and measures are realized which aimed at the optimization of logistics processes, areas, handling devices, including material flow. In order to meet these requirements, the implementation of information technologies is realized, then coordination of JIT processes, creation of packaging regulations. Lust but not least, logistics concepts and projects are created. Effective realization of all these activities requires the use of the latest technologies, where the virtual and augmented logistics have an irreplaceable role.

Last but not least, logistics concepts and projects are being created. Effective implementation of all these activities requires the use of the latest technologies, among which virtual and augmented reality have an irreplaceable role.

3.1 Application of virtual and augmented reality in automotive

As it was mentioned, virtual reality and especially augmented reality dispose of with a huge potential for the field of logistics processes. Classical virtual reality offers its application especially in analytical activities in the field of projection and design of logistics processes. It can be used primarily in connection with a digital twin or the philosophy of digital enterprise.

In logistics, it is possible to find out, thanks to virtual reality, whether it will be possible to mount parts and components according to the proposed technological procedure. Another use of virtual reality is a virtual simulation of various natural and physical phenomena, such as night driving or temperature loads. Virtual reality finds great use in CRASH tests. Virtual reality makes work faster and easier. The environment in virtual reality is in the long term aspect cheaper than the creation of physical models and prototypes (Figure 2). Virtual reality saves money significantly.

Figure 2

Example of virtual reality application within storage logistics of small components.

3.2 Use of the method of video-mapping in storage logistics

One of the main activities within storage logistics is the preparation of products for distribution and their picking. Within this activity, it can often lead to errors, which are most often presented by frequent replacement of components, their wrong allocation and inadequate fixation to the palette, and this can have affected to damages during transport. One of the ways to prevent these adverse phenomena is to use augmented reality within the method of video-mapping.

This method is based on a projection in which a 3D image is projected onto the static object. Video-mapping can project image accurately and without distortion on any surface. It is a direct method of augmented reality. It is realized by projectors in combination with different sensors and it creates texturing of the physical object. It works thanks to laser projection. For right realization of this method, it is needed their precise calibration (Figure 3).

Figure 3

Principle of the method of video-mapping within storage logistics [16].

3.3 Determination and evaluation of the effectiveness of applying the tools of virtual and augmented reality

The use of virtual and augmented reality in automotive industry needs to be evaluated for the need of knowing the effectiveness and benefits which they bring. The evaluation criteria with the highest interest of users are presented by financial and time criteria. The financial criteria are primarily focused on the monitoring of the reduction of costs associated with errors due to for example prolonging of the process of picking, product damage or more handling operations. Time criteria monitor reducing of the time of picking, time of control and time for the eventual reduction of different types of failures.

For example, we can use the method of time shot for the evaluation of the method of virtual and augmented reality, with comparison of the time of the picking process before the implementation of the method of virtual and augmented reality with the time after its implementation. The result must be clear in favour of implementing one of the methods. Other methods, we can use are various statistical and economic analysis. Their application must be realized with an emphasis on logistics processes,where these tools of virtual and augmented reality are used.

3.4 Description of materials and methods used in the research

Two methods of analysis – system analysis and time studies were chosen and used for the research described in this paper.

3.4.1 System analysis

“System analysis is one of the methods used in the analysis of logistics systems in a company. This analysis applies the systemic approach. The problem for analysis must be separated into subsystems and elements. System analysis focuses on examining the existing whole, evaluating the realization of its function, target behaviour of parts and also as a whole [17]”.

The course of system analysis was realized in three stages:

verification of correctness of objectives formulation,
analysis of the structure of the system and its behaviour,
design of options for improving the existing system.

3.4.2 Time studies

“Time studies primarily serve to standardize work, but on the other hand, they can serve as a basis for improving work tasks. The outputs of this analysis can be used as a basis for detecting activities that do not add value and also their origin [18]. Time studies are ranked among the methods of direct observation. They are divided into two groups:

Continual – data are obtained by continuous measurement,
Moment – data are obtained by measuring random moments during the working day [18]”.

Work measurement was performed at the workplace where virtual and augmented reality tools were installed in real-time. Through a time study, the following were monitored:

shot of the working day
moment observation
chronometry [18].

During the analysis, the main focus was on 5 main areas, which assessed the monitored processes, activities, that were performed, wastage of time, but also activities without value. The analysis focused on the objective of the actual work of the worker by picking and the way of work realization. Based on the analysis of these factors, it was possible to identify, combine, connect and simplify individual activities. The analyses followed the main and secondary objectives of the process of picking [18]”.

4 Example of augmented reality use by consignment picking

The picking process places high demands on accuracy, time and reliability. The aim is to ensure zero replacement of goods, and their right fixation and allocation on the transport and handling devices. Furthermore, within this process, components cannot be damaged.

Use of the method of video-mapping, the device displays instruction on a specific part, on the part carrier, or on the floor of the hall. Laser projections determine the correct position of the part on the palette. Text, figures and videos illustrate and show the optimal fixation and protection of the parts and components. For monitoring of displayed information, it is not needed a special device.

The system detects if any of the parts are positioned wrong and it helps workers in positioning them correctly.

The augmented reality minimizes errors by packing and increases the safety of the working environment.

The system of video-mapping consists of lasing projector with ultra-high-resolution and HD cameras (Figure 4). The important part of the whole system is the adaptation of the cameras to the intensity of the lightening in the hall so that the devices reliably ensure projection and also scan the surrounding.

Figure 4

Projectors of the system of Video-mapping.

The system must provide a solution that will not delay individual workers in their work by material picking. The whole system stars working by the way that the worker loads to the system the specific information by a bar code. The first help of the system starts with a light indication of the specific location, where to allocate the palette RACK (Figure 5).

Figure 5

Marking of the place for allocation of the pallet.

Subsequently, the parts are assembled according to the list. Before the allocation of parts in the position, the worker checks the identity of the part by reading the bar code (Figure 6). The system evaluates if the part meets the requirement. If the part does not match, the system lights up red as a warning symbol. In case of a match, the system starts with help. Text information is presented on the floor and blue colour presents the right position of the part allocation on the palette. After positioning the part in the correct, enlightened position, the blue colour of the enlightened position is changed to green for a moment and then goes out (Figure 7).Workers complete due to text and light helps the whole palette.

Figure 6

The download of material code.

Figure 7

The lighting of parts after allocation in the right position.

After assembling of parts, the entire palette is lighted in green to indicate that all parts are in agreement. This confirms the required allocation of all assembled parts in the position and the secure fixation of the parts to the pallet. Textual information is displayed on the floor that the pallet is completely ready.

5 Conclusion

Implementation of augmented reality –Video-mapping for the workplace of parts assembling on the palette, saves a certain amount of time per worker (Table 1).

Table 1

Input data about the sale, monthly average prices and revenues.

Name of operation	Time of assembling
	Original condition (min)	Use of video-mapping (min)
Preparation of material from the warehouse	30	30
Preparation of pallette	3	2,5
Control of parts accuracy	13	9,5
Assembling and fixation of parts to the palette	24	21

Total time [min]	70	63

If more workers are involved to complete one pallet, the saved time will logically increase. From the point of view of investment, the acquisition, implementation and start-up of this system was a significant intervention in finance. Save time and eliminate errors altogether, due to implementation of vide-mapping is a great asset. The other side of the matter is the importance and expected return on investment to implement this system. The return on investment is expected within one year.

The aim of the research, described in this paper, was to implement tools of the augmented reality to the process of shipments picking within the processes of storage logistics with the aim to increase their efficiency and shortening the time for preparation. This conclusion was fully fulfilled and the achieved savings (10%) present a significant benefit which is reflected in the overall logistics performance and represents considerable savings in logistics costs. On the base of these facts, further research will focus on expanding of application to the area of storage logistics. The aim will be to investigate the continuity of the described application. This intention is based on the fact that this solution can bring savings in the area of process time and economic costs. In terms of the initial phase of the research, the paper is the first of the forthcoming series, which will be devoted to the research of augmented reality and its application possibilities and benefits in the area of storage logistics in automotive.

Acknowledgement

This work is a part of these projects VEGA 1/0600/20, VEGA 1/0403/18, VEGA 1/0638/19, KEGA 013TUKE-4/2019, KEGA 012TUKE-4/2019 and SK-SRB-18-0053.

References

[1] Karkula M, Stryhunivska O. The use of virtual factory concepts and models for facilitating logistic systems and operations. In: Carpathian logistics congress (CLC’ 2016). Zakopane, Poland: Tanger LTD, 2017, pp. 456–462.Search in Google Scholar

[2] Stolfa J, Stolfa S, O’Connor RV, et al. Chances for virtual and augmented reality along the value chain. In: Stolfa, J and Stolfa, S and OConnor, RV and Messnarz R (ed) Systems, software and services process improvement (EUROSPI 2017). Berlin, Germany: Springer-Verlag Berlin, 2017, pp. 352–359.10.1007/978-3-319-64218-5Search in Google Scholar

[3] Agard C, von Davier AA. The virtual world and reality of testing Building Virtual Assessments. In: Jiao, H and Lissitz R (ed) Technology enhanced innovative assessment: development, modeling, and scoring from an interdisciplinary perspective. Charlotte, USA: Informatio Age Publishing-IAP, 2018, pp. 1–29.Search in Google Scholar

[4] Lang S, Kota M, Weigert D, et al. Mixed reality in production and logistics: Discussing the application potentials of Microsoft HoloLens (TM). In: Ginters, E. Paulauskas, V. and Estrada M (ed) ICTE in transportation and logistics2018 (ICTE 2018). Amsterdam, Netherlands: Elsevier Science BV, 2019, pp. 118–129.10.1016/j.procs.2019.01.115Search in Google Scholar

[5] Guo A, Wu X, Shen Z, et al. Order Picking with Head-Up Displays. Computer (Long Beach Calif) 2015; 48: 16–24.10.1109/MC.2015.166Search in Google Scholar

[6] Reif R, Guenthner WA, Schwerdtfeger B, et al. Evaluation of an Augmented Reality Supported Picking System Under Practical Conditions. Comput Graph forum 2010; 29: 2–12.10.1111/j.1467-8659.2009.01538.xSearch in Google Scholar

[7] Hanson R, Falkenstrom W, Miettinen M. Augmented reality as a means of conveying picking information in kit preparation for mixed-model assembly. Comput Ind Eng 2017; 113: 570–575.10.1016/j.cie.2017.09.048Search in Google Scholar

[8] Schwerdtfeger B, Reif R, Guenthner WA, et al. Pick-by-vision: there is something to pick at the end of the augmented tunnel. VIirtual Real 2011; 15: 213–223.10.1007/s10055-011-0187-9Search in Google Scholar

[9] Kubasakova I, Kampf R, Stopka O. Logistics information and communication technology. Commun - Sci Lett Univ Zilina 2014; 16: 9–13.10.26552/com.C.2014.2.9-13Search in Google Scholar

[10] Stopka O, Kampf R. Determining the most suitable layout of space for the loading units’ handling in the maritime port. Transport 2016; 33: 280–290.10.3846/16484142.2016.1174882Search in Google Scholar

[11] Chocholáč J, Boháčová L, Kučera T, et al. Innovation of the Process of Inventorying of the Selected Transport Units: Case Study in the Automotive Industry. LOGI – Sci J Transp Logist 2017; 8: 48–55.10.1515/logi-2017-0006Search in Google Scholar

[12] Kampf R, Hlatká M, Bartuska L. Optimization of Production Logistics. Adv Sci Technol Res J 2018; 12: 151–156.10.12913/22998624/100351Search in Google Scholar

[13] Stopka O, Chovancová M. Optimization process of the stock quantity based on a set of criteria when considering the interaction among logistics chain components. In: 22nd International Scientific on Conference Transport Means, 3-5 October 2018. Kaunas, Lithuania: Kaunas University of Technology, Lithuania, 2018, pp. 737–742.Search in Google Scholar

[14] Augmented reality. https://content.thriveglobal.com/wp-content/uploads/2017/11/augmentedreality.jpg 2019; 1.Search in Google Scholar

[15] Virtual reality. https://hips.hearstapps.com/roa.h-cdn.co/assets/cm/14/48/1024x683/547be6554c901e4_-_tregside_volkswagen-lg.jpg?resize=480:* 2019; 1.Search in Google Scholar

[16] Video-mapping. https://lightguidesys.com/wp-content/uploads/2016/08/LGS-System-Architecture-wLogos-2016.jpgSearch in Google Scholar

[17] Malindžák D, Takala J. Projecting logistics systems. Košice, Slovakia: Expres Publicit s.r.o., Slovakia, 2005.Search in Google Scholar

[18] Krisťak J, Kyseľ M, Vyšňanský M. Analysis, measurement and standardization of work. Žilina, Slovakia, 2010.Search in Google Scholar

Received: 2019-10-10

Accepted: 2019-11-20

Published Online: 2020-01-31

This work is licensed under the Creative Commons Attribution 4.0 International License.