The reform of the teaching mode of aesthetic education for university students based on digital media technology

3.1 Unity3D-based virtual roaming scene creation

Digital media technology can provide more intuitive, vivid, and vivid aesthetic education content, which makes it easier for college students to accept and understand. At the same time, digital media technology can also make college students more actively participate in the process of aesthetic education through interaction and experience and improve their learning interest and enthusiasm. The application of digital media technology can expand the form and way of college students’ aesthetic education. The traditional aesthetic education mainly focuses on classroom teaching, which is monotonous in form and boring in content, and is difficult to arouse the interest of college students. Digital media technology can integrate aesthetic education into all aspects of college students’ daily life through the Internet, multimedia, VR, and other ways to make aesthetic education more interesting and effective. Digital media technology can provide personalized educational content and methods according to different aesthetic needs and characteristics of college students, so that college students can get more comprehensive and in-depth experience and understanding in the process of receiving aesthetic education. The theoretical framework of the research includes the following four aspects: (1) Digital media technology: This article studies the 3D modelling of ancient cultural buildings and the construction of virtual roaming scenes by using digital media technology. Digital media technology involves many fields such as computer graphics, VR, and human–computer interaction, which provides a new way for the protection and inheritance of ancient cultural architecture. (2) VR technology: Through VR technology, students can feel the heritage and style of ancient cultural buildings more deeply in the classroom and improve students’ aesthetic ability. VR technology can create an immersive interactive experience, so that students can feel the charm of ancient cultural buildings. (3) Artificial intelligence algorithm: The dynamic time warping (DTW) algorithm and grid collider algorithm used in the research are both classic algorithms in the field of artificial intelligence. The DTW algorithm is used for gesture recognition and can deal with time series matching of gesture sequences effectively. The grid collider algorithm is used to detect collision events in gesture sequences, which provides important supplementary information for gesture recognition. (4) Teaching reform concept: The research aims to explore the method of using digital media technology and VR technology to realize the reform of aesthetic education teaching methods in universities. Through innovative teaching methods, stimulate students’ interest in learning and improve students’ learning effect. At the same time, the method designed by the institute can also enhance students’ learning interaction and cultivate students’ innovative thinking and practical ability.

In university aesthetic education classes, it is essential to develop students’ appreciation of the beauty of various types of architecture around the world, but simply using videos or pictures does not provide students with a more realistic and stunning experience. To this end, the study uses VR technology to create roaming scenes of architecture and uses somatosensory technology for human-computer interaction to bring a richer teaching experience to students. The research uses geometric modelling as the main approach and other modelling methods as a supplement to create 3D models of cultural buildings. Geometric modelling techniques model data by representing, controlling, analysing, and outputting geometric entities with geometric and topological information reflecting the shape, position and presentation of the structural drawing. Maya modelling software was chosen for the study to model the interior and exterior of the building. Similar and repetitive details are often found in the external structures of buildings, for which a modular modelling approach was used to simplify the work and improve modelling efficiency. Many ancient buildings come with frescoes as well as inscriptions inside the building. In the modelling process, it is necessary to consider not only the structure of the model, but also the seamless stitching of textures such as frescoes at a later stage, and the study unfolds the UV of the model into a whole on the basis of a single-sided infill, which is able to achieve perfect stitching of the internal textures.

The 3D model alone is far from satisfying the user’s need for realism in VR, and to have a better visual space, you need to add mapping textures to the model. The first element of mapping is the UV spread, also known as UV mapping, and Maya software comes with a variety of mapping methods that use a predetermined rule to project UV texture coordinates onto the surface of the model, automatically creating an association between the texture image and the surface. The results of UV mapping using different mapping methods are shown in Figure 1.

Figure 1

UV mapping with different mapping methods. (a) Planar mapping results. (b) Cylinder mapping results. (c) Spherical mapping results. (d) Automatic mapping.

After UV mapping, the images were studied for rendering in Photoshop to complete the model, Unity3D is an emerging engine with a strong presence in the film and television industry and has great strengths in creating roaming scenes. In order to achieve a realistic roaming effect, characters must react when they encounter objects in the scene, and characters stop moving forward when they encounter buildings and are not able to walk through walls. The study uses the Unity3D engine to add colliders to the model. Mesh colliders are complex colliders that fit into the structure of the model and can achieve good interaction. However, it consumes a lot of computational resources and affects the performance of the scene, so other colliders need to be selected depending on the situation. In some cases where interaction is not required, prototype colliders can be used instead, saving resources and optimizing scene performance. The external environment of the virtual scene mainly includes the sky, mountains, and features such as flowers and trees. The study uses Unity’s own sky box material to simulate the virtual sky environment. The study also uses a manual approach for the external terrain of the building. Combining the above operations, the study created a roaming 3D scene of a cultural building based on Unity3D.

3.2 Kinect-based depth image pre-processing in roaming scenes

Kinect is a 3D body camera that not only acquires colour images of targets but also captures depth images of targets and has features such as human skeleton, face recognition, and voice recognition. The main methods of environmental depth measurement are triangulation, time-of-flight, and structured light measurement. The structured light measurement method is mainly based on optical coding technology, which is at the heart of Kinect’s depth image acquisition. The light source for optical encoding is a laser scatter, the main characteristic of which is that the scatter is highly random and changes pattern depending on the distance and size of the Kinect. the Kinect laser scatter imaging principle is shown in Figure 2.

Figure 2

Kinect laser speckle imaging principle.

Before recognizing the scattered images of a spatial object, the light source is calibrated: assuming that a reference plane is taken every 10 cm, 30 scattered images can be obtained with this calibration. When the sensor is used to take a picture of the object to be measured, the scattered image of the object to be measured is obtained, and this image is correlated with the previous 30 images to obtain 30 correlated images. The peaks on the correlation images are the position of the object to be measured, and all peaks are superimposed and differenced to obtain a 3D image of the object to be measured, i.e., a depth image. This results in the height and width of the object being imaged and the actual physical position of the object not corresponding to each other. The original depth information captured by Kinect is not the actual spatial distance of the object. Therefore, it is necessary to correct for errors in the depth data and the actual distance to the object. The formula for converting the depth value to the actual distance is based on the Kinect calibration principle

In equation (1), d raw is the depth value of a pixel point on the depth image, H = 3.5 × 10 − 4 rad , K = 12.36 cm , L = 1.18 rad , O = 3.7 cm . The actual depth distance of the pixel is obtained from equation (1) at d . The 3D coordinates of the pixel point in real space are calculated from the depth distance in equation (2).

In equation (2), F = 0.0021 and w × h are the resolutions of the Kinect. The RGB images acquired by the Kinect sensor have some visual deviations from the depth images, and to correct for these deviations, the depth data is converted to the corresponding RGB data. When capturing depth images with the Kinect, the infrared component of the strong light finder will cover the laser scatter on the surface of the target to be measured, and the CMOS sensor is not sensitive to the scatter pattern in these areas, resulting in a hollow noise in the final depth image. For this reason, the study uses median filtering to filter out the noise. The median filtering principle is shown in Figure 3.

Figure 3

Schematic diagram of median filtering principle.

The median filter for noise reduction of the data is calculated as

In equation (3), f ( r , t ) represents the grey value of any pixel point within the domain window w , N f ( i , j ) is the solid field of grey values, and median w is the intermediate value for finding that taken grey in the window. Combining the above operations, the study converts the depth information into real distances and calibrates and denoises the image.

3.3 Human gesture recognition during roaming

When performing dynamic recognition, it is first necessary to track the various movements of the user. The study uses Kinect to capture the depth data of the human body and extract the morphology of the body to obtain a depth image. The depth image features are then extracted from the depth image, and then pixel-by-pixel information is inferred based on the depth image features using machine learning algorithms for body part recognition. All pixel information is aggregated using clustering algorithms to form a reliable prediction of 3D skeletal joint positions as the final output, and then the evaluation system uses this for each pixel of the output to determine the position of the joint point. An illustration of the depth difference feature in the depth image of the human skeletal nodes is shown in Figure 4.

Figure 4

Schematic diagram of depth difference features in bone node depth images.

The study combines the random forest algorithm and the mean shift algorithm to track the human skeleton and accurately locate the human joints to obtain the 3D coordinates of the joints. To reduce the computational complexity and improve the recognition accuracy, six joints of the left hand, right hand, left wrist, right wrist, left elbow, and right elbow are selected as feature vectors. The forearm movement of the hand can be represented by the wrist joint as well as the elbow key data, and a feature vector consists of the 3D coordinates of the six joint points, which can be written as

In equation (4), ( x , y , z ) is the coordinate of the corresponding joint point and n is the human skeletal joint data at the time of t n . Considering that the elbow and wrist joints as well as the hand joints move excessively during the gesture movement, while the shoulder joints contribute little value to the gesture movement. Therefore, the centre of the left/right shoulder was chosen as the reference point for the study, which can be written as

After obtaining the reference point A by means of equation (5), the data is normalized using

In equation (6), O is the coordinate vector corresponding to the skeletal joint point before normalization, O norm is the vector after normalization, LS is the coordinate vector corresponding to the left shoulder joint point, and RS is the coordinate vector corresponding to the right joint point. The position of the shoulder joint does not change significantly during the gesture recognition process, and by using it as a reference point, the variation in feature vectors caused by different user sizes and distance sensors and thus by the gesture recognition process is eliminated. Invalid gesture frame sequences can interfere with the gesture recognition effect during dynamic recognition. The study improves the real-time performance of the dynamic gesture recognition system by detecting the gesture starting point and focus, segmenting the before and after dynamic gesture frame sequences, extracting the gesture frames with high correlation Xu Lei, and removing the invalid frame sequences.

The study uses the DTW algorithm for dynamic gesture recognition. The DTW algorithm is a typical template matching algorithm that finds an effective time-calibrated matching path between a sequence of gestures and a sequence of template gestures and solves for the regularization function corresponding to the minimum cumulative distance when the sequence is calibrated. W ( n ) . The DTW regularization path is shown in Figure 5.

Figure 5

Tuning path of the DTW algorithm.

The path formed by the grid points passing between the starting and ending points is the regularized path, as follows

In equation (7), the k -th element of W w k represents the path of the k -th matching. The regularized paths satisfy the three constraints of continuity and monotonicity and the same starting point and end point. To determine the optimal regularization path, the minimum cost function of the regularization path is defined as follows:

The distance at each point on the matching path is accumulated. When performing dynamic gesture recognition, the gesture sequence to be measured is matched with the reference gesture sequence by the DTW algorithm, and the gesture corresponding to the minimum distance value output is the final recognition result. The cumulative distance is shown as

However, the DTW algorithm is an iterative computation of the data, and as the sequence of gestures increases, the complexity of the algorithm and the computational effort increases, resulting in a lower recognition rate, and the need to set a reference gesture template before each recognition also increases the computational effort. To address this, the study uses a diamond shape to constrain the global path, and the slope of the path curve is controlled to be in the range of 0.5–1. This way, when using the DTW algorithm for gesture matching, only the matching distance of the corresponding frame of the grid point within the diamond needs to be calculated. The global path constraint is shown in Figure 6.

Figure 6

Global path restrictions.

In addition, the study sets a distortion threshold to solve the problem of too many reference gesture templates leading to increased matching complexity. Combining the above operations, the study uses the DTW algorithm to achieve dynamic recognition of gestures and realize forward, left, backward, and right turn operations using gestures in roaming scenes.

Ancient Chinese architecture is the oldest and most complete architectural system in the world and has long adhered to the principle of combining mechanics and aesthetics. As an important component of art, architecture provides a valuable resource for aesthetic education, and architectural art education is an important way to implement aesthetic education. When students use the institute to design a virtual scene reenactment system, the class begins with the teacher organizing students to enter the scene and acquire human skeletal data through a Kinect body sensor to connect the body camera to the Unity3D game engine. The students enter the external scene and use the movement of the hand skeleton points to control the virtual mouse to move forward, backward, left, and right in the scene. This allows them to tour the interior of the building and experience the aesthetic structure of the building. If you exit the building after the tour, when you hit the trigger collider placed at the entrance, you will recede to the outer scene interface. This allows students to experience the beauty of architecture up close and personal without having to leave home, enhancing their appreciation of beauty and thus promoting their interest in learning about art and bringing them a new teaching experience beyond the books themselves.

By designing and developing a virtual scene reproduction system and using Kinect and Unity3D game engine, this study enables students to experience the beauty of ancient architecture in an immersive way, thereby improving their ability to appreciate beauty and stimulating their interest in art learning. In the traditional education mode, students can only learn about ancient buildings through books or pictures. Although such teaching methods can provide certain information, they cannot make students feel the physical beauty and spatial sense of buildings. Therefore, through VR technology, this study allows students to personally enter the virtual ancient architecture scene, control the movement of virtual characters through their own actions, and further understand the structure and design concept of ancient architecture. At the same time, in other aesthetic education classes such as music and art, students can feel the beauty more directly through these 3D fields, and their perception of the learning content has been further improved.

For the educational opportunities at the university level, the significance of this study lies in providing a new way and means of education. Due to the influence of many factors such as region, funds, and time, many students may not be able to visit the ancient architecture in person, and the virtual scene reproduction system developed in this research can provide an effective alternative for these students. In addition, this research can also promote interdisciplinary cooperation and exchange. Architectural art education is an important way to implement aesthetic education, which needs the cross-cooperation of many disciplines such as aesthetics, architecture, and computer science. Through the development of such a VR system, cooperation and exchange between different disciplines can be promoted, and the comprehensive quality and innovative ability of students can be improved. To sum up, this study is of great significance for improving the aesthetic education received by college students and can also expand educational opportunities at the university level and promote interdisciplinary cooperation and exchange.