Bringing the queen mother of the west to life: Digital reconstruction and analysis of Taoist Celestial Beings Worshiping mural’s apparel

2.1 Development of digital protection of murals

The ancient frescoes that have survived throughout history are not only testimonies of ancient civilization but also of research value in various fields such as culture, art, and history. Despite this, these are immovable cultural relics and can be damaged by both natural and man-made causes, making it hard to deeply observe and experience in person. The current conservation of frescoes is mainly reflected in two aspects. One is the conservation of physical entities of murals [7], including the treatment of mural diseases by physical and chemical means, etc., which may cause secondary damage to murals. Additionally, digital technology has been used to conserve murals [8].

Numerous nations have been starting digitalizing cultural heritage in recent times. At the end of the last century, the British Museum and the Louvre Museum in France began digitizing their collections. In 2011, Google launched the Art Project, which cooperated with museums around the world, using Google Street View technology to take pictures of museum interiors and historical paintings in the museums, which were then converted into ultra-high-resolution images for global Internet users to enjoy. To summarize, heritage revitalization has become a fundamental research priority to protect, preserve, and utilize historical and cultural heritage. In particular, data storage, virtual restoration [9], and virtual display of frescoes [10] are the primary methods of digital conservation of frescoes at present.

2.2 Protection status of Worshiping

In 2021, the Shanxi Museum restored the mural Worshiping using 3D printing technology, restored the colors using digital restoration technology, and presented them as animation during the exhibition. The next year, Shanxi Province launched two digital collections of Yongle Palace, which still presented the mural content in two-dimensional form. Additionally, previously published studies about the costumes of the gods and goddesses in Worshiping were limited to costume culture, religious culture [11], and costume aesthetics [12]. Despite these prior research studies by numerous scholars, the digital reconstruction of Worshiping had not been studied previously.

In short, the research and 3D virtual reconstruction of the Yongle Palace mural costumes have positive effects on the innovative design of both heritage exhibitions and digital collections.

2.3 3D virtual display of paintings

3D virtual simulation technology of clothing was mainly used in pattern making [13], fashion style design [14], clothing interaction design [15], and functional clothing design [16]. Furthermore, it is frequently used for costume restoration and 3D virtual displays. Currently, scholars have realized the 3D restoration of costumes in some famous paintings, such as Tang Zhao ling frescoes [17], Tang Tomb Murals [18], Han Xizai Banquet Painting [19], Spring Outing Painting of Madam Guo [20], and DaoLian Painting [21].

However, these studies only had a 3D virtual display of the mural costumes, the reconstruction of the appearance of the figures was not involved in the construction of the human models. Moreover, simulated costumes were mostly single-layered for modeling convenience.

The three-dimensional digital reconstruction of fresco-like cultural relics proposed in this article differs from the previous digital conservation of the physical entities of frescoes. The appearance of mannequins is taken into consideration, as opposed to the virtual restoration of clothing in paintings that is done by other academics. Its ultimate purpose is to reconstruct a virtual world conveyed by frescoes and to enhance the three-dimensional virtual display and interaction of the story content and context presented by frescoes. For example, in the project of the dynamic digital scroll of “Thousand Miles of Rivers and Mountains 3.0,” a 3D engine technology with real-time layered rendering has been used to create 3D models of the original painting’s mountains, buildings, and other elements, giving the audience an immersive and interactive experience.

3.1 General scheme

In this article, we selected Style3D virtual fitting software for 3D digital reconstruction of characters and clothing due to its simplicity and efficiency. Through Style3D, the structure of the fabric is able to be replicated digitally, resulting in a more realistic display of the characters and their respective costumes.

As a physical entity in two dimensions, the mural remains a digital two-dimensional model, even if the entire element is mapped to virtual space. However, the transition from a physical two-dimensional mural to a virtual one alone does not allow one to experience and perceive the world as it was imagined by the ancient people. Therefore, in the link between physical entities and digital models, it is necessary to include, in addition to technical aspects, the analysis of the 3D structure of elements in frescoes.

First, the semantic aspects of the costume are analyzed based on the mural images. Second, human body characteristics are estimated by consulting the literature and image measurement methods and a mannequin is built. Third, based on the previous analysis of clothing styles, colors, and patterns, a layer-by-layer virtual fitting from inside to outside is performed on the previously established mannequins to obtain clothing models. Finally, a virtual display system is built through QR codes.

The illustration of the proposed pipeline is shown in Figure 2.

Figure 2

Illustration of the proposed pipeline.

3.2 Semantic analysis of the costume

Mark Western Mother as I with Roman numerals, and mark the four goddesses beside her as II to V in clockwise order (Figure 3a). The comparison shows that the costumes of the goddesses in the frescoes are similar. The clothing styles are large-sleeved robes, long skirts, etc., only the colors, patterns, and number of layers of clothing are different.

Figure 3

Semantic clothing analyses. (a) Mural image, (b) Goddess I. western mother, (c) Goddess II, (d) Goddess III, (e) Goddess IV, and (f) Goddess V.

The semantic analysis is essential as it has a major influence on the results of the 3D virtual reconstruction in this study, with implications for planning education and tourism projects. The meaning of the costumes of these five goddesses is concerned with three aspects: the secular hierarchy, the era, and the religion.

First, the dress system of celestial beings in Worshiping, including the five goddesses, all inherited the secular dress system to a certain extent. From this, we can understand the layers and styles of their clothing. For example, the form of the dress of the Western Mother follows the dress system of the empress that dates back to the Qin and Han Dynasties [22]. She wears a pale ochre robe with large sleeves decorated with gold clouds and phoenix patterns on the edges, and a green square scarf with a turtle-back pattern on the cuffs; in the inner layer, she wears a stone-green pleated skirt; at the innermost layer, she wears a white Zhongdan (Chinese shirt, which was worn in the middle of a ceremonial robe or dress in ancient China) which can be seen from the cuffs and lower hem of the skirt; at the waist, she wears a vermillion knee covering and a large belt and ribbon hanging to the ground; in addition, she has a cape that wraps around her arms and drops over her knees. The other four goddesses also dressed up as aristocratic women in ancient China, with lower levels than empresses.

Second, the costume of these goddesses reflects the characteristics of traditional clothing from different eras in ancient China. Their costume is part of the traditional clothing system of Han ethnic group, incorporating Mongolian characteristics because of historical background. For example, the adoption of the left lapel in the robes of Western Mother and Goddess V conforms to the habits of Mongolian people, which is usually the right lapel in Han people. And the reason is the influence of the Mongols who ruled China during the Yuan Dynasty. In China at the end of the thirteenth century, after the Yuan rulers had conquered the neighboring countries, various kinds of gold and silver jewelry and technical handicrafts were introduced from Persia and elsewhere. The Yuan, therefore, were fond of adding gold and inlaying treasures to their garments. The author of the mural used the gelled patterning and gilding painting techniques on the edge of large-sleeve robe of Western Mother, showing that it was a popular textile technology “Nashiya” in the Yuan Dynasty. Nashiya is the Persian phonetic translation, which refers to the weaving process with gold woven gold brocade [23]. In addition, the lower edge of the pleated skirt worn by Western Mother is decorated with a circle of pearls, which were favored by the Yuan rulers. Moreover, the skirt waist of Goddess III and Goddess IV is higher than those of the other three, which was located below the chest and prevalent during the Sui, Tang, and Five Dynasties.

Third, the costumes of the five goddesses reflect the ancients’ imagination of Taoist immortals. This can be identified by the details of the costume, such as the halo on the head, the green feathers on the elbows of the sleeves, the red ribbons on the shoulders, the baby pattern on the right breast, and the pearls and jade ornaments on the collar and waist. In addition, Taoism holds the I Ching as its main ideology, and the center of the phoenix crown of Western Mother is decorated with the symbol of “Kun Gua,” which implies the Taoist ideology of “Have Ample Virtue and Carry All Things.” These details mainly come from the creation based on religious doctrine, which will be researched in the future study.

Based on the above analysis and combined with all 25 images of goddesses in Worshiping (Appendix 1), the following rules were summarized: (1) Their clothes wrapped around the waist have a fixed matching method, including ribbons, knee coverings, belts, and leather belts. (2) The color matching of their clothing is mainly based on the contrast between cold and warm. (3) Some styles of clothing have fixed color systems, such as white as the main color for ribbons and red as the knee covering. (4) Leather belts are usually arranged in square or circular shapes. Based on the above, we made artistic creations of styles and colors for the covered parts of the clothing concerning the visible clothing to enrich the display effect. For example, for Goddesses III and IV, some costumes of their lower bodies were invisible, so we added ribbons and set them to white. Other clothing is also designed with regard to the style and color rules we summarized above, of which the detailed steps will not be described.

In summary, we plotted the style diagrams of the main costumes of the five goddesses and extracted the colors and patterns from murals, as shown in Figure 3.

3.3 Body shape estimation under clothing

Although there are currently non-contact anthropometric methods for two-dimensional images [24], this method only applies to situations where the wearer’s clothing is fitted. The clothing in the murals presents the characteristics of a large volume, so we use the method of manually locating feature points for anthropometry. However, the Western Mother is seated, and the postures of other goddesses are also various. So, part of their bodies is usually covered by each other in the painting. Therefore, we cannot measure the key dimensions of the human body through image measurement.

However, the rules of human body shape in murals can be obtained by combining the results of image measurement. First, it can be known that the head–body ratio of the women with full-body portraits in Worshiping shows a distribution between 6.5 and 7.5 by measuring the pixel length of the character’s head and body. It is in line with the basic Asian standard of proportion, suggesting that the mural is realistic. And then, Relevant data show that the average height of Chinese women in the Song Dynasty was 157.17 cm [25], which is close to the medium height of Chinese women. And the corresponding standard clothing size for Chinese women is 160/84A. So, the height and chest circumference of the four goddesses next to Western Mother are set at 160 and 84 cm, respectively. However, the Western Mother is taller than the other immortals as the leader of the female immortals. So, the height of the mannequin of Western Mother is set to be 165 cm, and the bust is 88 cm. After entering the height and chest measurement into the software, the dimensions of other parts can be automatically adjusted in the software based on this. Some data will be given directly by the software, while other key data required for making clothing need to be measured on the three-dimensional model. The data are detailed in Table 1. For the data that need to be measured manually, the results given in the table are the average results of three measurements.

To make the 3D mannequin from the software resemble the goddess of Worshiping, an experienced artist was consulted to make the following adjustments to the 3D mannequin obtained from the software: the width of the face was increased, the width of the eyes was reduced, the mouth was raised, the eyelashes were hidden, the eyeliner was drawn, the eyebrows were thinned, and the skin tone was whitened (Figure 4).

Figure 4

Facial modification of mannequins. (a) Goddess I. western mother, (b) Goddess II, (c) Goddess III, (d) Goddess IV, and (e) Goddess V.

3.4 Reconstruction of multi-layer garments

The human body model has been established, and the next step is to reconstruct the clothing models. We take the clothing modeling of Western Mother as an example to introduce the detailed steps of layer-by-layer modeling.

3.4.2 Clothing pattern drawing and virtual simulation

In the second step, we draw the garment pattern and perform garment virtual simulations on the established mannequin. According to the above analysis, the patterns of the main clothing of the five goddesses are shown in Figure 5.

Figure 5

Main style garment patterns. (a) Cross collar Zhongdan, (b) curved collar Zhongdan, (c) large-sleeved robe, (d) half-arm shawl, (e) pleated skirt, and (f) skirt (Goddess II–V).

The sequence number of each picture in Figure 3 represents the inside and outside sequence of clothing wearing, and ① is the innermost layer. We perform virtual simulation layer by layer in this order. The simulation of each garment follows the following process: First, arrange the garment pattern in the corresponding parts. Second, do the virtual sewing. Third, perform the virtual fitting, and flexibly use the selection move tool to adjust the shape of the garment. Finally, set the physical properties of the fabric to improve its realism.

We need to conduct virtual fitting in multiple layers and take Western Mother as an example to describe clothing modeling layer by layer. After simulating each garment, it was frozen to preserve its shape before simulating the outer garment. First, simulate the Zhongdan. Because it is a left lapel, when arranging the pattern, focus on arranging the right lapel on the outer layer of the left lapel. After the simulation is completed, freeze the Zhongdan, and arrange the pleated skirt on the outer layer of the Zhongdan for simulation. During the experiment, it was found that the inner and outer garment fabrics conflicted with each other and the simulation speed was slow, so the part that could be covered by the coat was deleted, leaving only the collar. Then, arrange the large sleeve robe on their outer layer to simulate. The outer layer of the big sleeve robe is ribbon.

The ribbon hangs down to the ground, with the upper part being covered by the knee covering. In the outer layer of the knee covering, the belt is wrapped around the waist and hangs to the ground. We cut the belt into a belt and two straps that hang down and sew the hanging part to the belt. The silk cape cannot be arranged in a conventional way and needs to use the fixed pin tool to adjust its wearing form when the simulation is turned on. In addition, the silk cape is stacked on the shoulders. Therefore, add equidistant internal lines in the pattern and use the folded pleat tool to adjust the fold angle of each internal line. The layer-by-layer simulation steps are shown in Figure 6.

Figure 6

Overview of the garment reconstruction method. (a) Establish a mannequin, (b) simulate Zhongdan (c) simulate robe and skirt, (d) simulate all garment, (e) add colors and normal maps, (f) pattern generation, and (g) add patterns.

3.5 Establishment of virtual display system

After building the model, a real-time virtual display system is established. The image is rendered in the software to obtain the front views of five goddesses after the virtual reconstruction is completed (Figure 7). Scan the QR code in the bottom right corner to view the 3D model and information on the official Style3D platform for the model of five goddesses.

Figure 7

Digital reconstruction effect. (a) Goddess I. western mother, (b) Goddess II, (c) Goddess III, (d) Goddess IV, and (e) Goddess V.

4.1 Structure of evaluation system

This evaluation considers detailed information such as model appearance, clothing style pattern, and so on. Establishing the evaluation system based on these factors, the analytic hierarchy process applies to divide the evaluation index system into a criterion layer and an indicator layer. The first layer represents the total target factor set Z = (Z ₁, Z ₂, Z ₃); the second layer represents the subtarget factor set Z ₁ = (Z ₁₁), Z ₂ = (Z ₂₁, Z ₂₂, Z ₂₃, Z ₂₄), and Z ₃ = (Z ₃₁, Z ₃₂, Z ₃₃), to establish the evaluation system as shown in Table 4.

4.2 Determine the evaluation set of virtual reconstruction effect

The evaluation set is the set of evaluation results that evaluators can produce for each factor, and the evaluations of different evaluation indicators are different. A 5-point rating scale was used as the evaluation scale, V = (v1, v2, v3, v4, v5) = (excellent, good, average, poor, very poor).

4.3 Determine the weights of each indicator

Different indicators have different impacts on the evaluation of virtual recovery effects, so it is essential to determine the weights of each indicator.

First, the evaluation matrix G is established by inviting five experts to compare and rate each indicator using the 9-quartile scale method (Appendix 2), with G corresponding to Z = (Z ₁, Z ₂, Z ₃), G1 corresponding to Z ₁ = (Z ₁₁), G2 corresponding to Z ₂ = (Z ₂₁, Z ₂₂, Z ₂₃, Z ₂₄), and G3 corresponding to Z ₃ = (Z ₃₁, Z ₃₂, Z ₃₃). Second, the hierarchical single ranking is conducted. The hierarchical single ranking refers to the ranking of the importance of each factor at the level for a factor at the previous level. The solution vector corresponding to the maximum eigenvalue of the judgment matrix, after normalization, is the ranking weight of the relative importance of the corresponding factors at the same level to a factor at the upper level. Third, perform consistency checks on the judgment matrix. If it passes the test, it proves to be valid, otherwise, the expert needs to repeat the comparison and recalculate the judgment matrix.

For example, the judgment matrix for primary indicators, based on the scores assigned by the first expert, is as follows:

The maximum eigenvalue and eigenvector of matrix G were calculated using Matlab, and the eigenvectors were normalized. The maximum eigenvalue λ _max = 3.0037 of the matrix G in equation (1) and the eigenvector normalized to w = (0.1220, 0.6483, 0.2297). These data represent the weight assigned by the first expert to the first level indicator, so the weight of the model appearance is 0.1220, the weight of the garment silhouette is 0.6483, and the weight of the garment fabrics is 0.2297.

To ensure coordination between the elements of the matrix G, the consistency index (CI) and consistency ratio (CR) are calculated by the following equations:

In equation (2), λ _max is the maximum eigenvalue of the judgment matrix and n is the order of the judgment matrix. In equation (3), the average random consistency index (RI) depends on the order of the matrix (n). When CR ≤ 0.1, the judgment matrix is considered having passed the consistency test. For matrix G in equation (1), CI = 0.00185 is calculated, and RI = 0.58 when n = 3 is known (Appendix 3), so the CR = CI/RI ≈ 0.0032 < 0.1, which passes the consistency test. Therefore, these data are valid.

The same method is used to obtain the weights assigned to each indicator by the other four experts. The weighting coefficients of primary indicators were obtained by synthesizing the weight vectors of the five experts by arithmetic averaging.

Similarly, the weight coefficients of the secondary indicators were obtained, and the comprehensive weight coefficients of each indicator were calculated, as shown in Table 5.

4.4 Establishment of single-factor fuzzy evaluation matrix

After determining the evaluation index weights, the evaluation data of the secondary indicators were summarized and counted by means of a questionnaire (Appendix 4). The respondents to the questionnaire were undergraduate students majoring in clothing-related fields. A total of 105 questionnaires were collected, of which 100 were valid, with a valid rate of 95.2%. The survey statistical results are shown in Table 6. According to the statistical results, the second-level single-factor fuzzy comprehensive evaluation matrix R was obtained.

4.5 Fuzzy comprehensive effect evaluation

Combining the weights of each indicator and the fuzzy comprehensive evaluation principle, the following formula was used to calculate the comprehensive evaluation value (B) of the virtual recovery effect.

In equation (4), A is the combined weight coefficient of each indicator, and R is the second-level single-factor fuzzy comprehensive evaluation matrix. B = (0.2386, 0.5761, 0.1801, 0.0051, 0) was calculated. The comprehensive evaluation results show that the affiliation degree of “excellent” is 0.2386, “good” is 0.5761, “average” is 0.1801, “poor” is 0.0051, and “very poor” is 0. According to the principle of maximum affiliation, the reconstruction effect is “good.”

Worshiping is one of the most outstanding Chinese religious murals in terms of artistic achievement. In this article, a 3D digital reconstruction practice is conducted on the example of the five goddesses centered on the Western Mother of Worshiping in Yongle Palace. A user study was conducted using the FAHP, and the survey results showed that the reconstruction results were “good.”