Advances in personalized modelling and virtual display of ethnic clothing for intelligent customization

2.1.1 Human body modelling based on 3D scanning technology

3D body scanning combines information processing, electronics, computer graphics, computer vision, and other technologies to get single or multiple frames of data to reconstruct a static digital model. 3D body scanning is mainly completed to collect high-precision point cloud images of the human body, extract features, and other work [9].

The Cyberware WBX Full Body Colour 3D Scanner is capable of performing hundreds of thousands of scans of all parts of the human body. The system is equipped with four scanning heads, which rapidly scan each part of the human body from head to foot, generating a measurement data value every 2 mm to map out an accurate 3D dataset. The route for human body modelling based on 3D scanning technology is shown in Figure 2. The database of young male body shapes in Inner Mongolia has been established for the first time based on the principle of technical route and the advantages of human body shapes, to expand the boundaries of human body shape research, and to help the digital development of the clothing industry in Inner Mongolia.

Figure 2

A technical route for human modelling based on 3D scanning technology.

Yu and Kim [10] developed a dress form reflecting the body shape of middle-aged Korean women based on 3D human body data in Korean Size-6. Wen and Shih [11] developed a military body armour that protects and adapts using a K-means algorithm to create clusters of different body types for both sexes. Kolose et al. [12] developed a specific sizing system using anthropometric data from the New Zealand Defence Force Anthropometric Survey. All the above studies are built based on large-scale anthropometric data, taking advantage of 3D body scanning to establish a human body type database, which provides accurate data for clothing design, sizing systems, and smart prototypes, but cannot achieve the large number of personalized human digital model technologies required for smart customization. To be more accurately oriented to the development of specific clothing products, scholars have launched a study on the characteristics of the human body shape in local areas. Yoon et al. [13] broke through the use of traditional length data by defining six lateral shape directional vector angles based on three ground markers on the body shape, as well as a 3D surface created by standard landmarks. Then, the y-axis directional angle of the spatial vector was used to evaluate lateral body shape, and a logistic regression analysis was used to establish a classification and prediction model for upper lateral body shape. And established a classification and prediction model for upper body shape using logistic regression analysis. This method can be used to determine individual body shapes and can also interactively provide accurate sizing information to automated clothing pattern-making systems. Kim and Do [14] obtained foot shapes of elderly Korean men through a body scanner to help shoemakers produce multifunctional shoes. The 3D body scanning data are useful for studying obese body shapes [15], middle-aged and elderly people [16], and other special body shapes, accumulating accurate body shape data for people who need customized services and improving the quality of customized patterns. The idea is to summarize the original hundreds of individual data into dozens of characteristic parameters that can be highly correlated with customized clothing through mathematical and statistical methods [17], with a single parameter characterization [18] (length, angle) or a comprehensive mathematical relationship representation of several parts [19,20].

The focus of such digital human models (DHMs) is mainly on the measured body size data, ignoring the level of visualization of the specific model, which cannot satisfy or appeal to the visual experience of the customers at the sales end. Therefore, to build a personalized digital human body for e-commerce, Liu et al. [21] proposed a multiple modelling method (MMM) to create a realistic digital human body model. The TC² is used to obtain the whole body point cloud data and then partition the human body module (Figure 3a). The forward modelling method is used to model the hand and foot (Figure 3b). The photo modelling method is used to model the head (Figure 3c) and the torso with the reverse modelling method (RMM) (Figure 3d). The hands, feet, head, and torso are integrated into a static model, and the virtual skeleton is bound to the avatar to create a DHM. In RMMs, the layout method is usually used to reconstruct various parts of the human body. First, extract the cross-section of the human body from the point cloud, and then fit the curve to obtain the human body slice. In general, the more curves, the more accurate it is to model the leg by this method. However, too many curves lead to rough surfaces of the model. It was found that a leg model which is fitted by lofting 20–25 curves can meet the needs of fashion design very well through trial and error. To construct human body parts such as arms, shoulders, and the middle of the torso with irregular point cloud data distribution as accurately as possible, it is first necessary to determine the boundary lines of each part based on the changes in human body curves. Then, the position of the cutting plane is determined based on the characteristic point coordinates of the human body curve and adopted spatial geometry and point clouds to fit surfaces. Finally, a human body part model can be established well. The hybrid modelling technology integrates the advantages of multiple modelling techniques to achieve personalized goals, which is a new approach to human body modelling.

Figure 3

MMM based on 3D body scanning including the process of a–d.

Human body modelling based on 3D body scanning technology mainly serves the manufacturing end of the clothing industry. The current problem is dealing with noise data and filling incomplete surfaces. Some details are still difficult to solve, including the shadow part of the armpit and crotch, the light absorption effect of hair and skin, data processing, etc. The follow-up should provide services for the industrial sales end from scanning technology and human body model reconstruction and establish a personalized digital human body.

2.1.2 Parametric-based human modelling

The parametric human modelling method combines ergonomic principles to establish a statistically obtained virtual human, which can change the body shape of the virtual human by inputting a set of low-dimensional feature parameters and outputting the corresponding 3D model of the human body. It is an essential human modelling tool due to its simple operation and easy popularization in commercial virtual fitting software, mainly including SCAPE, Skin Multi-Person Linear (SMPL), SMPL-X, and other high-accurate and high-speed parametric human body models [22].

Anguelov et al. [23] proposed a SCAPE parametric human model based on edge deformation (Figure 4) and achieved a high-quality animated surface model for moving sequences through motion capture. On this basis, Zhou et al. [24] proposed a simple and easy-to-use image modification technique to reconstruct 3D skeletons through 2D images for realistic image remodelling. Wang [25] synthesized a new parametric sample model of the human body by changing the feature wireframe. Parametric human modelling can achieve many modelling and flexible adjustments of inherent parameters. However, this method differs greatly from natural human surface deformation, and the base model in the existing software can hardly meet personalized customization. The human body reconstruction is affected by the parameter settings. If the parameters are not comprehensive, accurate human body reproduction cannot be achieved. Otherwise, too many parameters will reduce the modelling efficiency and cannot complete the desired effect.

Figure 4

The parametric human model of SCAPE.

2.1.3 Image-based human modelling

The image-based human modelling method refers to the use of real photos, drawn images, video images, depth images, etc., to obtain the two-dimensional information of human body images and the matching image processing algorithms to extract human body representation information, such as contours, feature dimensions, etc., and transform them into 3D spatial data to build a 3D human body model. The specific technical route is shown in Figure 5.

Figure 5

A technical route for image-based modelling.

The first problem of the image-based modelling method is to reconstruct parameters and size extraction of the 3D human body model. Gu et al. [26] used the method of “feature point - insertion point - feature curve - basal plane - human model” to establish 15 generation rules for feature section curves, achieving 3D virtual modelling of the female upper body and automatic generation of personalized panels. Cai et al. [27] classified the body shape of 178 female college students based on their waist, abdomen, and buttocks, achieving automatic recognition of body shape based on 2D photos. Both of the above two methods use “3D scanning + photos,” and the human body model built is more accurate. Li et al. [28] reconstructed a detailed 3D human body from multi-view images by combining voxel super-resolution with the learning of implicit representations. Zhang and Xiao [29] incorporated the topology of SMPL template mesh and flexible mesh deformation and first used a typical 2D CNN architecture ResNet-50 to extract perceptual features from the input image and fuse the image features into 3D space by perceptual feature transformation. This method can qualitatively generate 3D human body meshes with more details and can build more accurate dynamic pose models. Li et al. [30] effectively encoded the deformation by representing 3D human body shapes through vertex-based deformation. The aforementioned three studies focus on the visual communication of the human body’s dynamic poses and shapes, without involving the human body dimensions, and thus the models are not highly accurate. Lu et al. [31] improved the accuracy of image acquisition and constructed a fast and lightweight deep learning network called Fast Body Net to generate end-to-end human models. The consumer-grade RGBD camera proposed in this research can provide excellent applications in real-time display and interaction in VR.

Human body modelling based on image sequence has low equipment construction cost, convenient image acquisition, and is suitable for client input in the customization process. It has a certain value in the virtual fitting room on the sales side. However, the overall accuracy of the human body model could be higher. It should be combined with 3D human body scanning to obtain a more accurate model simultaneously. In addition, there is an increased requirement for human clothing, which still requires the human body to wear only underwear or no clothes. These methods need further research to reconstruct the natural human facial shape due to the complexity of facial structure.

2.1.4 Software-specific human modelling

Specific software can create or recreate human body parts and can render and interactively control them. Commonly used software includes Matlab, Geomagic, 3D Studio MAX, OpenGL, Poser, and Maya [8]. Matlab and Geomagic are reverse engineering software [32] that can reverse reconstruct the human body model, and that requires 3D human scanning models. 3D Studio MAX [33] and OpenGL [34] can build complex objects and carry out animation design and interactive control, which can be applied to virtual clothing design and virtual fitting systems. Poser can collect many vertex data of complex human surfaces and export them, then combine them with other software to build dynamic human models with diverse poses [35]. Maya has a powerful rendering function that can create lifelike characters [17] and environments, which is widely used in the field of film and television animation and graphic design. However, the human body model built by Maya has poor versatility, which is challenging to integrate into actual production and is rarely used in the clothing field.

Accurate human body data are the basis of clothing customization, and a realistic human shape model is the key for enhancing customer consumption experience. These two points are the bottleneck of digital human body modelling technology for smart customization. With the increasingly wide coverage of 5G networks, the demand for the digital human body for smart customization is not only limited to accurate data at the manufacturing end but has been expanded to areas such as personalized human body and interaction design at the sales end. With the rapid development of human body modelling technology, the subsequent research focuses on:

1. Large-scale personalized human modelling research.

2. Quickly generate realistic and personalized digital bodies.

3. Consider a detailed accurate model of the head and face.

4. Research on digital human body modelling technology is interfaced with industry applications and clients to be able to meet the needs of customized sales and production.

2.2.1 Physical simulation method for forward design

The idea of the physical simulation method is to study how to sew 2D patterns into 3D clothing and then import the 2D patterns into 3D software. Then, change the 3D shape by adjusting 2D patterns, and display the 3D dress effect in real-time [37]. This method matches the workflow of the manufacturing end of the clothing customization process, so it is called the forward design idea. To achieve realistic clothing draping effects and animation effects, realistic simulation of fabric is the prerequisite for the realization of 3D clothing modelling. Through the establishment of mechanical models, energy, and other physical quantities, the motion of each part of the fabric is transformed into the mass motion and particle model under the action of various forces [38] to simulate the constraint control of spatial motion deformation of 3D clothing patterns [39]. The spatial relationship between clothing and the human body is defined through the numerical solution, collision detection, and response [5].

Jin et al. [40] proposed two multi-precision fabric modelling methods based on machine learning to solve complex calculation problems and the long-time consumption of fabric physical simulation methods. Virtual sewing technology is another major difficulty of physical simulation methods, involving the conversion of 2D–3D clothing patterns, the boundary processing of sewing alignment, the constraint processing of the sewing process, etc. Zhang et al. [41] established a virtual clothing model with seams perfectly matching the sewing of the 2D patterns (Figure 7), which makes the clothing model of the virtual interface closer to the sewing state of the clothing in reality, and provides a more effective way for the intelligent customization client to achieve an interactive design with the manufacturing end. Chen et al. [42] proposed a novel Neural Sewing Machine, which can learn the representations of clothing with different shapes and topologies, and has been successfully applied to 3D clothing reconstruction and controllable manipulation.

Figure 7

Model of clothing with seam structure.

2.2.2 Geometric simulation method for the reverse design

The geometric simulation method aims to study how to design clothing directly on a 3D human model, confirm the clothing style by adjusting the 3D clothing display effect, and finally unfold it into a workable 2D pattern [37]. This method is based on the target results of the reverse generation of production elements, the opposite of the workflow of the clothing customization process, so it is called the reverse design idea. The basis of modelling is divided into human body models, clothing models, and pictures, using a data-driven approach to clothing modelling.

Kim and Vendrovsky [43] proposed a method to drive clothing deformation by human skeleton pose, adding collision detection to linear skin deformation to obtain a more ideal clothing model. Zakharkin et al. [44] proposed a new method to model clothing based on the human body point cloud, capturing the appearance of clothing from video and re-rendering. Geng et al. [45] proposed a data storage technique that allows users to draw 3D sketches on human surfaces and use contour lines as boundary lines by interpolation to obtain an interactive design of clothing models (Figure 8). Holte [46] scanned the real clothes using RGB-D sensors, then created the clothing model with the clothing modelling software MD, textured the surface, and approximated the colour of the obscured part by K-Nearest Neighbors, showing the results of digital clothing by scanning real clothes. Xu et al. [47] proposed a new method for constructing 3D virtual clothing models from photographs. The method only needs to input the front image and back image, to process various states of clothing: model, human body model, or plane. The predicted boundary is used to estimate the size information of clothing and deform the template clothing grid to generate the final 3D model. The method of geometric modelling is to generate a clothing model by changing the surface geometry of the 3D model, which is simple and efficient, but focuses on the appearance of the contour effect and gives less consideration to the physical properties of the clothing, which has poor draping and insufficiently realistic folds.

Figure 8

The clothing model was created by outlining the contour boundary lines on the human surface.

2.2.3 Hybrid modelling method

The hybrid modelling method combines the advantages of physical modelling and geometric modelling. Wang and Tang [48] added the calculation of clothing tension during the sewing process of 2D patterns to control the deformation effect of the clothing model under tension, to improve the realism of the clothing model (Figure 9). It takes into account the physical properties of the model while focusing on the realistic effect of the model, which also has a positive significance for the correction of 2D patterns. However, this method is based on fitted silhouette clothing, and the modelling effect for clothing with loose volume or pleats needs to be further verified.

Figure 9

Hybrid method clothing model.

At present, customized clothing is mostly produced by forward design, which has problems such as poor fit, the slow transmission of interactive information, a long design and production cycle, and high prices. The reverse design is often used for online sales and customer fitting in custom stores, which has poor interactivity and weak and untimely data support on the manufacturing end, such as personalized data, automatic correction, matching patterns, etc. It is also a key direction for future research.

In addition, there is an increasing interest in simulating the process of dressing [49] besides modelling single 3D clothing, and the overlapping function of multiple clothing is also worth exploring for research. Hu et al. [50] involved multi-layer animations of clothing taken by a 3D scanner in clothing modelling for the first time. The naked body of the object was scanned with an RGBD camera, and then the subjects wearing each layer of clothing were scanned again. The joint model formulated according to the previous step will be fitted to the newly scanned data. The new clothing was segmented semi-automatic and added as additional layers to the multi-layer clothing model. However, there are limitations to this method, as the segmentation of the clothing needs to be manually adjusted. The ability to scan the human body in any pose and fully automate clothing extraction is the key direction for future research of this technology.