Benefit evaluation of building energy-saving renovation projects based on BWM weighting method

3.1 Constructing an evaluating index system for ESR benefits of existing residential buildings

For managing building ESR projects, the positive and negative benefits are obtained by comparing input and output. If output exceeds the input, it means that the project has brought positive benefits and income, and vice versa, it indicates that the project has brought negative benefits and losses [13,14,15]. For example, evaluating the benefits of residential energy renovation is based on evaluating the ratio of input to output to quantify the cost of renovation and the benefits after renovation. Value engineering is a systematic approach that aims to achieve as many functionalities as possible with minimal cost throughout the entire lifecycle. In fact, value engineering has also studied the correlation between cost and effectiveness [16,17]. Therefore, this study uses the concept of value engineering as a guide, combines the interactions of various energy renovation project entities, and uses statistical methods as a tool to establish a set of existing residential ESR benefit-evaluating index systems on the foundation of value engineering. The specific process of the evaluating index system established through research is shown in Figure 1.

Figure 1

Constructing process of the evaluating index system of the existing residential ESR based on value engineering.

As shown in Figure 1, when establishing an evaluation criteria system, it is necessary to first analyze and define functional and cost indicators with a value engineering orientation. Next, by reading a large amount of relevant literature and conducting on-site investigations, questionnaire surveys, and other steps, it will identify an indicator that can meet the needs of multi-dimensional evaluation. While ensuring the operability and practicality of this evaluating index system, it is necessary to reasonably screen and simplify the initial evaluation indicators. After distributing survey questionnaires to experts in relevant fields and collecting their suggestions, the most representative evaluation indicators are selected. Finally, using statistical analysis methods, this existing residential evaluation index system based on value engineering is determined. Value engineering regards function as its core, and for different products or objects, function represents their purpose or benefits, that is, the role and practicality assigned to the product, reflecting its value in use. Figure 2 shows the logical relationship between functional indicators and different objects.

Figure 2

Logical relationships between functional metrics and different objects: (a) user requirements – product logic relationship diagram and (b) participant requirements – product logic diagram.

Overall, consumers’ product choices are often based on their specific needs for product functionality. Functionality is actually a concrete manifestation of user needs, while products serve as carriers of functionality. Considering that the goal of energy-saving transformation is to enhance public welfare, such reform activities are very important. Therefore, in this situation, cost management work becomes extremely important, as ESR needs to focus on its quality to meet the needs of residents and other stakeholders [18]. Based on the above considerations, the existing evaluation index system for the benefits of residential ESR mainly revolves around functional indicators and cost indicators. This study constructs this evaluation system through a combination of literature statistics and questionnaire survey methods. First, by organizing 50 literature on evaluating ESR benefits of existing residential buildings published on Web of Science and CNKI from 2013 to 2022, they are categorized based on functional and cost indicators, and it summarizes and determines the specific names and evaluation categories of each indicator [19]. During the expert research phase, the team gathered expert opinions from different fields and backgrounds to ensure the comprehensiveness and representation of the evaluation indicators. Through a diversified expert team, the influence of a single expert opinion on the construction of the index system could be reduced, and the objectivity and scientificity of the index system could be improved. Second, in the statistical analysis stage, the team systematically collated and screened the collected data. By using the principles and methods of statistics, the reliability, validity, and consistency of the data were tested, so as to ensure the scientific and accurate evaluation results. The 95 questionnaires collected contain a total of 2,470 data on 26 initial indicators. Through statistical analysis of these data, the identification degree of each indicator is calculated. Eq. (1) describes the calculation process:

Value engineering adopts the ABC classification method, which draws on Pareto’s 80–20 principle, which only retains indicators with a recognition level of over 80% and eliminates six indicators with a recognition level of less than 80%. After such screening steps, the existing evaluation indicators for ESR benefits of residential buildings obtained in this study are shown in Table 1.

3.2 Design of benefit evaluation model for building ESR projects based on BWM weighting method

The existing ESR of residential buildings involves many stakeholders and has complex and dynamic characteristics [20]. In order to solve these complex, dynamic, and difficult-to-quantify problems, this study uses BWM and fuzzy comprehensive evaluation methods to construct a value engineering-based evaluation model for the efficiency of existing residential ESR. It calculates the weights of various indicators using the BWM method and then uses the fuzzy comprehensive evaluating method to determine the functional and cost coefficients of ESR. It quantifies the renovation benefits through value engineering formulas and evaluates them in sequence. The operation process of the evaluation model for ESR benefits of existing residential buildings on the foundation of value engineering is shown in Figure 3.

Figure 3

Operation flow of efficiency evaluation model of ESR of existing residential buildings on the foundation of value engineering.

When calculating indicator weights using the BWM method, it is first necessary to determine the decision criteria set. In this step, c 1 , c 2 , … , c n is used as the judgment criterion to evaluate the target. Taking car purchasing as an example, the decision criteria can be {Quality ( c 1 ), Price ( c 2 ), Comfort ( c 3 ), Safety ( c 4 ), and Style ( c 5 )}. After determining the judgment criteria, it is necessary to develop the optimal and worst criteria. Numbers between 1 and 9 are utilized to determine the best standard preference relative to other standards, and the expression for the best standard preference is shown in Eq. (2):

After determining the best standard preference, the number between 1 and 9 is used to determine the worst standard preference relative to other standards. Eq. (3) describes the worst standard preference:

Finally, the optimal weight needs to be determined, and its expression is shown in Eq. (4):

In order to achieve the optimal indicator state, it is necessary to determine the maximum absolute error, and the expression for the maximum absolute error is shown in Eq. (5):

In Eq. (5), w j represents the standard optimal weight, a j W represents the worst standard preference, and a B j represents the best standard preference. Eq. (6) describes the standard optimal weight constraint conditions:

In order to better utilize the maximum absolute error to select indicator weights, the study uses Eq. (7) to convert the maximum absolute error solution into a linear programming problem:

In Eq. (7), ξ represents the maximum absolute error value. In addition, the ratio of ξ to the consistency coefficient CI is the consistency ratio, and the expression for the consistency ratio is shown in Eq. (8):

Afterwards, Eq. (9) is utilized to describe the relationship between the consistency coefficients CI and ξ :

Finally, it describes the calculation process of the consistency coefficient through Eq. (10):

The lower the consistency ratio, the higher the reliability of the comparison. If the consistency ratio CR is less than or equal to 0.25, the evaluation result is consistent, indicating that this result is acceptable. If the consistency ratio exceeds 0.25, experts need to adjust the comparative relationship between the initial evaluation indicators and calculate again according to the above steps until consistency is achieved. This process can ensure the accuracy and reliability of the evaluation results. Transforming qualitative linguistic information into quantitative data is an important step in evaluating the benefits of energy-saving transformation. Because different experts may assign different semantic values to the same indicator, this can lead to confusion in the definition. This subjectivity can affect the reliability of the evaluation results and can lead to inconsistent evaluations between different projects. In order to reduce the influence of subjectivity on the evaluation results, the research adopts trapezoidal fuzzy numbers to convert qualitative language information into quantitative data and ensures that expert scores follow logical rules through consistency test, so as to improve the reliability and consistency of the evaluation results. The operation rules of trapezoidal fuzzy numbers are similar to those of vectors, which describe the basic operations of trapezoidal fuzzy numbers and are used for fuzzy evaluation and other related calculations. Assuming two trapezoidal fuzzy numbers A 1 = ( a 1 , b 1 , c 1 , d 1 ) and A 2 = ( a 2 , b 2 , c 2 , d 2 ) existing, the expression for fuzzy evaluation addition operation is shown in Eq. (11)

The expression for fuzzy evaluation subtraction operation is shown in Eq. (12)

Eq. (13) describes the multiplication operation of the fuzzy evaluation method

Eq. (14) describes the multiplication operation of the fuzzy evaluation method

According to the fuzzy operation rules, the calculation for the fuzzy center calculation process of trapezoidal fuzzy numbers is shown in Eq. (15)

In Eq. (15), x stands for an independent variable of continuous increasing and decreasing functions of membership function, and y stands for an independent variable of continuous increasing and decreasing functions of the inverse function of the membership function. This study considers the impact of two cumulative errors, functional and cost coefficients, and Figure 4 shows 16 different combinations of measurement permutations.

Figure 4

Sixteen different permutations and combinations of measures.

From Figure 4, it can be seen that there are four criteria for determining the initial level of energy-saving transformation benefits: excellent, good, middle, and bad. Due to the fact that the value coefficient is calculated through functional coefficient and cost coefficient, there may be some errors in calculating both coefficients. Therefore, the value coefficient can be seen as a comprehensive reflection of cumulative error.

4.1 Analysis of the evaluation results of ESR benefits

When evaluating residential construction projects, experts need clear scoring rules and content to ensure the accuracy of the research, especially the relative importance of each element. Fifteen experts were selected as research subjects from relevant fields, and they reached a consensus on the importance of pursuing ESR of residential buildings and energy efficiency evaluation. This research was mainly based on individual scoring to obtain indicator weights, and then calculated the average score and weight. Experts have passed consistency tests on the ratings of each criterion layer standard and indicator layer standard, indicating that their ratings follow logical rules. Table 2 shows the detailed results of the overall weight values throughout the entire process.

After obtaining the overall weight value, to better evaluate ESR benefits, the research comprehensively considered the perspectives of project participants, industry experts, and community residents to ensure that the evaluation results truly reflect the contribution of ESR to the public interest. Therefore, the study selected 29 evaluators with different knowledge backgrounds to evaluate the ESR benefits of residential buildings. Due to the varying degrees of understanding and management methods of ESR among participants, the study used semantic evaluation values with different accuracies to accurately reflect their viewpoints. Table 3 shows the trapezoidal fuzzy numbers corresponding to different semantic evaluation values.

In Table 3, the trapezoidal fuzzy numbers evaluated as “good,” “general,” and “poor” by residents were (0.67, 1, 1, 1), (0, 0.33, 0.67, 1), and (0, 0, 0, 0.33), respectively. The trapezoidal fuzzy numbers evaluated as “good” and “general” by the participants were (0.60, 0.72, 0.80, 1) and (0.31, 0.48, 0.360, 0.72), respectively, which differed significantly from the data evaluated by residents. Meanwhile, the trapezoidal fuzzy numbers evaluated by experts as “good” were (0.65, 0.71, 0.88, 1), which were closer to the residents’ evaluation values. In addition, the trapezoidal fuzzy numbers evaluated by participants and experts as “very good” and “very poor” were very close. The above results indicated that there might be certain differences in the evaluation results of the same standard among different groups when evaluating engineering. Therefore, when conducting engineering evaluations, it should consider the evaluation opinions of different groups to ensure that these evaluating results are more objective and comprehensive. Figure 5 summarizes the evaluation information for subordinate indicators of technical effectiveness.

Figure 5

Summary of evaluation information of B1 subordinate indicators.

From Figure 5, the participants’ evaluations of C11, C12, and C13 were all above general, and the number of people who think the three indicators are very good was 6, 2, and 5, and the number of people who think the three indicators are very good was 8, 8, and 8, respectively, which is far better than the number of people who think the three indicators are average. The results showed that the participants had high satisfaction with the three indexes. In addition, experts and residents rated the three indicators between poor and very good, and most rated them as general. The results showed that experts and residents believed that there is still much room for improvement in the three indicators. After that, the fuzzy evaluation value of each functional sub-index of the residential building energy efficiency reform project was calculated. The calculation results were summarized, as shown in Table 4, and the cost coefficient of the energy efficiency reform project was determined to be 0.412. These functional and cost coefficients were applied to the value engineering formula, resulting in a value coefficient of 0.997. Based on the existing evaluation standards for the benefits of ESR in residential buildings, it was inferred that the overall benefits of the ESR project have been rated as excellent, indicating that the project had successfully achieved the reform goals. As a demonstration project for ESR in Qingdao, it has been widely recognized by the government, community residents, and other participants during and after the reform. This fully proved the feasibility and accuracy of the evaluation index system and evaluating model constructed in this study.

From Table 4, the fuzzy evaluating values of sub-indicators between the same criterion layers were not significantly different, while there were significant differences in the fuzzy evaluating values of sub-indicators between different criterion layers. Among them, the fuzzy evaluation values of each sub-indicator of the B1 criterion layer indicator were (0.523, 0.631, 0.796, 0.962), (0.634, 0.712, 0.835, 0.902), and (0.634, 0.792, 0.815, 0.938), respectively. The fuzzy evaluation values of each sub-indicator of the B2 criterion layer indicator were (0.782, 0.826, 0.902, 0.968), (0.646, 0.846, 0.895, 0.975), (0.762, 0.806, 0.914, 0.958), and (0.514, 0.658, 0.846, 1), respectively. There was a significant difference in the fuzzy evaluation values of sub-indicators between the B1 criterion layer and the B2 criterion layer. There may be differences in the evaluation of engineering quality among different groups. When conducting engineering evaluations, it should consider all parties’ opinions to ensure that the evaluation results are more accurate and fair.

4.2 Empirical comparative analysis of ESR evaluation models

The cost coefficient of project’s ESR was slightly higher than the functional coefficient. In order to make targeted improvements, it is necessary to determine the reasons for this situation. The functional coefficient was composed of weighted evaluation values of standard layer indicators B1 (technical effect), B2 (economic benefit), B3 (social benefit), and B4 (ecological benefit). The cost coefficient was composed of the weighted evaluation values of the standard layer indicators B5 (renovation cost) and B6 (management cost). The study analyzed the coefficient-weighted evaluation values of all indicators, and the results are shown in Figure 6.

Figure 6

Coefficient weighted evaluation value of ESR of the project: (a) function coefficient and (b) cost coefficient.

As shown in Figure 6, the weighted evaluation values of economic and ecological benefits were relatively high, indicating that ESR projects have effectively saved resources such as water, electricity, and coal, while achieving significant results in reducing carbon emissions and improving the ecological environment. However, the weighted evaluation values of technological and social benefits were relatively low, so future improvement measures to improve functional coefficients should be taken in these two aspects. The weighted evaluation value of renovation cost was 0.226, which was significantly higher than the weighted value of management cost of 0.183. This indicated that the cost of decision-making, design, construction, and maintenance was relatively high throughout the entire cycle of ESR, which was consistent with project’s actual situation. Therefore, reducing the cost coefficient mainly relied on reducing renovation costs. To analyze this benefit evaluation model’s effectiveness for building ESR projects based on the BWM weighting method, the evaluation index model constructed by cost analysis and life cycle analysis was compared. The evaluation scores of indicators for different models are shown in Figure 6.

From Figure 7, the average index score of this benefit evaluation model for building ESR projects based on the BWM weighting method was higher than 0.80 points. The average scores of the evaluation index models constructed for cost analysis and life cycle analysis were 0.78 and 0.57, respectively. Compared with the BWM weighting method, they decreased by 0.02 and 0.23, respectively. The effectiveness of the benefit evaluation model for building ESR projects based on the BWM weighting method was the best. In order to more comprehensively verify the long-term effectiveness and sustainability of the proposed efficiency evaluation model for building ESR projects, as well as the universality of the research results, the model was used to conduct long-term quality evaluation on the efficiency of building ESR projects in different regions, and the quality monitoring data were statistically analyzed, as shown in Table 5.

Figure 7

Index evaluation scores of different models. (a) BWM, (b) costing, and (c) life cycle.

From the data in Table 5, ESR projects in different regions and building types had excellent performance in terms of facilities and equipment perfection, plane and spatial layout rationality, and structural safety, and their scores were generally above 0.70. Among them, the residential buildings in Qingdao scored the highest in the perfection of facilities and equipment and structural safety, respectively, 0.84 and 0.89, showing good equipment maintenance and structural safety. In terms of economic benefits, office buildings in Shanghai scored the highest at 0.81, reflecting high energy efficiency. The relatively low social benefit score, generally between 0.50 and 0.70, may be related to the different impact of different building types on the lives of surrounding residents. The score of ecological benefits was generally above 0.70, and the residential buildings in Qingdao had the highest score of 0.80, which showed the long-term effect of ESR on improving ecological efficiency. The function coefficient and cost coefficient were different in different regions and building types, but the value coefficient was generally close to 1, indicating that the overall benefit evaluation grade of ESR projects was higher. The above data validate the long-term validity and sustainability of the proposed evaluation model across different regions and building types, as well as the generality of the findings.