Self-optimization examination system based on improved particle swarm optimization

2.1 Discussion

The massive open online courses (MOOC) established by some famous colleges all around would become popular increasingly in recent years. The purpose of the MOOC is to share the excellent education resources of famous colleges with many students in the world through the Internet. The students can learn the newest knowledge and technology the computer and the Internet, instead of participating in the courses in these famous universities face to face. The most distinguishing feature of MOOC is that there are a huge number of page views every day. It is a challenge for teachers to master the learning status and learning quality of each student in a timely manner under this new teaching mode.

The examination system is a significant part of educational assistance instruction and is usually to track the study situation of every student by the periodical test. The development and maintenance of the examination system need not only the technological support of programmers but also assistance coming from some educational experts. The testing results reflect the study status of every student hardly if the examination system lacks regular maintenance, especially to the content and attributes adjustment in the examination-question database. It is impossible to maintain the examination-question database by the people in the MOOC timely, because the maintenance is complex and tremendous. These attributes about every question adjusted by the educational experts are affected by the subjective consciousness more or less and the effect of the examination paper according to these attributes may fall. It is not difficult to imagine the consequence of using this examination system to evaluate the learning status of the students who assimilate knowledge from the MOOC.

Artificial intelligence (AI) is a process to simulate human beings’ consciousness and thinking. Through AI technology, machines or computers have self-improvement and self-repairment abilities without the help of a human. AI technology has been applied in many fields and made plenty of research productions, which is an important direction for future computer development. The intelligent examination system not only has the ability to automatically compose a testing sheet under some special requirements but also can dynamically adjust the parameters of the testing questions and the testing contents of every paper according to the testing results lacking help from the teachers or educational experts. The advantages of the intelligent system are to save the high cost and time of maintenance and avoid excessive dependence on the educational experts elevating the testing objectivity.

2.2 Related work

The research on the design and development of the examination system has lasted for many years. Many professors or experts have made substantial research studies on the innovation of the system. van ver Linden proposed a test design based on the item response theory [1]. In this model, a linear programming algorithm is to maximize the test content under several demands. A computer-assisted testing and education system used on the web is developed by Chou and applied in science and engineering education [2]. A new test sheet composition method based on the tabu search and biogeography-based optimization is presented by Hong Duan and his colleagues [3]. The experiment testified to the improvement in the speed and success rate in composing testing sheets. A composition test-sheet optimization under the multiple assessment requirements and a mixed integer mathematical model are illustrated by Gwo-Jen Hwang and his team [4]. Gui-Xia designed a new composition examination sheets method based on an improved genetic algorithm that can solve multi-objects and multi-constraints problems [5]. An auto-composing test paper algorithm based on the ant colony method is put forward to solve the problem of how to compose a high-quality test sheet [6]. Yin et al. Chang took advantage of the particle swarm optimization to realize a composing examination sheets algorithm [7]. Fragulis et al. describes a dynamic examination application plugin based on the philosophy, which is an open-source software. The system can choose different questions in the pool of questions randomly according to the demands of the users and the students can be tested by the random exam [8]. Nandini and Maheswari applied semantic relation features to develop an online examination system that can assess descriptive answers automatically, and the system provided the score to show the understanding level of the subject [9]. Rohatgi and Dwivedi implement a testing web service with a regression test selection based on a regression testing framework [10]. Tekin and his colleagues designed a personalized online education system that is different from the existing system and is not one-size-fits-all [11]. Liu presented an improved particle swarm optimization to train the fuzzy neural network [12]. Su and Huang proposed a deep learning-based method to recommend the personalized and most effective anti-cancer drugs [13]. Zhang and Piao presented a non-dominated sorting genetic strategy to realize local and edge parallel processing to reduce delay and energy consumption in practical applications [14]. Ni and Zhang proposed a new architecture ApproxECIoT to adjust the size of the sample stratum according to the variance of each stratum while maintaining the given memory budget [15]. An evolutionary game algorithm based on reinforcement learning in practical application scenarios is designed and verified the performance [16]. A personalized web-based education system and parallel testing system are designed to satisfy some different demands [17,18,19,20,21].

4.1 Particle swarm optimization

Particle swarm optimization (PSO) algorithm is a successful case that evolved from biosimulation and is a significant application to solve optimization problems. In 1995, Professor Kenndy and Eberhart found some regular rules by observing some activities when birds search for food over a long period of time and designing the PSO algorithm by simulating the searching activity of the birds. The particle swarm optimization has been applied to many fields successfully and achieved many significant results. Some near-optimal solutions may solve some problems or phenomena that cannot be solved in real society through the PSO algorithm [22]. The PSO’s advantages include higher speed in optimization and easier realization in application mainly. With these advantages, particle swarm optimization has become a study spot and has been applied in many fields successfully, in which the traditional methods are hard to solve or the optimal results have some areas to be promoted. Some researchers have made massive innovations to optimize the PSO according to the variety of application areas. Zhang et al. presented an adaptive BBPSO by adding an adaptive disturbance variable to elevate the variety of swarm. A variable-size cooperative coevolutionary particle swarm optimization algorithm is proposed by Song to solve multi-target optimization [23]. Zhang et al. reported that cost-based feature selection problems can be solved by a multi-objective PSO [24]. These particle swarm optimizations optimized by different methods have been applied in many fields successfully and achieved many significant results.

4.2 Self-optimization examination system based on improved PSO

A self-optimization examination system based on improved particle swarm optimization (SOESIPSO) is presented to ameliorate the accuracy of the attributes about every question that decided the questions’ quality according to the testing results without the experts’ support. The examination questions’ quality is a significant part of the examination system and guides the composing examination paper algorithm to constitute a testing paper. A testing paper arrives at a high quality hardly without the high-grade examination questions even though the efficiency and quality of the composing testing paper algorithm are excellent. The SOESIPSO can efficiently elevate the evaluating quality of the testing paper and avoid expensive testing system maintenance.

4.2.3 Global optimum and local optimum

There are two significant elements that are global optimum and local optimum in the particle swarm optimization. The global optimum is the best value in the particle swarm during the optimum process and the local optimums are the highest value in the optimization process for every particle. The global optimum is only one in the PSO but the number of the local optimum equals the particles in the swarm. The function of two elements is playing an important role in the optimization quality and speed.

The research studies about the improvement of the particle swam optimization algorithm concentrate on the local optimums usually, because they focus on the individual optimum excessively making the optimizational result fall into local pitfall easily. A modified particle swarm optimization based on the dynamic k-means algorithm is presented in order to surmount the defect in the local optimum of the PSO. These rules deciding the k are various, although some researchers had applied the k-means to improve the particle swarm optimization in the past research studies. In this article, the concrete method computing the local optimum is to average the local optimums of k particles that are nearest to the target particle and the difference rates between them are less than 5% shown in the formula (16).

(16) Difference Rate i = ∣ LocalBest target − LocalBest j ∣ LocalBest target .

The difference rates between target particle and every particle in the nearest k particles should be calculated. If the difference rate is less than 5%, the particle will be considered into the average range to compute the local optimum of the target particle shown at (17). The value of the k is 5 percent of the total amount of the particle swarm and should be rounded off.

(17) Local Best j = ∑ i = 1 k Local Best i k .

The global best value in the SOESIPSO is the minimum value in the absolute difference in the discrimination, difficulty, time, and reliability.

The Optimization process of the self-optimization examination system based on improved particle swarm optimization is shown in Figure 1.

Figure 1

Optimization processes.

5.1 Testing question database

The experiments in this paper were accomplished on the examination questions library about Database Development, which imparts the fundamental operation and maintenance of the database mainly. The type in the examination questions library includes the only-choice question, multi-choice question, fill-in question, and operational question. Table 1 shows the concrete distribution of the testing question type.

Chart (a) reflects the examination types in a testing paper, which consists of five types of examination questions shown in Figure 2. Among them, operating question accounts for the most outstanding share, 50%. The multiple-choice question takes up 1/5. The other questions occupy the same proportions.

Figure 2

The constitution in a testing paper: (a) examination-type constitution; (b) examination-difficulty constitution.

The illustration of the difficulty distribution highlights the middle problems from the second diagram shown in Figure 2(b), covering half of the question types. In addition, the little easy questions and the little difficulty (15% respectively), the three items make up 4/5 of all the questions for an examination. The remaining 1/5 fells into the easy and difficult, with a percentage of 10%, respectively.

5.2 The performance of the SOESIPSO

There are two experiments will be shown in this section to testify to the optimizational performance of the SOESIPSO. In the first experiment, the PSO based on the dynamic k-means algorithm is used to optimize attributes of the questions in the examination questions library with various parameter combinations (C1 = C2 = 2, C1 = C2 = 1.5, C1 = 2, and C2 = 1.5 or C1 = 1.5 and C2 = 2 with rand = 0.3 or rand = 0.5) and the parameters with the best performance are selected through comparison. The second experiment is to validate the improved effects of the SOESIPSO by comparing the performances of the self-optimization examination system with the improved PSO and the unimproved one.

The composing examination paper algorithm is a random selection method, which generates a testing paper by randomly discovering testing questions under the condition of satisfying all of the constraints. These parameters in the examination questions library are assigned randomly from 0 to 1 without the help of educational experts. There are 30 particles in the particle swarm. The experiment platform is some personal computers with an Inter(R) Core™ i5-3230M CPU, 2.60 GHz, and 16 GB RAM. The programs used in this experiment are finished in Java.

For the analysis of the optimizing performance through the training process, 50 global optimums with the best fitness in particle swarm are selected in every 20 generations from 1 to 1,000 generations for every parameter combination shown in Figure 3.

Figure 3

The estimation performances of the SOESIPSO with different parameters: (a) performances by C1 = C2 = 1.5, rand = 0.3; (b) performances by C1 = C2 = 1.5, rand = 0.5; (c) performances by C1 = C2 = 2, rand = 0.3; (d) performances by C1 = C2 = 2, rand = 0.5; (e) performances by C1 = 1.5, C2 = 2, rand = 0.3; (f) performances by C1 = 1.5, C2 = 2, rand = 0.5; (g) performances by C1 = 2, C2 = 1.5, rand = 0.3; (h) performances by C1 = 2, C2 = 1.5, rand = 0.3.

Figure 3(h) and (g) show the performance of the evaluation function in a testing system when the parameter C1 = 2 and C2 = 1.5 and rand = 0.3 or rand = 0.5. The evaluation value clearly decreased from 47 to about 15 before about 630 generations, which is shown in Figure 3(h) and (g). As the optimization continues, the evaluation value is around 13 means the study power is in a stable situation without further decline until 1,000 generations.

Like the optimizing process of the C1 and C2 assigned 2 and 1.5, respectively, the evaluation function is in the stable learning stage with a declining evaluation value in the initial stage when the C1 is 1.5 and C2 is 2, which is shown in Figure 3(e) and (f). After the optimization arrives to about 800 generations which is later than the generation arriving at the stable situation when the parameter C1 and C2, respectively, are equal to 2 and 1.5, the evaluation function begins to stabilize, and the evaluation value at around 15, plus some rebounds from time to time.

From the above analysis and Figure 3 from (e) to (h), we can see that the SOESIPSO with C1 = 1.5 and C2 = 2 or C1 = 2 and C2 = 1.5 can optimize the testing papers to some extent and the optimization speed of the C1 = 2 and C2 =1.5 is faster than the C1 = 1.5 and C2 = 2 in the initial stage; however, the particle swarm optimization cannot continually optimize the evaluation function when the best fitness just achieved around from 13 to 15, which is approximate optimization instead of full. Therefore, these parameters combination C1 = 1.5 and C2 = 2 or C1 = 2 and C2 = 1.5 do not be considered by the SOESIPSO.

When the C1 and C2 are equal to 2, the optimizing speed is comparatively stable and the evaluation value is about 41 before 400 generations. The speed of the optimization is clearly accelerated from 360 generations to about 840 generations (excepting unusual phenomenon) and the evaluation value evidently decreased from around 40 to 15 as shown in Figure 3(c)–(d). After the 840 generations, the evaluation function tends to be in a near stable situation, and the optimizing value arrives at 12 around, accompanying some little zigzag phenomenon.

Figure 3(a) and (b) show the optimizing condition of the SOESIPSO using the C1 = C2 = 1.5. Before the 380 generations, the learning effect of the SOESIPSO is not evident and the best fitness of the evaluation function averages at about 41; that is, the stable period is larger than the parameter C1 and C2 which are equal to 2. The global optimum in the particle swarm is almost stably lessened to 10 after the 380 generation until the 1,000 generation accompanied with little slight rebounds from time to time.

There are two differences between the SOESIPSO with C1 = C2 = 2 and the C1 = C2 = 1.5 in the training 1,000 generations. The first one is the study ability. The optimization speed of the SOESIPSO with C1 = C2 = 2 is faster than the C1 = C2 = 1.5 before 380 generations. For example, the global optimum is around 39 when the C1 = C2 = 2 in the 380 generations but the C1 = C2 = 1.5 is around 41 at the same generation. The second difference is the best fitness of the evaluation function steadily reduced until 1,000 generations after the initial learning stage when the parameters C1 is 1.5 and equal to C2, but the best fitness of the SOESIPSO with the parameters both C1 and C2 equal to 2 is hard to keep decreasing after the 840 generations.

Because the volatility of the best fitness decreases when the parameter rand is initially assigned to 0.5 is higher than the rand which is initialized to 0.3 in the optimization process without considering whether the C1 is equal to C2, the rand assigned to 0.5 is eliminated from consideration.

Therefore, a set of best parameters (C1 = 1.5, C2 = 1.5, and rand = 0.3) will be selected by comparing the difference performance in 1,000 generations to further experiment from the above of all the analysis.

5.3 Comparison of the optimizing results

An experiment is designed to testify the performance of the SOESIPSO by comparing optimization results among the PSO, the PSO based on the k-means, and the PSO based on the dynamic k-means. Both the C1 and C2 are equal to 1.5 and the rand = 3 in self-optimization examination system. The results of the comparative experiment are displayed in Figure 4(a) and (b) separately in consideration of the showing effect.

Figure 4

Performance comparison: (a) performance before 1,000 generations; (b) performance from 1,000 to 2,000 generations.

The comparative performances of the self-optimization examination system (SOES) based on different PSO algorithms before 1,000 generations are shown in Figure 3(a). The performances of these self-optimization examination systems are not different clearly before 260 generations in which the optimal results are not obvious and these best-values rebound around 43. The performance of the self-optimization examination system based on the dynamic k-means begins to diversify first and the optimization speed is faster than the others from the 260 generations. At the same time, the performance of the SOES based on the PSO is not stable and the fluctuation is great, although the changing tendency of best value is declining. The optimization result of the SOES based on the k-means PSO begins to decline clearly until about 460 generations and the optimization speed is similar to the SOES based on the dynamic k-means from the 460 generations to the 800 generations. The global optimums of three self-optimization examination systems based on different PSOs are 10.5, 4.99, and 3.53, respectively, when the training optimization has arrived at 1,000 generations.

Figure 4(b) displays the optimization performance of the SOES based on various PSOs from the 1,000 generations to the 2,000 generations. The global optimum is less than 1 which is an acceptable result when the experiment optimized by the PSO based on the dynamic k-means arrived at 1,262 generations, but the PSO and the PSO based on k-means need 1,819 generations and 1,376 generations, respectively, reaching acceptable results in Figure 4(b). The optimization performance of the PSO based on the dynamic k-means is superior to the PSO and the PSO based on the k-means, although the optimization speeds of these algorithms are faster than the former and the global optimum has dropped from 10.5 to 0.5 and 4.99 to 0.12 with little zigzag phenomenon, respectively, in 1,000 generations. The optimization values of the particle swarm optimization based on the dynamic k-means are from 0.97 to 0.07 stable when the optimization generation is from 1,262 to 2,000.

From the above analysis, the particle swarm optimization based on the dynamic k-means can effectively improve the capability of the evaluation function in the computer testing system, and the optimization effects and speed are superior to the particle swarm optimization and the particle swarm optimization based on the k-means. The particle swarm optimization based on the dynamic k-means algorithm makes the examination system has the self-optimization ability without the educational experts’ help.