A Modified Binary Pigeon-Inspired Algorithm for Solving the Multi-dimensional Knapsack Problem

1 Introduction

Complex optimization is a class of problems that cut across different disciplines such as engineering design, production and manufacturing systems, economics and so on. Due to their practical applicability, numerous efficient and robust computational and soft computing techniques have been proposed to solve different classes of these problems. One of the classical examples of complex optimization problems that has numerous applications in different areas is the multidimensional knapsack problem (MKP). Basically, the practical applications of MKP can be found in cryptography [10, 21, 24], warehouse location problem, production scheduling problem, assignment problem, and reliability problem [23]. In the optimization context, the MKP as a resource allocation model focused on selecting a subset of objects that generate the maximum profit, whilst satisfying the set of constraints on capacities of the knapsack. Generating efficient techniques for the MKP as one of the categories of the NP-hard problem has been subject of investigation over the past few decades by researchers in the fields of artificial intelligence and operational research. The complexity of the MKP motivated the workers in the domain to proposed several techniques that are classified into traditional-based and metaheuristic-based techniques. The traditional-based technique could be utilized to generates the exact solution for the MKP, however, its time complexity increases exponentially with the size of the problem and thus could exhibit some weaknesses when utilized for large-scale problems of high dimensionality. The common examples of the traditional-based techniques employed to tackle the MKP are branch and bound [30], dynamic programming [33] and so on. Whereas the metaheuristic-based method could be employed to produce a near-optimal solution within reasonable computational times when compared to the traditional-based method and thus makes the metaheuristic-based method to be more suitable when utilized to handle large-scale problems. Typically, this method could be classified into two: local search-based methods that are designed to handle a single solution at a time and some examples of local search-based methods utilized for the MKP are simulated annealing [11], tabu search [17]; [39]. Similarly, the population-based methods, developed to tackle many solutions at a time are categorized into Evolutionary-based Algorithms (EAs) and Swarm Intelligence-based (SI) algorithms. Few of examples of the population-based algorithms that have been successfully utilized, modified and hybridized to solve different combinatorial optimization and engineering applications like MKP are ant algorithms [18], artificial bee colony algorithm [31], bat algorithm [45], cuckoo search algorithm [13], flower pollination algorithm [1], fruit fly optimization algorithm [26], monkey algorithm [44], differential evolution algorithm [32], genetic algorithm [28], harmony search algorithm [20]. particle swarm optimization [4, 8, 9, 15], symbiotic organisms search algorithm [38], wind-driven optimization [43]. It is worthy of mentioning that from the literature, numerous swarm-intelligence algorithmic techniques have successfully been proposed, modified and hybridized to tackled the different formulations of 0 - 1 multidimensional knapsack problem. Summarily, the SI - based algorithms have the following advantages which include: simplicity, highly adaptable, highly scalable, self-organized, flexible and collective robustness. Due to these advantages, many studies have proposed their usage in solving complex problems in the field of artificial intelligence and operational research [5, 6, 29, 34, 35, 36]. However, few limitations trace to SI - Based algorithms are time-critical applications, parameter tuning, premature convergence and stagnation in local optimum [2].

The pigeon-inspired optimization (PIO) algorithm is a category of swarm intelligence algorithm that is newly introduced to tackle the air robot path planning proposed in 2014 by Duan and Qiao [12]. It is a stochastic nature-inspired technique that has proven to be an easy but powerful population-based search technique inspired by the behavior of a swarm of pigeons. Homing pigeons can easily locate their homes in accordance with three homing operators: magnetic field, sun, and landmarks. In the context of this algorithm, a map and compass model is formulated according to the magnetic field and sun, while the presentation .of the landmark operator model is done based on landmarks [40]. Note that pigeons perceived the magnetic field through the nose from the magnetic particles taken to the brain by trigeminal nerve [25]. It is observed that the pigeons probably employ the usage of different navigational tools during diverse parts of their journey. At the initial stage of the journey, each pigeon could possibly depend more on compass-like tools, while in the middle and sometimes switch to landmarks in order to reexamine their routes and thus makes amendments [14].Note that the PIO algorithm has been successfully utilized to tackle many complex optimization problems like orbital space-craft formation reconfiguration [41], unmanned aerial vehicles [16, 42]. Studies have proven that the PIO algorithm had a better or comparable performance with some of the existing nature-inspired algorithms like the artificial bee colony, genetic algorithm, particle swarm optimization [27]. In recent time, the binary version of the classical PIO is proposed to tackle the MKP in [7] where its performance is better than some of the existing methods. However, from the experimental results, it can be observed that the BPIO converges prematurely due to lost diversity during the search activities.

The main motivation of this paper is to modify the BPIO algorithm with integration of simple crossover element of evolutionary algorithm to improve the performance. It is worthy to mention that the contribution of this research is in two folds which include:

1. Modification of Landmark operator of the BPIO to improve the diversity of the search space and exploring the interaction between pigeons to navigate the most promising regions of the search space effectively.

2. " The performance of the proposed modified is evaluated on the standard MKP benchmark datasets published by Operations Research Library (OR-Library).

Therefore, this paper presents a modified version binary pigeon-inspired optimization (Modified-BPIO) algorithm in which its landmark operator is modifies by the incorporation of crossover concept from the evolutionary algorithm when utilized to solve the 0-1 multidimensional knapsack problem. The performance of the modified-BPIO is evaluated on the standard MKP benchmark datasets published by Operations Research Library (OR-Library). Experimentally, the computational results show that the performance of the modified-BPIO is better when compared with some of the existing nature-inspired algorithms that worked on the same dataset.

The rest of the paper is organized as follows: Section 2 presents the description of the MKP, while the description of the basic PIO is given in Section 3, this is followed by the proposed binary approach as discusses with detailed information in Section 4. Next, the experimental results and discussions are given in Section 5. Finally, conclusions are considered in Section 6.

2 Multidimensional Knapsack problem (MKP) formulation

The MKP consists of a set of m knapsacks with a set of m capacities ^c = {c_i|i = 0, . .. , m − 1} , and a set of n entities ^e = {e_i|i = 0, . .. , n − 1}. The binary variables X_i(i = 0, . .. , n − 1) correspond to the chosen items to be carried in m knapsacks. The X_i assumes value of 1 if entity i is in the knapsack and 0 otherwise. Each item e_i has an associated profit P_i ≥ 0 and weight W_ij ≥ 0 for each knapsack j. The goal is to find the best combination of n entities by maximizing the sum of profits P_i multiplied by the binary variable X_i, which is mathematically represented as shown in Eq. (1).

Their constraints are the capacity C_j ≥ 0 of each knapsack. Therefore, the sum of the values of X_i multiplied by W_ij must be less than or equal to C_j as given in Eq. (2). Note that this formulation is adopted from [3].

3 Pigeon-Inspired Optimization Algorithm

PIO is a novel nature-inspired, metaheuristic algorithm that has been utilized for solving global optimization problems. It is based on imitating the natural homing pigeon behavior. Studies on the behavior of pigeon in the recent time, have shown that the pigeon can follow their path using some landmark features such as like main terrains, railways, and rivers rather than move directly for their destination. In the optimization context, the migration of pigeons can be formulated using the two mathematical models: map and compass operator, and landmark operator.

4 Modified-BPIO Optimization for the MKP

In this section, the proposed Modified-BPIO version that is based on the integration of a crossover strategy within the landmark operation of the BPIO for solving the MKP is presented. The solution approach utilized in order to achieve the aim of this paper is presented in the next subsections:

4.2 Initialization of the Pigeons Memory (PM)

The Pigeon Memory (PM) is referred to as the allocation of memory space of size where each row consists of a solution vector representing an MKP solution as in (10).

(10) P M = x 1 1 x 1 2 ⋯ x 1 N x 2 1 x 2 2 ⋯ x 2 N ⋮ ⋮ ⋱ ⋮ x P N 1 x P N 2 ⋯ x P N N f x 1 f x 2 ⋮ f x P N .

Note that for the problem under consideration, the pigeons in PM are generated using a 0-1 binary representation which is an obvious choice since it represents the underlying 0-1 integer variables. Therefore, this study utilizes a n-bit binary string representation, where n is the number of variables in the MKP, a value of 0 or 1 at the i^th path means that or 1 in the MKP solution, respectively. Figure 1 shows the binary representation of each pigeon (i.e. solution) in the search space for the MKP.

Figure 1

MKP solution representation

It is worthy of mentioning that in some case the solution (i.e. pigeon) generated based on the above procedure may not be feasible. Thus in order to avoid the utilization of such solutions in the population, a penalty function method is employed. Therefore, the fitness cost function is given in Eq. (11)

(11) fitness(S)=∑i=1npis[ i ]×Pen[ i ]

Where Pen is a penalty cost for an infeasible pigeon.

Accordingly, a population of pigeon (i.e. the population of solution) is initialized randomly with their paths and velocities. Then, fitness cost of each pigeon are evaluated and sorted in ascending order in PM based to their fitness costs (i.e. f(x₁) ≥ f(x₂) ≥ . .. ≥ f(x_PN)) to obtain the global best path.

5 Time complexity of the proposed Modified BPIO algorithm

The time complexity of the proposed BPIO algorithm is calculated based on section 4 as itemized in Table 1. Note that the computational time required to calculate the objective function is neglected because it is different from one problem to another. As shown in Table 1, the time complexity of the proposed Modified BPIO algorithm is O(MIN × C_R × MC_R × d). TThe step that requires a large time complexity in the algorithm is Step 4.3 that takes more computational time

6 Computational Experiments, Results, and Discussions

In this section, the performance of the proposed Modified-BPIO for solving multidimensional knapsack problem is presented and it is coded using Visual Basic.net and run on a Personal Computer with Intel Core i3 1.9 GHz, 4 GB memory running on Windows 10. The MKP benchmarks obtained from the OR library that is utilized to evaluate the proposed Modified-BPIO are one small and two big data sets namely: MKNAP 1, MK-NAPCB 1 and MKNAPCB 4. Note that seven instances from a small dataset and ten problem instances from both big datasets are utilized in the evaluation. The characteristics of these datasets are provided in Tables 3 and 2. As shown in Table 1, the first row shows the problem index, while the second row indicates the size of the problem i.e. number of objects and lastly, the third row represents the number of knapsack dimensions.

Similarly, the characteristic of the problem instances MKNNAPCB 1 and MKNNAPCB 4 are provided in Table 3,where column 1 and 2 represents the instance name and the problem size for the MKNNAPCB 1 respectively while column 3 and 4 shows the instance name and the problem size for the MKNNAPCB 4 respectively

6.1 Experimental Design

The computational experiments are designed to study the influence of integrating the crossover component to the landmark operator in the Modified-BPIO, it is worthy of mentioning that three varying values crossover parameter are employed in the experiments. The parameter settings were chosen carefully based on our preliminary experiments over multidimensional knapsack problems. The settings of these parameters should not be considered as the optimal set of values but a generalised set of values since the performance of the proposed algorithm is fairly well over the test problems. The detailed parameter settings for the Modified-BPIO such as using three convergence scenario is provided in Table 4. Similarly, other values of the remaining parameters are adopted from [7]. For instance, the population size (P_N), map and compass operator rate (MC_R) and the maximum iterations number (MIN) are fixed at 250, 0.5 and 1500 respectively.

6.2 Experimental results

In this section, experimental results of the Modified-BPIO using the three experimental scenarios are provided in Table 5, 6 and 7, where the values in the fourth to sixth columns of these tables represent the fitness values of 10 runs (highest is best). For each experimental scenario, the best results and mean of each problem instance over 10 runs are provided. The best results obtained by the proposed methods for each instance of the three datasets are highlighted in bold. As shown in Table 4, the three experimental scenarios of the Modified-BPIO achieved equal results in virtually all the instances of the smallest dataset except in instance 7, where Case 3 obtained the best results.

Similarly, as presented in Table 6 and 7, the performance of the proposed method for each experimental scenarios for the two hard datasets show that Case 2 with a crossover rate of 0.2 achieved best results in six instances of the MKNAPCB 1 dataset while Case 3 with crossover rate 0.3 came 2 by obtained best results in the remaining 4 instance. Finally, on MKNAPCB 4 dataset, Case 2 of the Modified-BPIO achieved best results in 5 instances of the dataset, while Case 3 and 1 obtained best results in 4 and 2 instances of the dataset respectively.

7 Comparative Analysis with Existing Techniques

The results produced by the proposed Modified-BPIO is compared with other existing techniques that worked on the same MKP benchmark, which are Binary cuckoo search algorithm BCS [13], Standard binary particle swarm optimization with penalty function PSO-P [19], quantum inspired cuckoo search QICSA [22] and a binary pigeon-inspired optimization algorithm Binary-PIO [7]. The performance of the Modified-BPIO is comparable to the other existing techniques in the small instance of the MKP benchmark. Virtually, all the methods obtained optimal or near-optimal values for all the instances of the MKNAP 1 as provided in Table 8.

Furthermore, Tables 9 shows the results of the proposed Modified-BPIO in comparison with these techniques from the literature. As shown in Table 9, the best results are highlighted in bold. Apparently, the Modified-PIO is able to obtain feasible solutions for all problem instances of both datasets. Furthermore, the proposed technique obtained best results in nine out of ten instances of the MKNPACB 1 dataset when compared with previous methods, however, the BPIO achieved the best result in the remaining one instance (i.e. in MKNAPCB 1 5.100.10). Similarly, the proposed Modified-BPIO achieved the first rank in seven out of 10 instances of the MKNAPCB 4 dataset while the BPIO came second in the remaining problem instances. This shows that modification of the landmark component of the BPIO aided the algorithm to navigate the solution space rigorously to obtain a good solution.

To show the performance of the proposed Modified-BPIO against the BPIO, a Wilcoxon Signed Rank is presented in Table 10 where the R+, R-, and p-value are computed for all the pairwise comparisons concerning Modified-BPIO. As shown in Table 10, the Modified-BPIO shows a significant improvement over BPIO, with a level of significance less than α = 0.01 on MKNAPCP 1 instance while their performance is relatively the same on MKNAPCP 4. Note that the two algorithms (i.e. Modified-BPIO and BPIO) used the same number of instances thus makes the Wilcoxon Signed Rank easier to test.

8 Conclusion

This paper presents a new modified swarm-intelligence method based on pigeon-inspired optimization algorithm for solving the multidimensional knapsack problem (MKP). The PIO is a class of swarm intelligence population-based algorithm that is proposed for continuous optimization problem and recently adopted to cope with the nature of 0 - 1 multidimensional knapsack problem. Experimentally, the BPIO lost the diversity of the solution space quickly during the search operations. In a bid to alleviate this problem, the crossover operator is integrated after the landmark operator of the BPIO. The performance of the Modified-BPIO is evaluated on some MKP benchmarks taken from OR-Library. The results of the experiments proved that the Modified-BPIO is better than existing algorithms that worked on the problem. Note that the proposed Modified-BPIO is tailored for solving the MKP only, in order to justify its performance, further investigation on its performance to other discrete optimization problems like traveling thief problem, timetabling and scheduling problem is very necessary as one of the future research. More so, the performance of the Modified-BPIO algorithm could be investigated for the ambient assisted living application and finally, the algorithm could be hybridized with expectation-maximization and k-means algorithm for clustering problems. Finally, the usage of late acceptance strategy in the map and compass operator of the Modified-BPIO can be investigated for its advantages in escaping local optima.