Smoothing Approximation to the Square-Root Exact Penalty Function

Assumption 1

f(x) satisfies the following coercive condition

Under Assumption 1, there exists a box X such that G([P]) ⊂ int(X), where G([P]) is the set of global minima of problem [P], int(X) denotes the interior of the set X. Consider the following problem

Let G([P′]) denote the set of global minima of problem [P′]. Then G([P′]) = G([P]).

Assumption 2

The set G([P]) is a finite set.

Then we consider the penalty problem of the form

Let p(u)=(max{0,u})12, that is,

then

For any ϵ > 0, let

It follows that

It is easy to see that p_ϵ(u) is continuously differentiable on R. Furthermore, we can obtain that p_ϵ(u) → p(u) as ϵ→0.

Figure 1 shows the behavior of p(u) (represented by the solid line), p_0.1(u) (represented by the dot line), p_0.01(u) (represented by the broken line) and p_0.001(u) (represented by the dash and dot line).

Figure 1

The behavior of p_ϵ(u) and p(u)

Let

Then φ_{q, ϵ}(x) is continuously differentiable on Rⁿ. Consider the following smoothed optimization problem

Lemma 1

For any x ∈ X, ϵ > 0,

Proof

Note that

When g_i(x) ∈ (0, ϵ], let

since

we have

Then

This completes the proof.

Theorem 2

Let {ϵ_j → 0 be a sequence of positive numbers and assume thatx^jis a solution to min_{x ∈ X}φ_q,ϵj(x) for someq > 0. Letxbe an accumulating point of the sequence {x^j}. Thenxis an optimal solution to min_{x ∈ X}φ_q(x).

Proof

Because x^j is a solution to min_{x ∈ X}φ_{q, ϵj}(x), we have

By Lemma 1, we have

and

It follows that

Let j → 0, we have

Theorem 3

Letxj∗∈Xbe an optimal solution of problem [LOP′] andx_{q, ϵ} ∈ Xbe an optimal solution of problem [SP] for someq > 0 andϵ > 0. Then

Proof

By Lemma 1, we have

Corollary 4

Suppose that Assumptions 1 and 2 hold, and that for anyx^* ∈ G([P]), there exists a λ^* ∈ R+msuch that the pair (x^*, λ^*) satisfies the second order sufficiency condition defined in [2]. Letx^* ∈ Xbe a global solution of problem [P] andx_{q, ϵ} ∈ Xbe a global solution of problem [SP] forϵ > 0. Then there existsq^* > 0 such that for anyq > q^*,

whereq^*is defined in Corollary 2.3in [13].

Proof

By Corollary 2.3 in [13], we have that x^* is a global solution of problem [LOP′] for q > q^* for an appropriately chosen q^* > 0. Then by Theorem 3, we have

Since ∑i=1mp(gi(x∗))=0, we have

Definition 5

For ϵ > 0, a point x ∈ X is said to be an ϵ-feasible solution of problem [P], if g_i(x) ≤ ϵ for any i ∈ I.

Theorem 6

Letxj∗∈Xbe an optimal solution of problem [LOP′] andx_{q, ϵ} ∈ Xbe an optimal solution of problem [SP]. Furthermore, letxj∗be a feasible solution of problem [P] andx_{q, ϵ}be anϵ-feasible solution of problem [P], then we have

Proof

It is clear that ∑i=1mp(gi(xq∗))=0. By Theorem 3 we have

which implies

By (3), we have

Then it follows (4) and (5) that

This completes the proof.

Remark 8

From 0 < η < 1 and N > 1, we can easily obtain the sequence {ϵ_j} is decreasing to 0 and the sequence {q_j} is increasing to + ∞ as j ⟶ +∞.

Now we prove the convergence of the algorithm under mild conditions.

Theorem 9

Suppose that for anyq ∈ [q₀, +∞), ϵ ∈ (0, ϵ₀], the set

Let {xq∗} be the sequence generated by Algorithm 7. If {xq∗} has limit point, then any limit point of {xq∗} is the solution of [P] for any m ≥ 3.

Proof

Let x be any limit point of { xq∗}. Then there exists a natural number set J ⊆ N, such that xq∗ → x, j ∈ J. If we can prove that (i) x∈ G₀ and (ii) f(x) ≤ ∈ f_{x∈ G0}f(x) hold, then x is the optimal solution of [P].

1. Suppose to contrary that x ∉ G₀, then there exist δ₀ > 0, i₀∈ I and a subset J₁ ⊂ J such that g_i0 ( xq∗)≥ δ₀ for any j ∈ J₁.

   If ϵ_j ≥ g_i0 (xq∗) ≥ δ₀, it follows from Step 2 in Algorithm 7 and (3) that

   f(xj∗)+13qjϵj−1δ032+23mqjϵj12≤φqj,ϵj(xj∗)≤φqj,ϵj(x)=f(x)+23mqjϵj12,∀x∈G0.

   Thus,

   f(xj∗)+13qjϵj−1δ032≤φqj,ϵj(xj∗)−23mqjϵj12≤f(x),∀x∈G0,

   which contradicts with ϵ_j → 0 and q_j → +∞.

   If gi0(xj∗)≥δ0>ϵjorgi0(xj∗)>ϵj≥δ0, it follows from Step 2 in Algorithm 7 and (3) that

   f(xj∗)+qjδ012+qj(m−1)ϵj12≤φqj,ϵj(xj∗)≤φqj,ϵj(x)=f(x)+23mqjϵj12,∀x∈G0.

   Thus,

   f(xj∗)+qjδ012+(13m−1)qjϵj12≤φqj,ϵj(xj∗)≤f(x),∀x∈G0,

   which contradicts with ϵ_j → 0 and q_j → + ∞ when m ≥ 3.

   Then we have x ∈ G₀.

2. For any x ∈ G₀, it holds that

   f(xj∗)≤φqj,ϵj(xj∗)≤φqj,ϵj(x)=f(x),

   then f(x)≤ inf_{x∈ G0}f(x) holds. This completes the proof.

Example 1

(Example 4.2 in [14] and Example 2 in [15])

Let x⁰=(1, 1,1, 1), q₀=2.0, ϵ₀=0.1, η=0.1, N=2, the results by Algorithm 7 are shown in Table 1.

The obtained approximately optimal solution is x^*=(0.1843219, 0.8502275, 1.992824, −0.9814662) with corresponding objective function value −44.233076. From [14], the obtained approximately optimal solution is x^*=(0.169234, 0.835656, 2.008690, −0.964901 ) with corresponding objective function value −44.233582. From [15], the obtained approximately optimal solution is x^*=(0.1585001, 0.8339736, 2.014753, −0.959688) with corresponding objective function value −44.22965.

For the j’ th iteration of the algorithm, we define a constraint error e_j by

It is clear that xj∗ is ϵ-feasible to (P) when e_j < ϵ.

Example 2

(Example 4.3 in [14])

Starting point x⁰=(2,2,2), q₀=100, ϵ₀=10, η=0.01, N=10, we obtain the results by Algorithm 7 shown in Table 2.

The results show that the obtained approximate global solution is xj∗=(2.499999, 4.220525, 0.968073) with objective function value f( xj∗)=944.215671. From [14], we know that the global solution is (2.500000, 4.220720, 0.967224) with global optimal value 944.215662.

Example 3

(Example 4.5 in [14])

Let x⁰=(3,3,3,3,1,3), q₀=1000, ϵ₀=0.1, η=0.01, N=2, numerical results by Algorithm 7 are shown in Table 3.

Let x⁰=(4,4,4,4,1,4), q₀=1000, ϵ₀=0.1, η=0.01, N=2, the results by Algorithm 7 are shown in Table 4.

Let x⁰=(9,9,5,9,1,9), q₀=1000, ϵ₀=0.1, η=0.01, N=2, the results by Algorithm 7 are shown in Table 5.

The obtained approximately optimal solution is x^*=(1.739321,8.260678,0.322508,0.416937, 0.999876, 7.583312) with corresponding objective function value 117.001623. From [14], the obtained approximately optimal solution is x^*=(1.847052,8.152948,0.607878,0.244707,0.994467, 7.766359 ) with corresponding objective function value 117.038781.

One can see that the numerical results in Table 3, Table 4 and Table 5 are similar. This means that Algorithm 7 does not completely depend on how to choose a starting point in this example.

Demonstrated by the numerical examples, Algorithm 7 is applicable for finding approximate global solutions for inequality-constrained global optimization problems.

According to our experience, initially q₀ may be 0.1, 1, 5, 10, 100, 1000 or 10000, N = 2, 5, 10 or 100, and the iteration formula q = N q. The initial value of ϵ₀ may be 10, 5, 1, 0.5 or 0.1, η=0.5, 0.1, 0.05 or 0.01, and the iteration formula ϵ=ηϵ.