Home Solutions to the coupled Schrödinger systems with steep potential well and critical exponent
Article Open Access

Solutions to the coupled Schrödinger systems with steep potential well and critical exponent

  • Zongyan Lv EMAIL logo and Zhongwei Tang
Published/Copyright: September 13, 2024

Abstract

In the present paper, we consider the coupled Schrödinger systems with critical exponent:

Δ u i + λ V i ( x ) + a i u i = j = 1 d β i j u j 3 u i u i  in  R 3 , u i H 1 ( R N ) , i = 1,2 , , d ,

where d ≥ 2, β ii > 0 for every i, β ij = β ji when ij, λ > 0 is a parameter and 0 V i L loc  R N have a common bottom int  V i 1 ( 0 ) composed of 0 0 1 connected components Ω k k = 1 0 , where int  V i 1 ( 0 ) is the interior of the zero set V i 1 ( 0 ) = x R N V i ( x ) = 0 of V i . We study the existence of least energy positive solutions to this system which are trapped near the zero sets int  V i 1 ( 0 ) for λ > 0 large for weakly cooperative case β i j > 0 s m a l l and for purely competitive case β i j 0 . Besides, when d = 2, we construct a one-bump fully nontrivial solution which is localised at one prescribed components Ω k k = 1 for large λ.

2010 Mathematics Subject Classification: 35J20; 35J60; 35B09

1 Introduction

Consider the following elliptic system with d ≥ 2 equations

(1.1) Δ u i + λ V i ( x ) + a i u i = j = 1 d β i j u j 3 u i u i  in  R 3 , u i H 1 ( R N ) , i = 1,2 , , d ,

where 2 * = 2 N N 2 = 6 is the Sobolev critical exponent. Also, we assume that β ii > 0 for every i, β ij = β ji when ij. System (1.1) appears when looking for standing wave solutions Ψ i ( x , t ) = e i λ i t u i ( x ) of time dependent coupled nonlinear Schrödinger system

ı t Ψ i + Δ Ψ i + V i ( x ) Ψ + j = 1 d β i j Ψ j p Ψ i p 2 Ψ i = 0 , i = 1 , , d ,

where ı is the imaginary unit and V i ( x ) : R N R is a given potential. This system originates from many physical models: for example, Bose–Einstein condensation (see [1]). In quantum mechanics, the solutions Ψ i (i = 1, … d) are the corresponding condensate amplitudes, β ii represent self-interactions within the same component, while β ij (ij) describe the strength and type of interactions between different components u i and u j . Furthermore, β ij > 0 represents the interaction is cooperative, while when β ij < 0, interaction is competitive.

We begin by presenting the basic assumptions on the potentials V i (x) and a i :

  • (A 1) V i ( x ) C R 3 , R satisfies V i (x) ≥ 0. Ω V i = int  V i 1 ( 0 ) , i = 1, …, d, are nonempty bounded sets and have smooth boundaries. Moreover, Ω ̄ V i = V i 1 ( 0 ) , i = 1, …, d, and Ω ̄ Ω ̄ V 1 = = Ω ̄ V d .

  • (A 2) There exist M > 0 such that D V i x R 3 V i ( x ) M , i = 1, …, d, are nonempty and have finite measures.

  • (A 3) a i ∈ (−μ 1(Ω), − μ (Ω)), i = 1, …, d, where μ 1(Ω) denotes the first eigenvalue of −Δ with Dirichlet boundary conditions and

μ ( Ω ) = π 2 4 R 0 2  with  R 0 = sup R x Ω , B R ( x ) Ω . 

This type of potentials are often referred to as steep potential well when λ is large. There are enormous investigations on non-linear Schrödinger equations and Schrödinger systems with steep potential well, for instance, [2]–[13]. For the type of muti-bump solutions, we refer to [2], [4], [8], [13]. Cao and Noussair in [4] (see also Ding and Tanaka in [8]), constructed the multi-bump solutions to the nonlinear Schrödinger equation with steep potential well under the assumptions that the bottom of the potential is composed of several connected bounded domains, if λ is large enough. Moreover, they showed that the solutions are trapped in the bottom Ω of the potentials as λ → +∞.

For the Sobolev critical case, Guo and the second author in [14] considered the muti-bump solutions for the following problem:

Δ u + ( λ V ( x ) a ) u = | u | 2 * 2 u , x R N ,

where N ≥ 5, V(x) ≥ 0 and its zero sets are not empty, λ > 0 is a parameter, a > 0 small such that the operator −Δ + λV(x) − a is definite. By using local mountain pass technique combining Contraction Image Principle, they constructed the multi-bump solution. Later on in [15], using a flow argument and a combination of global linking and local linking methods, Guo and the second author proved that the sign-changing bump solutions which is trapped in a neighborhood of Ω i , for λ sufficiently large, if the zero sets of V(x) have several isolated connected components Ω1, …, Ω k such that the interior of Ω i is not empty. In such case, the a > 0 is a constant such that the operator (−Δ + λV(x) − a) might be indefinite for λ large.

In recent years, the Schrödinger systems have been widely studied by many researchers, and related results can been seen in [16]–[18]. In [17], An and Yang dealt with the following weakly coupled nonlinear Schrödinger system

Δ u 1 + a 1 ( x ) u 1 = u 1 2 p 2 u 1 + b u 1 p 2 u 2 p u 1 , x R N , Δ u 2 + a 2 ( x ) u 2 = u 2 2 p 2 u 2 + b u 2 p 2 u 1 p u 2 , x R N ,

where N 1 , b R is a coupling constant, 2 p 2 , 2 * , 2 * = 2 N / ( N 2 ) if N ≥ 3 and +∞ if N = 1, 2, a 1(x) and a 2(x) are two positive functions. By some suitable conditions and constructing creatively two types of two-dimensional mountain-pass geometries, they obtained a positive synchronized solution for |b| > 0 small and a positive segregated solution for b < 0, respectively. Later, Chen and Pistoia in [16] considered the existence of segregated non-radial solutions for nonlinear Schrödinger systems with a large number of components in a weak fully attractive or repulsive regime in presence of a suitable external radial potential. Moreover, we mention that Wang, Wang and Wei ([18]) studied the existence of the normalized solutions for the following coupled elliptic system with quadratic nonlinearity

Δ u λ 1 u = μ 1 | u | u + β u v  in  R N , Δ v λ 2 v = μ 2 | v | v + β 2 u 2  in  R N ,

where u, v satisfying the additional condition

R N u 2 d x = a 1 , R N v 2 d x = a 2 .

They proved the existence of minimizer for the system with L 2-subcritical growth (N ≤ 3). They also proved the existence results for different ranges of the coupling parameter β > 0 with L 2-supercritical growth (N = 5).

Recently, Liu, Song and Zou in [19] considered the following Schrödinger systems with Sobolev critical exponent in dimension three:

(1.2) Δ u i + a i u i = j = 1 d β i j u j 3 u i u i  in  Ω R 3 , u i H 0 1 ( Ω ) , i = 1,2 , , d .

Clearly, the corresponding energy functional of (1.2) is

(1.3) J ( u ) = 1 2 i = 1 d u i i 2 1 2 p i , j = 1 d Ω β i j u i p u j p d x ,

then the critical point of (1.3) lies on the level

(1.4) C inf u N J ( u ) ,

where the Nehari type set

N u ( H 0 1 ( Ω ) ) d : u i 0 , u i i 2 = j = 1 d Ω β i j u i 3 u j 3  for every  i = 1,2 , . . , d

and

u i i 2 Ω u i 2 + a i u i 2 d x .

In this paper, based on the results of [19], we first prove that Schrödinger systems (1.1) for weakly cooperative case (β ij > 0 small) and for purely competitive case (β ij ≤ 0) admits a least energy solution u λ which achieves C λ (defined in (1.9)) for λ > 0 large. In addition, we shall show that u λ converge as λ → ∞ towards a least energy solution u of (1.2). Furthermore, for β 12 > max{β 0, β 1} (defined in (4.6)), we manage to construct one-bump solution to the Schrödinger systems (1.1) with d = 2.

Before the statement of the main theorem, we introduce some notations first. Set

E λ , i u L 2 ( R 3 ) : R 3 | u | 2 < , R 3 V i ( x ) u 2 < .

Then by the condition (A 3), for every a i ∈ (−λ 1(Ω), − λ*(Ω)), i = 1, …, d, E λ,1, …, E λ,d are the Hilbert spaces equipped with the following inner products

u , v λ , i R 3 u v + ( λ V i ( x ) + a i ) u v d x ,

The corresponding norms are given by

u i λ , i 2 R 3 | u i | 2 + ( λ V i ( x ) + a i ) u i 2 d x .

It is easy to see that E λ , i , λ , i is a Banach space. For the convenience, We denote the Hilbert spaces E λ , i , λ , i by E λ,i . Let E λ E λ,1 × E λ,2 ×⋯ × E λ,d be the Hilbert space with the inner product.

From [20], by the assumptions (A 2), (A 3), it is easy to see that E λ is continuously embedded in H 1 R 3 for λ properly large. Moreover, there is a positive number ν 0 independent of λ such that for λ > 0 large enough,

R 3 ( | u i | 2 + ( λ V i ( x ) + a i ) u i 2 ) d x ν 0 ( R 3 | u i | 2 + u i 2 d x ) .

It is worth noting that we can choose B R 1 ( 0 ) such that Ω B R 1 ( 0 ) and take Λ1 properly large such that Λ1 M 1,i > − a i , where M 1 , i inf | x | R 1 V i ( x ) > 0 . Then for λ ≥Λ1,

(1.5) R 3 ( | u i | 2 + ( λ V i ( x ) + a i ) + u 2 ) d x R 3 | u i | 2 d x .

Throughout this paper, we always work under the following assumptions

(1.6) β i i > 0 i = 1,2 , , d , β i j = β j i i , j = 1,2 , , d , i j , p = 3 .

Note that β ij = β ji , which provides a variational structure, then the solutions of (1.1) correspond to the critical points of the C 1 – energy functional J λ : E λ R defined by

(1.7) J λ ( u ) = 1 2 i = 1 d u i λ , i 2 1 6 i , j = 1 d R 3 β i j u i 3 u j 3 d x

where u = u 1 , , u d .

We call a solution is trivial if all its components are vanishing. We call a solution is semi-trivial if there exist at least one (but not all) vanishing component. We call a solution is fully nontrivial if all of its components are nontrivial. In particular, we mainly focus on the existence of least energy positive solutions, which attain the least energy positive level

(1.8) C L E S inf J λ ( u ) : u  is a solution of  ( 1.1 )  such that  u i > 0  for all  i = 1,2 , , d

Consider the Nehari type set

N λ u E λ : u i 0 , u i λ , i 2 = j = 1 d R 3 β i j u i 3 u j 3  for every  i = 1,2 , . . , d ,

and the infimum of J λ on the set N λ

(1.9) C λ inf u N λ J λ ( u ) = inf u N λ 1 3 i = 1 d u i λ , i 2

where J λ is defined in (1.7). It is easy to see that C L E S = C λ if C λ is attained on N λ , where C L E S is defined in (1.8).

Our mian results of this paper are the following

Theorem 1.1.

Assume that (A 1), (A 2), (A 3) and (1.6) hold. There exists constant K > 0 such that

0 < β i j < K , i , j = 1,2 , , d , i j

and for λ large enough, then C is achieved by a positive u ̄ λ N λ , and the system (1.1) has a least energy positive solution. Furthermore, for any sequences λ n → +∞, { u ̄ λ n } has a subsequence converging to u ̄ such that u ̄ is a least energy positive solution of (1.2). Namely u ̄ solves (1.2) and J ( u ̄ ) = C λ (defined in (1.4)).

Remark 1.2.

For d = 1, system (1.2) reduces to the following

(1.10) Δ u + a 1 u = β 11 u 5 , u H 0 1 ( Ω ) .

From [21], [22], we learn that (1.10) is the Brezis-Nirenberg problem and (1.10) has a positive least energy solution ω C 2 ( Ω ) C ( Ω ̄ ) with energy

m β 11 : = 1 3 Ω ω 2 + a 1 ω 2 d x < 1 3 β 11 1 2 S 3 2 .

Similarly, for d = 1, system (1.1) reduces to the following problem

(1.11) Δ u 1 + λ V 1 ( x ) + a 1 u = β 11 u 1 4 u 1 , u H 1 ( R 3 ) ,

where β 11 > 0. Using the method from [20], it is easy to see that there is a least energy positive solution ω λ of (1.11) which achieves

(1.12) m λ , 1 1 2 ω λ λ , 1 2 1 6 R 3 β 11 ω λ 6 d x = 1 3 ω λ λ , 1 2 < 1 3 β 11 1 2 S 3 2 .

Moreover, for any sequences λ n → +∞, { ω λ n } has a subsequence converging to ω such that ω is a least energy positive solution of (1.10). Namely, lim λ + m λ , 1 = m β 11 .

Theorem 1.3.

Assume that (A 1), (A 2), (A 3) and (1.6) hold and

β i j 0 ,  for any  i j .

and for λ large enough, then C is achieved by a positive u ̂ λ N λ , and the system (1.1) has a least energy positive solution. Furthermore, for any sequences λ n → +∞, { u ̂ λ n } has a subsequence converging to u ̂ such that u ̂ is a least energy positive solution of (1.2). Namely u ̂ solves (1.2) and J ( u ̂ ) = C λ (defined in (1.4)).

In order to construct one-bump solution to system (1.1) with d = 2 equations in Section 4, we assume further that

(A 4) Ω consists of 0 components: Ω = Ω 1 Ω 2 Ω 0 and Ω ̄ k Ω ̄ = , for all k

Remark 1.4.

By the assumption A 4 , there is a positive number ρ > 0, such that

Ω k 3 ρ Ω 3 ρ = ,  for  k , 1 k , 0 ,

where Ω ρ x R N : dist { x , Ω } < ρ for any domain Ω R N .

Remark 1.5.

From the assumptions (A 1) − (A 3), we can see that the zero set Ω of V i (x)(i = 1, 2) is a bounded domain in R 3 and thus the operator −Δ has discrete spectrum in H 0 1 Ω k 3 ρ ( k = 1,2 , , 0 ) and we can denote its eigenvalues by 0 < μ 1 k < μ 2 k < < μ m k < , where k = 1, 2, …, 0.

(A 5) For every 1 ≤ k 0, the operator −Δ + a i defined on H 0 1 Ω k 3 ρ is positively definite. Namely, μ ̄ < a i < μ ̂ , where i = 1, 2 and

(1.13) μ ̄ min 1 k 0 μ 1 k and μ ̂ max 1 k 0 μ * k μ * k μ Ω k 3 ρ = π 2 4 R k 2  with  R k = sup R x Ω , B R ( x ) Ω k 3 ρ . 

Theorem 1.6.

Assume that (A 1), (A 2), (A 4), (A 5) and (1.6) hold. For β 12 > max{β 0, β 1} (defined in (4.6)), then for any k ∈ {1, 2, …, 0} and a 1, a 2 > 0 sufficiently small, there exits Λ > 0 such that for λ ≥ Λ, (1.1) admits a solution u λ k such that, for any sequences λ n + , u λ n k has a subsequence converging to u k such that u k ≡ 0 for x R N \ Ω k and u k is a fully nontrivial solution of (4.2). That is u k solves (4.2) and J ̃ k ( u k ) = C 0 k (defined in (4.3)) .

This paper is organised as follows. Section 2 is devoted to the proof of Theorem 1.1. Section 3 is devoted to the proof of Theorem 1.3. Finally, we give the proof of Theorem 1.6 in Section 4.

Throughout the paper we use the notation ‖u‖ to denote the H 1-norm and |u| p to denote the L p -norm. The notation ⇀ denotes weak convergence. Capital latter C stands for positive constant, which may depend on some parameters and whose precise value can change from line to line. Let S be the Sobolev best constant of D 1,2 R 3 L 6 R 3 ,

S = inf u D 1,2 R 3 \ { 0 } R 3 | u | 2 d x R 3 | u | 6 d x 1 3 ,

where D 1,2 R 3 = u L 2 R 3 : | u | L 2 R 3 with norm  u D 1,2 R 3 | u | 2 1 2 . Set

R + d = x = x 1 , , x d : x i > 0 ,  for every  i = 1,2 , , d .

For a vector X = x 1 , , x d R d , denote the transpose of X by X T and define the norm by

| X | = x 1 2 + + x d 2 .

For a subset I ⊂ {1, …, d} with |I| = q, we denote the number of elements in set I by |I| and define

u i i I = u i 1 , , u i q

where I = i 1 , , i q and i 1 < i 2 < … < i q .

2 Least energy positive solutions for the weakly cooperative case

In this section, given I ⊆ {1, 2, …, d} with |I| = q, 1 ≤ qd, we consider the following subsystem

(2.1) Δ u i + λ V i ( x ) + a i u i = j I β i j u j 3 u i u i  in  R N , i I u i H 1 ( R N ) , i I .

and define

J λ , I u I 1 2 i I u i λ , i 2 1 6 i , j I R 3 β i j u i 3 u j 3 d x , N λ , I u I E I : u i 0  and  u i λ , i 2 j I R 3 β i j u i 3 u j 3 d x = 0 , i I , C λ , I inf u I N λ , I J λ , I u I = inf u I N λ , I 1 3 i I u i λ , i 2 .

Lemma 2.1.

Take

(2.2) C ̄ = d 3 max 1 i d 1 β i i S 3 2

then for every I ⊆ {1, 2, …, d}, there holds C λ , I C ̄ .

Proof.

By the definition of N λ , I and N I , we know that N I N λ , I which implies that C λ , I C I . From [19], we see that C λ , I C I C ̄ .□

Define

(2.3) K 1 = 7 S 3 12 ( 6 C ̄ ) 2 .

Lemma 2.2.

If

β i i > 0 i = 1,2 , , d , 0 < β i j < K 1 i , j = 1,2 , , d , i j

and for every I { 1,2 , , d } , u N λ , I with J λ , I ( u ) 2 C ̄ , then there exists constant C 2 > C 1 > 0 dependent only on K 1, a i , β ii , such that

C 1 R 3 u i 6 C 2  for every  i I .

Proof.

For any u N λ , I with J λ , I ( u ) 2 C ̄ , we have i I u i λ , i 2 6 C ̄ . Therefore,

S R 3 u i 6 d x 1 3 R N | u i | 2 d x u i λ , i 2 i I u i λ , i 2 6 C ̄ ,

that is R 3 u i 6 d x C 2 . On the other hand, we have

S R 3 u i 6 d x 1 3 u i λ , i 2 = j I R 3 β i j u i 3 u j 3 d x d max i = 1,2 , , d K 1 , β i i 6 C ̄ S 3 2 R 3 u i 6 d x 1 2 ,

which yields that R 3 u i 6 C 1 .□

Before proceeding, we introduce some notations. For every I ⊆ {1, 2, …, d} with |I| = q, we define the matrix A I ( u ) = a i j ( u ) ( i , j ) I 2 by

(2.4) a i i ( u ) = 4 R 3 β i i u i 6 d x + j I , j i R 3 β i j u i 3 u j 3 d x , i I , a i j ( u ) = 3 R 3 β i j u i 3 u j 3 d x , i , j I , i j .

Set

Γ I = u E λ : A I ( u )  is strictly diagonally dominant  .

The following lemma shows that N λ , I Γ I is a natural constraint.

Lemma 2.3.

For every I ⊆ {1, 2, …, d}, the set N λ , I Γ I is a smooth manifold. Moreover, the constrained critical points of J λ,I on N λ , I Γ I are free critical points of J λ,I . In other words, N λ , I Γ I is a natural constraint.

The proof of this lemma follows a similar approach to that of Lemma 4.1 in [19], and for brevity, we omit it here.

Lemma 2.4.

Assume that

β i i > 0 i = 1,2 , , d , 0 < β i j < K 1 i , j = 1,2 , , d , i j

then we have

N λ , I u E λ : J λ , I ( u ) 2 C ̄ Γ I

Moreover, the constrained critical points of J λ,I on N λ , I satisfying J λ , I ( u ) 2 C ̄ are free critical points of J λ,I .

Proof.

u N λ , I u E λ : J λ , I ( u ) 2 C ̄ . We will prove that A I (u) is strictly diagonally dominant, that is

4 R 3 β i i u i 6 d x + j I , j i R 3 β i j u i 3 u j 3 d x 3 j I , j i R 3 β i j | u i | 3 | u j | 3 d x > 0 , i I .

Notice that β ij > 0 and u N λ , I , we only need to show

4 u i λ , i 2 6 j I , j i R 3 β i j u i 3 u j 3 d x > 0 , i I

In fact, thanks to the choice of K 1, we have

6 j I , j i R 3 β i j | u i | 3 | u j | 3 d x 6 K 1 S 3 j I , j i ( R 3 | u i | i 2 d x ) 3 2 ( R 3 | u i | i 2 d x ) 3 2 6 K 1 S 3 j I , j i u i λ , i 3 u j λ , j 3 6 K 1 S 3 ( 6 C ̄ ) 2 u i λ , i 2 7 2 u i λ , i 2 .

Thus, by Lemma 2.2 we have

4 u i λ , i 2 6 j I , j i R 3 β i j u i 3 u j 3 d x 1 2 u i λ , i 2 1 2 S R 3 u i 6 1 3 1 2 S C 1 1 3

It follows that

4 R 3 β i i u i 6 d x + j I , j i R 3 β i j u i 3 u j 3 d x 3 j I , j i R 3 β i j | u i | 3 | u j | 3 d x 1 2 S C 1 1 3 > 0

which means that A I (u) is strictly diagonally dominant. Therefore,

N λ , I u E λ : J λ , I ( u ) 2 C ̄ Γ I ,

and so

N λ , I u E λ : J λ , I ( u ) 2 C ̄ N λ , I Γ I

By Lemma 2.3 we know that the constrained critical points of J λ,I on N λ , I satisfying J λ , I ( u ) 2 C ̄ are free critical points of J λ,I . This completes the proof.□

Next, we construct a Palais–Smale sequence at level C λ , I .

Lemma 2.5.

(Existence of Palais–Smale sequence) Assume that

β i i > 0 i = 1,2 , , d , 0 < β i j < K 1 i , j = 1,2 , , d , i j .

Then for every I ⊆ {1, 2, …, d}, there exists a sequence u n N λ , I satisfying

lim n J λ , I u n = C λ , I , lim n J λ , I u n = 0 .

Proof.

By the definition of C λ , I , there exists a minimizing sequence u n N λ , I with u n = u i , n i I satisfying

(2.5) J λ , I u n C λ , I , J λ , I u n i I μ i , n G λ , i u n = o ( 1 )

where G λ , i ( u ) u i λ , i 2 R 3 j I β i j u i u j 3 3 d x .

By Lemma 2.1, we can assume that J λ , I ( u n ) 2 C ̄ for n large enough, then following Lemma 2.4 we have

(2.6) 4 β i i u i , n 6 6 + j I , j i R 3 β i j u i , n 3 u j , n 3 d x 3 j I , j i R 3 β i j | u i , n | 3 u j , n 3 d x 1 2 S C 1 1 3  for  i I .

Suppose that v n is the minimum eigenvalues of A I u n . By Gershgorin circle theorem and (2.6) we have

(2.7) v n 1 2 S C 1 1 3

where C 1 is independent on n. Note that u n N λ , I , then test the second equation in (2.5) with 0 , , u i , n , , 0 , i I and multiply by μ n = μ i , n i I , by (2.7) we have

o ( 1 ) μ n μ n A I u n μ n T v n μ n 2 1 2 S C 1 1 3 μ n 2

where A I u n is defined in (2.4). It follows that μ i,n → 0 as n → ∞. Since for every φ H 1 ( R 3 ) , G λ , i u n φ is uniformly bounded, we have J λ , I u n φ = o ( φ ) , which yields that J λ , I u n 0 in H 1 ( R 3 ) . Therefore, u n is a standard Palais–Smale sequence.□

Lemma 2.6.

(See [23]) Assume that u n u, v n v in H 1 ( R 3 ) as n → ∞ and 1 < p < + ∞. Then, up to subsequence, there holds

lim n R 3 u n p v n p u n u p v n v p | u | p | v | p = 0

Lemma 2.7.

Assume that there holds

β i i > 0 i = 1,2 , , d , β i j 0 i , j = 1,2 , , d , i j ,

then we have

C λ , I i I m λ , i I { 1 , , d }

where m λ,i is defined in (1.12).

Proof.

Without loss of generality, we only prove that C λ i = 1 d m λ , i . We will prove this statement in two steps.

Step 1: We claim that there exists a 1 , , a d R + d such that a 1 ω λ , 1 , , a d ω λ , d N λ . We define the polynomial function F : R + d R

F t 1 , , t d = J t 1 ω λ , 1 , , t d ω λ , d = 1 2 i = 1 d t i 2 ω λ , i λ , i 2 1 6 i , j = 1 d t i 3 t j 3 R 3 β i j ω λ , i 3 ω λ , j 3 d x ,

where R + d = x = x 1 , , x d : x i > 0 f o r i = 1 , , d . Following the standard steps in [19], a 1 ω λ , 1 , , a d ω λ , d N λ can be verified.

Step 2: We claim that C λ i = 1 d m λ , i . By the definition of ω λ,i and β ij > 0 for any ij we see that

C λ J λ a 1 ω λ , 1 , , a d ω λ , d 1 2 i = 1 d a i 2 ω λ , i λ , i 2 1 6 i = 1 d a i 6 R 3 β i i ω λ , i 6 d x = i = 1 d 3 2 a i 2 1 2 a i 6 m λ , i i = 1 d m λ , i .

This completes the proof.□

The following proposition will play a critical role in proving that C is achieved by a solution with d nontrivial components. Define

(2.8) K 2 = min K 1 , S 3 4 ( 6 C ̄ ) 2

where K 1 is defined in (2.3). Then we have the following energy estimate.

Proposition 2.8.

Assume that there holds

β i i > 0 i = 1,2 , , d , 0 < β i j < K 2 i , j = 1,2 , , d , i j

Given I ⊆ {1, 2, …, d}, suppose that C λ , Q is achieved by u Q for every QI, then

C λ , I min C λ , Q + i I \ Q m λ , i : Q I .

Next, we present the proof of this proposition. Without loss of generality, we fix 1 ≤ qd − 1 and prove that

(2.9) C λ C λ , { 1 , , q } + i = q + 1 d m λ , i

Before proving (2.9), let us firstly prove the following Lemma 2.9 and Lemma 2.10.

Lemma 2.9.

Assume that there holds

β i i > 0 i = 1,2 , , d , 0 < β i j < K 2 i , j = 1,2 , , d , i j .

Given 1 ≤ qd − 1, if C λ , { 1 , , q } is achieved by u q = u 1 , , u q N λ , { 1 , , q } , then

max t 1 , , t q > 0 f q t 1 , , t q = f q ( 1 , , 1 ) = C λ , { 1 , , q }

Proof.

Notice that C λ , { 1 , , q } is achieved by u q = u 1 , , u q N λ , { 1 , , q } , then by Lemma 2.1 we have J λ , { 1 , , q } u q = C λ , { 1 , , q } < 2 C ̄ . Consider the polynomial function f q : R + q R

(2.10) f q t 1 , , t q = J λ , { 1 , , q } t 1 u 1 , , t q u q 1 2 i = 1 q t i 2 u i λ , i 2 1 6 i , j = 1 q t i 3 t j 3 R 3 β i j | u i | 3 u j 3 d x

Define the matrix B q u q = b i j u q by

b i j u q = R 3 β i j | u i | 3 u j 3 d x , i , j = 1,2 , , q .

We claim that the matrix B q u q is positive definite. We will prove that B q (u) is strictly diagonally dominant, that is

(2.11) R 3 β i i u i 6 d x j = 1 j i q R 3 β i j | u i | 3 u j 3 d x > 0

Note that u q N λ , { 1 , , q } , then the inequality (2.11) is true if we show

u i λ , i 2 2 j = 1 j i q R 3 β i j u i 3 u j 3 d x > 0

By the definition of K 2 we have

2 j = 1 j i q R 3 β i j u i 3 u j 3 d x 2 K 2 S λ 3 j = 1 j i q R 3 | u i | i 2 d x 3 2 R 3 | u i | i 2 d x 3 2 2 K 2 S λ 3 j = 1 j i q u i λ , i 3 u j λ , j 3 2 K 2 S λ 3 ( 6 C ̄ λ ) 2 u i λ , i 2 < 1 2 u i λ , i 2 .

Thus,

u i λ , i 2 2 j = 1 j i q R 3 β i j u i 3 u j 3 d x 1 2 u i λ , i 2 1 2 S C 1 1 3 .

Therefore, B q (u) is strictly diagonally dominant, and so B q (u) is positive definite. It follows that there exists a constant C > 0 such that

f q t 1 , , t q = 1 2 i = 1 q t i 2 u i λ , i 2 1 6 i , j = 1 q t i 3 t j 3 R 3 β i j u i 3 u j 3 d x 1 2 i = 1 q t i 2 u i λ , i 2 C 6 i = 1 q t i 6 ,  as  | t | + ,

which implies that f q t 1 , , t q has a global maximum in R + q ̄ . Here, t = t 1 , , t q . Similar to the proof of Step 2 in Lemma 2.7, we can get that the global maximum point of f q x 1 , , x q can not belong to R + q ̄ , which implies that the global maximum point of f q x 1 , , x q is a interior point in R + q . Therefore, the global maximum point of f q is a critical point. Next, we will show that f q has a unique critical point, which proof is very similar to Lemma 2.7 in [19], so we omit it.□

Lemma 2.10.

Assume that there holds

β i i > 0 i = 1,2 , , d , 0 < β i j < K 2 i , j = 1,2 , , d , i j .

Given 1 ≤ qd − 1, if C 1 , q is attained by u q = u 1 , , u q N 1 , q , then there exists t ̃ i > 0 , i = 1 , , d , such that

t ̃ 1 u 1 , , t ̃ q u q , t ̃ q + 1 ω λ , q + 1 , , t ̃ d ω λ , d N

where ω λ,i is a least energy positive solution of (1.11).

The proof of this lemma follows a similar approach to Lemma 2.8 in [19] and for brevity, we omit it here.

Proof of Proposition 2.8.

Without loss of generality, we prove that

C λ C λ , { 1 , , q } + i = q + 1 d m λ , i

Assume that C λ , { 1 , , q } is achieved by u q = u 1 , , u q . By Lemma 2.10, there exists t ̃ 1 , , t ̃ d such that t ̃ 1 u 1 , , t ̃ q u q , t ̃ q + 1 ω λ , q + 1 , , t ̃ d ω λ , d N λ . Note that β ij > 0 for any ij, then by a direct calculation we have

(2.12) J t ̃ 1 u 1 , , t ̃ q + 1 ω λ , q + 1 , , t ̃ d ω λ , d 1 2 i = 1 q t i ̃ 2 u i λ , i 2 1 6 i , j = 1 q t i ̃ 3 t j ̃ 3 R 3 β i j u i 3 u j 3 d x + 1 2 i = q + 1 d t i ̃ 2 ω λ , i λ , i 2 1 6 i = q + 1 d t i ̃ 6 R 3 β i i ω λ , i 6 d x = : f t 1 ̃ , , t q ̃ + g t ̃ q + 1 , , t ̃ d ,

where f t 1 , , t q is defined in (2.10) and

g t q + 1 , , t d 1 2 i = q + 1 d t i 2 ω λ , i λ , i 2 1 6 i = q + 1 d t i 6 R 3 β i j ω λ , i 6 d x .

Notice that ω λ , i λ , i 2 = R 3 β i i ω λ , i 6 d x = 3 m λ , i , it is easy to show that

(2.13) g t ̃ q + 1 , , t ̃ d max t q + 1 , , t d > 0 g t q + 1 , , t d = i = q + 1 d m λ , i

By Lemma 2.9 we get that

(2.14) f t ̃ 1 , , t ̃ q max t 1 , , t q > 0 f t 1 , , t q = f ( 1 , , 1 ) = C λ , { 1 , , q } .

We deduce from (2.12)(2.14) that

C λ J λ t ̃ 1 u 1 , , t ̃ q u q , t ̃ q + 1 ω λ , q + 1 , , t ̃ d ω λ , d f t ̃ 1 , , t ̃ q + g t ̃ q + 1 , , t ̃ d C λ , { 1 , , q } + i = q + 1 d m λ , i

This completes the proof of Proposition 2.8.□

Proof of Theorem 1.1

Next, we present the proof of Theorem 1.1. Recall that m λ , i < 1 3 β i i 1 2 S 3 2 (see (1.12)) for every 1 ≤ id. Set

(2.15) α = 1 2 min 1 i d β i i 1 S 3 3 m λ , i 2 > 0 ,

then we have

(2.16) 3 m λ , i 2 < β i i 1 S 3 α , 1 i d .

Denote

K 3 = min 1 i d β i i S 3 ( 6 C ̄ ) 2 α  and  K = min K 1 , K 2 , K 3 ,

where K 1 is defined in (2.3), K 2 is defined in (2.8), α is fixed in (2.15). From now on, we assume that β ij satisfies 0 < β ij < K for any ij.□

Conclusion of the Proof of Theorem 1.1

We will proceed by mathematical induction on the number of the equations in the subsystem. Set |I| = M, that is M the number of the equations in the subsystem, and M = 1, …, d. If M = 1, by Remark 1.2, we see that Theorem 1.1 is true.

We suppose by induction hypothesis that Theorem 1.1 holds true for every level C λ , I with |I| ≤ M for some 1 ≤ Md − 1. We need prove Theorem 1.1 for C λ , I with |I| = M + 1. Without loss of generality, we will present the proof for I = {1, …, M + 1}. By induction hypothesis we know that Proposition 2.8 is true for C λ , I . For simplicity, we will write { u ̄ n } instead of { u ̄ λ , n } . By Lemma 2.5, there exists a sequence { u ̄ n } N λ , I satisfying

lim n J λ , I u ̄ n = C λ , I , lim n J λ , I u ̄ n = 0

then u ̄ i , n is uniformly bounded in H 1 ( R 3 ) , i = 1,2 , , M + 1 . Passing to subsequence, we may assume that

(2.17) u ̄ i , n u ̄ λ , i  weakly in  H 1 ( R 3 ) , u ̄ i , n u ̄ λ , i  weakly in L 2 ( R 3 ) .

It is standard to see that J λ , I ( u ̄ λ ) = 0 and

u ̄ λ , i λ , i 2 = j = 1 M + 1 R 3 β i j u ̄ λ , i 3 u ̄ λ , j 3 d x  for every  i = 1,2 , . . , M + 1

Denote σ i , n = u ̄ i , n u ̄ λ , i , i = 1,2 , , M + 1 , and so σ i,n → 0 weakly in H 1 ( R 3 ) . We deduce from (2.17) that

(2.18) u ̄ i , n λ , i 2 = σ i , n λ , i 2 + u ̄ λ , i λ , i 2 + o ( 1 ) and R 3 u ̄ i , n 2 = R 3 σ i , n 2 + R 3 u ̄ λ , i 2 + o ( 1 )

and by Lemma 2.6 we have

(2.19) R 3 u ̄ i , n 3 u ̄ j , n 3 = R 3 σ i , n 3 σ j , n 3 + R 3 u ̄ λ , i 3 u ̄ λ , j 3 + o ( 1 ) .

By (2.18) and (2.19) we have

J λ , I u ̄ n = J λ , I ( u ̄ λ ) + 1 3 i = 1 M + 1 R 3 σ i , n 2 + ( λ V i ( x ) + a i ) σ i , n 2 d x + o ( 1 ) .

Passing to subsequence, we may assume that

lim n R 3 σ i , n 2 + ( λ V i ( x ) + a i ) σ i , n 2 d x = k i 0 , i = 1,2 , , M + 1 .

Thus,

(2.20) 0 J λ , I ( u ̄ λ ) J λ , I ( u ̄ λ ) + 1 3 i = 1 M + 1 k i = lim n J λ , I u ̄ n = C λ , I

Next, we will show that all u ̄ λ , i 0,1 i M + 1 by using a contradiction argument.

Case 1: u ̄ λ , i 0 for every 1 ≤ iM + 1. Firstly, we claim that k i > 0, i = 1, 2, …, M + 1. By contradiction, without loss of generality, we assume that k 1 = 0, notice that σ 1 , n = u ̄ 1 , n , then we know that σ 1,n → 0 strongly in H 1 ( R 3 ) and u ̄ 1 , n 0 strongly in H 1 ( R 3 ) . Hence, by Sobolev inequality we have

lim n R 3 u ̄ 1 , n 6 = 0 .

On the other hand, by Lemma 2.2, we see that

lim n R 3 u ̄ 1 , n 6 C 1 > 0

which is a contradiction. Therefore, k i > 0, i = 1, 2, …, M + 1. Notice that J λ , I u ̄ n 2 C λ , I 2 C ̄ for n large enough, thus

σ i , n λ , i 2 j = 1 M + 1 R 3 ( | σ j , n | 2 + ( λ V j ( x ) + a j ) σ j , n 2 ) d x + 3 J λ , I ( u ̄ λ ) + o ( 1 ) = 3 J λ , I ( u ̄ n ) 6 C ̄ .

Hence,

(2.21) 0 < k i 6 C ̄

Since u ̄ n N λ , I and σ i , n = u ̄ i , n , then we have i = 1 M + 1 σ i , n λ , i 2 = i = 1 M + 1 u ̄ i , n λ , i 2 6 C ̄ . Therefore,

σ i , n λ , i 2 = β i i R 3 σ i , n 6 d x + j = 1 , j i M + 1 β i j R 3 σ i , n 3 σ j , n 3 d x β i i S 3 R 3 σ i , n 2 d x 3 + K S 3 R 3 σ i , n 2 3 2 j = 1 , j i M + 1 R 3 σ j , n 2 3 2 β i i S 3 σ i , n λ , i 6 + K S 3 ( 6 C ̄ ) 3 2 R 3 σ i , n 2 + o ( 1 ) 3 2 β i i S 3 σ i , n λ , i 6 + K S 3 ( 6 C ̄ ) 3 2 σ i , n λ , i 2 + o ( 1 ) 3 2

Let n → ∞, we have

k i β i i S 3 k i 3 + K S 3 ( 6 C ̄ ) 3 2 k i 3 2

Combining this with (2.21), we get

1 β i i S 3 k i 2 + K S 3 ( 6 C ̄ ) 2

Then by the definition of K, K 4 and (2.16) we get

k i 2 β i i 1 S 3 K ( 6 C ̄ ) 2 β i i β i i 1 S 3 α > 3 m λ , i 2 ,

which implies k i > 3m λ,i . By Lemma 2.7 and (2.20) we have

i = 1 M + 1 m λ , i C λ , I = lim n J λ , I u ̄ n = J λ , I ( u ̄ λ ) + 1 3 i = 1 M + 1 k i = 1 3 i = 1 M + 1 k i > i = 1 M + 1 m λ , i

that is a contradiction. Therefore, Case 1 is impossible.

Case 2: Only one component of u ̄ λ is not zero.

Without loss of generality, we assume that u ̄ λ , 1 0 , and u ̄ λ , i 0,2 i p + 1 . Similarly to Case 1, we can prove that k i > 3m λ,i > 0 for every 2 ≤ iM + 1. Notice that u ̄ λ , 1 , 0 , , 0 is a solution of (1.11), then J u ̄ λ , 1 , 0 , , 0 m λ , 1 . Combining this with Lemma 2.7 and (2.20) we know that

i = 1 M + 1 m λ , i C λ , I = lim n J λ , I u ̄ n = J λ , I u ̄ λ , 1 , 0 , , 0 + 1 3 i = 1 M + 1 k i m λ , 1 + 1 3 i = 2 M + 1 k i > i = 1 M + 1 m λ , i ,

that is a contradiction. Therefore, Case 2 is impossible.

Case 3: There are q components of u ̄ λ that are not zero, 2 ≤ qM.

Without of loss generality, we may assume that u ̄ λ , 1 , , u ̄ λ , q 0 and u ̄ λ , q + 1 , , u ̄ λ , M + 1 0 . Similarly to Case 1, we have k i > 3m λ,i , q + 1 ≤ iM + 1. Note that u ̄ λ , 1 , u ̄ λ , 2 , , u ̄ λ , q , 0 , , 0 is a solution of subsystem and u ̄ λ , 1 , u ̄ λ , 2 , , u ̄ λ , q N λ , { 1 , q } , then J λ , I ( u ̄ λ ) C λ , { 1 , q } . Combining this with Proposition 2.8 and (2.20), we have

C λ , { 1 , q } + i = q + 1 M + 1 m λ , i C λ , I = lim n J λ , I u ̄ n = J λ , I ( u ̄ λ ) + 1 3 i = 1 M + 1 k i > C λ , { 1 , q } + i = q + 1 M + 1 m λ , i

that is a contradiction. Therefore, Case 3 is impossible.

Since Case 1, Case 2 and Case 3 are impossible, then we get that all components of u ̄ λ = u ̄ λ , 1 , , u ̄ λ , M + 1 are not zero. Therefore u ̄ λ N λ , I . Combining this with (2.20) we see that

C λ , I J λ , I ( u ̄ λ ) J λ , I ( u ̄ λ ) + 1 3 i = 1 M + 1 k i = lim n J λ , I u ̄ λ , n = C λ , I ,

which yields that J λ , I ( u ̄ λ ) = C λ , I . Obviously,

u ̄ λ = u ̄ 1 , , u ̄ M + 1 N λ , I  and  J λ , I ( u ̄ λ ) = C λ , I

It follows from Lemma 2.1 and Lemma 2.4 that u ̄ λ , I is a nonnegative critical point of J λ,I , and | u ̄ λ , 1 | , , | u ̄ λ , M + 1 | is a nonnegative solution of system (2.1). By the maximum principle, we know that | u ̄ λ , i | > 0 in R 3 , 1 i M + 1 . Therefore, u ̄ λ is a least energy positive solution of subsystem (2.1) with I = {1, …, M + 1}. We proceed by repeating this step, then we obtain a least energy positive solution of subsystem (2.1) with I = {1, …, d}.

Next, we will study the asymptotic behavior of C λ as λ → +∞. Then we give the proof of the rest of the Theorem 1.1.

Claim: For every I ⊆ {1, 2, …, d}, there holds

lim λ + C λ , I = C I ,

where C I inf N I J I (defined in (1.3)),

N I u I H q : u i 0  and  Ω | u i | 2 d x j I Ω β i j u i 3 u j 3 d x = 0 , i I

and H q H 0 1 ( Ω ) q . By the definition of N λ , I and N I , we know that N I N λ , I which implies that C λ , I C I . Moreover, C λ , I is strictly increasing with respect to λ. In fact, let λ > μ and C λ , I is achieved by u N λ , I . Then J λ , I ( u ) = C λ , I , u N λ , I . Note that

u i μ , i 2 < u i λ , i 2 = j = 1 q R 3 β i j | u i | 3 u j 3 d x .

Then there exists t = (t 1, …, t I ), where 0 < t i < 1, iI such that t u N μ , I . This implies that

C μ , I J μ , I ( t u ) = 1 3 i = 1 I t i 2 u i μ , i 2 < 1 3 i = 1 I u i λ , i 2 = J λ , I ( u ) = C λ , I .

Thus the limit of C λ , I exists as λ → +∞.

Assume that lim λ + C λ , I C I . Then for any λ n → +∞ as n → +∞, we have lim λ n + C λ n , I C I . For n large enough, let u n ( u 1 , n , , u I , n ) E λ n satisfies J λ n , I u n = C λ n , I and J λ n , I u n = 0 . It is easy to see that {u n } is bounded in E λ n , namely u n λ n C for some C > 0. And {u i,n } is bounded in H 1 R 3 for iI. Then up to a subsequence, we have

(2.22) u i , n u i  in  H 1 R 3 , u i , n u i  in  L 6 R 3 , u i , n u i  in  L l o c 2 R 3 , u i , n u i  a.e. in  R 3 .

Firstly, we claim that u Ω c = 0 , Ω c x : x R N \ Ω . If not, we have u Ω c 0 . Then there exists a compact subset F ⊂ Ω c with dist{F, Ω} > 0 such that u F 0 and for any iI

F u i , n 2 d x F u i 2 d x > 0 ,  as  n .

Moreover, by assumption A 1 , there exists ϵ 0 > 0 such that V i (x) ≥ϵ 0 for any xF. then

C λ n , I = J λ n , I u n = 1 3 i I u i , n λ n , i 2 1 3 i I R N λ n V i ( x ) u i , n 2 d x + a i u i , n 2 2 1 3 i I F λ n ϵ 0 u i , n 2 d x + a i u i , n 2 2 +  as  n + .

This contradiction shows that u Ω c = 0 , by the smooth assumption on Ω we have u = ( u i ) i I H | I | ( Ω ) .

Adapting the same method, we shall proceed by mathematical induction on the number of the equations in the subsystem. Set |I| = M. If M = 1, by Remark 1.2, we see that Claim is true.

We suppose by induction hypothesis that Claim holds true for every level C λ , I with |I| ≤ M for some 1 ≤ Md − 1. We need prove Claim also holds for C λ , I with |I| = M + 1. Without loss of generality, we will present the proof for I = {1, …, M + 1}.

By induction hypothesis we know that Proposition 2.8 is true for C λ n , I . By Claim, there exists a sequence u n N λ n , I satisfying

lim n J λ n , I u n = lim n C λ n , I C I , lim n J λ n , I u n = 0

And for each Φ = ( ϕ i ) i I H | I | ( Ω ) , we have

(2.23) 0 = J λ n , I u n , Φ = i I R n u i , n ϕ i + ( λ n V i + a i ) u i , n ϕ i d x j I R 3 β i j u j , n 3 | u i , n | u i , n ϕ i d x i I Ω u i ϕ i + a 0 , i u i ϕ i d x j I Ω β i j u j 3 | u i | u i ϕ i d x = J I ( u ) , Φ ,  as  n + ,

Thus J I ( u ) = 0 . Furthermore,

J I ( u ) = J I ( u ) 1 2 J I ( u ) , u = 1 3 i = 1 I u i i 2 0 .

Denote σ i,n = u i,n u i , i = 1, 2, …, M + 1, and so σ i,n → 0 weakly in H 1 ( R 3 ) .

By Brézis–Lieb’s Lemma, we have

R N u i , n 2 d x = Ω | u i | 2 d x + R N σ i , n 2 d x + o ( 1 ) ,

From Lemma 2.6 we have

(2.24) R 3 u i , n 3 u j , n 3 d x = R 3 σ i , n 3 σ j , n 3 d x + Ω u i 3 u j 3 d x + o ( 1 ) .

and

R N ( λ n V i ( x ) + a i ) u i , n 2 d x = a i Ω u i 2 d x + R N ( λ n V i ( x ) + a i ) σ i , n 2 d x + 2 a i Ω u i σ i , n d x = a i Ω u i 2 d x + R N ( λ n V i ( x ) + a i ) σ i , n 2 d x + o ( 1 ) .

Thus, we deduce that

(2.25) u i , n λ n , i 2 = σ i , n λ n , i 2 + Ω | u i | 2 d x + o ( 1 )

and

J λ n , I u n = J I ( u ) + 1 3 i = 1 M + 1 σ i , n λ n , i 2 + o ( 1 ) .

Passing to subsequence, we may assume that

lim n R 3 σ i , n 2 + ( λ n V i ( x ) + a i ) σ i , n 2 d x = k ̂ i 0 , i = 1,2 , , M + 1 .

Thus,

(2.26) 0 J I ( u ) J I ( u ) + 1 3 i = 1 M + 1 k ̂ i = lim n J λ n , I u n = lim n C λ n , I C I

Next, repeating the proceding steps, we get that all components of u = u 1 , , u M + 1 are not zero. Therefore u N I . Combining this with (2.26) we see that

C I J I ( u ) J I ( u ) + 1 3 i = 1 M + 1 k ̂ i = lim n J λ n , I u n = lim n C λ n , I C I ,

which yields that J I ( u ) = C I . Obviously,

u ̄ = u ̄ 1 , , u ̄ M + 1 N I  and  J I ( u ̄ ) = C I

From [19], we know that u ̄ is a least energy positive solution of subsystem (2.1) with I = {1, …, d}. This completes the proof.

3 Least energy positive solutions for the purely competitive case

In this section, we consider the purely competitive case β ij ≤ 0 and present the proof of Theorem 1.3. Recall the definitions of N λ , I , J λ , I and C λ , I in Section 2, where I ⊆ {1, 2, …, d}. To prove Theorem 1.3, we need the following several fundamental lemmas.

Lemma 3.1.

Given I ⊆ {1, 2, …, d}. Assume that β ij ≤ 0 for all ij, then there exist C 3 > 0 such that for any u I N λ , I there holds

R 3 u i 6 d x C 3 , i I .

Proof.

This follows directly from u I N λ , I and

S R 3 u i 6 d x 1 3 R 3 u i , n 2 d x u i λ , i 2 β i i u i 6 6 .

Lemma 3.2.

Given I ⊆ {1, 2, …, d}, N λ , I is a smooth manifold. Moreover, the constrained critical points of J λ,I on N λ , I are free critical point of J λ,I .

Proof.

The proof of this lemma can be completed by the method analogous to that used in Lemma 2.4. Recall the proof of Lemma 2.4, the key point is to show that A I (u) is strictly diagonally dominant. However, in the purely competitive case, the result is straightforward. Since β ij < 0 and u i ≢ 0, we have

a i i ( u ) j I , j i a i j ( u ) = 4 β i i u i 6 6 + j I , j i β i j u i u j 3 3 3 j I , j i β i j u i u j 3 3 = 4 β i i u i 6 6 + 4 j I , j i β i j u i u j 3 3 = 4 u i λ , i 2 > 0 , i I ,

which implies that A I (u) is strictly diagonally dominant in the purely competitive case. Then using the same arguments as in the proof of Lemma 2.4, we can easily carry out the proof of this lemma.□

Lemma 3.3.

Given I ⊆ {1, …, d} and for any λ > Λ0, there exists a sequence u n N λ , I satisfying

lim n J λ , I u n = C λ , I , lim n J λ , I u n = 0  in  H 1 ( R 3 ) .

Proof.

The proof is similar to that of Lemma 2.5, so we only sketch it. Recall the proof of Lemma 2.5, the crucial step is to show that the inequality (2.6) holds. Since β ij ≤ 0 and u n N λ , I , by Lemma 3.1 we have

4 β i i u i , n 6 6 + j I , j i R 3 β i j u i , n 3 u j , n 3 d x 3 j I , j i R 3 β i j | u i , n | 3 | u j , n | 3 d x = 4 u i , n λ , i 2 4 S u i , n 6 2 4 S C 3 1 3 > 0  for every  i I .

The remainder of the argument is analogous to that in Lemma 2.5.□

Lemma 3.4.

Given I ⊆ {1, …, d}, if for λ large enough and

C λ , I < min C λ , Γ + 1 3 i I \ Γ β i i 1 2 S 3 2 :  for any  Γ I ,

then C λ , I is attained by J λ,I on N λ , I .

Proof.

Based on Lemma 4.3, there exists a sequence u n N λ , I satisfying

lim n J λ , I u n = C λ , I , lim n J λ , I u n = 0  in  H 1 ( R 3 ) .

Thus, u n is bounded in H 1 ( R 3 ) | I | . So, after passing to subsequence, we may assume

u i , n u i weakly in H 1 ( R 3 ) , u i , n u i strongly in L loc 2 ( R 3 ) .

By a standard argument, u = u i i I is a solution to the subsystem (1.1). We assert that u is fully nontrivial.

If the assertion is false, then we may assume that some components of u are trivial. Let Γ i I : u i 0 . Then, for each i ∈ Γ, we have u i,n → 0 in L loc 2 ( R 3 ) . By Lemma 3.1 and Sobolev inequality we see that u i , n 2 2 C , where C is independent on n.

As u n N λ , I and β ij ≤ 0, we get that

(3.1) u i , n λ , i 2 β i i u i , n 6 6 β i i S 3 u i , n 2 6 + o ( 1 ) .

On the other hand, for λ large enough, we see that

(3.2) u i , n λ , i 2 u i , n 2 2 + o ( 1 )

Combining (3.1) with (3.2) we know that β i i 1 2 S 3 2 u i , n 2 2 + o ( 1 ) for every i ∈ Γ. Since u solves (2.2), we obtain

C λ , I = lim n J λ , I u n = lim n 1 3 i Γ u i , n λ , i 2 + i Γ u i , n λ , i 2 lim inf n 1 3 i Γ u i , n λ , i 2 + i Γ u i , n 2 2 lim inf n 1 3 i Γ u i , n λ , i 2 + 1 3 i Γ β i i 1 2 S 3 2 1 3 i Γ u i λ , i 2 + 1 3 i Γ β i i 1 2 S 3 2 = J λ , I \ Γ ( u ) + 1 3 i Γ β i i 1 2 S 3 2 C λ , I \ Γ + 1 3 i Γ β i i 1 2 S 3 2 ,

which leads to a contradiction. Therefore, u is fully nontrivial. This implies that u N λ , I , and

C λ , I J ( u ) lim inf n J u n = C λ , I

Hence, J λ , I ( u ) = C λ , I . This completes the proof.□

Lemma 3.5.

Suppose that β ij ≤ 0 for all ij, for λ large enough, then

(3.3) C λ < min C λ , I + 1 3 i I β i i 1 2 S 3 2 : I { 1 , , d } .

Proof.

We proceed to prove this statement by induction on the number of equations. For the case d = 1, this statement follows from Remark 1.2. Assume that the statement is true for every subsystem with |I| ≤ d − 1. Then the statement (3.3) reduces to

(3.4) C λ < min C λ , I + 1 3 i I β i i 1 2 S 3 2 : | I | = d 1 .

Without loss of generality, we may assume that I = {1, …, d − 1}. By Lemma 3.4 and our induction hypothesis, there exists a least energy positive solution u 1 , , u d 1 to the corresponding subsystem with I = {1, …, d − 1} and J u 1 , , u d 1 = C λ , I .

For simplicity, we may assume 0 ∈ Ω and B R 0 ( 0 ) is the largest ball contained in Ω, then we take

φ ( x ) = cos π | x | 2 R 0 , x B R 0 ( 0 ) , 0 , x Ω \ B R 0 ( 0 ) ,

and set w ɛ (x) ≔ U ɛ (x)φ(x). where

U ε ( x ) U ε , 0 ( x ) = 3 ε 2 1 4 ε 2 + | x | 2 1 2 .

From [24], we can see that

(3.5) Ω w ε 2 = S 3 2 + 3 2 R 0 π 3 ε + O ε 2 , Ω w ε 6 = S 3 2 + O ε 2 , Ω w ε 2 = 2 3 π ε R 0 + O ε 2 ,

and

(3.6) Ω w ε 3 B R 0 ( 0 ) U ε 3 = C B R 0 ( 0 ) ε ε 2 + | x | 2 3 2 d x C ε 3 2 | ln ε | + O ε 3 2 .

Note that by the standard regularity theory we have u i C 0 ( Ω ̄ ) . Therefore,

(3.7) R 3 u i 3 w ε 3 = Ω u i 3 w ε 3 max x Ω ̄ u i ( x ) 3 Ω w ε 3 C ε 3 2 | ln ε | + O ε 3 2 .

To show that (3.4) holds, we need the following claim.

Claim: There exist r, R > 0 independent of ɛ and t ɛ,1, …, t ɛ,d ∈ [r, R] such that

u ε = t ε , 1 u 1 , , t ε , d 1 u d 1 , t ε , d w ε N .

The proof of Claim follows a similar approach to that of Lemma 4.1 in [19], and for brevity, we omit it here. Then

C λ J λ u ε 1 2 i = 1 d 1 t ε , i 2 u i λ , i 2 1 6 i = 1 d 1 t ε , i 6 β i i u i 6 6 1 6 i , j = 1 , j i d 1 t ε , i 3 t ε , j 3 β i j u i u j 3 3 + 1 2 t ε , d 2 w ε λ , d 2 1 6 t ε , d 6 β d d w ε 6 6 + 1 3 i = 1 d 1 R 4 t ε , d 2 β i d u i w ε 3 3 Ψ t ε , 1 , , t ε , d 1 + Φ t ε , d .

As u 1 , , u d 1 is a least energy positive solution to the corresponding subsystem, then (1, …, 1) is a critical point of Ψ. Using the same method of Propositon 4.1 in [19], we get that the critical point is unique and

max t 1 , , t d 1 > 0 Ψ t 1 , , t d 1 = Ψ ( 1 , , 1 ) = J I u 1 , , u d 1 = C λ , I .

By (3.5)(3.7) and R is independent of ɛ, we know that

1 3 i = 1 d 1 R 4 t 2 β i d u i w ε 3 3 = o ( ε ) t 2  for  ε  small enough  ,

and so

Φ ( t ) = 1 2 S 3 2 + 2 3 π R 0 a d + π 2 4 R 0 2 ε + o ( ε ) + O ε 2 t 2 1 6 β d d S 3 2 + O ε 2 t 6

Since a d λ 1 ( Ω ) , λ * ( Ω ) , where λ * ( Ω ) = π 2 4 R 0 2 , it is standard to see that max t > 0 Φ ( t ) < 1 3 β d d 1 2 S 3 2 for ɛ small enough. It follows that

C λ max t 1 , , t d 1 > 0 Ψ t 1 , , t d 1 + max t > 0 Φ ( t ) < C λ , I + 1 3 β d d 1 2 S 3 2  for  ε  small enough 

This completes the proof.□

Conclusion of the proof of Theorem 1.3

Following directly from Lemma 3.2, Lemma 3.4 and Lemma 3.5, we get that u ̂ λ = u ̂ λ , 1 , , u ̂ λ , d is a fully nontrivial solution of system (1.1) and J λ ( u ̂ λ ) = C λ . Set u ̂ λ = u ̂ λ , 1 , , u ̂ λ , d , then u ̂ λ is a nonnegative solution of system (1.1) and J λ ( u ̂ λ ) = C λ . By the maximum principle, we see that u ̂ λ is a least energy positive solution of system (1.1).

Next, we will show the asymptotic behavior of C λ as λ → +∞. Then we give the proof of the rest of the Theorem 1.3. Similar to the Claim in Theorem 1.1, we also have C λ , I C I . Moreover, C λ , I is strictly increasing with respect to λ. Thus, for any λ n → +∞ as n → +∞, we have lim λ n + C λ n , I C I . We assume that u n is such that C λ n , I is achieved, then u n λ n is bounded. Then up to a subsequence, we obtain (2.22) and u ̂ Ω c = 0 , Ω c x : x R N \ Ω .

Similarly, using the mathematical induction on the number of the equations in the subsystem and repeating the proceding process, we get that all components of u ̂ = u ̂ 1 , , u ̂ M + 1 are not zero. Therefore u ̂ N I and J I ( u ̂ ) = C I . Thus, we see that

u ̂ = u 1 , , u M + 1 N I  and  J I ( u ̂ ) = C I

From [19], we know that u ̂ is a least energy positive solution of subsystem (2.1) with I = {1, …, d}. This completes the proof.

4 One-bump solution

In this section, we will consider the existence of one-bump solutions of the following system for λ large,

(4.1) Δ u 1 + λ V 1 ( x ) + a 1 u 1 = β 11 u 1 6 + β 12 u 2 3 u 1 u 1  in  R 3 , Δ u 2 + λ V 2 ( x ) + a 2 u 2 = β 22 u 2 6 + β 21 u 1 3 u 2 u 2  in  R 3 , u i H 1 ( R N ) , i = 1,2 ,

For any k ∈ {1, 2, …, }, the following problem is somehow the limit problem of (4.1)

(4.2) Δ u 1 + a 1 u 1 = β 11 u 1 6 + β 12 u 2 3 u 1 u 1  in  Ω k , Δ u 2 + a 2 u 2 = β 22 u 2 6 + β 21 u 1 3 u 2 u 2  in  Ω k , u i H 0 1 ( Ω k ) , i = 1,2 .

Now, we define the variational functional by:

J ̃ k ( u ) 1 2 i = 1 2 Ω k | u i | 2 + a i u i 2 d x 1 6 i , j = 1 2 Ω k β i j u i 3 u j 3 d x ,

the corresponding Nehari manifold

M = u H 2 , k : u 0 , i = 1 2 Ω k | u i | 2 + a i u i 2 d x = i , j = 1 2 Ω β i j u i 3 u j 3 ,

and the level of J ̃ k

A = inf { J ̃ k ( u ) : u M } ,

where H 2 , k H 0 1 ( Ω k ) × H 0 1 ( Ω k ) .

According to the Theorem 1.2 in [19], we know that under the condition (A3) and (1.7) and β ij ≥ 0 for any ij, then A is attained and system (4.2) has a ground state solution.

Define

(4.3) C 0 k inf γ Γ 0 k max t [ 0,1 ] J ̃ k ( γ ( t ) )

where

Γ 0 k = γ C [ 0,1 ] , H 2 , k : γ ( 0 ) = 0 , γ ( 1 ) = e k a n d J ̃ k ( γ ( 1 ) ) < 0

It is easy to see that

(4.4) C 0 k = inf u H 2 , k \ { 0 } max t > 0 J ̃ k ( t u ) = inf u M J ̃ k ( u ) = A .

Now we define the variational functional of (1.10) in Ω k by

(4.5) J k i ( u ) 1 2 Ω k | u i | 2 + a i u i 2 d x 1 6 Ω k β i i u i 6 d x

Furthermore, according to Lemma 4.2 in [25] (or [26]), we know that there exist β 0 and β 1 such that for β 12 > max{β 0, β 1}, A < min { m β 11 , m β 22 } holds, where

(4.6) β 0 2 19 β 22 β 11 8 and β 1 9 2 min m β 11 2 , m β 22 2 .

Then, we have the following Lemma

Lemma 4.1.

For β 12 > max{β 0, β 1}, let ( u ̃ 1 , u ̃ 2 ) is such that C 0 k is achieved. Then u ̃ 1 0 , u ̃ 2 0 .

Proof.

In fact, suppose u ̃ 1 ( x ) 0 , then

C 0 k = J ̃ k ( u ̃ 1 , u ̃ 2 ) = J ̃ k ( 0 , u ̃ 2 ) = max t > 0 J ̃ k ( 0 , t u ̃ 2 ) = max t > 0 J k 2 ( t u ̃ 2 ) = m β 22 2 > A

which contradicts to (4.4). We complete the proof.□

From Lemma 4.1 and (4.4), we observe that under the condition β 12 > max{β 0, β 1},

(4.7) C 0 k = inf u H 2 , k \ { 0 } max t > 0 J ̃ k ( t u ) = inf u N 2 J ̃ k ( u )

holds, where N 2 is the Nehari type set

N 2 u H 2 , k : u i 0 , i = 1 2 Ω k | u i | 2 + a i u i 2 d x = i , j = 1 2 Ω k β i j u i 3 u j 3 .

4.1 Penalization of the nonlinearity

Let us denote

H ( x , u 1 , u 2 ) = χ Ω k ρ ( x ) 1 6 β 11 u 1 6 + 1 6 β 22 u 2 6 + 1 3 β 12 u 1 3 u 2 3 + 1 χ Ω k ρ ( x ) P ( u 1 , u 2 )

To construct the nonlinearity P, for each a > 0 we consider X a C 0 ( [ 0 , + ) , R ) satisfying

(4.8) 0 X a 1 , X a ( t ) = 1 , t [ 0 , a ] , 0 , t > 2 a , X a ( t ) 0 , X a ( t ) c ̂ t

where c ̂ is a positive constant, and define

(4.9) P ( u 1 , u 2 ) = X a ( | ( u 1 , u 2 ) | ) 1 6 β 11 u 1 6 + 1 6 β 22 u 2 6 + 1 3 β 12 u 1 3 u 2 3 .

For later purpose, let us mention that we can choose a as follows:

(4.10) 0 < a < min ( 1 3 c ̂ β 12 ) 1 4 , ( μ ̄ a i ( c ̂ + 16 3 ) β 12 ) 1 4

where μ ̄ defined in (1.13). We define the modified functional by:

Φ λ , k ( u 1 , u 2 ) 1 2 R 3 | u 1 | 2 + λ V 1 ( x ) + a 1 u 1 2 + | u 2 | 2 + λ V 2 ( x ) + a 2 u 2 2 d x R 3 H ( x , u 1 , u 2 ) d x

Then one can check that a critical point of Φ λ,k corresponds to a solution of the following equation

(4.11) Δ u 1 + λ V 1 ( x ) + a 1 u 1 = H u 1 ( x , u 1 , u 2 )  in  R 3 , Δ u 2 + λ V 2 ( x ) + a 2 u 2 = H u 2 ( x , u 1 , u 2 )  in  R 3 , u i H 1 ( R N ) , i = 1,2 ,

By the definition of P(u 1, u 2), we see that P ( u 1 , u 2 ) = 1 6 β 11 u 1 6 + 1 6 β 22 u 2 6 + 1 3 β 12 u 1 3 u 2 3 if 0 < |(u 1, u 2)| < a. Thus a solution u of (4.11) is also a solution of the original problem (4.1) if 0 < |(u 1, u 2)| < a for all x R N \ Ω k ρ .

4.2 Compactness of the modified functional

In this subsection, we will show that the functional Φ λ,k satisfies the Palais–Smale condition under certain energy level.

Firstly, we define the following minimax value.

(4.12) C λ k = inf γ Γ λ k sup t [ 0,1 ] Φ λ , k ( γ ( t ) ) ,

where

Γ λ k = γ C [ 0,1 ] , E λ γ ( 0 ) = 0 , γ ( 1 ) = e k a n d Φ λ , k ( γ ( 1 ) ) < 0 .

Lemma 4.2.

Suppose that {u n } is a (P.S.) c sequence of the modified functional Φ λ,k , that is a sequence satisfying

Φ λ , k u n c , Φ λ , k u n 0 .

Then there exists a positive constant Λ2 > 0 such that for any λ Λ 2 , u n is bounded. That is there exists a constant C which is independent of λ and n such that

(4.13) lim n sup i = 1 2 u i , n λ , i 2 C

Proof.

By direct computation, we have

P ( u 1 , n , u 2 , n ) 1 6 [ P u 1 , n ( u 1 , n , u 2 , n ) u 1 , n + P u 2 , n ( u 1 , n , u 2 , n ) u 2 , n ] = | u n | 6 X a ( | u n | ) i , j = 1 2 β i j 6 | u i , n | 3 | u j , n | 3

and

(4.14) | | v n | 6 X a ( | v n | ) i , j = 1 2 β i j 6 | v i , n | 3 | v j , n | 3 | | v n | 6 | X a ( | v n | ) | β 12 6 | v 1 , n | 6 + 2 | v 1 , n | 3 | v 2 , n | 3 + | v 2 , n | 6 | v n | 6 | X a ( | v n | ) | β 12 6 | v 1 , n | 2 + | v 2 , n | 2 2 ( | v 1 , n | + | v 2 , n | ) 2 c ̂ β 12 a 4 | v 1 , n | 2 + | v 2 , n | 2 .

Since {u n } is a (P.S.) c sequence, we have

c + o ( 1 ) + ϵ n u n = Φ λ , k u n 1 6 Φ λ , k u n u n = 1 3 i = 1 2 u i , n λ , i 2 R N \ Ω k ρ P ( u 1 , n , u 2 , n ) 1 6 [ P u 1 , n ( u 1 , n , u 2 , n ) u 1 , n + P u 2 , n ( u 1 , n , u 2 , n ) u 2 , n ] d x = 1 3 i = 1 2 u i , n λ , i 2 + R N \ Ω k ρ | u n | 6 X a ( | u n | ) i , j = 1 2 β i j 6 | u i , n | 3 | u j , n | 3 d x , 1 3 i = 1 2 u i , n λ , i 2 R N \ Ω k 2 ρ C β 12 a 4 | u 1 , n | 2 + | u 2 , n | 2 d x 1 3 c ̂ β 12 a 4 i = 1 2 u i , n λ , i 2

where ϵ n → 0 as n → ∞. By (4.10), it is easy to see that the estimate (4.14) holds for some constant C > 0 independent of λ ≥ 0.□

Lemma 4.3.

The functional Φ λ,k (u 1, u 2) has Mountain-Pass geometry structure:

  1. there exist two constants α and ρ 0 such that Φ λ,k (u 1, u 2) ≥ α for all i = 1 2 u i λ , i = ρ 0 ;

  2. there is a e k E λ such that Φ λ,k (e 1,k , e 2,k ) < 0.

Proof.

Using (4.11) and Sobolev inequality, we have

Φ λ , k ( u 1 , u 2 ) = 1 2 i = 1 2 u i λ , i 2 i , j = 1 2 Ω k ρ β i j | u i | 3 | u j | 3 d x R N \ Ω k ρ P ( u 1 , u 2 ) d x 1 2 i = 1 2 u i λ , i 2 i , j = 1 2 R 3 β i j | u i | 3 | u j | 3 d x 1 2 i = 1 2 u i λ , i 2 β 12 ( S 3 u 1 λ , 1 6 + S 3 u 2 λ , 2 6 + 2 S 3 u 1 λ , 1 3 u 2 λ , 2 3 ) i = 1 2 ( 1 2 u i λ , i 2 β 12 S 3 j = 1 2 u i λ , i 3 u j λ , j 3 ) .

Choosing ρ 0 small enough, we can get the conclusion (1).

On the other hand, for any given u 0 H 0 1 ( Ω k ) , we have

Φ λ , k ( t u 0 , t u 0 ) = t 2 2 i = 1 2 R 3 | u 0 | 2 + a i u 0 2 d x i , j = 1 2 t 6 Ω k ρ β i j | u 0 | 6 d x

Thus, we can take e k = (tu 0, tu 0) with t sufficiently large such that the conclusion (2) is true.□

From [23], we consider the polynomial function Q ( x ) i , j = 1 2 β i j x i 3 x j 3 and denote by X the set of solutions to the maximization problem

(4.15) Q ( τ ) = max | X | = 1 Q ( X ) = Q max , τ = τ 1 , τ 2 , | τ | = 1 .

It follows from the results of [19], we know that C 0 k < 1 3 Q max 1 2 S 3 2 . Hence 0 < α < C λ k < 1 3 Q max 1 2 S 3 2 . By Lemma 4.3, using the standard Mountain Pass Lemma, we see that there is a (P.S.) c sequence {u n } ⊂ E λ of the functional Φ λ,k such that C λ k < 1 3 Q max 1 2 S 3 2 .

Proposition 4.4.

Suppose that {(u 1,n , u 2,n )} is a ( P . S . ) c λ k sequence for Φ λ,k with

C λ k < 1 3 Q max 1 2 S 3 2

where S is the best Sobolev constant. Then there exists a subsequence of {(u 1,n , u 2,n )} which converge strongly in E λ to a critical point u of Φ λ,k such that Φ λ , k ( u 1 , u 2 ) = c λ k .

Proof.

By Lemma 4.2, we know that {(u 1,n , u 2,n )} is bounded. Thus there exists a subsequence of {(u 1,n , u 2,n )} (still denoted by {(u 1,n , u 2,n )}) such that

u i , n u i  weakly in  L 2 R 3 , u i , n u i  a.e in  R 3 , u i , n u i  weakly in  L 2 * R 3 , u i , n u i  strongly in  L l o c 2 R 3

where i = 1, 2. Then by standard arguments, we can see that Φ λ , k ( u 1 , u 2 ) = 0 and Φ λ,k (u 1, u 2) ≥ 0.

Next we show that u i,n u i strongly in E λ , i = 1, 2. Let v i,n = u i,n u, i = 1, 2, it follows from the Brezis–Lieb’s Lemma that ( v 1 , n , v 2 , n ) is also a Palais–Smale sequence of Φ λ,k satisfying Φ λ , k v 1 , n , v 2 , n 0 and

lim n Φ λ , k v 1 , n , v 2 , n = C λ k Φ λ , k ( u 1 , u 2 ) C λ k C 0 k < 1 3 Q max 1 2 S 3 2 .

Hence it is sufficient to prove that v i,n → 0 strongly in E λ , i = 1, 2. We show this by contradiction. Without loss of generality, up to a subsequence, we assume on the contrary that lim n v 1 , n λ , 1 2 + v 2 , n λ , 2 2 = b > 0 . Thus v 1 , n , v 2 , n is also bounded in E λ , we have

(4.16) o ( 1 ) = Φ λ , k v 1 , n , v 2 , n v 1 , n , v 2 , n = i = 1 2 R 3 v i , n 2 + ( λ V i ( x ) + a i ) v i , n 2 ( x ) d x i , j = 1 2 Ω k ρ β i j | v i , n | 3 | v j , n | 3 d x R 3 \ Ω k ρ ( P v 1 , n ( v 1 , n , v 2 , n ) v 1 , n + P v 2 , n ( v 1 , n , v 2 , n ) v 2 , n ) d x i = 1 2 R 3 v i , n 2 + ( λ V i ( x ) a i ) v i , n 2 ( x ) d x i , j = 1 2 R 3 β i j | v i , n | 3 | v j , n | 3 d x

which indicates that

lim n i , j = 1 2 R 3 β i j | v i , n | 3 | v j , n | 3 d x lim n i = 1 2 R 3 v i , n 2 + ( λ V i ( x ) a i ) v i , n 2 ( x ) d x = b > 0

By Lemma 4.2, we can see that

Φ λ , k ( v n ) 1 6 Φ λ , k ( v n ) ( v n ) = 1 3 i = 1 2 R 3 [ | v i , n | 2 + ( λ V i ( x ) a i ) v i , n 2 ( x ) ] d x R 3 \ Ω k ρ P ( v 1 , n , v 2 , n ) 1 6 P v 1 , n ( v 1 , n , v 2 , n ) v 1 , n + P v 2 , n ( v 1 , n , v 2 , n ) v 2 , n d x = 1 3 i = 1 2 R 3 [ | v i , n | 2 + ( λ V i ( x ) a i ) v i , n 2 ( x ) ] d x + R 3 \ Ω k ρ | v n | 6 X a ( | v n | ) i , j = 1 2 β i j 6 | v i , n | 3 | v j , n | 3 d x .

According to the condition (A2), we can choose R ̃ > 0 such that D V i B 2 R ̃ ( 0 ) , then we have V i (x) > M i for any x R 3 \ B 2 R ̃ ( 0 ) . It follows that

R 3 \ B 2 R ̃ ( 0 ) v i , n 2 d x 1 λ M i R 3 \ B 2 R ̃ ( 0 ) λ V i ( x ) v i , n 2 d x 1 2 λ M i R 3 \ B 2 R ̃ ( 0 ) ( λ V i ( x ) + a i ) v i , n 2 d x C 2 λ M i

where C is a positive constant independent of λ and large n. Thus, for λ large, using (4.14), we have

| R 3 \ Ω k ρ | v n | 6 X a ( | v n | ) i , j = 1 2 β i j 6 | v i , n | 3 | v j , n | 3 d x | R 3 \ Ω k ρ c ̂ β 12 a 4 | v 1 , n | 2 + | v 2 , n | 2 d x = o λ ( 1 ) ,

where o λ (1) denotes a quantity that tends to 0 as λ → ∞. For λ large, it follows that

1 3 b = 1 3 i = 1 2 R 3 v i , n 2 + ( λ V i ( x ) + a i ) v i , n 2 ( x ) d x lim n Φ λ , k v n + o λ ( 1 ) C λ k + o λ ( 1 ) C 0 k + o λ ( 1 ) < 1 3 Q max 1 2 S 3 2 ,

which implies that b < Q max 1 2 S 3 2 .

By (4.15) and we set u ( x ) R 1 2 i = 1 2 u i ( x ) 2 , then we obtain

i , j = 1 2 R 3 β i j | v i , n | 3 | v j , n | 3 d x = R 3 f ( u ) d x = R 3 f u u ( x ) R 1 u ( x ) R 1 6 d x Q max R 3 u 1 2 + u 2 2 3 d x = Q max R 3 u 1 6 d x + R 3 u 2 6 d x + R 3 3 u 1 4 u 2 2 d x + R 3 3 u 1 2 u 2 4 d x = Q max R 3 u 1 6 d x 1 3 + R 3 u 2 6 d x 1 3 3 ,

which implies that

(4.17) R 3 u 1 6 d x 1 3 + R 3 u 2 6 d x 1 3 b Q max 1 1 3 .

By (1.5) and (4.17), then for λ large, we have

(4.18) b lim n i = 1 2 R 3 v i , n 2 d x lim n S R 3 v 1 , n 6 d x 1 3 + S R 3 v 2 , n 6 d x 1 3 S b 1 3 Q max 1 3 .

It follows that b S 3 2 Q max 1 2 . There is a contradiction and hence v i,n → 0 strongly in E λ , i = 1, 2.□

According to Lemma 4.3 and Proposition 4.4, we have proved the following existence result which is the main gredient of this subsection.

Proposition 4.5.

For any k ∈ {1, 2, …, }, there exists a critical point u λ k of the functional Φ λ,k (u) such that Φ λ , k u λ k = C λ k , where C λ k is defined in (4.12).

4.3 Asymptotic behavior of the one-bump solutions

In this subsection, we study the asymptotic behavior of one-bump solutions u λ k ( 1 k ) obtained in Proposition 4.5 as λ large. We have the following result.

Proposition 4.6.

Suppose u λ k = u 1 , λ k , u 1 , λ k , ( 1 k ) are the critical point of the functional Φ λ,k (u) obtained in Proposition 4.5. Then we have u λ k u k in H 1 ( R 3 ) ( 1 k ) , as λ → ∞, where u k = u 1 k , u 2 k is a fully nontrivial solution of (4.2) such that J ̃ k u k = C 0 k .

Proof.

For any sequence { λ n } n = 1 with λ n → ∞ as n → ∞, let us denote u λ n k is the corresponding critical points of Φ λ,k obtained in Proposition 4.5. To finish the proof of Proposition 4.6, we only need to show that up to a subsequence, u i , λ n k u i k ( i = 1,2 ) , as n → ∞. Indeed, since Φ λ , k u λ n k = C λ n k C 0 k and Φ λ , k u λ n k = 0 for all n ≥ 1, it is easy

i = 1 2 u i , λ n k i = 1 2 u i , λ n k λ n , i C ,

where ‖ ⋅‖ denotes the norm of H 1 ( R 3 ) . Thus up to a subsequence we may assume that u i , λ n k u i k ( i = 1,2 ) weakly in H 1 ( R 3 ) . We first show that u i k = 0 in R 3 \ Ω , where i = 1, 2 and Ω = Ω1 ∪ Ω2 ∪ …, Ω is the zero set V i 1 ( 0 ) . Let C m = x R N : V i ( x ) 1 m , then

C m u i , λ n k 2 d x 1 λ n m C m λ n V i ( x ) u i , λ n k 2 d x 1 2 λ n m C m λ n V i ( x ) a i u i , λ n k 2 d x 1 2 λ n m u i , λ n k λ n 2 1 2 λ n m C .

It follows that

C m u i k 2 d x lim inf n C m u i , λ n k 2 d x = 0 , as  n ,

which indicates that u i k 0 in C m . We get u i k 0 in m = 1 C m = R N \ Ω , where i = 1, 2. Next, for any ( ϕ 1 , ϕ 2 ) C 0 Ω k × C 0 Ω k , k { 1,2 , , } , we have

Φ λ n , k ( u 1 , n , u 2 , n ) ( ϕ 1 , ϕ 2 ) Φ λ n , k ( u 1 , n , u 2 , n ) λ n * ( ϕ 1 , ϕ 2 ) λ n 0

here we use the fact that ( ϕ 1 , ϕ 2 ) λ n indeed does not dependent on λ n . Thus we have

Ω k ( u 1 ϕ 1 + a 1 u 1 ϕ 1 ) d x + Ω k ( u 2 ϕ 2 + a 1 u 1 ϕ 1 ) d x = Ω k H u 1 ( x , u , v ) ϕ 1 + H u 2 ( x , u , v ) ϕ 2 d x .

By the definition of H(x, u 1, u 2), we know that for k ∈ {1, 2, …, } satisfies (4.2). For jk, setting (ϕ 1, ϕ 2) = (u 1, u 2) we have

Ω j | u 1 | 2 + a 1 u 1 2 + | u 2 | 2 + a 2 u 2 2 P u 1 ( u 1 , u 2 ) u 1 P u 2 ( u 1 , u 2 ) u 2 d x = 0

We have

Ω j P u 1 ( u 1 , u 2 ) u 1 + P u 2 ( u 1 , u 2 ) u 2 d x = Ω j | u | 6 X a ( | u | ) + X a ( | u | ) i , j = 1 2 β i j 6 | u i | 3 | u j | 3 d x C a 4 Ω j u 1 2 + u 2 2 d x ,

where C ( c ̂ + 16 3 ) β 12 . Thus, we have

0 = i = 1 2 Ω j | u i | 2 + a i u i 2 d x Ω j P u 1 ( u 1 , u 2 ) u 1 + P u 2 ( u 1 , u 2 ) u 2 d x i = 1 2 Ω j | u i | 2 + a i u i 2 d x C a 4 Ω j u 1 2 + u 2 2 d x 1 a i μ ̄ C a 4 μ ̄ i = 1 2 Ω j | u i | 2 d x > 0 .

There is a contradiction. Thus (u 1, u 2) ≡ (0, 0) in Ω j for jk.

In the following, we will show that u k = u 1 k , u 2 k is a fully nontrivial solution of (4.2). It is sufficient to show that u i , λ n k u i k ( i = 1,2 ) strongly in H 1 ( R 3 ) , as n → ∞. Indeed, if u i , λ n k u i k ( i = 1,2 ) strongly in H 1 R 3 which in turn implies that u i , λ n k u i k ( i = 1,2 ) strongly in L 2 * ( R 3 ) . We have

0 < Φ λ n , k u λ n k = 1 3 i = 1 2 Ω k 2 ρ u i , λ n k 2 * d x R N \ Ω k ρ P u 1 , λ n k , u 2 , λ n k 1 2 P u 1 , λ n k u 1 , λ n k , u 2 , λ n k u 1 , λ n k + P u 2 , λ n k u 1 , λ n k , u 2 , λ n k u 2 , λ n k d x 1 3 i = 1 2 Ω k 2 ρ u i , λ n k 2 * d x 1 3 i = 1 2 Ω k | u i k | 2 * d x

which implies that u k = u 1 k , u 2 k is a fully nontrivial solution of (4.2).

By the definition of C 0 k , one can see that J ̃ k ( u k ) C 0 k . Moreover

J ̃ k ( u k ) lim inf n Φ λ n , k u λ n k = lim inf n C λ n k C 0 k ,

which indicates that J ̃ k ( u k ) = C 0 k and thus u k is a fully nontrivial solution of (4.2).

At last, we come to show that, u i , λ n k u i k ( i = 1,2 ) strongly in H 1 ( R 3 ) , as n → ∞. We show this by a contradiction argument. Let us denote v i , n u i , λ n k u i k ( i = 1,2 ) , taking into account of (1.5), it is sufficient to prove that v i , n λ n , i 0 as n → ∞. Suppose on the contrary that, up to a subsequence,

lim inf n i = 1 2 v i , n λ n , i 2 = lim inf n i = 1 2 R 3 v i , n 2 + λ n V i ( x ) + a i v i , n 2 d x = b > 0 .

Since J ̃ k ( u k ) > 0 , for n large enough, we see that

(4.19) Φ λ n , k ( v 1 , n , v 2 , n ) = Φ λ n , k u λ n k J ̃ k ( u k ) + o ( 1 ) Φ λ n , k u λ n k + o ( 1 ) C 0 k + o ( 1 ) < 1 3 Q max 1 2 S 3 2 .

Note that u i , λ n k u i k ( i = 1,2 ) weakly in H 1 ( R 3 ) , similar to (4.16), we have

0 = Φ λ n , k u 1 , λ n k , u 2 , λ n k u 1 , λ n k , u 2 , λ n k J ̃ k u 1 k , u 2 k u 1 k , u 2 k = Φ λ n , k v 1 , n , v 2 , n v 1 , n , v 2 , n + o ( 1 ) i = 1 2 R N v i , n 2 + ( λ V i ( x ) a i ) v i , n 2 ( x ) d x i , j = 1 2 R 3 β i j | v i , n | 3 | v j , n | 3 d x

which implies that

lim n i , j = 1 2 R 3 β i j | v i , n | 3 | v j , n | 3 d x lim n i = 1 2 R N v i , n 2 + ( λ V i ( x ) a i ) v i , n 2 ( x ) d x = b > 0 .

Combining with (4.19), we obtain that b < Q max 1 2 S 3 2 .

Similar to (4.18), for λ large, we have

b = lim n i = 1 2 R N v i , n 2 + ( λ V i ( x ) a i ) v i , n 2 ( x ) d x lim n i = 1 2 R N v i , n 2 d x lim n S R N v 1 , n 6 d x 1 3 + S R N v 2 , n 6 d x 1 3 S ( b ) 1 3 Q max 1 3 .

Thus we get b S 3 2 Q max 1 2 . There is a contradiction and we proved that v i , n λ n , i 2 0 ( i = 1,2 ) as n → ∞. This concludes the proof of Proposition 4.6.□

4.4 L estimate of the critical points for modified functional

In this subsection, we will show that the critical point u λ k = u 1 , λ k , u 2 , λ k of Φ λ,k obtained in Proposition 4.5 is indeed a solution of the original problem (4.2). More precisely, we have

Proposition 4.7.

Fix M* > 0, there is a constant δ > 0 such that for any critical points u λ k of Φ λ , k u λ k with Φ λ , k u λ k M * , we have:

i = 1 2 u i , λ k ( x ) δ λ ,

which implies that there is a Λ* > 0 such that for λ Λ * , u λ k ( x ) a for any x R 3 \ Ω k ρ and hence u λ k ( x ) is also a solution of the original problem (4.2).

The similar arguments can be found in the paper by (Guo and Tang [14]), for the completeness, we give the details of the proof. Before giving the proof of Proposition 4.7, we firstly present an L estimate for the solutions u λ k outside of Ω k ρ with i = 1 2 u i , λ k λ C . More precisely, we have

Lemma 4.8.

Suppose u λ k are the critical points of Φ λ,k such that i = 1 2 u i , λ k λ C , where C is a constant independent of λ. Then we have for some constant C 0 > 0 independent of λ such that for λ large,

(4.20) i = 1 2 u i , λ k ( x ) L R 3 \ Ω k ρ C 0 .

Proof.

Firstly, by Proposition 4.6, it is easy to see

lim λ R 3 \ Ω k ρ u i , λ k 2 * d x = 0 , i = 1,2 .

Hence, for a small number η 0 > 0 (which we will be specified later), we have, for λ large enough,

(4.21) R 3 \ Ω k ρ u i , λ k 2 * d x 2 η 0 .

Now we are ready to use Moser’s iteration argument to obtain the desired estimates. Let ψ denote a smooth cut-off function and γ > 1 an arbitrary number, both of them will be specified later. Multiply (4.11) by ψ 2 u i , λ k γ , we have

i = 1 2 R N ψ 2 u i , λ k γ u i , λ k + ( λ V i ( x ) + a i ) u i , λ k ψ 2 u i , λ k γ d x = i = 1 2 R N H u i x , u 1 , λ k , u 2 , λ k ψ 2 u i , λ k γ d x .

By a direct computation, we have

(4.22) γ i = 1 2 R N ψ 2 u i , λ k γ 1 | u i , λ k | 2 d x + 2 i = 1 2 R N ψ u i , λ k γ u i , λ k ψ d x + R N ( λ V i ( x ) + a i ) u i , λ k ψ 2 u i , λ k γ d x = i = 1 2 R N H u i x , u 1 , λ k , u 2 , λ k ψ 2 u i , λ k γ d x .

On the other hand, by Hölder’s inequality and Young’s inequality, we have

2 R N ψ u i , λ k γ u i , λ k ψ d x γ 2 R N ψ 2 u i , λ k γ 1 | u i , λ k | 2 d x + 2 γ R N | ψ | 2 u i , λ k γ + 1 d x .

Note that for any x R N and u ≥ 0, we have g i ( x , u ) u 2 * 1 . Thus the inequality (4.22) leads to

γ 2 i = 1 2 R N ψ 2 u i , λ k γ 1 | u i , λ k | 2 d x + i = 1 2 R N ( λ V i ( x ) + a i ) u i , λ k ψ 2 u i , λ k γ d x 2 γ i = 1 2 R N | ψ | 2 u i , λ k γ + 1 d x + R N ψ 2 β 11 u 1 , λ k 5 + γ + β 12 u 1 , λ k 2 + γ u 2 , λ k 3 + β 22 u 2 , λ k 5 + γ + β 21 u 1 , λ k 3 u 2 , λ k 2 + γ d x

since V i (x) ≥ 0 (i = 1, 2), it deduces that

(4.23) i = 1 2 R N ψ 2 u i , λ k γ 1 | u i , λ k | 2 d x 4 β 2 i = 1 2 R N | ψ | 2 u i , λ k γ + 1 d x + i = 1 2 2 | a i | γ R N ψ 2 u i , λ k γ + 1 d x + 2 γ R N ψ 2 β 11 u 1 , λ k 5 + γ + β 12 u 1 , λ k 2 + γ u 2 , λ k 3 + β 22 u 2 , λ k 5 + γ + β 21 u 1 , λ k 3 u 2 , λ k 2 + γ d x

By Sobolev imbedding theorem, we have

(4.24) S R N ψ u i , λ k γ + 1 2 6 d x 1 3 R N ψ u i , λ k γ + 1 2 2 d x ( γ + 1 ) 2 R N ψ 2 u i , λ k γ 1 | u i , λ k | 2 d x + 2 R N | ψ | 2 u i , λ k γ + 1 d x .

Combining with (4.23) and (4.24), we get

(4.25) i = 1 2 S R N ψ u i , λ k γ + 1 2 6 d x 1 3 i = 1 2 2 | a i | ( γ + 1 ) 2 γ R N ψ 2 u i , λ k γ + 1 d x + 4 ( γ + 1 ) 2 γ 2 + 2 i = 1 2 R N u i , λ k γ + 1 | ψ | 2 d x + 2 ( γ + 1 ) 2 γ R N ψ 2 β 11 u 1 , λ k 5 + γ + β 12 u 1 , λ k 2 + γ u 2 , λ k 3 + β 22 u 2 , λ k 5 + γ + β 21 u 1 , λ k 3 u 2 , λ k 2 + γ d x .

Now for y R 3 \ Ω k 8 r , we specify the cut-off function ψ by

ψ = 1 , x B 2 r ( y ) , 0 , x R 3 \ B 4 r ( y )

with | ψ | C r . By Hölder’s inequality, we have

(4.26) R N ψ 2 u 1 , λ k 5 + γ d x R N ψ 2 u 1 , λ k 1 + γ 3 d x 1 3 R N \ Ω k u 1 , λ k 6 d x 2 3 2 η 0 2 3 R N ψ 2 u 1 , λ k 1 + γ 3 d x 1 3 ,

and

(4.27) R N ψ 2 u 1 , λ k 2 + γ u 2 , λ k 3 d x R N ψ 2 u 1 , λ k 1 + γ 3 d x 1 3 R N \ Ω k u 2 , λ k 3 u 1 , λ k 3 2 d x 2 3 R N ψ 2 u 1 , λ k 1 + γ 3 d x 1 3 2 η 0 2 3 .

Similar to (4.26) and (4.27), we have

(4.28) R N ψ 2 u 2 , λ k 5 + γ d x 2 η 0 2 3 R N ψ 2 u 2 , λ k 1 + γ 3 d x 1 3 ,

and

(4.29) R N ψ 2 u 2 , λ k 2 + γ u 1 , λ k 3 d x R N ψ 2 u 2 , λ k 1 + γ 3 d x 1 3 2 η 0 2 3 .

Take γ = 5 and η 0 > 0 is such that 2 ( γ + 1 ) 2 γ 2 β 12 2 η 0 2 3 = S 2 , then (4.25) becomes

i = 1 2 S R N ψ u i , λ k 3 6 d x 1 3 i = 1 2 4 | a i | ( γ + 1 ) 2 γ R N ψ 2 u i , λ k 6 d x + 8 ( γ + 1 ) 2 γ 2 + 4 i = 1 2 R N u i , λ k 6 | ψ | 2 d x

which implies that for any y R N \ Ω k ,

(4.30) i = 1 2 B 2 r ( y ) u i , λ k 18 d x C ( r ) i = 1 2 B 4 r ( y ) u i , λ k 6 d x .

In the following, we will use above estimates combining with Moser’s iteration argument to prove (4.20). Let Z 1 , λ = u 1 , λ k γ + 1 2 and Z 2 , λ = u 2 , λ k γ + 1 2 , where γ > 1 will be chosen later, then (4.25) becomes

(4.31) i = 1 2 S R N ψ Z i , λ 6 d x 1 3 i = 1 2 2 | a i | ( γ + 1 ) 2 γ R N ψ 2 Z i , λ 2 d x + 4 ( γ + 1 ) 2 γ 2 + 2 i = 1 2 R N Z i , λ 2 | ψ | 2 d x + 2 ( γ + 1 ) 2 γ R N ψ 2 β 11 u 1 , λ k 4 Z 1 , λ 2 + β 12 u 1 , λ k Z 1 , λ 2 u 2 , λ k 3 + β 22 u 2 , λ k 4 Z 2 , λ 2 + β 21 u 1 , λ k 3 Z 2 , λ 2 u 2 , λ k d x .

where ψ is a cut-off function supported in B 2r (y) with y R N \ Ω k and r will be specified later in each step of the iteration process.

Using Hölder’s inequality again, we notice that

R N ψ 2 u 1 , λ k 4 Z 1 , λ 2 d x R N ψ Z 1 , λ 18 7 d x 7 9 B 2 r ( y ) u 1 , λ k 18 d x 2 9

and

R N ψ 2 Z 1 , λ 2 u 1 , λ k u 2 , λ k 3 d x R N ψ Z 1 , λ 18 7 d x 7 9 B 2 r ( y ) u 2 , λ k 18 d x 1 6 B 2 r ( y ) u 1 , λ k 18 d x 1 9 .

where q = N 2 N 2 = 9 and 2 < 2 q q 2 = 18 7 < 6 . Using the fact that y R N \ Ω k and also (4.30), (4.21), the last term in (4.31) can be estimated as follows

R N ψ 2 i = 1 2 β i i u i , λ k 4 Z i , λ 2 + i = 1 , i j 2 β i j u i , λ k Z i , λ 2 u j , λ k 3 d x 2 β 12 [ 2 η 0 ] 2 9 i = 1 2 R N ψ Z i , λ 18 7 7 9 .

Thus, for any ɛ > 0, we have

ψ Z i , λ L 18 7 ( R N ) ε ψ Z i , λ L 6 ( R N ) + ε 1 2 ψ Z i , λ L 2 ( R N )

where i = 1, 2 . It follows from (4.31), we have

(4.32) i = 1 2 S R N ψ Z i , λ 6 d x 1 3 i = 1 2 2 | a i | ( γ + 1 ) 2 γ R N ψ 2 Z i , λ 2 d x + 4 ( γ + 1 ) 2 γ 2 + 2 i = 1 2 R N Z i , λ 2 | ψ | 2 d x + 2 ( γ + 1 ) 2 γ C β i = 1 2 ε ψ Z i , λ L 6 ( R N ) + ε 1 2 ψ Z i , λ L 2 ( R N )

where C β 2 β 12 [ 2 η 0 ] 2 9 . Setting ε S γ 4 ( γ + 1 ) 2 C β , we obtain from (4.32) that

(4.33) i = 1 2 R N ψ Z i , λ 6 d x 1 3 2 S 4 | a i | + 2 C ̃ ( γ + 1 ) 3 i = 1 2 R N ψ 2 Z i , λ 2 d x + 2 C ̃ S i = 1 2 R N Z i , λ 2 | ψ | 2 d x

where C ̃ is a constant independent of β. Now for rr 2 < r 1 ≤ 2r, we choose ψ such that ψ ≡ 1 in B r 2 ( y ) and ψ ≡ 0 in R N \ B r 1 ( y ) . Then by a direct computation, we deduce from (4.33) that

i = 1 2 Z i , λ L 6 B r 2 ( y ) 2 C ̃ 1 2 R 2 r 1 r 2 2 ( γ + 1 ) 3 i = 1 2 Z i , λ L 2 B r 1 ( y ) 2 ,

i.e.

(4.34) i = 1 2 Z i , λ L 6 B r 2 ( y ) 2 i = 1 2 Z i , λ L 6 B r 2 ( y ) 2 1 2 C ̃ 2 R r 1 r 2 h 3 2 i = 1 2 Z i , λ L 2 B r 1 ( y ) 2 1 2 C ̃ 2 R r 1 r 2 h 3 2 i = 1 2 Z i , λ L 2 B r 1 ( y ) ,

where h = 1 + γ, R = max{r, 1} and C ̃ 1 2 and C ̃ 2 are constants independent of r, γ. Set

N ( p , r ) = i = 1 2 B r ( y ) u i , λ k p d x 1 p

Then we can rewrite (4.34) in terms of N(⋅, ⋅):

(4.35) N k h , r 2 2 C ̃ 2 R r 1 r 2 2 h h N h N h , r 1 ,

where k = N N 2 = 3 . Let p = 2* and h = h m = p 3 m , r m = r 1 + 2 m , for m = 0, 1, 2, …. It follows from (4.35) that

N p 3 m + 1 , r m + 1 = N 3 h m , r m + 1 2 C ̃ 2 R r m r m + 1 2 h m h m N h m N h m , r m = 4 C ̃ 2 R r p 3 2 2 p 3 m 3 3 2 2 2 p m 3 m N p 3 m , r m 4 C ̃ 2 R r p 3 2 2 p j = 0 3 j 3 3 2 2 2 p j = 0 j 3 j N ( p , 2 r ) .

Let m → ∞, we have

i = 1 2 sup x B r ( y ) u i , λ k ( x ) = lim s N ( s , r ) C ̃ 3 N ( p , 2 r ) = C ̃ 3 u i , λ k L 6 B 2 r ( y ) C ̃ 3 R N \ Ω k u i , λ k 6 1 6 C ̃ 3 2 η 0 1 6 ,

where

C ̃ 3 = 4 C ̃ 2 R r p N 4 2 p j = 0 3 j 3 3 2 2 2 p j = 0 j 3 j .

Since the above estimate is independent of y R N \ Ω k , we indeed have proved (4.20) with C 0 = C ̃ 3 2 η 0 1 6 .□

A direct result of the arguments in the proof of Lemma 4.8 is the following exterior Harnack-type inequality:

Lemma 4.9.

Suppose all the assumptions of Lemma 4.8 are satisfied. Then for any 0 < r < 1 4 ρ , there is a constant C > 0 such that for any y R N \ Ω k , it holds

sup x B r ( y ) u i , λ k B 2 r ( y ) u i , λ k 6 d x 1 6 .

Now we are ready to present the proof of Proposition 4.7.

Proof of Proposition 4.7.

By the proof of Proposition 4.5, one can find a constant C* > 0 independent of λ such that for λ ≥Λ*, we have

(4.36) i = 1 2 u i , λ k λ , i C *

On the other hand, by the assumption on V i (x), there is a positive number M ̃ > 0 such that V i ( x ) M ̃ for all x R N \ Ω ρ , where Ω = int  V i 1 ( 0 ) is the interior of the zero set V i (x) and i = 1, 2. Thus for λ large enough, it holds that λ V i ( x ) a i λ 2 M ̃ , for all x R N \ Ω ρ . As a consequence of (4.36), we have

R N \ Ω ρ λ 2 M ̃ u i , λ k 2 d x R N \ Ω ρ u i , λ k 2 + λ 2 M ̃ u i , λ k 2 d x C *

which implies that

(4.37) R N \ Ω ρ u i , λ k 2 d x 1 λ 2 C * M ̃ .

And we may assume that 2 C * M ̃ 1 , otherwise we can take M is properly large.

Note that for any 0 < r < 1 4 ρ and q > 6 fixed, by the interpolation inequality, we have for any y R N \ Ω ,

(4.38) B 2 r ( y ) u i , λ k 6 d x B 2 r ( y ) u i , λ k 2 d x α B 2 r ( y ) u i , λ k q d x 1 α ,

where α ∈ (0, 1) is such that 6 = 2α + (1 − α)q i.e., α = q 6 q 2 . We may choose q > 6 such that α = 1 2 . By Lemma 4.8, we have

(4.39) B 2 r ( y ) u i , λ k q d x 1 α C 0 q ( 2 r ) N ω 3 ( 1 α ) C 0 q 2 ρ N ω 3 1 2

where ω 3 is the volume of the unit ball B 1(0). Combining (4.37)(4.39), we obtain that, for any y R N \ Ω 2 ρ

B 2 r ( y ) u i , λ k 6 d x C 0 q 2 ρ N ω 3 1 2 B 2 r ( y ) u i , λ k 2 d x α C 0 q 2 ρ N ω 3 1 2 1 λ 2 M a 0 α C ̃ 4 λ

where C ̃ 4 = 2 M a 0 C 0 q 2 ρ N ω 3 1 2 . As a consequence of Lemma 4.9, we have for any y R N \ Ω 2 ρ ,

sup x B r ( y ) u i , λ k C ̃ 4 λ ,

which implies that

| u i , λ k ( x ) | L R N \ Ω 2 ρ C ̃ 4 λ .

Proof of Theorem 1.6

According to Proposition 4.4, Proposition 4.5, Proposition 4.6 and Proposition 4.7, we complete the proof.□


Corresponding author: Zongyan Lv, Center for Mathematical Sciences, Wuhan University of Technology, Wuhan 430070, P.R. China, E-mail: 

Acknowledgments

The author Zongyan Lv thanks Chengxiang Zhang and Qihan He for the valuable discussions when preparing the paper. The authors also wound like to thank the referees for many valuable comments helping to improve the paper.

  1. Research ethics: Not applicable.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: The author states no conflict of interest.

  4. Research funding: None declared.

  5. Data availability: Not applicable.

References

[1] E. Timmermans, “Phase separation of Bose-Einstein condensates,” Phys. Rev. Lett., vol. 81, no. 26, pp. 5718–5721, 1998. https://doi.org/10.1103/physrevlett.81.5718.Search in Google Scholar

[2] L. Marcello and Z. W. Tang, “Multi-bump bound states for a system of nonlinear Schrödinger equations,” J. Differ. Equ., vol. 252, no. 5, pp. 3630–3657, 2012. https://doi.org/10.1016/j.jde.2011.11.017.Search in Google Scholar

[3] N. Ikoma and Y. Miyamoto, “Stable standing waves of nonlinear Schrödinger equations with potentials and general nonlinearities,” Calc. Var. Partial Differ. Equ., vol. 59, no. 2, p. 20, 2020. https://doi.org/10.1007/s00526-020-1703-0.Search in Google Scholar

[4] D. M. Cao and E. S. Noussair, “Multi-bump standing waves with a critical frequency for nonlinear Schrödinger equations,” J. Differ. Equ., vol. 203, no. 2, pp. 292–312, 2004. https://doi.org/10.1016/j.jde.2004.05.003.Search in Google Scholar

[5] T. Bartsch, A. Pankov, and Z. Q. Wang, “Nonlinear Schrödinger equations with steep potential well,” Commun. Contemp. Math., vol. 3, no. 4, pp. 549–569, 2001. https://doi.org/10.1142/s0219199701000494.Search in Google Scholar

[6] A. Ambrosetti, A. Malchiodi, and S. Secchi, “Multiplicity results for some nonlinear Schrödinger equations with potentials,” Arch. Ration. Mech. Anal., vol. 159, no. 3, pp. 253–271, 2001. https://doi.org/10.1007/s002050100152.Search in Google Scholar

[7] Y. Z. Wu, T. F. Wu, and W. M. Zou, “On a two-component Bose-Einstein condensate with steep potential wells,” Ann. Mat. Pura Appl., vol. 196, no. 5, pp. 1695–1737, 2017.10.1007/s10231-017-0635-6Search in Google Scholar

[8] Y. H. Ding and K. Tanaka, “Multiplicity of positive solutions of a nonlinear Schrödinger equation,” Manuscripta Math., vol. 112, no. 1, pp. 109–135, 2003. https://doi.org/10.1007/s00229-003-0397-x.Search in Google Scholar

[9] T. Bartsch and Z. W. Tang, “Multibump solutions of nonlinear Schrödinger equations with steep potential well and indefinite potential. Discrete Contin,” Dyn. Syst., vol. 33, no. 1, pp. 7–26, 2013. https://doi.org/10.3934/dcds.2013.33.7.Search in Google Scholar

[10] J. Byeon and Z. Q. Wang, “Standing waves with a critical frequency for nonlinear Schrödinger equations,” Arch. Ration. Mech. Anal., vol. 165, no. 4, pp. 295–316, 2002. https://doi.org/10.1007/s00205-002-0225-6.Search in Google Scholar

[11] T. Bartsch and Z. Q. Wang, “Existence and multiplicity results for some superlinear elliptic problems on RN${\mathbb{R}}^{N}$,” Commun. Partial Differential Equations, vol. 20, nos. 9–10, pp. 1725–1741, 1995.10.1080/03605309508821149Search in Google Scholar

[12] T. Bartsch and Z. Q. Wang, “Sign changing solutions of nonlinear Schrödinger equations,” Topol. Methods Nonlinear Anal., vol. 13, no. 2, pp. 191–198, 1999. https://doi.org/10.12775/tmna.1999.010.Search in Google Scholar

[13] Y. Z. Wu and W. M. Zou, “On a critical Schrödinger system in R4${\mathbb{R}}^{4}$ with steep potential wells,” Nonlinear Anal., vol. 191, 2020, Art. no. 111643. https://doi.org/10.1016/j.na.2019.111643.Search in Google Scholar

[14] Y. X. Guo and Z. W. Tang, “Multi-bump solutions for Schrödinger equation involving critical growth and potential wells,” Discrete Contin. Dyn. Syst., vol. 35, no. 8, pp. 3393–3415, 2015. https://doi.org/10.3934/dcds.2015.35.3393.Search in Google Scholar

[15] Y. X. Guo and Z. W. Tang, “Sign changing bump solutions for Schrödinger equations involving critical growth and indefinite potential wells,” J. Differ. Equ., vol. 259, no. 11, pp. 6038–6071, 2015. https://doi.org/10.1016/j.jde.2015.07.015.Search in Google Scholar

[16] H. X. Chen and A. Pistoia, “Segregated solutions for nonlinear Schrödinger systems with a large number of components,” Adv. Nonlinear Stud., vol. 24, no. 2, pp. 436–450, 2024. https://doi.org/10.1515/ans-2022-0076.Search in Google Scholar

[17] X. M. An and J. Yang, “Existence and asymptotic behavior of solitary waves for a weakly coupled Schrödinger system,” Adv. Nonlinear Stud., vol. 22, no. 1, pp. 159–183, 2022. https://doi.org/10.1515/ans-2022-0008.Search in Google Scholar

[18] J. Wang, X. Wang, and S. Wei, “Existence of normalized solutions for the coupled elliptic system with quadratic nonlinearity,” Adv. Nonlinear Stud., vol. 22, no. 1, pp. 203–227, 2022. https://doi.org/10.1515/ans-2022-0010.Search in Google Scholar

[19] T. H. Liu, Y. Song, and W. M. Zou, “Least energy positive solutions for d-coupled Schrödinger systems with critical exponent in dimension three,” J. Differ. Equ., vol. 367, no. 7, pp. 40–78, 2023.10.1016/j.jde.2023.04.039Search in Google Scholar

[20] Z. W. Tang, “Least energy solutions for semilinear Schrödinger equations involving critical growth and indefinite potentials,” Commun. Pure Appl. Math., vol. 13, no. 1, pp. 237–248, 2014. https://doi.org/10.3934/cpaa.2014.13.237.Search in Google Scholar

[21] H. Brézis and L. Nirenberg, “Positive solutions of nonlinear elliptic equations involving critical Sobolev exponents,” Commun. Pure Appl. Math., vol. 36, no. 4, pp. 437–477, 1983. https://doi.org/10.1002/cpa.3160360405.Search in Google Scholar

[22] G. Talenti, “Best constant in Sobolev inequality,” Ann. Mat. Pura Appl., vol. 110, no. 4, pp. 353–372, 1976. https://doi.org/10.1007/bf02418013.Search in Google Scholar

[23] Q. H. He and J. Yang, “Quantitative properties of ground-states to an M-coupled system with critical exponent in RN${\mathbb{R}}^{N}$,” Sci. China Math., vol. 61, no. 4, pp. 709–726, 2018. https://doi.org/10.1007/s11425-016-0464-4.Search in Google Scholar

[24] M. Willem, Minimax Theorems, Progr. Nonlinear Differential Equations Appl, vol. 24, Boston, MA, Birkhäuser Boston, Inc., 1996.10.1007/978-1-4612-4146-1Search in Google Scholar

[25] H. Y. Ye and Y. F. Peng, “Positive least energy solutions for a coupled Schrödinger system with critical exponent,” J. Math. Anal. Appl., vol. 417, no. 1, pp. 308–326, 2014. https://doi.org/10.1016/j.jmaa.2014.03.028.Search in Google Scholar

[26] S. Kim, “On vector solutions for coupled nonlinear Schrödinger equations with critical exponents,” Commun. Pure Appl. Math., vol. 12, no. 3, pp. 1259–1277, 2013. https://doi.org/10.3934/cpaa.2013.12.1259.Search in Google Scholar

Received: 2024-02-10
Accepted: 2024-08-04
Published Online: 2024-09-13

© 2024 the author(s), published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 11.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/ans-2023-0149/html
Scroll to top button