Remarks on the vanishing discount problem for infinite systems of Hamilton–Jacobi–Bellman equations

Kengo Terai

doi:10.1515/acv-2020-0116

Article

Remarks on the vanishing discount problem for infinite systems of Hamilton–Jacobi–Bellman equations

Kengo Terai

Published/Copyright: April 16, 2021

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information Explore this Subject

From the journal Advances in Calculus of Variations Volume 16 Issue 1

Abstract

This paper is concerned with the asymptotic analysis of infinite systems of weakly coupled stationary Hamilton–Jacobi–Bellman equations as the discount factor tends to zero. With a specific Hamiltonian, we show the convergence of the solution and prove the solvability of the corresponding ergodic problem.

Keywords: Hamilton–Jacobi–Bellman equations; infinite systems; viscosity solutions; asymptotic analysis; vanishing discount

MSC 2010: 35B40; 35F21; 49L25

Communicated by Hitoshi Ishii

Funding source: Japan Society for the Promotion of Science

Award Identifier / Grant number: 20J10824

Funding statement: This work was supported by Grant-in-Aid for JSPS Fellows Grant number 20J10824.

A Appendix

Here, we study the discount approximation with the conditions that F ∈ C ⁢ ( 𝕋 d × ℝ d × I ) , f ∈ C ⁢ ( 𝕋 d × I ) and k ∈ C ⁢ ( I × I ) . Then, as in [11, Theorem 4.5], the unique viscosity solution v α to (1.1) belongs to C ⁢ ( 𝕋 d × I ) . The aim of this section is to prove the following refined asymptotics.

Theorem A.1.

Suppose that I ⊂ R is a closed and finite interval. Suppose that (A1)–(A5) hold. Assume that F ∈ C ⁢ ( T d × R d × I ) , f ∈ C ⁢ ( T d × I ) and k ∈ C ⁢ ( I × I ) . Let v α ∈ C ⁢ ( T d × I ) be the viscosity solution to (1.1). Then there exists v ∈ C ⁢ ( T d × I ) , which is a viscosity solution to (1.2) with c = 0 , satisfying that v α → v uniformly on T d × I as α → 0 .

To prove the above, we consider a different type of half-relaxed limits of v α . Instead of Proposition 3.5, we give another result on the stability of viscosity solutions.

Proposition A.2.

Let assumptions (B1) and (B2) hold. Assume that H ∈ C ⁢ ( T d × R d × I ) and k ∈ C ⁢ ( I × I ) . Let v α ∈ C ⁢ ( T d × I ) be the viscosity solution to (1.1). Additionally, suppose that there exist c : I → R and x 0 ∈ T d such that uniformly in ξ ∈ I ,

(A.1) lim α → 0 ⁡ α ⁢ v α ⁢ ( x 0 , ξ ) + Θ ⁢ v α ⁢ ( x 0 , ξ ) = - c ⁢ ( ξ ) .

Let v ~ α ⁢ ( x , ξ ) := v α ⁢ ( x , ξ ) - v α ⁢ ( x 0 , ξ ) and set

(A.2) v ¯ ⁢ ( x , ξ ) = lim r → 0 ⁡ sup ⁡ { v ~ α ⁢ ( y , η ) ∣ | x - y | + | ξ - η | + α ⩽ r }

and

(A.3) v ¯ ⁢ ( x , ξ ) = lim r → 0 ⁡ inf ⁡ { v ~ α ⁢ ( y , η ) ∣ | x - y | + | ξ - η | + α ⩽ r } .

Then v ¯ and v ¯ are a subsolution and a supersolution, respectively, to

H ⁢ ( x , D ⁢ v , ξ ) + ∫ I k ⁢ ( ξ , η ) ⁢ { v ⁢ ( x , ξ ) - v ⁢ ( x , η ) } ⁢ 𝑑 η = c in ⁢ 𝕋 d × I .

Proof.

Here, we only show that v ¯ is a subsolution. Take ξ ∈ I , ϕ ∈ C 1 ⁢ ( 𝕋 d ) and x ^ ∈ 𝕋 d such that v ¯ ⁢ ( ⋅ , ξ ) - ϕ ⁢ ( ⋅ ) attains a strict maximum at x 0 ∈ 𝕋 d . Let { y j } j ∈ ℕ , { ξ j } j ∈ ℕ and { α j } j ∈ ℕ such that y j → x ^ , ξ j → ξ , α j → 0 and v ~ α j ⁢ ( y j , ξ j ) → v ¯ ⁢ ( x ^ , ξ ) .

On the other hand, let x j ∈ 𝕋 d such that

max x ∈ 𝕋 d ⁡ v ~ α j ⁢ ( x , ξ j ) - ϕ ⁢ ( x ) = v ~ α j ⁢ ( x j , ξ j ) - ϕ ⁢ ( x j ) .

Moreover, we can choose a subsequence satisfying x j → x ¯ for some x ¯ ∈ 𝕋 d . Then we get

v ¯ ⁢ ( x ^ , ξ ) - ϕ ⁢ ( x ^ ) ⩽ lim inf j → ∞ ⁡ v ~ α j ⁢ ( y j , ξ j ) - ϕ ⁢ ( y j )

⩽ lim inf j → ∞ ⁡ v ~ α j ⁢ ( x j , ξ j ) - ϕ ⁢ ( x j )

⩽ lim sup j → ∞ ⁡ v ~ α j ⁢ ( x j , ξ j ) - ϕ ⁢ ( x ¯ )

⩽ v ¯ ⁢ ( x ¯ , ξ ) - ϕ ⁢ ( x ¯ ) .

Thus, we have

x ^ = x ¯ and lim j → ∞ ⁡ v ~ α j ⁢ ( x j , ξ j ) = v ¯ ⁢ ( x ¯ , ξ ) .

Because v α j is a subsolution to (1.1), we have

α j v ~ α j ( x j , ξ j ) + H ( x j , D ϕ ( x j ) , ξ j ) + α j v α j ( x 0 , ξ j ) + Θ v α j ( x 0 . ξ j ) ⩽ - Θ v ~ α j ( x j , ξ j ) .

In light of (3.7), v ~ α is bounded on 𝕋 d × I . Due to (A.1), we have

α j v ~ α j ( x j , ξ j ) + H ( x j , D ϕ ( x j ) , ξ j ) + α j v α j ( x 0 , ξ j ) + Θ v α j ( x 0 . ξ j ) → H ( x ^ , D ϕ ( x ^ ) , ξ ) - c ( ξ ) .

By Fatou’s lemma, we obtain

lim sup j → ∞ ⁡ ∫ I k ⁢ ( ξ j , η ) ⁢ v ~ α j ⁢ ( x j , η ) ⁢ 𝑑 η ⩽ ∫ I lim sup j → ∞ ⁡ k ⁢ ( ξ j , η ) ⁢ v ~ α j ⁢ ( x j , η ) ⁢ d ⁢ η ⩽ ∫ I k ⁢ ( ξ , η ) ⁢ v ¯ ⁢ ( x ^ , η ) ⁢ 𝑑 η .

Hence, it follows that

H ⁢ ( x ^ , D ⁢ ϕ ⁢ ( x ^ ) , ξ ) + ∫ I k ⁢ ( ξ , η ) ⁢ { v ¯ ⁢ ( x ^ , ξ ) - v ¯ ⁢ ( x ^ , η ) } ⁢ 𝑑 η ⩽ c ⁢ ( ξ ) . ∎

Proof of Theorem A.1.

Note that 𝒜 = 𝒜 ~ and 𝒵 = 𝒜 × I because f ∈ C ⁢ ( 𝕋 d × I ) . Take x 0 ∈ 𝒜 . Define v ¯ and v ¯ as (A.2) and (A.3), respectively. In light of Proposition 4.1, for all ξ ∈ I and α > 0 , we get

α ⁢ v α ⁢ ( x 0 , ξ ) + Θ ⁢ v α ⁢ ( x 0 , ξ ) = 0 ,

which tells us that (A.1) holds with c ⁢ ( ξ ) ≡ 0 . By Proposition A.2, v ¯ and v ¯ are a subsolution and a supersolution to (4.2) with c = 0 , respectively.

By the definition, we see v ¯ ⩾ v ¯ in 𝕋 d × I . On the other hand, Proposition 4.1 and (3.7) imply v ¯ = v ¯ = 0 in 𝒜 × I . Then Theorem 4.2 guarantees v ¯ ⩽ v ¯ in 𝕋 d × I . Hence, v ¯ = v ¯ = : v in 𝕋 d × I . Because v ¯ and - v ¯ are upper semi-continuous in 𝕋 d × I , v is continuous on 𝕋 d × I . Noting that 𝕋 d × I is compact, by the basic property of half-relaxed limits, we can see v α → v uniformly on 𝕋 d × I as α → 0 . ∎

Acknowledgements

The author would like to thank Professor Hiroyoshi Mitake for his helpful comments and suggestions.

References

[1] F. Camilli, O. Ley, P. Loreti and V. D. Nguyen, Large time behavior of weakly coupled systems of first-order Hamilton–Jacobi equations, NoDEA Nonlinear Differential Equations Appl. 19 (2012), no. 6, 719–749. 10.1007/s00030-011-0149-7Search in Google Scholar

[2] A. Davini, A. Fathi, R. Iturriaga and M. Zavidovique, Convergence of the solutions of the discounted Hamilton–Jacobi equation: Convergence of the discounted solutions, Invent. Math. 206 (2016), no. 1, 29–55. 10.1007/s00222-016-0648-6Search in Google Scholar

[3] A. Davini and M. Zavidovique, Convergence of the solutions of discounted Hamilton–Jacobi systems, Adv. Calc. Var. (2019), 10.1515/acv-2018-0037. 10.1515/acv-2018-0037Search in Google Scholar

[4] H. Engler and S. M. Lenhart, Viscosity solutions for weakly coupled systems of Hamilton–Jacobi equations, Proc. London Math. Soc. (3) 63 (1991), no. 1, 212–240. 10.1112/plms/s3-63.1.212Search in Google Scholar

[5] W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, Appl. Math. (N. Y.) 25, Springer, New York, 1993. Search in Google Scholar

[6] H. Ishii, An example in the vanishing discount problem for monotone systems of Hamilton–Jacobi equations, preprint (2020), https://arxiv.org/abs/2006.02769. Search in Google Scholar

[7] H. Ishii, The vanishing discount problem for monotone systems of Hamilton–Jacobi equations. Part 1: Linear coupling, Math. Eng. 3 (2021), no. 4, Paper No. 032. 10.3934/mine.2021032Search in Google Scholar

[8] H. Ishii and L. Jin, The vanishing discount problem for monotone systems of Hamilton–Jacobi equations. Part 2: Nonlinear coupling, Calc. Var. Partial Differential Equations 59 (2020), no. 4, Paper No. 140. 10.1007/s00526-020-01768-8Search in Google Scholar

[9] H. Ishii and S. Koike, Viscosity solutions for monotone systems of second-order elliptic PDEs, Comm. Partial Differential Equations 16 (1991), no. 6–7, 1095–1128. 10.1080/03605309108820791Search in Google Scholar

[10] H. Ishii, H. Mitake and H. V. Tran, The vanishing discount problem and viscosity Mather measures. Part 1: The problem on a torus, J. Math. Pures Appl. (9) 108 (2017), no. 2, 125–149. 10.1016/j.matpur.2016.10.013Search in Google Scholar

[11] H. Ishii and K. Shimano, Asymptotic analysis for a class of infinite systems of first-order PDE: Nonlinear parabolic PDE in the singular limit, Comm. Partial Differential Equations 28 (2003), no. 1–2, 409–438. 10.1081/PDE-120019388Search in Google Scholar

[12] P.-L. Lions, G. Papanicolaou and S. R. S. Varadhan, Homogenization of Hamilton–Jacobi equations, unpublished work, 1987. Search in Google Scholar

[13] H. Mitake and H. V. Tran, Remarks on the large time behavior of viscosity solutions of quasi-monotone weakly coupled systems of Hamilton–Jacobi equations, Asymptot. Anal. 77 (2012), no. 1–2, 43–70. 10.3233/ASY-2011-1071Search in Google Scholar

[14] H. Mitake and H. V. Tran, Selection problems for a discount degenerate viscous Hamilton–Jacobi equation, Adv. Math. 306 (2017), 684–703. 10.1016/j.aim.2016.10.032Search in Google Scholar

[15] G. Namah and J.-M. Roquejoffre, Remarks on the long time behaviour of the solutions of Hamilton–Jacobi equations, Comm. Partial Differential Equations 24 (1999), no. 5–6, 883–893. 10.1080/03605309908821451Search in Google Scholar

[16] K. Shimano, Homogenization and penalization of functional first-order PDE, NoDEA Nonlinear Differential Equations Appl. 13 (2006), no. 1, 1–21. 10.1007/s00030-005-0019-2Search in Google Scholar

[17] G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Springer, New York, 1998. 10.1007/978-1-4612-0627-9Search in Google Scholar

[18] B. Ziliotto, Convergence of the solutions of the discounted Hamilton–Jacobi equation: A counterexample, J. Math. Pures Appl. (9) 128 (2019), 330–338. 10.1016/j.matpur.2019.04.005Search in Google Scholar

Received: 2020-12-07

Revised: 2021-02-22

Accepted: 2021-03-02

Published Online: 2021-04-16

Published in Print: 2023-01-01

You are currently not able to access this content.

Articles in the same Issue

https://doi.org/10.1515/acv-2020-0116

Keywords for this article

Hamilton–Jacobi–Bellman equations; infinite systems; viscosity solutions; asymptotic analysis; vanishing discount