Estimations of Means and Variances in a Markov Linear Model

Theorem 15 (Multidimensional Anscombe)

Let Y ( i ) := ( Y 1 ( i ) , Y 2 ( i ) , … , Y m ( i ) ) for i ≥ 1 be a sequence of i.i.d. real-valued random vectors with E ⁡ [ Y ( i ) ] = 0 ∈ R m and covariance matrix Σ. Let N ⁢ ( t ) be a random integer-valued random variable such that N ⁢ ( t ) t → t → ∞ a.s. θ ∈ R + . Then

Proof

Let n ⁢ ( t ) := ⌊ θ ⁢ t ⌋ , S k := ∑ i = 1 k Y ( i ) , and S k ( j ) = ∑ i = 1 k Y j ( i ) . Then

The first observation is that, since n ⁢ ( t ) is deterministic, due to the multidimensional central limit theorem, we have that

where Σ is the covariance matrix of the random vector Y ( 1 ) . Next, let ε ∈ ( 0 , 1 3 ) be given and

Then

by the union bound. Let σ i 2 := E ⁡ [ ( Y i ( 1 ) ) 2 ] < ∞ . Then we also know that

for all i = 1 , … , m , where the last inequality is valid for 𝑡 sufficiently large. Plugging this last estimation in inequality (A.2) yields, for 𝑡 sufficiently large,

for any ε ∈ ( 0 , 1 3 ) . Since 𝜀 can be chosen arbitrarily small, we deduce that

By noticing that n ⁢ ( t ) N ⁢ ( t ) → t → ∞ prob. 1 and using the multidimensional version of Slutsky’s theorem [13, Lemma 2.8], we deduce that

where the last convergence is indeed in probability since it is a convergence in distribution to a constant. Using this last equation, equation (A.1), and the multidimensional Slutsky theorem, we conclude that

Theorem 16 (Anscombe’s Multivariate Delta Method)

Let θ ∈ R k , and let { T n } n ∈ N be a sequence of 𝑘-dimensional random vectors and { X n } n ∈ N a sequence of natural-valued random variables such that

Furthermore, let h : R k → R m be once differentiable at 𝜃 with the gradient matrix ∇ ⁡ h ⁢ ( θ ) . Then

Proof

By the definition of differentiability of a vector field, we have that

where | R 2 ⁢ ( x ) | → x → θ 0 . In particular, we have that

On the other hand, it follows from the assumptions and the definition of R 2 that

Therefore, using the multidimensional Slutsky theorem [13, Lemma 2.8], we get that

where the last convergence is in probability because it is towards a constant. Using once more the multidimensional Slutsky theorem together with equations (B.1), (B.2), we conclude that