Integrative analysis with a system of semiparametric projection non-linear regression models

Let g(x) be the density of X, the joint density of (Y, X) is p(y,x|θ,f)=p(y|x,θ,f)g(x). Let

be the log-likelihood ratio. Let P be the probability measure of p(y,x|θ0,f0), Pq(Y,X|θ,f)=∫​q(y,x|θ,f)dP(y,x), the true mean of q; and let Pn q(Y,X|θ,f)=(1/n)∑i=1nq(yi,xi|θ,f)​, the empirical mean of q based on the data {(yi ,xi): i=1,…,n}.

Note that Pq(Y,X|θ,f) is the negative Kullback-Leibler divergence of p(y,x|θ,f) from p(y,x|θ0,f0), and as a function of (θ, f), it is always non-positive, attaining its maximum value of 0 at (θ, f) = (θ₀, f₀). Since (θ^,f^) is the MLE of (θ₀, f₀),

Denote d(θ,f; θ0 ,f0)=||θ−θ0||+∑j=13sups∈Dj |fj(s)−fj0(s)|​. By our model specification, Pq(Y,X|θ,f) is continuous with respect to (θ, f); also, the model is identifiable, so (θ₀, f₀) is the unique maximizer of Pq(Y,X|θ,f). Thus for all η > 0,

so if we show that 𝒜={q(y,x|θ,f):(θ,f)∈(Θ,ℱ3)} is a P-Glivenko-Cantelli class, then by Theorem 5.8 in van der Vaart [28],

which is the desired result.

Now we show that 𝒜 is P-Glivenko-Cantelli. For any function g, let ||g||L1 (P)=∫​|g(y,x)|dP(y,x), and N[ ](ϵ,𝒜,L1 (P)) be the minimum number of ϵ-brackets needed to cover 𝒜 under norm ‖⋅‖L1 (P). We first show that N[ ](ϵ,𝒜,L1(P)) is finite ∀ϵ>0.

By our specification of p(y,x|θ,f) and with (C1), it can be checked that q(y,x|θ,f) is boundedly differentiable with respect to θ, and boundedly Gáteaux differentiable with respect to f. In fact, let ℓ(θ,f|y,x) be the log-likelihood, ℓ˙θ(θ,f|y,x)=∂ℓ(θ,f|y,x)/∂θ be the score for θ, and ℓ˙fj(θ,f|y,x)[h]=∂ℓ(θ,fj+λh|y,x)/∂λ|λ=0 be the Gáteaux derivative of ℓ(θ,f|y,x) with respect to f_j in the direction h. Then for some (θ, f) lies between (θ₁, f₁) and (θ₂, f₂),

By our model specification, and (C1)–(C4) (note that (C1) together with ‖β‖= implies Θ is bounded), both ℓ˙θ(θ,f|y,x) and ℓ˙fj(θ,f|y,x)[h] have finite second moment. Consequently, by Taylor expansion and HÖlder’s inequality, there are constants 0 < C_j < ∞ (j = 1, 4), such that

So,

By (C1), N[ ](ϵ/(4C4),Θ,‖⋅‖)=O(1/ϵd), with d=dim(Θ). Since ℱ is a collection of bounded monotone functions, by Theorem 2.7.5 in van der Vaart and Wellner [29], for some constant 0 < C < ∞, and for all r ≥ 1,

Thus, for some generic constant 0<C<∞,

and so by Theorem 2.4.1 in van der Vaart and Wellner [29], 𝒜 is a Glivenko-Cantelli class with respect to P.

Below we identify the asymptotic covariance matrices of n(β^−β0) and n(α^−α0) (j = 1, 2,3 ).

We first compute the efficient score for β and α. For a function g(⋅), denote g˙(⋅) for its derivative, ℓ˙β(θ,f|y,x)=∂ℓ(θ,f|y,x)/∂β, and ℓ˙α(θ,f|y,x)=∂ℓ(θ,f|y,x)/∂α. Denote f˙(x⊺ β)⊗x=(f˙1(x⊺ β1)x⊺ ,f˙2(x⊺ β2)x⊺ ,f˙3(x⊺ β3)x⊺)⊺. We have

are the scores for β and α. We assume dim(β) > 1. The score operator for f at direction g = (g₁, g₂, g₃)^⊺ is, with g(x⊺ β)=(g1(x⊺ β1),g2(x⊺ β2),g3(x⊺ β3))⊺,

Note that ℓ˙θ*(θ0,f0|y,x)=ℓ˙θ(θ0,f0|y,x)−ℓ˙f(θ0,f0|y,x)[g*] is the efficient score for θ, where g*=(g1*,…,gd*)⊺(d=dim(θ)) is the least favorable direction, ℓ˙f(θ0,f0|y,x)[g*]=(ℓ˙f(θ0,f0|y,x)[g1],…,ℓ˙f(θ0,f0|y,x)[gd])⊺, and g^* is determined by

Let P_n and P as defined in the proof of Theorem 1, Z = (Y, X), and ℓ˜θ(θ0,f0|Z) be given latter, with n(Pn−P)ℓ˜θ(θ0,f0|Z)→DN(0, *I(θ0|f0)). Below we will show

This will complete the proof.

Since we proved the asymptotic equivalent of (θ^,f^) and (θ˜,f˜), now we derive the asymptotic distribution of θ^ as if it were obtained along with a version of f^ which has the adequate smoothness, in particular, it has consistent a derivative f˙^ with ||f˙^−f˙0||→P0. f˙^ will appear in ℓ˜˙θ(θ^,f^|z), which will be used in (A.2) latter. Due to the constraint ||β||=1 and α = 1, we need Lagrange’s method. Let

and

Applying the constraint ||β||2=1 and ||α||2=1 to get η=Pn{β⊺ℓ˙β(θ,f|Z)}/2 and ζ=Pn{α⊺ℓ˙α(θ,f|Z)}/2, and then Pn ℓ˜˙β(θ,f,η,ζ|Z)=Pn ℓ˜˙β(θ,f|Z) and Pn ℓ˜˙α(θ,f,η,ζ|Z)=Pn ℓ˜˙α(θ,f|Z), with

Denote ℓ˜˙θ(θ,f|z)=(ℓ˜˙β⊺(θ,f|z),ℓ˜˙α⊺(θ,f|z),ℓ˙Ω⊺(θ,f|z))⊺.

Since (θ^,f^) is the MLE, Pn ℓ˜˙θ(θ^,f^|Z)=0 and Pn ℓ˙f(θ^,f^|Z)[g*]=0. Also, it is apparent that Pℓ˜˙θ(θ0,f0|Z)=0 and Pℓ˙f(θ0,f0|Z)[g*]=0. Let ℳ1={ℓ˜˙θ(θ,f|z): θ∈Θ,f∈ℱ3} and ℳ2={ℓ˙f(θ,f|z)[g*]: θ∈Θ,f∈ℱ3}. Using the entropy calculation method in the proof of Theorem 1, it can be checked that ℳ₁ and ℳ₂ are Donsker classes. Since by Theorem 1, ‖θ^−θ0‖+∑​j=13‖f^j−fj0‖→0 (a.s.), also, conditions (C1)–(C4) and the given model imply ℓ˜˙θ(θ,f|z) and ℓ˙f(θ,f|z)[g*] are continuous in (θ, f) and with finite second moments, thus by Corollary 2.3.12 (ii) in van der Vaart and Wellner [29], we have

where the o_p(1) is in the vector sense, and

From these facts we get

and

Denote ℓ¨θθ(θ,f|z)=∂ℓ(θ,f|z)/(∂θ∂θ⊺), ℓ¨θfj(θ,f)[g]=∂ℓ˙θ(θ,fj+λg|z)/∂λ|λ=0, and define ℓ¨fjθ(θ,f|z)[g], ℓ¨fi fj(θ,f|z)[g1,g2], and ℓ˜¨θθ(θ,fj|z), ℓ˜¨θfj(θ,f|z)[g] similarly. In particular, ℓ˜¨θf(θ,fj|z)[g]=ℓ¨θfj(θ,f|z)[g], and ℓ˜¨θθ(θ,f|z)=(ℓij)1≤i,j≤3, with ℓ11=ℓ¨ββ(θ,f|z)−β⊺ ℓ˙β(θ,f|z)I1−β{β⊺ ℓ¨ββ(θ,f|z)+ℓ˙β⊺(θ,f|z)}, where I₁ is the dim(β)-dimensional identity matrix; ℓ12=ℓ21⊺=ℓ¨βα(θ,f|z)−β⊺ ℓ¨βα(θ,f|z)β, ℓ13=ℓ31⊺=ℓ¨βΩ(θ,f|z)−β⊺ ℓ¨βΩ(θ,f|z)β, ℓ22=ℓ¨αα(θ,f|z)−α⊺ ℓ˙α(θ,f|z)I2−α{α⊺ ℓ¨αα(θ,f|z)+ℓ˙α⊺(θ,f|z)}, ℓ23=ℓ32⊺=ℓ¨αΩ(θ,f|z)−α⊺ ℓ¨αΩ(θ,f|z)α, and ℓ33=ℓ¨ΩΩ(θ,f|z), here we treat Ω as a vector in the computation of partial derivatives.

Using Taylor expansion, and note that ‖θ^−θ0‖2+∑j=13‖f^j−fj0‖2 is bounded, so by the Lemma and dominated convergence E(‖θ^−θ0‖2+∑​j=13‖f^j−fj0‖2)=O(n−2/3), also, Pℓ˜¨θfj(θ0,f0|Z)[f^j−fj0]=Pℓ¨θf(θ0,f0|Z)[f^j−fj0], and so

The above and (A.2) give

Similarly,

The above and (A.3) give

Now, subtracting (A.4) from (A.5) we get

It is known that P(ℓ˙θ(θ0,f0|Z)ℓ˙θ⊺(θ0,f0|Z))=−Pℓ¨θθ(θ0,f0|Z). By the same way, −Pℓ¨θf(θ0,f0|Z)[g]=P(ℓ˙θ(θ0,f0|Z)ℓ˙f(θ0,f0|Z)[g]) for all g, and −P(ℓ¨ff(θ0,f0|Z)[g1,g2])=P(ℓ˙f(θ0,f0|Z)[g1]ℓ˙f(θ0,f0|Z)[g2]) for all g₁, g₂. Note that (A.1) holds for any g. Thus,

So (A.6) is

Denote J*(θ0|f0)=P(ℓ˜¨θθ(θ0,f0|Z)−ℓ¨fθ(θ0,f0|Z)[g*]) and ℓθ*˜˙(θ0,f0|Z)=ℓ˜˙θ(θ0,f0|Z)−ℓ˙f(θ0,f0|Z)[g*], and I*(θ0|f0)=E(θ0,f0)[ℓ˜˙∗θ(θ0,f0|Z)ℓ˜˙θ∗T(θ0,f0|Z)]. Since ||β^||=1, J*− is a d-dimensional matrix. Let J*− be its Moore-Penrose inverse, then the above is written as

which gives the desired result.

The mean in the right hand side above is