On functional reproducing kernels

Weiqi Zhou

doi:10.1515/math-2023-0102

Article Open Access

On functional reproducing kernels

Weiqi Zhou

Published/Copyright: August 4, 2023

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information Explore this Subject

$Open Mathematics$

From the journal Open Mathematics Volume 21 Issue 1

Abstract

We show that even if a Hilbert space does not admit a reproducing kernel, there could still be a kernel function that realizes the Riesz representation map. Constructions in spaces that are the Fourier transform of weighted L 2 spaces are given. With a mild assumption on the weight function, we are able to reproduce Riesz representatives of all functionals through a limit procedure from computable integrals over compact sets, despite that the kernel is not necessarily in the underlying Hilbert space. Distributional kernels are also discussed.

Keywords: reproducing kernels; Riesz representations; Sobolev spaces; weighted L 2 spaces; positive definite

MSC 2010: 46E22; 46E35; 47B34; 42A85

1 Motivations

A Hilbert space H is said to be a reproducing kernel Hilbert spaces if there exists an element K ∈ H ⊗ H ( ⊗ denotes the tensor product) such that all elements in H can be reproduced by an inner product with K :

f ( y ) = ⟨ f ( ⋅ ) , K ( ⋅ , y ) ⟩ H , ∀ f ∈ H .

The history of relevant theories dates back to [1,2]. Examples of reproducing kernel Hilbert space include spaces of band-limited functions, certain Sobolev spaces, etc.

Reproducing kernel Hilbert spaces can be characterized as spaces in which evaluation functionals are well defined and bounded with respect to the underlying norm. In practice, it is also common to consider other types of bounded linear functionals. A classical application is the linear optimal recovery problem, i.e., to approximate λ ( f ) for unknown functional λ and unknown function f from a set of known functional values λ 1 ( f ) , … , λ n ( f ) (see the exposition in [3]), typical examples of such functionals are, e.g., local averages of f around distinct points x 1 , … , x n .

By the Riesz representation theorem for Hilbert spaces, given a Hilbert space H , any functional in the dual H ∗ can be represented by an element in H , i.e., there is an isometry ρ : H ∗ → H such that

⟨ f , g ⟩ = ⟨ f , ρ ( g ) ⟩ H

holds for all f ∈ H and g ∈ H ∗ . The isometry ρ is then called the Riesz representation map, and ρ ( g ) is said to be the Riesz representative of the functional g . The notion ⟨ ⋅ , ⋅ ⟩ without subscript will always be used for dual pairings in this note while ⟨ ⋅ , ⋅ ⟩ H will be used for the inner product on some Hilbert space H .

If H is a reproducing kernel Hilbert space of functions, then Riesz representatives can be easily found by applying corresponding functionals to the kernel. This gives great convenience in practice for storage and computation since an abstract dual pairing between a functional (distribution) and a function now becomes a concrete integral between two continuous functions, which can be assessed numerically by, e.g., quadrature formulas. Such methods are generally referred to as kernel methods, and are proven to be successful in many application areas [4–6].

The purpose of this short note is to report the possibility for spaces without reproducing kernels to have a similar kernel that can reproduce functionals, i.e., to obtain a continuous Riesz representative of a functional by applying the functional to the kernel with an integral formula in a properly defined way. Naturally, some restrictions are needed.

Among various ways of constructing a reproducing kernel Hilbert space, we shall focus on the so-called native space of a given continuous positive weight function u . The construction of native spaces requires that u − 1 ∈ L 1 , we will relax this condition to u − 1 ∈ L ∞ and reproduce Riesz representatives of all functionals through a limit procedure from computable integrals over compact sets, despite that the kernel is not necessarily in the underlying Hilbert space. Distributional kernels are also discussed.

The next section provides prerequisite materials, while the main results are presented in the last section.

2 Preliminaries

Denote S and S ′ , respectively, as the Schwartz class and the space of tempered distributions in R d . We adopt the following form of the Fourier transform on S ,

f ˆ ( ξ ) = ∫ R d f ( x ) e − 2 π i ξ ⋅ x d x

and extend it to S ′ by dual pairing. Both f ˆ and F x → ξ f denote the Fourier transform of f , while the Fourier inverse transform of f is denoted as f ˇ or F ξ → y − 1 f . Subscripts in F x → ξ and F ξ → y − 1 indicate how variables are mapped; they may also be omitted if this is clear from the context.

Denote C c ∞ as the space of compactly supported smooth functions, and C c 0 as the space of compactly supported continuous functions. The Riesz representation theorem for measures states that the dual of C c 0 is the space of signed Radon measures.

Given a continuous and positive function u on R d , we may associate it with a weighted L 2 space defined as

H u = { f ∈ S ′ : f ˆ u 1 ∕ 2 ∈ L 2 } ,

whose dual space may be similarly defined as

H u ∗ = { g ∈ S ′ : g ˆ u − 1 ∕ 2 ∈ L 2 }

( u − 1 ∕ 2 is well defined since u is positive and thus has no zeros). Their norms are simply

‖ f ‖ H u = ‖ f ˆ u 1 ∕ 2 ‖ L 2 , ‖ g ‖ H u ∗ = ‖ g ˆ u − 1 ∕ 2 ‖ L 2 ,

with dual pairing

⟨ f , g ⟩ = ∫ R d f ˆ ( ξ ) g ˆ ( ξ ) ¯ d ξ .

For convenience, let us further write

Φ ( x ) = ( F − 1 u − 1 ) ( x )

if F − 1 u − 1 happens to be a function.

Lemma 1

If u is a continuous and positive function with lim ‖ x ‖ → ∞ u ( x ) = ∞ , then C c ∞ is dense in H u ∗ .

Proof

As u is positive and continuous, the condition lim x → ∞ u ( x ) = ∞ implies that u − 1 is bounded, thus L 2 is continuously embedded into H u ∗ since

‖ g ‖ H u ∗ 2 = ‖ ∣ g ˆ ∣ 2 u − 1 ‖ L 1 ≤ ‖ ∣ g ˆ ∣ 2 ‖ L 1 ‖ u − 1 ‖ L ∞ = ‖ g ˆ ‖ L 2 2 ‖ u − 1 ‖ L ∞ = ‖ g ‖ L 2 2 ‖ u − 1 ‖ L ∞

holds for any g ∈ L 2 . It then suffices to show that S is dense in H u ∗ since C c ∞ is dense in S with respect to the L 2 norm.

Let C be a compact set and denote g ˆ c , u c − 1 , u c as the restriction of g ˆ , u , u − 1 to C , respectively. Clearly, u c is bounded because u is continuous, therefore

‖ g ˆ c ‖ L 2 2 = ‖ ∣ g ˆ c ∣ 2 ‖ L 1 ≤ ‖ ∣ g ˆ c ∣ 2 u c − 1 ‖ L 1 ‖ u c ‖ L ∞ ≤ ‖ g ˆ u − 1 2 ‖ L 2 2 ‖ u c ‖ L ∞ = ‖ g ‖ H u ∗ 2 ‖ u c ‖ L ∞ ,

which indicates that g ˆ c is in L 2 and can thus be approximated by Schwartz class functions. Now, for any given ε > 0 , we can choose C large enough and some h ∈ S so that

‖ ( g ˆ − g ˆ c ) u − 1 2 ‖ L 2 2 = ‖ ∣ g − g c ∣ 2 ⋅ u − 1 ‖ L 1 ≤ ε , ‖ g ˆ c − h ‖ L 2 2 = ‖ ∣ g ˆ c − h ∣ 2 ‖ L 1 ≤ ε ,

consequently

‖ g − h ˇ ‖ H u ∗ 2 = ‖ ( g ˆ − h ) u − 1 2 ‖ L 2 2 = ‖ ∣ g ˆ − h ∣ 2 ⋅ u − 1 ‖ L 1 ≤ 2 ( ‖ ∣ g ˆ − g ˆ c ∣ 2 ⋅ u − 1 ‖ L 1 + ‖ ∣ g ˆ c − h ∣ 2 ⋅ u − 1 ‖ L 1 ) ≤ 2 ( ε + ‖ ∣ g ˆ c − h ∣ 2 ‖ L 1 ‖ u − 1 ‖ L ∞ ) ≤ 2 ( 1 + ‖ u − 1 ‖ L ∞ ) ε ,

which shows that S is dense in H u ∗ since h ˇ 2 ∈ S .□

The speed u ( x ) → ∞ as x → ∞ determines how fast u − 1 ( x ) decays to 0 as x → ∞ , which is critical for the reproducing property of H u . If u − 1 ∈ L 1 , then H u becomes a reproducing kernel Hilbert space, which has a positive definite convolution kernel Φ ( x − y ) . This is the construction of the so-called native spaces (see, e.g., [5, Chapter 10]) of Φ , among which Sobolev spaces are standard examples. Relevant theories go back to [7].

If u − 1 ∉ L 1 , then H u in general does not admit a reproducing kernel. Nevertheless, it is still possible to realize the Riesz representation map for linear functionals in H u ∗ using u − 1 .

Lemma 2

For any g ∈ H u ∗ , the map

ρ : g ↦ F − 1 ( g ˆ u − 1 )

realizes the Riesz representation map, i.e.,

⟨ f , ρ ( g ) ⟩ H u = ⟨ f , g ⟩

holds for every f ∈ H u and g ∈ H u ∗ .

Proof

By virtue of this definition, we obtain

‖ ρ ( g ) ‖ H u 2 = ∫ R d ∣ g ˆ ( ξ ) u − 1 ( ξ ) ∣ 2 u ( ξ ) d ξ = ∫ R d ∣ g ˆ ( ξ ) ∣ 2 u − 1 ( ξ ) d ξ = ‖ g ‖ H u ∗ 2 ,

and

⟨ f , ρ ( g ) ⟩ H u = ∫ R d f ˆ ( ξ ) g ˆ ( ξ ) u − 1 ( ξ ) ¯ u ( ξ ) d ξ = ∫ R d f ˆ ( ξ ) g ˆ ( ξ ) ¯ d ξ = ⟨ f , g ⟩ ,

which shows that ρ is an isometry that reproduces any functional g .□

lim ‖ x ‖ → ∞ u ( x ) = ∞ ,

then u − 1 is bounded and

‖ f ‖ L 2 2 = ‖ f ˆ ‖ L 2 2 = ‖ ∣ f ˆ ∣ 2 ‖ L 1 = ‖ ∣ f ˆ ∣ 2 u u − 1 ‖ L 1 ≤ ‖ ∣ f ˆ ∣ 2 u ‖ L 1 ‖ u − 1 ‖ L ∞ = ‖ f ‖ H u 2 ‖ u − 1 ‖ L ∞

holds for every f ∈ H u , which shows that H u is then continuously embedded into L 2 , and thus F − 1 ( g ˆ u − 1 ) is at least an L 2 function, which is sufficient for computing integrations already. This is useful for implementing the functional g , which may itself be a distribution. As L 2 is formed by equivalence classes of functions, to access the value of F − 1 ( g ˆ u − 1 ) one often further requires it to be continuous. If g and u − 1 are nice enough, then F − 1 ( g ˆ u − 1 ) may be easily computed by the convolution theorem. However, without extra assumptions, a convolution between g and F − 1 u − 1 might not exist.

3 Main results

Theorem 1

Let u be a continuous and positive function on R d with lim ‖ x ‖ → ∞ u ( x ) = ∞ and Φ ( x ) = ( F − 1 u − 1 ) ( x ) a locally integrable function (i.e., absolutely integrable on any compact set). Denote Φ y ( x ) = ( F − 1 u − 1 ) ( x − y ) for each y ∈ R d . If { g n ( x ) } n ∈ N is a sequence of functions in C c ∞ that converges to g in H u ∗ , then the Riesz representative of any functional g ∈ H u ∗ can be reproduced as follows:

(1) g ˜ ( y ) = lim n → ∞ ∫ R d g n ( x ) d Φ y ( x ) .

Proof

The existence of { g n ( x ) } n ∈ N with g n → g in H u ∗ is guaranteed by Lemma 1. For any f in C c ∞ , we have

∫ R d f ( x ) Φ y ( x ) d x < ∞ ,

as Φ is assumed to be locally integrable, thus d Φ y ( x ) represents a signed Radon measure (see, e.g., [8, Theorem 7.2]), therefore the integral in the right-hand side of equation (1) is well defined.

Now by Lemma 2, the Riesz representative of g is simply

ρ ( g ) = F − 1 ( g ˆ u − 1 ) = lim n → ∞ F − 1 ( g ˆ n u − 1 ) .

The assumption lim x → ∞ u ( x ) = ∞ implies that lim x → ∞ u − 1 = 0 , i.e., u − 1 ∈ S ′ , while g n ∈ C c ∞ implies that g n ∈ S . As g n is compactly supported, the convolution theorem now applies, and hence

F − 1 ( g ˆ n u − 1 ) = ∫ R d g n ( x ) Φ ( x − y ) d x ,

which establishes equation (1).□

Explicit constructions of g n can be obtained following the approach in Lemma 1. If u − 1 ∉ L 1 , then the kernel Φ will have a singularity at 0 since u is positive. For a concluding remark, we point out that if we do not insist on an integral form, then it would not be difficult to realize the Riesz representation map by a distributional kernel.

Proposition 1

Let H be a Hilbert space with dual H ∗ ; if S is continuously and densely embedded in H, then there is a distribution K ∈ S ′ ⊗ S ′ and a Hilbert subspace H ˜ ⊆ H ∗ in which S is also continuously and densely embedded such that K reproduces all functionals in H ˜ .

Proof

By assumption, we obtain the continuous embedding (see [9, Chapter 14]):

S ↪ H ↪ H ∗ ↪ S ′ .

The Riesz representation map restricted to S can thus be viewed as a continuous linear map from S into S ′ , then by the Schwartz kernel theorem on nuclear spaces (see [10, Chapter 5.2] for the original form and [11, p. 172] for its generalization on nuclear spaces), it can be represented by a distribution kernel in S ′ ⊗ S ′ (this tensor product is well defined also by the Schwartz kernel theorem), i.e.,

⟨ f ⊗ g , K ⟩ = ⟨ f , g ⟩

holds for all f , g ∈ S and thus extends to entire H ⊗ H ˜ by density.□

There is no guarantee that the kernel obtained this way is a function, e.g., let H = L 2 ( R ) , then the distribution kernel asserted by Proposition 1 is simply the Dirac distribution supported at the origin. See perhaps [12] for more on reproducing kernels on distribution spaces.

Funding information: Funded by the Jiangsu Education Department (The General Program for Natural Science Research in Jiangsu Higher Institutions: Grant No. 21KJD110001).
Conflict of interest: There is no conflict of interest.
Data availability statement: There is no data associated with this article.

References

[1] S. Bergman, Über die Entwicklung der harmonischen Funktionen der Ebene und des Raumes nach Orthogonalfunktionen, Math. Ann. 86 (1922), 238–271, DOI: https://doi.org/10.1007/BF01457987. 10.1007/BF01457987Search in Google Scholar

[2] N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc. 68 (1950), 337–404, DOI: http://doi.org/10.1090/S0002-9947-1950-0051437-7. 10.1090/S0002-9947-1950-0051437-7Search in Google Scholar

[3] R. Schaback, Native Hilbert spaces for radial basis functions, in: M. Müller, M. Buhmann, D. Mache, M. Felten (Eds.), New Developments in Approximation Theory, Vol. 132, Birkhäuser, Basel, 1999, pp. 255–282. 10.1007/978-3-0348-8696-3_16Search in Google Scholar

[4] M. Buhmann, Radial Basis Functions: Theory and Implementations, Cambridge Univ. Press, Cambridge, 2003. 10.1017/CBO9780511543241Search in Google Scholar

[5] H. Wendland, Scattered Data Approximation, Cambridge Univ. Press, Cambridge, 2004. 10.1017/CBO9780511617539Search in Google Scholar

[6] A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics, Springer Science+Business Media, New York, 2011. Search in Google Scholar

[7] S. Bochner, Vorlesungen über Fouriersche Integrale, Monatsh. Math. 40 (1933), A27, DOI: http://doi.org/10.1007/BF01708925. 10.1007/BF01708925Search in Google Scholar

[8] G. Folland, Real Analysis, John Wiley & Sons, New York, 1999. Search in Google Scholar

[9] J. Simon, Banach, Fréchet, Hilbert and Neumann Spaces, John Wiley & Sons, London, 2017. 10.1002/9781119426516Search in Google Scholar

[10] L. Hörmander, The Analysis of Linear Partial Differential Operators I: Distribution Theory and Fourier Analysis, Springer-Verlag, Berlin, 1983. Search in Google Scholar

[11] H. H. Schaefer and M. P. H. Wolff, Topological Vector Spaces, Springer, New York, 1999. 10.1007/978-1-4612-1468-7Search in Google Scholar

[12] R. Meidan, Reproducing-kernel Hilbert spaces of distributions and generalized stochastic processes, SIAM J. Math. Anal. 10 (1979), no. 1, 62–70, DOI: http://doi.org/10.1137/0510007. 10.1137/0510007Search in Google Scholar

Received: 2022-12-13

Revised: 2023-05-20

Accepted: 2023-07-24

Published Online: 2023-08-04

This work is licensed under the Creative Commons Attribution 4.0 International License.

Articles in the same Issue

https://doi.org/10.1515/math-2023-0102

Keywords for this article

reproducing kernels; Riesz representations; Sobolev spaces; weighted L 2 spaces; positive definite

Creative Commons

BY 4.0