A Tensor Decomposition Algorithm for Large ODEs with Conservation Laws
-
Sergey V. Dolgov
Abstract
We propose an algorithm for solution of high-dimensional evolutionary equations (ODEs and discretized time-dependent PDEs) in the Tensor Train (TT) decomposition, assuming that the solution and the right-hand side of the ODE admit such a decomposition with a low storage. A linear ODE, discretized via one-step or Chebyshev differentiation schemes, turns into a large linear system. The tensor decomposition allows to solve this system for several time points simultaneously using an extension of the Alternating Least Squares algorithm. This method computes a reduced TT model of the solution, but in contrast to traditional offline-online reduction schemes, solving the original large problem is never required. Instead, the method solves a sequence of reduced Galerkin problems, which can be set up efficiently due to the TT decomposition of the right-hand side. The reduced system allows a fast estimation of the time discretization error, and hence adaptation of the time steps. Besides, conservation laws can be preserved exactly in the reduced model by expanding the approximation subspace with the generating vectors of the linear invariants and correction of the Euclidean norm. In numerical experiments with the transport and the chemical master equations, we demonstrate that the new method is faster than traditional time stepping and stochastic simulation algorithms, whereas the invariants are preserved up to the machine precision irrespectively of the TT approximation accuracy.
Funding source: Engineering and Physical Sciences Research Council
Award Identifier / Grant number: EP/M019004/1
Funding statement: The author acknowledges funding from the EPSRC fellowship EP/M019004/1.
References
[1] A. C. Antoulas, D. C. Sorensen and S. Gugercin, A survey of model reduction methods for large-scale systems, Structured Matrices in Mathematics, Computer Science, and Engineering. I (Boulder 1999), Contemp. Math. 280, American Mathematical Society, Providence (2001), 193–219. 10.1090/conm/280/04630Suche in Google Scholar
[2] P. Benner, S. Gugercin and K. Willcox, A survey of projection-based model reduction methods for parametric dynamical systems, SIAM Rev. 57 (2015), no. 4, 483–531. 10.1137/130932715Suche in Google Scholar
[3] H.-J. Bungartz and M. Griebel, Sparse grids, Acta Numer. 13 (2004), 147–269. 10.1017/CBO9780511569975.003Suche in Google Scholar
[4] G. D. Byrne and A. C. Hindmarsh, A polyalgorithm for the numerical solution of ordinary differential equations, ACM Trans. Math. Software 1 (1975), no. 1, 71–96. 10.1145/355626.355636Suche in Google Scholar
[5] V. de Silva and L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem, SIAM J. Matrix Anal. Appl. 30 (2008), no. 3, 1084–1127. 10.1137/06066518XSuche in Google Scholar
[6] S. Dolgov and B. Khoromskij, Two-level QTT-Tucker format for optimized tensor calculus, SIAM J. Matrix Anal. Appl. 34 (2013), no. 2, 593–623. 10.1137/120882597Suche in Google Scholar
[7] S. Dolgov and B. Khoromskij, Simultaneous state-time approximation of the chemical master equation using tensor product formats, Numer. Linear Algebra Appl. 22 (2015), no. 2, 197–219. 10.1002/nla.1942Suche in Google Scholar
[8] S. V. Dolgov, B. N. Khoromskij and I. V. Oseledets, Fast solution of parabolic problems in the tensor train/quantized tensor train format with initial application to the Fokker–Planck equation, SIAM J. Sci. Comput. 34 (2012), no. 6, A3016–A3038. 10.1137/120864210Suche in Google Scholar
[9] S. V. Dolgov and D. V. Savostyanov, Alternating minimal energy methods for linear systems in higher dimensions, SIAM J. Sci. Comput. 36 (2014), no. 5, A2248–A2271. 10.1137/140953289Suche in Google Scholar
[10] M. Fannes, B. Nachtergaele and R. F. Werner, Finitely correlated states on quantum spin chains, Comm. Math. Phys. 144 (1992), no. 3, 443–490. 10.1007/BF02099178Suche in Google Scholar
[11] D. T. Gillespie, A general method for numerically simulating the stochastic time evolution of coupled chemical reactions, J. Comput. Phys. 22 (1976), no. 4, 403–434. 10.1016/0021-9991(76)90041-3Suche in Google Scholar
[12] I. G. Graham, F. Y. Kuo, D. Nuyens, R. Scheichl and I. H. Sloan, Quasi-Monte Carlo methods for elliptic PDEs with random coefficients and applications, J. Comput. Phys. 230 (2011), no. 10, 3668–3694. 10.1016/j.jcp.2011.01.023Suche in Google Scholar
[13] L. Grasedyck, Hierarchical singular value decomposition of tensors, SIAM J. Matrix Anal. Appl. 31 (2009/10), no. 4, 2029–2054. 10.1137/090764189Suche in Google Scholar
[14] A. Gupta and M. Khammash, Determining the long-term behavior of cell populations: A new procedure for detecting ergodicity in large stochastic reaction networks, IFAC Proc. 47 (2014), no. 3, 1711–1716. 10.3182/20140824-6-ZA-1003.00291Suche in Google Scholar
[15] W. Hackbusch, Tensor Spaces and Numerical Tensor Calculus, Springer Ser. Comput. Math. 42, Springer, Heidelberg, 2012. 10.1007/978-3-642-28027-6Suche in Google Scholar
[16] M. Hegland, C. Burden, L. Santoso, S. MacNamara and H. Booth, A solver for the stochastic master equation applied to gene regulatory networks, J. Comput. Appl. Math. 205 (2007), no. 2, 708–724. 10.1016/j.cam.2006.02.053Suche in Google Scholar
[17] F. L. Hitchcock, Multiple invariants and generalized rank of a p-way matrix or tensor, J. Math. Phys. 7 (1927), no. 1, 39–79. 10.1002/sapm19287139Suche in Google Scholar
[18] S. Holtz, T. Rohwedder and R. Schneider, The alternating linear scheme for tensor optimization in the tensor train format, SIAM J. Sci. Comput. 34 (2012), no. 2, A683–A713. 10.1137/100818893Suche in Google Scholar
[19] T. Jahnke, On reduced models for the chemical master equation, Multiscale Model. Simul. 9 (2011), no. 4, 1646–1676. 10.1137/110821500Suche in Google Scholar
[20] T. Jahnke and W. Huisinga, A dynamical low-rank approach to the chemical master equation, Bull. Math. Biol. 70 (2008), no. 8, 2283–2302. 10.1007/s11538-008-9346-xSuche in Google Scholar PubMed
[21] E. Jeckelmann, Dynamical density–matrix renormalization–group method, Phys. Rev. B 66 (2002), Article ID 045114. 10.1103/PhysRevB.66.045114Suche in Google Scholar
[22] V. A. Kazeev, B. N. Khoromskij and E. E. Tyrtyshnikov, Multilevel Toeplitz matrices generated by tensor-structured vectors and convolution with logarithmic complexity, SIAM J. Sci. Comput. 35 (2013), no. 3, A1511–A1536. 10.1137/110844830Suche in Google Scholar
[23] V. A. Kazeev, O. Reichmann and C. Schwab, hp-DG-QTT solution of high-dimensional degenerate diffusion equations, Technical Report 2012-11, ETH SAM, Zürich, 2012. Suche in Google Scholar
[24] G. Kerschen, J.-C. Golinval, A. F. Vakakis and L. A. Bergman, The method of proper orthogonal decomposition for dynamical characterization and order reduction of mechanical systems: an overview, Nonlinear Dynam. 41 (2005), no. 1–3, 147–169. 10.1007/s11071-005-2803-2Suche in Google Scholar
[25]
B. N. Khoromskij,
[26] B. N. Khoromskij, Tensor numerical methods for multidimensional PDEs: Theoretical analysis and initial applications, CEMRACS 2013—Modelling and Simulation of Complex Systems: Stochastic and Deterministic Approaches, ESAIM Proc. Surveys 48, EDP Science, Les Ulis (2015), 1–28. 10.1051/proc/201448001Suche in Google Scholar
[27] O. Koch and C. Lubich, Dynamical tensor approximation, SIAM J. Matrix Anal. Appl. 31 (2010), no. 5, 2360–2375. 10.1137/09076578XSuche in Google Scholar
[28] T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev. 51 (2009), no. 3, 455–500. 10.1137/07070111XSuche in Google Scholar
[29] C. Lubich, I. V. Oseledets and B. Vandereycken, Time integration of tensor trains, SIAM J. Numer. Anal. 53 (2015), no. 2, 917–941. 10.1137/140976546Suche in Google Scholar
[30] J. L. Lumley, The structure of inhomogeneous turbulent flows, Atmospheric Turbulence and Radio Wave Propagation, Nauka, Moscow (1967), 166–178. Suche in Google Scholar
[31] C. Moler and C. Van Loan, Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later, SIAM Rev. 45 (2003), no. 1, 3–49. 10.1137/S00361445024180Suche in Google Scholar
[32] B. Munsky and M. Khammash, A multiple time interval finite state projection algorithm for the solution to the chemical master equation, J. Comput. Phys. 226 (2007), no. 1, 818–835. 10.1016/j.jcp.2007.05.016Suche in Google Scholar
[33] H. Niederreiter, Quasi-Monte Carlo methods and pseudo-random numbers, Bull. Amer. Math. Soc. 84 (1978), no. 6, 957–1041. 10.1090/S0002-9904-1978-14532-7Suche in Google Scholar
[34] A. Nouy, A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations, Comput. Methods Appl. Mech. Engrg. 199 (2010), no. 23–24, 1603–1626. 10.1016/j.cma.2010.01.009Suche in Google Scholar
[35] I. V. Oseledets, Tensor-train decomposition, SIAM J. Sci. Comput. 33 (2011), no. 5, 2295–2317. 10.1137/090752286Suche in Google Scholar
[36] I. V. Oseledets and S. V. Dolgov, Solution of linear systems and matrix inversion in the TT-format, SIAM J. Sci. Comput. 34 (2012), no. 5, A2718–A2739. 10.1137/110833142Suche in Google Scholar
[37] T. Rohwedder and A. Uschmajew, On local convergence of alternating schemes for optimization of convex problems in the tensor train format, SIAM J. Numer. Anal. 51 (2013), no. 2, 1134–1162. 10.1137/110857520Suche in Google Scholar
[38] D. V. Savostyanov, S. V. Dolgov, J. M. Werner and I. Kuprov, Exact NMR simulation of protein-size spin systems using tensor train formalism, Phys. Rev. B 90 (2014), Article ID 085139. 10.1103/PhysRevB.90.085139Suche in Google Scholar
[39] U. Schollwöck, The density-matrix renormalization group in the age of matrix product states, Ann. Physics 326 (2011), no. 1, 96–192. 10.1016/j.aop.2010.09.012Suche in Google Scholar
[40] D. Schötzau, hp-DGFEM for parabolic evolution problems. Applications to diffusion and viscous incompressible fluid flow, PhD thesis, ETH, Zürich, 1999. 10.1007/s100920050004Suche in Google Scholar
[41] L. Sirovich, Turbulence and the dynamics of coherent structures. I. Coherent structures, Quart. Appl. Math. 45 (1987), no. 3, 561–571. 10.1090/qam/910462Suche in Google Scholar
[42] C. G. Small and J. Wang, Numerical Methods for Nonlinear Estimating Equations, Oxford Statist. Sci. Ser. 29, The Clarendon Press, Oxford, 2003. 10.1093/acprof:oso/9780198506881.001.0001Suche in Google Scholar
[43] S. A. Smoljak, Quadrature and interpolation formulae on tensor products of certain function classes, Dokl. Akad. Nauk SSSR 148 (1963), 1042–1045. Suche in Google Scholar
[44] E. Tadmor, The exponential accuracy of Fourier and Chebyshev differencing methods, SIAM J. Numer. Anal. 23 (1986), no. 1, 1–10. 10.1137/0723001Suche in Google Scholar
[45] L. N. Trefethen, Spectral Methods in MATLAB, Software Environ. Tools 10, Society for Industrial and Applied Mathematics, Philadelphia, 2000. 10.1137/1.9780898719598Suche in Google Scholar
[46] G. Vidal, Efficient simulation of one-dimensional quantum many-body systems, Phys. Rev. Lett. 93 (2004), Article ID 040502. 10.1103/PhysRevLett.93.040502Suche in Google Scholar PubMed
[47] T. von Petersdorff and C. Schwab, Numerical solution of parabolic equations in high dimensions, M2AN Math. Model. Numer. Anal. 38 (2004), no. 1, 93–127. 10.1051/m2an:2004005Suche in Google Scholar
[48] S. R. White, Density-matrix algorithms for quantum renormalization groups, Phys. Rev. B 48 (1993), no. 14, 10345–10356. 10.1103/PhysRevB.48.10345Suche in Google Scholar PubMed
[49] S. R. White and A. E. Feiguin, Real-time evolution using the density matrix renormalization group, Phys. Rev. Lett. 93 (2004), Article ID 076401. 10.1103/PhysRevLett.93.076401Suche in Google Scholar PubMed
© 2018 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Frontmatter
- Tensor Numerical Methods: Actual Theory and Recent Applications
- A Low-Rank Inexact Newton–Krylov Method for Stochastic Eigenvalue Problems
- A Tensor Decomposition Algorithm for Large ODEs with Conservation Laws
- Non-intrusive Tensor Reconstruction for High-Dimensional Random PDEs
- Quasi-Optimal Rank-Structured Approximation to Multidimensional Parabolic Problems by Cayley Transform and Chebyshev Interpolation
- Projection Methods for Dynamical Low-Rank Approximation of High-Dimensional Problems
- Tensor Train Spectral Method for Learning of Hidden Markov Models (HMM)
- Tucker Tensor Analysis of Matérn Functions in Spatial Statistics
- Low-Rank Space-Time Decoupled Isogeometric Analysis for Parabolic Problems with Varying Coefficients
- Approximate Solution of Linear Systems with Laplace-like Operators via Cross Approximation in the Frequency Domain
- Rayleigh Quotient Methods for Estimating Common Roots of Noisy Univariate Polynomials
Artikel in diesem Heft
- Frontmatter
- Tensor Numerical Methods: Actual Theory and Recent Applications
- A Low-Rank Inexact Newton–Krylov Method for Stochastic Eigenvalue Problems
- A Tensor Decomposition Algorithm for Large ODEs with Conservation Laws
- Non-intrusive Tensor Reconstruction for High-Dimensional Random PDEs
- Quasi-Optimal Rank-Structured Approximation to Multidimensional Parabolic Problems by Cayley Transform and Chebyshev Interpolation
- Projection Methods for Dynamical Low-Rank Approximation of High-Dimensional Problems
- Tensor Train Spectral Method for Learning of Hidden Markov Models (HMM)
- Tucker Tensor Analysis of Matérn Functions in Spatial Statistics
- Low-Rank Space-Time Decoupled Isogeometric Analysis for Parabolic Problems with Varying Coefficients
- Approximate Solution of Linear Systems with Laplace-like Operators via Cross Approximation in the Frequency Domain
- Rayleigh Quotient Methods for Estimating Common Roots of Noisy Univariate Polynomials
