A Tensor Decomposition Algorithm for Large ODEs with Conservation Laws
-
Sergey V. Dolgov
Abstract
We propose an algorithm for solution of high-dimensional evolutionary equations (ODEs and discretized time-dependent PDEs) in the Tensor Train (TT) decomposition, assuming that the solution and the right-hand side of the ODE admit such a decomposition with a low storage. A linear ODE, discretized via one-step or Chebyshev differentiation schemes, turns into a large linear system. The tensor decomposition allows to solve this system for several time points simultaneously using an extension of the Alternating Least Squares algorithm. This method computes a reduced TT model of the solution, but in contrast to traditional offline-online reduction schemes, solving the original large problem is never required. Instead, the method solves a sequence of reduced Galerkin problems, which can be set up efficiently due to the TT decomposition of the right-hand side. The reduced system allows a fast estimation of the time discretization error, and hence adaptation of the time steps. Besides, conservation laws can be preserved exactly in the reduced model by expanding the approximation subspace with the generating vectors of the linear invariants and correction of the Euclidean norm. In numerical experiments with the transport and the chemical master equations, we demonstrate that the new method is faster than traditional time stepping and stochastic simulation algorithms, whereas the invariants are preserved up to the machine precision irrespectively of the TT approximation accuracy.
Funding source: Engineering and Physical Sciences Research Council
Award Identifier / Grant number: EP/M019004/1
Funding statement: The author acknowledges funding from the EPSRC fellowship EP/M019004/1.
References
[1] A. C. Antoulas, D. C. Sorensen and S. Gugercin, A survey of model reduction methods for large-scale systems, Structured Matrices in Mathematics, Computer Science, and Engineering. I (Boulder 1999), Contemp. Math. 280, American Mathematical Society, Providence (2001), 193–219. 10.1090/conm/280/04630Search in Google Scholar
[2] P. Benner, S. Gugercin and K. Willcox, A survey of projection-based model reduction methods for parametric dynamical systems, SIAM Rev. 57 (2015), no. 4, 483–531. 10.1137/130932715Search in Google Scholar
[3] H.-J. Bungartz and M. Griebel, Sparse grids, Acta Numer. 13 (2004), 147–269. 10.1017/CBO9780511569975.003Search in Google Scholar
[4] G. D. Byrne and A. C. Hindmarsh, A polyalgorithm for the numerical solution of ordinary differential equations, ACM Trans. Math. Software 1 (1975), no. 1, 71–96. 10.1145/355626.355636Search in Google Scholar
[5] V. de Silva and L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem, SIAM J. Matrix Anal. Appl. 30 (2008), no. 3, 1084–1127. 10.1137/06066518XSearch in Google Scholar
[6] S. Dolgov and B. Khoromskij, Two-level QTT-Tucker format for optimized tensor calculus, SIAM J. Matrix Anal. Appl. 34 (2013), no. 2, 593–623. 10.1137/120882597Search in Google Scholar
[7] S. Dolgov and B. Khoromskij, Simultaneous state-time approximation of the chemical master equation using tensor product formats, Numer. Linear Algebra Appl. 22 (2015), no. 2, 197–219. 10.1002/nla.1942Search in Google Scholar
[8] S. V. Dolgov, B. N. Khoromskij and I. V. Oseledets, Fast solution of parabolic problems in the tensor train/quantized tensor train format with initial application to the Fokker–Planck equation, SIAM J. Sci. Comput. 34 (2012), no. 6, A3016–A3038. 10.1137/120864210Search in Google Scholar
[9] S. V. Dolgov and D. V. Savostyanov, Alternating minimal energy methods for linear systems in higher dimensions, SIAM J. Sci. Comput. 36 (2014), no. 5, A2248–A2271. 10.1137/140953289Search in Google Scholar
[10] M. Fannes, B. Nachtergaele and R. F. Werner, Finitely correlated states on quantum spin chains, Comm. Math. Phys. 144 (1992), no. 3, 443–490. 10.1007/BF02099178Search in Google Scholar
[11] D. T. Gillespie, A general method for numerically simulating the stochastic time evolution of coupled chemical reactions, J. Comput. Phys. 22 (1976), no. 4, 403–434. 10.1016/0021-9991(76)90041-3Search in Google Scholar
[12] I. G. Graham, F. Y. Kuo, D. Nuyens, R. Scheichl and I. H. Sloan, Quasi-Monte Carlo methods for elliptic PDEs with random coefficients and applications, J. Comput. Phys. 230 (2011), no. 10, 3668–3694. 10.1016/j.jcp.2011.01.023Search in Google Scholar
[13] L. Grasedyck, Hierarchical singular value decomposition of tensors, SIAM J. Matrix Anal. Appl. 31 (2009/10), no. 4, 2029–2054. 10.1137/090764189Search in Google Scholar
[14] A. Gupta and M. Khammash, Determining the long-term behavior of cell populations: A new procedure for detecting ergodicity in large stochastic reaction networks, IFAC Proc. 47 (2014), no. 3, 1711–1716. 10.3182/20140824-6-ZA-1003.00291Search in Google Scholar
[15] W. Hackbusch, Tensor Spaces and Numerical Tensor Calculus, Springer Ser. Comput. Math. 42, Springer, Heidelberg, 2012. 10.1007/978-3-642-28027-6Search in Google Scholar
[16] M. Hegland, C. Burden, L. Santoso, S. MacNamara and H. Booth, A solver for the stochastic master equation applied to gene regulatory networks, J. Comput. Appl. Math. 205 (2007), no. 2, 708–724. 10.1016/j.cam.2006.02.053Search in Google Scholar
[17] F. L. Hitchcock, Multiple invariants and generalized rank of a p-way matrix or tensor, J. Math. Phys. 7 (1927), no. 1, 39–79. 10.1002/sapm19287139Search in Google Scholar
[18] S. Holtz, T. Rohwedder and R. Schneider, The alternating linear scheme for tensor optimization in the tensor train format, SIAM J. Sci. Comput. 34 (2012), no. 2, A683–A713. 10.1137/100818893Search in Google Scholar
[19] T. Jahnke, On reduced models for the chemical master equation, Multiscale Model. Simul. 9 (2011), no. 4, 1646–1676. 10.1137/110821500Search in Google Scholar
[20] T. Jahnke and W. Huisinga, A dynamical low-rank approach to the chemical master equation, Bull. Math. Biol. 70 (2008), no. 8, 2283–2302. 10.1007/s11538-008-9346-xSearch in Google Scholar PubMed
[21] E. Jeckelmann, Dynamical density–matrix renormalization–group method, Phys. Rev. B 66 (2002), Article ID 045114. 10.1103/PhysRevB.66.045114Search in Google Scholar
[22] V. A. Kazeev, B. N. Khoromskij and E. E. Tyrtyshnikov, Multilevel Toeplitz matrices generated by tensor-structured vectors and convolution with logarithmic complexity, SIAM J. Sci. Comput. 35 (2013), no. 3, A1511–A1536. 10.1137/110844830Search in Google Scholar
[23] V. A. Kazeev, O. Reichmann and C. Schwab, hp-DG-QTT solution of high-dimensional degenerate diffusion equations, Technical Report 2012-11, ETH SAM, Zürich, 2012. Search in Google Scholar
[24] G. Kerschen, J.-C. Golinval, A. F. Vakakis and L. A. Bergman, The method of proper orthogonal decomposition for dynamical characterization and order reduction of mechanical systems: an overview, Nonlinear Dynam. 41 (2005), no. 1–3, 147–169. 10.1007/s11071-005-2803-2Search in Google Scholar
[25]
B. N. Khoromskij,
[26] B. N. Khoromskij, Tensor numerical methods for multidimensional PDEs: Theoretical analysis and initial applications, CEMRACS 2013—Modelling and Simulation of Complex Systems: Stochastic and Deterministic Approaches, ESAIM Proc. Surveys 48, EDP Science, Les Ulis (2015), 1–28. 10.1051/proc/201448001Search in Google Scholar
[27] O. Koch and C. Lubich, Dynamical tensor approximation, SIAM J. Matrix Anal. Appl. 31 (2010), no. 5, 2360–2375. 10.1137/09076578XSearch in Google Scholar
[28] T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev. 51 (2009), no. 3, 455–500. 10.1137/07070111XSearch in Google Scholar
[29] C. Lubich, I. V. Oseledets and B. Vandereycken, Time integration of tensor trains, SIAM J. Numer. Anal. 53 (2015), no. 2, 917–941. 10.1137/140976546Search in Google Scholar
[30] J. L. Lumley, The structure of inhomogeneous turbulent flows, Atmospheric Turbulence and Radio Wave Propagation, Nauka, Moscow (1967), 166–178. Search in Google Scholar
[31] C. Moler and C. Van Loan, Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later, SIAM Rev. 45 (2003), no. 1, 3–49. 10.1137/S00361445024180Search in Google Scholar
[32] B. Munsky and M. Khammash, A multiple time interval finite state projection algorithm for the solution to the chemical master equation, J. Comput. Phys. 226 (2007), no. 1, 818–835. 10.1016/j.jcp.2007.05.016Search in Google Scholar
[33] H. Niederreiter, Quasi-Monte Carlo methods and pseudo-random numbers, Bull. Amer. Math. Soc. 84 (1978), no. 6, 957–1041. 10.1090/S0002-9904-1978-14532-7Search in Google Scholar
[34] A. Nouy, A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations, Comput. Methods Appl. Mech. Engrg. 199 (2010), no. 23–24, 1603–1626. 10.1016/j.cma.2010.01.009Search in Google Scholar
[35] I. V. Oseledets, Tensor-train decomposition, SIAM J. Sci. Comput. 33 (2011), no. 5, 2295–2317. 10.1137/090752286Search in Google Scholar
[36] I. V. Oseledets and S. V. Dolgov, Solution of linear systems and matrix inversion in the TT-format, SIAM J. Sci. Comput. 34 (2012), no. 5, A2718–A2739. 10.1137/110833142Search in Google Scholar
[37] T. Rohwedder and A. Uschmajew, On local convergence of alternating schemes for optimization of convex problems in the tensor train format, SIAM J. Numer. Anal. 51 (2013), no. 2, 1134–1162. 10.1137/110857520Search in Google Scholar
[38] D. V. Savostyanov, S. V. Dolgov, J. M. Werner and I. Kuprov, Exact NMR simulation of protein-size spin systems using tensor train formalism, Phys. Rev. B 90 (2014), Article ID 085139. 10.1103/PhysRevB.90.085139Search in Google Scholar
[39] U. Schollwöck, The density-matrix renormalization group in the age of matrix product states, Ann. Physics 326 (2011), no. 1, 96–192. 10.1016/j.aop.2010.09.012Search in Google Scholar
[40] D. Schötzau, hp-DGFEM for parabolic evolution problems. Applications to diffusion and viscous incompressible fluid flow, PhD thesis, ETH, Zürich, 1999. 10.1007/s100920050004Search in Google Scholar
[41] L. Sirovich, Turbulence and the dynamics of coherent structures. I. Coherent structures, Quart. Appl. Math. 45 (1987), no. 3, 561–571. 10.1090/qam/910462Search in Google Scholar
[42] C. G. Small and J. Wang, Numerical Methods for Nonlinear Estimating Equations, Oxford Statist. Sci. Ser. 29, The Clarendon Press, Oxford, 2003. 10.1093/acprof:oso/9780198506881.001.0001Search in Google Scholar
[43] S. A. Smoljak, Quadrature and interpolation formulae on tensor products of certain function classes, Dokl. Akad. Nauk SSSR 148 (1963), 1042–1045. Search in Google Scholar
[44] E. Tadmor, The exponential accuracy of Fourier and Chebyshev differencing methods, SIAM J. Numer. Anal. 23 (1986), no. 1, 1–10. 10.1137/0723001Search in Google Scholar
[45] L. N. Trefethen, Spectral Methods in MATLAB, Software Environ. Tools 10, Society for Industrial and Applied Mathematics, Philadelphia, 2000. 10.1137/1.9780898719598Search in Google Scholar
[46] G. Vidal, Efficient simulation of one-dimensional quantum many-body systems, Phys. Rev. Lett. 93 (2004), Article ID 040502. 10.1103/PhysRevLett.93.040502Search in Google Scholar PubMed
[47] T. von Petersdorff and C. Schwab, Numerical solution of parabolic equations in high dimensions, M2AN Math. Model. Numer. Anal. 38 (2004), no. 1, 93–127. 10.1051/m2an:2004005Search in Google Scholar
[48] S. R. White, Density-matrix algorithms for quantum renormalization groups, Phys. Rev. B 48 (1993), no. 14, 10345–10356. 10.1103/PhysRevB.48.10345Search in Google Scholar PubMed
[49] S. R. White and A. E. Feiguin, Real-time evolution using the density matrix renormalization group, Phys. Rev. Lett. 93 (2004), Article ID 076401. 10.1103/PhysRevLett.93.076401Search in Google Scholar PubMed
© 2018 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Tensor Numerical Methods: Actual Theory and Recent Applications
- A Low-Rank Inexact Newton–Krylov Method for Stochastic Eigenvalue Problems
- A Tensor Decomposition Algorithm for Large ODEs with Conservation Laws
- Non-intrusive Tensor Reconstruction for High-Dimensional Random PDEs
- Quasi-Optimal Rank-Structured Approximation to Multidimensional Parabolic Problems by Cayley Transform and Chebyshev Interpolation
- Projection Methods for Dynamical Low-Rank Approximation of High-Dimensional Problems
- Tensor Train Spectral Method for Learning of Hidden Markov Models (HMM)
- Tucker Tensor Analysis of Matérn Functions in Spatial Statistics
- Low-Rank Space-Time Decoupled Isogeometric Analysis for Parabolic Problems with Varying Coefficients
- Approximate Solution of Linear Systems with Laplace-like Operators via Cross Approximation in the Frequency Domain
- Rayleigh Quotient Methods for Estimating Common Roots of Noisy Univariate Polynomials
Articles in the same Issue
- Frontmatter
- Tensor Numerical Methods: Actual Theory and Recent Applications
- A Low-Rank Inexact Newton–Krylov Method for Stochastic Eigenvalue Problems
- A Tensor Decomposition Algorithm for Large ODEs with Conservation Laws
- Non-intrusive Tensor Reconstruction for High-Dimensional Random PDEs
- Quasi-Optimal Rank-Structured Approximation to Multidimensional Parabolic Problems by Cayley Transform and Chebyshev Interpolation
- Projection Methods for Dynamical Low-Rank Approximation of High-Dimensional Problems
- Tensor Train Spectral Method for Learning of Hidden Markov Models (HMM)
- Tucker Tensor Analysis of Matérn Functions in Spatial Statistics
- Low-Rank Space-Time Decoupled Isogeometric Analysis for Parabolic Problems with Varying Coefficients
- Approximate Solution of Linear Systems with Laplace-like Operators via Cross Approximation in the Frequency Domain
- Rayleigh Quotient Methods for Estimating Common Roots of Noisy Univariate Polynomials
