Kernel Embedding Based Variational Approach for Low-Dimensional Approximation of Dynamical Systems

References

[1] Z. Bai, D. Lu and B. Vandereycken, Robust Rayleigh quotient minimization and nonlinear eigenvalue problems, SIAM J. Sci. Comput. 40 (2018), no. 5, A3495–A3522. 10.1137/18M1167681Search in Google Scholar

[2] S. L. Brunton, B. W. Brunton, J. L. Proctor and J. N. Kutz, Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control, PloS one 11 (2016), no. 2, Article ID e0150171. 10.1371/journal.pone.0150171Search in Google Scholar PubMed PubMed Central

[3] K. K. Chen, J. H. Tu and C. W. Rowley, Variants of dynamic mode decomposition: Boundary condition, Koopman, and Fourier analyses, J. Nonlinear Sci. 22 (2012), no. 6, 887–915. 10.1007/s00332-012-9130-9Search in Google Scholar

[4] W. Chen, H. Sidky and A. L. Ferguson, Nonlinear discovery of slow molecular modes using state-free reversible vampnets, J. Chem. Phys. 150 (2019), no. 21, Article ID 214114. 10.1063/1.5092521Search in Google Scholar PubMed

[5] J. D. Chodera and F. Noé, Markov state models of biomolecular conformational dynamics, Curr. Opin. Struct. Biol. 25 (2014), 135–144. 10.1016/j.sbi.2014.04.002Search in Google Scholar PubMed PubMed Central

[6] R. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal. 21 (2006), no. 1, 5–30. 10.1016/j.acha.2006.04.006Search in Google Scholar

[7] N. D. Conrad, M. Weber and C. Schütte, Finding dominant structures of nonreversible Markov processes, Multiscale Model. Simul. 14 (2016), no. 4, 1319–1340. 10.1137/15M1032272Search in Google Scholar

[8] P. Deuflhard and M. Weber, Robust Perron cluster analysis in conformation dynamics, Linear Algebra Appl. 398 (2005), 161–184. 10.1016/j.laa.2004.10.026Search in Google Scholar

[9] P. Drineas and M. W. Mahoney, On the Nyström method for approximating a Gram matrix for improved kernel-based learning, J. Mach. Learn. Res. 6 (2005), 2153–2175. 10.1007/11503415_22Search in Google Scholar

[10] K. Fackeldey, P. Koltai, P. Névir, H. Rust, A. Schild and M. Weber, From metastable to coherent sets—time-discretization schemes, Chaos 29 (2019), no. 1, Article ID 012101. 10.1063/1.5058128Search in Google Scholar PubMed

[11] K. Fukumizu, L. Song and A. Gretton, Kernel Bayes’ rule: Bayesian inference with positive definite kernels, J. Mach. Learn. Res. 14 (2013), 3753–3783. Search in Google Scholar

[12] D. Goswami, E. Thackray and D. A. Paley, Constrained Ulam dynamic mode decomposition: Approximation of the Perron–Frobenius operator for deterministic and stochastic systems, IEEE Control Syst. Lett. 2 (2018), no. 4, 809–814. 10.1109/LCSYS.2018.2849552Search in Google Scholar

[13] J. Han, L. Zhang and W. E, Solving many-electron Schrödinger equation using deep neural networks, J. Comput. Phys. 399 (2019), Article ID 108929. 10.1016/j.jcp.2019.108929Search in Google Scholar

[14] Y. Hashimoto, I. Ishikawa, M. Ikeda, F. Komura and Y. Kawahara, Kernel mean embeddings of von Neumann-algebra-valued measures, preprint (2020), https://arxiv.org/abs/2007.14698. Search in Google Scholar

[15] Y. Hashimoto, I. Ishikawa, M. Ikeda, Y. Matsuo and Y. Kawahara, Krylov subspace method for nonlinear dynamical systems with random noise, J. Mach. Learn. Res. 21 (2020), Paper No. 172. Search in Google Scholar

[16] T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning. Vol. 1, Springer Ser. Statist., Springer, New York, 2001. 10.1007/978-0-387-21606-5Search in Google Scholar

[17] J. Hermann, Z. Schätzle and F. Noé, Deep-neural-network solution of the electronic Schrödinger equation, Nature Chem. 12 (2020), no. 10, 891–897. 10.1038/s41557-020-0544-ySearch in Google Scholar PubMed

[18] I. T. Jolliffe and J. Cadima, Principal component analysis: A review and recent developments, Philos. Trans. Roy. Soc. A 374 (2016), no. 2065, 10.1098/rsta.2015.0202. 10.1098/rsta.2015.0202Search in Google Scholar PubMed PubMed Central

[19] V. R. Joseph and L. Kang, Regression-based inverse distance weighting with applications to computer experiments, Technometrics 53 (2011), no. 3, 254–265. 10.1198/TECH.2011.09154Search in Google Scholar

[20] O. Junge and P. Koltai, Discretization of the Frobenius–Perron operator using a sparse Haar tensor basis: The sparse Ulam method, SIAM J. Numer. Anal. 47 (2009), no. 5, 3464–3485. 10.1137/080716864Search in Google Scholar

[21] Y. Kawahara, Dynamic mode decomposition with reproducing kernels for koopman spectral analysis, Advances in Neural Information Processing Systems, ACM, New York (2016), 911–919. Search in Google Scholar

[22] S. Klus, P. Koltai and C. Schütte, On the numerical approximation of the Perron–Frobenius and Koopman operator, J. Comput. Dyn. 3 (2016), no. 1, 51–79. 10.3934/jcd.2016003Search in Google Scholar

[23] S. Klus, F. Nüske and B. Hamzi, Kernel-based approximation of the Koopman generator and Schrödinger operator, Entropy 22 (2020), no. 7, Paper No. 722. 10.3390/e22070722Search in Google Scholar PubMed PubMed Central

[24] S. Klus, F. Nüske, P. Koltai, H. Wu, I. Kevrekidis, C. Schütte and F. Noé, Data-driven model reduction and transfer operator approximation, J. Nonlinear Sci. 28 (2018), no. 3, 985–1010. 10.1007/s00332-017-9437-7Search in Google Scholar

[25] S. Klus, F. Nüske, S. Peitz, J.-H. Niemann, C. Clementi and C. Schütte, Data-driven approximation of the Koopman generator: Model reduction, system identification, and control, Phys. D 406 (2020), Article ID 132416. 10.1016/j.physd.2020.132416Search in Google Scholar

[26] S. Klus, I. Schuster and K. Muandet, Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces, J. Nonlinear Sci. 30 (2020), no. 1, 283–315. 10.1007/s00332-019-09574-zSearch in Google Scholar

[27] A. Konrad, B. Y. Zhao, A. D. Joseph and R. Ludwig, A Markov-based channel model algorithm for wireless networks, Wireless Netw. 9 (2003), no. 3, 189–199. 10.1145/381591.381602Search in Google Scholar

[28] M. Korda and I. Mezić, On convergence of extended dynamic mode decomposition to the Koopman operator, J. Nonlinear Sci. 28 (2018), no. 2, 687–710. 10.1007/s00332-017-9423-0Search in Google Scholar

[29] J. N. Kutz, S. L. Brunton, B. W. Brunton and J. L. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 2016. 10.1137/1.9781611974508Search in Google Scholar

[30] S. Lafon, Y. Keller and R. R. Coifman, Data fusion and multicue data matching by diffusion maps, IEEE Trans. Pattern Anal. Mach. Intell. 28 (2006), no. 11, 1784–1797. 10.1109/TPAMI.2006.223Search in Google Scholar PubMed

[31] P. L. Lai and C. Fyfe, Kernel and nonlinear canonical correlation analysis, Int. J. Neural Syst. 10 (2000), no. 05, 365–377. 10.1142/S012906570000034XSearch in Google Scholar PubMed

[32] Q. Li, F. Dietrich, E. M. Bollt and I. G. Kevrekidis, Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator, Chaos 27 (2017), no. 10, Article ID 103111. 10.1063/1.4993854Search in Google Scholar PubMed

[33] O. Lindenbaum, M. Salhov, A. Yeredor and A. Averbuch, Gaussian bandwidth selection for manifold learning and classification, Data Min. Knowl. Discov. 34 (2020), no. 6, 1676–1712. 10.1007/s10618-020-00692-xSearch in Google Scholar PubMed PubMed Central

[34] O. Lindenbaum, A. Yeredor and A. Averbuch, Bandwidth selection for kernel-based classification, 2016 IEEE International Conference on the Science of Electrical Engineering (ICSEE), IEEE Press, Piscataway (2016), 1–5. 10.1109/ICSEE.2016.7806089Search in Google Scholar

[35] B. Lusch, J. N. Kutz and S. L. Brunton, Deep learning for universal linear embeddings of nonlinear dynamics, Nature Commun. 9 (2018), no. 1, 1–10. 10.1038/s41467-018-07210-0Search in Google Scholar PubMed PubMed Central

[36] A. Mardt, L. Pasquali, F. Noé and H. Wu, Deep learning Markov and Koopman models with physical constraints, Proc. Mach. Learn. Res. (PMLR) 107 (2020), 451–475. Search in Google Scholar

[37] A. Mardt, L. Pasquali, H. Wu and F. Noé, Vampnets for deep learning of molecular kinetics, Nature Commun. 9 (2018), no. 1, 1–11. 10.1038/s41467-017-02388-1Search in Google Scholar PubMed PubMed Central

[38] R. T. McGibbon and V. S. Pande, Variational cross-validation of slow dynamical modes in molecular kinetics, J. Chem. Phys. 142 (2015), no. 12, Article ID 124105. 10.1063/1.4916292Search in Google Scholar PubMed PubMed Central

[39] I. Mezić, Analysis of fluid flows via spectral properties of the Koopman operator, Annu. Rev. Fluid Mech. 45 (2013), 357–378. 10.1146/annurev-fluid-011212-140652Search in Google Scholar

[40] K. Mills, M. Spanner and I. Tamblyn, Deep learning and the Schrödinger equation, Phys. Rev. A 96 (2017), no. 4, Article ID 042113. 10.1103/PhysRevA.96.042113Search in Google Scholar

[41] H. Q. Minh, P. Niyogi and Y. Yao, Mercer’s theorem, feature maps, and smoothing, Learning Theory, Lecture Notes in Comput. Sci. 4005, Springer, Berlin (2006), 154–168. 10.1007/11776420_14Search in Google Scholar

[42] L. Molgedey and H. G. Schuster, Separation of a mixture of independent signals using time delayed correlations, Phys. Rev. Lett. 72 (1994), 3634–3637. 10.1103/PhysRevLett.72.3634Search in Google Scholar PubMed

[43] J. R. Moreno, G. Carleo and A. Georges, Deep learning the Hohenberg-Kohn maps of density functional theory, Phys. Rev. Lett. 125 (2020), no. 7, Article ID 076402. 10.1103/PhysRevLett.125.076402Search in Google Scholar PubMed

[44] R. Nagai, R. Akashi and O. Sugino, Completing density functional theory by machine learning hidden messages from molecules, npj Comput. Mat. 6 (2020), no. 1, 1–8. 10.1038/s41524-020-0310-0Search in Google Scholar

[45] J. Nelson, R. Tiwari and S. Sanvito, Machine learning density functional theory for the Hubbard model, Phys. Rev. B 99 (2019), no. 7, Article ID 075132. 10.1103/PhysRevB.99.075132Search in Google Scholar

[46] F. Noé and F. Nüske, A variational approach to modeling slow processes in stochastic dynamical systems, Multiscale Model. Simul. 11 (2013), no. 2, 635–655. 10.1137/110858616Search in Google Scholar

[47] F. Nuske, B. G. Keller, G. Perezhernandez, A. S. J. S. Mey and F. Noe, Variational approach to molecular kinetics, J. Chem. Theory Comput. 10 (2014), no. 4, 1739–1752. 10.1021/ct4009156Search in Google Scholar PubMed

[48] F. Núske, R. Schneider, F. Vitalini and F. Noé, Variational tensor approach for approximating the rare event kinetics of macromolecular systems, J. Chem. Phys. 144 (2016), no. 5, 149–153. 10.1063/1.4940774Search in Google Scholar PubMed

[49] G. Perezhernandez, F. Paul, T. Giorgino, G. De Fabritiis and F. Noe, Identification of slow molecular order parameters for Markov model construction, J. Chem. Phys. 139 (2013), Article ID 015102. 10.1063/1.4811489Search in Google Scholar PubMed

[50] W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery, Numerical Recipes: The Art of Scientific Computing, Cambridge University, Cambridge, 2007. Search in Google Scholar

[51] J.-H. Prinz, H. Wu, M. Sarich, B. Keller, M. Senne, M. Held, J. D. Chodera, C. Schütte and F. Noé, Markov models of molecular kinetics: Generation and validation, J. Chem. Phys. 134 (2011), no. 17, Article ID 174105. 10.1063/1.3565032Search in Google Scholar PubMed

[52] A. Rahimi and B. Recht, Random features for large-scale kernel machines, Advances in Neural Information Processing Systems, ACM, New York (2008), 1177–1184. Search in Google Scholar

[53] S. Röblitz and M. Weber, Fuzzy spectral clustering by PCCA+: Application to Markov state models and data classification, Adv. Data Anal. Classif. 7 (2013), no. 2, 147–179. 10.1007/s11634-013-0134-6Search in Google Scholar

[54] P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech. 656 (2010), 5–28. 10.1017/S0022112010001217Search in Google Scholar

[55] C. Schütte, A. Fischer, W. Huisinga and P. Deuflhard, A direct approach to conformational dynamics based on hybrid Monte Carlo, J. Comput. Phys. 151 (1999), 146–168. 10.1006/jcph.1999.6231Search in Google Scholar

[56] C. R. Schwantes and V. S. Pande, Improvements in Markov state model construction reveal many non native interactions in the folding of ntl9, J. Chem. Theory Comput. 9 (2013), no. 4, 2000–2009. 10.1021/ct300878aSearch in Google Scholar PubMed PubMed Central

[57] C. R. Schwantes and V. S. Pande, Modeling molecular kinetics with tica and the kernel trick, J. Chem. Theory Comput. 11 (2015), no. 2, 600–608. 10.1021/ct5007357Search in Google Scholar PubMed PubMed Central

[58] A. S. Sharma, I. Mezi and B. J. Mckeon, Correspondence between koopman mode decomposition, resolvent mode decomposition, and invariant solutions of the Navier–Stokes equations, Phys. Rev. Fluids 1 (2016), Article ID 03240. 10.1103/PhysRevFluids.1.032402Search in Google Scholar

[59] A. Singer, R. Erban, I. G. Kevrekidis and R. R. Coifman, Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps, Proc. Natl. Acad. Sci. 106 (2009), no. 38, 16090–16095. 10.1073/pnas.0905547106Search in Google Scholar PubMed PubMed Central

[60] A. Smola, A. Gretton, L. Song and B. Schölkopf, A Hilbert space embedding for distributions, International Conference on Algorithmic Learning Theory, Springer, Berlin (2007), 13–31. 10.1007/978-3-540-75225-7_5Search in Google Scholar

[61] L. Song, K. Fukumizu and A. Gretton, Kernel embeddings of conditional distributions: A unified kernel framework for nonparametric inference in graphical models, IEEE Signal Process. Mag. 30 (2013), no. 4, 98–111. 10.1109/MSP.2013.2252713Search in Google Scholar

[62] L. Song, J. Huang, A. Smola and K. Fukumizu, Hilbert space embeddings of conditional distributions with applications to dynamical systems, Proceedings of the 26th Annual International Conference on Machine Learning, ACM, New York (2009), 961–968. 10.1145/1553374.1553497Search in Google Scholar

[63] B. K. Sriperumbudur, K. Fukumizu and G. R. G. Lanckriet, Universality, characteristic kernels and RKHS embedding of measures, J. Mach. Learn. Res. 12 (2011), 2389–2410. Search in Google Scholar

[64] A. Szabo and N. S. Ostlund, Modern Quantum Chemistry: Introduction to Advanced Electronic Structure Theory, Courier Corporation, North Chelmsford, 2012. Search in Google Scholar

[65] J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton and J. N. Kutz, On dynamic mode decomposition: theory and applications, J. Comput. Dyn. 1 (2014), no. 2, 391–421. 10.3934/jcd.2014.1.391Search in Google Scholar

[66] M. O. Williams, I. G. Kevrekidis and C. W. Rowley, A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci. 25 (2015), no. 6, 1307–1346. 10.1007/s00332-015-9258-5Search in Google Scholar

[67] M. O. Williams, C. W. Rowley and I. G. Kevrekidis, A kernel-based method for data-driven Koopman spectral analysis, J. Comput. Dyn. 2 (2015), no. 2, 247–265. 10.3934/jcd.2015005Search in Google Scholar

[68] H. Wu, A. Mardt, L. Pasquali and F. Noe, Deep generative Markov state models, Advances in Neural Information Processing Systems, ACM. New York (2018), 3975–3984. Search in Google Scholar

[69] H. Wu and F. Noé, Gaussian Markov transition models of molecular kinetics, J. Chem. Phys. 142 (2015), no. 8, Article ID 084104. 10.1063/1.4913214Search in Google Scholar PubMed

[70] H. Wu and F. Noé, Variational approach for learning Markov processes from time series data, J. Nonlinear Sci. 30 (2020), no. 1, 23–66. 10.1007/s00332-019-09567-ySearch in Google Scholar

[71] H. Wu, F. Núske, F. Paul, S. Klus, P. Koltai and F. Noé, Variational Koopman models: Slow collective variables and molecular kinetics from short off-equilibrium simulations, J. Chem. Phys. 146 (2017), no. 15, Article ID 154104. 10.1063/1.4979344Search in Google Scholar PubMed

[72] H. Wu, F. Paul, C. Wehmeyer and F. Noé, Multiensemble Markov models of molecular thermodynamics and kinetics, Proc. Natl. Acad. Sci. 113 (2016), no. 23, E3221–E3230. 10.1073/pnas.1525092113Search in Google Scholar PubMed PubMed Central

[73] M. Yue, J. J. Han and K. S. Trivedi, Composite performance and availability analysis of wirelesscommunication networks, IEEE Trans. Vehicular Technol. 50 (2001), no. 5, 1216–1223. 10.1109/25.950322Search in Google Scholar