Bayesian methods in integrative structure modeling

2.1 Probabilistic modeling

Let’s first discuss the modeling step. Here, the term model is understood in a very broad sense, and by M we denote all of our modeling assumptions in their entirety. The model parameters of primary interest are the 3D coordinates X of the structural representation of the biomolecular system. Because our major goal is to make inferences about X, the structural parameters are listed explicitly in our probabilistic formulation. However, the model M comprises not only the structural representation X, but all assumptions that are required to model the data.

If D denotes the structural data that we want to use for structure determination, then the modeling process involves formulating a likelihood Pr(D | X, M), which is the probability of measuring data D assuming that the system adopts structural coordinates X. The likelihood connects the observed data D with the structural parameters X via the model M. To establish such a connection might require additional parameters that are not of primary interest, but still necessary to formulate the model. Here, we absorb these nuisance parameters into the model M. Nuisance parameters are very common in integrative modeling and typically needed for a complete description of the data. For example, NMR data require the introduction of calibration factors or alignment tensors. Another important type of nuisance parameter are weighting factors that reflect the amount of noise in the data (Habeck et al. 2006).

Typically the likelihood is constructed as follows. First, we derive a forward model f that predicts idealized data (mock data) that should ideally match the observed data. In practice, a perfect match between observed and mock data is highly unlikely due to experimental noise or shortcomings of the model itself (e.g. approximations in the forward model, limited resolution of the representation, etc.). Second, we choose an error model that accounts for deviations of observed data D and mock data f(X). A common choice is a Gaussian error model, but other choices might be more appropriate such as a lognormal distribution for distance data, or a Poisson distribution for counts.

The second ingredient of a Bayesian model is the prior probability Pr(X | M). The prior expresses what we assume about reasonable structural models independent of the data. Depending on the representation, more or less detailed knowledge about X is available. At atomic resolution, we can use literature values for bond lengths, bond angles and other stereochemical parameters which are encoded in molecular mechanics force fields. At coarser resolution, common terms such as excluded volume or statistical potentials will be incorporated into the prior.

Bayes’ theorem stipulates that the posterior distribution Pr(X | D, M) is proportional to the product of the prior and the likelihood:

The proportionality factor Pr(D | M) is called the model evidence and not needed to compute structural models. However, the model evidence is crucial when we want to compare different modeling assumptions summarized by M (Knuth et al. 2015).

Equation (1) nicely separates the two stages of applying Bayesian inference in practice: Setting up the right hand side involves making choices about the model that describes the data and the structure. These choices need to be made by the modeler and are therefore, in parts, subjective; they should be challenged by alternative models. Bayesian inference demands us to make all of the modeling assumptions explicit and estimate them as well from the data, if we are uncertain about them. The left hand side is the actual inference, which is apparent from the fact that the role of the data D and the structural coordinates X is swapped. Equation (1) is a direct consequence of the rules of probability theory. The second stage of applying Bayesian inference involves computation of the factors on the left hand side, i.e. drawing samples from the posterior distribution and evaluating the model evidence.

In the Bayesian view, the posterior Pr(X | D, M) captures everything that can be said about the structural model X given the experimental data D in the light of all our modeling assumptions M. Assuming that the model is correct (or at least useful) the posterior is the actual result of a structure determination exercise. The resulting posterior probability quantifies to which extent the structural coordinates are determined by all of the information that is incorporated into the model.

Figure 2 schematically shows possible posterior shapes reflecting different states of uncertainty. If the posterior peaks sharply at a single structure, then the experimental data in combination with the background information suffice to determine the structure unambiguously. This typically requires high-resolution data from X-ray, cryo-EM or NMR. Next, the posterior could still exhibit a single dominant peak which is broadened due to the lack of data. To represent the increased uncertainty, we should not only try to locate the maximum of the peak, but also generate structural models that reflect the uncertainty under the peak. These structures are suboptimal solutions which are missed by an optimization approach. In the most general situation which is typical for integrative modeling, the posterior distribution will exhibit multiple peaks. Each peak corresponds to a structure ensemble that explains the data (at least to some degree). The higher the peak, the better is the agreement between observed and mock data. But not only the height of the peak, also the probability mass under the peak is important, because it reflects how common a structure is, which again argues for the use of sampling (rather than optimization) methods for structure modeling.

Figure 2:

The posterior probability Pr(X | D, M) reflects the uncertainty in the structure when taking all data and model assumptions into account.

2.2 Bayesian computation via posterior sampling

Often the posterior is converted into a scoring function by taking its negative logarithm (Rout and Sali 2019). Highly probable structures then correspond to structures with a low scoring function-logPr(D | X,M)-logPr(X | M). The first term is a restraint energy that favors structural models X minimizing the discrepancy between the observed data and mock data computed from X. The second term involving the log prior acts like a regularizer. Structural models with high posterior probability can be computed by minimizing the score.

However, optimization approaches are limited in that they reduce the full posterior distribution to a few points in conformation space. From a truly Bayesian perspective, the solution of a structure determination problem is given by the posterior distribution Pr(X | D,M) itself. Therefore, we should try to capture the shape of the posterior as closely as possible. A single or few structures cannot capture the shape of the posterior distribution unless it collapsed into a single peak. This is very unlikely to happen in integrative modeling where varied data of different quality and amount need to be combined.

An advantage of the Bayesian approach is that all sources of information about the structure of a biomolecular system enter with an appropriate strength. Likelihoods are normalized with respect to the data and therefore there is no freedom or need to set weighting factors. The width of the likelihood (which determines the weight of the log-likelihood) is determined by the quality of the data and is either known or can be estimated along with the structure parameters (Habeck et al. 2006; Rieping et al. 2005).

To infer a probabilistic model and take full advantage of the Bayesian approach, we can use statistical sampling methods and generate representative model realizations from the posterior distribution. Posterior samples can be used in several ways to assess the reliability of a structural model. The uncertainty of model parameters is readily available in the form of posterior histograms. Uncertain parameters are represented by broad histograms; ambiguous model parameters are indicated by multimodal posterior histograms.

In addition to the posterior probability, the model evidence, also known as marginal likelihood, is an important result of a Bayesian analysis. It is desirable to provide an estimate of the model evidence in addition to posterior samples, because different modeling approaches can be compared in an objective way via the estimated model evidences. Alternative models could differ in the choice of representation of a structure (e.g. its resolution), the forward model that connects structural data D with the structural representation, the error model or prior information.

3.1 Structural modeling with cryo-EM maps

Cryo-electron microscopy (cryo-EM) has emerged as one of the most powerful experimental methods for structure determination of large macromolecular complexes. Structural models are obtained from cryo-EM maps mainly by solving one of two tasks: rigidbody docking or flexible fitting.

A common approach to obtain a structural model of a multi-component complex is to first place the subunit structures rigidly into the EM map and then refine these structures flexibly. However, this approach requires that high-resolution structures or homology models of the subunits are available, and that the subunits do not undergo a drastic conformational change during the assembly of the complex. A growing number of computational tools for structural modeling with cryo-EM maps has been developed (Villa and Lasker 2014). These are typically non-Bayesian approaches that maximize the cross-correlation coefficient.

Here, the discussion is restricted to Bayesian approaches for structural modeling with cryo-EM maps. Bonomi et al. (2019) proposed a method for rigid multi-body docking that combines Bayesian inference with coarse-grained Gaussian mixture model (GMM) representations of the experimental map and subunit structures. They use Monte Carlo sampling of the rigid transformations within the IMP software. The approach is benchmarked on 21 protein/DNA complexes consisting of 2–7 subunits at intermediate resolution of 10 Å. On 16 examples, the approach achieves a good RMSD of 2.2 Å on average. However, there are a few instances where the correct assembly is missed either partially or completely. The reason is most likely inefficient sampling of rigid body poses in multi-component complexes. A general Bayesian framework for rigid and flexible fitting of protein structures into cryo-EM maps has been introduced by Habeck (2017) and implemented in the ISD software (also see Introduction). Both atomic and coarse-grained representations of molecular systems are supported. In flexible fitting, the structural parameters are updated with Hamiltonian Monte Carlo. On the Flex-EM benchmark (Topf et al. 2008), ISD produces fits that are systematically better than Flex-EM.

With the improvement of EM reconstructions towards atomic resolution (Yip et al. 2020), full-atom modeling is the final bottleneck to also obtain an atomic model in addition to the density map. This is the domain of flexible fitting of atomic structures into EM maps. Blau et al. (2016) have combined molecular dynamics (MD) with a cryo-EM fitting score that they derive in a Bayesian fashion. Vuillemot and Jonić (2021) combine normal-mode based fitting with cryo-EM maps within a Bayesian framework. A more recent development is CryoFold (Shekhar et al. 2021) which combines the Bayesian MELD approach (MacCallum et al. 2015) with other modeling tools. CryoFold generates an ensemble of high-quality structural models without requiring high-resolution subunit structures as input. However, the density map needs to be of a sufficiently high resolution of about 3–5 Å.

3.2 Coarse-graining of biomolecular systems

Choosing a good representation for structural modeling is an essential part of integrative approaches. Coarse-graining (CG) lumps groups of atoms together into structural building blocks, typically spherical beads, while aiming to preserve the structural and energetic properties as best as possible. Representing biomolecules with CG models seems appropriate due to their hierarchical architecture. The goal is to bridge between multiple spatial and temporal scales. Various CG representations of proteins, lipids and other biomolecules have been developed to enable MD simulations of large systems that are not suitable for all-atom simulations (Jin et al. 2022; Noid 2013; Pak and Voth 2018).

In the context of integrative modeling, CG representations are used to cope with the lack of data, resolution and the large size of the systems. Statistical potentials and potentials of mean force have a long tradition in protein modeling and structure prediction. Here, we focus on recent Bayesian approaches to coarse-graining. Bottom-up approaches typically run atomistic simulations from which CG models are learned. Farrell et al. (2015) have developed a general Bayesian framework to build, calibrate and validate CG models of atomistic systems. Model selection is applied to a family of CG models. Predictive coarse-graining is a Bayesian approach for learning CG models from simulations based on a combination of MAP and Monte Carlo simulation (Schöberl et al. 2017). BICePs focuses on the generation of conformational ensembles (Voelz et al. 2021). Posterior sampling is coupled to model selection techniques to optimize different types of hyperparameters. The framework can be used to optimize reference potentials or CG representations (Ge and Voelz 2018). Ultra-coarse-graining is more relevant to modeling of large biomolecular systems. Here many atoms are grouped together to form a bead (Jin and Voth 2018).

The CG approaches mentioned so far try to map the molecular representation so as to preserve the energetic parameters as best as possible. More pragmatic approaches focus on low-resolution representation of molecular structures. In cryo-EM modeling, Kawabata (2008) proposed the use of Gaussian mixture models (GMMs) to represent density maps as well as atomic structures. The GMM representation has been adopted by a several modeling approaches, most prominently by IMP (see work by Bonomi et al. (2019) and the discussion in Section 3.1 for an application). CG representations can be learned both from static atomic biomolecular structures and cryo-EM reconstructions using a Bayesian mixture model approach (Chen and Habeck 2017). This approach estimates an optimal mapping between atoms and beads, the bead positions as well as the non-bonded radius of the particles and their interaction strengths.

Another application is to infer CG models from projection images obtained with cryoEM (Joubert and Habeck 2015; Vakili and Habeck 2021). A highly coarse-grained representation of the 3D structure is generated from 2D projection images via posterior sampling. The unknown structure is represented by hundreds to thousands of beads rather than a density map over a voxel grid. The particles are arranged such that the 2D projection images are reproduced as best as possible. Along with the particle positions the posterior sampling algorithm estimates the unknown projection directions and can therefore be used for initial structure determination in single-particle analysis. The approach samples reconstructions that are of a quality and resolution similar to reconstructions generated with other approaches for initial model generation such as PRIME (Elmlund et al. 2013). However, it remains unclear if the particle-based reconstruction approach can be scaled to a larger number of particles (>10,000 particles) and thereby yield initial reconstructions at higher resolutions.

3.3 Structural modeling of chromosomes and genomes

An emerging field is structural modeling of entire chromosomes or genomes. This development is powered by high-throughput chromosome conformation capture (Hi–C) techniques. Single-cell Hi–C (Nagano et al. 2013) probes through-space contacts between different chromosomal regions and can be used for structural modeling in a fashion similar to NMR distance bounds. Due to limitations in resolution, but also the sheer size of chromosomes only highly coarse-grained representations of chromosomes are amenable to structural modeling.

Among the multitude of chromosome structure modeling software, we only mention Bayesian approaches (Carstens et al. 2016; Hu et al. 2013; Meng et al. 2021; Rosenthal et al. 2019; Wang et al. 2015; Xie et al. 2022). Typically, the chromosome fiber is represented with a highly coarse-grained polymer model, a “beads on a string” model, a chain of beads that represent 50–500 kilobases of chromatin. Carstens et al. (2016) applies the ISD framework to sparse contacts from single-cell Hi-C. Chromosome structures are generated from the posterior distribution with Monte Carlo methods. The structural models computed with ISD are consistent with independent measurements from fluorescence microscopy. More recently, Meng et al. (2021) have developed Si–C, a Bayesian approach for genome structure modeling from single-cell Hi–C data. Si–C works with beads at 10 kb–100 kb resolution and can utilize whole-genome data measured on single cells (Stevens et al. 2017). The data were measured on haploid cells and allow the reconstruction of the structure of an entire genome. The models are better resolved than those originally obtained by Stevens et al. (2017). Because with higher resolution the data become increasingly sparse, it is unclear which resolution (i.e. number of beads) still yields well-defined models. Using an RMSD-based criterion, the authors find that a resolution with at least 0.2 contacts per bead on average is required to generate well-defined ensembles. The calculated “super-resolution” structures are validated against independent imaging data and yield unprecedented details on the architecture of entire genomes. Although the resolution of these models is still orders of magnitude below base-pair resolution, the models are useful to study the global architecture of genome organization such as the formation of topologically associated domains (TADs) or A/B compartments.

3.4 Inference of conformational heterogeneity

Biomolecular systems are highly dynamic and undergo conformational changes. Therefore, representing the structure of a biomolecular system as a single conformational state is limited. Some data might even require modeling with a conformational ensemble composed of multiple states. For example, methods such as NMR or bulk Hi-C observe ensemble averages rather than single-state data. Multiple states are observed with XLMS or cryo-EM.

There has been an abundance of research activity aimed at developing Bayesian and maximum entropy methods for the inference of ensembles representing diverse conformational states (Bonomi et al. 2017). Typical input data are ensemble-averaged NMR or solution-scattering data. Several groups have combined the principle of maximum entropy with a Bayesian framework (Beauchamp et al. 2014; Bonomi et al. 2016; Bottaro et al. 2020; Hummer and Köfinger 2015). BioEn uses reweighting techniques in combination within a Bayesian framework and an entropic prior (Hummer and Köfinger 2015). Metainference infers a multi-state model (Bonomi et al. 2016). BME is another combination of Bayesian methods and maximum entropy (Bottaro et al. 2020) and has recently been applied to RNAs (Zhang and Frank 2021).

Various Bayesian approaches specialize on generating protein ensembles from small-angle scattering (SAS) data (Antonov et al. 2016; Pesce and Lindorff-Larsen 2021; Potrzebowski et al. 2018; Spill et al. 2021). Model selection techniques have been used to determine the size of the ensemble (Bowerman et al. 2019; Potrzebowski et al. 2018). Ensembles with full-atom detail are generated using the Bayesian approach of Shevchuk and Hub (2017). A method to infer ensembles of disordered protein states from NMR, FRET and SAXS data has been developed by Lincoff et al. (2020).

The principle of maximum entropy has also been applied to chromosome structure modeling (Lin et al. 2021). An application of Bayesian techniques to ensemble modeling based on bulk Hi–C data chromosome modeling has been developed. To choose the number of states, Carstens et al. (2020) have used the model evidence, which is estimated with an MCMC approach.

4.1 Modeling challenges

Molecular representation The choice of molecular representation has a strong impact on the usefulness and success of an integrative modeling exercise. Depending on the resolution and amount of data, a more or less detailed representation is adequate. A multi-scale representation might be required when data of different quality and resolution are mixed. In particular, as we approach cellular-scale systems, there is a large degree of arbitrariness in the choice of representation. Even if high-resolution structures of the subunits of a macromolecular complex are available, it is not generally clear what the optimal model representation is. The degree of coarse-graining should be driven by the resolution and the amount of data. Likewise, the number of states should also be sampled in ensemble modeling. To address some of these challenges, Viswanath and Sali (2019) have proposed a method to optimize the molecular representation for integrative modeling.

Bridging between the molecular and cellular scale A formidable challenge is to integrate data showing the system at largely different scales. This requires the development of multi-scale representations that describe the same molecular component at different resolutions such that they can be linked to the different types of input data. A multi-scale model needs to connect the different scales and their associated conformational degrees of freedom. Different parts of a structural model might be better resolved than others. A multi-scale model should represent a hierarchy of spatial scales and automatically adapt its maximum resolution so as to avoid overfitting. The model should also quantify non-specific interactions between biomolecules. Moreover, since the structures should be embedded into the cellular context, also structural representations of the cellular environment will become necessary (Im et al. 2016). New representations as the one proposed by Singla et al. (2022) might be useful to bridge between molecular and cellular scales. This representation builds biological structures from tetrahedra and is commonly used in numerical analysis to represent complicated three-dimensional shapes. The representation might be useful for modeling assemblies in crowded cellular environments with many diverse components.

Probabilistic models for structural data A diverse array of structural data is already used to build integrative structures (Rout and Sali 2019) and will become even more heterogeneous in the future. Each method requires a probabilistic model that links the data to the structural representation of the system. The probabilistic model should be realistic in the sense that the main characteristics of the data and the noise are captured adequately. At the same time, the model should be computationally efficient. For example, SAXS curves can be computed with high accuracy and detail, but evaluating these models is computationally expensive. Efficient models for computing approximate EM maps exist, but the noise models are typically very limited in that they ignore correlations between neighboring voxels.

4.2 Computational challenges

Software Bayesian integrative modeling is enabled by software packages such as the Integrative Modeling Platform (IMP) (Russel et al. 2012; Saltzberg et al. 2021), BioEn (Hummer and Köfinger 2015) or PLUMED-ISDB (Bonomi and Camilloni 2017). In the future, it will be important to make Bayesian integrative modeling available to non-experts. Given the diversity of the data and the systems there are many challenges that a software has to meet. The software should be easy to install, run and used with diverse structural data. A related issue is to train users of Bayesian integrative modeling software in probabilistic reasoning and modeling.

Efficient conformational sampling Conformational sampling refers to the process of generating representative structures X from the posterior Pr(X | D, M). Since X tends to be high-dimensional, direct sampling is inefficient and not viable. Therefore, one typically resorts to Markov chain Monte Carlo (MCMC) sampling which explores the conformation space by running a Markov chain. Conformational sampling is more or less challenging depending on the quality and amount of data. MCMC algorithms tend to converge very slowly. Faster MCMC algorithms and implementations are required to enable Bayesian integrative modeling of ever larger systems. Also pragmatic approaches such as jump-starting an MCMC simulation from optimized structures should be supported.

Automation of posterior inference MCMC algorithms typically come with a number of parameters that need to be optimized by the user. An example is the temperature schedule of parallel tempering simulations, which is one of the working horses of exhaustive posterior sampling in integrative modeling. Often, a large fraction of computation time is spent on optimizing algorithmic parameters so as to guarantee the convergence of the MCMC simulation. Adaptive MCMC methods that optimize algorithmic parameters in the course of simulation need to be developed to enable Bayesian integrative modeling also for non-experts with limited computational resources.

Validation of posterior sampling MCMC is an approximate method and can fail. A problem with all sampling methods is that they rely on asymptotic guarantees, meaning that they will produce correct samples only in the long run. But there is no sufficient measure that assesses whether a Markov chain has already converged. An MCMC simulation can fail in several ways. The simulation might miss an important peak of the posterior distribution, i.e. a structure that is a good explanation of the data. The simulation might misrepresent the probability mass under a peak, meaning that the frequency with which a structure is visited by the Markov chain might be biased and fail to reflect the true importance of the structure. Viswanath et al. (2017) have developed several techniques to assess the exhaustiveness of conformational sampling. These need to be extended in the future.

4.3 Deposition, validation and analysis of Bayesian integrative structures

Finally, there are a couple of challenges that are related to the deposition, validation and analysis of integrative structures obtained with a Bayesian approach.

Deposition of posterior ensembles In the Bayesian view, the solution of an integrative structure determination is the full posterior distribution Pr(X | D,M) rather than a single structure or a few structures. But how can we adequately represent a posterior distribution with finite means? How many structures suffice to capture the posterior? These questions are even more pressing, if ensembles of structures are inferred that represent diverse conformational states. Thinning approaches that reduce the number of MCMC samples could help us compress the large amount of data and information contained in posterior samples. The Protein Data Bank (PDB) has created an archive for integrative structures (Vallat et al. 2021). PDB-Dev provides timely support for integrative structure modeling, but might require future extensions to represent all relevant aspects of Bayesian integrative modeling. For example, it is desirable to report the complete posterior sampling protocol as well as samples of additional nuisance parameters. Moreover, the modeling software and deposition system should aim to ensure that the structure determination can be reproduced.

Validation of integrative structures Statistical methods, in particular those based on Bayesian inference, enable us to make sound statements about the quality of an integrative structure. New quality measures for the validation of integrative structures need to be developed based on concepts and techniques from Bayesian inference. A prerequisite for statistical model assessment is that we sample conformational space exhaustively. Therefore, an important aspect is to also improve and validate conformational sampling techniques (Viswanath et al. 2017).

Although the number of integrative structures continues to grow, the community still lacks a generally accepted quality measure for structures obtained by integrating data from multiple experimental sources. Estimated data weights (Habeck et al. 2006) should prove useful for the validation of integrative structures. From a Bayesian perspective, the data weights have an intuitive interpretation: They are the inverse squared error of a dataset, so high-quality data with low noise levels will be assigned a high weight.

Other statistical figures of merit should also be reported along with the structure. For example, the entropy of the posterior measures the effective number of structures needed to represent the posterior. Another largely unexplored quantity is the model evidence Pr(D | M). The model evidence is the goodness of fit averaged over all possible conformations X. Since it balances goodness of fit against model complexity, the model evidence embodies Occam’s razor and can be used to select among competing models of explaining the data.

Arroyuelo et al. (2021) have recently proposed various Bayesian tools to assess the quality of an NMR ensemble. These tools report per residue quality measures that allow the authors to identify problematic residues. The approach requires atomic structures and NMR chemical shifts. But some of the strategies might be transferable to integrative structures. We envision that based on PDB-Dev similar reference distributions can be derived for coarse-grained representations and other types of data.

Analysis of posterior samples A Bayesian integrative structure is encoded in the marginal posterior distribution of the structural parameters. This distribution needs to be approximated with efficient clustering methods that analyze the large amount of structural samples. Based on a cluster analysis we can then report the number of clusters, theirs weights and precisions to obtain insights into the shape and quality of the posterior ensemble. High-quality data will result in a single, narrow cluster, whereas ambiguous and sparse data result in several, broad clusters. This information should feed back into the validation of integrative structures. To validate structural models at atomic resolution it is also important to compare the models with related structures in the PDB. Bayesian methods for structural alignment of proteins have been developed by Rodriguez and Schmidler (2014) and Fallaize et al. (2020). In addition, powerful probabilistic methods for characterizing structure ensembles are also available (Theobald and Wuttke 2008). However, when working at non-atomic resolution, new tools for assessing the precision of structural models need to be developed. PrISM identifies regions of low and high precision in ensembles of bead models (Ullanat et al. 2022) and color-codes them in a graphical representation. That way, reliable and ambiguous regions can easily be identified. In addition, it is desirable to devise new visual analytics tools for Bayesian integrative structures. These tools could show distributions of structural degrees of freedom and additional model parameters.

5.1 High-resolution integrative modeling powered by ML-based structure prediction

Protein structures predicted with ML-based systems such as AlphaFold2 (AF2) and RoseTTAFold are already used in integrative modeling. For example, the combination of cryo-ET maps with high-resolution structure prediction has enabled near-atomic modeling of the nuclear pore complex (Mosalaganti et al. 2022). Another example is the integration of intermediate-resolution cryo-EM and AF2 to obtain a detailed model of the cytoplasmic ring of the nuclear pore complex (Fontana et al. 2022). A natural next step in protein structure prediction was to use ML-systems to predict protein complexes. In their assessment of the impact of AF2 on structural biology, Akdel et al. (2022) found that AF2 is already capable of predicting the structure of some protein complexes and outperforms standard docking approaches. Several extensions of AF2 and other ML systems have been proposed that will also facilitate integrative modeling. AF2 has recently been extended to specifically target protein complexes (Evans et al. 2021). AlphaFold-multimer outperforms AF2 on predicting protein complexes. While the performance is already quite impressive, the predictions of protein complexes tends to be more accurate for symmetric complexes rather than heteromeric assemblies. To improve the prediction of heteromeric complexes, FoldDock (Bryant et al. 2022a) combines AF2 with docking methods. AF2Complex predicts physical contacts between subunit structures without requiring paired sequence alignments (Gao et al. 2022). Several large-scale modeling studies have been carried out to predict the structures of entire interactomes. Humphreys et al. (2021) applied RoseTTAFold and AF2 to predict protein complexes and model their 3D structures. FoldDock has been used to predict complex structures of the human protein interaction network (Burke et al. 2023). Bryant et al. (2022b) use AF-Multimer and FoldDock to predict the assemblies that comprise as many as 30 chains. Nonetheless large assemblies still pose a formidable challenge and typically require additional information. Another challenge is that systems like AF2 do not predict all accessible conformations of a biomolecule (Lane 2023) and miss out on the structure of nucleic acids (Tunyasuvunakool 2022). RoseTTAFoldNA predicts complexes of proteins and nucleic acids (Baek et al. 2022). More applications of AF2 and RoseTTAFold relevant to challenges in integrative modeling are under development. For example, Terwilliger et al. (2022) combined AF2 with experimental information. Stein and Mchaourab (2022) have modified AF2 so as to generate conformational ensembles. There is no foreseeable end to the development of ML methods in protein structure prediction in the near future. For example, language models have recently be used to enable rapid and highly reliable prediction of protein structures (Lin et al. 2023). For integrative modeling, we expect that physics-based modeling in combination with experimental data and ML-based models of subunits will continue to be the most promising approach given the scarcity and heterogeneity of training data.

5.2 Learning of coarse-grained representations and forcefields

In principle, Bayesian methods can be used to infer any type of parameter including parameters of the forcefield used in the prior distribution Pr(X | M). The technical difficulty of learning these hyperparameters is that the normalizing constant of the prior depends on the parameters, but is not available in closed form resulting in a doubly intractable model. Approximate MCMC algorithms have been developed to infer doubly intractable models. Various approaches to learn forcefields with Bayesian methods have been proposed (Bottaro and Lindorff-Larsen 2018; Ge and Voelz 2018; Habeck 2014; Köfinger and Hummer 2021; Madin et al. 2022). These need to be improved and should become a part of integrative modeling itself.

Machine learning also impacts coarse-graining of proteins and their forcefields (Ding et al. 2023; Durumeric et al. 2023). A promising development is the use of representation learning in the context of molecular structures. These models could serve as prior distributions over the structural degrees of freedom. Particle-based simulator models have been inferred for deformable objects and fluids (Li et al. 2018) and might also be learnable for biomolecular systems. Wang et al. (2022) use generative models to train CG representations of molecules including short peptides.

5.3 Improved posterior sampling via coarse-graining and variational inference

Uncertainty quantification requires posterior sampling. Therefore, drawing structural models from the posterior distribution is at the core of a truly Bayesian approach to integrative modeling. It is also a major bottleneck in scaling the Bayesian approach to large complexes and cellular systems. Coarse-graining might not only help to cope with the scarcity of data, but also benefit posterior sampling and should be exploited for sampling in a more systematic fashion.

Posterior sampling typically relies on Markov chain Monte Carlo simulation. The current working horse in Bayesian integrative modeling are MCMC methods such as Hamiltonian Monte Carlo and parallel tempering (Rieping et al. 2005; Saltzberg et al. 2021). A viable alternative might be to learn a variational approximation of the posterior. Therefore, ML techniques powered by deep generative models could also help overcome the sampling problem in integrative modeling. For example, Boltzmann generators introduced by Noé et al. (2019) use deep architectures for sampling the Boltzmann distribution of biomolecules. These should be applicable to posterior distributions arising in integrative structure determination. More, recently Monroe and Shen (2022) have demonstrated learning of efficient, collective Monte Carlo moves with variational autoencoders.

5.4 Modeling the dynamics of biomolecular systems

Biological function involves dynamical changes in the structure and composition of biomolecular systems. Therefore, integrative modeling has to go beyond static structures and infer structure and dynamics simultaneously from the data. Inference of conformational ensembles and multi-state models is a first step to go beyond static models. Simultaneous determination of protein structure and dynamics from cryo-EM data has been proposed by Bonomi et al. (2018). Giraldo-Barreto et al. (2021) extract free-energy profiles from cryo-EM micrographs using a Bayesian approach called BIFE. BIFE recovers free-energy profiles along a path collective variable from cryo-EM images. On a real data set, not only the most probable conformation but also metastable states are found and activation barriers are estimated thereby providing a more complete characterization of the thermodynamic ensemble. Recent developments in representation learning also enable the reconstruction of heterogeneous structures directly from cryo-EM data. CryoDRGN uses a variational autoencoder (VAE) to learn ensembles of cryo-EM structures from projection images (Zhong et al. 2021).

What are efficient representations of dynamic structures? An explicit representation is a trajectory of successive conformational states. Also in the context of representing biomolecular dynamics, deep generative models inspired by recent developments in representation learning are a promising approach (Hoseini et al. 2021). It is becoming apparent that going beyond structures and searching for efficient and expressive representations will become important. In the light of the ongoing success of ML-based approaches to structure prediction, also more conceptual questions such as “What is a structural model?” will become relevant. We are used to think of structural models as an explicit set of 3D coordinates. Implicit models that generate coordinate arrays, potentially at varying resolution, might replace explicit representations. This is would have vast implications on the way biomolecular structures are represented, stored and used. Guo et al. (2021) generate protein structures with graph VAEs. Diffusion models such as torsional diffusion (Jing et al. 2022) or EigenFold (Jing et al. 2023) can generate protein structures and could be efficient representations of conformationally heterogeneous and flexible protein structures. More generally, differential programs such as simulators will play an important role in modeling biophysical processes (AlQuraishi and Sorger 2021). For example, Ingraham et al. (2019) propose a differentiable simulator to model protein structures.

5.5 Emerging data

A number of emerging experimental techniques will provide additional sources for integrative modeling. These data need to be supported by new probabilistic models linking the data to the structural degrees of freedom. New types of data will help us bridge the gap between the atomic and cellular scale. For example, tomographic methods such as cryo-ET and soft X-ray tomography provide 3D maps of organelles and other cellular compartments (Loconte et al. 2022, 2023). The 4D nucleome project collects detailed information about the structural organization of the cell nucleus at various stages of the cell cycle (Dekker et al. 2017). A multitude of proximity ligation and imaging methods enables the study of genome organization (Jerkovic and Cavalli 2021; Misteli 2020) including ChromEMT (Ou et al. 2017). These developments are complemented by light microscopy techniques that allow for structural structures of protein complexes and larger structural units (Sieben et al. 2018; Sigal et al. 2018). Correlative microscopy combining fluorescence with electron microscopy allows the visualization of whole cells at many scales (Hoffman et al. 2020). Multiscale modeling of biological systems will require neighborhood information obtained by mapping physical and functional proximities (Schaffer and Ideker 2021). There is a growing array of proximity ligation assays that allows the characterization the structure of genomes, chromosomes or large RNAs. Advanced cross-linking techniques uncover interactions within and between interacting proteins (Braberg et al. 2020; Graziadei and Rappsilber 2022; Mintseris and Gygi 2020; O’Reilly et al. 2020; Sae-Lee et al. 2022; Yperman et al. 2021). We also see a continued improvement of cryo-EM maps towards atomic resolution (Yip et al. 2020) and the emergence of single-molecule diffraction techniques based on, for example, free electron lasers (von Ardenne et al. 2018).

5.6 Structural cell modeling

The long-term goal of integrative modeling is the creation of physical molecular models of cellular components and of entire cells (Singla et al. 2018; Zhong et al. 2022). To achieve this goal, a large array of data and additional information needs to be fed into the modeling procedure to produce a consistent model of the cell. A promising approach is to couple simulation software such as cellPACK (Johnson et al. 2015) to cellular data. An impressive demonstration of this approach is the structural model of a mycoplasma cell (Maritan et al. 2022). Structural cell modeling is facing a number of challenges (Im et al. 2016). Bayesian metamodeling of cellular systems is a highly exciting development that asims to tackle these challenges within a Bayesian framework (Raveh et al. 2021).