Visual rating scales of brain atrophy in dementia: a systematic review of reliability and diagnostic accuracy

3.2.1 Medial temporal atrophy (MTA)

The Scheltens scale focuses on the bilateral medial temporal lobe (MTL) atrophy, looking at the width of the choroid fissure, the width of the temporal horn and the height of the hippocampus in the T1-weighted coronal plane (Scheltens et al. 1992). The degree of atrophy in each of these regions is combined to produce a score reflecting overall MTL atrophy. The scale is composed of five increments (Figure S4) ranging from 0 to 4 in which scores ≤1 indicating the absence of AD, and scores ≥2 indicating the presence of AD (Harper et al. 2015). The Scheltens et al. (1992) scale has been included in the research criteria for the diagnosis of AD (Dubois et al. 2007).

The inter-rater agreement measured with Cohen’s weighted kappa (wκ) in the original paper ranged from 0.72 to 0.84 (Scheltens et al. 1992). The level of expertise in the visual inspection of the MR images can affect ability to find the landmarks and then on the assessment of the level of atrophy; most of the studies in the review included neuroradiologists and radiologists. In general, the inter-rater agreement was fair to excellent (ranging from 0.3 to 0.91), generally higher for the rating of total MTA score than for the right and left sides. Furthermore, even in non-expert readers the agreement was moderate (Boutet et al. 2012), suggesting that the MTA scale can be easily and reliably used by non-experts. Moreover, the implementation of reference images or a training set protocol can affect the performance of the scale (Harper et al. 2015). Most of the studies included reference images or training protocols for the readers and, indeed, the final agreement was high. Lastly, to assess the consistency between measures, the studies have also included the intra-rater agreement for all the images processed or for a pull of them. In our sample, the intra-rater agreement ranged between 0.45 and 0.96, suggesting a moderate to near perfect agreement within the same rater.

The MRI parameters, as the weight, may also have an impact on the readability of the MR images: Fischbach-Boulanger et al. (2018) made a comparison between T1-weighted and T2-weighted images. In the study the inter-rater agreement was fair for both the two weights, suggesting similar abilities of the raters in the detection of the hippocampus atrophy through the MTA scale (Fischbach-Boulanger et al. 2018). Interestingly, Fischbach-Boulanger et al. (2018) also found that in the T2-weighted images, raters overestimated the atrophy compared to the T1-weighted images. Lastly, Molinder et al. (2021) pointed out the differences in the agreement considering the strength of the magnetic field. Considering both the 0.5 T and the 1.5 T, the inter-rater agreement was fair and the intra-rater agreement was substantial, showing that the magnetic field strength does not affect the rater’s score.

Moderate negative correlations have been detected between MTA scores volumetric measurements of the hippocampus, such as gray matter density or volume (Boutet et al. 2012; Fischbach-Boulanger et al. 2018; Fumagalli et al. 2018; Harper et al. 2016; Mårtensson et al. 2020; Molinder et al. 2021; Sheng et al. 2020; Wittens et al. 2024; Yuan et al. 2019). Furthermore, Enkirch et al. (2018) also found correlations between the MTA score and the cerebrospinal fluid (CSF) amyloid markers. Interestingly, the authors also assessed the entorhinal cortex atrophy by using the ERICA scale, which seemed to have higher diagnostic accuracy than the MTA scale in AD (Enkirch et al. 2018).

In the original paper of Scheltens et al. (1992) a sensitivity of 81 % and specificity of 67 % for AD versus age-matched controls was reported. The implementation of higher MRI field strength had further increased the reliability of the scale (Harper et al. 2015). For example, Koedam et al. (2011) using a magnetic field of 3 T found that the specificity for AD versus age-matched controls reached 92 %, and in the study of Wittens et al. (2024) the specificity was even higher up to 94 %. Surprisingly, the visual evaluation of T2-weighted images had higher discriminating power than the T1-weighted (Fischbach-Boulanger et al. 2018).

Furthermore, automated tools are now available to assess hippocampal atrophy and differentiate between physiological and pathological aging. Considering AD versus HC, comparable predictive and discriminatory abilities have been found between automated tools (e.g., Westman et al. 2011; Min et al. 2017), volumetric measures (e.g., Boutet et al. 2012; Molinder et al. 2021), and the MTA scale.

The implementation of the MTA scale was also extended with the purpose of differentiating between the prodromal stage of the neurodegenerative disease (i.e., MCI) or the risk factor for the onset of the AD, such as in people with subjective cognitive decline (SCD). Sheng et al. (2020) found a fair discriminative power between amnestic MCI (aMCI) and HC at the MTA scale which was comparable to the discriminative power obtained by the hippocampal GM density measured by volumetric measures. Good discriminative power between MCI and HC has also been detected by Wittens et al. (2024), and by Westman et al. (2011). Furthermore, MTA was also implemented for the differentiation of typical AD from atypical AD (i.e., posterior cortical atrophy (PCA) in Fumagalli et al. (2020)).

Moreover, the MTA scale has been also used for the longitudinal analyses, assessing the conversion to AD (Boutet et al. 2012; Ferreira et al. 2015). The authors concluded that the accuracy with an automatic tool was significantly better than the visual assessment for all non-expert readers regarding the differentiation between converted MCI (cMCI) and HC. While, for the stable MCI (sMCI) compared to HC, the difference between the accuracy of automatic and visual assessments was not significant (Boutet et al. 2012). However, the MTA scale is useful for determining MCI prognosis (Ferreira et al. 2015). Lastly, MTA scale has also been implemented for the differentiation of non-AD neurodegenerative diseases (e.g., Harper et al. 2016; Falgàs et al. 2020; Molinder et al. 2021; Falgàs et al. 2024), genetic forms (Falgàs et al. 2020; Fumagalli et al. 2018), and disease severity (Yuan et al. 2019), obtaining a fair to good discriminatory ability.

Software-based analyses have also been conducted to distinguish the early stages of neurodegenerative diseases from physiological brain changes in the brain. For example, Mårtensson et al. (2020) found that the software analysis had similar reliability MTA for differentiating MCI and SCD (Mårtensson et al. 2020).

To summarize, MTA scale was originally developed to assess AD-related atrophy but has recently been extended even to the prodromal stages and to other neurodegenerative diseases. The visual assessment of brain atrophy is reliable, consistent with volumetric hippocampal measures, and discriminates between neurodegenerative diseases and physiological aging. The results are reported in Tables 1 and 2.

3.2.2 Anterior temporal (AT)

The scale implemented by Davies et al. (2006) and Kipps et al. (2007) focuses on the bilateral anterior temporal lobe atrophy in the T1-weighted coronal plane by using five increments (Harper et al. 2015; Figure S4). The scale was implemented for the frontotemporal dementia (FTD) population to assess the prognosis (Davies et al. 2006). Moreover, the implementation of the AT scale has been extended to genetic variants (Falgàs et al. 2020; Fumagalli et al. 2018), to the assessment of other neurodegenerative forms related to the frontotemporal lobar degeneration (FTLD) (Harper et al. 2016), such as primary progressive aphasia (PPA) (Falgàs et al. 2020, 2024), and to other neurodegenerative diseases, such as AD (Fumagalli et al. 2020; Yuan et al. 2019).

The inter-rater and intra-rater agreement measured with Cohen’s kappa (κ) was 0.71 and 0.83, respectively (Davies et al. 2006). In our sample, the inter-rater agreement ranged from 0.57 to 0.93, meaning a moderate to near perfect agreement. The intra-rater agreement ranged from 0.63 to 0.95, meaning substantial to near perfect agreement. Interestingly, the highest agreement scores were related to the assessment of PPA variants (Falgàs et al. 2024). Furthermore, most of the studies found negative correlations between the AT score and GM atrophy, suggesting that it is a reliable measure of anterior temporal lobe atrophy (Falgàs et al. 2024; Fumagalli et al. 2018, 2020; Yuan et al. 2019).

The AT scale accurately discriminates between FTLD and both AD and Lewy body dementia (LBD) (Harper et al. 2016); between PPA variants from HC (Falgàs et al. 2020), except for the non-fluent PPA (nfvPPA) (Falgàs et al. 2024); among the PPA variants, mainly in differentiating the semantic PPA (svPPA) from the other variants (Falgàs et al. 2024); and between AD from HC (Fumagalli et al. 2020; Yuan et al. 2019). In our sample, no study has assessed the longitudinal changes in the AT brain atrophy to understand the predictive diagnostic accuracy of the prognosis.

To summarize, AT as a measure of the brain atrophy in the anterior temporal areas is reliable, consistent with volumetric measures and discriminates between neurodegenerative diseases, between different forms of PPA, and physiological aging (for the studies details see Tables 1 and 2).

3.3.1 Fronto-insula (FI)

The fronto-insula (FI) scale (Davies et al. 2009; Fumagalli et al. 2014) assesses the atrophy of the circular sulcus of the insula in the coronal plane on the slice where the anterior commissure becomes visible and the two following posterior (Falgàs et al. 2024) by using four points rating (from 0: no atrophy to 3: severe; Figure S4). The inter-rater and the intra-rater agreement ranged between substantial and near to perfect. Negative correlations were found between the FI scale and the GM density (Harper et al. 2016) and atrophy (Falgàs et al. 2024; Fumagalli et al. 2018; Yuan et al. 2019), suggesting that the scale is a reliable measure of insula atrophy. The scale had good discriminatory ability in distinguishing between the early onset of AD and HC (Falgàs et al. 2020; Harper et al. 2016), and AD severity (Yuan et al. 2019). While the discriminatory abilities between genetic mutations [poor: Fumagalli et al. (2018); excellent: Falgàs et al. (2020)], and PPA variants [poor: Falgàs et al. (2024); good: Falgàs et al. (2020)] are still entangled. The results are reported in Tables 1 and 3.

3.3.2 Orbitofrontal (OF)

The orbitofrontal (OF) scale (Davies et al. 2009; Fumagalli et al. 2014) evaluates the olfactory sulcus in the coronal plane by using four grade system (Figure S4) ranging from 0 (no atrophy) to 3 (severe atrophy) on the most anterior slice where the corpus callosum becomes visible (Falgàs et al. 2024). The inter-rater and the intra-rater agreement ranged between substantial and near to perfect. The negative correlations found between the OF scale and the GM density (Harper et al. 2016) and atrophy (Falgàs et al. 2024; Fumagalli et al. 2018, 2020; Yuan et al. 2019) suggest that the scale is a reliable proxy of olfactory sulcus enlargement. The scale had poor discriminatory values in distinguishing LBD and AD (Harper et al. 2016), LBD and HC (Harper et al. 2016), and FTD and AD (Harper et al. 2016). Furthermore, OF scale is not the best for discriminating PPA variants (Falgàs et al. 2024), but the discriminatory ability is fair between svPPA and HC (Falgàs et al. 2020). However, it fairly distinguishes AD and HC (Fumagalli et al. 2020; Yuan et al. 2019), even in early-onset AD (Falgàs et al. 2020), FTD and HC (Falgàs et al. 2020), and is excellent in the genetic forms of FTD (Falgàs et al. 2020). The results are reported in Tables 1 and 3.

3.3.3 Anterior cingulate (AC)

The anterior cingulate (AC) scale (Davies et al. 2009; Fumagalli et al. 2014) assesses the cingulate sulcus in the coronal plane with four grade system (Figure S4) ranging from 0 (no atrophy) to 3 (severe atrophy) on the most anterior slice where the corpus callosum becomes visible (Falgàs et al. 2024). Both the inter-rater and the intra-rater agreement ranged between substantial and near to perfect. The negative correlations found between the AC scale and the GM density (Harper et al. 2016) and atrophy (Falgàs et al. 2024; Fumagalli et al. 2018; Yuan et al. 2019) suggest that the scale is a reliable measure of anterior cingulate sulcus enlargement. The scale had poor discriminatory abilities in distinguishing LBD and AD + FTD (Harper et al. 2016), prodromal FTD and HC (Benussi et al. 2024), and was not the best for distinguishing PPA variants (Falgàs et al. 2024). However, it showed fair ability in the distinction between AD and HC (Falgàs et al. 2020; Yuan et al. 2019), between nfvPPA and HC (Falgàs et al. 2024), good between FTD and HC (Falgàs et al. 2020), and excellent abilities in distinguishing FTD genetic variants from HC (Falgàs et al. 2020). Interestingly, the combination of plasma neurofilament light and AC scale showed an excellent accuracy in distinguishing prodromal FTD and HC (Benussi et al. 2024). The results are reported in Tables 1 and 3.

3.3.4 Global cortical atrophy-frontal areas (GCA-F)

Two studies evaluated the global atrophy score (GCA) only the in frontal lobe (GCA-F) (Pasquier et al. 1996; Figure S4), finding an inter-rater agreement of wκ = 0.59 (Ferreira et al. 2016) and Spearman’s rho of 0.257 (Vanhoenacker et al. 2017), and a substantial (wκ = 0.7) intra-rater agreement (Ferreira et al. 2016). Negative correlations were found between CGA-F scores and both GM volume (Ferreira et al. 2016) and GM thickness (Ferreira et al. 2016), suggesting that GCA-F is a reliable measure of frontal lobe atrophy. Among the articles included, no study has assessed the diagnostic accuracy of the scale for neurodegenerative diseases. General information about the studies is included in Table 1.

To summarize, the visual rating scale assessing the frontal areas showed high reliability and correlations with GM volumetric measures. The discriminative abilities are excellent in distinguishing FTD genetic forms from controls, and the FI scale seems the most trustable in differentiating other neurodegenerative forms from controls.

3.4.1 Posterior atrophy (PA)

The Koedam scale focuses on the parietal lobe atrophy, looking at the posterior cingulate sulcus, precuneus, and the parieto–temporal sulcus in the T1-weighted sagittal and coronal plane (Harper et al. 2015; Koedam et al. 2011). The degree of atrophy in each of these regions is combined to produce a score reflecting overall PA atrophy. The scale is composed of four increments (Figure S4) ranging from 0 to 3 in which scores <1 indicating the absence of PA (Harper et al. 2015).

Both the inter-rater agreement and the intra-rater agreement of the scale are substantial to near perfect (ranging between 0.69–0.85 and 0.71–0.95), respectively. In the original study by Koedam et al. (2011), raters did not undergo a training session; while in other studies (Fumagalli et al. 2018; Harper et al. 2016; Sheng et al. 2020) raters received a training protocol. The presence of training for the raters in this case seems not to have an impact on the raters’ consistency and reliability.

The degree of atrophy measured by the PA scale negatively correlated with GM volume in the same posterior areas in all of the studies (Falgàs et al. 2024; Fumagalli et al. 2018, 2020; Harper et al. 2016; Möller et al. 2014; Sheng et al. 2020; Yuan et al. 2019).

The PA scale accurately discriminates between pathological and physiological aging (Falgàs et al. 2020; Ferreira et al. 2015; Möller et al. 2014; Yuan et al. 2019), between different forms of neurodegeneration (Koedam et al. 2011), between those MCI that will convert to AD from the stable MCI (Ferreira et al. 2015), and even in the genetic variants (Falgàs et al. 2020; Fumagalli et al. 2018). Furthermore, the PA scale can fairly distinguish between HC and MCI patients (Sheng et al. 2020). Furthermore, the discriminative power of the scale was similar to the automated classification method (Harper et al. 2016). Interestingly, the combination of MTA and PA scales increased the discriminative ability than the single scale scores both in the AD (Falgàs et al. 2020; Koedam et al. 2011) and in the MCI (Sheng et al. 2020) population.

To summarize, PA as a measure of the brain atrophy in the parietal areas is reliable, consistent with volumetric measures, and discriminates between neurodegenerative diseases and physiological aging. The combination of PA with other visual rating scales, such as the MTA, increases the discriminative power of each scale. Results are reported in Tables 1 and 4.

3.5.1 Global cortical atrophy (GCA)

The global cortical atrophy (GCA) evaluates brain atrophy in 13 regions in each hemisphere and the final score is given by the sum of each 13 regions scores (Pasquier et al. 1996). This scale was designed to assess atrophy following stroke and was not specifically developed to evaluate neurodegenerative dementias. However, it has also been applied in the assessment of neurodegenerative diseases. The inter-rater and intra-rater agreement of the GCA scale was evaluated by Koedam et al. (2011), reporting values of wκ > 0.7 and wκ = 0.85, respectively, while Ferreira et al. (2015) reported a lower value of intra-rater agreement (wκ = 0.7) but still substantial. However, poor diagnostic (AD vs HC, AUC = 0.653) and predictive performances (sMCI vs cMCI, AUC = 0.581) were found (Ferreira et al. 2015). General information about the studies is included in Table 1.