Problems with Tests of the Missingness Mechanism in Quantitative Policy Studies
-
Christopher H. Rhoads
Policy analysts involved in quantitative research have many options for handling missing data. The method chosen will often greatly influence the substantive policy conclusions that will be drawn from the data. The most frequent methods for handling missing data assume that the data are missing at random (MAR). The current paper notes that an omnibus, nonparametric test of the MAR assumption is impossible using the observed data alone. Nonetheless various purported tests of the missingness mechanism (including tests of MAR) appear in the literature. The current paper clarifies that all of these tests rely on some assumption that cannot be tested from the data. The paper notes that tests of the missingness mechanism are frequently misinterpreted and it clarifies the appropriate interpretation of such tests. Policy analysts are encouraged not to develop the false impression that modern procedures for handling missing data in conjunction with tests of the missingness mechanism provide protection against the ill effects of missing data. Any justification for a particular approach to handling missing data must be come from substantive knowledge of the missingness process, not from the data.
©2012 Walter de Gruyter GmbH & Co. KG, Berlin/Boston
Articles in the same Issue
- Article
- Improving Statistical Inference with Clustered Data
- Major Contributions to Quantitative Economics Sponsored by the Defense Community
- Problems with Tests of the Missingness Mechanism in Quantitative Policy Studies
- Climate Statistics and Public Policy
- Commentary and Ideas
- Data, Statistics, and Controversy: Making Science Research Data Intelligible
- Response or Comment
- Why and When "Flawed" Social Network Analyses Still Yield Valid Tests of no Contagion
- Comment on "Why and When 'Flawed' Social Network Analyses Still Yield Valid Tests of no Contagion"
Articles in the same Issue
- Article
- Improving Statistical Inference with Clustered Data
- Major Contributions to Quantitative Economics Sponsored by the Defense Community
- Problems with Tests of the Missingness Mechanism in Quantitative Policy Studies
- Climate Statistics and Public Policy
- Commentary and Ideas
- Data, Statistics, and Controversy: Making Science Research Data Intelligible
- Response or Comment
- Why and When "Flawed" Social Network Analyses Still Yield Valid Tests of no Contagion
- Comment on "Why and When 'Flawed' Social Network Analyses Still Yield Valid Tests of no Contagion"
