Handling missing data: an interactive guide
Mechanisms, methods and evidence, with simulations you can run in your browser
This page goes with the SGSSS Winter School session Handling missing data (10 December 2026). It covers the same ground as the slides, in order: what causes missing data, what you can and can't test, how to judge a method, and how complete records analysis, FIML and multiple imputation behave in practice.
Every simulation on this page runs live in your browser. It uses the design from Oatley, Gayle and Connelly (forthcoming), so you can check the claims for yourself rather than taking them on trust. Every dataset is generated from the same model:
y = 500 + 250 x1 + 400 x2 + 375 x3 + ε
The three covariates are correlated (r between 0.30 and 0.40), R² is 0.65, and only x2 has missing values. Because we made the data, we know the true coefficients. That is what lets us see whether a method gets them right.
Slides, Stata .do files, R scripts and datasets for every part are on the training page; direct links are in section 13.
1Missing data and its mechanisms
Missing values on their own cost you precision. Whether they also bias your results depends on why the values are missing. Rubin (1976) distinguishes three mechanisms:
| Mechanism | The chance a value is missing depends on | Example (an income question) |
|---|---|---|
| MCAR (missing completely at random) | Nothing: it is unrelated to any data | A server fault loses a random batch of responses |
| MAR (missing at random) | Only things you have observed | Older people skip the question, and age is recorded |
| MNAR (missing not at random) | The missing values themselves, beyond anything observed | High earners withhold income because it is high |
There is an important split inside MAR. If a covariate in your model drives the missingness, dropping incomplete cases selects on that covariate. The relationship between y and the x's is left intact, so a complete records analysis is still unbiased. If the outcome drives the missingness, dropping cases selects on y and distorts the relationship you are trying to estimate. Both are “MAR”, but they have opposite consequences.
2What the data can and can’t tell you
The standard diagnostics compare people with and without a missing value on the variables you can see: t-tests, chi-squared tests, a logistic regression of a missingness indicator, or Little’s MCAR test (mcartest in Stata, mcar_test() in R). If the groups differ, the data are not MCAR.
What no test can do is compare them on the values that are missing. The table below uses the sample from section 1. The last row is only possible because this is a simulation.
How much missing data is too much? Rules of thumb (5%, 10%) don’t tell you whether results are biased. The mechanism decides whether a method is valid; the proportion missing decides how much precision you lose. You can check this in the lab in section 5: complete records analysis is unbiased at 90% missing when x1 drives the missingness, and biased at 50% when y does.
3Three ways to judge a method
“Does this method work?” is really three separate questions. Methods fail them in every combination, so keep them apart.
| Criterion | Question | Measured by |
|---|---|---|
| Accuracy | Over many samples, is the average estimate equal to the true value? | Absolute percentage bias |
| Efficiency | Among accurate methods, how much do the estimates vary from sample to sample? | Empirical standard error |
| Calibration | Do the standard errors the method reports match how much its estimates actually vary? | Relative error in the SE; coverage of 95% intervals |
Bias matters most because it doesn’t shrink as the sample grows: a bigger sample just homes in on the wrong value with more confidence. Efficiency has a useful interpretation: if one method’s sampling variance is half another’s, it gets as much out of the data as the other would from twice as many cases.
A single dataset can’t show you any of this, because you only get one estimate. Section 4 shows what one sample looks like; section 5 repeats the experiment hundreds of times so you can see all three criteria.
4Nine methods on one sample
These are the nine methods compared in the paper, grouped by when they are valid:
| Group | Methods | Valid when |
|---|---|---|
| Inadequate | Mean imputation; regression imputation (filling in predicted values with no added noise); missing-indicator (dummy) method | Almost never in real data |
| Conditionally valid | Complete records analysis (listwise deletion); inverse probability weighting (IPW) | Missingness unrelated to y given x (deletion); correct model for who is missing (IPW) |
| Gold standard | Full information maximum likelihood (FIML); multiple imputation (MI), with or without auxiliary variables | Data are MAR |
5Repeated-sampling lab
This is the simulation study in miniature. Pick a mechanism and a missingness rate, then generate hundreds of datasets, apply every method to each, and see how the estimates behave across samples. The table reports the three criteria; the plots show the full spread of estimates.
6Why mean imputation fails
Mean imputation sets every missing x2 to the same value. With a single predictor the attenuation happens to cancel out, so it can look harmless. With correlated covariates, every imputed case sits on a flat line in the x2–x1 relationship. The regression then shares out the explained variance wrongly among the covariates, and every coefficient is biased, including those on fully observed variables.
The only setting where mean imputation is harmless for regression coefficients is when the covariates are uncorrelated, which real data almost never are. Simulation studies that use independent covariates therefore understate its damage.
7Full information maximum likelihood
FIML neither deletes nor fills in anything. Each case contributes to the likelihood whatever it has observed. A complete case contributes the joint density of y, x1, x2 and x3. A case missing x2 contributes the joint density of y, x1 and x3, with x2 integrated out. Because the means and covariances are shared across both kinds of case, the partial records still inform every parameter.
FIML assumes MAR and multivariate normality. Two practical points matter:
- The covariates must be part of the likelihood. In R’s lavaan the default
fixed.x = TRUEdrops cases with missing covariates, which turns it back into a complete records analysis. Setfixed.x = FALSEand check the N. Stata’smethod(mlmv)does this for you. - Binary and ordinal covariates are treated as normal. Software built for categorical variables (Mplus, Latent GOLD) handles them properly.
* Stata
sem (y <- x1 x2 x3), method(mlmv)
# R
fit <- lavaan::sem('y ~ x1 + x2 + x3', data = dat, missing = "fiml", fixed.x = FALSE)
8Inside multiple imputation
Multiple imputation repairs the two flaws of regression imputation. Each missing value gets a predicted value plus random noise, drawn using regression parameters that are themselves uncertain. This is repeated m times. You analyse each completed dataset and combine the results with Rubin’s rules:
| Quantity | Formula | Meaning |
|---|---|---|
| Pooled estimate | Q̄ = (1/m) Σ Q̂j | The average across imputations |
| Within-imputation variance | W̄ = (1/m) Σ Ŵj | Ordinary sampling variance |
| Between-imputation variance | B = Σ (Q̂j − Q̄)² / (m − 1) | Uncertainty due to the missing data |
| Total variance | T = W̄ + (1 + 1/m) B | What the pooled standard error is based on |
Build the imputation model carefully:
- Include every analysis variable, including the outcome. Leaving y out draws x2 as if it had nothing to do with y, which pulls its coefficient towards zero.
- Use the right model for each variable type: regression or predictive mean matching for continuous, logit, ologit or mlogit for categorical.
- Impute interactions and squared terms as “just another variable” in linear models (von Hippel, 2009).
- Only impute variables that have missing values; register the rest as regular.
9Auxiliary variables and the number of imputations
An auxiliary variable is one that isn’t in your analysis model but predicts the missing values, or whether they are missing. Earlier waves, linked administrative data and related survey items are typical sources. An auxiliary only helps where your analysis variable is missing, so it must be observed in those cases.
In the paper (1,000 replications, 90% missing), auxiliaries raised MI’s effective sample size from 1.48 to 2.09 times that of deletion. Going from 10 to 100 imputations added only a little more, at about nine times the run time. Common guidance on m: at least the percentage of incomplete cases (White, Royston and Wood, 2011), or von Hippel’s (2020) two-stage rule based on FMI. Below about 50% missing, m = 10 and m = 100 were indistinguishable.
10Published results
These are the full-scale results from the paper (1,000 replications, n = 1,000). Your runs in the lab should land close to them.
Covariate-driven MAR, 50% missing: coefficient on x2
Precision as missingness rises
Outcome-driven MAR and MNAR, 50% missing
Source: Oatley, Gayle and Connelly (forthcoming), Tables 4–6.
11A decision guide
Whatever the answers, don’t use mean, mode or median imputation, regression imputation or missing-indicator methods to estimate effects.
12Check yourself
Which mechanism is this?
A fieldwork tablet crashes on random days and loses those interviews.
MCAR. Deletion is unbiased; you only lose precision.
Older respondents skip a wellbeing item more often. Age is in the model, and wellbeing is a covariate.
MAR, covariate-driven. Deletion is unbiased. FIML or MI would be more precise.
In a model of GCSE attainment, parental education is missing more often for pupils with low attainment.
MAR, but outcome-driven. Deletion is biased. Use FIML, MI or IPW, with attainment in the imputation model.
People with high debts are less likely to report them, and nothing you observe predicts this.
MNAR. No standard method is guaranteed. Use auxiliaries, sensitivity analysis, and say so in the write-up.
Which method would you use?
40% missing on parental income; missingness is predicted by region and school type, both in your model.
Complete records analysis is valid. MI or FIML with auxiliaries would be more precise.
25% missing on a wellbeing covariate; pupils with poor attendance (your outcome) skip it more.
FIML or MI, with attendance in the imputation model.
Self-reported drinking is missing more often for heavy drinkers, and nothing observed predicts it well.
MI with auxiliaries, plus sensitivity analysis and an honest write-up.
13Code and reading
Session files
| Session | Stata | R |
|---|---|---|
| 1. Diagnosing missing data (sections 1–2) | session one.do | session one.R |
| 2. Deletion, single imputation and missing indicators (sections 4–6) | session two.do | session two.R |
| 3. FIML (section 7) | session three.do | session three.R |
| 4. Multiple imputation (sections 8–9) | session four.do | session four.R |
Datasets for session 1: datasetone.dta, datasettwo.dta, datasetthree.dta.
Multiple imputation in Stata
mi set mlong
mi register imputed age gender education income
mi register regular aux1 aux2
mi impute chained (regress) age income ///
(logit) gender ///
(ologit) education ///
= aux1 aux2, add(20) rseed(12345)
mi estimate: regress income age i.gender i.education
mi estimate, vartable dftable // RVI, FMI
Multiple imputation in R
library(mice)
meth <- make.method(df)
meth[c("age", "income")] <- "pmm"
meth["gender"] <- "logreg"
meth["education"] <- "polr"
meth[c("aux1", "aux2")] <- "" # predictors only
pred <- quickpred(df, minpuc = 0.2) # keep the outcome in
imp <- mice(df, m = 20, method = meth, predictorMatrix = pred, seed = 123)
summary(pool(with(imp, lm(income ~ age + gender + education))))
What to report
- The amount and pattern of missing data on each variable
- Your argument for the mechanism, and the diagnostics you ran
- The method, and why it suits that mechanism
- For MI: the software and command, the imputation model (variables, auxiliaries, model for each variable type), m, the iterations and the seed
- Convergence checks, FMI, and a comparison with complete records analysis
Reading
Carpenter, J.R. and Kenward, M.G. (2012) Multiple Imputation and its Application. Wiley.
Collins, L.M., Schafer, J.L. and Kam, C.-M. (2001) A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330–351.
Enders, C.K. (2010) Applied Missing Data Analysis. Guilford.
Little, R.J.A. (1992) Regression with missing X’s: a review. Journal of the American Statistical Association, 87(420), 1227–1237.
Little, R.J.A. and Rubin, D.B. (2019) Statistical Analysis with Missing Data, 3rd edn. Wiley.
Madley-Dowd, P. et al. (2019) The proportion of missing data should not be used to guide decisions on multiple imputation. Journal of Clinical Epidemiology, 110, 63–73.
Morris, T.P., White, I.R. and Crowther, M.J. (2019) Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102.
Oatley, S., Gayle, V. and Connelly, R. (forthcoming) A simulation study comparing missing data handling strategies: accuracy, efficiency and calibration under MCAR, MAR and MNAR.
Rubin, D.B. (1976) Inference and missing data. Biometrika, 63(3), 581–592.
Seaman, S.R. and White, I.R. (2013) Review of inverse probability weighting for dealing with missing data. Statistical Methods in Medical Research, 22(3), 278–295.
von Hippel, P.T. (2009) How to impute interactions, squares, and other transformed variables. Sociological Methodology, 39(1), 265–291.
White, I.R. and Carlin, J.B. (2010) Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Statistics in Medicine, 29(28), 2920–2931.
White, I.R., Royston, P. and Wood, A.M. (2011) Multiple imputation using chained equations: issues and guidance for practice. Statistics in Medicine, 30(4), 377–399.