Research methods guide
AIC vs. BIC: Choosing the Right Model Selection Criterion for Your Research

Introduction to Model Selection Criteria
AIC and BIC are two of the most widely used criteria for selecting statistical models. While they share a common goal—balancing model fit and complexity—they differ fundamentally in their approach. AIC prioritizes predictive accuracy, often favoring more complex models if they provide even marginal improvements. BIC, in contrast, emphasizes parsimony, penalizing complexity more heavily, especially as sample size grows. Understanding these differences is critical for researchers in fields ranging from econometrics to machine learning.
This guide explains the mathematical foundations of AIC and BIC, their practical applications, and when to use each criterion. We also address common pitfalls and provide software implementation notes for R, Python, and SPSS.
For researchers working with nested models (e.g., comparing ARIMA(1,1,1) vs. ARIMA(2,1,1)), this distinction becomes particularly important, as AIC and BIC may yield conflicting recommendations. By the end of this article, you will be equipped to make informed decisions about model selection based on your research goals.
Mathematical Foundations
AIC: The Prediction-Focused Criterion
The Akaike Information Criterion is defined as:
AIC = 2k − 2ln(ŷ)
where:
- k = number of estimated parameters in the model
- ln(ŷ) = maximized log-likelihood of the model
AIC penalizes model complexity with a constant term (2k) while rewarding fit through the log-likelihood term. Unlike BIC, AIC’s penalty does not scale with sample size, making it less conservative in larger datasets.
AIC is asymptotically efficient but inconsistent. This means it will select the true model with probability 1 as sample size grows only if the true model is among the candidates. However, it may still overfit by selecting overly complex models, even with large samples. For further reading on related modeling considerations, review the Correlation vs Regression guide.
BIC: The Parsimony-Focused Criterion
The Bayesian Information Criterion is defined as:
BIC = ln(n) * k − 2ln(ŷ)
where:
- n = sample size
- k = number of estimated parameters
- ln(ŷ) = maximized log-likelihood
BIC’s penalty term (ln(n) * k) grows with sample size, making it increasingly conservative as n increases. This stronger penalty often leads to simpler models compared to AIC. To better understand how sample characteristics affect statistical models, consult the Sample Size and Statistical Power guide.
BIC is consistent: if the true model is among the candidates, it will select it with probability 1 as n → ∞. This makes BIC particularly useful for theory testing, where identifying the true underlying model is the primary goal.
Key Differences: AIC vs. BIC
| Feature | AIC | BIC |
|---|---|---|
| Primary Goal | Prediction accuracy | Model parsimony |
| Penalty Term | 2k (constant) | ln(n) * k (scales with sample size) |
| Asymptotic Behavior | Efficient but inconsistent | Consistent |
| Preferred When | Forecasting, high-dimensional data | Theory testing, inference |
| Model Complexity | Tends to select more complex models | Tends to select simpler models |
When to Use AIC or BIC
Use AIC When…
- Your primary goal is prediction (e.g., forecasting time-series data, machine learning tasks).
- You are working with high-dimensional data where capturing nuanced patterns is critical.
- You suspect the true model is not among your candidates, and you want to minimize prediction error.
- You are comparing non-nested models (e.g., different functional forms).
Use BIC When…
- Your goal is inference (e.g., identifying the true underlying model).
- You prioritize parsimony and want to avoid overfitting.
- You have a large sample size, as BIC’s penalty becomes more severe.
- You are testing nested models and want to ensure consistency in model selection.
Empirical Examples and Trade-offs
In practice, AIC and BIC often yield different recommendations, especially when comparing nested models. For example, in time-series analysis:
- AIC might favor a higher-order ARIMA model (e.g., ARIMA(2,1,1)) if it provides a marginal improvement in fit.
- BIC might select a simpler model (e.g., ARIMA(1,1,1)) due to its stronger penalty for complexity.
This trade-off is visualized in empirical comparisons, where AIC tends to select more complex structures, while BIC favors parsimony. When evaluating categorical or limited dependent variables under similar frameworks, reviewing the Binary Logistic Regression guide offers valuable context.
Common Pitfalls and Best Practices
Pitfall 1: Ignoring Relative Nature
AIC and BIC are relative metrics. They are only meaningful when comparing models fitted to the same dataset. Comparing AIC/BIC values across different datasets is invalid.
Pitfall 2: Misinterpreting Consistency vs. Efficiency
AIC’s inconsistency means it may not select the true model, even with large samples. BIC’s consistency ensures it will select the true model if it is among the candidates, but this comes at the cost of potentially underfitting in prediction tasks.
Pitfall 3: Applying to Non-Nested Models Without Caution
While AIC and BIC can be used for non-nested models, their theoretical guarantees (e.g., consistency for BIC) assume the true model is among the candidates. Omitting relevant predictors can bias selection.
Pitfall 4: Overlooking Sample Size Dependence
BIC’s penalty (ln(n) * k) becomes increasingly severe as sample size grows. In small samples, AIC and BIC may yield similar results, but their divergence grows with n.
Software Implementation
Most statistical software automatically reports AIC and BIC for fitted models. Below are brief notes on implementation in R, Python, and SPSS:
R
In R, AIC and BIC are available for most model-fitting functions (e.g., lm(), arima()). For example:
model <- arima(data, order = c(1,1,1))
The output includes AIC and BIC values, which can be compared across models. For panel data structures, you may also explore the Fixed Effects vs. Random Effects guide.
Python
In Python, libraries like statsmodels and pmdarima provide AIC and BIC for fitted models:
from statsmodels.tsa.arima.model import ARIMA
The fit() method returns AIC and BIC, allowing direct comparison.
SPSS
SPSS reports AIC and BIC in the model summary output for regression and time-series models. Users can manually compare values across models.
Conclusion
Choosing between AIC and BIC depends on your research goals. If your priority is prediction, AIC is the better choice. If you seek parsimony or inference, BIC is more appropriate. Both criteria have strengths and limitations, and their application requires careful consideration of model assumptions, sample size, and the true model’s presence among candidates.
