BLUETHESIS / INDEX
FARequest guidance09302822091

SEARCH THE INDEX

Find the research topic you need.

Search services, articles and research resources.

Research methods guide

AIC vs. BIC: Choosing the Right Model Selection Criterion for Your Research

Comparison of AIC and BIC model selection criteria in quantitative research

Introduction to Model Selection Criteria

AIC and BIC are two of the most widely used criteria for selecting statistical models. While they share a common goal—balancing model fit and complexity—they differ fundamentally in their approach. AIC prioritizes predictive accuracy, often favoring more complex models if they provide even marginal improvements. BIC, in contrast, emphasizes parsimony, penalizing complexity more heavily, especially as sample size grows. Understanding these differences is critical for researchers in fields ranging from econometrics to machine learning.

This guide explains the mathematical foundations of AIC and BIC, their practical applications, and when to use each criterion. We also address common pitfalls and provide software implementation notes for R, Python, and SPSS.

For researchers working with nested models (e.g., comparing ARIMA(1,1,1) vs. ARIMA(2,1,1)), this distinction becomes particularly important, as AIC and BIC may yield conflicting recommendations. By the end of this article, you will be equipped to make informed decisions about model selection based on your research goals.

Mathematical Foundations

AIC: The Prediction-Focused Criterion

The Akaike Information Criterion is defined as:

AIC = 2k − 2ln(ŷ)

where:

  • k = number of estimated parameters in the model
  • ln(ŷ) = maximized log-likelihood of the model

AIC penalizes model complexity with a constant term (2k) while rewarding fit through the log-likelihood term. Unlike BIC, AIC’s penalty does not scale with sample size, making it less conservative in larger datasets.

AIC is asymptotically efficient but inconsistent. This means it will select the true model with probability 1 as sample size grows only if the true model is among the candidates. However, it may still overfit by selecting overly complex models, even with large samples. For further reading on related modeling considerations, review the Correlation vs Regression guide.

BIC: The Parsimony-Focused Criterion

The Bayesian Information Criterion is defined as:

BIC = ln(n) * k − 2ln(ŷ)

where:

  • n = sample size
  • k = number of estimated parameters
  • ln(ŷ) = maximized log-likelihood

BIC’s penalty term (ln(n) * k) grows with sample size, making it increasingly conservative as n increases. This stronger penalty often leads to simpler models compared to AIC. To better understand how sample characteristics affect statistical models, consult the Sample Size and Statistical Power guide.

BIC is consistent: if the true model is among the candidates, it will select it with probability 1 as n → ∞. This makes BIC particularly useful for theory testing, where identifying the true underlying model is the primary goal.

Key Differences: AIC vs. BIC

Feature AIC BIC
Primary Goal Prediction accuracy Model parsimony
Penalty Term 2k (constant) ln(n) * k (scales with sample size)
Asymptotic Behavior Efficient but inconsistent Consistent
Preferred When Forecasting, high-dimensional data Theory testing, inference
Model Complexity Tends to select more complex models Tends to select simpler models

When to Use AIC or BIC

Use AIC When…

  • Your primary goal is prediction (e.g., forecasting time-series data, machine learning tasks).
  • You are working with high-dimensional data where capturing nuanced patterns is critical.
  • You suspect the true model is not among your candidates, and you want to minimize prediction error.
  • You are comparing non-nested models (e.g., different functional forms).

Use BIC When…

  • Your goal is inference (e.g., identifying the true underlying model).
  • You prioritize parsimony and want to avoid overfitting.
  • You have a large sample size, as BIC’s penalty becomes more severe.
  • You are testing nested models and want to ensure consistency in model selection.

Empirical Examples and Trade-offs

In practice, AIC and BIC often yield different recommendations, especially when comparing nested models. For example, in time-series analysis:

  • AIC might favor a higher-order ARIMA model (e.g., ARIMA(2,1,1)) if it provides a marginal improvement in fit.
  • BIC might select a simpler model (e.g., ARIMA(1,1,1)) due to its stronger penalty for complexity.

This trade-off is visualized in empirical comparisons, where AIC tends to select more complex structures, while BIC favors parsimony. When evaluating categorical or limited dependent variables under similar frameworks, reviewing the Binary Logistic Regression guide offers valuable context.

Common Pitfalls and Best Practices

Pitfall 1: Ignoring Relative Nature

AIC and BIC are relative metrics. They are only meaningful when comparing models fitted to the same dataset. Comparing AIC/BIC values across different datasets is invalid.

Pitfall 2: Misinterpreting Consistency vs. Efficiency

AIC’s inconsistency means it may not select the true model, even with large samples. BIC’s consistency ensures it will select the true model if it is among the candidates, but this comes at the cost of potentially underfitting in prediction tasks.

Pitfall 3: Applying to Non-Nested Models Without Caution

While AIC and BIC can be used for non-nested models, their theoretical guarantees (e.g., consistency for BIC) assume the true model is among the candidates. Omitting relevant predictors can bias selection.

Pitfall 4: Overlooking Sample Size Dependence

BIC’s penalty (ln(n) * k) becomes increasingly severe as sample size grows. In small samples, AIC and BIC may yield similar results, but their divergence grows with n.

Software Implementation

Most statistical software automatically reports AIC and BIC for fitted models. Below are brief notes on implementation in R, Python, and SPSS:

R

In R, AIC and BIC are available for most model-fitting functions (e.g., lm(), arima()). For example:

model <- arima(data, order = c(1,1,1))

The output includes AIC and BIC values, which can be compared across models. For panel data structures, you may also explore the Fixed Effects vs. Random Effects guide.

Python

In Python, libraries like statsmodels and pmdarima provide AIC and BIC for fitted models:

from statsmodels.tsa.arima.model import ARIMA

The fit() method returns AIC and BIC, allowing direct comparison.

SPSS

SPSS reports AIC and BIC in the model summary output for regression and time-series models. Users can manually compare values across models.

Conclusion

Choosing between AIC and BIC depends on your research goals. If your priority is prediction, AIC is the better choice. If you seek parsimony or inference, BIC is more appropriate. Both criteria have strengths and limitations, and their application requires careful consideration of model assumptions, sample size, and the true model’s presence among candidates.