Rene Valenzuela
  • Home
  • Blog
  • Slides

On this page

  • References

What is … conformal prediction

predictive-modeling
Author

René Valenzuela

Published

January 9, 2024

Modified

January 28, 2024

Conformal prediction is a machine learning framework for uncertainty quantification. It produces statistically valid prediction regions for any underlying point predictor only assuming exchangeability of the data [1]. This is in contrast to traditional point prediction frameworks, which provide a single best estimate for a target variable and often do not quantify the uncertainty associated with that estimate.

Conformal prediction was originally designed for an on-line setting in which labels are predicted successively, each one being revealed before the next is predicted. It requires a user-specified significance level for which the algorithm should produce its predictions. This significance level restricts the frequency of errors that the algorithm is allowed to make.

From the blog post [2] some advantages of conformal prediction are:

  • Guaranteed coverage: Prediction sets generated by conformal prediction come with coverage guarantees of the true outcome. Conformal prediction does not depend on a well-calibrated model and coverage can also be guaranteed across classes or subgroups.
  • Easy to use: Conformal prediction approaches can be implemented from scratch with just a few lines of code.
  • Model-agnostic: Conformal prediction works with any machine learning model. It uses the normal outputs of whatever your preferred model is.
  • Distribution-free: Conformal prediction makes no assumptions about underlying distributions of data, i.e. it is a non-parametric method.
  • No retraining required: Conformal prediction can be used without retraining your model. It is another way of looking at and using, model outputs.

References

  1. Conformal Prediction (Wikipedia)
  2. Introduction To Conformal Prediction With Python
  3. Conformal Prediction for Machine Learning Classification —From the Ground Up

Copyright 2025, René Valenzuela

 

Thus website is built with Quarto