quantile-forest

quantile-forest offers a Python implementation of quantile regression forests compatible with scikit-learn.

Quantile regression forests (QRF) are a non-parametric, tree-based ensemble method for estimating conditional quantiles, with application to high-dimensional data and uncertainty estimation [1]. The estimators in this package are performant, Cython-optimized QRF implementations that extend the forest estimators available in scikit-learn to estimate conditional quantiles. The estimators can estimate arbitrary quantiles at prediction time without retraining and provide methods for out-of-bag estimation, calculating quantile ranks, and computing proximity counts. They support sample weights, multiple targets, monotonic constraints, and missing values. They are compatible with and can serve as drop-in replacements for the scikit-learn forest regressors.

Example of fitted model predictions and prediction intervals on California housing data (code)

Quick Start

To install quantile-forest from PyPI using pip:

pip install quantile-forest

To install quantile-forest from conda-forge using conda:

conda install quantile-forest -c conda-forge

Usage

A quantile forest is fit like a scikit-learn regressor, and predicts any quantiles of the target without being fit again. For example, to predict the median and a 95% prediction interval:

from sklearn import datasets
from sklearn.model_selection import train_test_split
from quantile_forest import RandomForestQuantileRegressor

X, y = datasets.load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)
qrf = RandomForestQuantileRegressor().fit(X_train, y_train)
y_pred = qrf.predict(X_test, quantiles=[0.025, 0.5, 0.975])  # one column per quantile

Documentation

An installation guide, API documentation, and examples can be found in the documentation. The release notes list the changes in each version.

References

[1] N. Meinshausen, "Quantile Regression Forests", Journal of Machine Learning Research, 7(Jun), 983-999, 2006. https://www.jmlr.org/papers/volume7/meinshausen06a/meinshausen06a.pdf

Citation

If you use this package in academic work, please consider citing https://joss.theoj.org/papers/10.21105/joss.05976:

@article{Johnson2024,
    doi = {10.21105/joss.05976},
    url = {https://doi.org/10.21105/joss.05976},
    year = {2024},
    publisher = {The Open Journal},
    volume = {9},
    number = {93},
    pages = {5976},
    author = {Reid A. Johnson},
    title = {quantile-forest: A Python Package for Quantile Regression Forests},
    journal = {Journal of Open Source Software}
}