Skip to content
Projects

rmitsuboshi/miniboosts has 56 GitHub stars and 9 forks. GitHot measured 0 stars in 24 hours, 0 in 7 days, and 0 in 15 days. Last checked 2026-11-15 14:04 UTC. Last push 2026-11-03.

How these counts are measured

Measured star growth

in 24h
0
in 7 days
0
in 15 days
0
in 1 month
Update delayed

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

A collection of boosting algorithms written in Rust 🦀

Machine Learning

README.md

Original

MiniBoosts

MiniBoosts is a Rust library for implementing and comparing boosting algorithms. Its algorithms share the Booster, WeakLearner, and Classifier traits, so you can use the same data and weak learner across experiments.

Since version 0.5.0, MiniBoosts ships as one miniboosts crate; algorithms and utilities are internal modules, not separate registry packages. Earlier releases have a different API; see the migration notes when upgrading. Rust 1.85 or newer is required (edition 2024).

Try the algorithms in the browser with the interactive demo, or use them from Python with pip install miniboosts (see Python bindings).

Getting started

Add MiniBoosts to your application's Cargo.toml:

[dependencies]
miniboosts = "0.5.2"

The optimization routines use Clarabel for linear, quadratic, and conic programming. No Gurobi installation or license is required.

Add the following to src/main.rs to train a binary classifier on a small in-memory dataset:

use std::io::BufReader;
use miniboosts::{Sample, prelude::*};

fn main() {
    let csv = b"x,class\n-2,-1\n-1,-1\n1,1\n2,1\n";
    let sample = Sample::from_reader(BufReader::new(&csv[..]), true)
        .unwrap()
        .set_target("class");
    let learner = DecisionTreeBuilder::new(&sample)
        .max_depth(2)
        .split_by(SplitBy::Entropy)
        .build();
    let mut booster = AdaBoost::init(&sample)
        .tolerance(0.1)
        .force_quit_at(20);
    let classifier = booster.run(&learner);
    assert_eq!(classifier.predict_all(&sample), vec![-1, -1, 1, 1]);
}

For CSV or SVMlight files, use SampleReader::default(). Binary classification labels must be -1 and 1, with both classes present in the training sample. The assertion checks predictions on this training sample; use a separate test set to evaluate performance on unseen data.

Python bindings

The optional Python package exposes the same Rust algorithms through classes such as AdaBoostClassifier, with fit(X, y), predict(X), and decision_function(X). Weak learners can be native Rust learners (decision trees, bounded linear functions, logistic regression, Gaussian naive Bayes, RBF bumps, finite classes) or compatible custom Python/sklearn estimators; confidence values are strictly validated in [-1, +1]. Gradient boosting and LogitBoost use regression weak learners (the native regression tree or sklearn regressors). Binary labels are encoded with a mapping warning and restored on prediction.

Install it from PyPI (prebuilt wheels for Linux, macOS, and Windows):

python -m pip install miniboosts

To build it from a checkout instead (requires Rust), run python -m pip install ./python. Rust users need no Python dependencies. See the Python guide for examples, algorithm parameters, the explicit sklearn PredictionAdapter, and limits.

Decision boundaries from Python

These figures use the Python API to run all 11 algorithms with the same data and depth-2 Rust decision trees. Point colors show training labels, background colors show predicted classes, and black lines mark zero ensemble confidence. train and test report accuracy; test points are generated independently.

Two moons

80 training points with coordinate noise, with a limit of 100 rounds. SoftBoost failed with solver status AlmostSolved at nu=5; its panel explicitly shows a separate run with nu=1. The other capped methods use nu=5.

Python decision boundaries on two moons

Overlapping spirals with label noise

160 training points with stronger coordinate noise; 20 training labels (12.5%) are flipped and marked with rings. Test labels are not flipped. The iteration limit is 60, and capped methods use nu=16. All 11 runs completed successfully. Fine-grained boundaries can fit corrupted labels without improving test accuracy.

Python decision boundaries on noisy spirals

These are illustrative runs, not a performance ranking or a comparison at equal optimization accuracy. (cap) marks an iteration-limit stop. In particular, CorrectiveERLP uses ShortStep and is capped at 60 in this figure; a separate 300-round-budget run stopped at round 91. Algorithm stopping conditions depend on weak-learner assumptions and do not by themselves certify global optimality.

Reproduction scripts, seeds, parameters, results, and environment records are available in the figure directory.

Algorithms

Import these types from miniboosts or miniboosts::prelude:

Type Source
AdaBoost Freund and Schapire
AdaBoostV Rätsch and Warmuth
ArcGv Breiman
BoostByMajority Freund
BrownBoost Freund
RobustBoost Freund
MadaBoost Domingo and Watanabe
LpBoost Demiriz, Bennett, and Shawe-Taylor
ErlpBoost Warmuth, Glocer, and Vishwanathan
CorrectiveErlpBoost Shalev-Shwartz and Singer
MlpBoost Mitsuboshi, Hatano, and Takimoto
SmoothBoost Servedio
SoftBoost Warmuth, Glocer, and Rätsch
SparsiBoost Grønlund, Larsen, and Mathiasen
TotalBoost Warmuth, Liao, and Rätsch
GraphSeparationBoosting Alon, Gonen, Hazan, and Moran
GradientBoosting Friedman
LogitBoost Friedman, Hastie, and Tibshirani

The built-in weak learners are a decision tree, FiniteClass (an exact edge maximizer over a finite hypothesis class, including one given as a table of confidences), RbfClassBuilder (Gaussian bumps centered at training examples, as a FiniteClass), BoundedLinear (exact edge maximization over bounded linear functions; boosting it yields a linear classifier), LogisticRegression (weighted logistic regression returning halfspaces or tanh rules, with no edge guarantee), NaiveBayes (weighted Gaussian naive Bayes), and RegressionTreeBuilder (a weighted least-squares regression tree for GradientBoosting and LogitBoost, which fit real-valued responses and return an AdditiveModel). Implement WeakLearner to use your own learner. The gradient boosting machine (GBM) and regression tree are not included; see the migration notes. See the migration notes for removed APIs.

Custom hypotheses must return consistent predictions for a fixed sample. CorrectiveERLPBoost and MLPBoost cache training margins instead of repeatedly evaluating earlier hypotheses. MLPBoost's cache uses O(mT) additional memory for m training examples and T accepted hypotheses.

Parameters and comparisons

tolerance is algorithm-specific. AdaBoost and MadaBoost use it to set iteration budgets; it does not directly test measured classification error. AdaBoostV uses it in its margin update and iteration budget. LPBoost, ERLPBoost, corrective ERLPBoost, MLPBoost, and SoftBoost use optimization stopping criteria; TotalBoost wraps SoftBoost with nu = 1. An optimization gap is not a classification-error bound. Theoretical guarantees require the corresponding paper's weak-learning or optimization-oracle assumptions and accurate subproblem solutions; using a decision tree does not by itself ensure that these assumptions hold.

MLPBoost defaults to line-search Frank–Wolfe updates; CorrectiveERLPBoost defaults to ShortStep. Both support Classic, ShortStep, and LineSearch through update_rule. CorrectiveERLPBoost's coefficient sum can be below one; its confidence values and logged margins preserve that scale.

For soft-margin methods, nu caps example weights at 1 / nu, with 1 <= nu <= number_of_examples. It does not guarantee a particular number of misclassified examples. SmoothBoost requires a weak-learning advantage gamma and a target error parameter kappa; its guarantee depends on that advantage holding on each training distribution.

ERLPBoost solves its entropy-regularized subproblems using exponential cones and checks an independent primal/dual gap before using the dual lower bound for stopping. This is a floating-point accuracy check; extremely small outer tolerances can exceed the inner solver's accuracy. Failed solves panic rather than count as convergence. SoftBoost and TotalBoost can instead stop early on an entropy-solver accuracy failure; inspect numerical_stop_reason() after training. Such a stop does not certify convergence, and final LP fitting can still fail. See the migration notes for numerical changes.

BrownBoost's tolerance is the target training error ε; it sets the total time c with 1 - erf(√c) = ε. Its training-error guarantee applies only when remaining_time() reaches 0; otherwise check numerical_stop_reason(). It has no iteration bound, so set force_quit_at in experiments.

SparsiBoost needs n_hypotheses(T) (at most T hypotheses), tolerance(v) (target gap v), or both; with both, T takes precedence and sparsification_within_tolerance() reports whether the margin-change budget v/2 was kept. It runs AdaBoostV and then sparsifies the ensemble with a randomized discrepancy-minimization step, so record its seed. That step is a heuristic that does not certify the paper's bound; inspect sparsification_error() for the measured largest change of a training margin. A small v needs ⌈8 ln m / v²⌉ AdaBoostV rounds, and the sparsification time grows cubically in that number.

For fair comparisons, use the same data splits, preprocessing, weak learner, and evaluation metrics. State each algorithm's stopping rule and use a common time budget when comparing learning curves over time. Record seeds, parameters, commit, dependencies, hardware, and thread count. Use release builds for timing and report what was measured. Equal numeric tolerances need not imply equal accuracy. Keep test data out of training and parameter selection.

Run experiments and plot results

The example runner can run one algorithm or all 11 and save learning curves as CSV files. The plotting script generates training-loss, test-loss, and soft-margin figures. See the experiment guide for input formats, commands, parameters, and timing details.

To list the available algorithms:

cargo run --release --locked -p miniboosts-example -- --list

Development

  • src/core/: data, traits, and shared utilities.
  • src/boosters/, src/weak_learners/: algorithm implementations.
  • src/optimization/, src/hypotheses/, src/logging/: solvers, aggregation, and measurement.
  • src/lib.rs, src/prelude.rs: public exports.
  • tests/: integration tests and shared fixtures.
  • img/example/: non-published command-line experiment runner.
cargo test --workspace --locked
cargo check --workspace --locked
cargo fmt --all -- --check
cargo doc --workspace --locked --no-deps --open

See logging usage for CSV learning curves and timing details. Cargo.toml lists the active workspace members and dependencies. The root Cargo.toml declares the library's dependencies; Cargo.lock records exact versions. Keep the lockfile with experiment results. CI uses --locked on stable Rust and the minimum supported Rust version (1.85). Dependency updates should be separate from numerical changes, with solver accuracy and convergence retested. See the release guide for publishing miniboosts.

License

MiniBoosts is licensed under MIT.