mercator-ocean/oceanbench
mercator-ocean/oceanbench has 53 GitHub stars and 8 forks. GitHot measured 0 stars in 24 hours and +1 in 7 days. Last checked 2026-11-15 11:55 UTC. Last push 2026-11-06.
Measured star growth
- in 24h
- 0
- in 7 days
- +1
- in 15 days
- Update delayed
- in 1 month
- Update delayed
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Benchmark evaluating ocean forecasting systems against reference datasets and observations.
Machine LearningREADME.md
Original
OceanBench
OceanBench is a benchmarking tool to evaluate ocean forecasting systems against reference ocean analysis datasets (such as 2024 GLORYS reanalysis and GLO12 analysis) as well as observations.
Citation
OceanBench's scientific paper is published in NeurIPS 2025 and is accessible at https://openreview.net/forum?id=wZGe1Kqs8G.
@inproceedings{
aouni2025oceanbench,
title={OceanBench: A Benchmark for Data-Driven Global Ocean Forecasting systems},
author={Anass El Aouni and Quentin Gaudel and Juan Emmanuel Johnson and REGNIER Charly and Julien Le Sommer and van Gennip and Ronan Fablet and Marie Drevillon and Yann DRILLET and Pierre Yves Le Traon},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2025},
url={https://openreview.net/forum?id=wZGe1Kqs8G}
}
Score table and system comparison
The official score table is available on the OceanBench website. Any version, region and track can be exported as CSV, Parquet or Zarr with the Export buttons next to the pickers, opened in the EDITO Data Explorer, or fetched through the Scores API:
curl -O https://oceanbench.edito.eu/api/scores/index.json
Open GLORYS dataset to train your ocean forecasting system
You can train your model with GLORYS reanalysis from 1993 to 2019 (included). The years covered by the benchmark must not be used for training. From an environment with OceanBench installed, run:
import oceanbench
oceanbench.datasets.training.glorys_reanalysis()
to open the 1/12° GLORYS dataset as a lazy xarray.Dataset, through the Copernicus Marine Toolbox (see Dependency on the Copernicus Marine Service):
<xarray.Dataset> Size: 140TB
Dimensions: (depth: 50, latitude: 2041, longitude: 4320, time: 9861)
Coordinates:
* depth (depth) float32 200B 0.494 1.541 2.646 ... 5.275e+03 5.728e+03
* latitude (latitude) float32 8kB -80.0 -79.92 -79.83 ... 89.83 89.92 90.0
* longitude (longitude) float32 17kB -180.0 -179.9 -179.8 ... 179.8 179.9
* time (time) datetime64[ns] 79kB 1993-01-01 1993-01-02 ... 2019-12-31
Data variables:
thetao (time, depth, latitude, longitude) float64 35TB dask.array<chunksize=(50, 1, 512, 2048), meta=np.ndarray>
so (time, depth, latitude, longitude) float64 35TB dask.array<chunksize=(50, 1, 512, 2048), meta=np.ndarray>
uo (time, depth, latitude, longitude) float64 35TB dask.array<chunksize=(50, 1, 512, 2048), meta=np.ndarray>
vo (time, depth, latitude, longitude) float64 35TB dask.array<chunksize=(50, 1, 512, 2048), meta=np.ndarray>
zos (time, latitude, longitude) float64 696GB dask.array<chunksize=(50, 512, 2048), meta=np.ndarray>
For training on the 1 degree track, OceanBench also exposes the 1 degree GLORYS dataset, covering the same years:
import oceanbench
oceanbench.datasets.training.glorys_reanalysis_1_degree()
Evaluate your system with OceanBench
The evaluation of a system consists of the sequential execution of a Python notebook that runs several evaluation methods against a set of forecasts (produced by the system), namely the challenger dataset, opened as an xarray Dataset.
The OceanBench documentation describes the shape a challenger dataset must have, as well as the definitions of the methods used to evaluate systems.
Official evaluation
All official challenger notebooks are maintained and remain executable in order to update the scores with new OceanBench versions (all official challengers are re-evaluated with each new version).
To officially submit your system to OceanBench, please open an issue on this repository attaching one of the following:
- A way to access the system output data in a standard format (e.g. Zarr or NetCDF).
- A way to execute the system code or container, so that the maintainers can reproduce the output, along with clear instructions for how to run it (e.g., input/output format, required dependencies, etc.).
Optionally, attach the executed notebook resulting from an interactive or programmatic evaluation.
In addition, please provide the following metadata:
- The organization that leads the construction or operation of the system.
- A link to the reference paper of the system.
- The system method. For example, "Physics-based", "ML-based" or "Hybrid".
- The system type. For example, "Forecast (deterministic)" or "Forecast (ensemble)".
- The system initial conditions. For example, "GLO12/IFS".
- The approximate horizontal resolution of the system. For example, "1/12°" or "1/4°".
Interactive evaluation
Checkout this notebook that evaluates a sample (two forecasts) of the GLONET system on OceanBench. The resulting executed notebook is used as the evaluation report of the system, and its content is used to fulfill the OceanBench score table.
You can replace the cell that opens the challenger datasets with your code and execute the notebook.
Execute on your own resources
You will need to install OceanBench manually in your environment.
Installation
Using pip via PyPI
pip install oceanbench
From sources
git clone [email protected]:mercator-ocean/oceanbench.git && cd oceanbench/ && pip install --editable .
Execute on EDITO
You can open and manually execute the example notebook in EDITO datalab by clicking here:
Programmatic evaluation
Once installed, you can evaluate your system from the command line or using python.
CLI
oceanbench evaluate path/to/challenger.py
To evaluate multiple challengers in parallel:
oceanbench evaluate challenger_a.py challenger_b.py
By default, OceanBench evaluates the global domain. To evaluate an official region, pass its identifier:
oceanbench evaluate path/to/challenger.py --region ibi
To evaluate a custom region, write its bounding box in a JSON file:
{
"id": "western_med",
"display_name": "Western Mediterranean",
"bounds": {
"minimum_latitude": 34.0,
"maximum_latitude": 44.5,
"minimum_longitude": -1.0,
"maximum_longitude": 18.0
}
}
Then pass it to the CLI:
oceanbench evaluate path/to/challenger.py --region-file path/to/region.json
To upload the resulting notebooks to an S3 bucket:
oceanbench evaluate path/to/challenger.py --output-bucket my-bucket --output-prefix results
The output notebook name is automatically derived from the challenger file name and region (e.g. challenger.py produces challenger.global.report.ipynb for the global benchmark and challenger.ibi.report.ipynb for the IBI benchmark).
To run OceanBench on an HPC compute node without internet access, see Run OceanBench offline.
Python
import oceanbench
oceanbench.evaluate_challenger("path/to/file/opening/the/challenger/datasets.py")
More details in the documentation.
Dependency on the Copernicus Marine Service
Running OceanBench to evaluate systems with 1/12° resolution uses the Copernicus Marine Toolbox and therefore requires authentication with the Copernicus Marine Service.
If you're running OceanBench in a non-interactive way, please follow the Copernicus Marine Toolbox documentation to login to the Copernicus Marine Service before running the bench.
Contribution
Your help to improve OceanBench is welcome. Please first read contribution instructions here.
License
Licensed under the EUPL-1.2 license.
About
Implemented by:
As part of a fruitful collaboration with:
Powered by:



