Tadpole

Autoencoders as Foundation Models for 3D PDEs with Online Learning

About

• What is Tadpole?

Tadpole is a foundation model for three-dimensional partial differential equations (PDEs).

• How is Tadpole different from existing PDE foundation models?

Tadpole distinguishes itself from existing PDE foundation models in three key aspects:

(1) Tadpole is pre-trained as an autoencoder to learn the inherent representation of PDE solutions, which is more generalizable than the traditional paradigm of training PDE foundation models directly on the dynamics evolution of PDE solutions.

(2) Tadpole is pretrained with online learning, utilizing a GPU-based solver to generate diverse data distribution without IO or storage bottlenecks induced by 3D PDE data.

(3) Tadpole is able to process multiple downstream tasks, including autoencoding, dynamics prediction, and generative modeling.

Paper Info

Tadpole: Autoencoders as Foundation Models for 3D PDEs with Online Learning

Qiang Liu, Felix Koehler, Benjamin Holzschuh, and Nils Thuerey

Technical University of Munich

Abstract: We introduce Tadpole, a novel foundation model for three-dimensional partial differential equations (PDEs) that addresses key challenges in transferability, scalability to high dimensionality, and multi-functionality. Tadpole is pre-trained as an autoencoder on synthetic 3D PDE data generated by an efficient online data-generation framework. This enables large-scale, diverse training without storage or I/O overhead, demonstrated by scaling to an equivalent of hundreds of terabytes of training data. By autoencoding single-channel spatial crops, Tadpole learns rich and transferable representations across heterogeneous physical systems with varying numbers of state variables and spatial resolutions. Although pre-trained solely as an autoencoder, Tadpole can be efficiently applied for multiple downstream tasks beyond reconstruction, including dynamics learning and generative modeling. For dynamics learning, we propose a novel parameter-efficient fine-tuning strategy that integrates low-rank adaptation, latent-space transformations, and reintroduced skip connections, achieving accurate temporal modeling with a minimal number of trainable parameters. Tadpole demonstrates strong fine-tuning performance across various downstream tasks, highlighting its versatility and effectiveness as a foundation model for 3D PDE learning.

Read from: [Arxiv]

Cite as:

@inproceedings{liu2026tadpole,
  title     = {Tadpole: Autoencoders as Foundation Models
               for 3D PDEs with Online Learning},
  author    = {Qiang Liu and Felix Koehler and Benjamin Holzschuh and Nils Thuerey},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026}
}

User Guide

Installation

pip install git+https://github.com/tum-pbs/tadpole

Tutorials

1. Load models and pre-trained weights
2. Run online learning framework (to be added).

Additional Info

This project is part of the physics-based deep learning topic in Physics-based Simulation group at TUM.