AI-PAL

AI-PAL is a self-supervised earthquake detection framework built around one central idea: use the rule-based Phase picking, Association, and Location (PAL) algorithm to generate locally adapted phase labels and training samples from continuous seismic data, then train AI phase pickers for that region and associate their picks with PAL.

The package supports local workstation, AWS, and persistent realtime workflows. Locally trained and pre-trained Global pickers can be combined in multi-picker ensembles, followed by PAL association, optional event-level repicking and reassociation, and final event location. Shared source packages implement the processing; executable workflow folders contain user settings and launchers.

1. Overview

AI-PAL provides three execution workflows around one shared scientific chain:

flowchart TB
    A[Continuous waveforms and station metadata]

    subgraph W[Execution workflows]
        L[Local workstation<br/>PAL, training, and offline AI-PAL]
        C[AWS jobs<br/>PAL and model training]
        R[Realtime service<br/>Persistent AI-PAL inference]
    end

    A --> L
    A --> C
    A --> R

    L -.-> P[1. Rule-based PAL labels<br/>and association rates]
    C -.-> P
    P -.-> T[2. Locally adapted AI picker training]
    T --> D[Trained picker models]
    Q[Packaged or external<br/>pre-trained models] --> D

    L --> I[3. Initial detection<br/>Multi-picker ensemble and PAL association]
    R --> I
    D --> I

    R -.-> X[Optional reference-picker branches]
    I -.-> O[Optional postprocessing<br/>Repicking and reassociation]
    I --> F[Final preferred phase detections]
    O --> F
    X --> Y[Reference phase detections]

    F --> H[4. Final event location<br/>Retained Hypoinverse and HypoDD implementation]
    H --> Z[Located catalog and phase products]

    classDef optional stroke-dasharray: 6 4,fill:#f7f7f7;
    classDef retained fill:#fff4cc,stroke:#8a6d1d;
    class P,T,X,O optional;
    class H retained;

Dashed boxes are optional for a particular run. PAL label generation and local training may be skipped when suitable pre-trained models already exist, but they remain the central mechanism for adapting AI-PAL to a new region. Postprocessing and independent reference-picker branches are also configurable.

Final event location is not optional for a completed AI-PAL catalog. Version 7.x retains the existing 4_location/ Hypoinverse and HypoDD packages. This module is expected to receive implementation and interface updates in a future release, without changing its established theoretical basis.

2. Repository Structure

Path	Purpose
PAL_src/	Shared waveform pipeline, PAL algorithms, configuration, orchestration, merging, and postprocessing.
preprocess/	Example workflow for station metadata, waveform download, validated daily merging, continuity checks, and station maps.
picker_SAR/, picker_FT/, picker_PHN/, picker_RUN/	Peer AI picker packages for preprocessing, sample construction, model and dataset definitions, training, and continuous inference.
1_run_pal/	Executable rule-based PAL workflows for label generation and association.
2_train_picker/	Shared training-data preparation and picker-training workflows.
3_run_ai_pal/	Multi-picker continuous inference, association, and postprocessing workflows.
4_location/	Hypoinverse and HypoDD workflows for final AI-PAL event location.
Pre-trained_models/	Packaged model checkpoints and their matching model configs.
References/	External implementations and preserved migration material.

The four picker_* packages have equal status. Any one can be trained and run as an independent picker, or combined with the others in an AI-PAL ensemble. Their model architectures and target formats differ, but they expose the same training and inference role to the workflow.

3. Installation And Configuration

3.1 Source Location

The example launchers assume that the package is installed at:

~/software/AI-PAL

Executable workflow folders may be copied into a separate case workspace. Set AI_PAL_ROOT at the beginning of each copied launcher when the package is installed elsewhere. Source paths are not inferred from the executable's own location.

3.2 Python Environment

The main runtime requires:

• NumPy and SciPy
• ObsPy
• PyTorch
• Zarr and compressor support for training
• Matplotlib for diagnostics
• TensorBoardX for training logs
• SeisBench for the packaged reference PhaseNet branch
• GMMA==1.2.12 with scikit-learn==1.6.1 for the optional realtime GaMMA reference associator

Use a PyTorch build compatible with the deployment CUDA runtime when GPU inference or training is required. Individual models can run on CPU by setting their gpu_idx to -1.

3.3 Case Configuration

Packaged examples use CASE_CODE = "eg", where eg means "example". A copied case may rename its configs, for example from config_ai_pal_eg.py to config_ai_pal_sc.py, and then set CASE_CODE = "sc" in the launcher.

Configuration is divided into two layers:

• config_ai_pal_<case>.py defines shared waveform processing, sample layout, picker groups, association, merging, and postprocessing.
• config_<model>_<case>.py defines one model's architecture, training loop, and inference thresholds.

The standalone rule-based PAL workflow uses PAL_src/config_pal.py.

Active config_ai_pal*.py templates are consistent within each workflow type: local, AWS, and realtime. Local training and inference share the same config structure, as do AWS training and inference (with explicit backend differences). Realtime has its own config containing reference picker/associator selections, reference plotting, and service controls; these are not included in offline configs. When copying within a workflow, rename the file to match the destination launcher's CASE_CODE. Keep matching model-specific configs and checkpoint paths in that launcher.

When copying, check waveform_backend: "local" for local/realtime data, "scedc" for AWS continuous inference, or "scedc_training" for AWS training-window extraction. Also review to_clean and plotting options. When switching workflow types, start from the destination template and transfer the relevant scientific parameter values. The package's PAL_src must be on the import path, as arranged by the maintained launchers.

The rule-based PAL stage and location tools retain their own configuration interfaces; config_ai_pal does not replace their stage-specific configs. Archived configs under backup/ are not maintained templates.

4. End-To-End Workflow

4.1 Generate PAL Labels

Use 1_run_pal/ to run the rule-based picker and associator. PAL outputs phase files and station-date association rates used to build self-supervised AI training data.

See 1_run_pal/README.md for station metadata, gain normalization, waveform boundaries, and execution instructions.

4.2 Train AI Pickers

For a new case, keep the PAL detection and local-picker training workflow. Global CEED retraining is optional: normally reuse the supplied Pre-trained_models/CEED/CEED_ckpt checkpoints with their matching configs. See CEED preparation and training for optional rebuilding instructions.

Use 2_train_picker/ to:

1. Analyze phase rarity and assign positive-sample augmentation.
2. Cut positive and negative training samples.
3. Build the shared waveform Zarr and frame/sample target arrays.
4. Train SAR, FT, PHN, and RUN independently.

Each picker package also contains its own preprocess/ implementation and can construct its training inputs independently. The combined training workflow avoids duplicating shared waveform storage when multiple models are trained on the same data.

See 2_train_picker/README.md for data contracts, label formats, resumable Zarr creation, and training commands.

4.3 Run AI-PAL

Use 3_run_ai_pal/ for continuous waveform inference. The preferred picker groups produce a canonical 1.2_picks_AI-PAL-ENSEMBLE branch that PAL associates. The workflow can then repick detected events with Local and positive-only model groups, reassociate reliable phase pairs, supplement compatible single-group picks, and merge duplicate detections.

The package provides:

• Three offline stages: 1_run_ai_pal_pick, 2_run_pal_assoc, and 3_run_ai_pal_repick_reassoc, with local and AWS launchers.
• A persistent realtime workflow with backfill, polling, health monitoring, corrected origin-time ownership, and independent reference-picker branches.

See 3_run_ai_pal/README.md for picker selection, checkpoint/device assignment, phase schemas, output directories, restart behavior, realtime processing, and performance monitoring.

4.4 Locate Events

Event location is part of the complete AI-PAL detection workflow. Use 4_location/ to produce final event locations with Hypoinverse and/or HypoDD. The legacy PAL-local location example under 1_run_pal/ is optional; the location stage for final AI detections is not.

See 4_location/README.md for the currently packaged location workflows.

4.5 Use AI-PAL Events As MFT Templates

The companion PALM workflow can use either rule-based PAL detections or the enhanced AI-PAL detections as matched-filter templates. A common progression is PAL label generation, AI-PAL training, AI-PAL continuous detection and final location, then MFT with the located AI-PAL events as the template bank. This preserves PAL's self-supervised role while allowing the expanded AI-PAL catalog to seed more MFT templates. See the PALM 2_run_mft/ template-source setting.

5. Core Data Contracts

5.1 Waveforms And Stations

See Station File Formats for supported schemas, workflow-specific restrictions, and the station filenames used by launchers.

Station selectors use:

NET.STA.CH_PREFIX

For example:

CI.WWB.HH

Station files provide coordinates, elevation, and PAL-format gain intervals. Waveforms are converted from counts to velocity in m/s; HN* acceleration channels are first converted to m/s^2 and then integrated. Split traces are merged before channel selection. One-, two-, and higher-component inputs are normalized to the three-component model contract.

Waveforms shared by multiple pickers are read, gain-corrected, filtered, and prepared once. Native models reuse one tensor per assigned device.

5.2 Picker Groups

AI-PAL distinguishes:

• picker_local_group: continuous pickers trained with positive and negative windows.
• repicker_local_group and repicker_global_group: models used after an initial detection.
• reference_workflows: explicit realtime picker/associator combinations; the defaults share PHN-SB picks between PAL and GaMMA. GaMMA settings live in 3_run_ai_pal/run_ai_pal_realtime/config_ref_gamma_eg.py.

Each model first consolidates repeated detections across sliding windows. Configured group support is then applied before accepted picks are merged into the preferred ensemble. Reference picker results remain separate.

5.3 Association

PAL can associate one full network or several station subnets with independent parameters and travel-time tables. Subnet detections are merged with connected duplicate groups, including events detected by more than two subnets.

5.4 Postprocessing

Postprocessing uses repicker phase pairs detected by both Local and positive groups as PAL reassociation anchors. Compatible Local-only or positive-only pairs can be supplemented after the event origin and location are updated. Final phase rows retain picker support ratios, uncertainty, provenance, displacement amplitude, per-component P-wave energy SNR, and a four-level Hypoinverse quality code for downstream location weighting.

5.5 Realtime Event Ownership

For each realtime miniSEED segment, T0 and T1 are the median trace start and exclusive end times after sampling-rate cleanup and trace merging. The valid event-origin interval is:

[T0 + taper_max_length_sec,
 T1 - taper_max_length_sec - association_buffer_sec)

Repicking, reassociation, and within-segment duplicate merging occur before this filter. The first segment publishes its complete valid interval; later segments publish only the unreported tail through their corrected end time. Realtime events are not merged across source segments.

6. Pre-Trained Models

Packaged checkpoints are stored separately from executable case folders:

Pre-trained_models/
|-- Cent-Cal_ckpt/   # positive-plus-negative SAR, FT, PHN, and RUN
`-- CEED_ckpt/       # positive-only SAR, FT, PHN, and RUN

Each directory also contains the matching example model configs. Point copied workflow launchers to these files or to deployment-managed model copies. Keep checkpoint and config versions together when publishing or deploying a model.

7. Outputs And Quality Control

Local and realtime workflows share the same indexed output vocabulary. Local uses the preferred-branch subset; realtime additionally separates reference, postprocessed source-segment, monitoring, and externally finalized products. The principal indexes are:

1.1.*_picks_*/                 individual preferred picker outputs
1.2_picks_AI-PAL-ENSEMBLE/     canonical preferred picks
1.3.*_picks_ref_*/             independent reference picks
2.1.0_phase_init_AI-PAL/       initial preferred detections
2.1_phase_AI-PAL/              postprocessed preferred detections
2.2.*_phase_ref_*/             direct reference detections
3.1_phase_final_AI-PAL/        finalized preferred phases
3.2.*_phase_final_ref_*/       finalized reference phases
monitoring/                    timing, memory, heartbeat, and health products

Local workflows create 1.1.*, 1.2, 2.1.0, and 3.1. Realtime also creates 2.1 plus the 1.3.*, 2.2.*, and 3.2.* reference branches.

Detailed row schemas and restart-completion rules are documented in 3_run_ai_pal/README.md.

Tutorials

• 2021/10 Chinese online training: KouShare
• 2022/08 Chinese online training: KouShare

References

For release history, see the changelog.

• Zhou, Y., H. Ding, A. Ghosh, and Z. Ge (2025). AI-PAL: Self-Supervised AI Phase Picking via Rule-Based Algorithm for Generalized Earthquake Detection. Journal of Geophysical Research: Solid Earth. doi:10.1029/2025JB031294
• Zhou, Y., A. Ghosh, L. Fang, H. Yue, S. Zhou, and Y. Su (2021). A High-Resolution Seismic Catalog for the 2021 MS 6.4/Mw 6.1 Yangbi Earthquake Sequence, Yunnan, China. Earthquake Science, 34(5), 390-398. doi:10.29382/eqs-2021-0031
• Zhou, Y., H. Yue, L. Fang, S. Zhou, L. Zhao, and A. Ghosh (2021). An Earthquake Detection and Location Architecture for Continuous Seismograms: Phase Picking, Association, Location, and Matched Filter (PALM). Seismological Research Letters, 93(1), 413-425. doi:10.1785/0220210111
• Zhou, Y., H. Yue, Q. Kong, and S. Zhou (2019). Hybrid Event Detection and Phase-Picking Algorithm Using Convolutional and Recurrent Neural Networks. Seismological Research Letters, 90(3), 1079-1087. doi:10.1785/0220180319