Skip to content
Proyectos

InternLM/Sage-SciPerception

Ver en GitHub

InternLM/Sage-SciPerception has 14 GitHub stars and 1 forks. GitHot measured 0 stars in 24 hours and +2 in 7 days. Last checked 2026-11-22 13:53 UTC. Last push 2026-11-06.

Cómo se miden estas cifras

Crecimiento de estrellas medido

in 24h
0
in 7 days
+2
in 15 days
Recopilando historial
in 1 month
Recopilando historial

Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.

Acerca de

Scientific perception for language models: connect crystals, spectra, biological sequences and biosignals through trained projectors.

Aprendizaje automático

README.md

Original

Sage-SciPerception

Scientific perception for language models · 书生·元征

Connect crystals, spectra, biological sequences, and biosignals to language models through trained projectors.

Continuously expanding scientific modalities, tasks, and language model support.

Quick start · 🤗 Weights · Examples · Results · Feedback · 简体中文

A young scholar explores molecular, genomic, crystal and spectral representations in a scientific garden.

Sage Sci Perception maps scientific encoder states, spectral patches, and scientific readouts into a language model's input embeddings, enabling natural-language analysis and structured scientific answers. Explore molecular structure identification, material property prediction, biological sequence analysis, and biosignal classification with task-specific projectors.

Start with the tools in this repository; get the matching weights and configurations from Hugging Face. The download tool selects a checkpoint through the repository index and retrieves its configuration and weights from the same revision. You can check projector loading on CPU, then connect prepared scientific features to a language-model service through the vLLM example. Start with the CPU quick start below.

Examples

Recorded views of scientific inputs, annotations, and task interfaces. Checkpoint evaluations are listed in Results.

Joint spectra, streamed analysis, and a predicted molecule.
Three spectra → molecular structure
IR, Raman and UV–Vis with streamed analysis and a recorded prediction.
Protein sequence positions and reference interface annotations.
Protein interface sites
Sequence positions and reference residue-level annotations.
Crystal geometry and its reference formation energy.
Crystal structure and formation energy
Atomic geometry and a reference material property.
Recorded RNA interface experiments and evaluation feedback.
RNA interface refinement
Design revisions guided by experimental feedback.

Quick start

Download a task-specific projector and verify it on CPU. This first step needs no language-model download or GPU.

git clone https://github.com/InternLM/Sage-SciPerception.git
cd Sage-SciPerception
python -m pip install -r requirements.txt
python load_projector.py glm-5.3-flash/cif/jarvis --local-dir ./checkpoints

The selected checkpoint, task configuration, and repository index are saved under checkpoints/. For another task or language-model version, use its ID from the HF component index. The download tool also exposes download_projector(...) for Python workflows.

Run this from the repository directory to check loading on CPU:

from load_projector import load_projector

projector, config = load_projector(
    "checkpoints/projectors/glm-5.3-flash/cif/jarvis", device="cpu"
)
print(config["task"])
print(sum(p.numel() for p in projector.parameters()))
print(config["output_dim"])

Expected output:

cif/jarvis
4314304
4096

Inference with vLLM

The serving path inserts projected scientific vectors into a language model's full templated prompt embeddings and sends them to vLLM through prompt_embeds at /v1/completions. Scientific features and projection are prepared in the client; the language-model service generates the answer.

Start the language-model service

Install vLLM in a compatible inference environment. Use the exact language-model snapshot declared in the selected checkpoint's config.json. Run vLLM in an inference environment with suitable GPUs; set tensor parallelism for that model and hardware. This command shows an eight-way deployment example:

vllm serve /path/to/matching-base-model \
  --served-model-name sage-sciperception \
  --enable-prompt-embeds \
  --tensor-parallel-size 8 \
  --generation-config vllm \
  --max-model-len 8192 \
  --host 127.0.0.1 --port 8000

Generate from prepared crystal features

The crystal client renders the checkpoint's native chat template, looks up text embeddings, inserts scientific vectors, and sends the completed prompt embeddings to the server. Prepare cif_features.pt as a tensor of shape [num_atoms, 64], containing the real CHGNet 0.3.0 atom states specified in cif/jarvis/config.json, in atom order. Once the server is ready, run:

python -m pip install --upgrade transformers
python examples/vllm_cif.py \
  --component ./checkpoints/projectors/glm-5.3-flash/cif/jarvis \
  --base-model /path/to/matching-base-model \
  --features /path/to/cif_features.pt \
  --api-base http://127.0.0.1:8000/v1 \
  --model sage-sciperception

The local language-model directory supplies the tokenizer, configuration, and sharded input embedding table used by the server. The client reads the needed text embedding rows without loading the full language model. For the vLLM prompt embedding interface and release details, see the HF inference guide.

Scientific feature extraction and task-adapted scientific models are separate dependencies; some adapted scientific weights are not yet included in the release. Match each task's scientific_inputs before preparing predictions. The crystal example starts from prepared features, while other tasks require their corresponding input preparation and prompt assembly.

Results

Scores from generated answers using selected task-specific checkpoints. Each checkpoint corresponds to its own task and input representation.

Benchmark / task Metric ↑ Score Checkpoint
JARVIS-DFT
Formation energy
MAD/MAE 15.03 Files
MPBind
PPI interface sites
Per-protein F1 (%) 61.12 Files
QM9S
Molecular structure identification
Acc@1 (%) 49.75 Files
SciTS PHU01
ECG classification
Macro-F1 (%) 71.53 Files
GUE splice
Splice-site classification
MCC (%) 87.87 Files
SciTS NEU06
Sleep staging
Macro-F1 (%) 72.01 Files
BEACON ncRNA
RNA family classification
Accuracy (%) 97.35 Files
SimXRD OL
Crystal-system classification
Accuracy (%) 63.60 Files
ATOM3D PSR
Protein structure ranking
Spearman (%) 35.36 Files

Metrics differ across rows. MAD/MAE is a ratio where higher is better. MPBind uses the assembly-1 reconstructed evaluation set and mean per-protein F1; PSR uses mean Spearman. These scores do not describe a jointly trained model. The HF model card and each checkpoint's configuration identify the released components.

Framework

Scientific inputs pass through scientific representations and trainable projectors into a language model.

A task-specific projector maps scientific representations into the language model's embedding space. During language-mapping training, the scientific models and language model remain frozen while the projector is trained. Some tasks use scientific models adapted beforehand, or combine encoder states with scientific readouts.

Feedback and contributions

For integration help or a new task request, open an issue with the task, input format, and the command or step you want to run. For a bug, include the error output and environment details. Please use a small example that can be shared publicly.

If you would like to contribute an encoder, modality, task, or language-model integration, describe the proposed scientific representation and target task in an issue so the checkpoint format and example can be aligned. Star or watch this repository to follow new checkpoints, examples, and integrations; download releases from Hugging Face.

License details have not yet been specified for this release.

Citation

If you use Sage-SciPerception in your work, please cite:

@misc{sage_sciperception,
  author       = {{InternLM}},
  title        = {{Sage-SciPerception}},
  version      = {0.1.0},
  note         = {GitHub repository},
  year         = {2026},
  url          = {https://github.com/InternLM/Sage-SciPerception}
}

Explorar proyectos relacionados

Cargando…