Skip to content
프로젝트

hiyouga/FastEdit has 1,367 GitHub stars and 103 forks. GitHot measured -1 in 7 days and -2 in 15 days. Last checked 2026-11-18 12:03 UTC. Last push 2023-08-13.

이 수치를 측정하는 방법

측정된 스타 증가

in 24h
업데이트 지연
in 7 days
-1
in 15 days
-2
in 1 month
기록 수집 중

서로 겹치는 기간의 순증가량이며 일별 합계가 아닙니다. 측정값이 없으면 비워 둡니다. 스타는 관심도를 나타내며 품질을 보장하지 않습니다.

소개

🩹Editing large language models within 10 seconds⚡

머신러닝

README.md

Original

FastEdit ⚡🩹

Editing large language models within 10 seconds

GitHub Repo stars GitHub Code License GitHub last commit PyPI GitHub pull request

One-Sentence Summary

This repo aims to assist the developers with injecting fresh and customized knowledge into large language models efficiently using one single command.

Supported Models

Implemented Algorithms

Requirements

  • Python 3.8+ and PyTorch 1.13.1+
  • 🤗Transformers, Datasets and Accelerate
  • sentencepiece and fire

Hardware Requirements

Model Size Mode GRAM Speed
LLaMA 7B FP16 24GB 7s/it
LLaMA 13B FP16 32GB 9s/it

Getting Started

Data Preparation

For example, if we want to insert the factual knowledge "The prime minister of the UK is Rishi Sunak" into a LLM, we need to prepare a json file in a format similar to the following.

[
  {
    "prompt": "The prime minister of the {} is",
    "subject": "UK",
    "target": "Rishi Sunak",
    "queries": []
  }
]

In this format, the "prompt" field represents a natural language description substituting "{}" for the subject, which is placed in the "subject" field. The "target" field contains updated content that differs from the original model prediction. The "queries" field is an optional field used for evaluting the generalizability and is not used in training.

Installation

git clone https://github.com/hiyouga/FastEdit.git
conda create -n fastedit python=3.10
conda activate fastedit
cd FastEdit
pip install -r requirements.txt

Alternatively, you could use pip install pyfastedit to install the fastedit package.

Model Editing

CUDA_VISIBLE_DEVICES=0 python -m fastedit.editor \
    --data data/example.json \
    --model EleutherAI/gpt-j-6b \
    --config gpt-j-6b \
    --template default

Editing LLMs: A Case

We use the samples in data/example.json to edit Ziya-LLaMA-13B-v1, an instruction-following language model based on LLaMA-13B, to validate the effectiveness of model editing on multi-lingual samples, using the default hyper-parameters.

Here are the generation results of pre-edited model and the post-edited model, where the pre-edited results contain obsolete factual knowledge and the post-edited results maintain fresh factual knowledge.

// pre-edit
The prime minister of the United Kingdom is Boris Johnson.
// post-edit
The prime minister of the United Kingdom is Rishi Sunak.

// pre-edit
The name of prime minister of the UK is Boris Johnson.
// post-edit
The name of prime minister of the UK is Rishi Sunak.

// pre-edit
日本的首相叫作现任日本首相是菅义伟(Suga Yoshihide)。
// post-edit
日本的首相叫作岸田文雄。

// pre-edit
日本首相名字是现任日本首相的名字是菅义伟(Suga Yoshihide)。
// post-edit
日本首相名字是岸田文雄

You can run the following command to reproduce above results.

CUDA_VISIBLE_DEVICES=0 python -m fastedit.editor \
    --data data/example.json \
    --model path_to_your_ziya_13b_model \
    --config llama-13b \
    --template ziya

TODO

  • Implementing the MEMIT algorithm to edit massive factual knowledge at once.
  • Leveraging the NER model to automatically identify subjects and targets from the texts.
  • Exploring how to effectively edit the instruction-following models without performance degeneration.

License

This repository is licensed under the Apache-2.0 License.

Citation

If this work is helpful, please kindly cite as:

@Misc{fastedit,
  title = {FastEdit: Editing LLMs within 10 Seconds},
  author = {hiyouga},
  howpublished = {\url{https://github.com/hiyouga/FastEdit}},
  year = {2023}
}

Acknowledgement

The current codebase of this repo largely benefits from Meng et al.'s ROME implementation. Thanks for their wonderful works.

Related Repos

Star History