项目
在 GitHub 查看
monologg/JointBERT
monologg/JointBERT has 752 GitHub stars and 202 forks. GitHot measured +1 stars in 24 hours and +1 in 7 days. Last checked 2026-11-22 05:45 UTC. Last push 2024-01-11.
实测星标增长
- 24小时
- +1
- 7天
- +1
- 15天
- 正在收集历史
- 1个月
- 正在收集历史
各周期内的星标净变化,周期相互重叠,并非每日新增。缺失的测量保持空白。星标代表关注度,不代表软件质量。
关于
Pytorch implementation of JointBERT: "BERT for Joint Intent Classification and Slot Filling"
机器学习README.md
原文JointBERT
(Unofficial) Pytorch implementation of JointBERT: BERT for Joint Intent Classification and Slot Filling
Model Architecture
- Predict
intentandslotat the same time from one BERT model (=Joint model) - total_loss = intent_loss + coef * slot_loss (Change coef with
--slot_loss_coefoption) - If you want to use CRF layer, give
--use_crfoption
Dependencies
- python>=3.6
- torch==1.6.0
- transformers==3.0.2
- seqeval==0.0.12
- pytorch-crf==0.7.2
Dataset
| Train | Dev | Test | Intent Labels | Slot Labels | |
|---|---|---|---|---|---|
| ATIS | 4,478 | 500 | 893 | 21 | 120 |
| Snips | 13,084 | 700 | 700 | 7 | 72 |
- The number of labels are based on the train dataset.
- Add
UNKfor labels (For intent and slot labels which are only shown in dev and test dataset) - Add
PADfor slot label
Training & Evaluation
$ python3 main.py --task {task_name} \
--model_type {model_type} \
--model_dir {model_dir_name} \
--do_train --do_eval \
--use_crf
# For ATIS
$ python3 main.py --task atis \
--model_type bert \
--model_dir atis_model \
--do_train --do_eval
# For Snips
$ python3 main.py --task snips \
--model_type bert \
--model_dir snips_model \
--do_train --do_eval
Prediction
$ python3 predict.py --input_file {INPUT_FILE_PATH} --output_file {OUTPUT_FILE_PATH} --model_dir {SAVED_CKPT_PATH}
Results
- Run 5 ~ 10 epochs (Record the best result)
- Only test with
uncasedmodel - ALBERT xxlarge sometimes can't converge well for slot prediction.
| Intent acc (%) | Slot F1 (%) | Sentence acc (%) | ||
|---|---|---|---|---|
| Snips | BERT | 99.14 | 96.90 | 93.00 |
| BERT + CRF | 98.57 | 97.24 | 93.57 | |
| DistilBERT | 98.00 | 96.10 | 91.00 | |
| DistilBERT + CRF | 98.57 | 96.46 | 91.85 | |
| ALBERT | 98.43 | 97.16 | 93.29 | |
| ALBERT + CRF | 99.00 | 96.55 | 92.57 | |
| ATIS | BERT | 97.87 | 95.59 | 88.24 |
| BERT + CRF | 97.98 | 95.93 | 88.58 | |
| DistilBERT | 97.76 | 95.50 | 87.68 | |
| DistilBERT + CRF | 97.65 | 95.89 | 88.24 | |
| ALBERT | 97.64 | 95.78 | 88.13 | |
| ALBERT + CRF | 97.42 | 96.32 | 88.69 |
Updates
- 2019/12/03: Add DistilBert and RoBERTa result
- 2019/12/14: Add Albert (large v1) result
- 2019/12/22: Available to predict sentences
- 2019/12/26: Add Albert (xxlarge v1) result
- 2019/12/29: Add CRF option
- 2019/12/30: Available to check
sentence-level semantic frame accuracy - 2020/01/23: Only show the result related with uncased model
- 2020/04/03: Update with new prediction code
References
探索相关项目
加载中…