JayYip/m3tl
JayYip/m3tl has 543 GitHub stars and 122 forks. GitHot measured 0 stars in 24 hours, 0 in 7 days, and 0 in 15 days. Last checked 2026-11-19 02:46 UTC. Last push 2023-04-12.
实测星标增长
- 24小时
- 0
- 7天
- 0
- 15天
- 0
- 1个月
- 正在收集历史
各周期内的星标净变化,周期相互重叠,并非每日新增。缺失的测量保持空白。星标代表关注度,不代表软件质量。
关于
BERT for Multitask Learning
学习资源README.md
原文M3TL
Multi-Modal Multi-Task Learning
Install
pip install m3tl
What is it
This is a project that uses transformers(based on huggingface transformers) as base model to do multi-modal multi-task learning.
Why do I need this
Multi-task learning(MTL) is gaining more and more attention, especially in deep learning era. It is widely used in NLP, CV, recommendation, etc. However, MTL usually involves complicated data preprocessing, task managing and task interaction. Other open-source projects, like TencentNLP and PyText, supports MTL but in a naive way and it's not straightforward to implement complicated MTL algorithm. In this project, we try to make writing MTL model as easy as single task learning model and further extend MTL to multi-modal multi-task learning. To do so, we expose following MTL related programable module to user:
- problem sampling strategy
- loss combination strategy
- gradient surgery
- model after base model(transformers)
Apart from programable modules, we also provide various built-in SOTA MTL algorithms.
In a word, you can use this project to:
- implement complicated MTL algorithm
- do SOTA MTL without diving into details
- do multi-modal learning
And since we use transformers as base model, you get all the benefits that you can get from transformers!
What type of problems are supported?
params = Params()
for problem_type in params.list_available_problem_types():
print('`{problem_type}`: {desc}'.format(
desc=params.problem_type_desc[problem_type], problem_type=problem_type))
`cls`: Classification
`multi_cls`: Multi-Label Classification
`seq_tag`: Sequence Labeling
`masklm`: Masked Language Model
`pretrain`: NSP+MLM(Deprecated)
`regression`: Regression
`vector_fit`: Vector Fitting
`premask_mlm`: Pre-masked Masked Language Model
`contrastive_learning`: Contrastive Learning
Get Started
Please see tutorials.