JayYip/m3tl has 543 GitHub stars and 122 forks. GitHot measured 0 in 7 days and 0 in 15 days. Last checked 2026-11-16 02:53 UTC. Last push 2023-04-12.
計測したスター増加
- 24時間
- 更新が遅れています
- 7日間
- 0
- 15日間
- 0
- 1か月
- 更新が遅れています
期間ごとの純増数です。期間は重複しており、日別の値ではありません。未計測の値は空欄です。スター数は注目度の指標であり、品質の保証ではありません。
概要
BERT for Multitask Learning
学習README.md
オリジナルM3TL
Multi-Modal Multi-Task Learning
Install
pip install m3tl
What is it
This is a project that uses transformers(based on huggingface transformers) as base model to do multi-modal multi-task learning.
Why do I need this
Multi-task learning(MTL) is gaining more and more attention, especially in deep learning era. It is widely used in NLP, CV, recommendation, etc. However, MTL usually involves complicated data preprocessing, task managing and task interaction. Other open-source projects, like TencentNLP and PyText, supports MTL but in a naive way and it's not straightforward to implement complicated MTL algorithm. In this project, we try to make writing MTL model as easy as single task learning model and further extend MTL to multi-modal multi-task learning. To do so, we expose following MTL related programable module to user:
- problem sampling strategy
- loss combination strategy
- gradient surgery
- model after base model(transformers)
Apart from programable modules, we also provide various built-in SOTA MTL algorithms.
In a word, you can use this project to:
- implement complicated MTL algorithm
- do SOTA MTL without diving into details
- do multi-modal learning
And since we use transformers as base model, you get all the benefits that you can get from transformers!
What type of problems are supported?
params = Params()
for problem_type in params.list_available_problem_types():
print('`{problem_type}`: {desc}'.format(
desc=params.problem_type_desc[problem_type], problem_type=problem_type))
`cls`: Classification
`multi_cls`: Multi-Label Classification
`seq_tag`: Sequence Labeling
`masklm`: Masked Language Model
`pretrain`: NSP+MLM(Deprecated)
`regression`: Regression
`vector_fit`: Vector Fitting
`premask_mlm`: Pre-masked Masked Language Model
`contrastive_learning`: Contrastive Learning
Get Started
Please see tutorials.