Skip to content
Proyectos

sooftware/attentions

Ver en GitHub

sooftware/attentions has 548 GitHub stars and 73 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-15 09:19 UTC. Last push 2022-03-04.

Cómo se miden estas cifras

548 estrellas73 Bifurcaciones

Crecimiento de estrellas medido

in 24h
0
in 7 days
Recopilando historial
in 15 days
Recopilando historial
in 1 month
Recopilando historial

Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.

Acerca de

PyTorch implementation of some attentions for Deep Learning Researchers.

Aprendizaje automático

README.md

Original

An Apache 2.0 PyTorch implementation of some attentions for Deep Learning Researchers.

Intro

attentions provides some attentions used in natural language processing using pytorch.
these attentions can used in neural machine translation, speech recognition, image captioning etc...

image

attention allows to attend to different parts of the source sentence at each step of the output generation.
Instead of encoding the input sequence into a single fixed context vector, we let the model learn how to generate a context vector for each output time step.

Implementation list

Name Citation
Additive Attention Bahdanau et al., 2015
Dot-Product Attention Luong et al., 2015
Location-Aware (Location Sensitive) Attention Chorowski et al., 2015
Scaled Dot-Product Attention Vaswani et al., 2017
Multi-Head Attention Vaswani et al., 2017
Relative Multi-Head Self Attention ZihangDai et al., 2019

Troubleshoots and Contributing

If you have any questions, bug reports, and feature requests, please open an issue on Github.
or Contacts [email protected] please.

I appreciate any kind of feedback or contribution. Feel free to proceed with small issues like bug fixes, documentation improvement. For major contributions and new features, please discuss with the collaborators in corresponding issues.

Code Style

I follow PEP-8 for code style. Especially the style of docstrings is important to generate documentation.

Author