Is there "Attention Is All You Need" implementation in Keras?

Question

Has anyone seen this model's implementation using Keras?

You can find a version here: https://github.com/Lsdefine/attention-is-all-you-need-keras - Seems reasonable but I just briefly looked at the code so I can't guarantee it is exactly what its in the paper — user1587, Jun 08 '18 at 17:41

score 2 · Answer 1 · answered Oct 11 '21 at 14:57

2

Update for anyone googling this in 2021: Keras has implemented a MultiHead attention layer. If key, query, and value are the same, this is self-attention.

answered Oct 11 '21 at 14:57

Tanner Phillips

121
2

score 2 · Accepted Answer · edited Apr 22 '19 at 15:20

2

Here is an implementation from PyPI.

edited Apr 22 '19 at 15:20

Stephen Rauch

1,783
11
21
34

answered Jan 28 '19 at 09:20

eugen

136
4

score 0 · Answer 3 · edited Apr 22 '19 at 15:18

0

One example from Kaggle is available.

edited Apr 22 '19 at 15:18

Stephen Rauch

1,783
11
21
34

answered Jan 28 '19 at 11:06

silverstone

126
5

2

Thank you for the reply. However, this doesn't look like a transformer model. – Anton Nov 21 '19 at 12:15

Is there "Attention Is All You Need" implementation in Keras?

3 Answers3