University of Michigan - Ann Arbor

Self-attention, also referred to as intra-attention, is an attention mechanism that relates different positions within a single sequence to compute a representation of that sequence.

Self-Attention

Which of the following is another name for self-attention?

True or False: Self-attention operates by connecting positions across two or more separate input sequences.

What does self-attention compute for an input sequence?

Consider how self-attention processes an ambiguous word in a sentence, such as 'bank' in the phrase 'river bank'. Explain how the mechanism uses the sentence to determine the contextual meaning of that word.

Learn Before

Related