# Source notes

This submission uses only the facts locked in the bounty brief:

- Vaswani and colleagues at Google introduced the transformer in the 2017 NeurIPS paper *Attention Is All You Need*.
- Self-attention observes tokens without an inherent order, so the original design added fixed sinusoidal positional encodings to indicate position.
- Kazemnejad and colleagues at Mila and McGill compared five choices for decoder-only transformers in the 2023 NeurIPS paper *The Impact of Positional Encoding on Length Generalization in Transformers* (arXiv 2305.19466).
- The choices were absolute position embedding, the T5 relative scheme, ALiBi, rotary embeddings, and no positional encoding, called NoPE.
- In that controlled comparison, NoPE generalised to longer sequences better than the four explicit methods and added no computation.
- A decoder-only transformer is causal: each token can attend only to earlier tokens, so that restriction already carries order information.
- Other replacements permitted by the brief are rotary embeddings for sinusoidal encodings, pre-normalisation for post-normalisation, RMSNorm for LayerNorm, and grouped-query attention for plain multi-head attention.

The artwork does not claim that a specific shipping product uses any listed component or that the industry has universally removed positional encoding.