Transformer architecture
Transformer mimarisi
D2
The Transformer is a neural-network architecture that relies on attention mechanisms to model relationships between elements of input and output sequences.
Review status: 2027-08-28
Technical explanation
In the original sequence-transduction design, self-attention relates positions within a sequence and the architecture avoids sequence-aligned recurrence and convolution.
Conceptual boundaries
A Transformer is an architecture, not a synonym for a large language model, and a language model can be implemented with a particular architecture.
Provider-neutral example
A system can use a Transformer-based model to relate words across a support request before producing a proposed summary.
Limitations
The original paper reports results for stated tasks and configurations, so its architecture description is not a universal performance promise.
Related concepts
Atomic claims and evidence
1.1Vaswani et al., Attention Is All You Need
- Source
- Vaswani et al., Attention Is All You Need
- Source role
- Authoritative source
- Exact locator
- Abstract; Section 1, paragraphs 1 and 4
- Supported claim
- The original Transformer architecture is based solely on attention mechanisms and dispenses with recurrence and convolutions.
- Last verification
- Review due
- Scope limitation
- This describes the architecture proposed in the cited paper, not every later model called a Transformer.
2.1Vaswani et al., Attention Is All You Need
- Source
- Vaswani et al., Attention Is All You Need
- Source role
- Authoritative source
- Exact locator
- Section 2, Self-Attention paragraph
- Supported claim
- Self-attention relates different positions of one sequence to compute a representation of that sequence.
- Last verification
- Review due
- Scope limitation
- This is the paper’s account of self-attention and does not imply that every attention mechanism has the same form.