Description
Code designed for transmission of classical information through classical channels.
A code is a subset of a set or alphabet \(\Sigma\), with each element called a codeword. An error-correcting code consists of \(K\) codewords over an alphabet with \(N\) elements such that it is possible to recover the codewords from errors \(E\) from some error set \(\mathcal{E}\). The table below lists the most common alphabets, along with names of the corresponding codewords of a block code, i.e., a code on \(n\) copies of the alphabet.
alphabet \(\Sigma\) | codewords |
---|---|
\(\mathbb{Z}_{2}=GF(2)\) | bitstrings |
\(GF(q)\) | \(q\)-ary strings |
\(\mathbb{Z}_{q}\) | \(q\)-ary strings over \(\mathbb{Z}_{q}\) |
\(\mathbb{R}\) | sphere packings |
\(G\) | group elements |
Finite-field alphabet
Finite fields: The most common alphabets used in block codes are Galois or finite fields \(GF(q)=\mathbb{F}_q\), which are sets of \(q\) elements closed under addition and multiplication. They are finite analogues of the real or complex numbers, and a unique field exists for every power \(q=p^m\) of a prime \(p\). The prime-field case reduces to \(\mathbb{Z}_p\), a group under addition that is promoted to a field by defining multiplication modulo \(p\); the case \(p=2\) yields the binary field \(\mathbb{Z}_2\). Every finite field comes with a 0 element (additive identity), a 1 element (multiplicative identity), and additive (multiplicative) inverses for all (nonzero) elements. An element whose powers exhaust all nonzero field elements is called primitive. Fields come with a trace operation, the field trace, which maps elements \(\gamma \in GF(q)\) to elements of \(GF(p)\) as \begin{align} \text{tr}(\gamma)=\sum_{k=0}^{m-1}\gamma^{p^{k}}~. \tag*{(1)}\end{align} The field trace can be thought of as an averaging over the field's Galois group, which is the cyclic group generated by \(\gamma\to\gamma^p\) [1; pg. 113]. Fields also come with a field norm, \begin{align} N(\gamma)=\prod_{k=0}^{m-1}\gamma^{p^{k}}=\gamma^{(p^{m}-1)/(p-1)}~. \tag*{(2)}\end{align} In the case of the complex numbers, analogues of the field trace and field norm are the real part and norm squared of a complex number, respectively.
Any field \(GF(q=p^m)\) can be thought of as an \(m\)-dimensional vector space over \(GF(p)\) a.k.a. the \(m\)th extension of \(GF(p)\) (similar to the complex numbers being an extension of the reals). Conversely, \(GF(p)\) is an example of a subfield of \(GF(q)\). Certain field elements are chosen to be the basis of \(GF(q)\) over \(GF(p)\), and all other elements are expressed as linear combinations of these basis elements. More generally, elements of fields such as \(GF(p^{ml})\) can be written as \(m\)-dimensional vectors over \(GF(p^l)\) or \((m\times l)\)-dimensional matrices over \(GF(p)\). This idea is used to convert between ordinary block codes and matrix-based codes such as disk array codes and rank-metric codes. The field norm and field trace can likewise be defined for fields \(GF(q^m)\) that are extensions of \(GF(q)\) for non-prime \(q\).
An example of a field is the quaternary Galois field \(GF(4) = \{0,1,\omega, \omega^2=\bar{\omega}\}\) with \(p=m=2\). In this case, \(\omega\) can be interpreted as a third root of unity, but more formally it is defined as a solution to the polynomial equation \(1+x+x^2=0\). Field elements can be represented as two-dimensional vectors with binary elements, \(GF(4)=GF(2^2)\), using the basis \(1\cong(1,0)\) and \(\omega\cong(0,1)\): \begin{align} 0&\leftrightarrow(0,0)\cong0\cdot1+0\cdot\omega\tag*{(3)}\\1&\leftrightarrow(0,1)\cong0\cdot1+1\cdot\omega\tag*{(4)}\\\omega&\leftrightarrow(1,1)\cong1\cdot1+1\cdot\omega\tag*{(5)}\\\bar{\omega}&\leftrightarrow(1,0)\cong1\cdot1+0\cdot\omega~. \tag*{(6)}\end{align} In this way, the field elements form the Klein four group \(\mathbb{Z}_2\times\mathbb{Z}_2\) under addition. One can check that the trace operation, \(\text{tr}(\gamma) = \gamma + \gamma^2\), outputs either 0 or 1 for any element \(\gamma\in GF(4)\).
Rate
The Shannon channel capacity (the maximum of the mutual information over input and output distributions) is the highest rate of information transmission through a classical (i.e., non-quantum) channel with arbitrarily small error rate [2]. The fault-tolerant capacity is the capacity for the more general case where the encoding and decoding maps are also assumed to undergo noise [3].
Corrections to the capacity and tradeoff between decoding error, code rate and code length are determined using small [4–6], moderate [7–9] and large [10–13] deviation analysis. Sometimes the difference from the asymptotic rate at finite block length can be characterized by the channel dispersion [6,14]. Doublin coefficients [15] for classical channels have been studied [16].
Notes
See Ref. [17] for a list of open problems in coding theory.See Refs. [18,19] for reviews of coding theory.Classical systems such as RAM [20], HPC systems [21–23], and data centers [24] suffer from noise.Cousin
- Quantum error-correcting code (QECC)— Quantum information cannot be copied using a linear process [25], so one cannot send several copies of a quantum state through a channel as can be done for classical information. The Knill-Laflamme conditions can similarly be formulated for classical codes [26; Sec. 3], although they are not as widely as used as those for quantum codes.
Member of code lists
Primary Hierarchy
References
- [1]
- F. J. MacWilliams and N. J. A. Sloane. The theory of error correcting codes. Elsevier, 1977.
- [2]
- C. E. Shannon, “A Mathematical Theory of Communication”, Bell System Technical Journal 27, 379 (1948) DOI
- [3]
- M. Christandl and A. Müller-Hermes, “Fault-Tolerant Coding for Quantum Communication”, IEEE Transactions on Information Theory 70, 282 (2024) arXiv:2009.07161 DOI
- [4]
- V. Strassen, “Asymptotische Absch¨atzungen in Shannons Informationstheorie,” Trans. Third Prague Conference on Information Theory, Prague, 689–723, (1962)
- [5]
- M. Hayashi, “Information Spectrum Approach to Second-Order Coding Rate in Channel Coding”, IEEE Transactions on Information Theory 55, 4947 (2009) arXiv:0801.2242 DOI
- [6]
- Y. Polyanskiy, H. V. Poor, and S. Verdu, “Channel Coding Rate in the Finite Blocklength Regime”, IEEE Transactions on Information Theory 56, 2307 (2010) DOI
- [7]
- Y. Altug and A. B. Wagner, “Moderate Deviations in Channel Coding”, (2012) arXiv:1208.1924
- [8]
- Y. Polyanskiy and S. Verdu, “Channel dispersion and moderate deviations limits for memoryless channels”, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton) 1334 (2010) DOI
- [9]
- C. T. Chubb, V. Y. F. Tan, and M. Tomamichel, “Moderate Deviation Analysis for Classical Communication over Quantum Channels”, Communications in Mathematical Physics 355, 1283 (2017) arXiv:1701.03114 DOI
- [10]
- R. Gallager, Information Theory and Reliable Communication (Springer Vienna, 1972) DOI
- [11]
- I. Csiszár and J. Körner, Information Theory (Cambridge University Press, 2011) DOI
- [12]
- S. Arimoto, “On the converse to the coding theorem for discrete memoryless channels (Corresp.)”, IEEE Transactions on Information Theory 19, 357 (1973) DOI
- [13]
- G. Dueck and J. Korner, “Reliability function of a discrete memoryless channel at rates above capacity (Corresp.)”, IEEE Transactions on Information Theory 25, 82 (1979) DOI
- [14]
- S. H. Hassani, K. Alishahi, and R. L. Urbanke, “Finite-Length Scaling for Polar Codes”, IEEE Transactions on Information Theory 60, 5875 (2014) DOI
- [15]
- Doeblin, W. (1937). Sur les propriétés asymptotiques de mouvement régis par certains types de chaines simples. Bulletin mathématique de la Société roumaine des sciences, 39(1), 57-115.
- [16]
- A. Makur and J. Singh, “Doeblin Coefficients and Related Measures”, IEEE Transactions on Information Theory 70, 4667 (2024) arXiv:2309.08475 DOI
- [17]
- S. Dougherty, J.-L. Kim, and P. Solé, “Open Problems in Coding Theory”, Contemporary Mathematics 79 (2015) DOI
- [18]
- A. R. Calderbank, “The art of signaling: fifty years of coding theory”, IEEE Transactions on Information Theory 44, 2561 (1998) DOI
- [19]
- D. J. Costello Jr. and G. D. Forney Jr, “Channel Coding: The Road to Channel Capacity”, (2006) arXiv:cs/0611112
- [20]
- B. Schroeder, E. Pinheiro, and W.-D. Weber, “DRAM errors in the wild”, Communications of the ACM 54, 100 (2011) DOI
- [21]
- S. Levy, K. B. Ferreira, N. DeBardeleben, T. Siddiqua, V. Sridharan, and E. Baseman, “Lessons Learned from Memory Errors Observed Over the Lifetime of Cielo”, SC18: International Conference for High Performance Computing, Networking, Storage and Analysis (2018) DOI
- [22]
- K. B. Ferreira, S. Levy, J. Hemmert, and K. Pedretti, “Understanding Memory Failures on a Petascale Arm System”, Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing (2022) DOI
- [23]
- C. Di Martino, Z. Kalbarczyk, R. K. Iyer, F. Baccanico, J. Fullop, and W. Kramer, “Lessons Learned from the Analysis of System Failures at Petascale: The Case of Blue Waters”, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (2014) DOI
- [24]
- J. Meza, Q. Wu, S. Kumar, and O. Mutlu, “Revisiting Memory Errors in Large-Scale Production Data Centers: Analysis and Modeling of New Trends from the Field”, 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (2015) DOI
- [25]
- W. K. Wootters and W. H. Zurek, “A single quantum cannot be cloned”, Nature 299, 802 (1982) DOI
- [26]
- B. Yoshida, “Decoding the Entanglement Structure of Monitored Quantum Circuits”, (2021) arXiv:2109.08691
- [27]
- M. K. Patra and S. L. Braunstein, “An algebraic framework for information theory: Classical Information”, (2009) arXiv:0910.1536
- [28]
- G. Kuperberg and N. Weaver, “A von Neumann algebra approach to quantum metrics”, (2010) arXiv:1005.0353
- [29]
- J. H. Conway and N. J. A. Sloane, “Orbit and coset analysis of the Golay and related codes”, IEEE Transactions on Information Theory 36, 1038 (1990) DOI
Page edit log
- Victor V. Albert (2022-11-06) — most recent
Cite as:
“Error-correcting code (ECC)”, The Error Correction Zoo (V. V. Albert & P. Faist, eds.), 2022. https://errorcorrectionzoo.org/c/ecc
Github: https://github.com/errorcorrectionzoo/eczoo_data/edit/main/codes/classical/ecc.yml.