Background
Communication links transmit information from here to there.
Computer memories transmit information from now to then.
In either case, noise causes the received data to differ slightly from the original data.
Elwyn Berlekamp, The technology of error correcting codes, Proc. IEEE, May 1980
Forward error-correction (FEC) is the art of introducing controlled redundancy into a digital message, in the form of additional check symbols, to achieve a high degree of reliability over space and time, despite errors or failures.
Coding theory is the science of designing efficient FEC codes, that are simple to encode, simple to decode, and that meet the target level of performance using as few check bits as possible.
After 70 years of continuing research in coding theory, it is fair to recognize that a significant amount of experience and knowledge has been accumulated on the design of efficient channel codes, culminating in practical solutions that theoretically achieve or at the very least closely approach the ultimate performance limit (channel capacity) at data rates as high as several tens or even hundreds of Gb/s.
Still, many challenges remain ahead in the field of channel coding.
Challenges ahead
Modern communication systems have evolved to the point they bear little ressemblence to their ancestors which have inspired Claude Shannon’s celebrated point-to-point communication paradigm at the heart of his fundamental work on information theory.
Reliable communication over networks with many sources and destinations requires coding techniques that go beyond simple routing and classical point-to-point source/channel codes.
At present, the fundamental performance limits in information networks, as well as the optimal protocols and coding techniques that achieve these limits, are only known and well understood for a few canonical examples.
Digital storage has followed a similar path. The advent of new applications such as cloud storage, distributed learning, caching, or computing, prompt the need for novel coding paradigms.
Beyond ensuring reliable communication or computing, code design has now to address new goals such as maximizing the number of users that can be simultaneously served by the system, or minimizing the expected service time.
Little is known, to date, about the optimal achievable trade-off in many such scenarios, and the design of efficient and practical codes for implementation in real systems is a challenging problem and an active area of on-going research.
Open problems remain even in the classical point-to-point communication setting of Shannon’s original work.
Short-packet machine-to-machine communications at the heart of Internet of Things (IoT) have revived interest in the design of efficient short FEC codes for messages ranging from a few tens up to a few hundred bits.
While FEC at medium to long blocklength is well understood and mastered, much less is known regarding optimal coding at short length. Only bounds or approximations to the performance limits in this regime are known, offering little clues about how to achieve them.
Whether message-passing decoding can be sufficient to achieve these optimal performance at short length, just as it is in the asymptotic regime, is still an open question.
Fortunately we now have new tools that may come in handy to address such problems
Project goals & methodology
Explore and assess how machine learning techniques can contribute to improve our understanding and practice of error correction coding
The central goal of AI4CODE
The plan
The method
