Claude Shannon's 1948 Communication Theory: Entropy, Noise, and Capacity

Claude Shannon’s 1948 paper is often reduced to one equation or credited with a vague invention of “the digital age.” Its lasting contribution is more precise: it formalized communication as a statistical engineering problem and related the amount of information a source produces to the ability of a channel to carry it despite noise. That separation created a way to ask what reliable transmission could achieve before choosing a particular code or circuit.

The paper appeared in two installments in The Bell System Technical Journal: Part I in July (volume 27, issue 3, pages 379–423) and Part II in October (issue 4, pages 623–656). The accessible institutional copy cited below is a corrected reprint of both installments, not a scan hosted by the original publisher. The publisher records and Bell Labs archive establish the journal metadata; the corrected reprint is the readable source for the paper’s technical content.

A model that isolates the engineering problem

Shannon begins with a source that selects a message, a transmitter that turns it into a signal, a channel that can perturb it, and a receiver that reconstructs a message at the destination. The model makes “correct communication” measurable without needing to interpret a message’s meaning. In the paper’s engineering framing, the significant question is whether the receiver can reproduce the chosen message, exactly or approximately, from a set of possible messages.

That choice was not a claim that meaning never matters. It was a deliberate boundary for analyzing a communication system: first model the possible messages, their statistics, and the channel’s noise; then ask how reliably signals can preserve the intended choice. Shannon explicitly places his argument alongside earlier work by Harry Nyquist and Ralph Hartley, while extending the discussion to channel noise and statistical structure in the source.

The same distinction is useful when diagnosing a modern link. A radio’s modulation rate, the number of payload bits in a packet, and the semantic value of an application request are different quantities. A capacity calculation does not tell a product team whether a particular message is important; it gives an upper bound under a specified channel model.

Entropy measures uncertainty in a source model

For a discrete source with outcomes xᵢ and probabilities pᵢ, Shannon defines entropy as H(X) = -Σ pᵢ log₂(pᵢ). With a base-2 logarithm, the unit is the bit. The paper notes that J. W. Tukey suggested the short name “bit”; Shannon did not present the underlying idea as appearing from nowhere. The logarithm connects a count of equally likely alternatives to the number of binary decisions needed to distinguish them.

Consider a simple memoryless source that emits A with probability 1/2, B with probability 1/4, and C with probability 1/4. Its entropy is 1.5 bits per symbol: -(0.5 log₂ 0.5 + 0.25 log₂ 0.25 + 0.25 log₂ 0.25). The prefix code A→0, B→10, C→11 has an average length of 1.5 bits per source symbol. The example shows how statistical imbalance creates a compression opportunity: likely symbols receive shorter codewords.

Entropy is an average for the assumed probability model, not the length of every message and not a measure of human significance. If a source’s distribution changes, the model and code may become inefficient. A real compressor also has framing, dictionaries, and finite-block overhead. Shannon’s source-coding result says that long sequences can be represented near their entropy rate under the stated source assumptions; it does not promise that every short file shrinks or that a chosen format is optimal for every input.

Capacity relates a noisy channel to reliable rate

Noise makes exact symbol-by-symbol transmission unreliable, but redundancy can let a receiver infer the original message. Shannon defines channel capacity as the maximum information rate permitted by a channel’s transition behavior. For a binary symmetric channel (each bit independently flips with probability p), the capacity is C = 1 - H₂(p) bits per channel use, where H₂(p) = -p log₂ p - (1-p) log₂(1-p).

At a 10% crossover probability, H₂(0.1) is about 0.469, so the ideal capacity is about 0.531 information bits per channel use. This is not a claim that a wire or radio should simply be operated at that raw rate. The result assumes the channel model is correct and applies to coding over sufficiently long blocks. A practical design also has to choose modulation, framing, latency, power, implementation complexity, and a target error rate.

The noisy-channel theorem is the striking limit: with suitable coding, rates below capacity can make the probability of error arbitrarily small in the asymptotic limit; a rate above capacity cannot be made reliable by a clever code alone. “Arbitrarily small” is not “zero errors in every finite transmission.” Long blocks can add delay and computation, and real links can violate the assumed stationary or independent-noise model. Capacity is a benchmark for a defined channel, not a service-level guarantee for a deployed network.

Read the two parts as one argument

Part I develops the discrete-source and discrete-channel foundations, including entropy, redundancy, and coding under noise. Part II extends the analysis to continuous signals and channels, including band limits and power constraints. Those extensions matter historically because they connect the abstract rate question to telephone and radio engineering, but formulas from a continuous Gaussian model must not be transplanted into an arbitrary digital network without checking the model and units.

For an engineering review, the paper suggests a disciplined sequence:

1. Define what counts as a source symbol and estimate its distribution from representative data.
2. State the channel model, units, bandwidth or symbol-rate limits, and how errors are measured.
3. Compare the source rate with the model’s capacity before selecting a code; leave margin for finite blocks and changing conditions.
4. Measure actual error rate, latency, and throughput on the deployed system. The theorem does not certify a particular modem or software stack.

This method is a better legacy than any single equation: separate source compression from channel protection, state assumptions, and distinguish a theoretical limit from an implementation result.

The 1948 communication paper should also not be conflated with Shannon’s later work on secrecy. Nokia Bell Labs’ journal archive lists “Communication theory of secrecy systems” as a separate 1949 article. That chronology matters: information transmission through noise and cryptographic secrecy are related subjects, but they pose different engineering questions.

Related:

• Public-Key Cryptography: From Classified Prehistory to Diffie-Hellman and RSA
• JPEG and T.81: How a Still-Image Coding Standard Became Interoperable

Sources:

• Corrected reprint of Shannon’s two-part paper, hosted by Harvard Mathematics
• July 1948 publisher record and DOI, Part I
• October 1948 publisher record and DOI, Part II
• Nokia Bell Labs archive of Bell System Technical Journal milestones

The Harvard-hosted PDF identifies itself as a corrected reprint of the July and October journal pages. The two DOI links identify the publisher records for the individual installments, while the Nokia Bell Labs archive lists both 1948 communication-theory installments and the separate 1949 secrecy-systems paper. Use the reprint for readable text and the journal records for publication metadata.