Why DNS Replaced a Centrally Distributed HOSTS.TXT File

Before DNS, Internet hosts commonly learned names and addresses from a master host table maintained by the Network Information Center at SRI. Sites fetched copies of HOSTS.TXT and used the local file for lookup. That architecture was simple when the community was small; it made one organization, one master namespace and repeated full-file distribution part of every site’s ability to name a growing network.

DNS changed two different things: it made the namespace hierarchical and it delegated authority over pieces of that hierarchy. Distributed servers and caching followed from those choices. The transition took years and coexisted with host tables; no single RFC instantly deleted HOSTS.TXT from every machine.

The host table was a replicated central database

The master table associated human-meaningful host names with network addresses and related properties. Organizations submitted changes to the NIC. The NIC validated/assembled a canonical file. Each participating host periodically transferred a new copy and parsed it into its local environment.

Lookups were fast once the file was local and required no live naming service. The failure mode was coordination. A new host or address change was not globally useful until the central table incorporated it and sites downloaded the update. Old copies remained plausible but stale.

The model also confused two meanings of “distributed.” Copies of the file were widely distributed, but authority to allocate and publish the shared namespace remained centralized. Replication reduced lookup dependence on the NIC while increasing update traffic and inconsistency between copies.

Growth stressed names, traffic and administration

As the ARPA Internet expanded, the table grew and changed more frequently. More sites fetched a larger file; more administrators submitted changes; name collisions and coordination became harder. Mail routing added information needs that a flat host-to-address list handled poorly.

RFC 830, published in 1982, discussed a distributed Internet name service and a gradual conversion of the host table. RFC 882’s 1983 rationale described the centralized host table and mail naming problems as related symptoms requiring a general naming system.

The bottleneck was not that one text file had exceeded a magic byte limit. It was the product of growth rate, update ownership, transfer load, synchronization delay, and a flat global naming policy. Compressing the file alone would not let each organization administer its own names safely.

Hierarchy made global uniqueness delegable

A tree-shaped namespace allows the same leftmost label to exist under different parents while the full domain name remains unique. An organization need not ask one central editor to approve every host label if authority for its subtree has been delegated.

This is the difference between a domain and a zone. A domain is a node and everything below it in the namespace. A zone is the contiguous portion for which a particular administrative authority publishes data, excluding subtrees delegated to other authorities.

The root remains a coordination point, but it delegates rather than storing every host record. Parent zones publish referrals to authoritative servers for children. Each delegated organization can update its own zone on its own schedule while following common protocol and data formats.

DNS therefore decentralizes routine administration without eliminating hierarchy or governance. Root-zone management, top-level-domain policy, registrar/registry processes, protocol parameters and dispute rules remain coordinated.

Mockapetris specified a general resource-record system

Paul Mockapetris at USC’s Information Sciences Institute authored RFC 882 and RFC 883 in November 1983. RFC 882 defined concepts and facilities: the domain-name tree, zones, name servers, resolvers, resource records, caching and referrals. RFC 883 specified implementation details and message formats.

A name in DNS identifies a set of resource records, not necessarily one machine. Records have owner name, type, class, lifetime and typed data. Address mappings are one application; authoritative-server information, aliases and mail routing also fit the model. Later standards revised specific record types while preserving the extensible RR architecture.

This corrects the common description of DNS as “a distributed phone book mapping names to IP addresses.” That analogy explains one lookup but hides delegation, multiple data types, aliases, service discovery, security records and names that have no address record at all.

Resolvers follow referrals across authority boundaries

A resolver starts with configured name-server knowledge and asks for a name/type. A server authoritative for the answer can return it. A server without the answer can refer the resolver toward servers closer to the responsible zone. Recursive resolvers commonly perform that work for clients and cache results.

The protocol separates where authority lives from where a query is answered. A recursive resolver can answer from cache without being authoritative. A secondary authoritative server can serve a synchronized copy without being the administrative source. Those roles matter during incident diagnosis.

Redundant authoritative servers reduce dependence on one machine, but only if they are independently reachable and carry consistent zone data. Distribution moves failure modes; it does not abolish them.

Caching trades load and latency for bounded staleness

Every cacheable DNS record carries a time-to-live chosen by the authority. Resolvers can reuse it until that lifetime expires, reducing repeated traffic and speeding lookups. During that interval, a changed authoritative record may coexist with older cached answers.

This is often called “DNS propagation,” as if a new value were actively pushed through every server. Ordinary DNS caching is mostly pull-based. Different resolvers refresh at different times depending on what they cached and its remaining TTL. Authorities cannot recall a still-valid cached record instantly.

TTL planning is consequently operational policy. Lower values can shorten future cache persistence but increase query load; lowering a TTL only immediately before a change does not erase copies cached under the previous longer TTL. Negative answers also gained standardized caching behavior later.

The design values availability and scalable lookup over instantaneous global consistency. That trade was explicit in early DNS documents, which regarded access to information as more important than perfect immediate agreement during updates.

Migration was scheduled, not instantaneous

RFC 881 proposed an implementation plan alongside the first DNS specifications. When progress slipped, RFC 921 published a revised October 1984 schedule. It called for domain registration, deployment of principal servers/resolvers, significant resolver use by May 1985, completion of the switch to domain-style resolution in July, and a September point after which the master host table no longer needed to be complete for the DARPA research community.

The schedule itself proves coexistence. DNS server information could appear in HOSTS.TXT; applications and system calls had to be modified; popular operating systems needed resolver implementations; administrators required new procedures. A host table and DNS could both answer names during the transition.

RFC 1034 and RFC 1035 replaced the original specifications in 1987, incorporating implementation experience and forming the enduring conceptual/protocol base. DNS was not “finished” in 1983.

The hosts file never disappeared

Modern systems still provide a local hosts file. It is useful for bootstrap, loopback names, emergency overrides and tightly controlled static mappings. Resolver order is host-specific and can consult local files before or after DNS according to configuration.

What disappeared was reliance on one globally complete, centrally edited file as the Internet’s normal naming database. A local override does not update authoritative DNS and may conceal a production error only on the administrator’s machine.

The survival of hosts files also warns against an oversimplified victory narrative. DNS replaced a global coordination/distribution mechanism, not the concept of static local name mappings.

DNS introduced new operational failure modes

Delegation can be lame or circular. Authoritative servers can disagree. Serial/transfer mistakes can leave secondaries stale. Cached data can outlive an emergency change. Recursive resolvers can fail, poison caches, leak queries or amplify denial-of-service traffic. A syntactically valid zone can direct mail or services to the wrong place.

Diagnosing DNS therefore requires asking which record type, which resolver, whether the answer is authoritative or cached, the TTL, the delegation path, and whether all authoritative servers agree. “It works on my machine” may reflect a local hosts entry or cache rather than healthy DNS.

These are costs of distributing authority and cached replicas. They are generally more manageable than requiring the entire Internet to download one frequently changing master table, but they are real design tradeoffs.

Original DNS did not authenticate answers

The 1980s protocol identifies servers and carries data but did not provide modern cryptographic origin authentication and integrity. Transaction IDs and source checks were not a chain of trust. Cache poisoning and forged referrals became serious as the network’s threat model changed.

DNS Security Extensions add signed data and delegation-of-trust records. The current core design was standardized in RFC 4033, RFC 4034 and RFC 4035 in 2005 after an earlier DNSSEC specification. Validating resolvers can distinguish authenticated data from bogus signatures when the chain is correctly deployed.

DNSSEC does not encrypt query names, make an incorrect signed record correct, secure registrars automatically, or configure applications to validate. Key generation, rollover, delegation records, signing and monitoring add operational work. It solves a different problem from the original scaling motivation.

The real replacement was an authority model

The host-table story is sometimes told as “a file became too large, so engineers put it on servers.” Servers alone would still leave every update with the NIC. The durable innovation was delegating a hierarchical namespace while standardizing how resolvers discover and cache authoritative data.

DNS scaled because organizations could own their zones, servers could refer queries across boundaries, and caches could absorb repeated demand. It accepted temporary disagreement bounded by policy instead of requiring every site to possess the newest global file before lookup.

That is why DNS outlived HOSTS.TXT as the Internet-wide system: not because text files stopped working, but because global naming needed distributed administration as much as distributed lookup. Related: No, Al Gore Never Said He ‘Invented the Internet’ — Here’s the Actual Quote · No, ‘The Internet’ and ‘The Web’ Are Not the Same Thing