HTTP Headers

HTTP headers look like small lines of metadata, but they control much of the real behaviour of the web. Caching, authentication, compression, cross-origin policy, content negotiation, and proxy behaviour are all shaped by headers. When something subtle breaks in a distributed system, headers are often involved.

Headers are part of the contract, not decoration

A response body tells you what the resource is. Headers tell clients and intermediaries how to treat it. Cache-Control decides whether a response may be stored. Content-Type tells the client how to interpret the bytes. ETag and Last-Modified enable conditional requests so caches and browsers can revalidate cheaply instead of downloading the full payload again.

That means incorrect headers can break correct application logic. A JSON response labelled as text/html, or a private response accidentally marked cacheable, can cause confusing bugs far from the original code.

Some headers are end-to-end and some are hop-by-hop

End-to-end headers are meant for the ultimate client or origin server. Hop-by-hop headers are consumed by a single transport hop and should not be forwarded blindly by proxies. Confusing the two leads to odd behaviour, especially when multiple reverse proxies, load balancers, or service meshes sit in the path.

Forwarded request context is especially delicate. Headers such as X-Forwarded-For, X-Forwarded-Proto, and standardised Forwarded values become security-sensitive the moment you rely on them for rate limiting, access control, or absolute URL generation. Trust them only from known proxies.

Header size and duplication matter operationally

Large cookies and verbose custom headers increase every request size. At scale this affects bandwidth, latency, and sometimes outright compatibility because proxies and servers enforce header size limits. Duplicate headers can also change meaning depending on the field. Some headers are safely repeatable; others are not.

Normalisation matters too. HTTP header names are case-insensitive, but applications and middleware sometimes mishandle them. When a bug only appears through one proxy chain, header normalisation is a prime suspect.

Security lives in headers more than many teams realise

Transport security, framing rules, content type sniffing, and cross-site protections are heavily header-driven. Strict-Transport-Security, Content-Security-Policy, X-Content-Type-Options, Referrer-Policy, and cookie attributes such as HttpOnly and SameSite all shape browser behaviour. Missing or weak values can create vulnerabilities even when the application code looks fine.

A good rule of thumb

Treat headers as a first-class API surface. Document the headers your service depends on, validate the ones you consume, and inspect them whenever behaviour differs between environments. In distributed systems, the body often tells you what happened. The headers often tell you why.

Caching headers deserve special respect because they involve invisible collaborators. Browsers, CDNs, reverse proxies, and API gateways may all cache differently if directives are vague. A response that works in local development can become stale or unexpectedly public in production because an intermediary took the header contract more literally than the application author did.