vvek

HTTP caching in plain words: Cache-Control, ETag and the headers worth setting

How browsers and CDNs reuse responses: freshness, revalidation with ETag, hashed asset URLs, and a small set of Cache-Control defaults that work.

Every web request that never reaches your server is one your server didn't have to handle. HTTP caching is the set of rules that makes that possible, and most of the trouble people have with it comes from one cause: not saying anything, and letting caches guess.

This article explains how caches decide what to do, which headers actually matter, and a few defaults you can use right away.

Two kinds of cache

A private cache belongs to one client, usually the browser. A shared cache sits between many clients and your server: a CDN, a reverse proxy, a corporate proxy.

That distinction matters when dealing with personal data. A page that shows someone's account must never be stored in a shared cache and served to the next visitor. That is what the private directive is for.

Fresh, stale, revalidated

A stored response has a lifetime. While its age is below that lifetime it is fresh, and the cache can hand it back without asking you anything. After that it is stale, and the cache should check with your server before reusing it.

The lifetime is set with max-age, in seconds:

Cache-Control: max-age=3600

That says: for the next hour, reuse this without asking.

If you send no Cache-Control at all, caches are still allowed to store the response and reuse it. The spec suggests a rough guess based on the Last-Modified date, roughly a tenth of the time since the last change, as MDN's caching guide describes. You don't want your behavior to rely on a guess, so set the header.

Revalidation: asking "has it changed?"

A stale response is not thrown away. If the cache has a validator for it, it can ask the server whether the copy is still good. The server answers 304 Not Modified with no body when nothing changed, which is much cheaper than sending the file again.

The usual validator is an ETag, an opaque token that changes when the content does:

HTTP/1.1 200 OK
ETag: "33a64df5"
Cache-Control: max-age=3600

Later, the client asks:

GET /index.html HTTP/1.1
If-None-Match: "33a64df5"

If the token still matches, the server replies 304 and the client keeps what it has. If not, it sends 200 with the new body. Last-Modified with If-Modified-Since works the same way using a timestamp; when both are present, If-None-Match wins. MDN prefers ETags because timestamps are harder to keep consistent across several servers.

A small Go handler illustrates the idea. The standard library already does the comparison for you if you set the header first:

func page(w http.ResponseWriter, r *http.Request) {
    body := render()
    sum := sha256.Sum256(body)
    etag := `"` + hex.EncodeToString(sum[:8]) + `"`

    w.Header().Set("ETag", etag)
    w.Header().Set("Cache-Control", "no-cache")
    if r.Header.Get("If-None-Match") == etag {
        w.WriteHeader(http.StatusNotModified)
        return
    }
    w.Write(body)
}

This still renders the page every time, so it saves bandwidth, not server work. For expensive pages, base the tag on something cheaper, like a version number or an updated-at timestamp.

What the directives really mean

The names are misleading, so let's be precise.

Directive What it does
max-age=N Fresh for N seconds; reuse without asking.
no-cache May be stored, but must be revalidated before every reuse.
no-store Do not store it at all.
private Only the user's own browser may store it.
public Storable even when the request carried Authorization.
immutable The content will never change, so skip revalidation on reload.

Warning

no-cache does not mean "do not cache". It means "cache, but check first". If you really want nothing stored, that is no-store, and it has a cost: MDN notes it also turns off the browser's back/forward cache and other benefits.

Make asset URLs change with their content

You cannot easily tell every browser in the world to drop a file. What you can do is never reuse a URL for different content.

If your build tool puts a content hash in the file name (app.3f9c2b.js), a change in the file means a new URL. The old URL can then be cached for as long as anyone likes:

Cache-Control: public, max-age=31536000, immutable

That is one year, and it is the example MDN gives for versioned assets. Most bundlers produce hashed names already; the work is in the server or CDN configuration that adds the header.

The HTML page that points to those files cannot be versioned this way, because people type its address. It should be revalidated every time, so a new deploy shows up straight away:

Cache-Control: no-cache
ETag: "9f2a1c"

Once a request carries an If-None-Match, the check is cheap, and the page stays correct.

flowchart LR
  A[Request] --> B{Stored copy?}
  B -- no --> F[Fetch from server]
  B -- yes --> C{Fresh?}
  C -- yes --> D[Use it]
  C -- no --> E[Ask server with If-None-Match]
  E -- 304 --> D
  E -- 200 --> F

Personal pages

For anything specific to a logged-in user, combine the two ideas:

Cache-Control: no-cache, private

It can be stored in the user's own browser, must be checked each time, and must not go into a shared cache. MDN also warns about request collapsing: a shared cache may hand one origin response to several clients that asked at the same moment, even for no-cache, so private is the directive that prevents that.

Vary: one URL, several versions

Caches key on the URL. If the same URL returns different content depending on a request header, say the language, tell the cache with Vary:

Vary: Accept-Language

Two guidelines from the same guide: avoid Vary: User-Agent, because it creates a huge number of variants and a poor hit rate, and do not use Vary: Cookie to separate personalised content. Use private for that.

A starting set of defaults

  1. Set Cache-Control on every response; do not leave it to guesswork.
  2. Hashed static assets: public, max-age=31536000, immutable.
  3. HTML documents: no-cache plus an ETag.
  4. Logged-in or personal responses: no-cache, private.
  5. Reach for no-store only for data that really must not be kept, such as a one-time token page.
  6. Avoid long strings of legacy directives. A pile of no-store, no-cache, max-age=0, must-revalidate, proxy-revalidate was meant for very old proxies, and MDN advises against it.

Checking your work

Open the browser's network panel and look at the Cache-Control, ETag and Age headers on a few requests. A normal reload revalidates; a forced reload fetches everything again. In code, fetch(url, { cache: "no-cache" }) behaves like a normal reload and { cache: "reload" } like a forced one, which is handy in tests.

With curl, you can try the conditional request by hand:

curl -sI https://example.com/ | grep -i etag
curl -sI -H 'If-None-Match: "33a64df5"' https://example.com/

A 304 on the second command means revalidation works.

One last point: you cannot reach into other people's caches and clear them. Browsers can be told to drop their cache with the Clear-Site-Data header, but proxies are out of reach unless they are managed ones, such as a CDN with a purge feature. Choose lifetimes you can live with, and rely on changing URLs, not on cleanup.

If you also build APIs that clients retry, the same care applies there; see idempotency keys for the write side, and rate limits from both sides for what happens when a cache is not enough.

Sources

Keep reading

A short email when we publish. Confirm once, unsubscribe any time.