
Key points
- Headers are metadata sent with every response. The browser and crawlers obey many of them.
- The ones to know:
Cache-Control,Content-Type,Content-Encoding,Location,Set-Cookie,X-Robots-Tagand the security family. - View them with
curl, browser DevTools or a header checker, and watch for version leaks such asServerandX-Powered-By.
What headers are
A request for a page and the reply both have two parts: headers, then a body. The reply's status line and headers look like this:
HTTP/2 200
content-type: text/html; charset=utf-8
content-encoding: br
cache-control: public, max-age=3600
strict-transport-security: max-age=31536000
x-robots-tag: noindex
Header names are not case sensitive. Your web server, CDN or application adds them, and any of these can change what you see.
The groups that matter
Caching
Cache-Controlsays how long browsers and CDNs may reuse a file, for examplemax-age=31536000, immutablefor versioned assets orno-storefor private pages.ETagandLast-Modifiedlet a browser ask "has this changed?" and receive a tiny "304 Not Modified" instead of the full file.Varytells caches which request headers change the response, such asAccept-Encoding.
More in compression and caching.
Content
Content-Typenames the format and character set. A wrong type or missing charset can cause broken text or blocked scripts.Content-Encodingshows the compression used, such asbr(Brotli) orgzip.Content-Lengthgives the body size in bytes, where known.
Redirects and status
- The status code (200, 301, 404, 503) is the first line, not a header, but it is read with the headers.
Locationgives the target of a redirect. See 301 vs 302.Retry-Aftertells clients when to try again after a 503 or 429.
Search engines
X-Robots-Tagcarries the same instructions as the robots meta tag (such asnoindex) and also works for PDFs and images. A leftovernoindexhere is a common reason a page will not appear in search, and it is easy to miss because nothing shows in the HTML.Linkcan carry canonical and hreflang information for non-HTML files.
Cookies
Set-Cookieshould carrySecure(https only),HttpOnly(not readable by scripts) and aSameSitevalue for sign-in or session cookies.
Security
Strict-Transport-Security,Content-Security-Policy,X-Content-Type-Options,Referrer-PolicyandPermissions-Policyharden the page in the browser. They have their own guide: security headers explained.
Information leaks
ServerandX-Powered-Byoften reveal software and exact versions. They do not break anything, but attackers use them to pick targets. Remove version numbers where you can.
How to view headers
- Online: paste an address into the HTTP header checker. It shows the status, redirect chain, every header with a plain-English note, and timing.
- Command line:
curl -sI https://example.com/prints the response headers. Some servers answer a HEAD request differently from a normal GET, so for a closer match usecurl -s -D - -o /dev/null https://example.com/. - Browser: open DevTools, go to the Network tab, reload, click the first request and read Response Headers.
Check the headers on the address a visitor actually ends up at. If the page redirects, the headers of the first response may be different from those of the final page.
Where to set them
It depends on your stack. Nginx uses add_header, Apache uses Header set, and hosts such as static site platforms and CDNs usually have a headers file or settings page. CMS plugins can also add some. After any change, test the live address rather than trusting the configuration.
# nginx
add_header X-Content-Type-Options "nosniff" always;
server_tokens off;
# Apache
Header always set X-Content-Type-Options "nosniff"
ServerTokens Prod
FAQ
What is the difference between request and response headers?
Request headers are sent by the browser, such as the language it prefers and cookies. Response headers are sent by the server, and they are the ones a header checker shows.
Can headers affect SEO?
Yes. The status code, redirects, X-Robots-Tag, canonical Link headers and caching all influence how crawlers treat a page, and server errors reduce crawling.
Why do I see headers I never set?
Your host, CDN or framework adds many by default, for example caching, compression and security headers. This is normal. Check the live response to see the final result.
Is it bad to show the Server header?
The header name alone is harmless, but exact version numbers help attackers find known flaws. Hide the version, or the header entirely, if your server allows it.
Why are headers different when I use curl?
curl sends different request headers from a browser, so some servers, CDNs or bot protections answer differently. Compare using the same address and method.