X-Robots-Tag
What Is the X-Robots-Tag? The X-Robots-Tag is an HTTP header that provides the same indexing and crawling directives as a meta robots tag, but is applied at the server level rather than within a page’s HTML. Its Use for Non-HTML Files Because the meta robots tag only works within HTML `<head>` sections, it can’t be […]