Robots Meta Tag
The robots meta tag is an HTML element placed in a page's `<head>` that provides page-level crawling and indexing instructions to search engine bots. Common directives include `index/noindex` (whether to add the page to the index) and `follow/nofollow` (whether to follow links on the page). Multiple directives can be combined, e.g., `<meta name="robots" content="noindex, nofollow">`.
Unlike the robots.txt file — which operates at the server level and controls crawl access — the robots meta tag operates at the page level and is only readable once a bot has already fetched the page. This distinction is important: if robots.txt blocks a URL, the bot never reads the meta tag. Specific bots like Googlebot can be targeted with bot-specific meta tags (e.g., `name="googlebot"`).
Why it matters for SEO
The robots meta tag gives granular, page-level control over how search engines index and link-follow behavior. Incorrect configurations — such as accidentally noindexing key pages — can cause significant ranking losses that are often difficult to diagnose without a technical audit.
Free tools to help with Robots Meta Tag
Ready to put Robots Meta Tag into practice?
LazySEO automates keyword research, content writing, and publishing — so you rank without the manual work.
Try LazySEO for $1