Robots Meta Tag
The robots meta tag is an HTML element placed in a page's `<head>` that provides page-level crawling and indexing instructions to search engine bots. Common directives include `index/noindex` (whether to add the page to the index) and `follow/nofollow` (whether to follow links on the page). Multiple directives can be combined, e.g., `<meta name="robots" content="noindex, nofollow">`.
Unlike the robots.txt file — which operates at the server level and controls crawl access — the robots meta tag operates at the page level and is only readable once a bot has already fetched the page. This distinction is important: if robots.txt blocks a URL, the bot never reads the meta tag. Specific bots like Googlebot can be targeted with bot-specific meta tags (e.g., `name="googlebot"`).
Why it matters for SEO
The robots meta tag gives granular, page-level control over how search engines index and link-follow behavior. Incorrect configurations — such as accidentally noindexing key pages — can cause significant ranking losses that are often difficult to diagnose without a technical audit.
Want to automate Robots Meta Tag optimization?
LazySEO handles this automatically across all your articles.
Free tools to help with Robots Meta Tag
Ready to put Robots Meta Tag into practice?
LazySEO automates keyword research, content writing, and publishing — so you rank without the manual work.
Try LazySEO Free