How To Optimize Robots.txt File For SEO in WordPress


Hello friends today we come to teach you that How To Add Custom Robots.txt File in wordpress blog? you just need to follow given below steps to Add Custom Robots.txt File in wordpress blog.

What is Robots.txt?

Robots.txt is a text file which contains few lines of simple code.this will use to allow and disallow the links to crawled in search engins means the search engin will read your robots.txt file before crawl your blog posts then they crawl your blog.and in robots.txt file we can allow and disallow the links to crawl in search engines

Video tutorial:-


How to add custom robots.txt in wordpress blog?

  • Go to your wordpress panel
  • install yoast seo plugin
  • Navigate to Settings >seo setting >Tools>file editor
  • Now paste your robots.txt file code in the box.


sitemap: https://www.

User-agent:  *
# disallow all files in these directories
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /archives/
disallow: /*?*
Disallow: *?replytocom
Disallow: /comments/feed/
User-agent: Mediapartners-Google*
Allow: /
User-agent: Googlebot-Image
Allow: /wp-content/uploads/

User-agent: Adsbot-Google
Allow: /

User-agent: Googlebot-Mobile
Allow: /

Replace with your blog


  • below given robot.txt file code is default code for all wordpress blogs


User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php


Step to Check Your Robots.txt File?

You can check this file on your blog by adding /robots.txt at last to your blog URL in the browser. like this:-           https://www.
then you wiil see the all codes of robot.txt file See below image.


Please enter your comment!
Please enter your name here