Hello friends today we come to teach you that How To Add Custom Robots.txt File in wordpress blog? you just need to follow given below steps to Add Custom Robots.txt File in wordpress blog.
What is Robots.txt?
Robots.txt is a text file which contains few lines of simple code.this will use to allow and disallow the links to crawled in search engins means the search engin will read your robots.txt file before crawl your blog posts then they crawl your blog.and in robots.txt file we can allow and disallow the links to crawl in search engines
How to add custom robots.txt in wordpress blog?
- Go to your wordpress panel
- install yoast seo plugin
- Navigate to Settings >seo setting >Tools>file editor
- Now paste your robots.txt file code in the box.
sitemap: https://www.126.96.36.199/sitemap.xml User-agent: * # disallow all files in these directories Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /archives/ disallow: /*?* Disallow: *?replytocom Disallow: /comments/feed/ User-agent: Mediapartners-Google* Allow: / User-agent: Googlebot-Image Allow: /wp-content/uploads/ User-agent: Adsbot-Google Allow: / User-agent: Googlebot-Mobile Allow: / Replace 188.8.131.52 with your blog
- below given robot.txt file code is default code for all wordpress blogs
Step to Check Your Robots.txt File?
You can check this file on your blog by adding /robots.txt at last to your blog URL in the browser. like this:- https://www.184.108.40.206/robots.txt
then you wiil see the all codes of robot.txt file See below image.