Custom robots.txt for Plus UI 3.1 – How to Add
A robots.txt file is important for SEO as it helps search engines understand which pages to index and which to ignore. In this guide, we will show you how to add a custom robots.txt file to your Blogger website using Plus UI 3.1.
Step 1: Open Blogger Dashboard
1. Go to Blogger and log in to your account.
2. Select the blog where you have installed Plus UI 3.1.
Step 2: Enable Custom robots.txt
1. In the left menu, click on Settings.
2. Scroll down to Crawlers and Indexing.
3. Find the option Custom robots.txt and enable it.
Step 3: Add the Custom robots.txt Code
1. Click on the Edit button next to Custom robots.txt.
2. Copy and paste the following code:
Custom robots.txt Code:
Custom robots.txt
If custom robots.txt option in Blog Settings is disabled, Blogger serves default robots.txt. But default robots.txt disallows bots to crawl all pages of and under /search segment which includes blog page (i.e. /search) and label pages as well (i.e. /search/label/Product), which is probably what you don't want.
To fix this, we can use robots.txt with custom rules allowing bots to crawl these pages as well. You can use the following custom robots.txt if you don't have idea about how does this work, otherwise feel free to write your own or modify it.
# all user agents
User-agent: *
# exclude pagination and query pages
Disallow: /search*updated-max=*
Disallow: /search*max-results=*
Disallow: /search*q=*
# include everything else
Allow: /
# sitemaps
Sitemap: https://www.example.com/sitemap.xml
Sitemap: https://www.example.com/sitemap-pages.xmlUser-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: https://www.example.com/sitemap.xmlCustom codes
This section will guide you to add custom codes in the theme. We recommend you add custom codes as described below for better managements.
Warning
Since you are adding your custom codes, we will not be responsible for any issues caused by it.
Custom meta tags
Custom meta tags can be added through Blogger HTML Editor.
- Go to Blogger HTML Editor.
-
Find the following line of code and add your custom meta tags below it.
<b:comment><!--[ Add additional meta tags (below) ]--></b:comment>
Custom codes
This section will guide you to add custom codes in the theme. We recommend you add custom codes as described below for better managements.
Warning
Since you are adding your custom codes, we will not be responsible for any issues caused by it.
Why Use a Custom robots.txt in Plus UI 3.1?
- Prevents duplicate content issues.
- Blocks unnecessary pages from being indexed (e.g., search pages).
- Helps improve SEO by guiding search engine bots properly.
Custom meta tags
Custom meta tags can be added through Blogger HTML Editor.
- Go to Blogger HTML Editor.
-
Find the following line of code and add your custom meta tags below it.
<b:comment><!--[ Add additional meta tags (below) ]--></b:comment>
Why Use Custom robots.txt in Plus UI 3.1?
- Helps in better SEO indexing.
- Prevents duplicate content issues.
- Blocks unnecessary pages from search engines.
Conclusion
Adding a custom robots.txt to Plus UI 3.1 is simple and improves your website's SEO. Follow these steps carefully, and always check if search engines are indexing your important pages properly.
