Blocking WordPress Posts And Pages From Crawlers

noindex nofollow WordPress posts and pages

As you may have noticed WordPress itself does not allow noindex, nofollow of individual posts and pages without using a plugin. The only option you have in WordPress Admin is to allow crawlers to index all pages or to index nothing.

There are however ways of working around WordPress deformities, actually, there are 3 different methods of blocking crawlers from crawling posts and pages. Below I discuss the easiest method, which is accomplished by editing the theme header.php template.

Its an extremely easy method of implementing <meta name=”robots” content=”noindex,nofollow”> and I include snapshots and examples.

Always remembering meta tags are not case sensitive, it don’t matter what you use (caps on or caps off) and you do not need to add <meta name=”robots” content=”index,follow”> By default search engines crawl pages and follow links when they don’t encounter the meta tag.

<Meta Name=”Robots” Content=”Noindex,Nofollow”>

In order to block individual posts or pages, you first need to know the post/page ID (its a number) The easiest method of locating the ID is to navigate to wp-admin, then click on all posts or all pages, choose the page you would like to block then click on edit (below snapshot)

Wordpress edit page

The number denoted in the below URL is the post ID. It refers to the row in the wp_posts database table.

Wordpress edit page noindex,nofollow

After locating the ID number of the post you want to noindex, you need to add the below code to the head section of your theme header.php template between <head> and </head>.

<?php if ($post->ID == #) { echo ‘<meta name=”robots” content=”noindex,nofollow”>’; } ?>

In the above code, # represents the ‘identity number’ of the post you want to noindex. In other words, if your post had an ID of 1194, the code would look as follows:

<?php if ($post->ID == 1194) { echo ‘<meta name=”robots” content=”noindex,nofollow”>’; } ?>

The above code has no limitations and restrictions. Hence, you and can block multiple pages, posts, attachment and even portfolios and galleries. Meaning it will work with any type of file stored in the wp_posts database table.

To noindex multiple posts you simply need to specify the identities of the posts that you want to block. For example, if you want to block crawlers from indexing posts and pages with ID 1194, 803, and 807. You can do this by simply adding them to the existing code.

<?php if ($post->ID == 1194 || $post->ID == 803 || $post->ID == 807) { echo ‘<meta name=”robots” content=”noindex,nofollow”>’; } ?>

Locating Your WordPress Theme Header.Php Template

There are actually three methods of editing the theme header.php template. The first and easiest method is in WordPress Admin. The other two methods are FTP and Cpanel (at your hosting company).

Adding <meta name=”robots” content=”noindex,nofollow”> via WordPress Admin

This method of adding <meta name=”robots” content=”noindex,nofollow”> via WordPress Admin is extremely simple. Simply navigate to your wp-admin, click on Appearance then click on Editor then open the Theme Header and insert the code between the <head> and </head> (below snapshot)

Adding noindex,nofollow via WordPress Admin

Make sure you configure everything correctly, it is also advised to verify that you have blocked the correct pages from crawlers. The simplest method of checking is to view the source code of the pages you have blocked. If you have added the code correctly, you will see <meta name=”robots” content=”noindex,nofollow”> in the source code. If not, the code has been added incorrectly. And also randomly check other pages to make sure they are not blocked.

Adding <meta name=”robots” content=”noindex,nofollow”> via Cpanel

Your WordPress theme header.php template is located at your hosting company (C-panel, public files) under File Manager.

Adding noindex,nofollow via Cpanel

Open the File Manager then click on wp-content.

 Cpanel file manager WordPress content

Choose your theme then open it (in this instance it was zerif-pro)

Cpanel file manager WordPress Zerif Pro

Highlight header.php, then at the top of the page click edit

Cpanel file manager WordPress header php

Insert the code between the <head> and </head> then click save.

Edit header php in Cpanel

Coincidentally. all changes made via C-panel at your hosting company will automatically reflect in your WordPress control panel Theme Header. And the same applies when editing via FTP.

Adding <meta name=”robots” content=”noindex,nofollow”> to theme header.php via FTP

If you have an FTP client installed you can edit the theme header.php template based on the same instructions as for editing via C-panel.