Blocking Pages From Google Search

There are a number of options available for blocking content from Google. You can use password-protection, robots.txt and the noindex meta tag

» SEO Help » Blocking Content

Blocking Content From Google

Blocking Content From Google

There are a number of options available for blocking content from Google. You can use password-protection, robots.txt and the noindex meta tag (not to forget, deleting the page!)

Blocking Google Via Password Protection

Google and other spiders can’t access password protected pages, so this is a very effective way to protect your personal information from appearing in the Google search, or for that matter, any search. You can achieve this by editing your htaccess file, that’s if you are using Apache Web Server. There are also a lot of tools available that will allow you to do this.

Blocking Google Via robots.txt

Files and directories can be blocked via robots.txt. The robots.txt file is uploaded to the root of your host (you would naturally need access to your C-Panel File Manager) Always remembering; robots.txt is not a guarantee. If Google discovers URLs on other sites they may still add it to index.

Example of robots.txt that allows all spiders to crawl all files and directories.

User-agent: *
Disallow:

Example of robots.txt that disallows all spiders to crawl all files and directories. The backslash disallows all robots (it blocks all files and directories from all robots)

User-agent: *
Disallow: /

Blocking Google Via Robots Meta Tag

When Google comes across the noindex meta tag they will completely drop that page from the search, regardless whether other sites link to it.

Example of robots meta tag, blocking all spiders from indexing and following links.

<meta name=”robots” content=”noindex,nofollow” />

Example of robots meta tag, allowing all spiders to index and follow links.

<meta name=”robots” content=”index,follow” />


...

Author - Terry

Terry is the owner of Cyber SEO Srvices and specializes in Optimization, Web Design and Digital Marketing and has been worked in the field for over 10 years and in Jan 2019 he qualified as a Google Ads Specialist and Consultant for Google Search, Display, Mobile, Shopping and Video Ads.

FOR PROFESSIONAL SEO ADVICE CONTACT US TODAY.

CONTACT US FOR ASSISTANCE!