Zurmo – How to Block Search Engine Robots

Many Webmasters and site administrators work hard to get their sites ranked highly with the major search engines. It’s a booming business with fast changing rules, and those who succeed in gaining a top ranking on google are sure to see an increase in traffic. So most of the shared hosting providers, including Arvixe, allow search engines to spider your web-root directory.

One of your considerations as a Zurmo administrator will be to manage the security and privacy of your CRM application. As such, one might legitimately ask oneself – if, indeed, it is really wise let Google or Bing, or Yahoo index your Zurmo instance.

This brief tutorial provide you with some basic ideas to block the “friendly” search engines from indexing your Zurmo instance. That is, those more-or-less benign search engines that respect meta tags and robots.txt files. Of course there are going to be bad actors out in the wild, who will stop at nothing including using nefarious or illegal methods to gain access to your systems, but that is topic for another post.

Generally if you have access to your web-root directory, (some call it the “document root” – on Arvixe personal hosting plans, it’s located under your home directory in a folder named ‘www’.) you can prevent site indexing using a robots.txt file.

In a nutshell, robots.txt is part of a standard protocol that well behaved search engine spiders are honor-bound to respect.

Simply create a text file with the following lines and drop it into your web-root directory.

#Robots.txt to prevent indexing of ZurmoCRM
User-agent: *
Disallow: /zurmo/

That’s all there really is to it. Now you can rest assured that at the very least, your Zurmo instance is not going to be exposed to the unwashed masses (via Google anyway).

Now if you are a belt-and-suspenders type of personality, or if you don’t have an Arvixe personal hosting account, and likewise may not have access to your web-root directory, you can still communicate your Zurmo no-indexing policy to the bots.

Fair Warning: The information that follows will entail modifying the Zurmo source in a non upgrade-safe way. What this means, is that should you feel comfortable enough with your web administration and coding skills to make this custom update, you may find that at some point in the future after updating your site to the latest version of Zurmo, this code change you are about to do, could disappear. Worse yet, if you make a mistake, you could render your Zurmo instance inoperable.  here are no guarantees that this change will not adversely affect your site. So be careful, and back up your site, before attempting the next bit.

First Step is to open the PageView.php file which controls how the XHTML headers are rendered.

# change to the  directory for the Zurno PageView.php file
cd www/zurmo/app/protected/core/views

#edit the file
vi PageView.php

Second Step is to add the following line to the return value of the protected function    renderXtmlHead().  he relevant snippet of code follows.

return '<head>' .
                   '<meta charset="utf-8">' .
                   '<meta http-equiv="X-UA-Compatible" content="IE=edge" />' . // Not Coding Standard

//ADD THE LINE BELOW TO PREVENT SITE INDEXING BY 'GOOD' SEARCH ENGINES
                   '
'<meta name="robots" content="noindex">' .

Remember to add the period at the end of the line, it’s the PHP concatenation operator. Without it, your change will amount to little more than a “500 Internal Server Error.” when you test.

Now you can sleep a little better, comforted in the knowledge that your Zurmo CRM instance is just a bit less exposed to pryin’ eyes.

(c) 2013 Windsor Wallaby. All rights reserved.

Tags: , , , , , , , | Posted under Zurmo | RSS 2.0

Author Spotlight

Windsor Wallaby

Windsor Wallaby

Windsor Wallaby is an independent and enthusiastic Zurmo CRM supporter and Open Source contributor. Active on the Zurmo user forums and a regular personality on the weekly Zurmo developer's conference call, Windsor is committed to building helping relationships by Listening, Learning, Doing, and Sharing. Windsor works with Zurmo CRM daily to track business opportunities and contacts. Windsor also integrates Zurmo as a core platform component for in-house and bespoke IT development projects.

Leave a Reply

Your email address will not be published. Required fields are marked *


× 7 = 28

You may use these HTML tags and attributes: <a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>