Today we are going to discuss how to manage all of the bots/spiders that come to your forums.
So what are bot/spiders you ask? Well it’s really quite simple bot/spiders are automated agents most commonly used by search engines to update their databases.
They are one part to your Search Engine Optimization. However the issue arises that sometimes they don’t use the correct session token for your site and can mess up page ranking and visitor numbers.
So what can we do to these? Well let’s take a look go ahead login navigate to your Admin CP and then click on system and you will be taken to the following page:
After you do that click on Spiders/Bots it will take you to a page that lists all of the bots and spiders like so:
I know that this list can be overwhelming. It’s quite simple it shows all spiders/bots that are used by all the companies who use them. You can edit any one your want and you can deactivate them too. So let’s just look at one edit so click on any one you want. For the purpose of this tutorial we are going to use the very first bot that shows up on the list which is: Adbot by google. So go ahead and click edit to the right and you will see the following page:
So these are the fields you can edit they are as follows:
- Bot Name: You can change this if you wish to but you don’t have to.
- Bot Style: This allows you to change what theme or style you have set on your forums to what the bot will see. It’s recommended to keep the style that you use.
- Bot Active: It’s best to keep them active. Not doing so denies the bot access to your site and can hurt your ranking on search engines.
- Agent Match: This is just a string of words that match the bot name.
- Bot Ip Address: This allows you to set a IP range. I don’t recommend changing this at all the default is perfect here.
So after you edit those click Save and you will be taken back to the main page like above. Now on the right hand side you Have the following options:
1. Deactivate: This allows you to deactivate the bot. Again it’s not a good idea because it can prevent your site from being indexed and can hurt your overall page ranking.
2. Edit: It lets you edit the Bot.
3. Delete: This allows you to delete it.
If you scroll all the way to the bottom you can mass select the bots and do mass option changes to them.
It’s that easy! If you have any questions please feel free to ask away!!!!!