I'm trying to create a rewrite rule to drop into .htaccess which accomplishes the following:
- If the page request is /siteindex.html
- And the User Agent IS NOT Googlebot
- redirect the request to the home page (i.e. /)
Here's the logic behind the request:
I created a site index file which is linked from the home page in order to "help" Google index a site. The good news is this has been effective.
However, despite using meta robots commands, Google has included the site index page in their search results. The visitor behavior I'm seeing in our apache log files suggests folks arriving at this page from search results are not traversing the site as I'd expect (the page is really designed for robots and not very interesting).
I've read through mod_rewrite at apache.org and engelschall's tutorial, but just can't get this right. Any suggestions would be appreciated. Thx !
The problem may be that this could redirect Googlebot as well unless you match EXACTLY the user agent string which of course isn't just "Googlebot" but something longer and more likely to change.
Alternately (and what I'd likely do) you can just look for Mozilla and in the user agent and redirect on that. Since IE and Netscape both put Mozilla in their user agent string you'd get probably 95% of the users. Add whatever the user agent string for Opera is and you'd bring it close to 99%.