Hi,

1. When viewing webstats, it says that googlebot visited today (13 May) as it has been visiting the site every single day the past week. But when I check Google Webmaster Tools, it says there that the last time the site was visited was 7 May. Can someone explain this difference?

2. I have hundreds of dynamic pages (php/MySQL), created on-the-fly with about 15 different pages as templates. Are the bots actually "visiting" all those pages? I'm asking since I created a sitemap, and there, only the templates were indexed.

Cheers,
Dan

1. When viewing webstats, it says that googlebot visited today (13 May) as it has been visiting the site every single day the past week. But when I check Google Webmaster Tools, it says there that the last time the site was visited was 7 May. Can someone explain this difference?

A GoogleBot is different from the analytics software( i hope you are using it). GoogleBot visits your site daily if your robots file has set permission for GoogleBot to spider your pages.

2. I have hundreds of dynamic pages (php/MySQL), created on-the-fly with about 15 different pages as templates. Are the bots actually "visiting" all those pages? I'm asking since I created a sitemap, and there, only the templates were indexed.

First of all the Bots visit all the pages, if the pages were in html format then it is easy for the bot to index, if it is somewhat in format like .php?p=35434 then it is difficult.

Generally SiteMap is only for your visitors for easier navigation and bots do not follow them, they only adopt the robots.txt file if present or adopt their own way.

Thanks Biju,

I am a bit confused by your replies, though. The Google Webmaster Tools specifically says that "Googlebot last successfully accessed your home page on May 7, 2007.". Hence, it's the bot, and on the stats it says it's visiting each day.

Your second reply is also confusing. You say "if it is somewhat in format like .php?p=35434 then it is difficult." and that's pretty much what I'm asking: Is it visiting (and indexing) dynamic pages or isn't it? By "difficult" it sounds as if you mean that it tries a few seconds, and then says "Ah..sod it" and leaves. :) I doubt that that's the way it works...

Perhasp you can elaborate?

Cheers,
Dan

(Oh, my reply sounded as if I'm irritated. I'm definitely not.)

To try to clarify what Biju was saying, google analytics and Googlebot are 2 different things.

Google Analytics is more targeted for how people got to you page, so when people come to your site it tells you the keywords, time etc of when and how people got to your page. The time that you may see in webmaster tools I believe is the last time you manually ran the analytics.

Googlebot is googles Spider bot that just scans all of your websites pages. SO Lets say you put the google analytics code onto your index page. And the bot decides that its going to visit other-page.html. Well this counts as a hit from googlebot but not towards your google analytics.

I hope this clears it up, I know my explanation may be vague but it's the only way I can put it to words.

Cody

To try to clarify what Biju was saying, google analytics and Googlebot are 2 different things.

Google Analytics is more targeted for how people got to you page, so when people come to your site it tells you the keywords, time etc of when and how people got to your page. The time that you may see in webmaster tools I believe is the last time you manually ran the analytics.

Googlebot is googles Spider bot that just scans all of your websites pages. SO Lets say you put the google analytics code onto your index page. And the bot decides that its going to visit other-page.html. Well this counts as a hit from googlebot but not towards your google analytics.

I hope this clears it up, I know my explanation may be vague but it's the only way I can put it to words.

Cody
Thanks Cody,

However, I think you are as confused as I am. :)

I don't "run" any analytic tool at all. I don't have any code on my pages. I just look at the text that says that the BOT visited my aite a certain day.

There's no definition of this bot at all (as far as I can see) whether it's the same or different than the regular bot. So, as far as I understand it IS the Googlebot you are referring to in second paragraph.

I'm sorry in case I appear to be hard to convince. I'm not, usually. But both yours and Biju's explanations seems to be just guesswork. No offense, I hope...but they do....

But I truly appreciate that you've tried.

Cheers,
Dan

Sorry, I thought you were talking about Google analytics and not webmaster tools.

I do apologize.

Cody

First of all the Bots visit all the pages, if the pages were in html format then it is easy for the bot to index, if it is somewhat in format like .php?p=35434 then it is difficult.

Generally SiteMap is only for your visitors for easier navigation and bots do not follow them, they only adopt the robots.txt file if present or adopt their own way.

Not quite true, the initial idea of the sitemap is to get indexed by Google.

Sorry, I thought you were talking about Google analytics and not webmaster tools.

I do apologize.

Cody
No problems Cody.

I think I have found the answer though:

At first, Googlebots obviously makes a brief visit - on a daily or near-daily basis. Just checking if there are any new links and/or major updates to the site compared to what's in their database.

If there is, it'll return later and scoop up the new data, index new pages and new texts, etc.

That explanation seems to hold water. Because my site stats registers even those little brief visits. But Googles Webmaster Tools only counts the "serious" visits when the actual updates to their database are being done.

Something like that....

Not quite true, the initial idea of the sitemap is to get indexed by Google.
I think it work both ways.