3 months ago I've moved about 30 sites from old server to new server with different IP and then all the pages in all sites are listed with url only on Google search results! No title, no desc, just url! Also I've checked log files and there was no googlebot traffic at all! What hapened??? (But I can see other spiders from other search engines come and index).
The new server is Windows 2003 server with IPSec enabled. The following ports are opened as I think it a must for hosting service:
I blocked all ICMP traffic (in & out), so the server does not reponse from ping requests.
I guess that not my server configuration problem (because other spider can visit and index!) But why no goolebot!
The 30 sites were very good ranking on google (usually from page 1 to page 3 on google result) before moving to new server. I also checked blacked list to see if my server IP was banned from google (thanks god there no my IPs on that list!).
So that's not server configuration problem, not IP banned problem, not search engine cloaking problem, and then ...what is really the problem?
The only thing I can guess is blocking ICMP traffic may cause trouble with googlebot! (But I dout about it!)
I've waiting for googlebot 3 months! Nothing hapened! Does google forget my sites?
Anybody has the same problem with me? Please help me!
This is just a stab in the dark but might the google bot be ping'ing each server before it connects? Which you just drop and so it moves on to the next link it's scraped. Does disabling icmp really buy you anything anyway? Might be worth re enabling it.
Interesting. It's the first time I've heard of an IP change causing a site to get yanked from Google. Have you tried writing them?
And no, it has nothing to do with ICMP. We block ICMP except on the mail servers IP address (some mail server software requires echo requests to verify a remote server), and have no problems with Google ranking.
yes, I've sent email to tell Google at [email protected] about my problem, and they suggested I should read guideline!
I know there's no Google cloaking here (because these site were good ranking on Google before moving to new server!), and there no blocking for standard web traffic (because other search engine can find and index my sites! MSN bot, Yahoo bot ...etc are indexing like crazy on my new server every day!)
So what is the problem? My customers continue asking me why their sites dont have good result (actually no result at all) on Google!
Hi JDMundo, I'm pretty sure there no my IPs on Google's blacklisted. Why? Because when I search on google like this:
site:mysite.com, it will return all pages for that site! But with url only instead of title, desc like other search results.
Same for all my sites in the new server. According to Google, if we search like this: "site:mysite.com", and if it says: there no result at all, then I'm sure that my site was banned. But as I said, there were results with my sites, but just url! People can not read url to understand what site is about :-)
I don't know what should I do to get googlebot come and index again!
The Googlebot is now very, very slow. Unfortunately it's possibly slower than most other search engines on the planet. It now takes on average around 3 months for a new site to be listed! Google is having quite a few problems with their search technology at the moment. I think it's something to do with when they doubled the index to 8billion pages (although I'm not sure).
The Googlebot should not require any other ports than port80. It functions like an "automatic web browser" that clicks any and all links it sees. If a browser can see the page, Googlebot should also be able to see it (offtopic: apart from frames and iframes).
My suggestion: try contacting Google again. They've been really helpfull to us when we've had similar problems being indexed by Google.
Yes!!!!!!!!!! "If a browser can see the page, Googlebot should also be able to see it " You are completely right! As I think, if the browser can see, so does the google! But why too long! Someone said because it uses IP cached for about 1 month, and I've waiting for 3 months! Anyway, I will try to contact Google again for my problem
(Actually I always think we don't have to open unecessary port like ICMP for googlebot!)
Chances are it has nothing to do with your ICMP or other server configs. Google saw the URL change to a new IP and knows there *may* have been a transfer of ownership for the site, and that it needs to be re-indexed. URLs that Googlebot knows about but have not been fully indexed (read, parsed, cached, etc) are listed without a description and such. Eventually Googlebot will catch up in its queue of sites to visit and fil in the details.