Google does use the IP address of your server as one of its factors in determining if your site is located in a particular geographical location and that in turn does markedly affect your sites rankings in that location.
You can check it out easily enough - just enter the same search term in several different Google domains (google.co.uk, google.com, google.cn etc) and look at the different results that appear.
Your TLD is also used in determining the location of your site and you can use Google webmaster tools to associate your site with a particular location.