1. there are lots of things that can go on from your connection to your site. many networks are in between usually. first thing to do is to do a "traceroute" to the ip address and see how the route is going.
2. if you see any timesouts, try to ping thru the time out points and see if there is actual congestion.
3. everyday at 2pm may be the high point for your isp and thus their pipes "could" become saturated at that point.
however, there is no conclusion as there could be a number of reasons why your site goes down everyday.
but I do not understand the ISP Cache topic....what do you mean by this?