I'm in the process of testing new code for our load balancers. I setup the lab as usual, make sure everything is working, and load the new code.

Everything seems to update fine, but one of the servers is failing health checks. I can ping it from the load balancer, but the health checks cannot reach it.

So, I doulbe and triple check the configuration, nothing. I wipe the config and rebuild it from scratch, still nothing. Okay, I figure it must be a bug in the code, so I revert back to the old code -- the problem is still there :(.

Finally, it occurs to me to make sure Apache is actually still running on the server...and, of course, it is not.

Today has long day written all over it.