I am noticing a weird mod_bandwidth related issue on my server right now, and I'm wondering if anyone can share some experties to help me sort this out.

Due to bandwidth restriction problems, I decided to put in mod_bandwidth on my server to help cut the bandwidth usage on a few file mirroring that I offer. I installed mod_bandwidth into apache and it seems to work without causing any crash, which is a good sign. I then proceeded to add this segment into the domain's section of /usr/local/apache/conf/httpd.conf

<VirtualHost 83.223.xxx.xxx>
<IfModule mod_bandwidth.c>
<Directory /home/psro/public_html/files>
BandWidthModule On
BandWidth all 131072
BandWidthPulse 1000000
I tested to grab files from that directory, and it does, indeed work as it suppose to (push data to me at ~128kb/s), so I was happy. But after a few days, I received complaints from people that they can't seem to download faster than 5kb/s from the directory. I tried to remove that segment and restarted apache, and people reflected that it works as if there's no cap, then putting it back on, and it went back down to 5kb/s. Is there something wrong with the above? Or is it their ISP's problem causing the download to mess up? Is there anything I can do to prevent the weird clog down from happening?