Results 1 to 8 of 8
Thread: prevent wget downloads
-
08-25-2005, 06:22 AM #1Junior Guru Wannabe
- Join Date
- Feb 2003
- Posts
- 34
prevent wget downloads
hi,
how to prevent wget downloads of my website
help required
thank you
-
08-25-2005, 07:03 AM #2Web Hosting Guru
- Join Date
- Jul 2004
- Location
- Reporting Live from Marrz
- Posts
- 257
.htaccess:
Code:SetEnvIfNoCase User-Agent "^Wget" bad_bot <Limit GET POST> Order Allow,Deny Allow from all Deny from env=bad_bot </Limit>
-
08-25-2005, 07:58 AM #3Junior Guru Wannabe
- Join Date
- Feb 2003
- Posts
- 34
I have figured out.. thank you "SupaDucta"
SetEnvIfNoCase User-Agent "^Wget" bad_bot
SetEnvIfNoCase User-Agent "^Wget/1.5.3" bad_bot
SetEnvIfNoCase User-Agent "^Wget/1.6" bad_bot
<Files ~ "\.(html|pdf|mp3|zip|rar|exe|gif|jpe?g|png|php|jsp) $">
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</files>
-
08-25-2005, 09:14 AM #4Web Hosting Guru
- Join Date
- Jul 2004
- Location
- Reporting Live from Marrz
- Posts
- 257
This line:
Code:SetEnvIfNoCase User-Agent "^Wget" bad_bot
Code:SetEnvIfNoCase User-Agent "^Wget/1.5.3" bad_bot SetEnvIfNoCase User-Agent "^Wget/1.6" bad_bot
Code:<Limit GET POST> Order Allow,Deny Allow from all Deny from env=bad_bot </Limit>
While this:
Code:<Files ~ "\.(html|pdf|mp3|zip|rar|exe|gif|jpe?g|png|php|jsp) $"> Order Allow,Deny Allow from all Deny from env=bad_bot </files>
-
08-26-2005, 12:12 AM #5Web Hosting Master
- Join Date
- Dec 2002
- Posts
- 1,304
Just remember that you cannot stop a determined person this way.
Having the source code to wget, I could probably easily substitue IE 5.5 for the user-agent string. This would deter most people though.
Also there is never any need to put php files in your list of non-allowed materials, since they cannot download your source (interpreted by the server first)."The only difference between a poor person and a rich person is what they do in their spare time."
"If youth is wasted on the young, then retirement is wasted on the old"
-
08-26-2005, 01:01 AM #6Web Hosting Master
- Join Date
- Nov 2001
- Posts
- 551
You don't need the source code... it is built in:
From the wget manual
`-U agent-string'
`--user-agent=agent-string'
Identify as agent-string to the HTTP server. The HTTP protocol allows the clients to identify themselves using a User-Agent header field. This enables distinguishing the WWW software, usually for statistical purposes or for tracing of protocol violations. Wget normally identifies as `Wget/version', version being the current version number of Wget. However, some sites have been known to impose the policy of tailoring the output according to the User-Agent-supplied information. While conceptually this is not such a bad idea, it has been abused by servers denying information to clients other than Mozilla or MS IE. In these cases it may be useful to "fake" the user-agent with this option. In the following example Wget masquerades as Mozilla 4.03 running on Solaris.
wget -U "Mozilla/4.03 [en] (X11; I; SunOS 5.5.1 sun4u)"
Use of this option is discouraged, unless you really know what you are doing.--
-
08-26-2005, 10:56 AM #7Web Hosting Master
- Join Date
- Dec 2002
- Posts
- 1,304
Touche
I suppose I could have taken 2 second to read the man pages, heh."The only difference between a poor person and a rich person is what they do in their spare time."
"If youth is wasted on the young, then retirement is wasted on the old"
-
06-19-2012, 02:43 AM #8New Member
- Join Date
- Mar 2012
- Posts
- 2
I am using this
But my Bandwidth Limit Exceeded.
my Internal server Error page hits 2,028,069 and use 2.79 GB
Wget hits 360,006
Googlebot hits 57,130 uses 82.91 GB Bandwidth.
what can i do?