Results 1 to 11 of 11
  1. #1
    Join Date
    Apr 2013
    Location
    Boston, MA
    Posts
    390

    Unhappy Hidden Files taking 100% of resources of the server

    Hello,

    When I log in my WHM and go to Server Status > Apache Status, I see something like this: http://d.pr/i/yg5f (It's a screenshot)

    Now, if I go to those locations, I can't find those folders. I've tried searching for them using grep but all I could find was lines on the access-log, like this:

    /home/capricho/access-logs/mydomain.com:1:187.11.243.174 - - [09/Dec/2013:16:10:11 -0200] "GET /.peide/ HTTP/1.1" 404 25904 "-" "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
    /home/capricho/access-logs/mydomain.com:2:179.208.77.63 - - [09/Dec/2013:16:10:11 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:3:187.2.126.241 - - [09/Dec/2013:16:10:12 -0200] "GET /.peide/ HTTP/1.1" 500 - "-" "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
    /home/capricho/access-logs/mydomain.com:4:200.161.233.99 - - [09/Dec/2013:16:10:11 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:5:200.17.161.191 - - [09/Dec/2013:16:10:11 -0200] "GET /.peide/ HTTP/1.1" 404 8808 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:6:186.212.12.119 - - [09/Dec/2013:16:10:11 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:7:177.75.218.181 - - [09/Dec/2013:16:10:12 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:8:186.219.113.92 - - [09/Dec/2013:16:10:12 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
    /home/capricho/access-logs/mydomain.com:9:189.41.143.188 - - [09/Dec/2013:16:10:13 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:10:177.157.160.210 - - [09/Dec/2013:16:10:13 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
    /home/capricho/access-logs/mydomain.com:11:189.2.184.100 - - [09/Dec/2013:16:10:13 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:12:179.223.165.130 - - [09/Dec/2013:16:10:14 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:13:179.104.97.224 - - [09/Dec/2013:16:10:14 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:14:189.54.167.4 - - [09/Dec/2013:16:10:17 -0200] "GET /.peide/ HTTP/1.1" 404 25904 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:15:186.215.195.195 - - [09/Dec/2013:16:10:18 -0200] "GET /.peide/ HTTP/1.1" 404 25904 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"
    /home/capricho/access-logs/mydomain.com:16:187.1.127.251 - - [09/Dec/2013:16:10:18 -0200] "GET /.peide/ HTTP/1.1" 404 25904 "-" "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
    /home/capricho/access-logs/mydomain.com:17:201.21.187.217 - - [09/Dec/2013:16:10:19 -0200] "GET /.peide/ HTTP/1.1" 404 240 "-" "Mozilla/4.0 (compatible; MSIE 6.0b; Windows 98; Win 9x 4.90)"


    These random 5 chars folders are in a lot of my domains. I've enabled mod_security and I can see it blocking a lot of them, but it can't block all of them.

    I managed to reduce the high load a little bit by setting up a new cPanel server with Cloudlinux, mod_security with paid Atomicorp rules, some security plugins and then I moved my websites to this new server, however the problem went to the new server with them. My server only is not dying because of Cloudlinux.

    I want to clean those folders but I can't find a way to do it. They are making the server very slow.

    is anyone here familiar to this?

    I contacted cPanel but Jesus (cPanel employee) said:

    Hello,

    Thank you for contacting cPanel support!

    From the looks of the logs, and my own review of the folder /home/domain/public_html, it looks like this site may have been either exploited through a security bug in wordpress, or the user's password may have been compromised. Unfortunately we can not provide security support for script level compromises. I've run a few diagnostics to ensure that your server is not compromised at a root level. You will need to seek out the services of a security specialist.

  2. #2
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,842
    Quote Originally Posted by v33usa View Post
    You will need to seek out the services of a security specialist.
    Good advice. Cleaning out the folders won't help - you need to fix whatever exploit is allowing their creation. I'd start by suspending the affected accounts as they're probably being used for some kind of illegal activity (spam / dos attacks etc.).
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  3. #3
    Join Date
    Mar 2009
    Location
    /home/khunj
    Posts
    432
    You cannot find them because they don't exist. Your log lines show a '404 Not Found' error:

    /home/capricho/access-logs/mydomain.com:1:187.11.243.174 - - [09/Dec/2013:16:10:11 -0200] "GET /.peide/ HTTP/1.1" 404 25904 "-" "Mozilla/4.0 (Windows; MSIE 6.0; Windows NT 5.2)"
    You could try to block the "MSIE 6.0" user-agent signature and return nothing (i.e., using mod_security 'drop' action), rather than returning 25904 bytes, though.
    NinTechNet
    ★ NinjaFirewall : Web Application Firewall for PHP and WordPress.
    ★ NinjaMonitoring : Monitor your website for suspicious activities.

  4. #4
    Not sure where cPanel the idea that you have been exploited.

    The server is returning 404, which means "Not found".

    Thus they are just attempts to hit a file which does not exist (Which is why you cannot find it.)

    This is most likely a server attack, perhaps of DOS/DDOS attempt.

    Your best defence against this will most likely be either (or all):
    - Fail2ban (With a filter to detect attempts at .peide and block the IP)
    - mod_evasive
    - mod_security (block user agent)
    - manual firewall blocking of each IP (A real pain in the butt....)

    But overall, I'd say you are most likely not exploited, just being DOS'd by someone who is angry with you for some reason.

    You may be able to prevent running out of resources also by disabling apaches keep-alive and tuning worker/prefork MPM to be more specific to the server.

    Hope this helps or clears up some stuff for you :-).

  5. #5
    Join Date
    Apr 2013
    Location
    Boston, MA
    Posts
    390
    Quote Originally Posted by khunj View Post
    You cannot find them because they don't exist. Your log lines show a '404 Not Found' error:



    You could try to block the "MSIE 6.0" user-agent signature and return nothing (i.e., using mod_security 'drop' action), rather than returning 25904 bytes, though.
    Quote Originally Posted by JakeMS View Post
    Not sure where cPanel the idea that you have been exploited.

    The server is returning 404, which means "Not found".

    Thus they are just attempts to hit a file which does not exist (Which is why you cannot find it.)

    This is most likely a server attack, perhaps of DOS/DDOS attempt.

    Your best defence against this will most likely be either (or all):
    - Fail2ban (With a filter to detect attempts at .peide and block the IP)
    - mod_evasive
    - mod_security (block user agent)
    - manual firewall blocking of each IP (A real pain in the butt....)

    But overall, I'd say you are most likely not exploited, just being DOS'd by someone who is angry with you for some reason.

    You may be able to prevent running out of resources also by disabling apaches keep-alive and tuning worker/prefork MPM to be more specific to the server.

    Hope this helps or clears up some stuff for you :-).
    Thanks for the information guys, I really appreciate it.

    I added these rules on mod_security so far and the load has decreased considerably:

    Code:
    SecRule REQUEST_HEADERS:User-Agent "^Mozilla/4.0 \(compatible; MSIE 6.0; Windows NT 5.1; SV1\)$" "log,drop,phase:1,id:111111111,msg:'Brute Force Attack Dropped'"
    
    SecRule REQUEST_HEADERS:User-Agent "^Mozilla/4.0 \(Windows; MSIE 6.0; Windows NT 5.2\)$" "log,drop,phase:1,id:111111112,msg:'Brute Force Attack Dropped'"
    
    SecRule REQUEST_HEADERS:User-Agent "^Mozilla/4.0 \(compatible; MSIE 6.0b; Windows 98; Win 9x 4.90\)$" "log,drop,phase:1,id:111111113,msg:'Brute Force Attack Dropped'"
    
    SecRule REQUEST_HEADERS:User-Agent "^Mozilla/4.0 \(Windows; MSIE 6.0; Windows NT 5.2\)$" "log,drop,phase:1,id:1111111114,msg:'Brute Force Attack Dropped'"
    
    SecRule REQUEST_METHOD "@streq POST" "chain,log,drop,phase:1,id:1111111115,msg:'Brute Force Attack Dropped'"
    SecRule REQUEST_URI "@beginsWith /.peide/"
    
    SecRule REQUEST_METHOD "@streq POST" "chain,log,drop,phase:1,id:1111111116,msg:'Brute Force Attack Dropped'"
    SecRule REQUEST_URI "@beginsWith /.xqhye/"
    
    SecRule REQUEST_METHOD "@streq POST" "chain,log,drop,phase:1,id:1111111117,msg:'Brute Force Attack Dropped'"
    SecRule REQUEST_URI "@beginsWith /.unzli/"
    
    SecRule REQUEST_METHOD "@streq POST" "chain,log,drop,phase:1,id:1111111118,msg:'Brute Force Attack Dropped'"
    SecRule REQUEST_URI "@beginsWith /.vhjus/"
    
    SecRule REQUEST_METHOD "@streq POST" "chain,log,drop,phase:1,id:1111111119,msg:'Brute Force Attack Dropped'"
    SecRule REQUEST_URI "@beginsWith /.vhjus/"
    However, some of these folders were indeed uploaded to my sites, and they were accessible on the net, which worries me that the server might still be infected.. I see that a lot of IPs are from DataShack, I've sent them abuse complaints but no response, so I blocked all their IP blocks on my firewall.

    Is there a way to make mod_security automatically block attackers' IPs on iptables?

    Thanks!!

  6. #6
    Join Date
    Apr 2013
    Location
    Boston, MA
    Posts
    390
    Nevermind, the rules above didn't solve it

    I will try the fail2ban method, but I have to search how I can do that first.

    Does anybody have all the IP blocks owned by DataShack?

    I blocked these in my firewall (csf), however a lot of IPs from DataShack are still passing through:

    Code:
    #Datashack IPs
    
    63.141.224.0/19
    69.197.185.0/24
    74.91.16.0/20
    107.150.32.0/19
    142.54.160.0/19
    173.208.138.0/24
    173.208.214.0/24
    173.208.215.0/24
    173.208.216.0/24
    173.208.218.0/24
    173.208.225.0/24
    173.208.226.0/24
    173.208.228.0/24
    192.151.144.0/20
    192.187.96.0/19
    198.204.224.0/19
    199.168.96.0/21
    192.187.99.0/24
    
    63.141.241.0/24
    69.30.192.0/18
    69.30.244.0/24
    69.197.128.0/18
    72.22.211.0/24
    72.22.215.0/24
    72.22.220.0/24
    107.151.64.0/18
    173.208.128.0/17
    173.208.180.0/24
    173.208.250.0/24
    190.5.208.0/22
    192.187.113.0/24
    198.204.240.0/21
    199.47.192.0/21
    199.245.59.0/24
    204.12.192.0/18
    
    #Datashack IPs

  7. #7
    Join Date
    Nov 2013
    Posts
    103
    Quote Originally Posted by JakeMS View Post
    The server is returning 404, which means "Not found".
    That doesn't mean anything. The file sizes are different and they should all be the same if it were really a 404 Not found response page. And there is even a code 500. Code 500 should never appear for regular static files.

    Either the web server is compromised, or there is an application responding configured to take URL paths. Eg. application URIs vs. static files.

    Those are probably payloads to already infected "clients" or the infections themselves (eg. payloads called by javascript). Check the contents of the response with tcpdump and grep.

  8. #8
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,842
    Did you seriously think a couple of forum users looking at a tiny fragment of apache log would know better than the cPanel staffer who's logged into your server and examined the compromised account?

    At this stage, firewall blocks and mod_security aren't going to help. You need to fix the root cause: the exploited user account(s). If you don't know how to do that then hire someone who does.
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  9. #9
    just a quick note: ls -al should show any hidden files and folders in a directory.
    Tara Roberts
    www.whmxtra.com

  10. #10
    Join Date
    Jan 2011
    Location
    Canada
    Posts
    934
    Quote Originally Posted by v33usa View Post
    However, some of these folders were indeed uploaded to my sites, and they were accessible on the net, which worries me that the server might still be infected.. I see that a lot of IPs are from DataShack, I've sent them abuse complaints but no response, so I blocked all their IP blocks on my firewall.
    It's futile to just ban DataShack IPs. From your first post, it looks like a botnet is using your server as a C&C - likely spam given the unusually higher number of .br IPs.

    A root scan is unlikely to find the backdoor that is installed. Neither will mod-security at this time. Get a security expert to look at your files. They should at the very least do a diff compare against all wordpress files against the latest and give you a clean package to reinstall a new secured server.

  11. #11
    Join Date
    Aug 2009
    Location
    Metro Detroit Area
    Posts
    1,619
    Quote Originally Posted by foobic View Post
    Did you seriously think a couple of forum users looking at a tiny fragment of apache log would know better than the cPanel staffer who's logged into your server and examined the compromised account?

    At this stage, firewall blocks and mod_security aren't going to help. You need to fix the root cause: the exploited user account(s). If you don't know how to do that then hire someone who does.
    ^^ This. Absolutely.
    HostMantis Affordable Web Hosting
    Shared Reseller VPS 24/7/365 Support Instant Activation
    CloudLinux CloudFlare Softaculous Premium Multi PHP cPanel
    Also offering Windows Hosting with Plesk 12 Multi PHP MSSQL ASP.NET

Similar Threads

  1. Replies: 3
    Last Post: 02-22-2010, 11:27 PM
  2. Hidden Files and Directories
    By Mechromancer in forum Hosting Security and Technology
    Replies: 2
    Last Post: 01-25-2007, 04:01 PM
  3. Java taking too much server resources.
    By ahmedaf in forum Hosting Security and Technology
    Replies: 1
    Last Post: 02-17-2006, 12:31 PM
  4. I can see hidden files through cuteftp why?
    By terran11355@ in forum Hosting Security and Technology
    Replies: 9
    Last Post: 10-12-2004, 02:07 PM
  5. PHP versus Plain HTML files & Server Resources
    By ochiba in forum Web Hosting
    Replies: 9
    Last Post: 03-22-2003, 10:59 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •