Results 1 to 21 of 21
Thread: Server Completely down
-
12-28-2013, 03:14 PM #1Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
Server Completely down
Greetings pals,
Please I really need someone here ot help me on my vps server.
The server is completely down and even ssh can't login.
I contacted my vps provider and here is their reply
"Seems your VPS is running with swap memory and it seems the server is getting high connections.
Please verify the snippet attached below :
=========
root@bestservers [~]# w
19:04:41 up 7 min, 1 user, load average: 120.86, 63.30, 26.02
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/1 122.167.99.132 18:58 21.00s 0.15s 0.02s w
root@bestservers [~]# free -m
total used free shared buffers cached
Mem: 2010 2001 8 0 4 47
-/+ buffers/cache: 1949 60
Swap: 3999 3334 665
=============
You need to investigate the issue and resolve it.
Hope you can understand."
I have no idea how to resolve this since the server is only accessible via VNC and I am not quite familiar with that.
I guess I caused this because the last thing I did on the server bfore it got crashed in increasing the php memory limit to 100MB. it was formally 32MB and I guess that is what is affecting the server now.
Can anybody here please assist and help me on this ? Please.
Thanks
-
12-28-2013, 04:31 PM #2Web Hosting Master
- Join Date
- Mar 2012
- Posts
- 1,421
I think that your best bet is to go to the Server Management offer section, there you will find some good individuals/companies that knows what they are doing.
--
-
12-28-2013, 05:38 PM #3WHT Addict
- Join Date
- Nov 2003
- Location
- Texas, United States
- Posts
- 115
If you and/or your provider can access the console, copy down the messages that may be displayed there. Also, try logging in through the console (it may still be very slow given your resource usage). If even console login doesn't work, reboot your VPS (or ask your provider to do it if you do not have the access necessary). That should clear up the immediate issue, allowing you (or a System Administration provider) to then gain access to your server and look around.
► Tim Benoit
-
12-28-2013, 05:44 PM #4Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
Yeah, I actually had my service provider to reboot the vps and stop cpane, sql and apache.
this gave me access to ssh.
While on ssh, I started cpanel, logged into whm and suspended the accounts with high cpu usage.and then started sql and other services.
this was eble to resolve the issue and on ssh, I run this command
/usr/local/cpanel/3rdparty/bin/mysqltuner.pl
Which gave me this recommendations
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Reduce your overall MySQL memory footprint for system stability
Enable the slow query log to troubleshoot bad queries
Adjust your join queries to always utilize indexes
Temporary table size is already large - reduce result set size
Reduce your SELECT DISTINCT queries without LIMIT clauses
Variables to adjust:
*** MySQL's maximum memory usage is dangerously high ***
*** Add RAM before increasing MySQL buffer variables ***
query_cache_limit (> 1M, or use smaller result sets)
join_buffer_size (> 32.0M, or always use indexes with joins)
So I am checking around now how to follow this recommendation and limit the MYSQL CPU RESOURCE USAGE and also MySQL's maximum memory before enabling those accounts back.
but the overall server performance is back up now.
hope this help someone.
Thanks
-
12-28-2013, 05:49 PM #5WHT Addict
- Join Date
- Nov 2003
- Location
- Texas, United States
- Posts
- 115
To modify the MySQL memory settings you would need to edit the /etc/my.cnf file and restart MySQL. There are many settings that are, or can be, in that file that affect memory utilization. MySQL has generic my-*.cnf files to use as a guideline typically stored in /usr/share/mysql/
Depending on the number of sites on your server, and what those sites are doing, you may need to increase your RAM above 2GB.► Tim Benoit
-
12-28-2013, 05:54 PM #6Mostly Retired!
- Join Date
- Nov 2002
- Location
- Portland, Oregon
- Posts
- 2,992
Most of these adjustments are made in /usr/local/lib/php.ini or my.cnf. Make a copy of these files before making any changes to it, even if it's not correctly configured -- as a rollback measure.
-
12-28-2013, 06:36 PM #7Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
Here is my my.cnf file configuration
[mysqld]
default-storage-engine=myisam
low_priority_updates=1
max_connections = 100
key_buffer = 192M
tmpdir=/home/tmp
skip-external-locking
max_allowed_packet = 16M
table_cache = 4096
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 2M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_type = 1
query_cache_size= 128M
query_cache_limit = 1M
thread_concurrency = 8
wait_timeout = 30
open_files_limit=17506
tmp_table_size = 256M
max_heap_table_size = 256M
join_buffer_size = 32M
table_definition_cache=1024
slow-query-log=/var/lib/mysql/slow.log
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 2M
local-infile=0
And here is the details from mysqltuner
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.5.32-cll
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 2G (Tables: 2560)
[--] Data in InnoDB tables: 15M (Tables: 581)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
[--] Data in MEMORY tables: 0B (Tables: 4)
[!!] Total fragmented tables: 573
-------- Performance Metrics -------------------------------------------------
[--] Up for: 1h 17m 54s (28K q [6.196 qps], 944 conn, TX: 117M, RX: 4M)
[--] Reads / Writes: 87% / 13%
[--] Total buffers: 720.0M global + 38.2M per thread (100 max threads)
[!!] Maximum possible memory usage: 4.4G (226% of installed RAM)
[OK] Slow queries: 0% (2/28K)
[OK] Highest usage of available connections: 8% (8/100)
[OK] Key buffer size / total MyISAM indexes: 192.0M/376.6M
[OK] Key buffer hit rate: 100.0% (21M cached / 2K reads)
[OK] Query cache efficiency: 67.8% (16K cached / 24K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 1K sorts)
[!!] Joins performed without indexes: 74
[OK] Temporary tables created on disk: 23% (785 on disk / 3K total)
[OK] Thread cache hit rate: 99% (8 created / 944 connections)
[!!] Table cache hit rate: 5% (3K open / 54K opened)
[OK] Open file limit used: 29% (5K/17K)
[OK] Table locks acquired immediately: 100% (10K immediate / 10K locks)
[OK] InnoDB data size / buffer pool: 15.6M/128.0M
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Reduce your overall MySQL memory footprint for system stability
Enable the slow query log to troubleshoot bad queries
Adjust your join queries to always utilize indexes
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
*** MySQL's maximum memory usage is dangerously high ***
*** Add RAM before increasing MySQL buffer variables ***
join_buffer_size (> 32.0M, or always use indexes with joins)
table_cache (> 4096)
I would appreciate it if you can guide me on the right part to edit in the cnf file.
Thanks
-
12-28-2013, 07:07 PM #8WHT Addict
- Join Date
- Nov 2003
- Location
- Texas, United States
- Posts
- 115
It highlighted the immediate issues fairly well:
1) Lots of tables are fragmented. This could be OK or could be bad. It would be a good idea to optimize them.
2) You have MySQL configured to use to to 4.4GB of RAM, however your server only has a total of 2GB of RAM. You will need to tweak down your memory parameters in my.cnf (use the example files I mentioned previously) and/or upgrade the RAM in your server. These would be for the *buffer* and *cache* variables in the file.
3) There have been some joins performed on non-indexed fields in the database. This means the query was performing a full table scan for those queries. Not optimal but may be the responsibility of the script developer to optimize their queries.
You could also enable slow query logging to help show you what queries performed were running "slow".► Tim Benoit
-
12-28-2013, 07:18 PM #9Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
Hello,
thanks alot for your support.'Just that I am lost here.
Can you please tell me where excatly I should edit on the cnf file ? if possible, please make the changes and give me the right one to use so that I can just over right the existing one.
Sorry for my noob questions please.
Thanks
-
12-28-2013, 08:31 PM #10WHT Addict
- Join Date
- Nov 2003
- Location
- Texas, United States
- Posts
- 115
Sent you a PM
► Tim Benoit
-
12-28-2013, 08:41 PM #11Poooooonnyyy :*
- Join Date
- Jan 2003
- Location
- Canada
- Posts
- 5,073
You're 3GB into swap.
You need to upgrade your RAM.
FranciscoBuyVM - OpenVZ & KVM Based VPS Servers - Chat with us
- All popular VPN methods supported
- Affordable offloaded MySQL & DDoS protection
- 5GB backup space, unmetered private LAN bandwidth & native IPv6 included. All with a strong serving of pony
-
12-29-2013, 12:15 AM #12Web Hosting Master
- Join Date
- Aug 2009
- Posts
- 3,207
it is a bit unusual when your server eat up such amount of swap. Either your server or the host node run out of memory...
█ ControlVM.com :: Elastic Cloud Hosting Alternatives to VPS ● Reliable Cloud Server
█ Enterprise Hosting @ Malaysia ● Germany ● USA ● Singapore
We Accept: Paypal ● Alipay ● GrabPay ● Credit/Debit Card ● FPX Bank Transfer ● Bank TT
-
12-29-2013, 12:24 AM #13Web Hosting Master
- Join Date
- Dec 2007
- Location
- LocalHost
- Posts
- 1,317
Can you please post current load and memory usage
Code:top
Code:free -m
█ YagHost - Fast Reliable Hosting Since 2009
█ Managed VPS - NVMe DirectAdmin
█ Web Hosting - NVMe SSD, AMD EPYC, 10 Gbps (US, Europe, Singapore)
-
12-29-2013, 02:51 AM #14Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
root@bestservers [~]# top
top - 07:50:27 up 10:26, 1 user, load average: 0.07, 0.06, 0.07
Tasks: 201 total, 1 running, 198 sleeping, 0 stopped, 2 zombie
Cpu(s): 0.4%us, 0.3%sy, 0.0%ni, 98.7%id, 0.3%wa, 0.3%hi, 0.1%si, 0.0%st
Mem: 2058696k total, 1882508k used, 176188k free, 87584k buffers
Swap: 4095992k total, 112k used, 4095880k free, 840980k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14434 root 15 0 12764 1196 840 R 0.3 0.1 0:00.26 top
1 root 15 0 10372 696 588 S 0.0 0.0 0:01.82 init
2 root RT -5 0 0 0 S 0.0 0.0 0:00.57 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0.0 0.0 0:00.51 migration/1
6 root 34 19 0 0 0 S 0.0 0.0 0:00.01 ksoftirqd/1
7 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
8 root RT -5 0 0 0 S 0.0 0.0 0:00.47 migration/2
9 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/2
10 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/2
11 root RT -5 0 0 0 S 0.0 0.0 0:00.57 migration/3
12 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/3
13 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/3
14 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/0
15 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1
16 root 10 -5 0 0 0 S 0.0 0.0 0:00.05 events/2
-
12-29-2013, 04:17 AM #15Web Hosting Master
- Join Date
- Dec 2007
- Location
- LocalHost
- Posts
- 1,317
█ YagHost - Fast Reliable Hosting Since 2009
█ Managed VPS - NVMe DirectAdmin
█ Web Hosting - NVMe SSD, AMD EPYC, 10 Gbps (US, Europe, Singapore)
-
12-29-2013, 11:32 AM #16Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
I noticed the site eating up the memory so I had to suspend them....
I just enabled them now and the server is dead again
-
12-29-2013, 11:44 AM #17Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
Please look into my cnf above and give me the recommended settings to use please.
The server is on Xen vitualization with Centus 5
-
12-29-2013, 02:54 PM #18Newbie
- Join Date
- Jun 2009
- Location
- Catalonia
- Posts
- 15
Hello,
I think there are several values too high, particularly join_buffer_size = 32M. The default value for this variable is 128K. Low it to 1 or 2MB and, I think, at least you will be able to start Mysql without crashing the server. You can see how a change in a variable can affect your memory consumption here: http://www.mysqlcalculator.com/.
Also, install and run the script tuning-primer.sh and see (after 48 hours running Mysql) what it recommends for optimizing your Mysql, you can compare it with Mysqltuner results. http://webhostingneeds.com/Install_M..._Primer_Script
The maximum memory usage displayed by Mysqltuner should not exceed 50% of available installed RAM, you can run Mysqltuner after changing any value to see if it fits in this limit.
Good luck.
-
12-29-2013, 10:26 PM #19Junior Guru Wannabe
- Join Date
- Apr 2012
- Posts
- 75
my.cnf file edited to the following
[mysqld]
default-storage-engine=myisam
low_priority_updates=1
max_connections = 100
key_buffer = 192M
tmpdir=/home/tmp
skip-external-locking
max_allowed_packet = 1M
table_cache = 2M
sort_buffer_size = 1M
read_buffer_size = 1M
read_rnd_buffer_size = 2M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_type = 1
query_cache_size= 64M
query_cache_limit = 1M
thread_concurrency = 8
wait_timeout = 30
open_files_limit=17506
tmp_table_size = 256M
max_heap_table_size = 256M
join_buffer_size = 1M
table_definition_cache=1024
slow-query-log=/var/lib/mysql/slow.log
[mysqldump]
quick
max_allowed_packet = 2M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 1M
write_buffer = 1M
[myisamchk]
key_buffer = 128M
sort_buffer_size = 128M
read_buffer = 1M
Please help on how o optimize further.
Thanks All
-
12-29-2013, 10:40 PM #20Web Hosting Master
- Join Date
- Dec 2007
- Location
- LocalHost
- Posts
- 1,317
Try enabling sites one-by-one. And find sites which are causing most problem.
Optimization can work upto certain limits only. If you websites need more RAM / CPU, you should consider upgrading your VPS.
I suggest you to keep heavy websites on a separate VPS so that your normal websites can run smoothly.█ YagHost - Fast Reliable Hosting Since 2009
█ Managed VPS - NVMe DirectAdmin
█ Web Hosting - NVMe SSD, AMD EPYC, 10 Gbps (US, Europe, Singapore)
-
12-29-2013, 10:43 PM #21Web Hosting Master
- Join Date
- Mar 2012
- Posts
- 1,421
Just a quick note here:
You have been dealing a few days with this. While is good to ask for help in the forum, nothing bad about this, if you had chosen to request service from a "Server Management" company/business, you could have used this time to manage other more important things, other than doing the trial and error and experimenting.
IMHO.--
Similar Threads
-
My server just completely overloaded? DDOS?
By chasebug in forum Hosting Security and TechnologyReplies: 5Last Post: 09-19-2010, 12:23 PM -
Server At PixelFX Completely Down!!
By chiehkai in forum Providers and Network Outages and UpdatesReplies: 8Last Post: 07-05-2006, 08:41 PM -
Server completely messed up.
By Coldmedia in forum Dedicated ServerReplies: 2Last Post: 04-09-2006, 01:32 PM -
remove domain name from server completely
By mshannaq in forum Dedicated ServerReplies: 1Last Post: 09-15-2005, 05:37 AM -
Please help me; I need a server pronto but am completely clueless
By D-elizabeth in forum Dedicated ServerReplies: 10Last Post: 12-11-2003, 12:11 AM