I have a VPS and I run daily backups via WHM. My home directory is 2.3GB in 16 accounts and I am backing up to another directory on the same drive (to be synced to Amazon S3 once the backup is done)

It takes cpbackup between 3 and 4 hours to complete the backup. Is that typical? Seems like it's awful slow.

Should I even care?

cPanel has a load safeguard where if the server is busy it will delay the backup process until the load is back to normal. So if your server is under a high load while the backup process is running, it might take hours. How many cores does your server have? What's the load like (sar -q)? Also, have you changed the extracpus=0 in /var/cpanel/cpanel.conf to reflect the correct number of cores/cpus? The reason being the load checker makes decision using this variable and could run sub optimally.

And yes, I think you should care. Your backup snapshot might not be consistent if it takes 4-5 hours to complete.

Regards
Joe / UNIXy

Hello,

There must be high load on server that caused delay for backup. What type of backup do you use Incremental or compressed?

Thanks for the input.

Heres the load output (thanks for the command, still learning linux..)12:00:06 AM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
12:10:01 AM 1 69 0.12 0.08 0.08
12:20:01 AM 1 69 0.00 0.02 0.04
12:30:01 AM 2 71 0.08 0.04 0.01
12:40:01 AM 1 66 0.00 0.05 0.02
12:50:01 AM 2 70 0.02 0.15 0.08
01:00:01 AM 1 71 0.22 0.09 0.06
01:10:01 AM 1 75 1.08 0.93 0.51
01:20:05 AM 0 79 2.31 2.14 1.29
01:30:02 AM 10 74 2.30 2.54 1.92
01:40:05 AM 5 73 2.13 2.65 2.33
01:50:06 AM 0 78 2.32 2.43 2.37
02:00:05 AM 1 81 2.81 2.62 2.46
02:10:08 AM 16 81 4.39 4.05 3.36
02:20:01 AM 1 81 3.02 3.04 3.09
02:30:02 AM 0 76 2.29 2.41 2.76
02:40:01 AM 2 76 3.05 2.75 2.72
02:50:01 AM 0 71 1.04 1.46 2.10
03:00:01 AM 2 70 1.37 1.23 1.65
03:10:01 AM 0 72 1.16 1.20 1.42
03:20:07 AM 10 72 2.76 1.93 1.73
03:30:05 AM 0 71 2.60 2.04 1.79
03:40:01 AM 1 73 1.21 1.43 1.56
03:50:02 AM 0 77 1.88 1.64 1.60
04:00:02 AM 0 87 3.46 3.13 2.38
04:10:02 AM 0 88 4.43 4.82 3.70
04:20:05 AM 0 77 2.79 3.75 3.73
04:30:01 AM 1 75 1.01 1.66 2.66
04:40:01 AM 1 76 1.11 1.08 1.84
04:50:02 AM 0 70 0.34 0.81 1.37
05:00:01 AM 5 69 0.04 0.16 0.73
05:10:07 AM 0 70 0.37 0.20 0.45
05:20:01 AM 1 69 0.29 0.12 0.27
Backup started at 1:15 and was done at 4:30

The spike at 4:00 is because of another cron that is started, which I will be moving to a later time.

cat /proc/cpuinfo shows 8 processors:
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4009.17
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

processor : 1
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4001.06
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

processor : 2
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4000.42
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

processor : 3
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 0
siblings : 4
core id : 3
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4000.00
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

processor : 4
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 1
siblings : 4
core id : 0
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4000.59
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

processor : 5
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 1
siblings : 4
core id : 1
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4000.83
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

processor : 6
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 1
siblings : 4
core id : 2
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4000.84
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

processor : 7
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350
stepping : 3
cpu MHz : 2000.090
cache size : 512 KB
physical id : 1
siblings : 4
core id : 3
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 4000.82
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

I didnt know about the extracpus thing. It's currently set to zero. I should change that to 8?


I am running compressed backups.

Thanks,
Mike

Hmm, looking closer, it seems its 2 quad core processors.

I didnt know about the extracpus thing. It's currently set to zero. I should change that to 8?


I am running compressed backups.

Thanks,
Mike

Hi Mike,

Yes, you have 8 cpus so set extracpus accordingly to 7 (7 extracpus) and restart cPanel (/etc/init.d/cpanel restart). The backup process will run relatively faster now. In addition, is your VPS configured with the correct timezone (run: date or zdump /etc/localtime)? A few hours can make a difference especially when traffic spikes up around backup kickoff time. Or push the backup to run say at 2AM/3AM if that helps (as root run: crontab -l|grep cpbackup).

Regards
Joe / UNIXy

Thanks! I have updated extracpus. Timezone is correct (previously fixed that).

Though I'm curious if this will change anything as I remember reading elsewhere of a problem that someone was having with load, and in the backup log is said something to the effect that the process was waiting for the load to come down before it would continue. I never see anything like that in my backup log. Guess we'll see tonight..

Thanks again!

Guess we'll see tonight..


Hi Mike,

Have you seen any improvement?

Regards
Joe / UNIXy

Yes and no. The following day, the backup still took 3.5 hours.
The day after that it only took 1:20, but then it was back to 3 hours the next day. Yesterday it took nearly 4.5 hours.

There might be some other factors that are slowing things down...in particular, other customers on the same VPS box. So I'm looking into other possibilities..and will ask the host to look into it as well to see if there is something going on on that box that is hogging resources.

Thanks for the help.

Yes and no. The following day, the backup still took 3.5 hours.
The day after that it only took 1:20, but then it was back to 3 hours the next day. Yesterday it took nearly 4.5 hours.

There might be some other factors that are slowing things down...in particular, other customers on the same VPS box. So I'm looking into other possibilities..and will ask the host to look into it as well to see if there is something going on on that box that is hogging resources.

Thanks for the help.

It could be disk IO problems too. Check the %wa (use top to find it).

So the whole box was rebooted early this morning because "the raid controller was being reported as offline". It happened with minutes of my normal backup time, so the backup did not complete.

I ran the backup script manually this morning. You know how long it took? 11 minutes! So there is a serious problem on this box. Who knows, maybe everyone is doing their backups at the same time so disk IO is through the roof. Gonna put a support ticket in. Thanks for the help.