Here is my server's CPU Usage:

Over the past fifteen minutes, the CPU has been heavily loaded.

15 minute CPU load average is 8.66
The processor is currently 40 degrees Celsius.
The system has been running for 1 day, 19 hours, and 55 minutes.

My questions:

Should I worry about this, or I have to take any action immediately ?
What max temperature can a CPU of RaQ4r stand for?
When a CPU is heavily loaded, what and how should I do to prevent it from being damaged? Is reboot the server suitable ?

Your info is not enough for me to give some comment.
Run "top" command and show us like the following data.

4:39am up 13:36, 25 users, load average: 0.72, 0.54, 0.55
99 processes: 98 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 2.8% user, 1.8% system, 0.0% nice, 95.3% idle
Mem: 257844K av, 251468K used, 6376K free, 97532K shrd, 102684K buff
Swap: 128484K av, 5732K used, 122752K free 82056K cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
1 root 0 0 108 56 40 S 0 0.0 0.0 0:05 init
2 root 0 0 0 0 0 SW 0 0.0 0.0 0:00 kflushd
3 root 0 0 0 0 0 SW 0 0.0 0.0 0:00 kpiod

Generally I would keep load average under 1.50

I think it is OK now.
------------------------------------------------
1:10am up 2 days, 4:06, 1 user, load average: 1.20, 1.32, 1.36
126 processes: 122 sleeping, 3 running, 1 zombie, 0 stopped
CPU states: 14.9% user, 19.5% system, 0.0% nice, 65.5% idle
Mem: 257816K av, 226180K used, 31636K free, 458832K shrd, 17008K buff
Swap: 131448K av, 27668K used, 103780K free 85248K cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
15650 admin 15 0 912 912 668 R 0 3.2 0.3 0:03 top
15995 playchin 16 0 1216 1216 836 R 0 2.1 0.4 0:00 photo.cgi
15994 wmanets 15 0 1188 1188 940 R 0 1.9 0.4 0:00 random.cgi
15992 wmanets 15 0 0 0 0 Z 0 1.7 0.0 0:00 random.cgi <
10846 httpd 1 0 14296 9160 7920 S 0 0.3 3.5 0:01 httpd
10853 httpd 2 0 14296 9156 7924 S 0 0.3 3.5 0:01 httpd
11523 httpd 6 0 14908 10M 8564 S 0 0.3 3.9 0:01 httpd
15750 httpd 11 0 14564 9412 8132 S 0 0.3 3.6 0:00 httpd
10838 httpd 0 0 14484 9348 7960 S 0 0.1 3.6 0:00 httpd
10841 httpd 11 0 14744 9.9M 8608 S 0 0.1 3.9 0:01 httpd
10856 httpd 5 0 15676 10M 8136 S 0 0.1 4.0 0:01 httpd
10862 httpd 6 0 14296 9160 7904 S 0 0.1 3.5 0:01 httpd
10878 httpd 7 0 14484 9352 7940 S 0 0.1 3.6 0:01 httpd
10881 httpd 0 0 14296 9160 7936 S 0 0.1 3.5 0:01 httpd
10898 httpd 7 0 14548 9448 7856 S 0 0.1 3.6 0:01 httpd
10911 httpd 5 0 14308 9168 7924 S 0 0.1 3.5 0:00 httpd
12336 httpd 10 0 14488 9352 7956 S 0 0.1 3.6 0:00 httpd