Hello,

My RAID5 is composed of
Disque Dur SCA Max6 : 3 IBM/Hitachi 36 Go U320 10000 RPM 80 pins DK32EJ
Carte SCSI : 1 Adaptec 2120S RAID 0,1,5
I'am running with a kernel 2.4.26-grsec but I don't know if I have forgotten an option?

I don't know why it is so slow even if only few people are online on the website

hdparm -tT /dev/sda1

/dev/sda1:
Timing buffer-cache reads: 128 MB in 0.21 seconds =609.52 MB/sec
Timing buffered disk reads: 64 MB in 20.26 seconds = 3.16 MB/sec

I seeked what happened with afacli

AFA0>
COMMAND: container list
Executing: container list
Cluster Num Total Oth Chunk Scsi Partition
Partner Label Type Size Ctr Size Usage B:ID:L Offset:Size
------- ----- ------ ------ --- ------ ------- ------ -------------
0 0 RAID-5 68.3GB 64KB Valid 0:00:0 64.0KB:34.1GB
/dev/sda scsi 0:01:0 64.0KB:34.1GB
0:02:0 64.0KB:34.1GB


AFA0>
COMMAND: disk list
Executing: disk list

B:ID:L Device Type Blocks Bytes/Block Usage Shared
------ -------------- --------- ----------- ---------------- ------
0:00:0 Disk 71687340 512 Initialized NO
0:01:0 Disk 71687340 512 Initialized NO
0:02:0 Disk 71687340 512 Initialized NO

ontroller details
Executing: controller details
Controller Information
----------------------
Remote Computer: .
Device Name: AFA0
Controller Type: No Info
Access Mode: READ-WRITE
Controller Serial Number: Last Six Digits = BA980E
Number of Buses: 1
Devices per Bus: 15
Controller CPU: i960 R series
Controller CPU Speed: 100 Mhz
Controller Memory: 64 Mbytes
Battery State: Not Present

Component Revisions
-------------------
CLI: 2.1-5 (Build #3898)
API: 2.1-5 (Build #3898)
Miniport Driver: 1.1-0 Beta (Build #9999)
Controller Software: 4.0-0 (Build #6011)
Controller BIOS: 4.0-0 (Build #6011)
Controller Firmware: (Build #6011)
Controller Hardware: 2.64

task list

Controller Tasks

TaskId Function Done% Container State Specific1 Specific2
------ -------- ------- --------- ----- --------- ---------

No tasks currently running on controller


or this: 1:01.0 RAID bus controller: Adaptec AAC-RAID (rev 01)
Subsystem: Adaptec: Unknown device 0286
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 32, cache line size 10
Interrupt: pin A routed to IRQ 11
Region 0: Memory at f8000000 (32-bit, prefetchable) [size=64M]
Expansion ROM at feae0000 [disabled] [size=64K]
Capabilities: [80] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

However I don't understand. What's wrong?

Thanks
Best Regards,

a dmesg output would be helpful.

I found this in the messages.log

Aug 16 23:29:42 sqlserv kernel: VFS: Mounted root (ext2 filesystem).
Aug 16 23:29:42 sqlserv kernel: SCSI subsystem driver Revision: 1.00
Aug 16 23:29:42 sqlserv kernel: Red Hat/Adaptec aacraid driver (1.1-3 Apr 18 2004 15:07:49)
Aug 16 23:29:42 sqlserv kernel: AAC0: kernel 4.0.4 build 6011
Aug 16 23:29:42 sqlserv kernel: AAC0: monitor 4.0.4 build 6011
Aug 16 23:29:42 sqlserv kernel: AAC0: bios 4.0.0 build 6011
Aug 16 23:29:42 sqlserv kernel: AAC0: serial ba980efafaf001
Aug 16 23:29:42 sqlserv kernel: AAC0: Non-DASD support enabled
Aug 16 23:29:42 sqlserv kernel: scsi0 : aacraid
Aug 16 23:29:42 sqlserv kernel: Vendor: ADAPTEC Model: scsi Rev: V1.0
Aug 16 23:29:42 sqlserv kernel: Type: Direct-Access ANSI SCSI revision: 02
Aug 16 23:29:42 sqlserv kernel: Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
Aug 16 23:29:42 sqlserv kernel: SCSI device sda: 143354880 512-byte hdwr sectors (73398 MB)
Aug 16 23:29:42 sqlserv kernel: Partition check:
Aug 16 23:29:42 sqlserv kernel: sda: sda1 sda2 sda3 sda4 < sda5 sda6 >
Aug 16 23:29:42 sqlserv kernel: Journalled Block Device driver loaded

Shenron,

1. During test - please shutdown as many services as you can (if somebody using your server - then test results is going to be slower)

2. please show the output of "vmstat"

3. use tiobench for tests, please provide test results for 40 & 512Mb files.
http://directory.fsf.org/sysadmin/Monitor/tiobench.html

ok, I'm going to install tiobench.

here the outpout of vmstat

vmstat
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 0 6628 600516 50060 1160800 2 1 14 12 7 8 2 2 8

./tiotest -f 40
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item | Time | Rate | Usr CPU | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write 160 MBs | 7.2 s | 22.207 MB/s | 0.6 % | 71.2 % |
| Random Write 16 MBs | 8.3 s | 1.877 MB/s | 0.2 % | 3.0 % |
| Read 160 MBs | 0.1 s | 1510.332 MB/s | 37.8 % | 330.4 % |
| Random Read 16 MBs | 0.0 s | 1589.037 MB/s | 101.7 % | 305.1 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write | 0.156 ms | 277.812 ms | 0.00000 | 0.00000 |
| Random Write | 3.257 ms | 6293.411 ms | 0.05000 | 0.00000 |
| Read | 0.009 ms | 10.395 ms | 0.00000 | 0.00000 |
| Random Read | 0.007 ms | 0.111 ms | 0.00000 | 0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total | 0.220 ms | 6293.411 ms | 0.00222 | 0.00000 |
`--------------+-----------------+-----------------+----------+-----------'



./tiotest -f 512
Tiotest results for 4 concurrent io threads:
,----------------------------------------------------------------------.
| Item | Time | Rate | Usr CPU | Sys CPU |
+-----------------------+----------+--------------+----------+---------+
| Write 2048 MBs | 90.8 s | 22.553 MB/s | 0.7 % | 59.8 % |
| Random Write 16 MBs | 10.2 s | 1.526 MB/s | 0.0 % | 2.2 % |
| Read 2048 MBs | 312.0 s | 6.564 MB/s | 0.2 % | 2.4 % |
| Random Read 16 MBs | 17.8 s | 0.877 MB/s | 0.1 % | 0.5 % |
`----------------------------------------------------------------------'
Tiotest latency results:
,-------------------------------------------------------------------------.
| Item | Average latency | Maximum latency | % >2 sec | % >10 sec |
+--------------+-----------------+-----------------+----------+-----------+
| Write | 0.531 ms | 991.636 ms | 0.00000 | 0.00000 |
| Random Write | 0.535 ms | 1903.176 ms | 0.00000 | 0.00000 |
| Read | 2.273 ms | 840.462 ms | 0.00000 | 0.00000 |
| Random Read | 17.636 ms | 355.047 ms | 0.00000 | 0.00000 |
|--------------+-----------------+-----------------+----------+-----------|
| Total | 1.460 ms | 1903.176 ms | 0.00000 | 0.00000 |
`--------------+-----------------+-----------------+----------+-----------'


hdparm -tT /dev/sda1

/dev/sda1:
Timing buffer-cache reads: 128 MB in 0.22 seconds =581.82 MB/sec
Timing buffered disk reads: 64 MB in 2.24 seconds = 28.57 MB/sec

28.57MB/sec is slow :confused:

Thanks :)

it looks like you have 2-year old firmware 6011 build on the 2120S.

try the newest firmware (7244 build):
http://www.adaptec.com/worldwide/support/driverdetail.jsp?sess=no&language=English+US&cat=/Product/ASR-2120S&filekey=asr2120s_fw_v7244.exe

and you must use 7244 driver set after firmware update:
http://www.adaptec.com/worldwide/support/driverdetail.jsp?sess=no&language=English+US&cat=/Product/ASR-2120S&filekey=aacraid_drv_1.1.4-2322.rpm

>>Timing buffered disk reads= 28.57 MB/sec
that's only about half of single SATA drive! hmmm....

Shenron, yes please do as cwl suggested:
1. update firmware
2. update driver

and please post result of test here again - would be very interesting. we're having the same problems with newest ibm server 4xSCSI u320 HDs in RAID5 are slow as 30MB/s :-((

I will do and give you results

For th firmware version 7244 http://www.adaptec.com/worldwide/support/driverdetail.jsp?sess=no&language=English+US&cat=/Product/ASR-2120S&filekey=asr2120s_fw_v7244.exe
It is written: This file is self extracting and will unzip to c:\adaptec\raid
How can it works with Linux Redhat ?

The server will boot from the raid so:

- If I update the firmware first. Does the server boot with the old driver which seems to be 1.1-3 ?

What must I do with the file "acu.exe" ?

Originally posted by Shenron
The server will boot from the raid so:

- If I update the firmware first. Does the server boot with the old driver which seems to be 1.1-3 ?

What must I do with the file "acu.exe" ?
I just updated the firmware on couple of 2120S the other days. I recall you need to use afu.exe instead of acu.ext

you need to download asr2120s_fw_v7244.zip to windows based PC, then
unzip
copy afu.exe and AC212001.ufi to 1.44M disk 1
copy AC212002.ufi to 1.44M disk 2

reboot the server with 2120S using any DOS bootable disk, then change disk to firmware disk #1
enter 'afu' at DOS prompt

I would install the aacraid_drv_1.1.4-2322.rpm first before firmware update. however, make a backup copy of old 'aacraid.o' driver first just in case. it should be under:
/lib/modules/<current kernel version.>/kernel/drivers/scsi/

I'm looking at the can, and it says right here "Kills on contact" . Okay, nevermind :D

I didn't find any aacraid.o in /lib/modules :confused:

[root@sqlserv bin]# cd /lib/modules/2.4.26-grsec/kernel/drivers/scsi/
[root@sqlserv scsi]# ls
3w-xxxx.o aha1740.o cpqfc.o eata_pio.o in2000.o osst.o psi240i.o scsi_mod.o st.o u14-34f.o
53c7,8xx.o aic7xxx dmx3191d.o fdomain.o initio.o pas16.o qla1280.o sd_mod.o sym53c416.o ultrastor.o
a100u2w.o aic7xxx_old.o dpt_i2o.o gdth.o ips.o pci2000.o qlogicfas.o seagate.o sym53c8xx_2 wd7000.o
advansys.o AM53C974.o dtc.o g_NCR5380.o megaraid.o pci2220i.o qlogicfc.o sg.o sym53c8xx.o
aha152x.o atp870u.o eata_dma.o ide-scsi.o NCR53c406a.o pcmcia qlogicisp.o sim710.o t128.o
aha1542.o BusLogic.o eata.o imm.o ncr53c8xx.o ppa.o scsi_debug.o sr_mod.o tmscsim.o
[root@sqlserv scsi]# find / -name aacraid.o
/usr/src/linux-2.4.26/drivers/scsi/aacraid/aacraid.o
[root@sqlserv scsi]#

but I can see it in lsmod

lsmod
Module Size Used by Not tainted
autofs 12484 0 (autoclean) (unused)
eepro100 21324 1
mii 4092 0 [eepro100]
ipchains 53516 14
ehci-hcd 20064 0 (unused)
usb-uhci 25732 0 (unused)
usbcore 77248 1 [ehci-hcd usb-uhci]
ext3 69856 4
jbd 51944 4 [ext3]
aacraid 35700 5
sd_mod 12796 10
scsi_mod 110664 2 [aacraid sd_mod]

[root@sqlserv scsi]# find / -name aacraid.o
/usr/src/linux-2.4.26/drivers/scsi/aacraid/aacraid.o

that's the one! make a backup before you update the driver and firmware:
cp aacraid.o aacraid.bak
or something like that....

My kernel is 2.4.26 with the grsecurity patch. grsecurity patch is configured as high security level. I said that because when I tried a hdparm with a kernel 2.4.18 without grsecurity, it was

hdparm -tT /dev/sda1

/dev/sda1:
Timing buffer-cache reads: 128 MB in 0.20 seconds =620.00 MB/sec
Timing buffered disk reads: 64 MB in 0.72 seconds = 85.12 MB/se
It is twice more !!!

Moreover I tried to update driver as you said. While the update, it was looking for kernel with lilo.conf and it don't update anythink with the kernel 2.4.26-grsec but the update seemed to be ok with the 2.4.18 kernel
Why?
My kernel is named "2.4.26-grsec" and the other is "2.4.18-smp". Is there problem with the name of the kernel?

Thanks,

Originally posted by Shenron
My kernel is 2.4.26 with the grsecurity patch. grsecurity patch is configured as high security level. I said that because when I tried a hdparm with a kernel 2.4.18 without grsecurity, it was

hdparm -tT /dev/sda1

/dev/sda1:
Timing buffer-cache reads: 128 MB in 0.20 seconds =620.00 MB/sec
Timing buffered disk reads: 64 MB in 0.72 seconds = 85.12 MB/se
It is twice more !!!

Moreover I tried to update driver as you said. While the update, it was looking for kernel with lilo.conf and it don't update anythink with the kernel 2.4.26-grsec but the update seemed to be ok with the 2.4.18 kernel
Why?
My kernel is named "2.4.26-grsec" and the other is "2.4.18-smp". Is there problem with the name of the kernel?

Thanks,
well, Timing buffered disk reads: 64 MB in 0.72 seconds = 85.12 MB/sec, is a very decent performance! but, Timing buffer-cache reads: 128 MB in 0.20 seconds =620.00 MB/sec, is still rather low, it should be in 2000+MB/sec range.

check you 2120S RAID BIOS and see whether the 64-meg on-card cache is turned on?

2.4.18smp is a different kernel than 2.4.26, and it does look like the 2.4.26grsec is somehow screwy. regardless, 2.4.18smp is old too....

Are you sure drivers are available with kernel 2.4.27?

When I tried :

mkinitrd -f -v /boot/initrd-2.4.27-grsec.img 2.4.27-grsec/
No module aacraid found for kernel 2.4.27-grsec/
find / -name aacraid
/proc/scsi/aacraid
/usr/src/linux-2.4.26/include/config/scsi/aacraid
/usr/src/linux-2.4.26/drivers/scsi/aacraid
/usr/src/linux-2.4.27/include/config/scsi/aacraid
/usr/src/linux-2.4.27/drivers/scsi/aacraid
/opt/Adaptec/aacraid


in /opt/Adaptect/aacraid there is
cat read.me


To patch your kernel sources:
1) Extract the patches from /opt/adaptec/aacraid/aacraid_patches.tgz
tar zxvf aacraid_patches.tgz
2) Determine which patch you need.
The patches are named aacraid-(kernel version).(distro).diff
Your kernel version can be determined with
uname -r
The distro is your distribution RH=RedHat, SuSE=SuSE, and TL=TurboLinux
For a generic kernel there is no distro - it is left blank
3) Run patch -p1 -d /opt/src/(your linux source dir) <(patch from step 2).diff

But the problem is that their last patch is kernel 2.4.22.

So how can I install my new kernel with raid or with the currently 2.4.26 kernel?

Thanks,

nope, I am not sure. looks like you need advices from linux experts on the WHT boards! I'm just a hardware guy.....

Shenron,

ive looked into rpm, there is "aacraid_source.tgz" file. untar it to the directory with your kernel sources and then make this kernel with this module

can you show the full output of hdparm /dev/sda1 I can only see a small section of that. It'll help me to see if anything needs tweaking.

Andrew

of course:


[root@sqlserv bin]# hdparm /dev/sda1

/dev/sda1:
readonly = 0 (off)
geometry = 8923/255/63, sectors = 16386237, start = 63

Originally posted by farpost
Shenron,

ive looked into rpm, there is "aacraid_source.tgz" file. untar it to the directory with your kernel sources and then make this kernel with this module Thanks, where must I untar this file?
Because after untar it, I'm trying "make"

make
make: *** no target. stop.

untar it to /usr/src/linux/drivers/scsi/aacraid directory

be sure you read HOWTO make the kernel from sources.

Ok, I'll replace old files with this new driver sources.
So I think that It will be compiled with the whole kernel? What should I check in the kernel config? What options must I choose in the kernel to have my raid running well?

Shenron,

I sugest you got through kernel howto first (just get it from www.linug.org). You could either compile it into your kernel (you will need remake your kernel), or make scsi module for your current kernel - but anyway it may require remake kernel anyway.

There are some options, it would be simpler and faster just go thru HOWTO, than I would ask you questions here.

btw i would you suggest to learn howto setup linux to be able to boot from 2 or more kernels. so even if you get unbootable kernel - you could use another one.