Web Hosting Talk : Web Hosting Main Forums : Dedicated Server : What is my bottleneck for file hosting using this server? HDD or Port

[funkywizard]

I accidnetally duplicated sda1 and sda2 twice. One should be sda1 / sda2 and another should be sdb1 / sdb2

[iwod]

Sorry for jumping in, assuming i am only serving a few numbers of files, and the total size of all files (100MB) fits well within the system Memory capacity (512MB).

Then could i assume that all file will be served from cache / Ram?
And therefore speed of Disk does not come into the equation?

Would CPU speed ever become a bottleneck? Let say i am only running on a Single Core Atom?

[omiKron]

Originally Posted by ayksolutions

I don't really have anything else to contribute to this thread since it has been so meticulously dissected by the posters. I just wanted to praise the people, especially Gabe, who contributed to this thread. It's truly helpful, not only for the OP, but for anyone else who reads this in the future. I hope more threads like this pop up on WHT instead of the usual bashing.

I concur; reading this thread was very informative. Thanks everyone for your contribution here.

[p2prockz]

Hello, I am back here to get some of your expertise regarding a RAID 10 setup.

It was pointed out that stripe size of 2MB is recommended for a software RAID 10 and for Hardware RAID10 we should go with the maximum stripe size allowed by the controller.

I have an issue where I paid $$ for a HW raid controller which only support a max stripe size of 256K

What should I do in this case to get the best performance?
Should I cancel the Hardware RAID controller and go with software raid?

Thanks in advance.

[funkywizard]

Originally Posted by p2prockz

Hello, I am back here to get some of your expertise regarding a RAID 10 setup.

It was pointed out that stripe size of 2MB is recommended for a software RAID 10 and for Hardware RAID10 we should go with the maximum stripe size allowed by the controller.

I have an issue where I paid $$ for a HW raid controller which only support a max stripe size of 256K

What should I do in this case to get the best performance?
Should I cancel the Hardware RAID controller and go with software raid?

Thanks in advance.

If you're primarily working with large files with sequential access, then definitely yes. For large numbers of simultaneous reads of large files, a 2MB stripe and a 512k readahead will provide at least 4x the performance of a 256k stripe.

[p2prockz]

Thanks a lot for the reply. Wish they had a thanks button for profiles.

[funkywizard]

Glad to help : )

[Joshak]

Wow, It depends on many things.. Its very hard for me to answer right away, I might get back to this one..

[p2prockz]

Hi All, Had setup software raid10 and I see the partition is re-syncing. Is resyncing done in the case of repairing the partition? or is it normal for it to run after the raid10 installation for the first time?

[root@server ~]# cat /proc/mdstat
Personalities : [raid10] [raid1]
md0 : active raid1 sda1[0] sdc1[2] sdd1[3] sdb1[1]
      487360 blocks [4/4] [UUUU]

md2 : active raid10 sdc3[2] sda3[0] sdb3[1] sdd3[3]
      5843927040 blocks super 1.2 2048K chunks 2 near-copies [4/4] [UUUU]
      [===>.................]  resync = 15.3% (896459072/5843927040) finish=74870.8min speed=1100K/sec

md1 : active raid10 sdc2[2] sda2[0] sdb2[1] sdd2[3]
      15621120 blocks super 1.1 512K chunks 2 near-copies [4/4] [UUUU]

unused devices: <none>

What kind of commands can I use to get the status related to RAID10? Can I get information directly from the HW controller? I am very new to RAID so any reading material would be helpful.

Thanks.

[p2prockz]

Originally Posted by funkywizard

Maybe, but there really isn't a whole lot to it:

1) If using software raid 10, make sure the raid stripe is 2MB
2) If using hardware raid 10, make sure the raid stripe is as large as the raid controller will allow
3) Set the linux readahead to 1/4 of the raid stripe (if using raid 10), or to 512k (if using raid 1 or no raid at all)
4) Disable "atimes" in /etc/fstab
5) All else being equal, more ram, more hard drives, or faster rpm drives, are better than less ram, fewer hard drives, or slower rpm drives

Hello again,

Just wondering would it be beneficial to set a stripe size even larger than 2MB? For example, since we have files larger than 100Mb can we set the strip size to let's say 4MB or 8MB?

Thanks.

[funkywizard]

Originally Posted by p2prockz

Hello again,

Just wondering would it be beneficial to set a stripe size even larger than 2MB? For example, since we have files larger than 100Mb can we set the strip size to let's say 4MB or 8MB?

Thanks.

There is no additional performance benefit when you use a readahead larger than 512KB, and in fact you can lose performance in some cases because you'll have to read the same data more than once if your readaheads are taking up too much ram. With a 2MB stripe and a 512k readahead, approximately 25% of your read requests will cross a stripe boundary, so 75% of the time, one read request causes one disk i/o on one drive, and 25% of the time you'll get one disk i/o on each of two drives. A 4mb stripe should drop this to 12.5%. So for 10 reads, instead of 12.5 i/o requests, you'd have 11.25 i/o requests. So potentially a 4MB stripe might be a little faster in this use pattern than a 2MB stripe, but it's a pretty small difference and there might be negative performance from having the stripe be too large, so I'd say 2MB is a safe bet. You could use 4MB if you want, I would expect that to be fine too, but I haven't tested that. Anything above 4MB when you have a 512k readahead won't do much for you. A readahead above 512k won't do much for you either.

[p2prockz]

Thank you again!