Per vlad's request, I have started a new thread comparing at least partially using Sun X4500's with ZFS software on top vs. applogic clusters.
Question #1: How exactly is the cluster able to push "1.2GBytes per second" across gigabit ethernet?
Question #2: Why does a thumper *HAVE* to be backed up when you're running RAID-Z on it with double fault tolerance (somewhat like raid-6) and on top of that have it mirrored to a SECOND thumper, essentially giving you raid6+1 with completely redundant hardware?
Question #3: IF you can give me a good reason why you STILL Have to back all of this up, what exactly is the cluster doing that prevents you from needing these backups. What does the cluster have that competes with ZFS snapshots? What happens if I just want to roll back a file?
That's a good starting point, we can work from there.
I see that you are using the internal disk, which is being mirror'd to a second physical disk somewhere on the grid across gigabit ethernet. Going by that, anything heavy IO is going to absolutely crawl on "the grid". Any heavy database will need far more IOPS than a single IDE/SATA or SCSI disk (although there shouldn't be SCSI in "commodity" hardware eh?) can handle.
Am I going to get a response yet or should we just assume anyone with heavy IO applications should stay far far away from the grid?