Streaming media is largely dependent on the network card and ram. I have some streaming servers dishing out over 10,000 32k streams and run at <5% cpu (this is a windows media server, but i'd imagine others work similar). You want to make sure you have an intel NIC as they give the best performance.
The only other factor is what kind of streaming you are doing. There is a difference in load between on demand and a live stream. I must admit I am more familiar with live streaming, so I cannot give comments for on demand streaming which would seem to rely more on disk access.