I was wondering, if providing streaming videos consume more resources than just providing pure ftp download for videos. From my knowledge, streaming would suck more memory than pure download, am i right? Do you think doing streaming videos is a little bit tough for a a Celeron 2.4Ghz 512MB linux box?

You are not married to the server. When the streaming becomes too resource intensive, start upgrading. Like there is any other decision to be made?

Depends on how you intend to do it but starting out a entry level dedicated should be fine. I've done both streaming video and audio on a celeron but since encoding was done on another box it just had to take the feed coming in and serve it to whoever connected. That wasn't too terrible on the CPU it became more of a bandwidth issue. I used Shoutcast and NSV for video btw

Its all relative
Streaming to 10 people, fine. Streaming to 1000, maybe not

As long as you're not encoding on that machine at the same time you will be fine. I have pushed 75 128k windows media streams out of a similar setup, without any preformance degradation. Bandwidth will be more of an issue than anything else.

streaming will be more effective if you have large video/audio files (on-demand mode) and users want to play the same time once or twice (not X multiple times)

Let me show you an example.
You have 50Mb video file (10 minutes real video, 128Kbps rate) that you offer:

a.) for download (via FTP or HTTP - doesnt matter)
b.) for real streaming (for example .ASF format)

Now, when user starts to download it (via media player, not "SAVE" mode on the disk), in order to see from minute 6 to 7, you will need to download 6 minutes of content and then your media player will start playing the video... and don't forget, the download will continue all the time until the file has been downloaded or you close the application

If you use real streaming, then file is NOT completely downloaded from the streaming server, but only the frames you are seeing in media player..... see the difference? If the user wants to play from minute 6 to 7, then only bandwidth for these frames will be transmitted to your media player...

Don't forget that multiple one-video file streaming watching from the server will add up your server bandwidth. If the user wants to play the file 100+X times, then of course download will be the best..