Results 1 to 19 of 19
  1. #1

    What do I really need?

    I can imagine you guys being sick of new members signing up and immediately posting a new thread without searching but I've scoured the forum for hours now and found nothing close to this. I could really need some help here.

    OK, I'm starting an online audio service business. So I need the visitors of my site to be able to privately upload huge amounts of data that I can receive.

    My guess is that I'll need something that can carry an upload of 3gb max, like 15 times a month.
    ~ 50gb bandwidth / month

    I don't need much space since I'll be deleting the uploaded file right after I've downloaded it myself.
    ~ 10gb space (in case some people upload at same time)

    In order to upload a file with this size, the speed is pretty crucial, I'm guessing like at least 5mbps?

    But what are my options here?

    I need an easy access for the visitors to upload, so FTP is sort of out of the question I guess. (I can't have that 3 out of 4 mails is about how to upload the files)

    A VPS will be too expensive and too hard to configure I believe.

    The best thing I believe is some sort of "dropbox", like yousendit offers for 15 bucks/month, but their plan is too small. Is there anything like out there like theirs?

    I've checked out the competition but most of them seems to make the transfer with FTP (and their clients needs to contact them first in order to get uploading information etc, it's this I want to avoid).

    I'm on my last leg here just looking for some help or directions.
    I know my questions isn't really sliced for this forum either...
    And as you probably know by now, I'm new to this.

    Every reply would be greatly appreciated.

    /edgewater

  2. #2
    Join Date
    Mar 2009
    Posts
    2,218
    It may be the visitors' upload speed that's the limiting factor. Are they using home internet connections, or do they have something with faster upload speeds?

    A while ago, my youngsters were in bands at Uni; and one of the things they would do is record songs using a multi-track recorder. Sometimes they'd want to send me the files for mixing.

    So, a song on 8 tracks at 48kHz 24-bit takes about one million bytes per second; and some of their songs would last 10 minutes, so that would be a 600-million-byte upload - for one song. To upload that song could take two hours. (And they'd have several songs to be mixed.)

  3. #3
    It will be the visitor that determines the upload speed. There are other factors to consider. If you don't wish to use FTP you'll need to develop a web app to handle the upload and you most likely will have issues running that on a shared server.

    For instance, if you developed a page in PHP to handle the uploads, large files will time out or exceed the max upload size set on the server. To overcome that you'll need at least a VPS, so you can set those parameters. Then there's the issue of browser time outs, network hiccups, etc., so you'll need to have a way to resume broken uploads.

    Much to overcome here if you don't want to use FTP.
    Sphere Web Hosting, LLC.
    █ Experience our Contagious Support™ and 100% Uptime SLA
    We don't just provide support, we care about your success - it's just the way we do things!
    █ Make your life a little easier and visit http://www.spherewebhosting.com

  4. #4
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,842
    Quote Originally Posted by edgewater View Post
    I need an easy access for the visitors to upload, so FTP is sort of out of the question I guess. (I can't have that 3 out of 4 mails is about how to upload the files)
    I guess what you're thinking about then is a browser upload (http). Not sure if it's even possible for files that size but it's certainly not recommended - how frustrating would it be to have your browser uploading for hours and then discover that the upload's failed and you have to do it all over again... Even ftp isn't a good solution for the same reason. Personally, I'd always use rsync to ensure that in the event of a broken connection (or for any other reason) the upload can be resumed easily and efficiently.

    To use rsync you'd need a host that offers ssh access (jailed is ok). That's not uncommon it is an extra requirement.

    To hide the complexity from the clients, perhaps look for a client-side application (or java applet?) they can install. If you can't find anything you could probably get a little custom app written to do this.

    Whatever method you use you'll need to be sure the host is ok with you doing this, because no matter what you'll have connections open for a long time. Definitely discuss it with any potential host first - it's so much easier than having to move later. For this sort of usage a VPS might actually be a better bet.
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  5. #5
    Join Date
    Mar 2009
    Location
    /usr/bin/perl
    Posts
    971
    Most shared hosts prohibit you using X% of any resource for N minutes, so I second the VPS idea. Perhaps a managed provider would be able to configure php, apache, etc for you and upload a script if you had one written. In the long run this will certainly be the more sustainable option.

  6. #6
    Join Date
    Jun 2004
    Location
    California
    Posts
    682
    Perhaps this is stupid, but tools like Winzip let you break a large file into smaller pieces. If you really want to avoid FTP, maybe your clients could upload over HTTP in 10mb chunks or something.

    I personally think it would be terrible from an end user point of view, but it is an option that could be explored... maybe.
    Scott Swezey
    TCL Fasteners, Inc.
    Automotive fasteners for collision shops.

  7. #7
    Join Date
    Mar 2009
    Posts
    2,218
    It might be better to split the job in two:

    1) A web server for providing the user interface
    2) An FTP server for handling the uploads. This can be separate from the web server - eg, Torqhost offers bulk storage accounts meeting the OP's needs for about 5 Euros a month.

    The web server would deliver a page with a file upload form; that would allow the user to select files from their local machine, and click an "upload" button. Javascript code within the web page would issue the necessary FTP commands.

    As the file uploading and storage would not be done on the web server, restrictions on run time and file sizes would not apply.

  8. #8
    Quote Originally Posted by edgewater View Post
    A VPS will be too expensive and too hard to configure I believe.
    There are a lot of good suggestions for doing it over the web, but because of the "too hard to configure" part you mentioned, they might be a bit beyond what you're able to do (although you could hire someone I suppose).

    You really might be better off to either:
    -Use FTP. Have a nice little walk-through on the site with screenshots (or make a tiny "how-to" video) to reduce the emails you receive about how to do it.
    -Grab an already existing service, like that YouSendIt you mentioned. You said their package is too small (the 15/month caps at 2GB per up and 6GB total). You could do the math and see if files might make it under the 2GB mark or could be split up. Otherwise you might be able to see if they can offer you a larger package (they have a "corporate" one but it doesn't mention the limits). No idea if that service resumes uploads or anything, but you could keep looking around.

    3GB is a LOT to upload on just about any consumer connection. I really tend to think that FTP would be an easier way to go - it seems to inherently handle resumes and retries pretty well. A customer might be pretty irked if they spend half a day uploading a file through a web interface and have it time-out, only to have to upload the entire thing again.

    Just my thoughts on the matter though.
    EyeglassRetailerReviews.com - Save money by buying glasses online!

  9. #9
    Join Date
    Apr 2009
    Posts
    643
    That would be great to see what have you been using before and in which way would you need to upgrade your account
    ASPnix Web Hosting - ASP.NET, MS SQL, AJAX, Hyper-V
    Microsoft Hosting and Virtualization

  10. #10
    Join Date
    Mar 2009
    Posts
    2,218
    Here's an example of a web page which provides FTPuploads to a separate server.

    (Before trying it, please decide whether you trust it and me enough to type in FTP details and passwords. I do not use the code myself, so cannot comment on its quality.)

    http://www.integralftp.com/

    The code for this is available as a free download at

    http://www.enterprisedt.com/products.../overview.html

    So if you run your own web server, you could modify the code to include login details (IP address, userid, passsword) for your preferred FTP service.

    Couple that with a file sharing service, and you can at least get started. I reckon a low-cost web host, plus the Torqhost file sharing package, would meet the stated requirements for about $10 a month.

    I've not used Torqhost, so cannot comment on their quality of service. Anyway, here's their web page:

    http://torqhost.com/bulk_storage_packages.php

    So, what can go wrong? I can think of a few things:

    1) The time for uploads can be long and frustrate users on home internet connections. There's nothing you can do about that, but you need to check what happens if a user interrupts an upload. Can it be restarted where it left off?

    2) You need to manage the FTP accounts somehow, so users cannot access each others' data. One basic method is to have a pool of FTP userids, each with their own upload directory, and assign one to a user who wants to upload data. After the data is uploaded, move it from the directory and change the FTP password.

    3) You need some way to know if an upload is complete.

    4) You need to ensure you have adequate backups; it would be embarrassing to ask a user to redo an eight-hour upload because you accidentally deleted or damaged the data.

    5) And you need to maintain logs of where and when files came from. One day, a user is going to upload the wrong file. You will spend time doing whatever it is you do, and when you are finished, the user will say "But you're working on the old version ..." Ideally, you'd need to provide a way for the user to verify that the file you're about to work on is the one they intended.

  11. #11
    Quote Originally Posted by SphereWebHosting
    It will be the visitor that determines the upload speed. There are other factors to consider....

    For instance, if you developed a page in PHP to handle the uploads, large files will time out.....

    Much to overcome here if you don't want to use FTP.
    Yeah I'm starting to think here that it has to be done with FTP or some advanced coding with a VPS, thanks alot for the info!

    Quote Originally Posted by foobic
    I guess what you're thinking about then is a browser upload (http). Not sure if it's even possible for files that size but it's certainly not recommended .... I'd always use rsync to ensure that in the event of a broken connection (or for any other reason) the upload can be resumed easily and efficiently.

    To hide the complexity from the clients, perhaps look for a client-side application (or java applet?) they can install. If you can't find anything you could probably get a little custom app written to do this.
    Your whole reply is and will be extremely helpful to me, especially about the rsync!
    Never heard of it and I've read some about it now, thank you!

    Quote Originally Posted by jarrodsl
    Most shared hosts prohibit you using X% of any resource for N minutes, so I second the VPS idea. Perhaps a managed provider would be able to configure php, apache, etc for you and upload a script if you had one written. In the long run this will certainly be the more sustainable option.
    Yeah it will probably be the main problem if going lowbudget..
    But the VPS-solution with advanced coding will probably have to wait a while until I have steady work coming in.
    Appreciate the reply mate.

    Quote Originally Posted by ScottSwezey
    Perhaps this is stupid, but tools like Winzip let you break a large file into smaller pieces. If you really want to avoid FTP, maybe your clients could upload over HTTP in 10mb chunks or something.

    I personally think it would be terrible from an end user point of view, but it is an option that could be explored... maybe.
    Yeah I thought about that, but 10mb, c'mon =)
    But like 700mb parts, but the thing here is ( like you said ), I don't think people would go for that. Imagine having the pain to go to computer each half hour to upload file after file that could have been done once... Thanks for your thoughts though

    Quote Originally Posted by jdi_knght
    There are a lot of good suggestions for doing it over the web, but because of the "too hard to configure" part you mentioned, they might be a bit beyond what you're able to do (although you could hire someone I suppose).

    You really might be better off to either:
    -Use FTP. Have a nice little walk-through on the site with screenshots (or make a tiny "how-to" video) to reduce the emails you receive about how to do it.
    -Grab an already existing service, like that YouSendIt you mentioned. You said their package is too small (the 15/month caps at 2GB per up and 6GB total). You could do the math and see if files might make it under the 2GB mark or could be split up. Otherwise you might be able to see if they can offer you a larger package...

    3GB is a LOT to upload on just about any consumer connection. I really tend to think that FTP would be an easier way to go - it seems to inherently handle resumes and retries pretty well. A customer might be pretty irked if they spend half a day uploading a file through a web interface and have it time-out, only to have to upload the entire thing again.
    Yeah the "too hard to configure" will be a problem, and I can't hire someone when I'm in this stage. But its an great idea if the site is lucky enough will grow.

    Good advice with the how-to video, that I'll do!
    And I've now written to yousendit and asked if they can make up something custom for me. - Thanks for the reply, appreciate it

    Quote Originally Posted by rumsfo
    That would be great to see what have you been using before and in which way would you need to upgrade your account
    I'm the ultimate first-timer, never even bought webhosting!

    Quote Originally Posted by tim27.....
    It might be better to split the job in two:

    1) A web server for providing the user interface
    2) An FTP server for handling the uploads. This can be separate from the web server....
    ______

    So if you run your own web server, you could modify the code to include login details (IP address, userid, passsword) for your preferred FTP service.

    Couple that with a file sharing service, and you can at least get started. I reckon a low-cost web host, plus the Torqhost file sharing package, would meet the stated requirements for about $10 a month.

    So, what can go wrong? I can think of a few things.......
    3 big-helping replies, I don't know how to express my gratitude in words.. And from another who's done work with audio!

    First off, I'll most possibly go with a low cost-webhost and the FTP-service from torqhost. Great advice there. I've been looking for pure ftp-storage like torqhost but nothing close to as cheap and good as them.

    Excellent with that checklist of "what can go wrong". That goes straight onto the paper pad.

    Good advice also with that webpage to ftp-upload-code, will definitely check that out and other solutions like it.

    As said, your help and info will be and is greatly appreciated, thanks!

  12. #12
    Join Date
    May 2008
    Location
    Melbourne, Australia
    Posts
    10,482
    I've not used Torqhost, so cannot comment on their quality of service. Anyway, here's their web page:

    http://torqhost.com/bulk_storage_packages.php
    You cannot recommend a Service you have never used ... Suppose someone did not read the rules.
    l Dedigeeks (Twitter) • Shared • Reseller • Cloud VPS • Since 2010
    l Leading AU Hosting Provider • Multiple locations - around the globe!
    l cPanel/WHM • R1Soft Backups • 24/7/365 Support • 99.9% Uptime Guarantee
    l www.yourcompanynamehere.com • Customer Service Rep. • Superior Service Guarantee

  13. #13
    Join Date
    Mar 2009
    Posts
    2,218
    Quote Originally Posted by ATH-Sean View Post
    You cannot recommend a Service you have never used ... Suppose someone did not read the rules.
    I never recommend any services.

  14. #14
    Join Date
    May 2008
    Location
    Melbourne, Australia
    Posts
    10,482
    I've not used Torqhost, so cannot comment on their quality of service. Anyway, here's their web page:

    http://torqhost.com/bulk_storage_packages.php
    You cannot give a direction to a service you have never tried yourself
    l Dedigeeks (Twitter) • Shared • Reseller • Cloud VPS • Since 2010
    l Leading AU Hosting Provider • Multiple locations - around the globe!
    l cPanel/WHM • R1Soft Backups • 24/7/365 Support • 99.9% Uptime Guarantee
    l www.yourcompanynamehere.com • Customer Service Rep. • Superior Service Guarantee

  15. #15
    Join Date
    Mar 2009
    Posts
    2,218
    Quote Originally Posted by ATH-Sean View Post
    You cannot give a direction to a service you have never tried yourself
    I've no idea why you think that.

  16. #16
    Quote Originally Posted by tim2718281 View Post
    I've no idea why you think that.
    I suppose he meant that you need to have personal experience with the host you are suggesting or talking about. If you have seen the goos review that would be kind of forwarding opinion but not describing your personal touch and that might be a bit dishonest for the web host you are with.

  17. #17
    Join Date
    Sep 2009
    Posts
    375
    Quote Originally Posted by edgewater View Post
    A VPS will be too expensive
    Well, it depends, there are hosts that offers affordable packages for a vps. You might try checking some at the offer forum.

  18. #18
    Join Date
    Nov 2009
    Posts
    512
    I would be tempted to look at VPS solution. Dosent have to cost a fortune ( having said that im not sure on your budget) you will certainly get the speed you need and also the flexability of other features.

    Not sure what experience you have OS wise linux, windows etc. Both would be faily simple to configure as an ftp server

  19. #19
    Join Date
    Sep 2006
    Location
    Toronto, Canada
    Posts
    10
    Quote Originally Posted by akirah View Post
    Well, it depends, there are hosts that offers affordable packages for a vps. You might try checking some at the offer forum.

    Agreed, check the forums


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •