Results 1 to 3 of 3
  1. #1
    Join Date
    Aug 2004
    Posts
    333

    Question curl_multi() - parallel downloads very slow?

    Some people said that with curl_multi() I can parallel download/open many pages without waiting slow pages to finish and that look very good for me because I need to check thousands pages every day, after few hours of work to add it to my current php code now I know that it is slow like when I use standard curl calls, for me this is not parallel downloads, for me this look like waiting for every page to finish and than go to next, like when I using standard curl calls in php!

    This is code that I using:
    http://www.onlineaspect.com/2009/01/...hout-blocking/

    Somebody know what can be problem?
    I`m on centos server without any load, 100Mbps dedicated connection...

    Thanks.

  2. #2

    Speeding up curl_multi

    Hi,

    The general trouble is that each group of requests has to wait for the slowest request to download. In a group of 50 requests, all it takes is a single laggy one to slow processing of the others.

    You could write a wrapper for this to cycle through requests like seen here:

    http://www.onlineaspect.com/2009/01/...hout-blocking/

    Hope that helps

  3. #3
    Join Date
    Aug 2004
    Posts
    333
    But, if you check my first post you`ll see that I already using it...

Similar Threads

  1. Browser Parallel Downloads
    By Server Management in forum Programming Discussion
    Replies: 1
    Last Post: 03-23-2013, 03:40 PM
  2. IIS downloads slow
    By thezombiehunter in forum Hosting Security and Technology
    Replies: 0
    Last Post: 12-13-2010, 04:51 PM
  3. Replies: 1
    Last Post: 07-10-2010, 03:21 PM
  4. Slow downloads
    By benoitr007 in forum Web Hosting
    Replies: 21
    Last Post: 04-29-2009, 02:12 AM
  5. Slow downloads
    By LP560 in forum Hosting Security and Technology
    Replies: 4
    Last Post: 04-08-2006, 05:35 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •