Results 1 to 16 of 16
  1. #1
    Join Date
    Dec 2002
    Location
    India
    Posts
    1,433

    slow single threaded http/ftp downloads

    Hello all,

    I have 2 dedicated servers located in very reputed dedicated server companies in the usa.

    one located with service provider A with a 100MBPS switch and burstable bandwidth

    In this server I always , I mean ALL THE time get download speeds of over and above 1500KiB/sec all the time for *single threaded http/ftp" downloads

    for example

    server# lynx "http://dev.mysql.com/get/Downloads/MySQL-5.0/mysql-5.0.27.tar.gz/from/http://mysql.mirrors.pair.com/"



    Read 21874 of 25261 KiB of data, 1651 KiB/sec.

    etc

    I have another server located with dedicated provider B with the same 100MBPS switch and burstable bandwidth but

    server# lynx "http://dev.mysql.com/get/Downloads/MySQL-5.0/mysql-5.0.27.tar.gz/from/http://mysql.mirrors.pair.com/"

    Read 7810 of 25261 KiB of data, 632 KiB/sec.

    i.e it starts at about 780KiB/sec and gradually reduces and tapers off to about 632 KiB/Sec and eventually settles at about 500KiB/sec or so

    what is really happening here in the case of server B?

    Is there some kind of *unknown* bandwidth throtlling that is being secretly applied in service provider B's case

    Im not sure whether this will make any difference but service provider A's server is freebsd and service provider's B server is Linux enterprise 4

    Also the technicians present in service provider's server B are saying that they are trying to fix this problem for the last 7 days

    Any pointers will be greatly appreciated and I can pass it onto the server B's techs

    I have given the above download url only as a sample but the same holds good for all the public downloadable urls like perl , php or mysql websites..I have tried in about 25 different urls

    Service provider A 's download speed rocks and is always about 1700 KiB/sec or so

    but

    Service provider B's download speed does not cross 650Kibs at no point of time for single threaded downloads

    I may not be able to show you the traceroutes since I do not want to reveal the locations of my servers

    also if I use a mult threaded download utility like axel to download these files the speed goes up appreciably on service provider's B server too.

    Could the "B" server linux kernel have been tweaked to allow only a certain amount of bandwidth for single threaded downloads ?

    Im not sure at all

    any help will be greatly appreciated
    Chris, Developer/Programmer,
    https://www.chrisranjana.com/
    Php laravel developers, Node Python developers,
    AWS Certified | Python developers | Nodejs Developers | Devops

  2. #2
    Join Date
    Sep 2005
    Location
    Southern California
    Posts
    179
    Keep in mind that just because you have a 100mbit connection to your switch doesnt mean that you have a 100mbit connection to the internet.

    In a datacenter switches are often 24 ports, and with less reputable hosts you may be sharing your 100mbit link with 20 other people on the same switch. If the switch has a gigabit uplink to the core/distribution switches, then thats fine. However i have seen many cases where a 24 port 10/100 switch has a 100mbit uplink.

    If a bunch of other people on the same switch are using a lot of bandwidth then that could explain your speed issues.

    As far as i know you cannot tweak a kernel to limit bandwidth, especially if you can thread it and get better speeds. It is possible the host has a packet shaper that is throttling your requests, however this is unlikely.

    If you want to test, you can try transferring data via http, https, and ftp, and compare speeds.

  3. #3
    Join Date
    Dec 2002
    Location
    India
    Posts
    1,433
    Quote Originally Posted by localhost127
    Keep in mind that just because you have a 100mbit connection to your switch doesnt mean that you have a 100mbit connection to the internet.
    Ok fair enough , but the speeds I get in server B doesn't even come close to 10MBPS ! leave alone 100mbit speeds !



    If a bunch of other people on the same switch are using a lot of bandwidth then that could explain your speed issues.
    This is possible but I have tested the speed at various times , mornings late evenings etc and etc but it never exceeds 500 to 600 Kib/s



    If you want to test, you can try transferring data via http, https, and ftp, and compare speeds.
    In server B ftp and sftp transfers are slightly faster but no where near the blazing speed of server A
    Chris, Developer/Programmer,
    https://www.chrisranjana.com/
    Php laravel developers, Node Python developers,
    AWS Certified | Python developers | Nodejs Developers | Devops

  4. #4
    Join Date
    Feb 2003
    Location
    Panorama City, CA
    Posts
    2,581
    Ask them maybe?
    Do an in house test..

    Full duplex problem maybe?
    Try and force 100?
    Remote Hands and Your Local Tech for the Los Angeles area.

    (310) 573-8050 - LinkedIn

  5. #5
    Join Date
    Dec 2002
    Location
    India
    Posts
    1,433
    Jeremy

    The box is on Linux RHEL

    How to know for sure that full duplex is not enabled ?

    SInce I have ROOT access I can find it out
    Chris, Developer/Programmer,
    https://www.chrisranjana.com/
    Php laravel developers, Node Python developers,
    AWS Certified | Python developers | Nodejs Developers | Devops

  6. #6
    Join Date
    Dec 2005
    Location
    Internet
    Posts
    1,352
    Run the following command:

    # ethtool eth0

    that will show you the answer. If you are unable to find it out from the results, please paste the output here so that we could help you
    Senior Server Administrator

  7. #7
    Join Date
    Feb 2003
    Location
    Panorama City, CA
    Posts
    2,581
    I like mii-tool its just easy on the eyes.
    Remote Hands and Your Local Tech for the Los Angeles area.

    (310) 573-8050 - LinkedIn

  8. #8
    Join Date
    Dec 2005
    Location
    Internet
    Posts
    1,352
    Quote Originally Posted by Jeremy
    I like mii-tool its just easy on the eyes.
    Mii-tool is an old man, one of the original linux tools. But, ethtool wil replace it and will get support from all the new NICs in near future
    Senior Server Administrator

  9. #9
    Join Date
    Dec 2002
    Location
    India
    Posts
    1,433
    Here it is


    ethtool eth1

    Settings for eth1:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes: Not reported
    Advertised auto-negotiation: No

    Speed: 100Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: off
    Supports Wake-on: umbg
    Wake-on: g
    Current message level: 0x00000007 (7)
    Link detected: yes

    should I be concerned about the bolded lines above ?
    Chris, Developer/Programmer,
    https://www.chrisranjana.com/
    Php laravel developers, Node Python developers,
    AWS Certified | Python developers | Nodejs Developers | Devops

  10. #10
    Join Date
    Dec 2002
    Location
    India
    Posts
    1,433
    also if I enable auto negotiation it comes as half duplex

    ethtool eth1
    Settings for eth1:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Advertised auto-negotiation: Yes
    Speed: 100Mb/s
    Duplex: Half
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    Supports Wake-on: umbg
    Wake-on: g
    Current message level: 0x00000007 (7)
    Link detected: yes
    Chris, Developer/Programmer,
    https://www.chrisranjana.com/
    Php laravel developers, Node Python developers,
    AWS Certified | Python developers | Nodejs Developers | Devops

  11. #11
    Join Date
    Sep 2005
    Location
    Southern California
    Posts
    179
    Half duplex is bad and would probably cause some of your problems. If you are having problems it is a decent idea to force full duplex. In fact, some people force full duplex even if they aren't having issues.

  12. #12
    Join Date
    Dec 2002
    Location
    India
    Posts
    1,433
    Thanks. so


    Advertised link modes: Not reported
    Advertised auto-negotiation: No
    is a non issue ?

    This happens only when full duplex is forced
    Chris, Developer/Programmer,
    https://www.chrisranjana.com/
    Php laravel developers, Node Python developers,
    AWS Certified | Python developers | Nodejs Developers | Devops

  13. #13
    Join Date
    Sep 2005
    Location
    Southern California
    Posts
    179
    Its not an issue, it is just letting you know what is going on.

  14. #14
    Join Date
    Dec 2002
    Location
    India
    Posts
    1,433
    ok and here is more info from your favorite tool

    mii-tool -v
    eth0: no autonegotiation, 100baseTx-HD, link ok
    product info: vendor 00:50:43, model 12 rev 2
    basic mode: autonegotiation enabled
    basic status: autonegotiation complete, link ok
    capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
    advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
    link partner: 100baseTx-HD
    eth1: no autonegotiation, 100baseTx-HD, link ok
    product info: vendor 00:50:43, model 12 rev 2
    basic mode: autonegotiation enabled
    basic status: autonegotiation complete, link ok
    capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
    advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
    link partner: 100baseTx-HD

    The link partner is shown as Half Duplex ???
    Chris, Developer/Programmer,
    https://www.chrisranjana.com/
    Php laravel developers, Node Python developers,
    AWS Certified | Python developers | Nodejs Developers | Devops

  15. #15
    Join Date
    Sep 2005
    Location
    Southern California
    Posts
    179
    Right, this is why you want to turn off auto negotiation and force it to 100 full

  16. #16
    Join Date
    Feb 2003
    Location
    Panorama City, CA
    Posts
    2,581
    You can add

    ETHTOOL_OPTS="autoneg off speed 100 duplex full" to /etc/sysconfig/network-scripts/ifcfg-eth0

    then restart the network should make it work.
    Remote Hands and Your Local Tech for the Los Angeles area.

    (310) 573-8050 - LinkedIn

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •