Results 1 to 13 of 13
  1. #1
    Join Date
    Oct 2010
    Location
    UK
    Posts
    127

    Bonding Interfaces - CentOS

    Hello,

    I would like to ask about best bonding mode for the best performance.

    - at the moment I use mode=4 (802.3ad).

    I also tried to test few other modes however when I change it in the /etc/modprobe.conf file I don't see any changes even if I restarted the networking service.
    Do I have to remove /proc/net/bonding/bond0 file so the changes will take affect?

    Regards

  2. #2
    Join Date
    Mar 2009
    Location
    NL
    Posts
    571
    LACP is the best in our experience (mode 4, which you are using).

    Network restart should be enough normally.
    YISP - High Bandwidth dedicated servers and colocation in YISP-AS(Amsterdam)!

    Website: http://www.yisp.com
    Contact: info "(AT)" yisp.com

  3. #3
    Join Date
    Oct 2010
    Location
    UK
    Posts
    127
    Thanks for reply.

    I've also noticed the following message during network restart, is this something to worry about it (bold warning):

    May 9 20:59:30 server kernel: bonding: bond0: Adding slave eth0.
    May 9 20:59:30 server kernel: bnx2: eth0: using MSIX
    May 9 20:59:30 server kernel: bonding: bond0: enslaving eth0 as a backup interface with a down link.
    May 9 20:59:30 server kernel: bonding: bond0: Adding slave eth1.
    May 9 20:59:30 server kernel: bonding: bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond
    May 9 20:59:31 server kernel: bnx2: eth1: using MSIX
    May 9 20:59:31 server kernel: bonding: bond0: enslaving eth1 as a backup interface with a down link.
    May 9 20:59:31 server kernel: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
    May 9 20:59:31 server kernel: bonding: bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond
    May 9 20:59:32 server kernel: bnx2: eth0 NIC Copper Link is Up, 100 Mbps full duplex
    May 9 20:59:32 server kernel: bonding: bond0: link status definitely up for interface eth0.
    May 9 20:59:32 server kernel: bnx2: eth1 NIC Copper Link is Up, 100 Mbps full duplex
    May 9 20:59:32 server kernel: bonding: bond0: link status definitely up for interface eth1.
    - status seems to be fine, have you change any other options (Bonding Driver Options) :

    cat /proc/net/bonding/bond0
    Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)

    Bonding Mode: IEEE 802.3ad Dynamic link aggregation
    Transmit Hash Policy: layer2 (0)
    MII Status: up
    MII Polling Interval (ms): 100
    Up Delay (ms): 0
    Down Delay (ms): 0

    802.3ad info
    LACP rate: slow
    Active Aggregator Info:
    Aggregator ID: 14
    Number of ports: 1
    Actor Key: 9
    Partner Key: 1
    Partner Mac Address: 00:00:00:00:00:00

    Slave Interface: eth0
    MII Status: up
    Link Failure Count: 0
    Permanent HW addr: 00:22:19:60:3b:61
    Aggregator ID: 14

    Slave Interface: eth1
    MII Status: up
    Link Failure Count: 0
    Permanent HW addr: 00:22:19:60:3b:63
    Aggregator ID: 15
    - I run some tests, both NICs are connected to 100Mbit ports and I have up to 100Mbit speed connection only but it should be doubled, am I right?

    - and why eth0 interface is utilized more that eth1

    netstat -i
    Kernel Interface table
    Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
    bond0 1500 0 1923738 0 0 0 1644037 0 0 0 BMmRU
    bond0:1 1500 0 - no statistics available - BMmRU
    eth0 1500 0 1802739 0 0 0 1642241 0 0 0 BMsRU
    eth1 1500 0 120999 0 0 0 1796 0 0 0 BMsRU
    lo 16436 0 5527429 0 0 0 5527429 0 0 0 LRU
    venet0 1500 0 60376947 0 0 0 58573887 0 751 0 BOPRU

  4. #4
    Join Date
    Nov 2003
    Posts
    538
    Quote Originally Posted by HTF1 View Post
    Thanks for reply.

    I've also noticed the following message during network restart, is this something to worry about it (bold warning):



    - status seems to be fine, have you change any other options (Bonding Driver Options) :



    - I run some tests, both NICs are connected to 100Mbit ports and I have up to 100Mbit speed connection only but it should be doubled, am I right?

    - and why eth0 interface is utilized more that eth1
    Regarding your warning "May 9 20:59:31 server kernel: bonding: bond0: Warning: No 802.3ad response from the link partner for any adapters in the bond"

    What is the device at the other end of the port channel? Another server? a Switch? If it is a switch what is the port configuration?

    Regarding performance:

    A single stream won't be sent out both channels in a link bond.

    So if you are using wget to download a file, the maximum speed of that download will be the same as the speed of one of the links. The benefit of link bonding is that you can download two files each at 100Mbps at the same time. Not a single file at 200Mbps.
    XLHost.com
    Dedicated Servers, Virtual Private Servers, and more since 1995.
    drew @ xlhost.com

  5. #5
    Join Date
    Oct 2010
    Location
    UK
    Posts
    127
    Thank you for reply.

    The server is located in DC so there is a switch on the other side of the port channel. I know that this mode requires same configuration on the switch but if everything is working fine is there some configuration that need to be done anyway? Is this feature fully utilize in this case

    Is there anything else that I could do it to tune this up?

    A single stream won't be sent out both channels in a link bond.

    So if you are using wget to download a file, the maximum speed of that download will be the same as the speed of one of the links. The benefit of link bonding is that you can download two files each at 100Mbps at the same time. Not a single file at 200Mbps.

    http://wiki.centos.org/TipsAndTricks/BondingInterfaces
    Bonding allows you to aggregate multiple ports into a single group, effectively combining the bandwidth into a single connection. Bonding also allows you to create multi-gigabit pipes to transport traffic through the highest traffic areas of your network. For example, you can aggregate three megabits ports into a three-megabits trunk port. That is equivalent with having one interface with three megabytes speed.

    Regards
    Last edited by darkspace_co; 05-10-2011 at 08:48 AM.

  6. #6
    Join Date
    Mar 2003
    Location
    WebHostingTalk
    Posts
    16,966
    Moved > Hosting Security and Technology.
    Specially 4 You
    .
    JoneSolutions.Com ( Jones.Solutions ) is on the net 24/7 providing stable and reliable web hosting solutions and services since 2001

  7. #7
    From the sound of it, the switch is not configured for bonding / lacp. In this situation, your network connection will still "work", but you won't get any benefit out of having multiple links. You'll want to contact the datacenter to verify that the switchports making up the bond configured on your server, are also setup as a LAG / LACP group on the switch.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  8. #8
    Join Date
    Oct 2010
    Location
    UK
    Posts
    127
    OK thanks for reply.
    I understand that all the other modes don't require this configuration on the switch. Could you please advice which one I can use as an alternative for mode 4 that would give me performance improvement - just in case if they won't be able to set this up for me.
    Last edited by darkspace_co; 05-11-2011 at 07:44 AM.

  9. #9
    Quote Originally Posted by HTF1 View Post
    OK thanks for reply.
    I understand that all the other modes don't require this configuration on the switch. Could you please advice which one I can use as an alternative for mode 4 that would give me performance improvement - just in case if they won't be able to set this up for me.
    Mode 4 is really the only mode I use because all the others can play hell with the switch's ARP or MAC tables, and are spotty at best for performance improvements.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  10. #10
    Join Date
    Oct 2010
    Location
    UK
    Posts
    127
    Hello,

    OK so it looks like mode=4 is not an option in my case (no configuration on the switch).

    Is it worth to set any other mode then (I'm planing to set mode 6 - balance-alb).
    Is there any other solution to improve performance and utilize both interfaces or maybe just us it as a two single interfaces.

    BTW: When I change bonding mode it's not working after network restart only after reboot - any ideas?
    Last edited by darkspace_co; 05-13-2011 at 11:33 AM.

  11. #11
    Join Date
    Mar 2002
    Location
    Philadelphia, PA
    Posts
    2,508
    Quote Originally Posted by HTF1 View Post
    Hello,
    BTW: When I change bonding mode it's not working after network restart only after reboot - any ideas?
    Can you post your bonding configuration? The two interfaces in the bond, the bond and modprobe? Will make it easier to resolve.
    Linux junkie | steward.io

  12. #12
    Join Date
    Oct 2010
    Location
    UK
    Posts
    127
    Quote Originally Posted by viGeek View Post
    Can you post your bonding configuration? The two interfaces in the bond, the bond and modprobe? Will make it easier to resolve.
    Yes of course:

    eth0
    DEVICE=eth0
    USERCTL=no
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    BOOTPROTO=none
    TYPE=Ethernet
    IPV6INIT=no
    PEERDNS=yes
    eth1
    DEVICE=eth1
    USERCTL=no
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    BOOTPROTO=none
    TYPE=Ethernet
    IPV6INIT=no
    PEERDNS=yes
    bond0
    DEVICE=bond0
    USERCTL=no
    ONBOOT=yes
    NETMASK=xxx.xxx.xxx.xxx
    BROADCAST=xxx.xxx.xxx.xxx
    IPADDR=xxx.xxx.xxx.xxx
    BOOTPROTO=none
    TYPE=Ethernet
    IPV6INIT=no
    PEERDNS=yes
    /etc/modprobe.conf
    alias bond0 bonding
    options bond0 miimon=100 mode=4
    alias eth0 bnx2
    alias eth1 bnx2
    alias eth2 bnx2
    alias eth3 bnx2
    alias scsi_hostadapter megaraid_sas
    alias scsi_hostadapter1 ata_piix

  13. #13
    Join Date
    Mar 2010
    Location
    Germany
    Posts
    681
    Quote Originally Posted by HTF1 View Post
    Hello,

    I would like to ask about best bonding mode for the best performance.

    - at the moment I use mode=4 (802.3ad).

    I also tried to test few other modes however when I change it in the /etc/modprobe.conf file I don't see any changes even if I restarted the networking service.
    Do I have to remove /proc/net/bonding/bond0 file so the changes will take affect?

    Regards
    You'd have to unload the bonding kernel module while networking services are stopped.
    Note that RedHat also demands you put the BONDING_OPTS=" ... " section in the ifcfg-bond0 file.

    I would also go with mode 4.
    Some hints:
    enable fast mode (faster detection of failures) and try the different loadbalancing settings.
    Check out my SSD guides for Samsung, HGST (Hitachi Global Storage) and Intel!

Similar Threads

  1. HP ProCruve 2824 and Mac Pro Bonding
    By Ricky Smith in forum Hosting Security and Technology
    Replies: 0
    Last Post: 09-21-2010, 10:16 PM
  2. MLPPP (ADSL Bonding)
    By TailoredVPS in forum Hosting Security and Technology
    Replies: 2
    Last Post: 10-25-2009, 08:15 PM
  3. NIC Bonding and Switches?
    By fog in forum Hosting Security and Technology
    Replies: 1
    Last Post: 07-02-2008, 01:27 AM
  4. Insurance and Bonding?
    By TubuNet in forum Web Hosting
    Replies: 2
    Last Post: 01-01-2007, 09:04 PM
  5. Hosting via ADSL Bonding
    By SME in forum Other Offers & Requests
    Replies: 21
    Last Post: 04-22-2002, 07:07 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •