Ethernet Network Cable Aggregation

Written by Don Schultz, trueCABLE Senior Technical Advisor, Fluke Networks Copper/Fiber CCTT, BICSI INST1, INSTC, INSTF Certified

Recommended blog articles before this blog topic: The Need for Speed and Just What is NBASE-T?.

We like to go fast. There is a thrill in speed, and not only does it get your adrenaline going it also can solve practical problems while being fun in the process.

Link aggregation, commonly referred to as bonding, is the concept of taking multiple Ethernet connections and logically combining them into one big pipe.

Think of it this way: You have four, one-inch hoses and each one can only handle so much water at a time. What you really want is the ability to emulate a four-inch pipe and get the water flow that such a big pipe can handle. Link aggregation is how you accomplish this. Except with Ethernet cable, you’re moving data as opposed to water.

What exactly can link aggregation accomplish?

It is not the panacea we would hope for. There are limitations to the technology and as folks who have read my blogs before know...there is no free lunch. For high-level purposes, we will refer to link aggregation generically as it can sometimes mean something specific. Here is what it CAN do:

• Increase bandwidth between the switch and shared resource (like a server), providing multiple incoming clients a dedicated connection automatically. This is called 802.3ad Link Aggregation and we will get into that a little later. The net effect is clients are not competing for a single line of communication, allowing the shared resource to more effectively handle simultaneous requests. This method does require a special switch and Network Interface Card (NIC) installed in the “server” that recognizes this kind of aggregation. 802.3ad also allows connecting two or more switches together using link aggregation to increase switch to switch bandwidth.
• Load balancing with or without fail-over called ALB (Adaptive Load Balancing). This is a software based link aggregation technique that does not require a special switch. ALB will require a NIC that recognizes how to do it, since in this case it is the NIC that is doing all of the brain work. This technique similarly reduces competition between clients for a single connection.
• Provide durability for your network. If a single Ethernet link should fail, the others can take over seamlessly...assuming you configure it that way.

Here is what link aggregation CANNOT do:

• Double, triple, or quadruple your actual download speed from that bigger device. You still get your individual line speed, so your computer will pull down files from that NAS (Network Attached Storage) or server at 1 Gigabit speeds just like before (assuming you combined Gigabit Ethernet connections at the storage device side).
• Make your Internet faster. You pay your ISP (Internet Service Provider) a monthly fee for a certain speed. That speed is likely 500 Mbp/s or less. 500 Mbp/s is still only half the speed of a single Gigabit Ethernet connection. Adding three more links into the mix will not help at all, and might even hurt.

As always, pictures say a lot and makes this concept easier to understand. I will present how you are likely currently setup and then two common scenarios where link aggregation is used to improve bandwidth.

Your network, simplified, probably looks like this:

In this setup, the network switch is dumb. Dumb means not configurable and the switch has no address of its own on the network. All Ethernet cables are 1 Gigabit, and even the one going to the “server” is 1 Gigabit. A “server” is not necessarily special, it just serves out data to clients. Any computer or storage device on a network can function as a server.
Here is the 802.3ad Link Aggregation scenario, where we focus on the network switch and file server:

In this setup, showing only the Ethernet switch and server for clarity, the switch is smart. Smart means managed and configurable. It has its own address on the network as well. In this case, two 1 Gigabit Ethernet connections have been logically combined into a LAG or Logical Access Group. This LAG functions as a single logical pipe to the computer. The shared resource has a two port Ethernet NIC and is setup to recognize the 802.3ad aggregation link.

Pros of this setup:

• Managed switches are powerful and allow for a great deal of configurability. You can even have multiple LAGs on one switch. Yup, you can have a two port LAG running to the server and another two port LAG connecting up a second smart switch.
• 802.3ad is an understood standard in the networking world. Each 802.3ad device operates and interoperates seamlessly with other vendor’s 802.3ad compatible equipment.
• Allows for a bigger pipe from the switch to the shared resource, which means the connection will support more clients.

Cons of this setup:

• Cost. Managed switches are expensive, especially ones that support multiple LAGs. The costs can be 10X higher or more for just the switch.
• You still don’t get true double, triple, or quadruple bandwidth at any single client device.
• Hidden costs. You increased your bandwidth to the shared resource. Can the storage device even support multiple clients accessing it at once without becoming the bottleneck itself?

Here is the software version of link aggregation, known as Adaptive Load Balancing and also referred to as Teaming.

Pretty similar picture isn’t it? On the outside, it looks the same in fact. What makes this setup different is the network switch is dumb. It does not care about fancy logical access groups and is so dumb it cannot even remember its own address (which can be forgiven...it does not have one anyway). What makes this unique, and potentially more cost effective than 802.3ad in some situations, is the NIC inside the computer or storage device is the smarty pants. The NIC is deciding how to divide up the traffic and this leads to more pros and cons!

Pros of this setup:

• Less costly. Dumb regular network switches are so common they cost very little today compared to years ago.
• Easier to implement. You do not need to be a network administrator to get this setup right. In some cases, like certain NAS devices, there are two ports on the back that automatically support adaptive load balancing and will set itself up for you!
• You get similar benefits of increased bandwidth between the switch and computer or storage device. Client traffic will be logically divided across two or more physical connections. The net effect is similar to 802.3ad, it is just implemented differently.

Cons of this setup are:

• This implementation is switch to computer/shared resource/server only. No switch to switch LAGs are allowed.
• The burden of this technology is placed upon the NICs in the computer or storage device, leaving you at the mercy of Operating System (OS) support and NIC driver quality.
• You still don’t get double, triple, or more bandwidth to any one client device.
• The shared resource needs to be up to snuff in regards to supporting multiple clients accessing at once, otherwise there is little benefit to be gained.

The primary take-away with link aggregation is that it does not solve all bandwidth bottlenecks. When properly implemented, link aggregation can relieve network congestion to a single centralized storage device but it won’t increase line speeds to any single client device accessing the shared resource.

What many people don’t know is that implementing link aggregation carries a penalty in terms of bandwidth.

The multiple NICs and switch ports add additional Ethernet frame protocol and TCP/IP packet overhead in order to keep everything straight. A client side device might see a 10% drop in total available peak bandwidth to the shared resource. The upside is the shared resource won’t get bogged down anymore assuming the shared resource can keep up. The net result is the client, despite the reduced bandwidth, will get better service from the shared resource. However, this is not always the case. How the network is setup and if all hardware is matched for the wider pipe needs to be taken into account.

Link aggregation has its uses, but nothing will ever beat a single, large pipe like a true 10 Gigabit (or higher) network setup. Of course, a 10 Gigabit network will cost you dearly and link aggregation can help keep costs down.

Remember, planning is key. Keep expectations reasonable and make the required changes where they make sense. Happy networking!

trueCABLE presents the information on our website, including the “Cable Academy” blog and live chat support, as a service to our customers and other visitors to our website subject to our website terms and conditions. While the information on this website is about data networking and electrical issues, it is not professional advice and any reliance on such material is at your own risk.