Skills: How to improve data center cabling?

When you step into most data centers, administrators usually first apologize for messy wiring. Wiring is simple - connect the server to the network switch with a cable. However, the cable is placed like a "hanging surface", which affects the space utilization and aesthetics of the cabinet. The chaotic scene is like a lot of snakes entwined and cannot be sorted out. We understand how things have evolved to this point, but we are unable to determine the root cause of this result. Is this chaos the result of the constant evolution of technology? More importantly, how do you improve data center cabling?

The article will start with three main challenges and explain how to design and implement a data center cabling infrastructure: topology, quantity, and cable type.

Data Center Topology

Data Center Cabling is different from traditional building wiring. Cable lengths are shorter and performance requirements are higher. Unlike regular offices with only two or four terminals, a single cabinet may require 24, 48, 96 or more network cables or fiber connections.

Three basic methods of data center wiring:

1. Point to point

2. End of line convergence

3. Convergence of cabinet tops (using fiber optic or Switch)

Introduce point-to-point wiring. This is the wiring method that the data center has used for many years, and the results are as we have seen before - a mess. Point-to-point means pulling the network cable under the floor, in the air (whether or not it is wired) or through the server cabinet. Cables are usually made on the spot or directly use existing links. Old cables are usually not removed or marked, making tracking and finding more difficult, resulting in data center personnel needing to be maintained in a “snack” environment. Servers and jumpers can confuse cable distribution, and forms become more severe when high-density network switches are installed in cabinets or elsewhere. In short, peer-to-peer mode is no longer good for data centers.

End-of-row convergence architecture

End-of-line convergence, also known as "regional convergence," is a common method of wiring in data centers in recent years. In the end-of-line cabling architecture, there are places dedicated to placing patch panels and aggregation switches, usually at the end of the same row of cabinets—sometimes in the middle—to facilitate the connection of the entire set of server cabinets. Patch panels are installed in each slot, and permanent cables are connected to the corresponding aggregation cabinet through these racks. Other cables are usually fiber optic and are used to connect the aggregation layer equipment to the core network, a.k.a main distributed framework or area. It can be thought of as a compressed version of the standard "star topology" in the data center. The server cabinet is like an office, and the aggregation cabinet is an intermediate distribution frame (IDF) or a communication room (TR).

End-of-row convergence simplifies the difficulty of adding hardware by simply connecting the server to the patch panel to which you want to connect, and then connecting the patch panel to the corresponding aggregation layer switch. Two short jumpers are required for each connection, which is also beneficial for future installation and maintenance.

Similarly, access layer switches are connected to the core network via fiber optics, so the entire installation process involves only simple jumpers. The “tail-of-line” aggregation minimizes the length of the required jumpers, as all patch panels have labels that are easy to track, and documents that record relevant information are easier to read. Not only that, because longer cables are no longer needed, there is no reason to keep old, long, and inferior cables. This avoids system performance degradation due to jumper compatibility issues.

Casing Convergence Architecture

Casing Convergence is the latest wiring method, but can be seen as a variant of the end-of-line convergence. The top-of-rack convergence depends on the server configuration density, and we will discuss how to choose the appropriate wiring scheme for the data center cabling architecture. If a cabinet is full of servers, the preferred solution is to install an access layer switch on each cabinet. If the server has dual NICs and there are fault tolerance requirements, consider installing one more switch. Typically, a lower-level switch will be installed for system management. All switches are connected to the core network via fiber patch cables, similar to end-of-line aggregation. Therefore, the effect of the top of the cabinet is the same as that of the end of the line, but it is more conducive to expansion - you can install cheap switches on each cabinet according to actual needs, and you can also complete the migration through reconfiguration. The impact of a single switch failure is much less than the failure of a large switch at the end of the line.

When you encounter a blade server that requires a specific model switch, the top of the cabinet is your best choice. In this case, all we have to do is connect the fiber to the core network. Therefore, as long as each cabinet has the same fiber optic equipment, network convergence can be achieved. Sometimes you will even encounter a "hybrid" installation, but the overall structure is not affected, so you only need to connect different blades to different switches. In general, ordinary fiber optic equipment can meet high performance applications.

Choosing the Best Data Center Cabling Architecture

How to choose the end of the line or the top of the cabinet depends on the device density, application throughput requirements and hardware connection details. In a large data center, two design approaches can leverage their strengths in different areas of the application. In short, the answer is not unique. The overall trend will be biased towards fiber optic applications, and the choice of cabling architecture is based on “on-demand”, which is the key to any business.

Storage networks have used Fibre Channel connections for many years and are built with cabinet tops. In a storage area network (SAN), fiber patch cords connect patch panels to storage-connected switches. But this requires an extra network per cabinet - such a solution is not very cost effective.

The latest trend is Fibre Channel over Ethernet (FCoE), which allows SAN connections to use the same topology and use fiber paths as transport networks. FCoE can reduce physical line requirements, increase bandwidth and aggregate networks and consolidate SAN management. Currently FCoE is still in its infancy, and not all manufacturers are compatible with this model. Moreover, depending on the network cable function, how to connect storage depends on the hardware network connection requirements and other factors. Nonetheless, FCoE may become a common standard supported by manufacturers in the near future.

Author: Robert E. McFarlane Translator: Chen Dewen