In this article, Jerry Blair takes a closer look at the topic of density within a colocation data center—and how it can affect cost, performance, and availability.
Balancing Space, Power, and Cooling
When executives are asked what they want to know about their company’s colocation data centers, they focus on whether applications are available, will they perform as advertised, and what the cost is. However, they often don’t ask about another important factor—density—which has a direct impact on cost, performance, and availability.
Density is the amount of power a data center can provide within a cabinet and what is required to run the application (servers and networking gear) per cabinet. The greater the density, the more compute resources that are available and the faster applications can run.
In this blog, we explain why colocation density should matter to executives, providing an overview of the three key factors IT can use to present the concept. This can help open the door to discussions on how much density the company requires and which colocation data center in which to invest.
As we demonstrate, costs can come down to a delicate balance between space, power, and cooling. While less density makes sense in some cases, it’s always a good idea to be able to acquire more power capacity to handle a hardware refresh or if compute requirements spike.
Density Factor #1 – Less Space Lowers Colocation Costs
The first density factor to consider is the space required to hold all the racks your business needs for servers and networking gear. Colocation providers factor how much space your environment consumes into your monthly rate. This may be directly defined by the square feet of cage space you consume or factored into the kW rate you pay for power (which includes floor space). Either way, it’s usually to your benefit to use the least amount of space possible while using higher power density per cabinet (watts per square foot).
One way to achieve this is to use taller racks. The traditional rack height for years was 42U, but now, many data center providers support 48U or even 52U. A 52U rack provides roughly 25% more space than a 42U rack, allowing you to dramatically shrink your footprint.
Just think, for every five 42U racks, you only need four 52U racks. In a 10,000SF data center hall that supports 400 racks, this allows for an additional 4,000 1U servers. The taller 52U cabinets can also save you on the cost of networking gear for the project.
Density Factor #2 – Space Density and Big Data Drive Higher Power
The drive for better performance—combined with the drop in price for multi-processor blades—has led many IT departments to move to denser server systems. Whether these are 1U servers, blade servers, or even hyperconverged systems, it’s now normal to see rack densities at 10kW of power (up from 3-5kW per rack).
On top of this, the extreme growth of data and the demand for analysis of Big Data has increased dramatically. Just think of all the modern services performed on the vast amounts of data that users currently store: contextual searches of their photos, scrubbing of real-time videos for offensive content, analysis of shopping patterns, and targeted marketing to name a few.
These services and many others all run extremely compute-intensive artificial intelligence, which demands high power consumption hardware. This has led to special HPC (High Performance Computing, i.e., supercomputers) hardware that can drive power consumption at the rack level to 20kW, 50kW, or even 100kW and above.
Density Factor #3 – Cooling Required to Offset Density Heat
Packing a lot of power into less space leads to the third density factor—cooling. While just about all data centers can provide the power for high densities, the issue is cooling the generated heat. It doesn’t do any good to provide 20kW of power to a rack if you can cool only 10kW. The same is true if there’s not enough airflow to properly cool the equipment in your rack.
The best way to address these issues is to have a conversation with your provider on what power densities they can cool and if they have tested to verify them. Additionally, it’s always helpful for them to have existing customers that they can point to as examples of successfully cooling high-density environments.
In general, a rack on a standard raised floor data center with standard server hardware (including blade systems), should support 8kW of active load with no extra cooling measures (standard hot aisle and cold aisle). Beyond that, you will often need some form of containment, whether it’s chimney cabinets, hot aisle, or cold aisle containment.
All of these methods will often support 15-25kW of active load, depending on the data center, and specific server hardware could potentially support a higher load. Going beyond 25kW per rack will almost always require some form of specialized cooling strategy, which can include chilled water rear-door heat exchangers, liquid immersion, or water/coolant piped directly to the server chips.
Again, the key to meeting your cooling requirement is to know your requirements and have the conversation with your provider on how best to meet them.
Use Case: More Density Reduces Networking Gear Costs by 50%
As an example of balancing density requirements, one of our clients had an HPC environment that required approximately 1500U of rack space. They considered footprints with 60 racks and 17kW per rack (effectively only filling half of each rack) as well as 30 racks and 34kW per rack.
The 17kW design was their default as that is the max that many data centers can support. However, this would have roughly doubled the cost of their expensive networking gear, approximately $30K per rack. By allowing the client to use the higher density 34kW design, we cut the number of required racks in half to allow them to save roughly $900K in network hardware costs.
Help to Assess Complex Options
As you assess the ideal colocation data center density for your enterprise applications, your options can be complex. If you work with one of the leading colocation providers, they can help you study the cost options and find the optimal balance among space, power, and cooling.
To determine how well a colocation provider solves this challenge, ask about the power densities they can support per rack and what methods they use to do so. Additionally, check to see if they have high-density customers today, request a description of what they are doing, how they met the density request, and what they have learned from supporting these high-density customers.
The colocation density your company ultimately needs will, likely, be driven by your application requirements along with the upfront installation costs compared to the ongoing power and cooling costs. Either way, the ROI will be more favorable if you form a long-term partnership with your colocation provider. You may not need high density cabinets today, but it would be critically important to know they are available and can scale quickly if the business requirements change tomorrow.
About the Author: As co-founder, Jerry Blair was instrumental in DataBank’s inception in 2005. In his role, Blair is charged with executing on the company’s sales strategy. With a successful track record spanning more than 20 years in senior sales management, Blair’s experience and proven ability to implement results-driven direct and channel-focused sales programs is a very welcome continued asset to the company. Prior to DataBank, Blair was Vice President of Sales for Switch and Data and LayerOne. He has also served as General Manager of Sales for Lucent Technologies and has held sales management positions with various industry leaders including ICG Communications, Nortel Communications and Wellfleet Communications.
Industry insights, exciting announcements, and innovative exhibitions from across the global ICT infrastructure industry will fill the Gaylord National Resort & Convention Center in National Harbor, MD August 30 – September 2, 2021, for ITW 2021. This year’s event titled ‘Global Connectivity Reimagined’ will provide a blended in-person and virtual experience, allowing local and international attendees to come together, however they’d like, wherever they are in the world.
This year, edge infrastructure provider, DataBank, is ITW’s Datacloud and Edge Track sponsor. Raul Martynek, CEO and telecom veteran of 20+ years, will share the stage with some of the industry’s top market drivers and disruptors, including Bill Barney, CEO, Turbidite and Chris Sharp, CTO, Digital Realty to discuss how telcos can capitalize on the cloud and the edge during a panel on Monday, August 30.
“The pandemic accelerated digital transformation across the ICT infrastructure landscape creating incredible opportunities for organizations of all types,” comments Martynek. “ITW brings together some of the industry’s top data center, cloud, and edge space industry leaders for an incredible opportunity to share new ideas and explore new opportunities to partner and collaborate with industry leaders and decision-makers.”
Over the past year, DataBank has played a significant role in expanding the landscape of digital transformation across the U.S. and Europe. The company has experienced tremendous growth with expansions in key markets including Dallas, Denver, Pittsburgh, Indianapolis, and Salt Lake City; and with the closing of its acquisition of zColo, DataBank now operates over 60 data centers.
In addition to hearing from Martynek at the Datacloud and Edge Track panel discussion, companies also have a chance to meet with DataBank key executives and top sales representatives at ITW. Interested companies can book a meeting by clicking here.
A Closer Look at DataBank’s Denver Metro
The mile-high city of Denver, located at the foothills of the Rocky Mountains, has become a magnet for business growth, talent, and innovation. Denver’s economy ranks fourth for fastest growth in the last five years among the top 20 cities and metros in the U.S. Additionally, the Denver Technical Center has driven demand for Denver colocation options.
Geographically speaking, Denver is an ideal location for network connectivity in the United States: Enterprises can easily connect with customers and partners on either coast or in major cities along the way. The entire metropolitan area enjoys relatively low electricity pricing, which can be a valuable advantage in a colocation facility.
Denver is also a low risk for natural disasters that are common in coastal cities or on the plains, making it the perfect destination for ongoing business continuity, operations resilience, and redundancy for disaster recovery.
DataBank is proud to have four Denver metro data center locations. Combined, they provide more than 50,000 square feet of data center space and 8.5 megawatts of power. Our facilities also offer access to a wide range of fiber, internet, and connectivity options and can be supported by DataBank’s managed services. Each facility is also a professional work environment with loading docks, package service, parking, conference rooms, office space, workspace, and WIFI.
There’s great promise in the Denver-area tech ecosystem. Boulder County is among the nation’s most educated populations and one of the highest concentrations of advanced technology workers. The area also features world-class researchers at the state’s flagship university, the University of Colorado, and more than a dozen major federal facilities.
In many ways, our presence in Denver offers the best-of-the-best approach to colocation. With the perfect location and the right partner delivering all the services you need, DataBank’s Denver metro is the perfect advantage for any company looking to compete regionally, nationally, or on a global level.
Interested in learning more? Discover what makes DataBank different, schedule a tour, or request a no-obligation quote today.
DataBank’s Denver Metro by the Numbers
8.5
Total Megawatts
113+
Total Carriers
53,000
Total Square Feet
DataBank’s Denver Metro Data Centers
DEN1: Englewood Data Center
393 Inverness Parkway, Englewood, CO
19,000 square feet, 2 MW of on-site power, an N+1 power design, and N+1 cooling
Explore this data center
DEN2: Centennial Data Center
6900 South Peoria Street, Centennial, CO
27,000 square feet, 4 MW of on-site power, an N+1 power design, and 8 carriers
Explore this data center
DEN3: Downtown Denver Data Center
1500 Champa Street, Denver, CO
1,000 square feet, 2 MW of on-site power, an N+1 power design, and 8 carriers
Explore this data center
DEN4: Westminster Data Center
7579 W. 103 Avenue, Westminster, CO
6,000 square feet, 500 kW of on-site power, an N+1 power design, and N+1 cooling
Explore this data center
New Strategies for Network Service Providers
In our first article in this series about edge computing, we talked about the edge from the perspective of hyperscale cloud providers. While the forces shaping edge computing are the same, the strategies and outcomes will be different for network service providers. By carefully evaluating their options, data center locations, and required architectures, these companies can make the best choice to operate at the mobile edge and capitalize on all that 5G has to offer.
It seems like you can’t go a day without hearing about the imminent arrival of 5G, and the transformative impact it will have on consumers and businesses.
As a quick overview, 5G is the fifth-generation technology standard for broadband cellular networks and will deliver quantum leaps in performance and capacity in three categories: enhanced Mobile Broadband (eMBB), massive Machine-Type Communications (mMTC), and Ultra-Reliable Low-Latency Communications (URLLC)—all critical factors for providing better applications, content experiences, and services in the near future.
The rollout of 5G officially started in some markets in 2019, yet its full implementation is expected to take a few more years before it will be available in all areas. Even so, adoption is expected to surge: According to recent research, by 2022, there will be approximately 1 billion 5G subscriptions worldwide. In the U.S. alone, an estimated 50 percent of all mobile connections are expected to be 5G-enabled by 2023.
Game-Changing Use Cases—and Implications
5G will inevitably lead to much more data creation and processing as well as internet exchange and peering at more locations since application support is central to its design. This, in turn, will require even more real-time data processing and storage. All of this will be needed to support next-generation business concepts and use cases such as:
- The increased popularity and rollout of Amazon Go-inspired stores that rely on on-premises, high performance computing clusters to support the automation, machine learning, and artificial intelligence workloads that make these stores so successful.
- Supermarkets that will continue to deploy innovative new technologies to monitor in-store customer behavior to identify shopping patterns and understand buyers’ decisions.
- Increased use of advanced analytics and facial recognition technology to reduce risk in large crowds or at popular events such as the Super Bowl.
Clearly there will be many more use cases and new commercial 5G products and services—many of which we can’t even predict yet—but all will require new ways of operating at the edge for today’s network service providers.
New Table Stakes For 5G Success
While some of the previous examples are speculative, it is certain that 5G will completely change the topology of networks from core to edge. This shift will create valuable new opportunities for
network providers as well as fiber companies and multi-tenant and modular data center providers.
Additionally, there will be an increase in the number of nodes—as well as the emergence of entirely new types of nodes—to support 5G’s low-latency capabilities. This will drive demand for secure data center space at more locations for compute, storage, and interconnection as well as the need for more networks and fiber between the nodes for aggregation.
The effort is already underway. We’re beginning to see examples where 5G network builders (Dish Networks, AT&T, Verizon) are collaborating with terrestrial fiber providers (Zayo), cloud computing companies (AWS) and data center providers such as DataBank. These companies are all working together to build the ideal network infrastructure to capitalize on all 5G has to offer.
The Shift from a Terrestrial-Powered Edge to a New Wireless Edge
In the past, the entire industry leaned toward a centralized model where servers, fiber optic cables, and data storage were consolidated in a relatively small number of traditional data centers and locations. This model helped companies scale data storage and processing while maximizing efficiency and reducing costs. For example, data was created, processed, and stored centrally using one or more public cloud availability zones or data centers in East, Central, and West Tier 1 markets.
These networks tended to be characterized by large network and fiber providers who were naturally drawn to Tier 1 markets and a high number of connections occurring to and from these zones and data centers. Additionally, internet exchange and peering was happening in the large carrier hotels within these regions.
Yet, with the emergence of 5G and the new era of mobile edge computing, far more data will be created at the edge and the “far edge” and will need to be processed and stored there to support IoT, robotics, gaming, autonomous machinery, and latency-sensitive applications and workloads.
In this case, these networks will be characterized by smaller, shorter connections to aggregation points. One example of this is Open Random Access Networks (O-RAN), a network architecture designed specifically to support 5G mobile networks. Instead of using a one-to-one correlation between remote radio units and baseband units, O-RAN now enables virtualization through the baseband pool, including networking, computing, and storage. O-RAN also uses open platforms and real-time virtualization technology to improve scalability and support multi-vendor, multi-technology environments. As a result, O-RAN hubs will be an important part of any mobile edge strategy, and one that network architects will need to carefully consider.
In these mobile edge environments, network providers will increasingly be drawn to Tier 2 and Tier 3 markets, expanded geography within Tier 1 markets and the multi-tenant data centers located in them. They are also expected to take advantage of modular data center platforms that can be “dropped” anywhere with closer proximity to fiber and cellular tower infrastructure. They will also be able to perform localized internet exchanges in facilities that are more open and neutral than past options.
Opportunities for Network and Fiber Providers
It’s important to note that wireless networks still need globs of terrestrial fiber to connect the nodes and perform important tasks such as aggregation. The expected spike in nodes and O-RAN infrastructures will require more fiber to connect them all, and more fiber will be needed as more Tier 2 metro markets continue to expand or build new data centers and adopt the 5G network architecture.
This means fiber will continue to be a crucial piece of the overall equation. With the incorporation of millimeter wave and mid-band spectrum, propagated from a larger number of nodes, fiber backhaul for capacity and to provide critical on-ramps to the cloud are essential. It is for this reason that Dish Networks recently signed a major fiber deal with Zayo to provide backhaul and fronthaul transport for its 5G networks.
Opportunities for Multi-Tenant Data Centers and Modular Data Center Operators
At the same, time, we believe 5G represents a significant opportunity for multi-tenant data centers and modular data center operators. More distributed compute and storage infrastructure will need more secure data center space to host edge computing, edge cloud nodes, and edge interconnections.
Yet, transitioning from 4G to 5G networks is not just an equipment upgrade; instead, it requires a fundamental change in network topology and new sites to operate at shorter distances and higher speeds (up to 250 mph). The industry is growing to keep up with demand. As evidence, research from Small Cell Forum cited the doubling of cell sites, from four million in 2019 to more than eight million in 2020. These numbers will continue to grow to keep pace with 5G.
Building 5G networks from scratch does have its advantages—and presents new opportunities within the industry. This decision allows network providers to be more strategic in their data center location, compute architecture, and interconnections. It also gives them more flexibility and control than working with a public cloud vendor who may claim to provide 5G edge computing, but in reality, is still heavily reliant on the same centralized model.
Multi-tenant data center providers, such as DataBank, who already operate multiple data centers in Tier 1/2/3 markets are well-positioned to support network aggregation points (such as O-RAN) and emerging computing and cloud initiatives. Additionally, modular data center operators can now create a mobile edge compute and internet exchange node wherever it’s needed.
About the Author: Raul Martynek joined DataBank in June of 2017 as the CEO and is a 20+ year veteran in the telecom and internet infrastructure sector, having held senior positions at several communications and networking companies as well as asset management firms. Raul earned a Bachelor of Arts in Political Science from Binghamton University and received a master’s degree in International Affairs from Columbia University School of International and Public Affairs.
Frequently Asked Questions