Use Best Practices to Design Data Center Facilities
Michael A. Bell
Data centers seldom meet the operational and capacity requirements of their initial designs. The principal goals in data center design are flexibility and scalability, which involve site location, building selection, floor layout, electrical system design, mechanical design and modularity.
Publication Date: 22 April 2005/ID Number: G00127434 Page 2 of 26
© 2005 Gartner, Inc. and/or its Affiliates. All Rights Reserved.
TABLE OF CONTENTS
1.0 Introdu ction ...............................................................................................................
...................
4
1.1 Density vs
. Capacity .......................................................................................................
5
1.2 Rack
Layout................................................................................................................
....
6
1.3 Rack
Units .................................................................................................................
.....
7
2.0 Design
Criteria ............................................................................................................
.................
8
2.1 Critical Bu ilding Systems ................................................................................................
9
2.2 Planning and
Designing................................................................................................
10
2.3 Locat ion ...................................................................................................................
.....
11
2.4 Site.......................................................................................................................
.........
12
2.5 Architecture ...............................................................................................................
...
13
2.6 Power Di stribution ........................................................................................................
14
2.7 Power
Supply ...............................................................................................................
15
2.8 Mechanica l Systems.....................................................................................................
16
2.9 Raised-Access Floor ....................................................................................................
17
2.10 Fire Detecti on and Suppr ession .................................................................................
18
3.0 Data Center Constr uction Costs — Tier 2.................................................................................
19
3.1 Data Center Constr uction Costs
— Tier 3....................................................................
20
4.0 Data Center Facilit ies Managemen t Options:............................................................................
22
5.0 Conclusion .................................................................................................................
................
23
Appendix A. Conferen ce Poll Re sults ............................................................................................
.
23
Appendix B. Ac ronym Key........................................................................................................
.......
25
LIST OF FIGURES
Figure 1. The First Rule of Data Center Fa cilities: Design for Fl exibility and Scalability ..................
5
Figure 2. The Density vs. Capacity Debate......................................................................................
.
6
Figure 3. Rack Layout Re sulting Powe r Demands............................................................................
7
Figure 4. Use Rack Unit —
The Primary Pl anning Fa ctor................................................................. 8
Figure 5. Design Criter ia: Scope and
Redundancy ...........................................................................
9
Figure 6. Critical
Building Systems............................................................................................
......
10
Figure 7. Engineering
Plan and Spac e Design ...............................................................................
11
Figure 8. Locati on Selection ...................................................................................................
.........
12
Figure 9. Site and Buildi ng Selection
Criter
ia.................................................................................
.
13
Figure 10. Site and Building Sele ction Criteria
(Continued)............................................................
14
Figure 11. Power
Distribution ..................................................................................................
........
15
Figure 12. UPS/Ba ckup Gener ators ...............................................................................................
.
16
Figure 13. Mechani cal Systems ..................................................................................................
....
17
Figure 14. Ra ised Floors .......................................................................................................
..........
18
Publication Date: 22 April 2005/ID Number: G00127434 Page 3 of 26
© 2005 Gartner, Inc. and/or its Affiliates. All Rights Reserved.
Figure 15. Fire Detect ion and S uppression..................................................................................... 19
Figure 16. Data Center Co nstruction Cost s —Tier 2.......................................................................
20
Figure 17. Constructi on Costs — Tier 3 .........................................................................................
.
21
Figure 18. Conference Poll Re sults: Facility Problems ...................................................................
24
Figure 19. Conference Poll
Results: Tier Levels .............................................................................
24
Figure 20. Conference Poll
Results: Power Rating .........................................................................
25
Publication Date: 22 April 2005/ID Number: G00127434 Page 4 of 26
© 2005 Gartner, Inc. and/or its Affiliates. All Rights Reserved.
ANALYSIS
Management Summary
Data center facilities rarely achieve the operati onal and capacity requirements specified in their initial designs. The advent of new technologies, su ch as blade servers, that require substantial incremental power and cooling capacity; the pressu re to consolidate multiple data centers into fewer locations; the need for incremental space; changes in operational procedures; and potential changes in safety and security regulations conv erge to impose constant facilities changes on the modern data center.
The overarching rule in data center facilities is to design for flexibility and scalability. This rule embraces several key principles in the site loca tion, building selection, floor layout, electrical system design, mechanical design, and the conc ept of modularity that enables the data center facility to change and adapt as needed, with minimum renovation and change to basic building systems. This research delves into specific guidelines on achieving a high level of flexibility and scalability in the data center. These best practices address site location, building selection, and principles in the design and provisioning of critical facilitie s systems. In doing so, it examines the following
Client Issues:
•
What is the impact of new hardware technolog ies on the infrastructure of data center facilities? •
What are the critical design considerations and best practices in the site selection and design of data center facilities?
•
What cost elements are associated with differ ent levels of data center reliability and fault tolerance? •
How should the managem ent of data center fa cilities be organized?
1.0 Introduction
Strategic Planning Assumption: Through 2009, 70 perce nt of data center facilities will fail to meet operational and capacity requirements without some level of renovation, expansion or relocation
(0.7 probability).
The key to a successful data center facility — one that is sustainable in the long term — is to consider it as a receptacle for equipment and ope rations, as well as an integrated system, in which each component must be considered in the context of flexibility and scalability (see Figure
1).
Publication Date: 22 April 2005/ID Number: G00127434 Page 5 of 26
© 2005 Gartner, Inc. and/or its Affiliates. All Rights Reserved.
Figure 1. The First Rule of Data Center F acilities: Design for Flexibility and Scalability
Forces
of
Change
New equipment
C onsolidations and expansions
N ew redundancy requirements
Incremental power requirements
Incremental cooling demands
C onstrained floor space
N ew safety and security regulations
C hanges in operational procedures
C hanges in mission
C ost pressu re s
Murphy's Law: "What can go wrong — will!"
Bell's Law: "Murphy was an optimist."
Forces
of
Change
New equipment
C onsolidations and expansions
N ew redundancy requirements
Incremental power requirements
Incremental cooling demands
C onstrained floor space
N ew safety and security regulations
C hanges in operational procedures
C hanges in mission
C ost pressu re s
Murphy's Law: "What can go wrong — will!"
Bell's Law: "Murphy was an optimist."
Source: Gartner Research (April 2005)
Action Item: Stress flexibility and scalability in a ll aspects of the data center facility, including location, building selection, space layout and building systems design.
1.1 Density vs. Capacity
Tactical Guidelines: As a general rule, plan for the data center to scale from 50 watts to
100 watts per square foot (that is, in a raised floor area); increase capacity on a modular basis. Assess the trade-offs between space and power in the total cost of the new facility.
Provide additional space between racks for air circulation.
Client Issue: What is the impact of new hardware technologies on the infrastructure of data center facilities?
Evidence is growing that data centers are being ov er-designed for electrical capacity, because of concerns about meeting the incremental pow er and cooling demands of modern server equipment (such as blade servers). Sufficient c apacity should be delivered relative to patch panels, conduits and intermediate distribution f eeds. Special attention should be focused on equipment densities relative to initial and longer-term electrical power capacities. Electrical capacity ranging from 30 watts to 70 watts per s quare foot for computer equipment is typically required. However, additional power is required for air conditioning, humidification, lighting, and uninterruptible power supply (UPS) and transform er losses. This additional demand could add one to one-and-a-half times more wattage to the electrical load, depending on equipment spacing and air-handling efficiencies. In Figure 2, heat densities are expressed in watts per square foot for the equipment footprint. Actual watts per square foot over the raised floor would be substantially less when aisle space — and other space — is included in the calculation.
Publication Date: 22 April 2005/ID Number: G00127434 Page 6 of 26
© 2005 Gartner, Inc. and/or its Affiliates. All Rights Reserved.
Figure 2. The Density vs. Capacity Debate
Product Heat Density Trends Chart
Pow er
Spa ce
Total cost
$
Space/Power Trade-off
• Focus on density vs. capacity
• Provide ample clearances between racks
• Provide expansion space
• U se actual connected load, not equipment nameplate load
• Be mindful of different measures • Plan to scale up modularly
Units of Measure
Heat Load per Product
Footprint
(watts/m
2
)
Heat Load per Produc t
Footprint
(watts/ft.
2
)
Year of Product Announcement
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
10,000
8,000
6,000
4,000
2,000
1,000
800
600
400
200
100
80
60
100,000
80,000
60,000
40,000
20,000
10,000
8,000
6,000
4,000
2,000
1,000
800
Tape Storage
Systems
Workstations
(standalone)
Servers and Disk
Storage Systems
(1.8–2.2 m tall)
Communication
Equipment
(frames)
Product Heat Density Trends Chart
Pow er
Spa ce
Total cost
$
Space/Power Trade-off
Pow er
Spa ce
Total cost
$
Space/Power Trade-off
• Focus on density vs. capacity
• Provide ample clearances between racks
• Provide expansion space
• U se actual connected load, not equipment nameplate load
• Be mindful of different measures • Plan to scale up modularly
Units of Measure
Heat Load per Product
Footprint
(watts/m
2
)
Heat Load per Produc t
Footprint
(watts/ft.
2
)
Year of Product Announcement
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
10,000
8,000
6,000
4,000
2,000
1,000
800
600
400
200
100
80
60
100,000
80,000
60,000
40,000
20,000
10,000
8,000
6,000
4,000
2,000
1,000
800
Tape Storage
Systems
Workstations
(standalone)
Servers and Disk
Storage Systems
(1.8–2.2 m tall)
Communication
Equipment
(frames)
Source: The Uptime Institute and the Thermal Management Consortium, a collaborative effort of Amdahl, Cisco
Systems, Compaq Computer, Cray, Dell, EMC, Hewlett-Packard, IBM, Intel, Lucent Technologies, Motorola, Nokia,
Nortel Networks, Sun Microsystems and Unisys
Gartner recommends that the data center be designed to scale from 50 watts to 100 watts per square foot for raised floor areas. Increment al capacity can be added on a modular basis.
Action Item: Focus on density vs. capacity; consider more-liberal spacing between racks to mitigate heat gain by improving airflow.
1.2 Rack Layout
Tactical Guideline: Distribute high-density racks throughout the layout to mitigate hot spots; use spot cooling as necessary.
Modern blade servers can be packed into a single-ra ck enclosure, resulting in power demands of
18 to 20 kilowatts per rack. The primary issue wi th dense-packing the layout with high-capacity