Power is and always will be an issue in the datacenter, especially as gear gets denser, faster, and hotter. Power circuits can be sized improperly from lack of due diligence or simply devolve as equipment moves in and out of racks. Having gone through a few datacenter build outs I wanted to share some of my methodology for power sizing and planning at the rack level.
Power analogy
Think of electrical circuits and delivery through copper wires like water through piping. There are 3 main facets that we are concerned with in sizing the power infrastructure: voltage, amperage, and power. Resistance, measured in ohms, is equivalent to the size of the water pipe, but since power cables are provided to us by electricians and vendors, we’re not overly concerned with that aspect here. (Although I should mention that it is important to ensure that the power cables used to connect to equipment power supplies are capable of max PSU draw). Voltage is equivalent to the pressure (force) of the water flow. Amperage is equivalent to the rate of flow (current) of the water and also helps us determine efficiency. The power of the circuit is determined by multiplying the force (voltage) by rate of flow (amps). Assuming a Power Factor of 1, this equation also determines volt-amps, most commonly rated in Kilovolt-amps (Kva). This of course changes a bit in a three-phase circuit but I’ll get to that later. In either case, increasing the force or current flow will result in more power generated.
Watts or volt-amps (single-phase) = Voltage x Amps
Efficiency
In most cases, when you increase the voltage supplied to a server, storage, or networking device, it becomes more efficient in the way it utilizes power. In other words higher voltage = reduction in current draw. For example, an HP DL380 G6 with a 750 watt power supply can, as rated by HP, draw up to ~8 amps @ 110 volts. If you provide this same server with a 208v power circuit its draw decreases to just over 4 amps. The consumed current continues to decrease as you increase the voltage. These power supplies can go up to 240v reducing the current draw to a nominal 3.6a. That’s a 45% difference in draw between the lowest and highest capable voltages. The advantages of higher voltage in power constrained environments is obvious.
Sizing
So how big do you go? Single phase? Three-phase? 110? 208? 220? An enterprise-class UPS typically has 3 power phases: A, B, and C. The utility power circuits attach here usually as 480v @ 600a mains, two for redundancy. That’s 1200 amps of draw and any number of voltage scenarios that can be built off of the breaker box on the other side of the UPS, assuming it can handle it. Each phase has a maximum current and voltage depending on the size of the breakers installed in the UPS. You can quickly get a general idea on how much current your datacenter is consuming by looking at the load levels of each phase on the UPS, displayed in amps.
Let’s assume that the power infrastructure has been properly built and that the internal UPS breakers are setup so that all available power is being distributed to breaker boxes evenly. We need to figure out what size circuits, called whips, need to be pulled from the breakers to the racks. You can calculate based on averages or peaks. Calculating based on peak loads will make for absolute certain that you have enough available juice to power all of your attached gear if it were running at 100% load. If this is unlikely to ever occur in your environment, you should be ok using the averages.
There are 2 factors you need to figure out to properly size your power circuits: total power required (watts) and how much draw (amps) at a given voltage. As I stated before, this number will vary depending on input voltage. All equipment power supplies are rated by the manufacturers of the devices. From each device you plan to place in a single rack, add the rated draw from each system’s power supply and the required watts of each. You will need to ensure that you can provide enough total power as well as draw.
For example we determined that we need: 6000 watts and 25 amps total with 208v delivery. To meet these demands, assuming two circuits per rack for redundancy, the minimum circuit size we could run would be 208v @ 20a single phase. This yields 4.2Kva per circuit or 8320 watts total delivered to the rack ((208 x 20)x2). If 110/120 power is all that you have available, then you need to re-calculate your equipment requirements because current draw will likely double at this reduced voltage.
If you aren’t completely filling your target racks then don’t forget about providing room for growth. You may only need 6Kva day one, but your rack is only half full. In the example above, the two 208v @ 30a circuits will give us 28% additional headroom. If we plan to double the same type and number of equipment in this rack then these circuits will not be enough. If we plan for a 50% potential increase then we need to provide enough power for 12000 watts of power and 50 amps of draw. To meet this demand we’ll need to increase our circuit size to 30 amps at 208 volts. This will provide 6.2Kva per circuit for a total of 12.4Kva. This is a solid and safe sizing estimate for our new rack.
Three-Phase Power
While extremely efficient and better for redundancy, 3-phase circuits can be more expensive because of the increased power output. This is where we have to remember the overall Kva output of our UPS. If not careful, then we can overrun its capabilities. To calculate the power rating of a 3-phase circuit, start with the formula above, then multiply by 1.732.
Watts or volt-amps (three-phase) = Voltage x Amps x 1.732
Referring back to the scenario above, we need 12000 watts and 50 amps for our new rack. In a single-phase solution we need 30 amp circuits running at 208 volts. If we took those specs and turned them into 3-phase circuits, we would get 10.8Kva per circuit or 21.6Kva for the entire rack, a lot more than we need. Because of the increased output of 3-phase power we could now actually reduce our circuit size to the rack. 208v @ 20a, 3-phase, yields 7.2Kva per circuit or 14.4Kva total which is closer to our calculated estimate. 3-phase power saves us 20 amps of draw from the UPS on this particular rack!
Your particular power requirements may reveal that 3-phase power is unfeasible as it might produce too much power for your given application potentially overwhelming your UPS. I just went through a scenario where single-phase actually made more sense based on current load of the UPS and our power requirements. There are certainly more factors that could be considered but this should give you a good basis for determining power needs in your environment.
References:
0 Comments