Architecture for Modular Data Centers
Cloud-Computing Economies of Scale AWS Executive Symposium James Hamilton, 2009.11.10 VP & Distinguished Engineer e: [email protected] w: mvdirona.com/jrh/work b: perspectives.mvdirona.com
Agenda Infrastructure Efficiency at Scale Cloud services really are different Where does the money go? Where does the power go?
Cloud Computing Economics Why utility computing makes sense economically
Amazon Web Services Specialization 11/10/2009
http://perspectives.mvdirona.com
2
Economies of Scale 2006 comparison of very large service with mid-size: (~1000 servers): Large Service [$13/Mb/s/mth]: $0.04/GB Medium [$95/Mb/s/mth]: $0.30/GB (7.1x) Large Service: $4.6/GB/year (2x in 2 Datacenters) Medium: $26.00/GB/year* (5.7x)
Large Service: Over 1.000 servers/admin Enterprise: ~140 servers/admin (7.1x)
Large block h/w purchases significantly more economic Large weekly purchases offer significant savings H/W Manufacturers willing & able to do custom designs at scale
Automation & custom s/w investments amortize well at scale Summary: scale economics strongly in play 11/10/2009
http://perspectives.mvdirona.com
3
Services Different from Enterprises Enterprise Approach: Largest cost is people – scales with servers (~100:1 common) Enterprise interests center around consolidation & utilization • Consolidate workload onto fewer, larger systems • Large SANs for storage & large routers for networking
Internet-Scale Services Approach: Largest costs is server & storage H/W • Typically followed by cooling, power distribution, power • Networking varies from very low to dominant depending upon service • People costs under 10% & often under 5% (>1000+:1 server:admin)
Services interests center around work-done-per-$ (or joule)
Observations: People costs shift from infrastructure to supporting the business Expect high-scale service techniques to spread to enterprise Focus instead on work done/$ & work done/joule 11/10/2009
http://perspectives.mvdirona.com
4
Where Does the Money Go? Assumptions (not an Amazon facility): Facility: ~$200M for 15MW facility, 82% is power dist & mech (15-year amort.) Servers: ~$2k/each, roughly 50,000 (3-year amort.) Average server power draw at 30% utilization: 80% Server to Networking equipment ratio: 2.5:1 (“Cost of a Cloud” data) Commercial Power: ~$0.07/kWhr
Monthly Costs
Servers
4%
15% 44% 19% 18%
Networking Equipment Power Distribution & Cooling Power Other Infrastructure
3yr server & 15 yr infrastructure amortization
Observations: • •
62% per month in IT gear of which 44% in servers & storage Net gear costs high & only 38% in shell, power, power distribution, & mech.
Details at: http://perspectives.mvdirona.com/2008/11/28/CostOfPowerInLargeScaleDataCenters.aspx & http://perspectives.mvdirona.com/2009/03/07/CostOfACloudResearchProblemsInDataCenterNetworks.aspx 11/10/2009 http://perspectives.mvdirona.com 5
PUE & DCiE Measure of datacenter infrastructure efficiency Power Usage Effectiveness PUE = (Total Facility Power)/(IT Equipment Power)
Datacenter Infrastructure Efficiency DCiE = (IT Equipment Power)/(Total Facility Power) * 100%
http://www.thegreengrid.org/en/Global/Content/white-papers/The-Green-Grid-Data-Center-Power-Efficiency-Metrics-PUE-and-DCiE 11/10/2009
http://perspectives.mvdirona.com
6
Where Does the Power Go? Assuming above average datacenter at PUE ~1.7 Each watt to server loses ~0.7W in power distribution & cooling IT load (servers, storage, & networking): 1/1.7=> 59% • Networking under 4%
Power losses easier to track than cooling: Power transmission & switching losses: ~8% Cooling losses remainder:100-(59+8) => 33%
Observations: Server efficiency & utilization improvements highly leveraged Cooling costs incredibly high at ~1/3 Net gear not large power consumer 11/10/2009
http://perspectives.mvdirona.com
7
Agenda Infrastructure Efficiency at Scale Cloud services really are different Where does the money go? Where does the power go?
Cloud Computing Economics Why utility computing makes sense economically
Amazon Web Services Specialization 11/10/2009
http://perspectives.mvdirona.com
8
Infrastructure at Scale Datacenter design efficiency Average datacenter efficiency low with PUE over 2.0 (Source: EPA) • Many with PUE well over 3.0
High scale cloud services in the 1.2 to 1.5 range Lower cost & much better for environment
Multiple datacenters At scale multiple datacenters can be used • Close to customer • Cross datacenter data redundancy
• Address international markets efficiently
Avoid massive upfront data cost & years to utilize 11/10/2009
http://perspectives.mvdirona.com
9
H/W Cost & Efficiency Optimization Service optimized hardware Custom cloud-scale design teams: • Dell DCS, SGI (Rackable), ZT Systems, Verari, HP, …
Purchasing power at volume Supply chain optimization Shorter chain drives much higher server utilization • Predicting next week easier than 4 to 6 months out
Less overbuy & less capacity risk
Networking transit costs rewards volume Cloud services unblocks new business & growth Remove dependence on precise capacity plan 11/10/2009
http://perspectives.mvdirona.com
10
Investments at Scale Deep automation only affordable when amortized over large user base Lack of automation drives both cost & human error fragility
S/W investments at scale Massive distributed systems investments such as Amazon Simple Storage Service & Elastic Block Store hard to justify without scale
Special Skills with deep focus Distributed systems engineers, power engineering, mechanical engineering, server h/w design, networking, supply chain, 24x7 operations staff, premium support,…
11/10/2009
http://perspectives.mvdirona.com
11
Utilization & Economics Server utilization problem 30% utilization VERY good &10% quite common • Expensive & not good for environment
Solution: pool number of heterogeneous services • Single reserve capacity pool far more efficient • Non-correlated peaks & law of large numbers
Pay as you go & pay as you grow model Don’t block the business Don’t over buy Transfers capital expense to variable expense Apply capital for business investments rather than infrastructure
Charge models drive good application owner behavior Cost encourages prioritization of work by application developers High scale needed to make a market for low priority work 11/10/2009
http://perspectives.mvdirona.com
12
Agenda Infrastructure Efficiency at Scale Cloud services really are different
Where does the money go? Where does the power go?
Cloud Computing Economics Why utility computing makes sense economically
Amazon Web Services Specialization 11/10/2009
http://perspectives.mvdirona.com
13
Amazon Cycle of Innovation 15+ years of operational excellence Managing secure, highly available, multi-datacenter infrastructure
Experienced at low margin cycle of innovation: Innovate Listen to customers Drive down costs & improve processes Pass on value to customers
AWS announced price reduction October 27: Up to 15% off all EC2 instance families & sizes 11/10/2009
http://perspectives.mvdirona.com
14
AWS Approach Broad set of services: Infrastructure Services SimpleDB Simple Storage Service CloudFront Simple Queue Service Elastic MapReduce Relational Database Service Elastic Block Store Premium Support Virtual Private Cloud
Payments & Billing Flexible Payment Services DevPay
On Demand Workforce Mechanical Turk
Alexa Web Services Web Information Service Top Sites
Merchant Services Fulfillment Web Service
“Open the hood” approach Simple, layerable building block services Component services are substitutable 11/10/2009
http://perspectives.mvdirona.com
15
AWS Scale Looking at Quantcast Top 500k sites (11/2009) 53% of cloud hosted sites are using AWS 27% growth in AWS hosted site count Oct to Nov More sites than all others combined
High growth workloads: data Intensive computing, commercial HPC, analysis, & optimization Scale supports deep investment in automation, monitoring, operations, & funds faster innovation Sources: • http://www.quantcast.com/top-sites-1 • http://www.jackofallclouds.com/2009/11/state-of-the-cloud-november-2009/ 11/10/2009
http://perspectives.mvdirona.com
16
AWS Pace of Innovation • Reserved Instances in EU • Elastic MapReduce • SQS in EU
• New SimpleDB Features • FPS General Availability
11/10/2009
• RDS Announced • High-Memory Instances • Reduced EC2 Pricing
• AWS Security Center
• EC2 Reserved Instances • EC2 with Windows • EC2 in EU • AWS Toolkit for Eclipse • Lower pricing tiers for CloudFront • AWS Management Console
• AWS Multi-Factor Authentication • Virtual Private Cloud • Lower Reserved Instance Pricing
• Elastic MapReduce in EU
• AWS Import/Export • New CloudFront Features • Monitoring, Auto Scaling & Elastic Load Balancing http://perspectives.mvdirona.com
• EBS Shared Snapshots • SimpleDB in EU • Monitoring in EU • Auto Scaling in EU • Elastic Load Balancing in EU 17
Summary Benefit from AWS scale Higher h/w & infrastructure utilization Better environmentally
Achieve fundamental cost shift Convert capital expense to variable cost Lower operating costs at same time Adapts to inaccurate capacity plans Pay as you go & pay as you grow
Gain business agility Obtain h/w resources for new project or expansion in minutes rather than months Lower infrastructure cost/risk supporting more innovation Higher productivity 11/10/2009
http://perspectives.mvdirona.com
18
More Information Amazon Web Services: •
http://aws.amazon.com
Designing & Deploying Internet-Scale Services: http://mvdirona.com/jrh/talksAndPapers/JamesRH_Lisa.pdf
Where does the power go & what to do about it: •
http://mvdirona.com/jrh/TalksAndPapers/JamesHamilton_AFCOM2009.pdf
Recovery-Oriented Computing: http://roc.cs.berkeley.edu/ http://www.cs.berkeley.edu/~pattrsn/talks/HPCAkeynote.ppt http://www.sciam.com/article.cfm?articleID=000DAA41-3B4E-1EB7-BDC0809EC588EEDF
Autopilot: Automatic Datacenter Operation: http://research.microsoft.com/users/misard/papers/osr2007.pdf
Perspectives Blog: http://perspectives.mvdirona.com
Email: [email protected] 11/10/2009
http://perspectives.mvdirona.com
19
19
Agenda Infrastructure Efficiency at Scale Cloud services really are different Where does the money go? Where does the power go?
Cloud Computing Economics Why utility computing makes sense economically
Amazon Web Services Specialization 11/10/2009
http://perspectives.mvdirona.com
2
Economies of Scale 2006 comparison of very large service with mid-size: (~1000 servers): Large Service [$13/Mb/s/mth]: $0.04/GB Medium [$95/Mb/s/mth]: $0.30/GB (7.1x) Large Service: $4.6/GB/year (2x in 2 Datacenters) Medium: $26.00/GB/year* (5.7x)
Large Service: Over 1.000 servers/admin Enterprise: ~140 servers/admin (7.1x)
Large block h/w purchases significantly more economic Large weekly purchases offer significant savings H/W Manufacturers willing & able to do custom designs at scale
Automation & custom s/w investments amortize well at scale Summary: scale economics strongly in play 11/10/2009
http://perspectives.mvdirona.com
3
Services Different from Enterprises Enterprise Approach: Largest cost is people – scales with servers (~100:1 common) Enterprise interests center around consolidation & utilization • Consolidate workload onto fewer, larger systems • Large SANs for storage & large routers for networking
Internet-Scale Services Approach: Largest costs is server & storage H/W • Typically followed by cooling, power distribution, power • Networking varies from very low to dominant depending upon service • People costs under 10% & often under 5% (>1000+:1 server:admin)
Services interests center around work-done-per-$ (or joule)
Observations: People costs shift from infrastructure to supporting the business Expect high-scale service techniques to spread to enterprise Focus instead on work done/$ & work done/joule 11/10/2009
http://perspectives.mvdirona.com
4
Where Does the Money Go? Assumptions (not an Amazon facility): Facility: ~$200M for 15MW facility, 82% is power dist & mech (15-year amort.) Servers: ~$2k/each, roughly 50,000 (3-year amort.) Average server power draw at 30% utilization: 80% Server to Networking equipment ratio: 2.5:1 (“Cost of a Cloud” data) Commercial Power: ~$0.07/kWhr
Monthly Costs
Servers
4%
15% 44% 19% 18%
Networking Equipment Power Distribution & Cooling Power Other Infrastructure
3yr server & 15 yr infrastructure amortization
Observations: • •
62% per month in IT gear of which 44% in servers & storage Net gear costs high & only 38% in shell, power, power distribution, & mech.
Details at: http://perspectives.mvdirona.com/2008/11/28/CostOfPowerInLargeScaleDataCenters.aspx & http://perspectives.mvdirona.com/2009/03/07/CostOfACloudResearchProblemsInDataCenterNetworks.aspx 11/10/2009 http://perspectives.mvdirona.com 5
PUE & DCiE Measure of datacenter infrastructure efficiency Power Usage Effectiveness PUE = (Total Facility Power)/(IT Equipment Power)
Datacenter Infrastructure Efficiency DCiE = (IT Equipment Power)/(Total Facility Power) * 100%
http://www.thegreengrid.org/en/Global/Content/white-papers/The-Green-Grid-Data-Center-Power-Efficiency-Metrics-PUE-and-DCiE 11/10/2009
http://perspectives.mvdirona.com
6
Where Does the Power Go? Assuming above average datacenter at PUE ~1.7 Each watt to server loses ~0.7W in power distribution & cooling IT load (servers, storage, & networking): 1/1.7=> 59% • Networking under 4%
Power losses easier to track than cooling: Power transmission & switching losses: ~8% Cooling losses remainder:100-(59+8) => 33%
Observations: Server efficiency & utilization improvements highly leveraged Cooling costs incredibly high at ~1/3 Net gear not large power consumer 11/10/2009
http://perspectives.mvdirona.com
7
Agenda Infrastructure Efficiency at Scale Cloud services really are different Where does the money go? Where does the power go?
Cloud Computing Economics Why utility computing makes sense economically
Amazon Web Services Specialization 11/10/2009
http://perspectives.mvdirona.com
8
Infrastructure at Scale Datacenter design efficiency Average datacenter efficiency low with PUE over 2.0 (Source: EPA) • Many with PUE well over 3.0
High scale cloud services in the 1.2 to 1.5 range Lower cost & much better for environment
Multiple datacenters At scale multiple datacenters can be used • Close to customer • Cross datacenter data redundancy
• Address international markets efficiently
Avoid massive upfront data cost & years to utilize 11/10/2009
http://perspectives.mvdirona.com
9
H/W Cost & Efficiency Optimization Service optimized hardware Custom cloud-scale design teams: • Dell DCS, SGI (Rackable), ZT Systems, Verari, HP, …
Purchasing power at volume Supply chain optimization Shorter chain drives much higher server utilization • Predicting next week easier than 4 to 6 months out
Less overbuy & less capacity risk
Networking transit costs rewards volume Cloud services unblocks new business & growth Remove dependence on precise capacity plan 11/10/2009
http://perspectives.mvdirona.com
10
Investments at Scale Deep automation only affordable when amortized over large user base Lack of automation drives both cost & human error fragility
S/W investments at scale Massive distributed systems investments such as Amazon Simple Storage Service & Elastic Block Store hard to justify without scale
Special Skills with deep focus Distributed systems engineers, power engineering, mechanical engineering, server h/w design, networking, supply chain, 24x7 operations staff, premium support,…
11/10/2009
http://perspectives.mvdirona.com
11
Utilization & Economics Server utilization problem 30% utilization VERY good &10% quite common • Expensive & not good for environment
Solution: pool number of heterogeneous services • Single reserve capacity pool far more efficient • Non-correlated peaks & law of large numbers
Pay as you go & pay as you grow model Don’t block the business Don’t over buy Transfers capital expense to variable expense Apply capital for business investments rather than infrastructure
Charge models drive good application owner behavior Cost encourages prioritization of work by application developers High scale needed to make a market for low priority work 11/10/2009
http://perspectives.mvdirona.com
12
Agenda Infrastructure Efficiency at Scale Cloud services really are different
Where does the money go? Where does the power go?
Cloud Computing Economics Why utility computing makes sense economically
Amazon Web Services Specialization 11/10/2009
http://perspectives.mvdirona.com
13
Amazon Cycle of Innovation 15+ years of operational excellence Managing secure, highly available, multi-datacenter infrastructure
Experienced at low margin cycle of innovation: Innovate Listen to customers Drive down costs & improve processes Pass on value to customers
AWS announced price reduction October 27: Up to 15% off all EC2 instance families & sizes 11/10/2009
http://perspectives.mvdirona.com
14
AWS Approach Broad set of services: Infrastructure Services SimpleDB Simple Storage Service CloudFront Simple Queue Service Elastic MapReduce Relational Database Service Elastic Block Store Premium Support Virtual Private Cloud
Payments & Billing Flexible Payment Services DevPay
On Demand Workforce Mechanical Turk
Alexa Web Services Web Information Service Top Sites
Merchant Services Fulfillment Web Service
“Open the hood” approach Simple, layerable building block services Component services are substitutable 11/10/2009
http://perspectives.mvdirona.com
15
AWS Scale Looking at Quantcast Top 500k sites (11/2009) 53% of cloud hosted sites are using AWS 27% growth in AWS hosted site count Oct to Nov More sites than all others combined
High growth workloads: data Intensive computing, commercial HPC, analysis, & optimization Scale supports deep investment in automation, monitoring, operations, & funds faster innovation Sources: • http://www.quantcast.com/top-sites-1 • http://www.jackofallclouds.com/2009/11/state-of-the-cloud-november-2009/ 11/10/2009
http://perspectives.mvdirona.com
16
AWS Pace of Innovation • Reserved Instances in EU • Elastic MapReduce • SQS in EU
• New SimpleDB Features • FPS General Availability
11/10/2009
• RDS Announced • High-Memory Instances • Reduced EC2 Pricing
• AWS Security Center
• EC2 Reserved Instances • EC2 with Windows • EC2 in EU • AWS Toolkit for Eclipse • Lower pricing tiers for CloudFront • AWS Management Console
• AWS Multi-Factor Authentication • Virtual Private Cloud • Lower Reserved Instance Pricing
• Elastic MapReduce in EU
• AWS Import/Export • New CloudFront Features • Monitoring, Auto Scaling & Elastic Load Balancing http://perspectives.mvdirona.com
• EBS Shared Snapshots • SimpleDB in EU • Monitoring in EU • Auto Scaling in EU • Elastic Load Balancing in EU 17
Summary Benefit from AWS scale Higher h/w & infrastructure utilization Better environmentally
Achieve fundamental cost shift Convert capital expense to variable cost Lower operating costs at same time Adapts to inaccurate capacity plans Pay as you go & pay as you grow
Gain business agility Obtain h/w resources for new project or expansion in minutes rather than months Lower infrastructure cost/risk supporting more innovation Higher productivity 11/10/2009
http://perspectives.mvdirona.com
18
More Information Amazon Web Services: •
http://aws.amazon.com
Designing & Deploying Internet-Scale Services: http://mvdirona.com/jrh/talksAndPapers/JamesRH_Lisa.pdf
Where does the power go & what to do about it: •
http://mvdirona.com/jrh/TalksAndPapers/JamesHamilton_AFCOM2009.pdf
Recovery-Oriented Computing: http://roc.cs.berkeley.edu/ http://www.cs.berkeley.edu/~pattrsn/talks/HPCAkeynote.ppt http://www.sciam.com/article.cfm?articleID=000DAA41-3B4E-1EB7-BDC0809EC588EEDF
Autopilot: Automatic Datacenter Operation: http://research.microsoft.com/users/misard/papers/osr2007.pdf
Perspectives Blog: http://perspectives.mvdirona.com
Email: [email protected] 11/10/2009
http://perspectives.mvdirona.com
19
19
