Transformation to Cloud Computing - Semantic Scholar
New York University, Course CSCI-GA.3033-010, Fall 2015
Nikolai Joukov John J. Rofrano Murthy V. Devarakonda Harigovind Ramasamy
Transformation to Cloud Computing
Outline ! Background on the course ! The approach to transformation
! Course Details
2
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
This Course is a Bit of a Horse of a Different Color
! This course it being taught by a team of IBM Researchers who are experts in their field ! There is no text book because “the book” is still being written ;-)
3
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Transformation to Cloud Computing
What is Cloud Computing? …and why would anyone want to transform to it?
4
Examples of Cloud Computing Computing as a Service SaaS, PaaS,IaaS
Utility Computing Shared Resources
5
On Demand Provisioning
Consider the Possibilities that Cloud Computing Opens Up?
“On the Cloud, nobody knows you’re a small business!” * Peter Steiner, The New Yorker, July 5, 1993 6
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Cloud is… All of The Above
7
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Some Cloud Infrastructure Services
8
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Some Cloud Platform Services
9
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Some Cloud Software Services
10
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
What we mean when we say “Cloud Computing” ! A fundamental shift from physical infrastructure to virtual infrastructure – Seen as a major paradigm shift
! A rapid way of provisioning and later releasing computing services on the network ! Some important characteristics: – Rapid/automated provisioning and (later) release of services – Can be Pay-as-you-go – Appearance of infinite resources – Could be managed or unmanaged – and more…
! Types of Clouds: – Based on service: • Software as a Service (SaaS) • Platform as a Service (PaaS) • Infrastructure as a Service (IaaS) – Based on ownership • Public Clouds • Private Clouds • Hybrid Clouds
11
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Transformation
What is Transformation to Cloud?
12
Transformation is Migration ! Since the cloud is virtual you cannot “move” a “physical” computer into a cloud – But you can move the disk image into the cloud (which can be just as good) • …unless you had lots of “junk” on your computer because you might not want that on your new cloud server (note: we call this “your mess for less”) – Or you can build a new server in the cloud that looks just like the old server • When we build a new server we sometimes call this “workload” migration because we are migrating the “work” that the server is doing into another server in the cloud. ! What if the cloud doesn’t support your server’s OS platform? – Now you have to “re-platform” to a new OS which could get tricky – This is a lot like building a new server but what about all of the software? Will it still work?
13
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Types of Migration ! Image Migration – Capture an image of the server and reconstitute in the cloud • P2V (Physical-to-Virtual) • P2I (Physical-to-Image)
! Application / Workload Migration – Build a new server to run the workload in the cloud • Re-install • Re-platform • Consolidation
– Build from a Template • IAAS • PAAS
– Build via Specification Cloning • Use the physical server specs to build an exact duplicate virtual server 14
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Image Migration
Workload Migration
Types of Clouds ! Unmanaged Clouds – Do-It-Yourself from scratch – Possibly bring your own image – Anything goes but you fix it when it breaks
! Managed Cloud
Unmanaged Cloud
– Assistance in on-boarding – Only certain image infrastructure is allowed – Bring your own image may not be supported
Managed Cloud
• Or image compliance must be adhered to
– Steady state is managed • Patch management • Upgrade management
15
Image Migration
Workload Migration
More Types of Clouds ! Clouds can be Public, Private, or Hybrid – Public to all on cloud provider’s infrastructure – Private to business on dedicated business infrastructure – Private to business on shared cloud provider’s infrastructure
Unmanaged Cloud
Managed Cloud
HYBRID PUBLIC
16
PRIVATE
Image Migration
Workload Migration
A transformation example you may have seen
17
* Example from Apple http://www.apple.com/support/switch101/
A transformation example you may have seen
Know your Target Platform
Migrate what you can
Test and fix Problems
18
* Example from Apple http://www.apple.com/support/switch101/
Why switch personal computer platforms? ! Will it save me money? ! Will I have fewer crashes? ! Will all my software work? ! Will the things work that I need? ! Will the service be better?
19
Why switch to cloud? A Business Perspective
! My machines, my OS
! Standardized OS and no worries about machines
! My management tools, my ! Standard set of tools, management processes standardized processes ! My delivery model ! Standard Cloud Delivery model My mess for less
20
All good and cheaper in the Cloud
If costs will be significantly lower in Cloud offerings compared to current costs, will businesses tolerate running two models in parallel? ! Initially this may be the case ! The future of business computing environments is likely to trend towards sourcing from Cloud providers. ! How will IT service providers manage the desire from businesses to have cost benefits of standard delivery models while retaining some level of customization (some of which is driven by industry/geo specific regulation) ! The transition is challenging and businesses want to understand how to do this with a short time to payback
21
Corporate legacy has positive and negative sides and forms a legacy value stack Legacy can be heritage or burden. Technologies can help customers to identify which legacy is heritage and which is burden, and help to take the appropriate actions to handle legacy.
Corporate legacy cuts across every aspect of the enterprise. Therefore, it can only be addressed effectively if it integrates Systems, People & Culture, and Processes around an agile business and information design
Legacy
Want to leverage & expand on it
Burden
Want to reduce, replace, control & get rid of it
Business process legacy
Software & data legacy
Infrastructure legacy
Legacy needs to be treated differently depending on the heritage/burden aspect. Any legacy related effort has higher value when targeting higher items in the legacy stack. 22
Future of Legacy
Increasing value
Heritage
“Bad” Legacy
Know-how legacy
“Good” Legacy
Business model legacy
Transformation is Not Easy ! Production workloads are running and can’t be disrupted ! System structure never fully known – Personnel that knew the systems changes – Documentation gets out of date ! Systems very heterogeneous – Several generations of technology – Line-of-business governance for distributed systems – Mergers and acquisitions ! Hence transformation cost often exceeds the expected benefits, and important transformations are not undertaken. – Big area for improvement!
23
Outline ! Background on the course ! The approach to transformation
! Course Details
24
What is this Course About? ! Enterprise IT needs constant changes: – New business needs – Reduce cost by new technology ! Examples: – Servers consume about 1.2% of the electrical power produced in the US – Many servers have around 10% utilization – very inefficient – Technology often gets 10-15 years old because upgrade too risky ! Gradual transformation vs complete overhaul ! We present approaches for typical enterprise IT transformation problems
Utilization of 9 servers in 24 hours (30-min slots), scale on the right
25
Cost
From An ROI perspective – successful enterprise IT transformation reducing risk/cost of migration and increasing steady state benefit
Migration Cost
Source System Cost for steady state Improved HW/SW Utilization
Steady state cost
Reduced Power Consumption
benefit Improved Operational Processes
Target System Cost for steady state
Design & Approach 26
Planning & Implementation
Realization
Case Study for this Course
27
Case Study Details ! ACME wants to migrate some of their IT Services to the cloud ! They “think” they have about 2000 – 3000 Servers running a variety of: – Linux 64-bit, 32-bit – AIX – Windows Server 2003, 2008, 2012 – Sun Solaris ! Running middleware such as: – DB2 – Oracle – MySQL – WebSphere Application Server – JBOSS – Tomcat – SAP ! In support of applications like: – Company Web Site – Human Resources – Manufacturing – Shipping and Logistics – Custom Written Departmental Applications 28
Conceptual Migration Workflow
Know your Target
29
Discover your Source
Analysis & Design
Provision & Configure
Migrate & Remediate
Test
Cut-over
Transformation Workflow Overview
30
Detailed Architecture for Enterprise IT Transformation Customer and infrastructure data collection (manual and automated) • Infrastructure data collection • Organizational data (owners ...) • Customer goals and plans
Analytics, Design, Planning • Target options • Technical compatibility • Business constraints • Placement optimization • Wave planning • ROI analysis
Non-customer sources • External product descriptions • Benchmarks
Procurement and physical setup • Hardware and software ordering • Setup of physical infrastructure if new
Migrate, modify • Core migration methods: • Lift-and-shift • Copy • Provision and backup-restore • Apply changes from design • Start
Test and remediate • Unit tests • Comparison tests • User acceptance tests If fails, back to “modify” or initial state
Consolidated data repositories • Customer data • Historic data • General technical and financial data • . Models
Synchronize and cutover • Educate • Resynchronize test system with source • Switch operation over • Registrations etc.
Migration Analytics • Operational Improvement
Project workflow and status visualization Benchmark assessment Macro design Micro design Contracts
Possibly retry For large migrations in waves = groupings migrated at different times. Possibly even micro-design and procurement in these waves Possibly several optimization phases
31
Lecture: Cloud as a Target for IT Transformation Part 1 ! Lecturer: Murthy Devarakonda, IBM Research, [email protected] ! Brief Introduction to Cloud Terms Used ! Cloud Economics: User perspective ! Cloud Economics: Provider perspective ! Reading: – Michael Armbrust, et al, "A View of Cloud Computing," Comm. of ACM, April 2010. – Murthy Devarakonda, et al, “Labor Cost Reduction with Cloud: An End-to-End View,” IBM Research Report (work in progress), will be available on the course web page
Lecture: Discovery ! This has two dimensions: What data to collect, and how. The what is determined by the repository and by the project workflow. The how must suit the data and the customer.
This part is automatable, but not fully automated in all phases at all customers
These parts also need standard templates
Customer data collection Infrastructure data collection • Standard templates for early customer overviews • Standard products brought by Services team • Additional scripts for data not otherwise available • Preexisting 3rd-party tools at customer • Including application and data dependencies • Including utilization discovery Organizational data collection • Roles, responsibilities, meeting data etc. • Credential acquisition data (approval processes, status of asking, finally credentials) • Business data for the infrastructure Customer goals and plans • Timelines, budgets • Focus: ROI, space, power ... • Constraints: facilities, vendors, consolidation layers ...
Adapter infrastructure • Prebuilt adapters for services team’s tools • .. and for standard APIs and typical tools present at customer • Library for building more adapters
Consolidated data repositories
Data collection workflow and visualization ! What data to start / finish collecting when ! What tools to use, depending on customer inputs about goals, installed tools, approval processes ! Status tracking of credential acquisition and data collection 33
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Lecture: Analytics, Design and Planning
! All the data used here are in the consolidated data repositories, or must be estimated based on those data Analytics Benchmark assessments ! Estimating gains and cost with few customer data, more historic data Technical compatibility ! Depending on consolidation layer: SW-OS, OS-VM or OS-HW, cloud image options, software upgrade and change options (possibly with prices) ! Simple size constraints ! Migration costs for different scenarios
Consolidated data repositories
Business constraint analysis ! Workload-level performance (rough – no benchmarks for complex changes) ! Availability, backup, disaster recovery ! Security and compliance ! Other mgmt processes: Maintenance windows, change mgmt, .... Optimization (ultimately for cost) ! Using resource utilization factors, for target systems from customer goals. ! Placement: Multi-dimensional bin packing: multiple utilization dimensions, and the technical and business constraints. Goal variable: typically cost, using the financial data. Might also be power or floor space. ! Rarer: storage and network usage optimization Wave planning ! Groupings of servers or business application for joint migration
Analytics, design and planning workflow ! Lots of reports and visualizations, as this is not fully automated yet ! ROI analysis may also include soft cost (licenses, server management ...). ! What-if-scenarios, storing important intermediate results, signoffs 34
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Procurement and physical setup
Lecture: Automatic Migration
! Where ‘the business occurs’ the actual doing of the migration, after all the planning and procurement has concluded. Migrate, Modify Core Could migration methods
Procurement and physical setup
Consolidated data repositories
• ! Copy: xxx Create an exact copy of the source servers, e.g. image copy. Copy this image between source and target directly or via intermediate. Live vs. non-live migrations • Provision and backup-restore: Provision a new server based on details provided in design starting with OS installation. Introduce software packages in a phased approach. Initiate backup (source) and restore (target) once new system available
Apply changes from design (sometimes in middle of migration) • Use-case driven modifications often needed. • Ability for direct operator intervention • Modification scope varies widely, and is package specific • For complex scenarios, code change may be necessary Start ! xxx • Either manual initiation or scheduled initiation. • Need for real-time situation awareness • New ideas: Provisioning to Cloud environments, image analysis.
Migrate, modify workflow ! Process constrained by schedule and resource availability per consolidated data repositories ! Requirement for exact data alignment between migration tooling and analysis ! Permit migration engineer overrides and in flight modification (constrained) 35
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Test and remediate
Lecture: Transformation and the Network Networking Aspect Customer and infrastructure data collection
Non-customer sources
Analytics, Design, Planning
Procurement and physical setup
Migrate and modify
Consolidated data repositories
Project workflow and status visualization
Test and remediate
Synchronize and cutover
Migration Analytics
Lecture: Network Transformation ! Large scale changes to the Network Infrastructure of a company ! Key Objective – Maximize effectiveness of the enterprise network while keeping costs under control ! Example: Firewall Infrastructure Consolidation – Over time, the firewall infrastructure of an enterprise grows organically, eventually resulting in management, maintenance, and cost issues – Reduce the number of firewalls, while retaining flow control restrictions Extranet 1
Intranet
DMZ
all traffic
ssh https
telnet http
all traffic
Extranet
Intranet
DMZ ssh https
all traffic all traffic httpsmtp
?
all traffic
httpsmtp
all traffic
Extranet 2
telnet http
all traffic
telnet http
Internet
Internet
Test and Remediate ! The point at which the ‘rubber meets the road’. Was the migration successful, if not, what are you going to do? Test and Remediate
Migrate and modify
User Tests ! ! Test xxx cases often developed during initial product development. ! For smaller systems, incremental improvements may have limited or inadequate test sets. ! For larger systems, full regression testing very costly. ! Risk, risk, risk. Remediation always ‘under the gun’ to complete ! Migration directed code coverage analysis tools provide value. Comparison Tests ! Configuration comparison ! Running pre and post migration systems in parallel; very complex but provides high assurance of successful migration
Consolidated data repositories
User Acceptance Testing ! Essential to have clear UAT understanding before migrations start! ! Heavy reliance on strong client / provider relationship ! Area with greatest risk ! Observation: Most time consuming phase of the migration ! Observation: Perform on new dev test system, then promote ! New ideas: Integrated model based testing
38
Test and remediate workflow ! Iterative process to ensure that target behaves in same fashion as the source. ! Essentially involves exercising pre agreed test cases until such time as UAT passes ! Application specific SME often needed to assist in troubleshooting. ! Very difficult to ensure scripts etc. correctly migrated (often not originally part of test plan) Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Synchronize and cutover
Synchronization and Cutover ! Finally, one can start using the new version of the system
Test and remediate
Synchronize and cut-over Synchronize ! Except with lift-and-shift, source system typically keeps running ! xxx most of the prior phases during ! So it’s now in a different state than the new / test system ! Resynchronize – typically using same copying tool instance as in migration Cut-over ! If possible, only switch DNS to new IP addresses ! Else change clients Registration and deregistration ! Enter new systems in asset management systems etc. ! Change status of old systems
Consolidated data repositories
User education ! If systems changed noticeably, teach users (probably started during test phase) Decommissioning ! Typically old systems are kept for a while ! At some point, really decommission hardware (may be for reuse)
Synchronization and cutover workflow ! Coordinate synchronization and cut-over, usually with short down-times
39
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Outline ! Background on the course ! The approach to transformation
! Course Details
40
Course Evaluation
! 50% Assignments (3 total) ! 20% Mid-term Exam ! 20% Final Exam ! 10% Class Attendance
Schedule from Start to Mid-term Exam Lecture Title 1. Transformation To Cloud Computing: Course and Projects Overview 2. Clouds and Other Transformation Goals
Lecturers Nikolai Joukov, Murthy Devarakonda, Hari Ramasamy, John Rofrano Murthy Devarakonda
Date 09/03
Description Introduction to cloud, IT transformation, and need for transformation in enterprise infrastructures. Description of course projects.
09/10
Economics of cloud, types of cloud
3. Enterprise IT Today, IT Discovery
Nikolai Joukov
09/17
Enterprise IT today is a mix of technologies from 80s, 90s, 00’s, and state-of-the-art technologies.
4. Workload Placement: Analytics, Design, and Planning
John Rofrano
09/24
Analysis to identify which workloads are candidates for migration to cloud. Algorithms to optimize placement of such workloads on the target cloud platform.
5. Automatic migration
John Rofrano
10/01
Automation technologies for physical-to-virtual migration, re-installation, and re-platforming of OS instances and applications.
10/08
Students present and discuss the results of their 1st assignment
Nikolai Joukov
10/15
Deep dive into Enterprise Data Storage Systems
Hari Ramasamy
10/22
Most transformation activities will involve changes to the networking infrastructure. We'll cover automation techniques for network reconfiguration during transformation to cloud.
10/29
Grades deadline is 11/03
6. Student Assignment #1 Review 7. Enterprise Storage Systems, DevOps Technologies 8. Transformation and the Enterprise Network
9. Mid-term
42
Schedule from Mid-term Exam to Final Exam
43
Lecture Title Lecturers 10. Student assignment #2 review
Date 11/05
Description Students present and discuss the results of their 2nd assignment
11. Experiences in Transformation to Hybrid Cloud: A Case Study for a Large Financial Enterprise
Hari Ramasamy
11/12
Nearly half of large enterprises will have hybrid cloud deployments by the end of 2017. We will describe a case study in hybrid cloud design spanning an onpremise private cloud, traditional IT infrastructure, and a public cloud.
12. Migration to Cloud Decision Support
Murthy Devarakonda
11/19
Framework tools for deciding what applications can and should be migrated to cloud
13. Student assignment #3 review
12/03
Students present and discuss the results of their 3rd assignment
14. Final Exam
12/11
Cloud Migration Assignments (x3) ! The assignments are designed to build on each other. ! The end goal is to migrate your first application workload into a cloud. – First assignment is to analyze the application structure and understand it's run time behavior. – Second assignment is to estimate the size and number of VMs needed, and migrate the applications to VMs (application starts out as a non-virtualized code) – Third assignment is put it on a cloud, run it, and measure it. ! This course will give you all of the skills necessary to accomplish these tasks
44
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Questions ?
Nikolai Joukov John J. Rofrano Murthy V. Devarakonda Harigovind Ramasamy
Transformation to Cloud Computing
Outline ! Background on the course ! The approach to transformation
! Course Details
2
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
This Course is a Bit of a Horse of a Different Color
! This course it being taught by a team of IBM Researchers who are experts in their field ! There is no text book because “the book” is still being written ;-)
3
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Transformation to Cloud Computing
What is Cloud Computing? …and why would anyone want to transform to it?
4
Examples of Cloud Computing Computing as a Service SaaS, PaaS,IaaS
Utility Computing Shared Resources
5
On Demand Provisioning
Consider the Possibilities that Cloud Computing Opens Up?
“On the Cloud, nobody knows you’re a small business!” * Peter Steiner, The New Yorker, July 5, 1993 6
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Cloud is… All of The Above
7
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Some Cloud Infrastructure Services
8
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Some Cloud Platform Services
9
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Some Cloud Software Services
10
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
What we mean when we say “Cloud Computing” ! A fundamental shift from physical infrastructure to virtual infrastructure – Seen as a major paradigm shift
! A rapid way of provisioning and later releasing computing services on the network ! Some important characteristics: – Rapid/automated provisioning and (later) release of services – Can be Pay-as-you-go – Appearance of infinite resources – Could be managed or unmanaged – and more…
! Types of Clouds: – Based on service: • Software as a Service (SaaS) • Platform as a Service (PaaS) • Infrastructure as a Service (IaaS) – Based on ownership • Public Clouds • Private Clouds • Hybrid Clouds
11
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Transformation
What is Transformation to Cloud?
12
Transformation is Migration ! Since the cloud is virtual you cannot “move” a “physical” computer into a cloud – But you can move the disk image into the cloud (which can be just as good) • …unless you had lots of “junk” on your computer because you might not want that on your new cloud server (note: we call this “your mess for less”) – Or you can build a new server in the cloud that looks just like the old server • When we build a new server we sometimes call this “workload” migration because we are migrating the “work” that the server is doing into another server in the cloud. ! What if the cloud doesn’t support your server’s OS platform? – Now you have to “re-platform” to a new OS which could get tricky – This is a lot like building a new server but what about all of the software? Will it still work?
13
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Types of Migration ! Image Migration – Capture an image of the server and reconstitute in the cloud • P2V (Physical-to-Virtual) • P2I (Physical-to-Image)
! Application / Workload Migration – Build a new server to run the workload in the cloud • Re-install • Re-platform • Consolidation
– Build from a Template • IAAS • PAAS
– Build via Specification Cloning • Use the physical server specs to build an exact duplicate virtual server 14
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Image Migration
Workload Migration
Types of Clouds ! Unmanaged Clouds – Do-It-Yourself from scratch – Possibly bring your own image – Anything goes but you fix it when it breaks
! Managed Cloud
Unmanaged Cloud
– Assistance in on-boarding – Only certain image infrastructure is allowed – Bring your own image may not be supported
Managed Cloud
• Or image compliance must be adhered to
– Steady state is managed • Patch management • Upgrade management
15
Image Migration
Workload Migration
More Types of Clouds ! Clouds can be Public, Private, or Hybrid – Public to all on cloud provider’s infrastructure – Private to business on dedicated business infrastructure – Private to business on shared cloud provider’s infrastructure
Unmanaged Cloud
Managed Cloud
HYBRID PUBLIC
16
PRIVATE
Image Migration
Workload Migration
A transformation example you may have seen
17
* Example from Apple http://www.apple.com/support/switch101/
A transformation example you may have seen
Know your Target Platform
Migrate what you can
Test and fix Problems
18
* Example from Apple http://www.apple.com/support/switch101/
Why switch personal computer platforms? ! Will it save me money? ! Will I have fewer crashes? ! Will all my software work? ! Will the things work that I need? ! Will the service be better?
19
Why switch to cloud? A Business Perspective
! My machines, my OS
! Standardized OS and no worries about machines
! My management tools, my ! Standard set of tools, management processes standardized processes ! My delivery model ! Standard Cloud Delivery model My mess for less
20
All good and cheaper in the Cloud
If costs will be significantly lower in Cloud offerings compared to current costs, will businesses tolerate running two models in parallel? ! Initially this may be the case ! The future of business computing environments is likely to trend towards sourcing from Cloud providers. ! How will IT service providers manage the desire from businesses to have cost benefits of standard delivery models while retaining some level of customization (some of which is driven by industry/geo specific regulation) ! The transition is challenging and businesses want to understand how to do this with a short time to payback
21
Corporate legacy has positive and negative sides and forms a legacy value stack Legacy can be heritage or burden. Technologies can help customers to identify which legacy is heritage and which is burden, and help to take the appropriate actions to handle legacy.
Corporate legacy cuts across every aspect of the enterprise. Therefore, it can only be addressed effectively if it integrates Systems, People & Culture, and Processes around an agile business and information design
Legacy
Want to leverage & expand on it
Burden
Want to reduce, replace, control & get rid of it
Business process legacy
Software & data legacy
Infrastructure legacy
Legacy needs to be treated differently depending on the heritage/burden aspect. Any legacy related effort has higher value when targeting higher items in the legacy stack. 22
Future of Legacy
Increasing value
Heritage
“Bad” Legacy
Know-how legacy
“Good” Legacy
Business model legacy
Transformation is Not Easy ! Production workloads are running and can’t be disrupted ! System structure never fully known – Personnel that knew the systems changes – Documentation gets out of date ! Systems very heterogeneous – Several generations of technology – Line-of-business governance for distributed systems – Mergers and acquisitions ! Hence transformation cost often exceeds the expected benefits, and important transformations are not undertaken. – Big area for improvement!
23
Outline ! Background on the course ! The approach to transformation
! Course Details
24
What is this Course About? ! Enterprise IT needs constant changes: – New business needs – Reduce cost by new technology ! Examples: – Servers consume about 1.2% of the electrical power produced in the US – Many servers have around 10% utilization – very inefficient – Technology often gets 10-15 years old because upgrade too risky ! Gradual transformation vs complete overhaul ! We present approaches for typical enterprise IT transformation problems
Utilization of 9 servers in 24 hours (30-min slots), scale on the right
25
Cost
From An ROI perspective – successful enterprise IT transformation reducing risk/cost of migration and increasing steady state benefit
Migration Cost
Source System Cost for steady state Improved HW/SW Utilization
Steady state cost
Reduced Power Consumption
benefit Improved Operational Processes
Target System Cost for steady state
Design & Approach 26
Planning & Implementation
Realization
Case Study for this Course
27
Case Study Details ! ACME wants to migrate some of their IT Services to the cloud ! They “think” they have about 2000 – 3000 Servers running a variety of: – Linux 64-bit, 32-bit – AIX – Windows Server 2003, 2008, 2012 – Sun Solaris ! Running middleware such as: – DB2 – Oracle – MySQL – WebSphere Application Server – JBOSS – Tomcat – SAP ! In support of applications like: – Company Web Site – Human Resources – Manufacturing – Shipping and Logistics – Custom Written Departmental Applications 28
Conceptual Migration Workflow
Know your Target
29
Discover your Source
Analysis & Design
Provision & Configure
Migrate & Remediate
Test
Cut-over
Transformation Workflow Overview
30
Detailed Architecture for Enterprise IT Transformation Customer and infrastructure data collection (manual and automated) • Infrastructure data collection • Organizational data (owners ...) • Customer goals and plans
Analytics, Design, Planning • Target options • Technical compatibility • Business constraints • Placement optimization • Wave planning • ROI analysis
Non-customer sources • External product descriptions • Benchmarks
Procurement and physical setup • Hardware and software ordering • Setup of physical infrastructure if new
Migrate, modify • Core migration methods: • Lift-and-shift • Copy • Provision and backup-restore • Apply changes from design • Start
Test and remediate • Unit tests • Comparison tests • User acceptance tests If fails, back to “modify” or initial state
Consolidated data repositories • Customer data • Historic data • General technical and financial data • . Models
Synchronize and cutover • Educate • Resynchronize test system with source • Switch operation over • Registrations etc.
Migration Analytics • Operational Improvement
Project workflow and status visualization Benchmark assessment Macro design Micro design Contracts
Possibly retry For large migrations in waves = groupings migrated at different times. Possibly even micro-design and procurement in these waves Possibly several optimization phases
31
Lecture: Cloud as a Target for IT Transformation Part 1 ! Lecturer: Murthy Devarakonda, IBM Research, [email protected] ! Brief Introduction to Cloud Terms Used ! Cloud Economics: User perspective ! Cloud Economics: Provider perspective ! Reading: – Michael Armbrust, et al, "A View of Cloud Computing," Comm. of ACM, April 2010. – Murthy Devarakonda, et al, “Labor Cost Reduction with Cloud: An End-to-End View,” IBM Research Report (work in progress), will be available on the course web page
Lecture: Discovery ! This has two dimensions: What data to collect, and how. The what is determined by the repository and by the project workflow. The how must suit the data and the customer.
This part is automatable, but not fully automated in all phases at all customers
These parts also need standard templates
Customer data collection Infrastructure data collection • Standard templates for early customer overviews • Standard products brought by Services team • Additional scripts for data not otherwise available • Preexisting 3rd-party tools at customer • Including application and data dependencies • Including utilization discovery Organizational data collection • Roles, responsibilities, meeting data etc. • Credential acquisition data (approval processes, status of asking, finally credentials) • Business data for the infrastructure Customer goals and plans • Timelines, budgets • Focus: ROI, space, power ... • Constraints: facilities, vendors, consolidation layers ...
Adapter infrastructure • Prebuilt adapters for services team’s tools • .. and for standard APIs and typical tools present at customer • Library for building more adapters
Consolidated data repositories
Data collection workflow and visualization ! What data to start / finish collecting when ! What tools to use, depending on customer inputs about goals, installed tools, approval processes ! Status tracking of credential acquisition and data collection 33
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Lecture: Analytics, Design and Planning
! All the data used here are in the consolidated data repositories, or must be estimated based on those data Analytics Benchmark assessments ! Estimating gains and cost with few customer data, more historic data Technical compatibility ! Depending on consolidation layer: SW-OS, OS-VM or OS-HW, cloud image options, software upgrade and change options (possibly with prices) ! Simple size constraints ! Migration costs for different scenarios
Consolidated data repositories
Business constraint analysis ! Workload-level performance (rough – no benchmarks for complex changes) ! Availability, backup, disaster recovery ! Security and compliance ! Other mgmt processes: Maintenance windows, change mgmt, .... Optimization (ultimately for cost) ! Using resource utilization factors, for target systems from customer goals. ! Placement: Multi-dimensional bin packing: multiple utilization dimensions, and the technical and business constraints. Goal variable: typically cost, using the financial data. Might also be power or floor space. ! Rarer: storage and network usage optimization Wave planning ! Groupings of servers or business application for joint migration
Analytics, design and planning workflow ! Lots of reports and visualizations, as this is not fully automated yet ! ROI analysis may also include soft cost (licenses, server management ...). ! What-if-scenarios, storing important intermediate results, signoffs 34
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Procurement and physical setup
Lecture: Automatic Migration
! Where ‘the business occurs’ the actual doing of the migration, after all the planning and procurement has concluded. Migrate, Modify Core Could migration methods
Procurement and physical setup
Consolidated data repositories
• ! Copy: xxx Create an exact copy of the source servers, e.g. image copy. Copy this image between source and target directly or via intermediate. Live vs. non-live migrations • Provision and backup-restore: Provision a new server based on details provided in design starting with OS installation. Introduce software packages in a phased approach. Initiate backup (source) and restore (target) once new system available
Apply changes from design (sometimes in middle of migration) • Use-case driven modifications often needed. • Ability for direct operator intervention • Modification scope varies widely, and is package specific • For complex scenarios, code change may be necessary Start ! xxx • Either manual initiation or scheduled initiation. • Need for real-time situation awareness • New ideas: Provisioning to Cloud environments, image analysis.
Migrate, modify workflow ! Process constrained by schedule and resource availability per consolidated data repositories ! Requirement for exact data alignment between migration tooling and analysis ! Permit migration engineer overrides and in flight modification (constrained) 35
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Test and remediate
Lecture: Transformation and the Network Networking Aspect Customer and infrastructure data collection
Non-customer sources
Analytics, Design, Planning
Procurement and physical setup
Migrate and modify
Consolidated data repositories
Project workflow and status visualization
Test and remediate
Synchronize and cutover
Migration Analytics
Lecture: Network Transformation ! Large scale changes to the Network Infrastructure of a company ! Key Objective – Maximize effectiveness of the enterprise network while keeping costs under control ! Example: Firewall Infrastructure Consolidation – Over time, the firewall infrastructure of an enterprise grows organically, eventually resulting in management, maintenance, and cost issues – Reduce the number of firewalls, while retaining flow control restrictions Extranet 1
Intranet
DMZ
all traffic
ssh https
telnet http
all traffic
Extranet
Intranet
DMZ ssh https
all traffic all traffic httpsmtp
?
all traffic
httpsmtp
all traffic
Extranet 2
telnet http
all traffic
telnet http
Internet
Internet
Test and Remediate ! The point at which the ‘rubber meets the road’. Was the migration successful, if not, what are you going to do? Test and Remediate
Migrate and modify
User Tests ! ! Test xxx cases often developed during initial product development. ! For smaller systems, incremental improvements may have limited or inadequate test sets. ! For larger systems, full regression testing very costly. ! Risk, risk, risk. Remediation always ‘under the gun’ to complete ! Migration directed code coverage analysis tools provide value. Comparison Tests ! Configuration comparison ! Running pre and post migration systems in parallel; very complex but provides high assurance of successful migration
Consolidated data repositories
User Acceptance Testing ! Essential to have clear UAT understanding before migrations start! ! Heavy reliance on strong client / provider relationship ! Area with greatest risk ! Observation: Most time consuming phase of the migration ! Observation: Perform on new dev test system, then promote ! New ideas: Integrated model based testing
38
Test and remediate workflow ! Iterative process to ensure that target behaves in same fashion as the source. ! Essentially involves exercising pre agreed test cases until such time as UAT passes ! Application specific SME often needed to assist in troubleshooting. ! Very difficult to ensure scripts etc. correctly migrated (often not originally part of test plan) Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Synchronize and cutover
Synchronization and Cutover ! Finally, one can start using the new version of the system
Test and remediate
Synchronize and cut-over Synchronize ! Except with lift-and-shift, source system typically keeps running ! xxx most of the prior phases during ! So it’s now in a different state than the new / test system ! Resynchronize – typically using same copying tool instance as in migration Cut-over ! If possible, only switch DNS to new IP addresses ! Else change clients Registration and deregistration ! Enter new systems in asset management systems etc. ! Change status of old systems
Consolidated data repositories
User education ! If systems changed noticeably, teach users (probably started during test phase) Decommissioning ! Typically old systems are kept for a while ! At some point, really decommission hardware (may be for reuse)
Synchronization and cutover workflow ! Coordinate synchronization and cut-over, usually with short down-times
39
Transformation to Cloud Computing, Columbia University (COMS E6998-12)
Outline ! Background on the course ! The approach to transformation
! Course Details
40
Course Evaluation
! 50% Assignments (3 total) ! 20% Mid-term Exam ! 20% Final Exam ! 10% Class Attendance
Schedule from Start to Mid-term Exam Lecture Title 1. Transformation To Cloud Computing: Course and Projects Overview 2. Clouds and Other Transformation Goals
Lecturers Nikolai Joukov, Murthy Devarakonda, Hari Ramasamy, John Rofrano Murthy Devarakonda
Date 09/03
Description Introduction to cloud, IT transformation, and need for transformation in enterprise infrastructures. Description of course projects.
09/10
Economics of cloud, types of cloud
3. Enterprise IT Today, IT Discovery
Nikolai Joukov
09/17
Enterprise IT today is a mix of technologies from 80s, 90s, 00’s, and state-of-the-art technologies.
4. Workload Placement: Analytics, Design, and Planning
John Rofrano
09/24
Analysis to identify which workloads are candidates for migration to cloud. Algorithms to optimize placement of such workloads on the target cloud platform.
5. Automatic migration
John Rofrano
10/01
Automation technologies for physical-to-virtual migration, re-installation, and re-platforming of OS instances and applications.
10/08
Students present and discuss the results of their 1st assignment
Nikolai Joukov
10/15
Deep dive into Enterprise Data Storage Systems
Hari Ramasamy
10/22
Most transformation activities will involve changes to the networking infrastructure. We'll cover automation techniques for network reconfiguration during transformation to cloud.
10/29
Grades deadline is 11/03
6. Student Assignment #1 Review 7. Enterprise Storage Systems, DevOps Technologies 8. Transformation and the Enterprise Network
9. Mid-term
42
Schedule from Mid-term Exam to Final Exam
43
Lecture Title Lecturers 10. Student assignment #2 review
Date 11/05
Description Students present and discuss the results of their 2nd assignment
11. Experiences in Transformation to Hybrid Cloud: A Case Study for a Large Financial Enterprise
Hari Ramasamy
11/12
Nearly half of large enterprises will have hybrid cloud deployments by the end of 2017. We will describe a case study in hybrid cloud design spanning an onpremise private cloud, traditional IT infrastructure, and a public cloud.
12. Migration to Cloud Decision Support
Murthy Devarakonda
11/19
Framework tools for deciding what applications can and should be migrated to cloud
13. Student assignment #3 review
12/03
Students present and discuss the results of their 3rd assignment
14. Final Exam
12/11
Cloud Migration Assignments (x3) ! The assignments are designed to build on each other. ! The end goal is to migrate your first application workload into a cloud. – First assignment is to analyze the application structure and understand it's run time behavior. – Second assignment is to estimate the size and number of VMs needed, and migrate the applications to VMs (application starts out as a non-virtualized code) – Third assignment is put it on a cloud, run it, and measure it. ! This course will give you all of the skills necessary to accomplish these tasks
44
CSCI-GA.3033-010 Transformation to Cloud Computing, New York University, Fall 2015
Questions ?
