Data-First Cloud Migration and the Migration Gap
By Paul Scott-Murphy, Jul 08, 2021
The history of information technology has been one of exponential growth, with data at the very center of all innovation. Fueled by increasingly rapid collection, creation, and processing of data, organizations rely on access to greater volumes of information to compete and provide value. Information is central to organization operations. Not only do we have more data, but it's of greater value and importance for business operations.
Given the value of this data, one of the core questions IT infrastructure stakeholders need to ask is: where should my data reside? How can I best access it? And, most crucially, how can I best use it to enhance my business? It’s not just a question of how cheaply and in what form data can be stored, but rather, what can be done with datasets.
It’s not just a question of how cheaply and in what form data can be stored, but rather, what can be done with datasets.
Traditional on-premises infrastructure lacks agility and requires up-front planning and investment to meet anticipated demands. It is difficult to meet customer expectations of immediate and personalized interaction using fixed infrastructure capacity without over provisioning. Additionally, modern approaches to artificial intelligence and machine learning technologies are not being led by on-premises implementations. Data science has moved from model-centric to data-centric approaches, where results that formerly could not be achieved now seem straightforward with the right (massive) volume of data. Modern data analytics has advanced by using what were previously unreasonably large sets of data, and it is only the capacity of shared cloud storage and other infrastructure that have made this possible.
As a result, the scope of IT in the cloud is larger and more complex -- but significantly more capable.
Data-first cloud migration
Modernizing enterprise IT ecosystems by introducing the cloud can be complex. Most organizations choose a staged approach to cloud migration. And given the role of data at the center of modern business, many benefit from a data-first approach. Today, companies are looking to move their data to the cloud for three key reasons:
- To reduce the cost of operations because of the efficiencies that emerge at scale in the cloud.
- To de-risk operations by taking an agile approach to building new services and functionality.
- To modernize systems, taking better advantage of cloud-based services that companies would otherwise either have to build, acquire, or integrate themselves on-premises.
A data-first approach is an obvious -- and safe -- approach for cloud migration. But there’s a catch because data sets are never actually static, and business does not wait. I’ll explain with a story:
In 1930, the Indiana Bell Telephone Company decided to build a new corporate headquarters in a location that overlapped their existing building in downtown Indianapolis. They faced a dilemma: if they knocked down the existing building and built a new one instead, they couldn’t maintain business operations during the construction, which was unacceptable. They needed to both build a new headquarters and eliminate the old one, all while keeping operations running.
To achieve this, over the course of a month or so, Bell shifted their existing building slowly, moving it 16 meters in one direction, rotating it by 90 degrees, and shifting it 30 meters in another direction to make space for the new building. And throughout this structural relocation, they maintained the operations of that building: all office workers came to work every morning and all water, telecommunications, and power services were maintained.
The analogy is that Bell found a way to maintain business operations throughout a significant change. Business ran like normal, but they also prepared for the future. The parallel with cloud migration is clear: the key to cloud data migration at scale — migrating large volumes of data that are critical to business operations without impeding those operations — is to take the same approach that Bell took nearly 100 years ago.
The key to data-first migration: Overcoming the migration gap
At WANdisco, we call it the data migration gap. It’s the challenge presented by the need to migrate large volumes of data that are business critical, without exceeding the budgeted allocation of effort or time, all while maintaining business operation.
To overcome the data migration gap and achieve a data-first migration, organizations need the ability to make data available in a new location while maintaining operations against those data during migration. This includes being able to work with datasets; ingest new data; and modify, access, query, and maintain all regular application and platform activities.
How does this happen? Another story will illustrate the concept:
Last year, the Huangpu district government decided to move the historic Lagena Primary School building in Shanghai to make space for a new commercial and office complex. Since the movement of the Indiana Bell building, which was done with hand-cranked rollers, technology has progressed, of course. The company that was contracted to move this building placed nearly 200 robots underneath it. Each robot worked in concert to “walk” the building 60 meters to its new site. Not surprisingly, the task took a fraction of the time of the Indiana Bell move.
Migration automation technology (particularly that from WANdisco) helps to overcome the data migration gap and run a successful data-first migration.
The LiveData Migrator solution is ideally suited to the dynamic data environments of data-intensive enterprises, enabling data-first migrations without application changes or business disruption. It facilitates migrations at any scale with only a single pass of the source data, and continues to replicate ongoing changes from source to target thereafter. This empowers migration stakeholders to bridge the data migration gap cost-effectively and efficiently, making overall cloud migration faster and more attainable for any enterprise.
Chief Technology Officer, WANdisco
Paul has overall responsibility for WANdisco’s product strategy, including the delivery of product to market and its success. This includes directing the product management team, product strategy, requirements definitions, feature management and prioritisation, roadmaps, coordination of product releases with customer and partner requirements and testing. Previously Regional Chief Technology Officer for TIBCO Software in Asia Pacific and Japan. Paul has a Bachelor of Science with first class honours and a Bachelor of Engineering with first class honours from the University of Western Australia.