How to Build and Manage a Winning Team for Effective Data Migration
By WANdisco
Oct 24, 2019
Cloud computing has become the preferred way to access and control computing resources and data. According to McAfee, 97% of organizations already use public or private cloud services — and not just for computing resources.
Big data is starting to jump the on-prem/cloud gap too. IDG found that 41% of enterprises are already migrating storage, archiving, backups and other file servers, and 21% plan to do so in the coming year. The cloud has completely transformed entire industries, computing paradigms and enterprises, and has become the ideal for storing and accessing big data.
The crucial question facing enterprises today is not the if of big data cloud migration, but rather the how. And chief among the numerous concerns surrounding migration is the overall management of the process.
Large, complex enterprise migrations consume significant human resources time and mindshare. There are a plethora of roles that need to be filled by qualified professionals, including:
- Cloud Architect
The migration project mastermind who defines the overall vision of a cloud solution, fleshes out the migration plan while taking into consideration the various trade-offs of each approach, and sets up the target environment with suitable governance controls. - Cloud Infrastructure Engineers
Execute the actual migration work, based on the plan the cloud architect created - developing resources, uploading data, writing scripts, and more. - Software Developers
Experts in the multiple applications that need to be migrated who design and deliver revised apps that can take advantage of the new cloud environment. - Technical Specialists
Solution engineers who specialize in areas impacted by migration - databases, networking, storage, security, identity management, and more. - Project Manager
Manages the project schedule to ensure that milestones are achieved on time, facilitates cross-team communication, tracks and manages risks, and analyzes dependencies.
Migration Management Best Practices
With your team in place, it is time to set up a strong management process. As a rule, migration is best managed centrally by the cloud infrastructure team, and not - as commonly occurs - by application teams.
App teams are often expected to manage migration projects because they already manage the mission-critical apps that run on top of on-prem Hadoop infrastructure. As a result, many assume that it makes sense for them to continue to do so during and after migration. Moreover, many different teams manage the different layers of an enterprise big data stack - the infrastructure team “only” manages the Hadoop system and everything below it.
Yet, from experience, it is very hard, if not impossible, for each application team to work out its own migration strategy. The reason? Data. Apps need active data to continue to function – and only the infrastructure teams can ensure this.
Thus, it’s preferable to have the infrastructure team manage the project – first deploying a file system-level migrator, then providing application teams with access to their data in the cloud. In this manner, and with the right tools, while the applications teams port their applications to the cloud, active business data can be kept up to date.
The Bottom Line
There are multiple paths by which organizations can move to the cloud. Adopting advanced technology like WANdisco LiveMigrator can enable a sophisticated non-blocking cloud migration approach. Yet beyond technology, to ensure a smooth transition to the cloud, organizations with complex and distributed data environments also need to minutely define the roles of migration responsibility. Managing teams and assigning roles for effective data migration begins with delineating the big data migration division of labor.
