What is LiveMigrator?
Migrate from Hadoop to cloud without disruption or downtime.
LiveMigrator is a safe and reliable, single pass cloud migration tool, to initiate a transfer of data from one cluster to another.
LiveMigrator automates your Hadoop to cloud migration at scale with application downtime and no risk of data loss, even when your data sets are under active change.
- Single pass data migration with guaranteed consistency
- Scalable for very large migrations
- Minimal disruption to operations
- Non-blocking to client nodes.
- Designed to maximize target ingest rate
Benefits of LiveMigrator
LiveMigrator enables you to transition to a LiveData environment which makes your data globally accessible and consistent where it is needed, avoiding the costs of a manual migration and the pattern of data silos that emerge when data cannot be kept consistent.
- 100% Data Availability during migration, lowest RPO/RTO, improves Data SLA's
- High Scalability and better performance with Big Data Sizes (100's TB - Multi PB)
- Immediate availability of migrated data
- Unaffected by outages
As changes occur anywhere in the donor system, LiveMigrator creates and ensures beneficiary data consistency.
No manual intervention required to handle out-of-sync conditions with guaranteed data consistency across clusters.
- No code maintenance, no manual migration
- Automated replication across all major commercial Hadoop distributions, storage and analytic services
- No repeated scans of the source storage lowers project cost, faster project completion, faster adoption of ML and AI
- Multi Cloud (Bi-Directional) Data Replication compatible with all Cloud Vendors
- No "Big Bang" cutover for applications
LiveMigrator Automates Cloud Migration
Zero Downtime. Zero Data Loss. 100% Consistency.
Hadoop & Object Storage: Works across a variety of big data source and target environments, including all major Hadoop and object storage technologies.
Petabyte Scale: Migrates big data sets at any scale to cloud storage without needing to halt changes made to the data sets during migration.
Selective Migration: Allows selection of which data sets should be migrated. Administrators define replication policies that control what data is replicated between clusters and selectively exclude data from migration to specific clusters in the new environment, or move it off to be archived.
One Pass: Migrates existing data sets with a single pass through the source storage system, eliminating the overhead of repeated scans.
One Click: With Automated Replication, just select data sets and push start. LiveMigrator notifies you when migration is complete with no scripts, no code maintenance, no transfer devices, no scheduling and no reviewing.
One Consensus: Incorporates changes made ot the data sets under migration with a strongly consistent distributed consensus engine, so that applications can continue to modify the donor system's data without causing divergence between donor and beneficiary.
Automatic Outage Recovery: The consensus engine used by LiveMigrator eliminates the need for manual response to system failures, including network outages and other disruptions.
Bandwidth Management: Optimizes bandwidth use by eliminating the need for repeated transfer of data, and enforces limits on bandwidth use.
Rapid Availability: Minimizes the time required to bring workloads to the cloud by making each data location available for use as soon as bandwidth allows.
Guaranteed Consistency: As changes occur anywhere in the donor system, LiveMigrator creates an ensures beneficiary data consistency.
The LiveMigrator Approach
Unique, patented consensus technology as the foundation for data replication
|Static Data Copy||Change Data Capture||Dual Ingest||LiveMigrator|
|Storage Systems Supported||Hadoop-compatible file systems||Structured databases for source
Structured, Hadoop-compatible or object storage for destination.
|Custom built, so arbitrary||
Hadoop-compatible file systems.
Object storage services
Other metadata repositories (Hive, Ranger, Sentry)
|How does data transfer occur?||Point-in-time||Continuously||Duplicate writes||Continuously|
|Who participates?||Single source, single destination||Single source, single destination||Client application and two destinations||Any number of peer replicas|
|When does data transfer begin?||Only after a file or object has been created in full||While content is created||While content is created||While content is created|
|Can applications modify data at source while transfer is ongoing?||Not if required outcome is for destination data to be a replica of source||YES||NO||YES|
|Can applications modify data at destination while transfer is ongoing?||NO||NO||NO||YES - there is no distinction between "source" and "destination" - all are peers with the same data available locally|
|Key Differentiation||Schedule a copy of data from source to target||Copy data from source to target as change occurs||Difficult to maintain, high risk of failures impacting operation||Data consistency, not just data replication, so you can actively use data (create, read, update, delete) in any location|
LiveDeltaLake ensures Databricks users get consistent business insights when using multiple cloud regions or providers. With LiveDeltaLake, any changes made to content in Managed Delta Lake on Databricks are replicated among these environments.Learn more about LiveDeltaLake
LiveMigrator: Migrate from on-premises HDFS to the Cloud with zero downtime, zero data loss and zero inconsistency.