Tuesday, March 12, 2019

Data Migration Service - AWS DMS

Data migration between various platforms. 

Have you ever thought about migrating your production database from one platform to another and dropped this idea later, because it was too risky, you were not ready to bare a downtime?

If yes, then please pay attention because this is what we are going to perform in this article. 

A few days back we’re trying to migrate our production MySQL RDS from AWS to GCP SQL, and we had to migrate data without downtime, accurate and real-time and that too without the help
of any Database Administrator.

After doing a bit research and evaluating few services we finally started working on AWS DMS 
(Data Migration Service) and figured out this is a great service to migrate a different kind of data. 

Let’s discuss some important features of AWS DMS: 

  • The source database remains fully operational during the migration.
  • Migrates the database securely, quickly and accurately.
  • No downtime required works as schema converter as well.
  • Supports various type or database like MySQL, MongoDB, PSQL etc.
  • Migrates real-time data also synchronize ongoing changes. 
  • Homogeneous migrations (migrations between the same engine types).
  • Heterogeneous migrations (migrations between different engine types).
  • Compatible with a long range of database platforms like RDS, Google SQL, on-premises etc. 
  • Inexpensive (Pricing is based on the compute resources used during the migration process).
This is a high-level overview of Data Migration Setup.

Let's perform step by step migration: 


Note: We've performed migration from AWS RDS to GCP SQL, you can choose database source and destination as per your requirement. 

Create replication instance:

A replication instance initiates the connection between the source and target databases, transfers the data, cache any changes that occur on the source database during the initial data load.

Use the fields to below to configure the parameters of your new replication instance including network and security information, encryption details, select instance class as per requirement.





After completing all mandatory fields click the next tab, and you will be redirected to Replication Instance tab. Grab a coffee quickly while the instance is getting ready.




Hope you are ready with your coffee because the instance is ready now.




Now let's create two endpoints “Source” and “Target”:


Click on “Run test” tab after completing all fields, make sure your Replication instance IP is whitelisted under security group. 

Create Target Endpoint:





Click on “Run test” tab again after completing all fields, make sure your replication instance IP is whitelisted under target DB authorization. 

Our replication setup is ready now, we've to create "Replication Task" to perform the migration. 

Create a “Replication Task” to start replication:

  • Task Name: any name
  • Replication Instance: The instance we’ve created above
  • Source Endpoint: The source database
  • Target Endpoint: The target database
  • Migration Type: Here I choose “Migrate existing data and replicate ongoing” because we needed ongoing changes.






Once all the fields are completed click on the “Create task” and you will be redirected to “Tasks” tab.

Verify the task status:


The task status is "Load Complete" and Validation Status is "Validated" that means migration has been performed successfully. 







1 comment:

  1. Data migrating without time lag , with more accuracy at real-time and that too without the help of any Database Administrator was really good.

    Database migration services
    VMware Cloud Migration
    Azure Cloud Migration
    AWS Cloud Migration
    Cloud Migration Services

    ReplyDelete

Lets Get Started With Packer

In this blogpost, we will see how to get started with packer. We will cover installation, writing a template for creating AWS AMI. To get t...