preface

“Databases are one of the trickiest products out there,” Kelly Stirman, former MongoDB (now Google Cloud) executive, said in an interview. Replacement costs are very high. The database transformation platform is seen as one of the riskiest proposals in all of IT, and the transition will be slow for existing workloads.”

The migration reasons

First of all, we need to understand that the database is working well, why migrate? Here are four.

  1. Upgrade to the latest version of database software to improve security and compliance.
  2. Consolidate data from multiple databases into a single database to achieve a consolidated view of the data.
  3. Move existing data to a new database to reduce costs, improve performance, and achieve scalability.
  4. Migrate from on-premise databases to cloud-based databases for better scalability and lower costs.

The first type is what we often do, such as upgrading from Oracle 11g to Oracle 12C or MySQL 5.7 to 8.0. The second type is also what we often do. We use Debezium, Oracle GoldenGate, Kafka, Otter, etc., to migrate the data of various cities and aggregate them into the aggregation library. Implement unified data view. We have also done the third type, that is, the existing databases such as SQL Server, Oracle, DB2, etc., are migrated to MySQL cluster +MyCat database middleware to achieve sub-database sub-table, elastic expansion. The fourth category is one that we have less exposure to, but it’s also one that’s thriving this year

The rise of cloud migration

Today we are going to discuss the fourth category, which is booming this year. Why is it booming? Let’s take a look at what foreign manufacturers have done this year.

First, Microsoft launched Azure Database Migration Service on February 20, 2020.

Their approach to migration is simple: use mysqldump to export the structured data, then import it to their database on the cloud, then use their page’s synchronous CDC tool for full data loading and incremental data synchronization.

On March 5, 2020, MongoDB, along with partners Informatica and Hitachi Vantara Pentaho, also launched a modern toolkit to help companies migrate from Oracle to MongoDB. Informatica is one of the more established ETL tools vendors.

Google also launched its own data migration service this year. Google is using a no-service approach. Can help users migrate MySQL, PostgreSQL, SQL Server to Cloud SQL.

Impressions of cloud migration

I also used some cloud migration platforms this year, even though there are so many platforms launching cloud migration. The impression mainly consists of the following points:

  1. With just a few clicks, you can get a quick experience of the migration, and the experience is great, especially for assessing the difficulty of the migration, which is a great way to get a preview of the overall migration.
  2. The migration process also comes with monitoring. It’s also a great experience, as in the past we might have hung several scripts and toggled back and forth to see the log. Now you can show the speed of the migration directly on the platform, how many rows have been migrated, and how long it will take, all of which are very friendly.
  3. Data check, check this platform also has, out of the data inconsistent or data garbled and other problems WE can also find the platform back pan.

Disadvantages: Error detection needs further improvement, the graphical interface masks several steps behind the operation, sometimes after an error, still need to log a lot of troubleshooting. For those cloud platforms, it may be necessary to find Support for investigation, after all, it may not be able to get operation logs.

Trends in cloud migration

To conclude, the Gartner report makes a “strategic assumption” that “75% of all databases will be deployed or migrated to the cloud by 2022. If this is a trend, there will be more cloud migration platform tools in the future. It could be a repackaged tool from an established software + cloud vendor, or it could be directly integrated into the migration platform with an off-the-shelf open source tool.

As traditional DBAs, migration will definitely become easier in the future. Tools help us to achieve a large number of manual evaluation, inspection operations. The valuable thing to do is to understand the underlying implementation of migration, which will help us better analyze the problem.