基于国产CPU环境的国产数据库历史数据迁移技术

Historical data migration technology of domestic database based on domestic CPU environment

摘要:

针对目前数据迁移方法存在数据迁移耗时长、存储空间最大占用率较高、迁移学习错误率高和被访问数据在线概率低的问题，开展基于国产CPU环境的国产数据库历史数据迁移技术的研究。首先在国产CPU环境中集群部署系统软硬件，提高历史数据在国产数据库之间的迁移速率。其次建立孤立森林模型，将历史数据输入孤立森林模型中展开趋势预测，剔除国产数据库中存在的异常数据，减少待迁移的数据量。最后，构建数据迁移模型，并采用交替优化策略求取模型最优解，完成国产数据库历史数据的迁移。实验结果表明，该方法的数据迁移时间为18 min，储存空间最大占用率在10%~25%之间，ALC指标值为0.78~0.95，被访问数据在线概率能够始终保持在97%以上，证明该方法数据迁移耗时较短，存储空间最大占用率较低，迁移学习的错误率低，访问效率高，具有较好的应用效果。

Abstract:

In view of the problems of long data migration, high maximum occupancy rate of storage space, high error rate of transfer learning and low online probability of visited data, the historical data migration technology of domestic database based on domestic Central Processing Unit(CPU) environment is studied. Firstly, the system software and hardware are clustered and deployed in the domestic CPU environment to improve the migration rate of historical data between domestic databases. Secondly, an isolation forest model is established, and the historical data is input into the isolation forest model for trend prediction, thereby eliminating the abnormal data in the domestic database, and reducing the amount of data to be migrated. Finally, a data migration model is constructed, and an alternating optimization strategy is adopted to find the optimal solution of the model, thus completing the migration of historical data in domestic databases. The experimental results show that the data migration time of this method is 18 minutes, and the maximum occupancy rate of storage space is between 10% and 25%, the ALC(Area under the Learning Curve) index value is 0.78~0.95, and the online probability of the accessed data can always be maintained at more than 97%, proving that this method has a short data migration time, a low maximum occupancy rate of storage space, a low error rate of migration learning, and high access efficiency, demonstrating good application effects.