SMARTMIG: Risk-modulated proactive data migration for maximizing storage system utility

Abstract

The goal of storage management is to maximize the overall utility of the storage system by continuously tuning the amount of resources allocated to multiple independent competing applications. Due to variations in access characteristics, service level objectives, and exception events such as failures and load surges, there is a need to invoke corrective actions such as data migration to modify the resources allocated to a given application. There is a significant body of research on automated data migration - their focus has primarily been on optimizing for the current system load without considering load forecasts; the scheduling of the migration operation today is currently heuristic and coarse-grained; finally, there is a need to factor in prediction inaccuracies and migration data-size (referred to as risks) in the decision-making. This paper proposes SMARTMIG: a framework for optimizing the storage utility by proactively scheduling data migration using time-series forecasts. SMARTMIG generates several plans for what data to migrate, where to migrate, how to migrate (i.e., the migration speed), and when to migrate. These plans are generated using constraint optimization, and their selection is modulated by risk analysis of the prediction accuracy and the migration overheads. For the experimental evaluation of SMARTMIG, we developed a detailed storage system simulator, and analyzed the quality of migration decisions made in different scenarios. Our results show that for a significant percentage of scenarios, SMARTMIG in a automated fashion minimizes the utility loss by 80% compared to no action invocation.