Reconciling end-to-end confidentiality and data reduction in cloud storage

Abstract

An increasingly common practice for users of storage systems is to perform end-to-end encryption to ensure the confidentiality of data stored on external storage systems or in the cloud. This practice, however, inhibits the benefits of deduplication and compression performed downstream from where data is encrypted; as a consequence, the required storage capacity increases, and so does the overall cost of the service. In this paper, we address this problem by proposing a framework that reconciles end-to-end encryption with downstream compression and deduplication. The proposed framework guarantees the confidentiality of data in transit and at rest, even after clients cancel a cloud storage subscription, without affecting the ability of storage systems to perform data reduction functions. The framework requires only minor modifications in storage applications that encrypt data, and no changes in a client's business applications. Additionally, we propose several secure data reduction algorithms to compress and deduplicate data without compromising its confidentiality, even if the data is originally encrypted with different keys. We present a comprehensive security analysis that shows that the framework is secure against malicious cloud administrators, other tenants and law enforcement agencies. Our prototype shows that, for a reasonable extra overhead in the time required to store data, the framework enables a considerable amount of storage capacity savings.