Enabling workload scalability, strong consistency and elasticity with transactional database replication

Abstract

The proliferation of internet-based services and applications has led to both a rapid growth and high variability of transactional workloads, which can negatively affect the performance of the underlying database system. However, most existing database systems do not offer any features to automatically support workload scalability (i.e., the ability to handle increasing workload demands) or elasticity (i.e., the ability to handle variations in those workloads). Database replication has been successfully employed in the past to scale performance and improve availability of relational databases but current approaches suffer from various issues including limited scalability, performance versus consistency tradeoffs, and requirements for database or application modifications. This thesis presents a new replication-based middleware system, called Hihooi, which is able to achieve workload scalability, strong consistency, and elasticity for existing transactional databases at a low cost. As a middleware system, Hihooi sits between the database engines and the clients, offering a single database view and masking the complexity of the underlying replication, which is used to increase throughput (via increasing the processing capacity of the system) and decrease latency (via spreading the load across the nodes). The novelty of Hihooi lies in its replication and transaction routing algorithms. In particular, Hihooi replicates all write statements asynchronously and applies them in parallel at the replica nodes, while ensuring that all replicas remain consistent with the primary copy. At the same time, a fine-grained transaction routing algorithm ensures that all read transactions are load balanced to the replicas, while maintaining strong consistency semantics. Finally, elasticity is achieved by supporting an easy and quick way to add and remove replicas from the cluster. A thorough experimental evaluation with several well-established benchmarks highlights how Hihooi is able to achieve almost linear workload scalability for existing transactional databases.