Sharding-JDBC: A Distributed Micro-Service Database Access Framework
Design and Implementation of a Scalable Database Architecture
In the realm of Internet databases, a critical challenge lies in ensuring high availability, storage stability, and concurrent access to massive data. To address this issue, we introduce Sharding-JDBC, a distributed micro-service database access framework designed to meet the demands of modern web applications.
Comparing Database Schema
Database programs can be broadly categorized into three schools: Relational Database Management Systems (RDBMS), NoSQL, and NewSQL. Each has its strengths and weaknesses:
- RDBMS: Supports native SQL, but has limitations in terms of flexibility and scalability. ACID compliance is a significant advantage, but distributed transactions using the XA protocol can be inefficient.
- NoSQL: Offers flexible transactions using the BASE approach, but lacks strong consistency. This can be a concern for applications requiring high data integrity.
- NewSQL: Combines the benefits of RDBMS and NoSQL, with support for SQL and flexible transactions. However, its SQL support is not perfect, and it requires a complete rewrite of the storage engine.
Advantages and Disadvantages of RDBMS Solutions
RDBMS solutions have several advantages, including:
- SQL-oriented: Supports native SQL, making it familiar to developers.
- Stable storage engine: Mature single-node transaction engine ensures high performance.
- Stand-alone performance: Meets performance requirements for small-scale applications.
However, RDBMS solutions also have some disadvantages:
- Limited concurrency: Single-node concurrent access frequency data carrying capacity is limited.
- Distributed transaction performance: Distributed transactions can be unacceptable due to the XA protocol’s inefficiency.
- Distributed expansion difficulty: Expanding RDBMS solutions to distributed environments can be challenging.
Dangdang Database Middle Layer
The Dangdang database middle layer focuses on addressing the issue of data fragmentation. This is achieved by:
- Fragmentation: Sub-tables are divided into separate libraries, allowing for separate read and write operations.
- Distributed primary key: Primary keys are distributed across multiple nodes, ensuring efficient data access.
However, the current implementation of the Dangdang database middle layer has some limitations:
- Limited support for XA transactions: The XA protocol is used for distributed transactions, but it can be inefficient.
- No support for complex queries: The current implementation does not support complex queries, such as OR, UNION, and subqueries.
Sharding-JDBC: A Distributed Micro-Service Database Access Framework
Sharding-JDBC is an open-source distributed database middleware that requires no additional deployment or dependence. It is designed to be scalable and flexible, with the following features:
- Complete implementation of sub-library sub-table: Separate read and write functions are supported, along with distributed primary keys.
- Flexible transaction management: Management functions include configuration data and dynamic source switching.
- Slice type: Slice type fragmentation and vertical fragmentation are supported, with policy-level slicing and horizontal partitioning.
- Implementation: Sharding-JDBC is implemented using a transparent proxy, which shields the fragmented rules and allows for seamless database access.
Why Sharding-JDBC?
Sharding-JDBC addresses the limitations of RDBMS solutions by:
- Resolving SQL parsing issues: Sharding-JDBC resolves SQL parsing issues by only parsing SQL when necessary.
- Separate read and write: Sharding-JDBC supports separate read and write operations, allowing for efficient data access.
- Flexible transaction management: Sharding-JDBC provides flexible transaction management functions, including configuration data and dynamic source switching.
Conclusion
Sharding-JDBC is a distributed micro-service database access framework designed to meet the demands of modern web applications. By addressing the limitations of RDBMS solutions, Sharding-JDBC provides a scalable and flexible solution for database access. Its features, including complete implementation of sub-library sub-table, flexible transaction management, and slice type fragmentation, make it an attractive choice for developers.