一文搞懂数据库分片:分库分表,只分库不分表,只分表不分库...
分片需要先配置好:
HoneyConfig config=HoneyConfig.getHoneyConfig();
config.multiDS_enable=true;
config.multiDS_sharding=true;
1. 分库分表
参考bee-exam工程 ShardingPaging.java
ShardingConfig.addShardingBean(Orders.class,new ShardingBean("ds[0..1].orders[0..5]", "orderid"));
说明: 为orders表指定分版规则。
[INFO] [Bee] fullNodes: {orders={ds0=[0, 1, 2], ds1=[3, 4, 5]}}
[INFO] [Bee] tabToDsMap: {orders0=ds0, orders1=ds0, orders2=ds0, orders3=ds1, orders4=ds1, orders5=ds1}
从日志也可以看出:ds0这个数据库里有表:orders0,orders1,orders2;
ds1里有表:orders3,orders4,orders5;
分片键(分片字段是:orderid)
如何通过orderid字段的值找到相应的表和库;
默认是使用求余来映射,如orderid的值为10, 10%6=4,则表是orders4,再通过表来确定库是ds1(这样可以不用指定库的规则)。
[INFO] ds[0..1].orders[0..5]
[INFO] [Bee] fullNodes: {orders={ds0=[0, 1, 2], ds1=[3, 4, 5]}}
[INFO] [Bee] tabToDsMap: {orders0=ds0, orders1=ds0, orders2=ds0, orders3=ds1, orders4=ds1, orders5=ds1}
fullNodes是该实体(表)的所有结点分布情况.
tabToDsMap是表名到ds名称的映射 map.
2.仅分库
ShardingBean shardingBean=new ShardingBean();shardingBean.setDsField(Orders_F.userid);shardingBean.setDsName("ds");shardingBean.setDsRule("userid%2");shardingBean.setFullNodes("ds[0..1].orders[]"); //只分库,不分表(每个库的表名都一个;有多个库)ShardingConfig.addShardingBean(Orders.class, shardingBean);
日志:
[INFO] ds[0..1].orders[]
[INFO] [Bee] fullNodes: {orders={ds0=[], ds1=[]}}
[INFO] [Bee] tabToDsMap: {orders=ds1} //反查时,只会返回后面的ds名称
3.仅分表
ShardingBean shardingBean=new ShardingBean();shardingBean.setTabField(Orders_F.userid);shardingBean.setFullNodes("ds[].orders[0..2]"); //只分表,不分库(只有一个库)ShardingConfig.addShardingBean(Orders.class, shardingBean);
[INFO] ds[].orders[0..2]
[INFO] [Bee] fullNodes: {orders={ds=[0, 1, 2]}}
[INFO] [Bee] tabToDsMap: {orders0=ds, orders1=ds, orders2=ds}
4. 使用日期的字段作为分片键
//只分表,不分库(只有一个库)Map<String, DataSource> dataSourceMap = new HashMap<>();dataSourceMap.put("ds", dataSource0); //只有一个库BeeFactory.getInstance().setDataSourceMap(dataSourceMap);ShardingBean shardingBean=new ShardingBean();shardingBean.setTabField(Orders_F.createtime);//shardingBean.setFullNodes("ds[].orders_[202001..202012]"); //只分表,不分库(只有一个库)shardingBean.setFullNodes("ds[].orders_[202003..202004]"); //只分表,不分库(只有一个库)shardingBean.setTabAlgorithmClass(DateCalculate.class);ShardingConfig.addShardingBean(Orders.class, shardingBean);
定义日期规则计算类DateCalculate
import org.teasoft.bee.sharding.algorithm.Calculate;public class DateCalculate implements Calculate{@Overridepublic String process(String rule, String shardingValue) {return shardingValue.substring(0, 7).replace("-", ""); //eg:"202004";//shardingValue为null时,不会流到这; MultiTenancy注解可以用默认值;但Sharding注解,ShardingBean会走全域查所有}
}
日志:
[INFO] ds[].orders_[202003..202004]
[INFO] [Bee] fullNodes: {orders={ds=[_202003, _202004]}}
[INFO] [Bee] tabToDsMap: {orders_202003=ds, orders_202004=ds}
[INFO] [Bee] orders, its sepTab is: '_'
[INFO] [Bee] Reset the tabName OneTime, tabName:orders_202003
[INFO] [Bee] select SQL: select id,userid,orderid,name,total,createtime,remark,sequence,abc,updatetime from orders_202003 where createtime=? limit ?,? [values]: 2020-03-02 11:29:28(String),0(Integer),2(Integer)
[INFO] [Bee] select SQL: ( ExecutableSql )
select id,userid,orderid,name,total,createtime,remark,sequence,abc,updatetime from orders_202003 where createtime='2020-03-02 11:29:28' limit 0,2 ;
[INFO] [Bee] ========= the current DataSource name is :ds
[INFO] [Bee] | <-- select rows: 2
[INFO] Orders1[id=100001,userid=111,orderid=null,name=Bee(ORM Framework),total=95.010000,createtime=2020-03-02 11:29:28,remark=test,sequence=12345601,abc=test1,updatetime=null]
[INFO] Orders1[id=100002,userid=112,orderid=null,name=Bee,total=96.010000,createtime=2020-03-02 11:29:28,remark=test%a,sequence=12345602,abc=test2,updatetime=2020-03-02 16:34:19]
5. 不规则,通过自定义设置映射
ShardingBean shardingBean=new ShardingBean();shardingBean.setTabField(Orders_F.createtime);shardingBean.setFullNodes("ds[].orders_[a,b]"); //只分表,不分库(只有一个库)shardingBean.setTabAlgorithmClass(CustomCalculate.class);ShardingConfig.addShardingBean(Orders.class, shardingBean);
日志:
[INFO] ds[].orders_[a,b]
[INFO] [Bee] fullNodes: {orders={ds=[_a, _b]}}
[INFO] [Bee] tabToDsMap: {orders_a=ds, orders_b=ds}
[INFO] [Bee] orders, its sepTab is: '_'
定义映射计算CustomCalculate
/*** 无规律分布,通过直接定义来设置映射* @author Administrator*/
public class CustomCalculate implements Calculate{@Overridepublic String process(String rule, String shardingValue) {String t=shardingValue.substring(0, 7).replace("-", "");
// if("202004".equals(t)) return "a";
// else return "b";//直接定义映射关系;不一定是日期字段,其它字段值也可以;只是一个例子Map<String,String> map=new HashMap<>();map.put("202003", "a");map.put("202004", "b");String r=map.get(t);if(r!=null) return r;else return "Default-Value"; //TODO//shardingValue为null时,不会流到这; MultiTenancy注解可以用默认值;但Sharding注解,ShardingBean会走全域查所有}
}
参考Java ORM 工具Bee,一个简单易用又功能强大的ORM;
天下大势,分久必合!
Hibernate/MyBatis+ plus +Sharding JDBC + Jpa+ Spring data+ GraphQL+ App ORM (Android, 鸿蒙)= Bee
Spring Cloud 微服务使用数据库更方便:Bee + Spring Boot; 轻松支持多数据源,Sharding, Mongodb.
要整合一堆的工具,还不如只用一个小巧又功能强大的工具。
https://gitee.com/automvc/bee
bee: Bee,互联网新时代的Java ORM框架,支持Sharding;JDBC,Android,HarmonyOS;支持多种关系型数据库,还支持NoSQL的Cassandra,Mongodb等;更快、更简单、更自动,开发速度快,运行快,更智能!