1.引言
读写分离:顾名思义,就是将插入、更新、删除等事务性的操作分发到主数据库master上,而将读请求分发到从数据库slave上,利用mysql主从同步,从数据库服务器将同步更新本机数据,这样就保证了两个数据库中数据一致性。
读写分离有几种实现方式,可以在应用层配置多个数据源,根据业务需求访问不同的数据,指定对应的策略,也可以使用动态切换数据源方式实现,也可以借助数据库中间件的方式实现,mycat就是基于中间件实现。
2.原理图
在应用层和数据库集群之间添加一个代理服务,应用层访问代理,代理根据请求类型(读/写)自动分流到不同的数据库服务器。
下面通过一个完整的示例,说明mycat如何实现读写分离。
3.读写分离实现
本示例需要利用mysql主从复制,需要提前搭建主从同步环境,可以参考上一篇博客https://blog.csdn.net/BruceLiu_code/article/details/104694515,提先搭建好主从复制。
3.1.读写分离实现
在主服务器上执行以下SQL:
create database septwriteread;
use septwriteread;
create table user(id int not null,name varchar(20));
create database septwriteread02;
use septwriteread02;
create table user(id int not null,name varchar(20));
1234567
同时,从库也同步过来了主库创建的数据库和表:
3.2.配置server.xml
<user name="root" defaultAccount="true">
<property name="password">root</property>
<property name="schemas">TESTDB</property>
<!-- 表级 DML 权限设置 -->
<!-- <privileges check="false"> <schema name="TESTDB" dml="0110" > <table name="tb01" dml="0000"></table> <table name="tb02" dml="1111"></table> </schema> </privileges> -->
</user>
<user name="user">
<property name="password">user</property>
<property name="schemas">TESTDB</property>
<property name="readOnly">true</property>
</user>
1234567891011121314151617181920
3.3.schema.xml配置分片节点、分片表等
<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">
<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">
<table name="user" primaryKey="id" rule="mod-long" dataNode="dn1,dn2" />
</schema>
<dataNode name="dn1" dataHost="localhost1" database="septwriteread" />
<dataNode name="dn2" dataHost="localhost1" database="septwriteread02" />
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="3" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<!-- 主数据库130 -->
<writeHost host="hostM1" url="192.168.70.128:3306" user="root" password="123">
<!-- 从数据库131 -->
<readHost host="hostS1" url="192.168.70.130:3306" user="root" password="123" />
</writeHost>
</dataHost>
</mycat:schema>
123456789101112131415161718192021222324
3.4.配置rule.xml
<function name="mod-long" class="io.mycat.route.function.PartitionByMod">
<!-- how many data nodes -->
<property name="count">2</property>
</function>
<tableRule name="mod-long">
<rule>
<columns>id</columns>
<algorithm>mod-long</algorithm>
</rule>
</tableRule>
1234567891011
在rule.xml修改配置文件
3.5.测试插入数据
在MyCat服务器192.168.70.129上面执行下面测试语句:
insert into user(id,name) values(1111111,database());
insert into user(id,name) values(2222222,database());
insert into user(id,name) values(3333333,database());
insert into user(id,name) values(4444444,database());
1234
重启mycat:
[root@centos1 bin]# ./mycat start
Starting Mycat-server...
[root@centos1 bin]# jps
16345 WrapperSimpleApp
16366 Jps
12345
登录mycat:
3.6.测试分析查询数据
select * from user;
1
下面分几种情况分析当balance取不同值,请求分发的情况如下:
修改log4j2.xml查看mycat.log debug日志级别
<Loggers>
<!--<AsyncLogger name="io.mycat" level="info" includeLocation="true" additivity="false">-->
<!--<AppenderRef ref="Console"/>-->
<!--<AppenderRef ref="RollingFile"/>-->
<!--</AsyncLogger>-->
<asyncRoot level="debug" includeLocation="true">
<!--<AppenderRef ref="Console" />-->
<AppenderRef ref="RollingFile"/>
</asyncRoot>
</Loggers>
123456789101112
【a】当balance=“3”,开启负载均衡,请求分发到writeHost对应的readHost。查看mycat.log debug日志,走从服务器192.168.70.130的hostS1节点。
2019-12-26 13:12:31.679 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.datasource.PhysicalDBPool.getRWBanlanceCon(PhysicalDBPool.java:551)) - select read source hostS1 for dataHost:localhost1
2019-12-26 13:12:31.679 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.mysql.nio.MySQLConnection.synAndDoExecute(MySQLConnection.java:463)) - con need syn ,total syn cmd 1 commands SET names utf8mb4;schema change:false con:MySQLConnection [id=11, lastTime=1577337151679, user=root, schema=septwriteread, old shema=septwriteread, borrowed=true, fromSlaveDB=true, threadId=21, charset=utf8, txIsolation=3, autocommit=true, attachment=dn1{
SHOW PROCEDURE STATUS WHERE Db='TESTDB'}, respHandler=SingleNodeHandler [node=dn1{
SHOW PROCEDURE STATUS WHERE Db='TESTDB'}, packetId=0], host=192.168.70.130, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
12
【b】当balance=“0”,表示不开启负载均衡,查看mycat.log debug日志,走主服务器192.168.70.128,可见所有读操作都发送到writeHost中。
修改schema.xml中balance负载均衡的值:
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<!-- 主数据库130 -->
<writeHost host="hostM1" url="192.168.70.128:3306" user="root" password="123">
<!-- 从数据库131 -->
<readHost host="hostS1" url="192.168.70.130:3306" user="root" password="123" />
</writeHost>
</dataHost>
1234567891011
然后重启mycat,在主库执行查询语句 select * from user;查看日志:
2019-12-26 13:15:48.866 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.datasource.PhysicalDBPool.getRWBanlanceCon(PhysicalDBPool.java:551)) - select read source hostM1 for dataHost:localhost1
2019-12-26 13:15:48.866 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.mysql.nio.MySQLConnection.synAndDoExecute(MySQLConnection.java:463)) - con need syn ,total syn cmd 1 commands SET names utf8mb4;schema change:false con:MySQLConnection [id=8, lastTime=1577337348866, user=root, schema=septwriteread, old shema=septwriteread, borrowed=true, fromSlaveDB=false, threadId=39, charset=utf8, txIsolation=3, autocommit=true, attachment=dn1{
SHOW STATUS}, respHandler=SingleNodeHandler [node=dn1{
SHOW STATUS}, packetId=0], host=192.168.70.128, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
12
【c】当balance=“2”,表示请求随机分发到writeHost或者readhost。查看mycat.log debug日志,走192.168.8.128或者192.168.8.130,所有读操作都随机发送到writeHost或者readHost中。
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="2" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<!-- 主数据库130 -->
<writeHost host="hostM1" url="192.168.70.128:3306" user="root" password="123">
<!-- 从数据库131 -->
<readHost host="hostS1" url="192.168.70.130:3306" user="root" password="123" />
</writeHost>
</dataHost>
1234567891011
然后重启mycat,在主库执行查询语句 select * from user;查看日志:
2019-12-26 13:19:48.138 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.datasource.PhysicalDBPool.getRWBanlanceCon(PhysicalDBPool.java:551)) - select read source hostM1 for dataHost:localhost1
2019-12-26 13:19:48.138 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.mysql.nio.MySQLConnection.synAndDoExecute(MySQLConnection.java:463)) - con need syn ,total syn cmd 1 commands SET names utf8mb4;schema change:false con:MySQLConnection [id=1, lastTime=1577337588138, user=root, schema=septwriteread, old shema=septwriteread, borrowed=true, fromSlaveDB=false, threadId=51, charset=utf8, txIsolation=3, autocommit=true, attachment=dn1{
SELECT *
FROM user
LIMIT 100}, respHandler=io.mycat.backend.mysql.nio.handler.MultiNodeQueryHandler@713db746, host=192.168.70.128, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
1234
2019-12-26 13:21:30.387 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.datasource.PhysicalDBPool.getRWBanlanceCon(PhysicalDBPool.java:551)) - select read source hostS1 for dataHost:localhost1
2019-12-26 13:21:30.387 DEBUG [$_NIOREACTOR-0-RW] (io.mycat.backend.mysql.nio.MySQLConnection.synAndDoExecute(MySQLConnection.java:436)) - not need syn connection :
MySQLConnection [id=13, lastTime=1577337690387, user=root, schema=septwriteread, old shema=septwriteread, borrowed=true, fromSlaveDB=true, threadId=27, charset=utf8, txIsolation=3, autocommit=true, attachment=dn1{
SHOW STATUS}, respHandler=SingleNodeHandler [node=dn1{
SHOW STATUS}, packetId=0], host=192.168.70.130, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
123
【d】当balance=1时,开启读写分离机制,所有读操作都发送到当前writeHost对应的readHost和备用的writeHost上,通过日志分析所有读操作都发送到hostS1或者hostM2【让全部的readHost及备用的writeHost参与select的负载均衡。 】
<dataHost name="localhost1" maxCon="1000" minCon="10" balance="2" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<!-- 主数据库130 -->
<writeHost host="hostM1" url="192.168.8.128:3306" user="root" password="123">
<!-- 从数据库131 -->
<readHost host="hostS1" url="192.168.8.130:3306" user="root" password="123" />
</writeHost>
<!-- stand by writehost -->
<writeHost host="hostM2" url="192.168.8.130:3306" user="root" password="123">
</writeHost>
</dataHost>
12345678910111213141516
4.总结
下面补充一下dataHost节点中比较重要的三个属性以及各自值的含义,具体如下。
balance指的负载均衡类型,目前的取值有4种:
1. balance=”0”, 不开启读写分离机制,所有读操作都发送到当前可用的writeHost上;
2. balance=”1”,全部的readHost与stand bywriteHost参与select语句的负载均衡,简单的说,当双主双从模式(M1->S1,M2->S2,并且M1与 M2互为主备),正常情况下,M2,S1,S2都参与select语句的负载均衡;
3. balance=”2”,所有读操作都随机的在writeHost、readhost上分发;
4. balance=”3”,所有读请求随机的分发到wiriterHost对应的readhost执行,writerHost不负担读压力;
1234
注意:balance=3只在1.4及其以后版本有,1.3没有;
writeType属性:
1.writeType=”0”,所有写操作发送到配置的第一个writeHost,第一个挂了切到还生存的第二个writeHost,重新启动后以切换后的为准,切换记录在配置文件中:dnindex.properties;
2.writeType=”1”,所有写操作都随机地发送到配置的writeHost,1.5以后废弃不推荐;
12
switchType指的是切换的模式,目前的取值也有4种:
1. switchType=’-1’ 表示不自动切换;
2. switchType=’1’ 默认值,表示自动切换;
3. switchType=’2’ 基于MySQL主从同步的状态决定是否切换,心跳语句为 show slave status;
4. switchType=’3’基于MySQLgalary cluster的切换机制(适合集群)(1.4.1),心跳语句为 show status like ‘wsrep%’;