Redis Cluster

来自姬鸿昌的知识库
Jihongchang（讨论 | 贡献）2023年2月20日 (一) 05:09的版本 （→‎cluster-master-0.conf）
(差异) ←上一版本 | 最后版本 (差异) | 下一版本→ (差异)
跳到导航 跳到搜索

图解

1

生成缩略图出错：无法将缩略图保存到目标地点

每个节点主数据不同，是数据的子集

利用多台服务器构建集群提高超大规模数据处理能力

同时提供高可用支持



2

生成缩略图出错：无法将缩略图保存到目标地点

Redis Cluster 集群采用 Hash Slot（哈希槽）分配

Redis 集群预分好 16384 个槽，初始化集群时平均规划给每一台 Redis Master


为什么是 16384?

在 Redis 集群中槽分配的元数据会不间断的在 Redis 集群中分发，以保证所有节点都知晓槽的分配情况

16384=16k，在发送心跳包时使用 char 进行 bitmap 压缩后是 2k（2*8(8 bit) * 1024(1k)=16K）

通常我们不会部署超过10000个 Redis 主节点，因此16384就够用了



配置

cluster-master-0.conf

bind 127.0.0.1
port 6379
masterauth "vn4sj5kbxdaG"
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
cluster-announce-port 6379
cluster-announce-bus-port 16379



cluster-master-1.conf

bind 127.0.0.1
port 6380
masterauth "vn4sj5kbxdaG"
cluster-enabled yes
cluster-config-file nodes-6380.conf
cluster-node-timeout 15000
cluster-announce-port 6380
cluster-announce-bus-port 16380



cluster-master-2.conf

bind 127.0.0.1
port 6381
masterauth "vn4sj5kbxdaG"
cluster-enabled yes
cluster-config-file nodes-6381.conf
cluster-node-timeout 15000
cluster-announce-port 6381
cluster-announce-bus-port 16381


cluster-slave-0.conf

bind 127.0.0.1
port 6479
masterauth "vn4sj5kbxdaG"
cluster-enabled yes
cluster-config-file nodes-6479.conf
cluster-node-timeout 15000
cluster-announce-port 6479
cluster-announce-bus-port 16479


cluster-slave-1.conf

bind 127.0.0.1
port 6480
cluster-enabled yes
masterauth "vn4sj5kbxdaG"
cluster-config-file nodes-6480.conf
cluster-node-timeout 15000
cluster-announce-port 6480
cluster-announce-bus-port 16480


cluster-slave-2.conf

bind 127.0.0.1
port 6481
masterauth "vn4sj5kbxdaG"
cluster-enabled yes
cluster-config-file nodes-6481.conf
cluster-node-timeout 15000
cluster-announce-port 6481
cluster-announce-bus-port 16481


配置说明

bind 127.0.0.1
port 6379
masterauth "vn4sj5kbxdaG"
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
cluster-announce-port 6379
cluster-announce-bus-port 16379
cluster-enabled 是否开启集群
cluster-config-file 生成的 node 文件，记录集群节点信息，默认为 nodes.conf
cluster-node-timeout 节点连接超时时间
cluster-announce-port 集群节点映射端口
cluster-announce-bus-port 集群节点总线端口，节点之间互相通信，常规端口+1万，记录每个 key slot 在集群中哪个节点上的 2k bitmap 心跳包用此端口进行传输
masterauth master 密码，因为 replica 在 master failover 时会变成 master，所以也要进行 masterauth 配置




注意

  • 集群配置中不允许出现 replicaof 的配置项，否则就会报
    F:\下载目录\Redis-x64-5.0.14.1>.\redis-server.exe .\config\cluster\redis-cluster0.conf
[31444] 20 Feb 10:59:14.659 #
*** FATAL CONFIG FILE ERROR ***

[31444] 20 Feb 10:59:14.659 # Reading the configuration file, at line 1345

[31444] 20 Feb 10:59:14.660 # >>> 'replicaof 127.0.0.1 6380'

[31444] 20 Feb 10:59:14.660 # replicaof directive not allowed in cluster mode




启动

master-0

F:\下载目录\Redis-x64-5.0.14.1>.\redis-server.exe .\config\cluster\cluster-master-0.conf
[21000] 20 Feb 11:05:53.729 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
[21000] 20 Feb 11:05:53.729 # Redis version=5.0.14.1, bits=64, commit=ec77f72d, modified=0, pid=21000, just started
[21000] 20 Feb 11:05:53.729 # Configuration loaded
[21000] 20 Feb 11:05:53.732 * No cluster configuration found, I'm 74168c6581ab94338a85097768ae4cb376ef3c75
……
[21000] 20 Feb 11:05:53.927 # I have keys for unassigned slot 9231. Taking responsibility for it.
[21000] 20 Feb 11:05:53.928 # I have keys for unassigned slot 15118. Taking responsibility for it.
[21000] 20 Feb 11:05:53.928 # I have keys for unassigned slot 15901. Taking responsibility for it.
[21000] 20 Feb 11:05:53.928 # I have keys for unassigned slot 16066. Taking responsibility for it.
[21000] 20 Feb 11:05:53.977 * Ready to accept connections

master-1

F:\下载目录\Redis-x64-5.0.14.1>.\redis-server.exe .\config\cluster\cluster-master-1.conf
……

master-2

F:\下载目录\Redis-x64-5.0.14.1>.\redis-server.exe .\config\cluster\cluster-master-2.conf
……

slave-0

F:\下载目录\Redis-x64-5.0.14.1>.\redis-server.exe .\config\cluster\cluster-slave-0.conf
……

slave-1

F:\下载目录\Redis-x64-5.0.14.1>.\redis-server.exe .\config\cluster\cluster-slave-1.conf
……

slave-2

F:\下载目录\Redis-x64-5.0.14.1>.\redis-server.exe .\config\cluster\cluster-slave-2.conf
……


构建集群

F:\下载目录\Redis-x64-5.0.14.1>.\redis-cli.exe -a vn4sj5kbxdaG --cluster create ^
More? 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 ^
More? 127.0.0.1:6479 127.0.0.1:6480 127.0.0.1:6481 ^
More? --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:6480 to 127.0.0.1:6379
Adding replica 127.0.0.1:6481 to 127.0.0.1:6380
Adding replica 127.0.0.1:6479 to 127.0.0.1:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 505bd2d9507614b50b1d0d658a30d45ab92d28a6 127.0.0.1:6379
   slots:[0-5460] (5461 slots) master
M: 8559cf0b8c7e9a1b43c8ac4ec7f733c8d81b6acd 127.0.0.1:6380
   slots:[5461-10922] (5462 slots) master
M: cc20ad1cbc8aec6c7fca0fb2b98b1e76021f16fa 127.0.0.1:6381
   slots:[10923-16383] (5461 slots) master
S: dd97a0972a7a37bacb3208b6e7453edef7093249 127.0.0.1:6479
   replicates cc20ad1cbc8aec6c7fca0fb2b98b1e76021f16fa
S: fbcd0ec66809598e5b6ca39ae3a1a5b4cda79381 127.0.0.1:6480
   replicates 505bd2d9507614b50b1d0d658a30d45ab92d28a6
S: 72b733de5475dd38b5f351df8f3fcfeb08c8ddbd 127.0.0.1:6481
   replicates 8559cf0b8c7e9a1b43c8ac4ec7f733c8d81b6acd
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 127.0.0.1:6379)
M: 505bd2d9507614b50b1d0d658a30d45ab92d28a6 127.0.0.1:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 8559cf0b8c7e9a1b43c8ac4ec7f733c8d81b6acd 127.0.0.1:6380
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 72b733de5475dd38b5f351df8f3fcfeb08c8ddbd 127.0.0.1:6481
   slots: (0 slots) slave
   replicates 8559cf0b8c7e9a1b43c8ac4ec7f733c8d81b6acd
S: fbcd0ec66809598e5b6ca39ae3a1a5b4cda79381 127.0.0.1:6480
   slots: (0 slots) slave
   replicates 505bd2d9507614b50b1d0d658a30d45ab92d28a6
M: cc20ad1cbc8aec6c7fca0fb2b98b1e76021f16fa 127.0.0.1:6381
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: dd97a0972a7a37bacb3208b6e7453edef7093249 127.0.0.1:6479
   slots: (0 slots) slave
   replicates cc20ad1cbc8aec6c7fca0fb2b98b1e76021f16fa
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

F:\下载目录\Redis-x64-5.0.14.1>

遇到的问题

第一次构建集群的时候报错了

F:\下载目录\Redis-x64-5.0.14.1>.\redis-cli.exe -a vn4sj5kbxdaG --cluster create ^
More? 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 ^
More? 127.0.0.1:6479 127.0.0.1:6480 127.0.0.1:6481 ^
More? --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:6480 to 127.0.0.1:6379
Adding replica 127.0.0.1:6481 to 127.0.0.1:6380
Adding replica 127.0.0.1:6479 to 127.0.0.1:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 74168c6581ab94338a85097768ae4cb376ef3c75 127.0.0.1:6379
   slots:[0-5460],[5798],[6257],[9015],[9231],[15118],[15901],[16066] (5461 slots) master
M: 1636a36903d5dfa2b3206215e453132c0862fb97 127.0.0.1:6380
   slots:[741],[1539],[1569],[3745],[5461-10922],[15118],[15901],[16066] (5462 slots) master
M: 8f4aa2533b66eb39540b388925269adb8a508c42 127.0.0.1:6381
   slots:[741],[1539],[1569],[3745],[5798],[6257],[9015],[9231],[10923-16383] (5461 slots) master
S: f1122af668cffa20904b81b58b743e8515717975 127.0.0.1:6479
   replicates 1636a36903d5dfa2b3206215e453132c0862fb97
S: ad836ce0fd7f6982a6113d3cfa062d02bea0f800 127.0.0.1:6480
   replicates 8f4aa2533b66eb39540b388925269adb8a508c42
S: 43b77791f26c4d7096286f9d3f2e7134b0f78141 127.0.0.1:6481
   replicates 74168c6581ab94338a85097768ae4cb376ef3c75
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 127.0.0.1:6379)
S: 74168c6581ab94338a85097768ae4cb376ef3c75 127.0.0.1:6379
   slots: (0 slots) slave
   replicates 43b77791f26c4d7096286f9d3f2e7134b0f78141
M: 43b77791f26c4d7096286f9d3f2e7134b0f78141 127.0.0.1:6481
   slots:[741],[1539],[1569],[3745],[5798],[6257],[9015],[9231],[15118],[15901],[16066] (11 slots) master
   5 additional replica(s)
S: f1122af668cffa20904b81b58b743e8515717975 127.0.0.1:6479
   slots: (0 slots) slave
   replicates 43b77791f26c4d7096286f9d3f2e7134b0f78141
S: 1636a36903d5dfa2b3206215e453132c0862fb97 127.0.0.1:6380
   slots: (0 slots) slave
   replicates 43b77791f26c4d7096286f9d3f2e7134b0f78141
S: ad836ce0fd7f6982a6113d3cfa062d02bea0f800 127.0.0.1:6480
   slots: (0 slots) slave
   replicates 43b77791f26c4d7096286f9d3f2e7134b0f78141
S: 8f4aa2533b66eb39540b388925269adb8a508c42 127.0.0.1:6381
   slots: (0 slots) slave
   replicates 43b77791f26c4d7096286f9d3f2e7134b0f78141
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[ERR] Not all 16384 slots are covered by nodes.

因为监听 6481 的这个 Redis 服务实例之前用过，已经分配了一些 slot，所以构建集群的时候其他5个实例都把它当 master 了

然后：

  1. 把所有 Redis 实例停了
  2. 把 *.rdb 和 nodes-6379.conf、nodes-6380.conf、nodes-6381.conf、nodes-6479.conf、nodes-6480.conf、nodes-6481.conf 都删了
  3. 挨个重启6个实例

然后再重新构建集群就好了




参考

https://www.bilibili.com/video/BV1F44y1C7N8

取自“https://www.jihongchang.top/index.php?title=Redis_Cluster&oldid=5679”