Redis High Availability Cluster Practice

1. Environmental Instructions

redis version: Redis-5.0.8
System version: Redhat6.8

Host IP
redispri1 110.50.1.55
redispri2 110.50.1.56
redispri3 110.50.1.57
redisbck1 110.50.1.57
redisbck2 110.50.1.59
redisbck3 110.50.1.60

2. Build a redis highly available cluster

  1. Each node closes NetworkManager, selinux=disable
    # /etc/init.d/NetworkManager stop
    # chkconfig NetworkManager off
    # sed -i "s/SELINUX=.*/SELINUX=disabled/g" /etc/selinux/config && setenforce 0
  2. Install Redis, GCC gcc-c++ libstdc+++ devel on each node
    # tar xvf redis-5.0.8.tar.gz -C /usr/local/
    # mv /usr/local/redis-5.0.8/ /usr/local/redis/
    # yum install  -y gcc gcc-c++ libstdc++-devel
    # cd /usr/local/redis/src && make MALLOC=libc && make install prefix=/uer/local/redis
  3. Each node sets up Iptables, opening port 6379/16379
    # iptables -I INPUT -p TCP --dport 6379 -j ACCEPT
    # iptables -I INPUT -p TCP --dport 16379 -j ACCEPT
    # /etc/init.d/iptables save
    # chkconfig iptables on
  4. Cluster Configuration
    #Create backup file storage, data file storage, log file storage, AOF mode file storage
    # mkdir -p /appdata/redis/{db,file,log,aof}
    #Modify profile parameters
    # cp /usr/local/redis/redis.conf /usr/local/redis/redis.conf.bak
    # vim /usr/local/redis/redis_6379.conf
    bind Current Node IP
    port 6379
    daemonize yes
    pidfile /var/run/redis_6379.pid
    #Verify environment debug, production environment notice
    loglevel debug       
    logfile /appdata/redis/log/redis.log
    save 900 1
    save 300 10
    save 60 10000
    dbfilename dump.rdb
    dir /appdata/redis/file/
    requirepass 202004
    maxclients 10000
    maxmemory 100M
    appendonly yes
    appendfilename appendonly.aof
    appendfsync everysec
    auto-aof-rewrite-percentage 100
    auto-aof-rewrite-min-size 64mb
    cluster-enabled yes
    cluster-config-file nodes-6379.conf
    cluster-node-timeout 5000
    #Disable emptying all records
    rename-command FLUSHALL ""   
    #Disable Empty Database
    rename-command FLUSHDB ""    
    #Configurable server when client connection is disabled
    #rename-command CONFIG ""     
    #Disable client connections to view all existing keys
    rename-command KEYS ""
  5. Start redis for each node
    # /usr/local/bin/redis-server /usr/local/redis/redis_6379.conf
    #bin directory file description
    redis-benchmark  #Scripts for performance testing
    redis-check-aof  #The script used for redis data persistence is to store one
    redis-check-rdb #Scripts are used for redis data persistence and are stored every once in a while
    redis-cli        #Scripts for client-to-redis connections
    redis-sentinel -- >redis-server  #Scripts are used on clusters, Sentinels
    redis-server     #Script for opening redis service
  6. Create Cluster
    # redis-cli --cluster create --cluster-replicas 1 110.50.1.55:6379 110.50.1.56:6379 110.50.1.57:6379 110.50.1.58:6379 110.50.1.59:6379 110.50.1.60:6379>>> Performing hash slots allocation on 6 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    Adding replica 110.50.1.59:6379 to 110.50.1.55:6379
    Adding replica 110.50.1.60:6379 to 110.50.1.56:6379
    Adding replica 110.50.1.58:6379 to 110.50.1.57:6379
    M: d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379
    slots:[0-5460] (5461 slots) master
    M: ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379
    slots:[5461-10922] (5462 slots) master
    M: 6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379
    slots:[10923-16383] (5461 slots) master
    S: b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379
    replicates 6a75af4717e1890a632ffffcad4d1e8b98cebcab
    S: 00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379
    replicates d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515
    S: f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379
    replicates ec79401490c99d77c878c9a8b7cfb08fe237b949
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    ..
    >>> Performing Cluster Check (using node 110.50.1.55:6379)
    M: d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379
    slots:[0-5460] (5461 slots) master
    1 additional replica(s)
    S: b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379
    slots: (0 slots) slave
    replicates 6a75af4717e1890a632ffffcad4d1e8b98cebcab
    S: 00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379
    slots: (0 slots) slave
    replicates d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515
    M: 6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379
    slots:[10923-16383] (5461 slots) master
    1 additional replica(s)
    S: f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379
    slots: (0 slots) slave
    replicates ec79401490c99d77c878c9a8b7cfb08fe237b949
    M: ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379
    slots:[5461-10922] (5462 slots) master
    1 additional replica(s)
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.
  7. View cluster status
    # redis-cli -c -h 110.50.1.55 -p 6379
    # View Cluster Nodes
    110.50.1.55:6379> cluster nodes
    b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379@16379 slave 6a75af4717e1890a632ffffcad4d1e8b98cebcab 0 1587694145212 4 connected
    00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379@16379 slave d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 0 1587694145000 5 connected
    6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379@16379 master - 0 1587694145000 3 connected 10923-16383
    f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379@16379 slave ec79401490c99d77c878c9a8b7cfb08fe237b949 0 1587694146623 6 connected
    d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379@16379 myself,master - 0 1587694146000 1 connected 0-5460
    ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379@16379 master - 0 1587694146219 2 connected 5461-10922
    # View cluster information
    110.50.1.55:6379> cluster info
    cluster_state:ok
    cluster_slots_assigned:16384
    cluster_slots_ok:16384
    cluster_slots_pfail:0
    cluster_slots_fail:0
    cluster_known_nodes:6
    cluster_size:3
    cluster_current_epoch:6
    cluster_my_epoch:1
    cluster_stats_messages_ping_sent:828
    cluster_stats_messages_pong_sent:826
    cluster_stats_messages_sent:1654
    cluster_stats_messages_ping_received:821
    cluster_stats_messages_pong_received:828
    cluster_stats_messages_meet_received:5
    cluster_stats_messages_received:1654
    # View Cluster Fragmentation
    110.50.1.55:6379> cluster slots
    1) 1) (integer) 10923
    2) (integer) 16383
    3) 1) "110.50.1.57"
      2) (integer) 6379
      3) "6a75af4717e1890a632ffffcad4d1e8b98cebcab"
    4) 1) "110.50.1.58"
      2) (integer) 6379
      3) "b38d96efe145a810dd48e3349002794c98649d72"
    2) 1) (integer) 0
    2) (integer) 5460
    3) 1) "110.50.1.55"
      2) (integer) 6379
      3) "d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515"
    4) 1) "110.50.1.59"
      2) (integer) 6379
      3) "00d4fdf753d8097637d3c96b1da6abe43ebb7e59"
    3) 1) (integer) 5461
    2) (integer) 10922
    3) 1) "110.50.1.56"
      2) (integer) 6379
      3) "ec79401490c99d77c878c9a8b7cfb08fe237b949"
    4) 1) "110.50.1.60"
      2) (integer) 6379
      3) "f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe"
  8. Set Cluster Password
    requirepass: Password for external services, clients to connect redis.
    Master auth:redis The password used by the master to connect redis from.
    This means that if you set the requirepass parameter on the master, you need to set the master auth parameter on the master as specified by the master password.This allows you to continue to synchronize the master data.
    Method one: Profile adds fields, but needs to be restarted
    requirepass "passwd"
    masterauth "passwd"
    Method 2: Log on redis to execute the following command without restarting the service
     config rewrite persists a config set to a Redis configuration file
    config set requirepass "passwd"
    config set masterauth "passwd"
    config rewrite
  9. Validation tests and conclusions

Shut down 1 primary node randomly, corresponding slave node is selected as primary node, cluster still serves external normally

*#Normal Cluster*
112.111.110.50.1.55:6379> cluster nodes
d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379@16379 myself,master - 0 1587699843000 9 connected 0-5460
00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379@16379 slave d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 0 1587699845503 9 connected
b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379@16379 slave 6a75af4717e1890a632ffffcad4d1e8b98cebcab 0 1587699844597 4 connected
ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379@16379 master - 0 1587699845504 10 connected 5461-10922
f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379@16379 slave ec79401490c99d77c878c9a8b7cfb08fe237b949 0 1587699844902 10 connected
6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379@16379 master - 0 1587699844000 3 connected 10923-16383

*#Close primary node110.50.1.57*
*#Node state is master, fail, previous slave110.50.1.58Selected as master*
111.110.50.1.55:6379> cluster nodes*
d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379@16379 myself,master - 0 1587699968000 9 connected 0-5460
00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379@16379 slave d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 0 1587699967723 9 connected
b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379@16379 master - 0 1587699969535 12 connected 10923-16383
ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379@16379 master - 0 1587699967519 10 connected 5461-10922
f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379@16379 slave ec79401490c99d77c878c9a8b7cfb08fe237b949 0 1587699969739 10 connected
6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379@16379 master,fail - 1587699941783 1587699941000 3 disconnected

*#Set up Key and view it randomly so that clusters are available*

111.110.50.1.55:6379> set helloworld 111
OK
110.50.1.58:6379> get helloworld
-> Redirected to slot [2739] located at 110.50.1.55:6379
"111"
110.50.1.59:6379> get helloworld
-> Redirected to slot [2739] located at 110.50.1.55:6379
"111"

Random manual shutdown of 1 slave node, the cluster still serves normally

*#Close Slave Node110.50.1.59*
*#Node status is slave,fail*
111.110.50.1.56:6379> cluster nodes
00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379@16379 slave,fail d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 1587703268981 1587703268000 9 disconnected
f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379@16379 slave ec79401490c99d77c878c9a8b7cfb08fe237b949 0 1587703278821 10 connected
b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379@16379 slave 6a75af4717e1890a632ffffcad4d1e8b98cebcab 0 1587703278000 13 connected
ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379@16379 myself,master - 0 1587703277000 10 connected 5461-10922
d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379@16379 master - 0 1587703277819 9 connected 0-5460
6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379@16379 master - 0 1587703278000 13 connected 10923-16383

*#Set up Key and view it randomly so that clusters are available*

111.110.50.1.56:6379> set helloworld 222
-> Redirected to slot [2739] located at 110.50.1.55:6379
OK
110.50.1.58:6379> get helloworld
-> Redirected to slot [2739] located at 110.50.1.55:6379
"222"

Close a pair of master-slaves randomly

*#Random active shutdown of a pair of primary and secondary nodes 110.50.1.57, 110.50.1.58*
*#Error CLUSTERDOWN The cluster is down, cluster is no longer available*
111.110.50.1.55:6379> cluster nodes
d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379@16379 myself,master - 0 1587703705000 9 connected 0-5460
00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379@16379 slave d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 0 1587703706000 9 connected
b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379@16379 slave,fail 6a75af4717e1890a632ffffcad4d1e8b98cebcab 1587703697645 1587703697546 13 disconnected
ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379@16379 master - 0 1587703705557 10 connected 5461-10922
f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379@16379 slave ec79401490c99d77c878c9a8b7cfb08fe237b949 0 1587703706165 10 connected
6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379@16379 master,fail - 1587703692596 1587703691995 13 disconnected 10923-16383
110.50.1.55:6379> set helloworld 333
(error) CLUSTERDOWN The cluster is down

Shut down two primary nodes randomly

*#Shut down two primary nodes randomly 110.50.1.57, 110.50.1.56*
*#Error CLUSTERDOWN The cluster is down, cluster is no longer available*
111.110.50.1.55:6379> cluster nodes
d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 110.50.1.55:6379@16379 myself,master - 0 1587704575000 9 connected 0-5460
00d4fdf753d8097637d3c96b1da6abe43ebb7e59 110.50.1.59:6379@16379 slave d851fcaf25e413e9e58d0ad96c91ac2d7b4ae515 0 1587704575494 9 connected
b38d96efe145a810dd48e3349002794c98649d72 110.50.1.58:6379@16379 slave 6a75af4717e1890a632ffffcad4d1e8b98cebcab 0 1587704573466 13 connected
ec79401490c99d77c878c9a8b7cfb08fe237b949 110.50.1.56:6379@16379 master,fail? - 1587704561741 1587704560000 10 disconnected 5461-10922
f66e8c1f3e7e3cd35a86b11dd0506b1fc24691fe 110.50.1.60:6379@16379 slave ec79401490c99d77c878c9a8b7cfb08fe237b949 0 1587704575496 10 connected
6a75af4717e1890a632ffffcad4d1e8b98cebcab 110.50.1.57:6379@16379 master,fail? - 1587704566981 1587704566000 13 disconnected 10923-16383
110.50.1.55:6379> set helloworld 444
(error) CLUSTERDOWN The cluster is down

Verification conclusion:
1) How does a cluster determine if a node is hung up and a new election takes place
Each node of the redis cluster has information about all the primary and secondary nodes of the cluster.They are ping-pong each other to determine if a node can be connected.If more than half of the nodes Ping a node without responding, the cluster considers the node down, connects its standby nodes, and chooses the new master
2) Cluster unavailable
a. If any master of the cluster hangs up and the current master has no slave. The cluster is not available
b. If more than half of the master s in the cluster are suspended, the cluster will not be available with or without slave s

3. Redis's own testing tools

# redis-benchmark -h 110.50.1.59 -p 6382 -t set,get -c 10 -n 1000000
====== SET ======
  1000000 requests completed in 45.44 seconds
  10 parallel clients
  3 bytes payload
  keep alive: 1

89.95% <= 1 milliseconds
99.99% <= 2 milliseconds
100.00% <= 3 milliseconds
100.00% <= 3 milliseconds
22005.59 requests per second

====== GET ======
  1000000 requests completed in 45.68 seconds
  10 parallel clients
  3 bytes payload
  keep alive: 1

90.05% <= 1 milliseconds
99.99% <= 2 milliseconds
100.00% <= 3 milliseconds
100.00% <= 5 milliseconds
100.00% <= 7 milliseconds
21891.90 requests per second

4. Problems encountered in the construction process and Solutions

1. Compilation errors

# make && make install prefix=/usr/local/redis/
ake[1]: Entering directory `/usr/local/redis/src'
    CC adlist.o
 In the file containing from adlist.c:34:
zmalloc.h:50:31: Error: jemalloc/jemalloc.h: No file or directory
 zmalloc.h:55:2: Error: #error "Newer version of jemalloc required"
make[1]: *** [adlist.o] Error 1
make[1]: Leaving directory `/usr/local/redis/src'
make: *** [all] Error 2

1) The src directory contains command scripts for redis server and redis cli, and should be compiled in this directory
2) There is a passage in README.

Allocator
Selecting a non-default memory allocator when building Redis is done by setting
the MALLOC environment variable. Redis is compiled and linked against libc
malloc by default, with the exception of jemalloc being the default on Linux systems. This default was picked because jemalloc has proven to have fewer
fragmentation problems than libc malloc.
To force compiling against libc malloc, use:
% make MALLOC=libc
To compile against jemalloc on Mac OS X systems, use:
% make MALLOC=jemalloc

Allocator, if there is an environment variable MALLOC, it will be used to create Redis.The default allocator is jemalloc, because jemalloc and libc have fewer fragmentation problems than they do.Compilation error because the system has no jemalloc and only libc.

Solution:
(1) Adding parameters
make MALLOC=libc
(2) Download and install jemalloc
https://github.com/jemalloc/jemalloc/releases
./configure && make && make install
2. Startup error

# dbfilename appendfilename cannot use a path
# /usr/local/bin/redis-server /usr/local/redis/redis_6379.conf 
*** FATAL CONFIG FILE ERROR ***
Reading the configuration file, at line 253
>>> 'dbfilename "/appdata/redis/db/dump.rdb"'
dbfilename can't be a path, just a filename

*** FATAL CONFIG FILE ERROR ***
Reading the configuration file, at line 706
>>> 'appendfilename "/appdata/redis/aof/appendonly.aof"'
appendfilename can't be a path, just a filename

# Commands cannot be commented after they are disabled
# /usr/local/bin/redis-server /usr/local/redis/redis_6379.conf
*** FATAL CONFIG FILE ERROR ***
Reading the configuration file, at line 325
>>> 'rename-command FLUSHALL ""   #Disable emptying all records'
Bad directive or wrong number of arguments

3. Connection Error

Whether it's in redis-server version 5.x or the old ruby way of creating clusters, redis passwords can't be configured in the create cluster section.
If a password is set, redis-cli --cluster create will report a user authentication failure error
 # redis-cli --cluster create --cluster-replicas 1 110.50.1.55:6379 110.50.1.56:6379 110.50.1.57:6379 110.50.1.58:6379 110.50.1.59:6379 110.50.1.60:6379
[ERR] Node 110.50.1.55:6379 NOAUTH Authentication required.
##################################################
Solution
 Remove passwords for all redis nodes when redis-cli --cluster create creates the cluster
 When the cluster configuration is complete, passwords are dynamically set for each node via config set tings (redis do not need to be restarted and remain valid after restart)
redis-cli -h 127.0.0.1 -p 6379 -c
 127.0.0.1:6379> config set requirepass'password'//Set password
 127.0.0.1:6379> config set master auth'password'//Set connection password
 127.0.0.1:6379> config rewrite //Write the config set operation to the configuration file
 Connect to the cluster after setting the password
redis-cli -h 127.0.0.1 -p 6379 -c -a password

5. redis-cli --cluster help

redis-cli --cluster help
Cluster Manager Commands:
  create         host1:port1 ... hostN:portN   #Create Cluster
                 --cluster-replicas <arg>      #Number of slaves
  check          host:port                     #Check Cluster
                 --cluster-search-multiple-owners #Check if slots are assigned to more than one node at a time
  info           host:port                     #View cluster status
  fix            host:port                     #Repair Cluster
                 --cluster-search-multiple-owners #Repair Slot Duplicate Allocation Problem
  reshard        host:port                     #Specify any node of the cluster to migrate slots and redivide slots
                 --cluster-from <arg>          #Which source nodes do you want to migrate slot s from? Migration can be done from multiple source nodes, separated by commas, passing the node id of the node, or directly--from all, so that the source node is all the nodes of the cluster. Without passing this parameter, the user is prompted for input during the migration process
                 --cluster-to <arg>            #slot requires the node id of the destination node to be migrated. Only one destination node can be filled in. Without passing this parameter, the user will be prompted for input during migration.
                 --cluster-slots <arg>         #The number of slot s to migrate, without passing this parameter, prompts the user for input during the migration process.
                 --cluster-yes                 #Specify confirmation input for migration
                 --cluster-timeout <arg>       #Set timeout for migrate command
                 --cluster-pipeline <arg>      #Define the number of keys taken out at a time by the cluster getkeysinslot command, using the default value of 10 if no key is passed
                 --cluster-replace             #Whether to replace directly to the target node
  rebalance      host:port                                      #Balance the number of cluster nodes slot s by specifying any node of the cluster 
                 --cluster-weight <node1=w1...nodeN=wN>         #Specify the weight of cluster nodes
                 --cluster-use-empty-masters                    #Settings allow primary nodes without slot s to participate, not allowed by default
                 --cluster-timeout <arg>                        #Set timeout for migrate command
                 --cluster-simulate                             #Simulate rebalance operation, do not actually perform migration operation
                 --cluster-pipeline <arg>                       #Defines the number of keys taken out at a time by the cluster getkeysinslot command, with a default of 10
                 --cluster-threshold <arg>                      #Migrate slot threshold beyond threshold, perform rebalance operation
                 --cluster-replace                              #Whether to replace directly to the target node
  add-node       new_host:new_port existing_host:existing_port  #Add node, add new node to specified cluster, add primary node by default
                 --cluster-slave                                #New node as slave, default random primary node
                 --cluster-master-id <arg>                      #Specify a primary node for the new node
  del-node       host:port node_id                              #Delete a given node and shut down the node service after success
  call           host:port command arg arg .. arg               #Execute commands on all nodes of the cluster
  set-timeout    host:port milliseconds                         #Set cluster-node-timeout
  import         host:port                                      #Importing external redis data into a cluster
                 --cluster-from <arg>                           #Import data from the specified instance into the cluster
                 --cluster-copy                                 #Specify copy when migrating
                 --cluster-replace                              #Specify replace when migrate
  help           

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

Tags: Linux Redis iptables SELinux yum

Posted on Mon, 18 May 2020 09:34:46 -0700 by kansaschuck