Redis full check verifies the data of two different redis instances

Redis full check checks whether redis data is consistent:

Verify the data of two different redis instances:

6986 is redis instance 1
6987 is redis instance 2

Log in to 6986 redis instance 1, simulate setting 4 key values

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6986 -a 'Y2hJKSGtuEq'
Warning: Using a password with '-a' option on the command line interface may not be safe.
127.0.0.1:6986> keys *
(empty list or set)
127.0.0.1:6986> set test001 001
OK
127.0.0.1:6986> set test002 002
OK
127.0.0.1:6986> set test003 003
OK
127.0.0.1:6986> set test004 004
OK
127.0.0.1:6986> keys *
1) "test003"
2) "test002"
3) "test004"
4) "test001"
127.0.0.1:6986>

Log in to 6987 redis instance 2, simulate setting 2 key values

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6987 -a 'Y2hJKSGtuEq'
Warning: Using a password with '-a' option on the command line interface may not be safe.
127.0.0.1:6987> keys *
(empty list or set)
127.0.0.1:6987> set test002 a02
OK
127.0.0.1:6987> set test004 a04
OK
127.0.0.1:6987> keys *
1) "test004"
2) "test002"
127.0.0.1:6987> 

-The m parameter specifies the verification mode, which is divided into four types:
-m parameter comparison mode: 1 for full comparison, 2 for the length of value only, 3 for the existence of key, 4 for full comparison, ignore the comparison of big key

From the key name, the key in the 6987 instance is a subset of the key in the 6986 instance:
If the verification mode is not specified, the default is to use - m 3 to check whether the key exists. It is recommended to use - m 1 full comparison mode when verifying different redis instance data

Take 6987 instance as the original instance and 6986 instance as the target instance for verification:

[root@localhost redis-full-check-1.4.7]# redis-full-check   -s 127.0.0.1:6987  -p 'Y2hJKSGtuEq' -t 127.0.0.1:6986 -a 'Y2hJKSGtuEq'  --log=log  --result=result
[root@localhost redis-full-check-1.4.7]# ls
log  redis-full-check  result  result.db.1  result.db.2  result.db.3

[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from key;
sqlite> select * from field;
sqlite>

In terms of the key name, since the key in the 6987 instance is a subset of the key in the 6986 instance, there is no difference

At the same time, you can see from the log:

[INFO 2020-02-08-12:29:32 main.go:65]: init log success
[INFO 2020-02-08-12:29:32 main.go:164]: configuration: {127.0.0.1:6987 Y2hJKSGtuEq auth 0 -1 127.0.0.1:6986 Y2hJKSGtuEq auth 0 -1 result.db result 3 2 unknown unknown unknown 15000 5 256 5 lo
g  false 16384  20445 false}
[INFO 2020-02-08-12:29:32 main.go:166]: ---------
[INFO 2020-02-08-12:29:32 full_check.go:238]: sourceDbType=0, p.sourcePhysicalDBList=[meaningless]
[INFO 2020-02-08-12:29:32 full_check.go:243]: db=0:keys=2
[INFO 2020-02-08-12:29:32 full_check.go:253]: ---------------- start 1th time compare
[INFO 2020-02-08-12:29:32 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:29:32 scan.go:20]: build connection[source redis addr: [127.0.0.1:6987]]
[INFO 2020-02-08-12:29:33 full_check.go:203]: stat:
times:1, db:0, dbkeys:2, finish:33%, finished:true
KeyScan:{2 2 0}

[INFO 2020-02-08-12:29:33 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:29:38 full_check.go:253]: ---------------- start 2th time compare
[INFO 2020-02-08-12:29:38 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:29:38 full_check.go:203]: stat:
times:2, db:0, finished:true
KeyScan:{0 0 0}

[INFO 2020-02-08-12:29:38 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:29:43 full_check.go:253]: ---------------- start 3th time compare
[INFO 2020-02-08-12:29:43 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:29:43 full_check.go:203]: stat:
times:3, db:0, finished:true
KeyScan:{0 0 0}

[INFO 2020-02-08-12:29:43 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 0 key(s) and 0 field(s) conflict

But if you compare the original instance with the target instance, you will find the inconsistency

Take 6986 instance as the original instance and 6987 instance as the target instance for verification:

redis-full-check   -s 127.0.0.1:6986  -p 'Y2hJKSGtuEq' -t 127.0.0.1:6987 -a 'Y2hJKSGtuEq'  --log=log  --result=result
[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from field;
sqlite> select * from key;
1|test001|string|lack_target|0|3|0
2|test003|string|lack_target|0|3|0

It is found that there are two more key s for instance 6986 than for instance 6987

You can also see the differences in the logs:

[INFO 2020-02-08-12:35:10 main.go:65]: init log success
[INFO 2020-02-08-12:35:10 main.go:164]: configuration: {127.0.0.1:6986 Y2hJKSGtuEq auth 0 -1 127.0.0.1:6987 Y2hJKSGtuEq auth 0 -1 result.db result 3 2 unknown unknown unknown 15000 5 256 5 lo
g  false 16384  20445 false}
[INFO 2020-02-08-12:35:10 main.go:166]: ---------
[INFO 2020-02-08-12:35:10 full_check.go:238]: sourceDbType=0, p.sourcePhysicalDBList=[meaningless]
[INFO 2020-02-08-12:35:10 full_check.go:243]: db=0:keys=4
[INFO 2020-02-08-12:35:10 full_check.go:253]: ---------------- start 1th time compare
[INFO 2020-02-08-12:35:10 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:35:10 scan.go:20]: build connection[source redis addr: [127.0.0.1:6986]]
[INFO 2020-02-08-12:35:11 full_check.go:203]: stat:
times:1, db:0, dbkeys:4, finish:33%, finished:true
KeyScan:{4 4 0}
KeyConflictInProcess|string|lack_target|{2 2 0}

[INFO 2020-02-08-12:35:11 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:35:16 full_check.go:253]: ---------------- start 2th time compare
[INFO 2020-02-08-12:35:16 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:35:17 full_check.go:203]: stat:
times:2, db:0, finished:true
KeyScan:{2 2 0}
KeyConflictInProcess|string|lack_target|{2 2 0}

[INFO 2020-02-08-12:35:17 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:35:22 full_check.go:253]: ---------------- start 3th time compare
[INFO 2020-02-08-12:35:22 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:35:23 full_check.go:203]: stat:
times:3, db:0, finished:true
KeyScan:{2 2 0}
KeyConflictAtLast|string|lack_target|{2 2 0}
[INFO 2020-02-08-12:35:23 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 4 key(s) and 0 field(s) conflict

value: field exists in source key and destination key, but the corresponding value of field is different
Log in to 6987redis instance, change key test004 to a04, and change key test003 A03

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6987 -a 'Y2hJKSGtuEq'
Warning: Using a password with '-a' option on the command line interface may not be safe.
127.0.0.1:6987> get test004
"a04"
127.0.0.1:6987> get test002
"002"
127.0.0.1:6987> get test001
"001"
127.0.0.1:6987> get test003
"a03"

At this time, the redis full check mode is used:

[root@localhost redis-full-check-1.4.7]# redis-full-check   -s 127.0.0.1:6987  -p 'Y2hJKSGtuEq' -t 127.0.0.1:6986 -a 'Y2hJKSGtuEq'  --log=log  --result=result -m 1
[root@localhost redis-full-check-1.4.7]# redis-full-check   -s 127.0.0.1:6987  -p 'Y2hJKSGtuEq' -t 127.0.0.1:6986 -a 'Y2hJKSGtuEq'  --log=log  --result=result -m 4

[root@localhost redis-full-check-1.4.7]#  redis-full-check   -s 127.0.0.1:6986  -p 'Y2hJKSGtuEq' -t 127.0.0.1:6987 -a 'Y2hJKSGtuEq'    --result=result -m 1
[INFO 2020-02-08-14:44:36 main.go:65]: init log success
........
........
[INFO 2020-02-08-14:44:49 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 4 key(s) and 0 field(s) conflict

Data inconsistency found in the log for 2 instances

It is found that the value values of key test003 and test004 in instance 6986 and instance 6987 are inconsistent:

[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from key;
1|test004|string|value|0|3|3
2|test003|string|value|0|3|3
sqlite> 

Log in to 6987redis instance and delete key test001:

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6987 -a 'Y2hJKSGtuEq'
Warning: Using a password with '-a' option on the command line interface may not be safe.
127.0.0.1:6987> get test001
"001"
127.0.0.1:6987> get test002
"002"
127.0.0.1:6987> get test003
"a03"
127.0.0.1:6987> get test004
"a04"
127.0.0.1:6987> del test002
(integer) 1
127.0.0.1:6987> keys *
1) "test001"
2) "test004"
3) "test003"

At this time, the redis full check mode is used:

[root@localhost redis-full-check-1.4.7]#  redis-full-check   -s 127.0.0.1:6986  -p 'Y2hJKSGtuEq' -t 127.0.0.1:6987 -a 'Y2hJKSGtuEq'    --result=result -m 1
[INFO 2020-02-08-14:54:25 main.go:65]: init log success
...............
...............
[INFO 2020-02-08-14:54:38 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 6 key(s) and 0 field(s) conflict

It is found that the key and value values in instance 6986 and instance 6987 are inconsistent:

[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from key;
1|test004|string|value|0|3|3
2|test003|string|value|0|3|3
3|test002|string|lack_target|0|3|0
sqlite> select * from field;

Tags: Linux Redis SQLite SQL

Posted on Sat, 08 Feb 2020 00:50:26 -0800 by jassh