[Redis5 source code learning] analysis of restore of redis command

Grape

Command syntax

Command Meaning: Deserialize a given serialized value and associate it with a given key.

Command format:

RESTORE key ttl serialized-value [REPLACE] [ABSTTL] [IDLETIME seconds] [FREQ frequency]

Command actual combat

redis> DEL mykey
0
redis> RESTORE mykey 0 "\n\x17\x17\x00\x00\x00\x12\x00\x00\x00\x03\x00\
                        x00\xc0\x01\x00\x04\xc0\x02\x00\x04\xc0\x03\x00\
                        xff\x04\x00u#<\xc0;.\xe9\xdd"
OK
redis> TYPE mykey
list
redis> LRANGE mykey 0 -1
1) "1"
2) "2"
3) "3"

Return value

If deserialization succeeds, OK is returned, otherwise an error is returned.

Source code analysis

Source code analysis is divided into several parts to explain.

Parameter processing

void restoreCommand(client *c) {
    long long ttl, lfu_freq = -1, lru_idle = -1, lru_clock = -1;
    rio payload;
    int j, type, replace = 0, absttl = 0;
    robj *obj;
    /* Analytic parameter */
    for (j = 4; j < c->argc; j++) {
        int additional = c->argc-j-1;
        if (!strcasecmp(c->argv[j]->ptr,"replace")) {
            replace = 1;
        } else if (!strcasecmp(c->argv[j]->ptr,"absttl")) {
            absttl = 1;
        } else if (!strcasecmp(c->argv[j]->ptr,"idletime") && additional >= 1 &&
                   lfu_freq == -1)
        {
            if (getLongLongFromObjectOrReply(c,c->argv[j+1],&lru_idle,NULL)
                    != C_OK) return;
            if (lru_idle < 0) {
                addReplyError(c,"Invalid IDLETIME value, must be >= 0");
                return;
            }
            lru_clock = LRU_CLOCK();
            j++; /* Consume additional arg. */
        } else if (!strcasecmp(c->argv[j]->ptr,"freq") && additional >= 1 &&
                   lru_idle == -1)
        {
            if (getLongLongFromObjectOrReply(c,c->argv[j+1],&lfu_freq,NULL)
                    != C_OK) return;
            if (lfu_freq < 0 || lfu_freq > 255) {
                addReplyError(c,"Invalid FREQ value, must be >= 0 and <= 255");
                return;
            }
            j++; /* Consume additional arg. */
        } else {
            addReply(c,shared.syntaxerr);
            return;
        }
    }

We mentioned the restore command format above. We can see that the fourth parameter is optional at the beginning, so the parsing parameter starts from j=4, and different operations will be performed according to different parameters during the traversal.
Let's look at these four commands in turn:

  1. Replace: If it is a replacement field, set the identifier bit replace to 1.
  2. Absttl: If it is an absttl field, set the identifier absttl to 1. If the ABSTTL modifier is used, ttl should indicate the absolute Unix timestamp (in milliseconds) in which the key will terminate.
  3. Idletime &&freq: These two parameters are explained in detail in the object command. In objec, ideltime is the number of seconds (read or write operations are not requested) that have been returned since the object stored at the specified key is idle. Freq is a logarithmic access frequency counter that returns the object stored at the specified key. When these two commands are parsed in this command, ideltime sets the lru_clock clock value. Set lru_freq in freq, set the frequency and determine whether it is between 0 and 255.

Check & corresponding mode operation

/* This is to make sure that the key exists. This operation is only performed when replace ment equals 0.*/
    if (!replace && lookupKeyWrite(c->db,c->argv[1]) != NULL) {
        addReply(c,shared.busykeyerr);
        return;
    }
        
    /* Check that ttl is legal and the rule is whether it is less than 0 and can be converted to numbers*/
    if (getLongLongFromObjectOrReply(c,c->argv[2],&ttl,NULL) != C_OK) {
        return;
    } else if (ttl < 0) {
        addReplyError(c,"Invalid TTL value, must be >= 0");
        return;
    }
    // Check RDB version and data checksum. If they do not match, an error is returned.
    if (verifyDumpPayload(c->argv[3]->ptr,sdslen(c->argv[3]->ptr)) == C_ERR)
    {
        addReplyError(c,"DUMP payload version or checksum are wrong");
        return;
    }
    rioInitWithBuffer(&payload,c->argv[3]->ptr);
    if (((type = rdbLoadObjectType(&payload)) == -1) ||
        ((obj = rdbLoadObject(type,&payload)) == NULL))
    {
        addReplyError(c,"Bad data format");
        return;
    }
    // If it's replace mode, delete key
    if (replace) dbDelete(c->db,c->argv[1]);
    

Add key

 /* Create the key and set the TTL if any */
    dbAdd(c->db,c->argv[1],obj);  //If the key exists, an error is reported
    if (ttl) {
        if (!absttl) ttl+=mstime();
        setExpire(c,c->db,c->argv[1],ttl);
    }
    //Create a new key.
    //Set if ttl exists.
    // If ttl is 0, there will be no expiration when the key is created, otherwise the specified expiration time (in milliseconds) will be set.
    objectSetLRUOrLFU(obj,lfu_freq,lru_idle,lru_clock);
    //Set frequency and time range limits
    signalModifiedKey(c->db,c->argv[1]);
    addReply(c,shared.ok);
    server.dirty++;
}

Expand

LFU's least commonly used page replacement algorithm recently

Least Frequently Used algorithm.LFU is the first page to be eliminated with the least number of visits in a certain period of time!

The core idea of the algorithm is that if data has been accessed many times in the past, it will be accessed more frequently in the future.
The specific algorithm flow is as follows:

LRU hasn't used permutation algorithm recently

Least Recently Used algorithm.LRU is the first to eliminate the longest time that has not been used.

The algorithm eliminates data according to the historical access records of data. The core idea of the algorithm is that if data has been accessed recently, the probability of being accessed in the future will be higher.
The specific algorithm flow is as follows:

FIFO FIFO FIFO First in First Out Permutation Algorithms

FIFO (First in First out), first in first out.
In fact, in the design concept of the operating system, many places have used the idea of first in, first out, such as job scheduling (first come first served). Why is this principle used in many places? Because this principle is simple, consistent with people's inertial thinking, fair and easy to implement, it can be achieved directly by using queues in the data structure.

Tags: PHP Redis Unix less

Posted on Tue, 08 Oct 2019 13:39:39 -0700 by TGWSE_GY