Redis-based Distributed Lock Implementation

Preface

This article focuses on how Redis-based distributed locking is implemented. It refers to a number of articles written by big guys and summarizes distributed locking.

Overview of Distributed Locks

In a multithreaded environment, in order to ensure that a block of code can only be accessed by one thread at a time, we can usually use synchronized syntax and ReetrantLock to guarantee that in Java, this is actually the way of local locking.But now companies are all prevalent distributed architectures, in a distributed environment, how to ensure that the threads of different nodes execute synchronously?

In fact, for distributed scenarios, we can use distributed locks, which are a way to control mutually exclusive access to shared resources between distributed systems.

For example, in a distributed system, multiple services are deployed on multiple machines. When a client and a user initiate a request for data insertion, if there is no guarantee of a distributed lock mechanism, multiple services on multiple machines may perform concurrent insertions, resulting in repeated insertions of data, which can cause problems for some businesses that do not allow extra data.Title.Distributed locks are designed to solve such problems and ensure mutually exclusive access to shared resources between multiple services. If one service preempts a distributed lock and the other services do not acquire a lock, no subsequent operations will be performed.The general meaning is as follows (not necessarily accurate):

 

 

Features of distributed locks

Distributed locks generally have the following characteristics:

  • Mutual exclusion: Only one thread can hold a lock at a time
  • Re-accessibility: The same thread on the same node can acquire a lock again if the lock is acquired
  • Lock Timeout: Supports lock timeouts as locks in J.U.C. Prevents deadlocks
  • High Performance and High Availability: Locking and unlocking require high efficiency and high availability to prevent distributed lock failures
  • Blocking and non-blocking: Ability to wake up in time from blocking

Implementation of Distributed Lock

We generally implement distributed locks in the following ways:

  • Database-based
  • Redis-based
  • Based on zookeeper

This article focuses on how to implement distributed locks based on Redis

Distributed Lock Implementation for Redis

1. Use the setnx+expire command (wrong practice)

Redis's SETNX command, setnx key value, sets the key to value, so that if the key does not exist, it will succeed. If the key exists, do nothing, return 1 successfully, and 0 unsuccessfully.SETNX is actually short for SET IF NOT Exists

Because distributed locks also require a timeout mechanism, we use the expire command to set them, so the core code for the setnx+expire command is as follows:

public boolean tryLock(String key,String requset,int timeout) {
    Long result = jedis.setnx(key, requset);
    // result = 1 Setup succeeded, otherwise setup failed
    if (result == 1L) {
        return jedis.expire(key, timeout) == 1L;
    } else {
        return false;
    }
}

 

In fact, the steps above are problematic. setnx and expire are two separate steps and are not atomic. If the first instruction is executed and an exception is applied or restarted, the lock cannot expire.

One improvement is to use Lua scripts to ensure atomicity (including setnx and expire instructions)

2. Use Lua scripts (containing setnx and expire instructions)

The code is as follows

public boolean tryLock_with_lua(String key, String UniqueId, int seconds) {
    String lua_scripts = "if redis.call('setnx',KEYS[1],ARGV[1]) == 1 then" +
            "redis.call('expire',KEYS[1],ARGV[2]) return 1 else return 0 end";
    List<String> keys = new ArrayList<>();
    List<String> values = new ArrayList<>();
    keys.add(key);
    values.add(UniqueId);
    values.add(String.valueOf(seconds));
    Object result = jedis.eval(lua_scripts, keys, values);
    //Determine success
    return result.equals(1L);
}

3. Use the set key value [EX seconds][PX milliseconds][NX|XX] command (correct practice)

Redis started at version 2.6.12, adding a series of options to the SET command:

SET key value[EX seconds][PX milliseconds][NX|XX]
  • EX seconds: Sets the expiration time in seconds
  • PX milliseconds: Set expiration time in milliseconds
  • NX: Set value only if key does not exist
  • XX: Set value only if key exists

The NX option of the set command is the same as the set nx command, and the code procedure is as follows:

public boolean tryLock_with_set(String key, String UniqueId, int seconds) {
    return "OK".equals(jedis.set(key, UniqueId, "NX", "EX", seconds));
}

Value must be unique. We can use UUID to set random strings to ensure uniqueness. Why?If value is not a random string but a fixed value, the following problems may exist:

  • 1. Client 1 acquired the lock successfully
  • 2. Client 1 blocked an operation for too long
  • 3. The key set expires and the lock is automatically released
  • 4. Client 2 acquired a lock for the same resource
  • 5. Client 1 recovers from blocking because the value is the same, so the lock held by Client 2 is released when the release lock operation is performed, which can cause problems

So in general, when releasing locks, we need to verify the value

Implementation of release lock

Value values need to be validated when releasing locks, which means we need to set a value when acquiring locks. We can't use del keys as a rude way because any client can unlock a direct del key, so when unlocking, we need to determine if the locks are their own, based on the value, and the code is as follows:

public boolean releaseLock_with_lua(String key,String value) {
    String luaScript = "if redis.call('get',KEYS[1]) == ARGV[1] then " +
            "return redis.call('del',KEYS[1]) else return 0 end";
    return jedis.eval(luaScript, Collections.singletonList(key), Collections.singletonList(value)).equals(1L);
}

Lua scripting is used here to ensure as much atomicity as possible.

Use the set key value [EX seconds][PX milliseconds][NX|XX] commandIt looks OK, but in fact there are problems in the Redis cluster. For example, A client gets a lock on Redis master node, but the locked key has not synchronized to the slave node yet, master failure, failover, a slave node is upgraded to master node, B client can also get a lock on the same key, but client A has already got the lock, which isCauses multiple clients to get locks.

So there are other scenarios for this Redis cluster scenario

4. Redlock algorithm and Redisson implementation

Redis author antirez proposes a more advanced implementation of Redlock based on a distributed environment, which works as follows:

Reference article below Redlock:Redis Distributed Lock Implementation and redis.io/topics/dist...

Suppose you have five separate Redis nodes (note that the nodes here can be five Redis single master instances or five Redis Cluster clusters, but not a cluster cluster cluster with five primary nodes):

  • Gets the current Unix time in milliseconds
  • Trying to acquire locks from five instances in turn, using the same key and unique value (such as UUID). When requesting a lock from Redis, the client should set a network connection and response timeout that is less than the lock's expiration time. For example, if your lock automatically expires for 10s, the timeout should be between 5 and 50 milliseconds to avoid being obeyedWith server-side Redis hanging up, the client is still deadly waiting for a response.If the server does not respond within the specified time, the client should try to get a lock from another Redis instance as soon as possible
  • The client uses the current time minus the time it started acquiring the lock (the time recorded in step 1) to get the time it took to acquire the lock, and the lock is successful only if it is acquired from most of the Reds nodes (N/2+1, here are three nodes) and it takes less time than the lock failure time.
  • If a lock is retrieved, the real effective time of the key is equal to the effective time minus the time used to acquire the lock (result of step 3 calculation)
  • If for some reason the acquisition of a lock fails (at least N/2+1 Redis instances have not acquired the lock or the acquisition time has exceeded the valid time), the client should unlock all Redis instances (even if some Redis instances have not succeeded in locking at all, preventing some nodes from acquiring the lock but the client has not responded and causing the next period of time to be unavailable)Re-acquire lock)

Redisson implements simple distributed locks

For Java users, we often use Jedis, Jedis is Redis's Java client. In addition to Jedis, Redisson is also a Java client. Jedis is a blocking I/O, while Netty is used at the bottom to implement non-blocking I/O. The client encapsulates locks and inherits the Lock interface of J.U.C. So we can use Redisson as ReentrantLock does.The process is as follows.

  1. Join the POM dependency first
<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson</artifactId>
    <version>3.10.6</version>
</dependency>
  1. Using Redisson, the code is as follows (similar to using ReentrantLock)
// 1. configuration file
Config config = new Config();
config.useSingleServer()
        .setAddress("redis://127.0.0.1:6379")
        .setPassword(RedisConfig.PASSWORD)
        .setDatabase(0);
//2. structure RedissonClient
RedissonClient redissonClient = Redisson.create(config);

//3. Set Lock Resource Name
RLock lock = redissonClient.getLock("redlock");
lock.lock();
try {
    System.out.println("Acquire locks successfully, implement business logic");
    Thread.sleep(10000);
} catch (InterruptedException e) {
    e.printStackTrace();
} finally {
    lock.unlock();
}
 

For the implementation of the Redlock algorithm, we can use RedissonRedLock in Redisson, referring to the Lark's article for details: mp.weixin.qq.com/s/8uhYult2h...

Distributed Lock Wheel Implemented by Redis

A simple distributed lock is implemented using a combination of SpringBoot + Jedis + AOP.

1. Custom Notes

Customize a comment whose method executes logic to acquire a distributed lock

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
public @interface RedisLock {
    /**
     * Business Key
     *
     * @return
     */
    String key();
    /**
     * The number of seconds the lock will expire, defaulting to 5 seconds
     *
     * @return
     */
    int expire() default 5;

    /**
     * Attempt to lock, maximum wait time
     *
     * @return
     */
    long waitTime() default Long.MIN_VALUE;
    /**
     * Unit of time out for locks
     *
     * @return
     */
    TimeUnit timeUnit() default TimeUnit.SECONDS;
}

2. AOP Interceptor Implementation

In AOP, we execute the logic to acquire and release distributed locks as follows:

@Aspect
@Component
public class LockMethodAspect {
    @Autowired
    private RedisLockHelper redisLockHelper;
    @Autowired
    private JedisUtil jedisUtil;
    private Logger logger = LoggerFactory.getLogger(LockMethodAspect.class);

    @Around("@annotation(com.redis.lock.annotation.RedisLock)")
    public Object around(ProceedingJoinPoint joinPoint) {
        Jedis jedis = jedisUtil.getJedis();
        MethodSignature signature = (MethodSignature) joinPoint.getSignature();
        Method method = signature.getMethod();

        RedisLock redisLock = method.getAnnotation(RedisLock.class);
        String value = UUID.randomUUID().toString();
        String key = redisLock.key();
        try {
            final boolean islock = redisLockHelper.lock(jedis,key, value, redisLock.expire(), redisLock.timeUnit());
            logger.info("isLock : {}",islock);
            if (!islock) {
                logger.error("Failed to acquire lock");
                throw new RuntimeException("Failed to acquire lock");
            }
            try {
                return joinPoint.proceed();
            } catch (Throwable throwable) {
                throw new RuntimeException("System exception");
            }
        }  finally {
            logger.info("Release lock");
            redisLockHelper.unlock(jedis,key, value);
            jedis.close();
        }
    }
}

 

3. Redis implements a distributed lock core class

@Component
public class RedisLockHelper {
    private long sleepTime = 100;
    /**
     * Acquire distributed locks directly using setnx + expire
     * Nonatomicity
     *
     * @param key
     * @param value
     * @param timeout
     * @return
     */
    public boolean lock_setnx(Jedis jedis,String key, String value, int timeout) {
        Long result = jedis.setnx(key, value);
        // result = 1 Setup succeeded, otherwise setup failed
        if (result == 1L) {
            return jedis.expire(key, timeout) == 1L;
        } else {
            return false;
        }
    }

    /**
     * Using the Lua script, the script uses the setnex+expire command to lock
     *
     * @param jedis
     * @param key
     * @param UniqueId
     * @param seconds
     * @return
     */
    public boolean Lock_with_lua(Jedis jedis,String key, String UniqueId, int seconds) {
        String lua_scripts = "if redis.call('setnx',KEYS[1],ARGV[1]) == 1 then" +
                "redis.call('expire',KEYS[1],ARGV[2]) return 1 else return 0 end";
        List<String> keys = new ArrayList<>();
        List<String> values = new ArrayList<>();
        keys.add(key);
        values.add(UniqueId);
        values.add(String.valueOf(seconds));
        Object result = jedis.eval(lua_scripts, keys, values);
        //Determine success
        return result.equals(1L);
    }

    /**
     * In Redis 2.6.12 and beyond, use the set key value [NX] [EX] command
     *
     * @param key
     * @param value
     * @param timeout
     * @return
     */
    public boolean lock(Jedis jedis,String key, String value, int timeout, TimeUnit timeUnit) {
        long seconds = timeUnit.toSeconds(timeout);
        return "OK".equals(jedis.set(key, value, "NX", "EX", seconds));
    }

    /**
     * Customize the time-out for acquiring locks
     *
     * @param jedis
     * @param key
     * @param value
     * @param timeout
     * @param waitTime
     * @param timeUnit
     * @return
     * @throws InterruptedException
     */
    public boolean lock_with_waitTime(Jedis jedis,String key, String value, int timeout, long waitTime,TimeUnit timeUnit) throws InterruptedException {
        long seconds = timeUnit.toSeconds(timeout);
        while (waitTime >= 0) {
            String result = jedis.set(key, value, "nx", "ex", seconds);
            if ("OK".equals(result)) {
                return true;
            }
            waitTime -= sleepTime;
            Thread.sleep(sleepTime);
        }
        return false;
    }
    /**
     * Wrong unlock method - delete key directly
     *
     * @param key
     */
    public void unlock_with_del(Jedis jedis,String key) {
        jedis.del(key);
    }

    /**
     * Use Lua scripts for unlock manipulation, value values are validated when unlocking
     *
     * @param jedis
     * @param key
     * @param value
     * @return
     */
    public boolean unlock(Jedis jedis,String key,String value) {
        String luaScript = "if redis.call('get',KEYS[1]) == ARGV[1] then " +
                "return redis.call('del',KEYS[1]) else return 0 end";
        return jedis.eval(luaScript, Collections.singletonList(key), Collections.singletonList(value)).equals(1L);
    }
}
 

4. Controller Layer Control

Define a TestController to test the distributed locks we implement

@RestController
public class TestController {
    @RedisLock(key = "redis_lock")
    @GetMapping("/index")
    public String index() {
        return "index";
    }
}

Summary

Distributed locks focus on mutual exclusion, with only one client acquiring a lock at any one time.In a real production environment, the implementation of distributed locks may be more complex, and I'm here mainly focused on Redis-based distributed lock implementation in a single machine environment. As for Redis cluster environment, there is not much involved. Interested friends can consult the relevant information.

Project source address: github.com/pjmike/redi...


Author: pjmike_pj
Links:https://juejin.im/post/5cc165816fb9a03202221dd5
Source: Excavation





Tags: Redis Jedis Java less

Posted on Thu, 28 May 2020 17:50:56 -0700 by outsidaz