Redis in microservice architecture

Learn how to use Redis with Spring Cloud and Spring Data to provide configuration servers, message brokers, and databases.

 

 

Redis can be widely used in microservice architecture. It may be one of the few popular software solutions that your application can leverage in many different ways. On request, it can act as a master database, cache, or message broker. Although it is also a key / value store, we can use it as a configuration server or discovery server in a microservice architecture. Although it is usually defined as an in memory data structure, we can also run it in persistent mode.

Through this article, I will show you some examples of using Redis with microservices built based on Spring Boot and Spring Cloud framework, combining my own knowledge and the knowledge I have learned in the recent course. These applications will use Redis publish / subscribe, Redis as cache or main database, and Redis as configuration server to communicate with each other asynchronously. This is a picture illustrating the architecture described.

 

 

 

Redis as configuration server

If you have built microservices using Spring Cloud, you may know something about Spring Cloud Config. It is responsible for providing distributed configuration mode for microservices. Unfortunately, Spring Cloud Config does not support Redis as a back-end repository for property sources. That's why I decided to derive a Spring Cloud Config project and implement this feature. I hope my implementation will soon be included in the official release of Spring Cloud. How can we use it? It's very simple. Let's see.

The current SNAPSHOT version of Spring Boot is 2.2.0.BUILD-SNAPSHOT, which is the same as the version used for Spring Cloud Config. When building Spring Cloud Config Server, we only need to include these two dependencies, as shown below.

 1 <parent>
 2     <groupId>org.springframework.boot</groupId>
 3     <artifactId>spring-boot-starter-parent</artifactId>
 4     <version>2.2.0.BUILD-SNAPSHOT</version>
 5 </parent>
 6 <artifactId>config-service</artifactId>
 7 <groupId>pl.piomin.services</groupId>
 8 <version>1.0-SNAPSHOT</version>
 9 <dependencies>
10     <dependency>
11         <groupId>org.springframework.cloud</groupId>
12         <artifactId>spring-cloud-config-server</artifactId>
13         <version>2.2.0.BUILD-SNAPSHOT</version>
14     </dependency>
15 </dependencies>

 

By default, Spring Cloud Config Server uses a Git repository backend. We need to activate a redisprofile to force it to use redis as the backend. If the address your redis instance listens to is not localhost:6379, you need to use the spring.redis. * property to override the automatically configured connection settings. This is our bootstrap.yml file.

1 spring:
2   application:
3     name: config-service
4   profiles:
5     active: redis
6   redis:
7     host: 192.168.99.100

 

The application main class should be annotated with @ EnableConfigServer.

1 @SpringBootApplication
2 @EnableConfigServer
3 public class ConfigApplication {
4     public static void main(String[] args) {
5         new SpringApplicationBuilder(ConfigApplication.class).run(args);
6     }
7 }

 

Before running the application, we need to start the Redis instance. This is a command that runs as a Docker container and is exposed on port 6379.

1 $ docker run -d --name redis -p 6379:6379 redis

 

The configuration of each application must be available in the key ${spring.application.name} or ${spring.application.name}-${spring.profiles.active[n]}.

We must use the key corresponding to the configuration property name to create the hash. Our sample application driver management uses three configuration properties: server.port for setting HTTP listening port, spring.redis.host for changing the default Redis address used as message agent and database, and sample.topic.name for setting name. Topics for asynchronous communication between microservices. This is the structure of Redis hash we created for driver management using RDBTools visualization.

 

 

This visualization is equivalent to running the Redis CLI command HGETALL, which returns all fields and values in the hash.

1 >> HGETALL driver-management
2 {
3   "server.port": "8100",
4   "sample.topic.name": "trips",
5   "spring.redis.host": "192.168.99.100"
6 }

 

After setting the key and value in Redis and running Spring Cloud Config Server with a valid redisprofile, we need to enable the distributed configuration function on the client. To do this, we just need to include the spring cloud starter config dependency in thepom.xml of each microservice.

1 <dependency>
2 <groupId>org.springframework.cloud</groupId>
3 <artifactId>spring-cloud-starter-config</artifactId>
4 </dependency>

 

We use the latest stable version of Spring Cloud.

 1 <dependencyManagement>
 2     <dependencies>
 3         <dependency>
 4             <groupId>org.springframework.cloud</groupId>
 5             <artifactId>spring-cloud-dependencies</artifactId>
 6             <version>Greenwich.SR1</version>
 7             <type>pom</type>
 8             <scope>import</scope>
 9         </dependency>
10     </dependencies>
11 </dependencyManagement>

 

The name of the application is obtained from the property spring.application.name at startup, so we need to provide the following bootstrap.yml file.

1 spring:
2   application:
3     name: driver-management

 

 

Redis as message agent

Now, we can continue to study the message broker, the second use case of Redis in the microservice based architecture. We will implement a typical asynchronous system, as shown in the figure below. After creating a new itinerary and completing the current itinerary, the micro service itinerary management will send the notification to Redis Pub / Sub. The notification is received by both driver management and passenger management who book a specific channel.

 

 

Our application is very simple. We only need to add the following dependencies to provide REST API and integrate with Redis Pub / Sub.

1 <dependency>
2     <groupId>org.springframework.boot</groupId>
3     <artifactId>spring-boot-starter-web</artifactId>
4 </dependency>
5 <dependency>
6     <groupId>org.springframework.boot</groupId>
7     <artifactId>spring-boot-starter-data-redis</artifactId>
8 </dependency>

 

We should use the channel name and publisher to register the bean. TripPublisher is responsible for sending messages to the target topic.

 1 @Configuration
 2 public class TripConfiguration {
 3     @Autowired
 4     RedisTemplate<?, ?> redisTemplate;
 5     @Bean
 6     TripPublisher redisPublisher() {
 7         return new TripPublisher(redisTemplate, topic());
 8     }
 9     @Bean
10     ChannelTopic topic() {
11         return new ChannelTopic("trips");
12     }
13 }

 

TripPublisher uses RedisTemplate to send messages to topics. Before sending, it will use Jackson 2jsonredisserializer to convert all messages in the object to JSON strings.

 1 public class TripPublisher {
 2     private static final Logger LOGGER = LoggerFactory.getLogger(TripPublisher.class);
 3     RedisTemplate<?, ?> redisTemplate;
 4     ChannelTopic topic;
 5     public TripPublisher(RedisTemplate<?, ?> redisTemplate, ChannelTopic topic) {
 6         this.redisTemplate = redisTemplate;
 7         this.redisTemplate.setValueSerializer(new Jackson2JsonRedisSerializer(Trip.class));
 8         this.topic = topic;
 9     }
10     public void publish(Trip trip) throws JsonProcessingException {
11         LOGGER.info("Sending: {}", trip);
12         redisTemplate.convertAndSend(topic.getTopic(), trip);
13     }
14 }

 

We have implemented logic at the publisher. Now, we can implement it on the subscriber side. We have two microservice driver management and passenger management that listen for notifications sent by the travel management microservice. We need to define the RedisMessageListenerContainer bean and set the message listener implementation class.

 1 @Configuration
 2 public class DriverConfiguration {
 3     @Autowired
 4     RedisConnectionFactory redisConnectionFactory;
 5     @Bean
 6     RedisMessageListenerContainer container() {
 7         RedisMessageListenerContainer container = new RedisMessageListenerContainer();
 8         container.addMessageListener(messageListener(), topic());
 9         container.setConnectionFactory(redisConnectionFactory);
10         return container;
11     }
12     @Bean
13     MessageListenerAdapter messageListener() {
14         return new MessageListenerAdapter(new DriverSubscriber());
15     }
16     @Bean
17     ChannelTopic topic() {
18         return new ChannelTopic("trips");
19     }
20 }

 

The class responsible for handling incoming notifications needs to implement MessageListenerinterface. When a message is received, the DriverSubscriber deserializes it from JSON to an object and changes the state of the driver.

 1 @Service
 2 public class DriverSubscriber implements MessageListener {
 3     private final Logger LOGGER = LoggerFactory.getLogger(DriverSubscriber.class);
 4     @Autowired
 5     DriverRepository repository;
 6     ObjectMapper mapper = new ObjectMapper();
 7     @Override
 8     public void onMessage(Message message, byte[] bytes) {
 9         try {
10             Trip trip = mapper.readValue(message.getBody(), Trip.class);
11             LOGGER.info("Message received: {}", trip.toString());
12             Optional<Driver> optDriver = repository.findById(trip.getDriverId());
13             if (optDriver.isPresent()) {
14                 Driver driver = optDriver.get();
15                 if (trip.getStatus() == TripStatus.DONE)
16                     driver.setStatus(DriverStatus.WAITING);
17                 else
18                     driver.setStatus(DriverStatus.BUSY);
19                 repository.save(driver);
20             }
21         } catch (IOException e) {
22             LOGGER.error("Error reading message", e);
23         }
24     }
25 }

 

 

Redis as the primary database

Although the main purpose of using Redis is to cache in memory or store as key / value, it can also act as the main database of the application. In this case, it's worth running Redis in persistent mode.

1 $ docker run -d --name redis -p 6379:6379 redis redis-server --appendonly yes

 

Use hash operation and mmap structure to store entities in Redis. Each entity needs to have a hash key and ID.

 1 @RedisHash("driver")
 2 public class Driver {
 3     @Id
 4     private Long id;
 5     private String name;
 6     @GeoIndexed
 7     private Point location;
 8     private DriverStatus status;
 9     // setters and getters ...
10 }

 

Fortunately, Spring Data Redis provides a well-known repository mode for Redis integration. To enable it, we should use the @ enablereredisrepositories annotation to configure or main class. When using Spring warehouse mode, we don't have to build any queries for Redis ourselves.

1 @Configuration
2 @EnableRedisRepositories
3 public class DriverConfiguration {
4 // logic ...
5 }

 

With the Spring Data repository, we don't need to build any Redis queries, just follow the name method of the Spring Data convention. For more details, see my previous article introduction to Spring Data Redis. For example purposes, we can use the default method implemented within Spring Data. This is a declaration of the repository interface in driver management.

1 public interface DriverRepository extends CrudRepository<Driver, Long> {}

 

Don't forget to enable the Spring Data repository by annotating the main application class or configuration class with @ enablereredisrepositories.

1 @Configuration
2 @EnableRedisRepositories
3 public class DriverConfiguration {
4 ...
5 }

 

conclusion

There are various use cases of redis in the microservice architecture. I just showed you how to easily use it with Spring Cloud and Spring Data to provide configuration servers, message brokers, and databases. Redis is generally regarded as cache storage, but I hope you will change this after reading this article.

Tags: Java Redis Spring Database snapshot

Posted on Thu, 26 Dec 2019 02:45:41 -0800 by Twentyoneth